domanic has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
apiarian has joined #ipfs
apiarian has quit [Client Quit]
espes__ has joined #ipfs
hiei has joined #ipfs
domanic has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
vsimon has joined #ipfs
domanic has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
dconroy has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
onikuma has quit [Quit: Leaving]
vsimon has quit [Remote host closed the connection]
uzzrroOOOO has joined #ipfs
vsimon has joined #ipfs
ralphtheninja has quit [Quit: leaving]
domanic has quit [Read error: Connection reset by peer]
TUSF has quit [Ping timeout: 240 seconds]
domanic has joined #ipfs
Neomex has quit [Remote host closed the connection]
domanic has quit [Read error: Connection reset by peer]
joocain2 has quit [Remote host closed the connection]
domanic has joined #ipfs
joocain2 has joined #ipfs
TUSF has joined #ipfs
TUSF has quit [Ping timeout: 256 seconds]
wmoh1 has quit [Read error: Connection reset by peer]
domanic has quit [Read error: Connection reset by peer]
sz0 has quit [Quit: Connection closed for inactivity]
domanic has joined #ipfs
jesse22 has joined #ipfs
domanic has quit [Ping timeout: 268 seconds]
wmoh1 has joined #ipfs
cmeik has joined #ipfs
cmeik is now known as cmeiklejohn
cmeiklejohn has quit [Changing host]
cmeiklejohn has joined #ipfs
<MikeFair>
whyrusleeping: still out there?
<MikeFair>
AphelionZ: There's also #orbitdb
vivus has quit [Quit: Leaving]
G3nka1 has joined #ipfs
asciiker has joined #ipfs
asciiker has quit [Read error: Connection reset by peer]
asciiker has joined #ipfs
tombusby has quit [Ping timeout: 272 seconds]
vsimon has quit [Remote host closed the connection]
tombusby has joined #ipfs
hipboi_ has quit [Ping timeout: 256 seconds]
hipboi_ has joined #ipfs
dconroy has joined #ipfs
domanic has joined #ipfs
leeola has quit [Quit: Connection closed for inactivity]
domanic has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
asciiker has quit [Read error: Connection reset by peer]
asciiker has joined #ipfs
domanic has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
cris has quit [Ping timeout: 240 seconds]
domanic has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
cris has joined #ipfs
kanzure has quit [Quit: leaving]
tombusby has quit [Remote host closed the connection]
tombusby has joined #ipfs
domanic has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
jesse22 has quit [Ping timeout: 260 seconds]
ygrek_ has joined #ipfs
witten has quit [Ping timeout: 260 seconds]
domanic has quit [Ping timeout: 276 seconds]
chriscool1 has joined #ipfs
<AphelionZ>
how does ipfs versioning work, exactly? do you have to keep track of the versions yourself or is there something built in?
<MikeFair>
You mean of code, or in store files?
<AphelionZ>
in the filesystem itself
<MikeFair>
Every "version" has a unique CID
<MikeFair>
(Content ID)
<AphelionZ>
from ipfs.io: IPFS provides historic versioning (like git) and makes it simple to set up resilient networks for mirroring of data.
<AphelionZ>
right, i know that
<MikeFair>
I think that statement is somewhat misleading
<MikeFair>
AFAIK there is no built-in system for version tracking
<AphelionZ>
ok
<MikeFair>
The information to do it exists
<AphelionZ>
is there docs? link?
<MikeFair>
Nothing I'm aware of; IPNS is the closest thing, but that only gives you a fixed reference to the "latest" version
<AphelionZ>
yeah, per node too.. and very slowly for the time being ;)
<AphelionZ>
i want to give my users versioning is all
<MikeFair>
IPNS is not only per node, you can publish arbitrary ipns addresses to your own keys
<AphelionZ>
well if i publish a name for a given content hash, it will typically generate the same ipns hash
<MikeFair>
(for example, I use ipns keys to represent two different domains I've played with ipfs hosting on)
<MikeFair>
AphelionZ: That's because you didn't use the --key parameter
<AphelionZ>
aha!
<AphelionZ>
but still i dont think thats what im looking for
<MikeFair>
AphelionZ: And the same restant hash is exactly the point ;)
<MikeFair>
AphelionZ: Well it kind of is
<MikeFair>
the reason is this: What are you versioning?
<AphelionZ>
files =)
<MikeFair>
that's what you think ;)
<MikeFair>
what's a file in ipfs
<MikeFair>
a CID
<MikeFair>
So a list of all the versions of a file is a list of CIDs
<AphelionZ>
right
<AphelionZ>
with you so far
<MikeFair>
Which CID means what you want?
<MikeFair>
ipfs doesn't understand "file" in the sense you think of it
<MikeFair>
a file doesn't have a name
<MikeFair>
such that named file could have versions
<MikeFair>
there's an ocean of CIDs
<MikeFair>
that's it
<MikeFair>
inside one of those CIDs is a directory listing
<MikeFair>
it has a string associated to another CID
dconroy has quit [Ping timeout: 240 seconds]
<MikeFair>
but if you change the directory listed (add or delete a file) -- you change the CID of the directory
<MikeFair>
So the directory also has a list of CIDs
<MikeFair>
there's no timestamps
<MikeFair>
(at least not that you could sort an order across the ocean of CIDs)
<MikeFair>
So you need a container to describe the history of file versions
<MikeFair>
Something like a directory, specific to that file
<AphelionZ>
yeah, i get ya
<AphelionZ>
so the long and short is... i need to keep track of this myself somehow
<MikeFair>
And you want that thing to have a fixed address that doesn't move everytime something changes
<MikeFair>
and that's what ipns provides
<AphelionZ>
gotcha
<MikeFair>
You know "labels" in version control systems?
<MikeFair>
fixed names to specific revisions of the source tree?
}ls{ has quit [Ping timeout: 256 seconds]
witten has joined #ipfs
<MikeFair>
You could make a directory of files to be revisioned, and inside each file is the CID for the file at that point; then publish that directory as an IPNS
<MikeFair>
address
* MikeFair
is kind of seeing something leveraging IPLD here
<MikeFair>
It's not in production as far as I know; but I think it's the planned successor to ipns
}ls{ has joined #ipfs
<MikeFair>
btw, out of curiousity, how would you expect your users to find/identify a historical version of a file?
dudebbn has quit [Read error: Connection reset by peer]
ygrek_ has quit [Ping timeout: 248 seconds]
dudebbn has joined #ipfs
anewuser has quit [Ping timeout: 256 seconds]
vsimon has joined #ipfs
<whyrusleeping>
MikeFair: you know, i'm alive
<MikeFair>
o/
* MikeFair
does a little jig in honor of whyrusleeping's lack of being dead.
<whyrusleeping>
yayyyyyyyy
<MikeFair>
I've just been hunting for ppl who are more clueful than I am regarding the potential of using IPNS in an IPLD link ;)
<MikeFair>
Or another way I can create a authorized distributed IPLD graph (multiple people maintaining different parts of the same IPLD structure)
<MikeFair>
And AphelionZ was wondering about any built-in functions for versioned file tracking ; I didn't know of any -- only that people have mused about how to construct one
samm has quit [Read error: Connection reset by peer]
<MikeFair>
In the IPLD instance, it's easy to see how people can directly publish their portion of the graph -- what I can't see is how that CID update gets rolled into the higher level (because that's not a graph portion they control)
ilyaigpetrov has joined #ipfs
<MikeFair>
The easiest metaphor is something like a JSON product description by a manufacturer; each company individually publishes their own description with the latest details for their product --- other people, retailers and wholesalers, put that link into their own catalogs
<MikeFair>
The manufacturer then makes some spelling corrections, and fixes a few other inaccuracies and publishes a new JSON description
<MikeFair>
In IPNS land, this is relatively easy because they can can just publish the new ipfs link to the updated CID
<MikeFair>
(using a key created explicity for that product)
<MikeFair>
But all the retailers/wholesalers catalogs now want to be updated to reflect the manufacturer's latest published details
<MikeFair>
If the IPLD link was to the IPNS address; there'd be nothing to do; next time the link was traversed, the ipns address would resolve to the updated address
<MikeFair>
Another option, though less favorable, is to make the IPLD structure explicitly fragmented; The ipns CID gets stored in the catalog as data, rather than a link, and the application has to explicitly resolve it to fetch the downstream information
<whyrusleeping>
hrmmm
<whyrusleeping>
the issue with IPNS in IPLD links, is that it breaks the immutability contract
<whyrusleeping>
stebalien says he would be okay with having a different link type
<whyrusleeping>
and you would have to explicitly traverse them
<MikeFair>
don't think I'd mind the explicit traversal (by which I'm picturing I'd have to retrieve the link reference, the code wouldn't recurse into it for me)
<whyrusleeping>
if thats okay, then i'm sure we could find some way to move forward
<MikeFair>
I don't think IPLD necessarily should get a --flatten command to auto-recurse through an IPLD structure
<MikeFair>
I see IPLD links as kin to symlinks and hardlinks in the FS
witten has quit [Ping timeout: 260 seconds]
<MikeFair>
mixing them with "-r" type flags can do really bad things (it's bad enough when you bog down only your own system; but dragging all the IPFS peers with you doesn't sound .... kind) :)
<MikeFair>
BUt I'd like to be able to do something like: ipfs dag get /ipns/IPNS-for-My-Catalog/products/0/manufacturer_details/
<MikeFair>
Where manufacturer_details is another ipns reference
<MikeFair>
perhaps {"/":"/ipns/IPNS_CID"}
<MikeFair>
What I don't want to have to do is two commands; (1) ipfs dag get .../manufacturer_details returns an ipns address, then (2) ipfs dag get returned_ipns_address
<MikeFair>
If "double look ups" to root the request explicitly to the lower portion of the graph using the CID is what you meant; that likely wouldn't work
<MikeFair>
But if the code refused to cross the link until the path explicitly referenced it, that'd be fine
<MikeFair>
I was going to attempt to code something myself (basically an "if link CID startswith /ipns; then ..."), but I couldn't even figure out where that code is
G3nka1 has quit [Remote host closed the connection]
G3nka1 has joined #ipfs
<MikeFair>
whyrusleeping: Any guidance on what I could look at/who I could talk to next to teach "ipfs dag get" to accept an IPNS CID; and for the link resolver to explicitly test for an "/ipns/" CID?
<gozala>
is it a generic thing that specific IPDLResolver defines on it’s own ?
<gozala>
meaning that representation of it is specific to a resolver instance ?
<gozala>
MikeFair: lidel ^^ ?
<whyrusleeping>
MikeFair: I would go make sure you've read a lot of the old ipld design threads (sorry, i don't have links right now)
Kotak has joined #ipfs
<whyrusleeping>
and then post an issue on ipfs/notes or something
<whyrusleeping>
that should start some discussion
<MikeFair>
Where are the old design threads? Closed out ipld GitHub issues?
<MikeFair>
I assume the way a link gets resolved is the node has to come to my peer, then my peer looks up the next CID (remote peers don't trverse links for my request for me)
<MikeFair>
Even just to look at the code I got lost trying to figure out where that resolution happens
<Kotak>
#HELP
<MikeFair>
And I'm with gozala, in that sometimes it's not very clear how I'd go about making and deploying something like a new interface structure
* MikeFair
looks over at Kotak. :)
<Xe>
Kotak: what do you need help with? you can just ask your question instead of asking to ask
<Kotak>
Will Do :)
<Kotak>
Thanks
witten has joined #ipfs
<gozala>
maybe daviddias or whyrusleeping can help me with my question ?
* lidel
opens left eye
<MikeFair>
gozala: My understanding of the dagNode in that context means "The JSON Object"
ulrichard has joined #ipfs
<gozala>
MikeFair: Yeah but question is who defines the shape of it ?
<gozala>
is it per IPDLResolver implementation ?
<gozala>
It seems like it to me, but I’m not certain
<MikeFair>
gozala: You mean the "IPLD Format" reference
sioa has joined #ipfs
<gozala>
it seems like there is IPDLFormatUtil<dagNode> { … }
<mithun>
Hi all, I tried using the ipfs.get() command to fetch the some data back from IPFS, but it returns back with a forEach is not a function. Then I tried the "https://gateway.ipfs.io/api/" which takes a lot of time and sometimes it gets stuck. Any idea on what should I do? FYI: I'm using solidity+truffle(nodejs)
<gozala>
MikeFair: and I think there is also some correlation with Resolver.resolve value
lgierth2[m] has quit [Ping timeout: 264 seconds]
randy[m] has quit [Ping timeout: 264 seconds]
cesalazar[m] has quit [Ping timeout: 264 seconds]
trhode84[m] has quit [Ping timeout: 264 seconds]
matiasnomad[m] has quit [Ping timeout: 264 seconds]
Bertschneider[m] has quit [Ping timeout: 264 seconds]
gejohn[m] has quit [Ping timeout: 264 seconds]
berunade[m] has quit [Ping timeout: 264 seconds]
foreseaz[m] has quit [Ping timeout: 264 seconds]
avatias0[m] has quit [Ping timeout: 264 seconds]
abbiya has quit [Ping timeout: 264 seconds]
ant0nio[m] has quit [Ping timeout: 264 seconds]
ehmry has quit [Ping timeout: 264 seconds]
am2on has quit [Ping timeout: 264 seconds]
ebarch has quit [Ping timeout: 264 seconds]
Matthew[m] has quit [Ping timeout: 264 seconds]
davidar_ has quit [Ping timeout: 264 seconds]
<MikeFair>
mithun: I don't follow "but it returns back with a forEach is not a function"
cesalazar[m] has joined #ipfs
trhode84[m] has joined #ipfs
matiasnomad[m] has joined #ipfs
Bertschneider[m] has joined #ipfs
randy[m] has joined #ipfs
<MikeFair>
gozala: The Reoslver, as I understand it, returns a single value that was requested
admin[m]1 has quit [Ping timeout: 264 seconds]
Smith[m]1 has quit [Ping timeout: 264 seconds]
DokterBob1 has quit [Ping timeout: 264 seconds]
aeaf123[m] has quit [Ping timeout: 264 seconds]
kevindetry[m] has quit [Ping timeout: 264 seconds]
scitec[m] has quit [Ping timeout: 264 seconds]
score_under[m] has quit [Ping timeout: 264 seconds]
cbluth[m] has quit [Ping timeout: 264 seconds]
Jacob[m] has quit [Ping timeout: 264 seconds]
iwxzr[m] has quit [Ping timeout: 264 seconds]
WiredDude[m] has quit [Ping timeout: 264 seconds]
scheutsch[m] has quit [Ping timeout: 264 seconds]
rombob[m] has quit [Ping timeout: 264 seconds]
pt467353[m] has quit [Ping timeout: 264 seconds]
fujexo[m] has quit [Ping timeout: 264 seconds]
ganlub has quit [Ping timeout: 264 seconds]
AndrewMcSwain[m] has quit [Ping timeout: 264 seconds]
M1trace[m] has quit [Ping timeout: 264 seconds]
juicenoose[m] has quit [Ping timeout: 264 seconds]
nocent has quit [Ping timeout: 264 seconds]
vq12[m] has quit [Ping timeout: 264 seconds]
zerga[m] has quit [Ping timeout: 264 seconds]
renatocan[m] has quit [Ping timeout: 264 seconds]
M-Dave has quit [Ping timeout: 264 seconds]
grensjo[m] has quit [Ping timeout: 264 seconds]
pierce has quit [Ping timeout: 264 seconds]
itorres has quit [Ping timeout: 264 seconds]
M-hash has quit [Ping timeout: 264 seconds]
jcgruenhage has quit [Ping timeout: 264 seconds]
chihiro[m] has quit [Ping timeout: 264 seconds]
lgierth2[m] has joined #ipfs
gejohn[m] has joined #ipfs
avatias0[m] has joined #ipfs
ant0nio[m] has joined #ipfs
foreseaz[m] has joined #ipfs
ebarch has joined #ipfs
<gozala>
MikeFair: yeah but what is the type of that value ?
davidar_ has joined #ipfs
abbiya has joined #ipfs
berunade[m] has joined #ipfs
<MikeFair>
gozala: That value will either be: a piece of JSON; or a link with rest of the path
ehmry has joined #ipfs
f4bio[m] has quit [Ping timeout: 264 seconds]
DEC05EBA[m] has quit [Ping timeout: 264 seconds]
klara[m] has quit [Ping timeout: 264 seconds]
neurocis[m] has quit [Ping timeout: 264 seconds]
M1trace[m] has joined #ipfs
scheutsch[m] has joined #ipfs
fujexo[m] has joined #ipfs
kevindetry[m] has joined #ipfs
aeaf123[m] has joined #ipfs
pt467353[m] has joined #ipfs
iwxzr[m] has joined #ipfs
score_under[m] has joined #ipfs
grensjo[m] has joined #ipfs
Jacob[m] has joined #ipfs
rombob[m] has joined #ipfs
scitec[m] has joined #ipfs
admin[m]1 has joined #ipfs
cbluth[m] has joined #ipfs
ganlub has joined #ipfs
juicenoose[m] has joined #ipfs
zerga[m] has joined #ipfs
WiredDude[m] has joined #ipfs
Smith[m]1 has joined #ipfs
chihiro[m] has joined #ipfs
vq12[m] has joined #ipfs
itorres has joined #ipfs
AndrewMcSwain[m] has joined #ipfs
renatocan[m] has joined #ipfs
DokterBob1 has joined #ipfs
trqx has quit [Remote host closed the connection]
M-Dave has joined #ipfs
am2on has joined #ipfs
pierce has joined #ipfs
nocent has joined #ipfs
M-hash has joined #ipfs
klara[m] has joined #ipfs
DEC05EBA[m] has joined #ipfs
f4bio[m] has joined #ipfs
neurocis[m] has joined #ipfs
Matthew[m] has joined #ipfs
jcgruenhage has joined #ipfs
<gozala>
MikeFair: is that a same piece of JSON that Util can deserialize ?
<MikeFair>
gozala: Use the "islink" and "remainderPath" to test
<MikeFair>
gozala: yes
<gozala>
ok so that is the correlation than that I was trying to figure out
<MikeFair>
that's my understanding, it's a binary version of the node stored in the network
<MikeFair>
but, resolve does sub-portions of the blob
<sioa>
Hey I am a relative programming noob. I came here from libp2p site. And I am trying to make a mastodon like p2p social network for my highschool. Can I do that with libp2p or do I need to first get a handle on what ipfs is?
<gozala>
MikeFair: I think I’m tired having hard time visualizing the whole picture
<MikeFair>
sioa: From what you just said, I don't think ipfs is quite at the level you'd be wanting; you'd have to focus on which features of the social network you meant and how much coding you want to put into it
<mithun>
@MikeFair, Yea I used the ipfs.get() inbuilt function from the ipfs github page. It returns the data with "a path and content", where the content is in readable stream. To iterate through the content it uses "forEach" method. For me it returns with the error of forEach is not a function
<MikeFair>
gozala: IPLD is using "structured data" as the content of the CID right? So if you take a Record Description Format (like JSON), you can serialize it into a binaryBlob and stick it a node with a CID reference
<MikeFair>
mithun: I'd check to make sure the object on which you are calling forEach has a value
<gozala>
MikeFair: ok
Fess_ has joined #ipfs
<MikeFair>
gozala: When you "resolve" you are asking for a single piece of that structured data
<sioa>
MikeFair: Let's just say I want to make a simple Twitter clone that works p2p. I am doing this because I think it would be a fun project and I am just researching the tools atm. So I do want to code as much as I can.
<gozala>
MikeFair: so say I have cid for son like { a: { b1: {c1: 1, c2:2}, b2: { d1:1 } }}
<gozala>
and I’m trying to resolve `a/b1`
<MikeFair>
gozala: Given a CID and a path, resolve can get the data; when you do this resolve has three cases: It finds the data requested and returns it; the path doesn't exist and so it's an err; or it tried to find the path but encountered a link instead
<MikeFair>
sioa: A simple web page that aggregates a bunch of tweets from different browsers is very doable --- I'd point you to OrbitDB
<mithun>
MikeFair, Rather than iterating through the contents, I tried displaying the whole retrieved value and displayed it on to my browser console. I was able to see both the path and content there, both having values..
<sioa>
MikeFair: Thanks, I will check it out.
<gozala>
serialized blob of that son ?
<gozala>
serialized blob of that json ?
<MikeFair>
sioa: Sorry I can't be of much more help -- I haven't done much browser side p2p coding yet
<gozala>
presumably path is “a/b1"
dionysus69 has joined #ipfs
<gozala>
and I have no clue wha the value in the callback would be
Fess has quit [Ping timeout: 265 seconds]
<MikeFair>
gozala: I think it's the binaryData that came from the stored dag node
<gozala>
is that a {c1: 1, c2:2} ?
<MikeFair>
callback is a function
Ronsor has joined #ipfs
<MikeFair>
the function to call with the result
Ronsor is now known as Guest17071
<gozala>
MikeFair: on I meant result passed to the callback
<gozala>
so it’s either link or value resolved
<MikeFair>
in the json you provided there are no llinks
<gozala>
and I wonder if value resolved is deserialized json segment
<MikeFair>
is the object you specified {c1: 1, c2 2{
<gozala>
oh so it’s an object not the binary blob
<MikeFair>
gozala: I'm not sure if it's the binary segment of that value, or the deserialized version
<MikeFair>
gozala: But I don't think it's the whole dag node
<sioa>
MikeFair: No worries. Thanks for the help. Anyway, what would be a good source to learn about IPFS and how the different projects like IPLD, libp2p relate to each other? I just learnt about this and it seems truly interesting.
<gozala>
MikeFair: so how do I know if result.value is link or a content (regardless if it’s binaryBlob or deserialized)
<MikeFair>
sioa: It's definitely interesting, and as far as I'm concerned is the foundation of making fancy networks that work like every high-tech science fiction novel written ;)
<gozala>
so it seems there should be a way to figure in resolve result what did I got
<mithun>
Does my ipfs version and the gateway version which I'm using cause a delay in fetching the data back from IPFS?
<mithun>
my ipfs version is0.4.10 and the gateway version is 0.4.14
<MikeFair>
mithun: how so? I don't see differing software versions on peer nodes contributing significantly to data fetch time
muvlon has quit [Ping timeout: 276 seconds]
<MikeFair>
gozala: Well if you got a "remainingPath" from resolve, the you definitely have a link ;)
<MikeFair>
gozala: The trick though is I've asked for a path that was a link, and got the result I expected (even though there wouldn't have been a remainingPath to fetch)
<MikeFair>
gozala: So yes, there's something to test if you have a resolved value or a link
<mithun>
MikeFair, when i try to use the gateway "https://gateway.ipfs.io/<hash>", it takes a lot of time to retrieve back the data. I thought it might be a difference in the version
<gozala>
MikeFair: This interface could really use some improvements
<MikeFair>
mithun: I've come to accept that this is the silent elephant in the room nobody seems to understand or talk about
<MikeFair>
mithun: Sometimes things resolve fast, and then explicitly, they go slow -- like minutes slow
<mithun>
Thanks MikeFair...
<MikeFair>
err inexplicably
Kotak has quit [Quit: Page closed]
<mithun>
@MikeFair, so what if I use a private ipfs swarm instance?
<MikeFair>
mithun: I'd expect everything to go really quickly in your own private swarm
<MikeFair>
mithun: I've especially noticed it immediately after an ipns update
<mithun>
@MikeFair, oke...I will try to go for a private instance then. Thanks Mike
<MikeFair>
(implementing the filesystem)
<MikeFair>
gozala: The way I understand it, there's a binary chunk of data -- you run the hash algorithm on that data and you get a CID --- you run the p2p XOR hopping algorithm and you figure out what peer that data belongs to ---- that peers stores the binary data against the CID
<MikeFair>
When someone requests the CID, the peer returns the binary blob
<mithun>
@MikeFair, Do you have any sample code for uploading an image onto IPFS using nodejs.
<MikeFair>
mithun: I don't, but any file upload example will do; nothing special about images
<MikeFair>
I've mostly stuck with the go-ipfs exe
<JCaesar>
(In danger of picking nits: You can't upload things to IFPS.)
<MikeFair>
Done a little bit of Go coding -- and a tiny handful of browser js-ipfs
<MikeFair>
JCaesar: If not "upload", what do you call it?
<MikeFair>
JCaesar: You don't really "copy it to"
<JCaesar>
"provide" maybe…
muvlon has joined #ipfs
<MikeFair>
JCaesar: But you don't really "provide" it to IPFS; once you do 'ipfs put' a copy is placed in the p2p swarm
<MikeFair>
JCaesar: Your peer "provides" CIDs
<JCaesar>
"provide xy over ipfs"?
<MikeFair>
That's kind of strange to me, because my node isn't controlling the providing.... "ipfs provides xy" <--- I see that
<JCaesar>
Hm, but maybe I misunderstand things… "a copy is placed in the p2p swarm" sounds like an upload indeed.
<MikeFair>
JCaesar: It's not like your local node is serving your files
<JCaesar>
Then it's an upload indeed… js-ipfs is different, I guess…
<JCaesar>
Sorry.
<MikeFair>
Hmmm, js-ipfs shouldn't be different
<MikeFair>
It's a separate swarm, but locating the provider of a CID should use the same algorithm
<MikeFair>
js-ipfs should take your local data, break it up into blocks, then "put" those blocks out into the swarm
faenil has quit [Quit: Lost terminal]
<mithun>
MikeFair: Thanks..Mike..we are stuck with a particular problem of being able to retrieve an Image from IPFS through the console but not in our frontend of our application. Was wondering whether you had some suggestions of sample projects which had done this or suggestions on which forum we could post our query for help
Xiti has quit [Quit: Xiti]
<MikeFair>
If your trying to cross from go-ipfs to js-ipfs then the problem might be separate swarms
<MikeFair>
I don't know if it's been resolved, but there was a time when the browsers using the js-ipfs API wouldn't trade CIDs with the console ipfs peer nodes
<MikeFair>
I never quite understood exactly why; I think it was because the browsers were acting as their own peer network
<gozala>
MikeFair: Does this matches your understanding of the interface
<whyrusleeping>
mithun: are you using js-ipfs or go-ipfs?
<whyrusleeping>
whats the setup here?
smuten_ has quit [Quit: leaving]
smuten has joined #ipfs
<MikeFair>
whyrusleeping: Sorry he must have stepped away; can you briefly explain to me why go-ipfs and js-ipfs programs don't magically just play nice with each other?
<whyrusleeping>
MikeFair: I think the main reason is that js-ipfs doesnt have the dht or any form of content routing
<whyrusleeping>
which means that you have to manually connect to peers that have the content you want
<mithun>
whyrusleeping,MikeFair: go-ipfs...and yes..I uploaded an image using the ipfs add and got the hash...used it with "http://ipfs.io/ipfs/<generatedhash>
<whyrusleeping>
mithun: and it never loads on ipfs.io?
<MikeFair>
whyrusleeping: So without routing, how do you find the peer with the content you're after?
<whyrusleeping>
MikeFair: really luck
<whyrusleeping>
js-ipfs *really* needs to fix this
<mithun>
whyrusleeping: It Does, sometimes...But then sometimes I have to wait for more than 5mins, hoping to get the data back..
<whyrusleeping>
its a bit of a WIP guide
<whyrusleeping>
but its the steps that I usually take when i cant get data from one node to another
<mithun>
whyrusleeping: will go through the doc and update you..
<whyrusleeping>
thanks, let me know if anything is unclear
<whyrusleeping>
i'll update the doc
rendar has joined #ipfs
<mithun>
whyrusleeping:sure, will do...Thanks
<MikeFair>
whyrusleeping: Thanks I'm looking forward to learning more about js-ipfs versus go-ipfs
asciiker has quit [Read error: Connection reset by peer]
asciiker has joined #ipfs
espadrine_ has quit [Ping timeout: 248 seconds]
<MikeFair>
Anyone still here? I was wondering if anyone had any thoughts on how to create a "Virtual Private Swarm"
<MikeFair>
Here's my thinking; there are distributed nodes that are more likely to want to exchange information because of context (like they are all part of the same global company)
<MikeFair>
Furthermore, these nodes are more likely to want the same sets of files, because employees are sharing closely related information (all about the same company)
<MikeFair>
Aside from particpating in the single global swarm -- it'd be useful (meaning it would make things faster) --- if these nodes could create a separate routing table just for themselves
<MikeFair>
This way, these peers have two 'swarms' they can interrogate for content.
xnbya has quit [Ping timeout: 240 seconds]
<MikeFair>
The "virtual private swarm" will have far fewer nodes, pinning more relevant content
<gozala>
daviddias: maybe you could respond whenever you get a chance
trqx has joined #ipfs
xnbya has joined #ipfs
Vaelatern has quit [Ping timeout: 248 seconds]
mtodor has joined #ipfs
NullbutC00L has joined #ipfs
dionysus69 has quit [Ping timeout: 264 seconds]
<MikeFair>
gozala: I think I can answer the "requires iteration questions
<MikeFair>
gozala: I beleive it's because you can't
<MikeFair>
You might be able to hide the iteration deeper in the API, but the network can't iteratively retrieve the data for you because it doesn't understand the data
<MikeFair>
We can only assume that your local node understands how to interpret the binaryBlobs
Taoki has quit [Ping timeout: 276 seconds]
<MikeFair>
Think about it like a directory structure and files
<MikeFair>
When you retrieve the CID for the directory, you get links to the CIDs for the files, but the network doesn't bring you all the files CIDs too
<MikeFair>
If you ask for a file, as a relative path from the directory's CID; the it pulls the directory, follows the path, then returns the file
<MikeFair>
That's an iterative process
<MikeFair>
In the case of IPLD, that file now has a structure that can be dug into too
asciiker has quit [Read error: Connection reset by peer]
asciiker has joined #ipfs
<gozala>
MikeFair: I think you misunderstood my question :) which was why not actually use tuple or a record instead of a table
<gozala>
It seems that it’s an array of (path, value) pairs
<gozala>
So it could be represented as [path, value] or a {path: “/a/b”, value: x}
<gozala>
But instead it’s a map with single entry
<MikeFair>
Because a list of links would be a more useful thing for asynchronous retrieval
<MikeFair>
forEach(pathName) map
<MikeFair>
retrieve(map[pathName])
<MikeFair>
something like that
<MikeFair>
You know, I'm not writing any code that uses these results atm, so I can't say what'd be better/easier
<gozala>
MikeFair: oh if map can have >1 entry sure
<gozala>
But I was under impression it doesn’t
<gozala>
As it’s array of them
<gozala>
And description calls it a tuple
<gozala>
Which is why I found it counterintuitive
<MikeFair>
We might be talking different API calls too
<MikeFair>
I saw one that returned "all links in this object"
Fess_ has quit [Ping timeout: 255 seconds]
<mithun>
whyrusleeping: I went through link which you gave and was able to get the data from two different nodes..i.e. from node A to node B
<MikeFair>
mithun: Did you have to do anything, or did it just work?
<gozala>
MikeFair: I was referring to resolver.tree
<gozala>
and it could go multiple level’s deep as well too
inetic has joined #ipfs
<MikeFair>
gozala: Oh, I wasn't thinking that: I was thinking [{"dir":"CID"}] (from that example)
<MikeFair>
gozala: [{"dir":"CID"}, {"dir2":"CID2"}, {"dir/subdir":"CID"}] (from a different example with more links in it)
<gozala>
MikeFair: so I vaguely remember old IPLD paper having more or less what I described
<gozala>
It could have changed though
<MikeFair>
gozala: Well you're absolutely right in that a link is described as an object {"/":"CID"}
<MikeFair>
That is the actual value you see in the DB
<gozala>
MikeFair: what DB ? Sorry my brain is fried, need to sleep
<gozala>
talk to you later
<MikeFair>
ipfs dag get /ipfs/zdpuAvseB5qenujhBMdZt4eiaefxJrFmEqAQMdLx7CaoYyQXY
<gozala>
thanks again for your input
<MikeFair>
gozala: Sorry, DB == IPLD stored structure
<MikeFair>
That dag object has a nodes array which are links
<MikeFair>
err which have links
bomb-on has quit [Quit: zzz]
<MikeFair>
ipfs dag get /ipfs/zdpuAvseB5qenujhBMdZt4eiaefxJrFmEqAQMdLx7CaoYyQXY/nodes/0/data
<MikeFair>
traverses one of those links
igorline has joined #ipfs
chriscool1 has joined #ipfs
raynold has quit [Quit: Connection closed for inactivity]
xzha has joined #ipfs
vsimon has quit [Remote host closed the connection]
chriscool1 has quit [Quit: Leaving.]
vsimon has joined #ipfs
bomb-on has joined #ipfs
colatkinson has joined #ipfs
igorline has quit [Ping timeout: 252 seconds]
maxzor has joined #ipfs
Mateon3 has joined #ipfs
asciiker has quit [Read error: Connection reset by peer]
Mateon2 has quit [Ping timeout: 240 seconds]
Mateon3 is now known as Mateon1
uzzrroOOOO has quit [Ping timeout: 256 seconds]
SunflowerSociety has joined #ipfs
igorline has joined #ipfs
colatkinson has quit [Ping timeout: 248 seconds]
mrBen2k2k2k has joined #ipfs
Caterpillar has quit [Ping timeout: 248 seconds]
igorline has quit [Ping timeout: 255 seconds]
colatkinson has joined #ipfs
colatkinson has quit [Remote host closed the connection]
rcat has joined #ipfs
vmx has joined #ipfs
ylp1 has joined #ipfs
knix_ has quit [Remote host closed the connection]
toxync01- has joined #ipfs
toxync01- is now known as toxync01_
toxync01 has quit [Ping timeout: 265 seconds]
toxync01_ is now known as toxync01
cxl000 has joined #ipfs
SunflowerSociety has quit [Read error: Connection reset by peer]
SunflowerSociety has joined #ipfs
pkx2 has joined #ipfs
kanzure has joined #ipfs
pkx2 has quit [Remote host closed the connection]
<MikeFair>
---- Another random thought ---- What if the browsers using js-ipfs somehow built their own DHT by only including the nodes that directly hosted their content of interest
<MikeFair>
I mean a broswer isn't really meant to act as a content host in the network for most definitions of "content host" --- they're more of a content consumer
<MikeFair>
The function they could provide though is speeding up the routing table of the DHT
<MikeFair>
There's something about the browsers maintaining the "quick lookup network" for what nodes are providing what CIDs (by popularity and interest) that seems to make sense here
<MikeFair>
There's also a CID batch list: "Nodes that requested these CIDs are likely to request these CIDs"; and somehow keeping a list of nodes providing those CIDs more prominently in their routing netowrk
SunflowerSociety has quit [Quit: Leaving]
maxzor has quit [Remote host closed the connection]
ONI_Ghost has joined #ipfs
maxzor has joined #ipfs
reit has quit [Ping timeout: 268 seconds]
sioa has quit [Ping timeout: 260 seconds]
PorcoRosso70 has quit [Ping timeout: 260 seconds]
G3nka1 has quit [Ping timeout: 256 seconds]
dimitarvp has joined #ipfs
cl0[m] has joined #ipfs
}ls{ has quit [Quit: real life interrupt]
Caterpillar has joined #ipfs
tombusby has quit [Ping timeout: 272 seconds]
demize has quit [Quit: Do Androids Dream of Electric Sheep?]
jkilpatr has quit [Ping timeout: 252 seconds]
tombusby has joined #ipfs
demize has joined #ipfs
bwn has quit [Ping timeout: 252 seconds]
ONI_Ghost has quit [Read error: Connection reset by peer]
ONI_Ghost has joined #ipfs
inetic has quit [Ping timeout: 260 seconds]
jkilpatr has joined #ipfs
uzzrroOOOO has joined #ipfs
vivus has joined #ipfs
igorline has joined #ipfs
mithun has quit [Quit: Page closed]
vedtam has joined #ipfs
vmx has quit [Ping timeout: 260 seconds]
<vedtam>
hi all!
<vedtam>
I am new to IPFS, but doing web and app development for almost 10 years
Neomex has joined #ipfs
<vedtam>
I am amaised by IFS and really cuorious what it could do for devs in the coming future
<vedtam>
I m wondering about a few things, maybe someone could help me understand
<vedtam>
say I wan't to deploy a website with a login page, what r my options in order to build a login functionality?
uzzrroOOOO has quit [Ping timeout: 260 seconds]
<JCaesar>
Stuff on IPFS is understood to be public, so I'm wondering, what that login would be for.
<vedtam>
say I want to build a blogging platform
<vedtam>
so only an owner could change certain informations within a page
<JCaesar>
I don't know if if IPFS is the right thing for that. You could have a look at what d.tube is doing, or at the forum on ipfs.io.
<mkg20001[m]>
zeronet's zeroblog platform might be what you are looking for
<vedtam>
I mean, the infromation would be public but this app/site would allow only the author to make changes to it via a web interface trought the gateway + odmain
<vedtam>
hmmm
<vedtam>
the thing is, I am planning to mix a blockchain into the app
<vedtam>
say, sending tips to users via ether/bitcoin transactions
<vedtam>
but on ethereum for example there is no cheap way of storing data, than on IPFS's homepage it emphasises that its the perfect match for the blockchian
<JCaesar>
That sounds like d.tube/steemit.
<vedtam>
aham
NullbutC00L has quit [Ping timeout: 260 seconds]
<vedtam>
damn, I liked so much how IPFS doing things
<vedtam>
:(
<JCaesar>
(d.tube is partially based on IPFS. I don't like their way of using it, though.)
<vedtam>
cuz if its suitable for static site hosting, i thought it could be hacked to get a way of securing content
reit has joined #ipfs
<JCaesar>
I think it can. And since I'm unhappy with how d.tube is doing things, I'd like to encourage you to try…
<vedtam>
the way I imagined to build it was by creating a "static site builder" where the user makes changes to its content, I would record the data to a JSON, than when saving the content, a renderer function would pick up this JSON, and spit out a static HTML file
<vedtam>
images would be compressed/converted to base64 and the whole thing bundled to a single HTML file
<vedtam>
could this be possible?
<JCaesar>
Should. But I'm wondering why you want to pack everything into a single file.
<vedtam>
from what I'we read, the garbage collector is kinda intens, so to avoid to much of a hassle of pinning and cashing multiple files, there would be a single file to take care of
<vedtam>
of cours optimising would be among the last steps, if multiple files gonna work, there will be no need to bundle
mildred_ has joined #ipfs
lord| has quit [Ping timeout: 268 seconds]
kumavis has quit [Ping timeout: 252 seconds]
bbielsa has quit [Ping timeout: 252 seconds]
chriscool has quit [Ping timeout: 252 seconds]
aaaaaaaaa____ has quit [Ping timeout: 252 seconds]
chriscool has joined #ipfs
aaaaaaaaa____ has joined #ipfs
<voker57>
you don't have to manage pinning multiple files, just pin a directory with blog in it and it's ok
massino_ has quit [Ping timeout: 252 seconds]
mildred_ has quit [Client Quit]
<vedtam>
hmmm
<vedtam>
right
<vedtam>
mate, thanks. I will be beack! Cya
<vedtam>
*back
kumavis has joined #ipfs
bbielsa has joined #ipfs
massino_ has joined #ipfs
voker57 has quit [Remote host closed the connection]
vedtam has quit [Quit: WeeChat 1.9.1]
voker57 has joined #ipfs
voker57 has joined #ipfs
Taoki has joined #ipfs
lord| has joined #ipfs
trqx has quit [Remote host closed the connection]
lord| has quit [Quit: WeeChat 2.0.1]
PorcoRosso70 has joined #ipfs
ralphtheninja has joined #ipfs
vmx has joined #ipfs
vyzo has quit [Ping timeout: 240 seconds]
vyzo has joined #ipfs
trqx has joined #ipfs
anewuser has joined #ipfs
igorline has quit [Ping timeout: 248 seconds]
Jesin has quit [Quit: Leaving]
mtodor_ has joined #ipfs
<JCaesar>
voker57: But where does that directory come from? If you compose it with a writable gateway and PUT…
mtodor has quit [Ping timeout: 276 seconds]
ONI_Ghost has quit [Read error: No route to host]
ONI_Ghost has joined #ipfs
<JCaesar>
Hm. I wonder if I could keep content online simply by asking various gateways for https://$GW/api/v0/refs?arg=/ipns/foobar.de&r
<Icefoz_>
JCaesar: You can but there's no guarentees.
<Icefoz_>
Nothing stops the gateway from fetching something, serving it, and then immediately throwing it away.
<lgierth>
ok there's no way i'm gonna read the backlog
<lgierth>
i was only gone 9 hours, how did you come up with 350 new lines
Rudde has quit [Remote host closed the connection]
reit has quit [Quit: Leaving]
inetic has joined #ipfs
igorline has joined #ipfs
rodolf0 has joined #ipfs
shizy has joined #ipfs
<mkg20001[m]>
lgierth: Is it possible to specify a relay in the p2p-circuit multiaddr like this: `<server-address>/p2p-circuit/ipfs/<dst-id>` ?
<MikeFair>
whyrusleeping: The "Virtual" part is take advantage of the global network of peers
* MikeFair
reads first.
<whyrusleeping>
ah
<whyrusleeping>
i gotcha
Encrypt has joined #ipfs
<MikeFair>
Yeah, so this would be that, but as a second routing table
<lgierth>
gozala: yeah connections are currently always duplex reliable streams. i'm not sure about the exact types in js-land. in addition to these "stream conns" we'll also have "packet conns" some time soon
<whyrusleeping>
yeah, i re-read through what you were asking
<whyrusleeping>
that would be really nice
<lgierth>
gozala: but that's gonna be a separate set of functions throughout the libp2p stack
<gozala>
lgierth: thanks
<lgierth>
btw those interface definitions are very much appreciated :D i could have used exactly that last weekend
mildred has joined #ipfs
<MikeFair>
gozala: The conn in that API call appears to be a connection to specific peer from the given the context, but I can't tell if it's a logical "node to node" connection (the addresses involved are peer ids); or a more specific socket connection (the addresses involved are ip addresses or other kind of physical network address)
<lgierth>
MikeFair: at that particular interface, we don't care. it's just a stream and we do stuff with it. the underlying things wrap it in encryption, multiplexing, etc. -- ip addresses/sockets are also completely transparent to this interface. they only matter to the respective transport module
<MikeFair>
lgierth: So if I interrogated the conn object, in the source/dest fields I'd expect to see peer ids then?
raynold has joined #ipfs
<lgierth>
ah good point, i think you can actually get the underlying multiaddr
<lgierth>
at least in go-land you can
<lgierth>
but that underlying addr might for example be a circuit relay address (circuit relay is half a transport too)
mildred has quit [Ping timeout: 256 seconds]
zautomata has joined #ipfs
* MikeFair
nods to lgierth and adds "Learn IPFS Network Layers Stack Model" to his list of "Things to understand before opening my mouth to much".
<raynold>
Ahh it's a wonderful day
* lgierth
adds "Document the IPFS stack properly" to his list
<MikeFair>
;) --- I think I understand what you're saying, but obviously a multiAddr and a circuit address aren't in IPv4; I'm assuming a circuit address is something like the IPv6 version of a routed connection.
<lgierth>
well it's actually on the list already but just got another big exclamation mark next to it
Neomex has quit [Read error: Connection reset by peer]
<MikeFair>
(I don't remember if a routed connection is exactly what IPv6 calls it; but you place the specific list IP addresses to hop through as part of the link/connection definition)
* MikeFair
waves to raynold. o/
<MikeFair>
whyrusleeping: Any thoughts on a short route for me to test the IPNS in IPLD links since we spoke last night?
<MikeFair>
lgierth: Would love to hear your thoughts on doing it too
<MikeFair>
lgierth: The question is: I'd like to use an IPNS link as the reference in an IPLD link
<MikeFair>
And give an ipns address to ipfs dag get
<MikeFair>
lgierth: The use case is building a distributedly authorized graph --- so different people are updating different portions of the graph, and since they don't have privileges to update portions of the graph they don't own, they can't update link references when they update their own sections
<lgierth>
that is *really* tricky when it comes to resolving through that graph
<MikeFair>
lgierth: Only if you auto-recurse ;)
<whyrusleeping>
lgierth: yeah, he just wants to be able to manually resolve through ipns links
<MikeFair>
lgierth: To which I say "Don't do that!!!! It hurts everyeone!!"
<lgierth>
i'd say that's a case for a relatively simple custom ipld resolver
<lgierth>
or ipld format, or what its called
ONI_Ghost has quit [Ping timeout: 265 seconds]
<lgierth>
one that understands what to do when it encounters an ipns reference
<lgierth>
i can't discuss a ton today, have to pack and put my room in order, flying to the US tomorrow morning
<MikeFair>
lgierth: I was also able to sort out, that if auto-recursion was going on, keeping track of the "CIDs resolved to from the root of this path" and not follow any dupes
<lgierth>
but i suggest you look into a custom ipld format for that. as for baking it directly into ipld itself, i'd say it's likely not worth the added complexity
<MikeFair>
lgierth: I'm not opposed to that but I am having a hard time finding any docs on how make one of those and use it.
<MikeFair>
(A lot of those 350 lines was gozala and I trying to parse through the resolver API ;) )
<MikeFair>
I'd actually really loveto be able to make custom formats
<whyrusleeping>
MikeFair: so what exactly do you want? Do you just want to build something hacky to play with?
<lgierth>
hehe got it
<MikeFair>
not only for links, but also for storage
<lgierth>
i think the tl;dr would be: define a multicodec code that means "this node is an ipns reference", and bind the respective ipld format implementation to that code. then for the actual resolution logic (the resolved content would not be covered by the reference's hash in the dag) you'll have to plug in IPNS
vyzo has quit [Ping timeout: 248 seconds]
<whyrusleeping>
MikeFair: in go or js?
<MikeFair>
lgierth: My thoughts on detecting "this is an ipns reference" was "the link CID starts with /ipns/"
<MikeFair>
whyrusleeping: I was hoping both, but I'll take either really ; I've got more experience with go ; but the JS codebase seems a bit easier to follow
<MikeFair>
I couldn't even locate where the ipld resolver code was happening
<MikeFair>
Well I was looking for the "Where do I hack in the 'test CID value for startswith /ipns/'" point
<MikeFair>
I like the idea of a custom ipld resolver
<MikeFair>
I guess I could add a "namespace" or 'ns' value to the link object --- absence or 'ipfs' == standard CID; otherwise branch on module/function name (in my case 'ipns')
<MikeFair>
whyrusleeping: Oh wait! Is that it in "SplitAbsPath" just above ResolveToLastNode?
<whyrusleeping>
ehmmm
<MikeFair>
if parts[0] == "ipfs" { parts = parts[1:] }
<whyrusleeping>
yes, but its not the droid youre looking for
<whyrusleeping>
that whole bit of code is for 'resolve a given path'
<MikeFair>
That strips out the 'ipfs' portion from the CID
<MikeFair>
Does "ipfs name publish" produce CIDv1 output?
walle303 has joined #ipfs
jcgruenhage has joined #ipfs
<MikeFair>
OR is that going to be part of the iprs takeover
raktajino is now known as Guest15087
<lgierth>
nope, currently only `ipfs dag` commands produce cidv1, and `ipfs add` if you pass certain flags
<lgierth>
for ipns keys and cidv1, we're waiting to properly integrate ed25519 keys
<lgierth>
gotta run! going swimming
* MikeFair
looks at lgierth and wonders if he understands what that means for my testing/proff of concept.
<MikeFair>
:)
<MikeFair>
glhf!
<MikeFair>
(good luck / have fun)
kaotisk has quit [Ping timeout: 260 seconds]
sim590 has quit [Ping timeout: 260 seconds]
<lgierth>
i got socialised on counterstrike 1.5 servers
<lgierth>
kthxbai
<MikeFair>
lgierth: Cna i ping you again on this when you're back?
sim590 has joined #ipfs
<MikeFair>
Would like to do it right, but looks like there's no way to get there from here ;)
<lgierth>
sure, i'll just not pay much attention to my computer while packing and cleaning ;) and tomorrow i'll be on the road for 21 hours
kaotisk has joined #ipfs
<MikeFair>
Well -- mostly it's just that if I use ipns name publish to make an ipns reference, but I can only use a v1 CID to make the custom IPLD resolver work; then I have no way to produce a link that I can put in IPLD to custom resolve
The_8472 has joined #ipfs
The_8472 has quit [*.net *.split]
<MikeFair>
lgierth: Unless I can somehow cast the v0 ipns CID into a v1 CID
<MikeFair>
cast/convert
The_8472 has joined #ipfs
angreifer has joined #ipfs
<MikeFair>
fwiw -- that CID also claims v0 CIDs are backwards compatible; so that ipns address starting with Qm, should be understood, and I'd just need to add code to resolve it?
<MikeFair>
err that CID doc
jkilpatr has quit [Ping timeout: 256 seconds]
guideline has joined #ipfs
zautomata has quit [Read error: Connection reset by peer]
<whyrusleeping>
Or really, I guess you don't necessarily need your own ipld type, but it would make things easier I think
<whyrusleeping>
you could just use the normal ipld cbor type, and have your own custom field for ipns links
<whyrusleeping>
like, {"?": "/ipns/QmFooBar"} instead of the "/" key
<whyrusleeping>
and your resolver could look for that, and do an ipns lookup
<whyrusleeping>
to return the next step in the path
vsimon has joined #ipfs
Ellenor is now known as Reinhilde
fredthomsen has quit [Quit: Leaving.]
jesse22 has joined #ipfs
igorline has quit [Ping timeout: 260 seconds]
Ecran has joined #ipfs
<jesse22>
hello. for the bitswap protocol do connections have to be Dialed-out to be used for blocks? or can blocks that exist on peers be transferred over connections that were initiated by peers that Dialed me?
<jesse22>
trying to figure out absolutely abysmal transfer speed issues. I should be seeing near gigabit speeds. getting 1-5 MB/s-ish.
talonz has quit [Quit: Leaving]
erde74 has quit [Quit: Leaving]
Fess_ has quit [Quit: Leaving]
reit has joined #ipfs
mtodor has joined #ipfs
Poeticode is now known as Poeticruel
mtodor has quit [Remote host closed the connection]
pawalls has joined #ipfs
espes__ has quit [Ping timeout: 265 seconds]
talonz has joined #ipfs
robattila256 has joined #ipfs
<whyrusleeping>
jesse22: what commands are you using?
ilyaigpetrov has quit [Quit: Connection closed for inactivity]
raynold has quit [Quit: Connection closed for inactivity]
<Taoki>
So I've been trying to find something out for like a month now: Is it possible, using browser JavaScript, to automatically upload a file then modify an IPNS address to point to it? Of course with protecting the IPNS with a password or private key, so only the owner of an account can make changes.
<Taoki>
This is essential for any plan to have a dynamic website, where users can update the settings of their account and have the account point to the updated file.
<natewalck>
whyrusleeping: I'm working with @jesse22 on this
<natewalck>
do you mean which daemon command?
<natewalck>
or which command are we using to monitor bandwidth usage?
<whyrusleeping>
natewalck: which command are you using to transfer files?
<natewalck>
curl?
<whyrusleeping>
curl?
<natewalck>
more or less
<whyrusleeping>
not like, ipfs get, or ipfs pin?
<natewalck>
There is a file on the IPFS network and we are curling it down from the local endpoint
<natewalck>
127.0.0.1:whatever/ipfs/HASH
<jesse22>
whyrusleeping: ipfs get or HTTP-via-gateway seem to offer similar performance
<Taoki>
Yep, seems like it: IPFS Companion in Firefox is seeing it.
ylp has quit [Quit: Leaving.]
<lgierth>
whyrusleeping: your {"?":"/ipns"} is new syntax -- we can do this without messing with ipld itself
<lgierth>
this is really just another ipld format with resolution logic that's a tad more involved that usual
igorline has joined #ipfs
<lgierth>
(when bringing in ipns, it's not strictly content-addressed anymore of course. permission-addressed or key-addressed would be a term i guess)
<jesse22>
Taoki: yes. you can see connected peers by executing 'ipfs swarm peers' for example
<Taoki>
wonderful
<Taoki>
Okay, so: Since I'm thinking of trying to develop a website for ipfs, I need to know one of the basic questions. How do I use IPFS JavaScript functions in Firefox?
<Taoki>
I assume simply running the Daemon in the background won't enable them in all browsers on my machine. Or will it?
<jesse22>
i don't know anything about the javascript libs, but i imagine you'll need to include them in page source. surely there's some getting started docs?
<Taoki>
One more thing I should take note of: Is it possible to create a local, sandboxed IPFS daemon? I'd rather not litter the whole network with a website that is a work in progress
jkilpatr has quit [Ping timeout: 256 seconds]
<jesse22>
Taoki: honestly? i wouldn't worry about 'littering' anything. files added to ipfs are only hosted on your node until some other node requests them
<jesse22>
that said you can remove all your Bootstrap hosts (ipfs bootstrap) and it shouldn't connect to anyone else when being started
<Taoki>
Right... that is fair. Though how do I remove temporaries then?
<Taoki>
Also is it possible to delete an IPNS once you created it?
<jesse22>
i've seen issues with ipns not working using a single node ,though
<Taoki>
Ah
MikeFair has quit [Ping timeout: 260 seconds]
[1]MikeFair is now known as MikeFair
<jesse22>
i don't know about deleting ipns - good question
<whyrusleeping>
jesse22: can you take a look at the performance if you use `ipfs pin add` to do the transfer?
<jesse22>
whyrusleeping: happy to, let me do that
<Taoki>
How do you delete files from your local node, once they are obsolete and you no longer need them wasting space on your drive?
<jesse22>
Taoki: unpin them and then garbage collect
igorline has quit [Ping timeout: 256 seconds]
<jesse22>
whyrusleeping: so far performance is similarly slow - bw poll is showing ~1.7 MB/s
<whyrusleeping>
jesse22: hrm... I expect it to be kinda slow (due to the graph traversal algorithms right now)
<whyrusleeping>
but 1.7MB/s seems a bit low
<whyrusleeping>
jesse22: whats the latency between the two machines?
<whyrusleeping>
lgierth: its not a new syntax
<whyrusleeping>
you can just make an object that has that in it
<whyrusleeping>
and write a handler for it in your resolver
<Taoki>
I'd still like to get back to my question for the time being in my case, as I've been trying to find an answer for over a month now. I believe it was:
<jesse22>
whyrusleeping: a wrinkle: we're running a private swarm - currently only ~15-ish nodes. so i'd expect the dht to be fairly small with minimal content at this stage
<whyrusleeping>
jesse22: that should be fine
<jesse22>
whyrusleeping: there's about 6-7 hosts that have the file cached. some hosts are very close - same LAN, some hosts are across a WAN
<Taoki>
I want to have an interface site that allows any user to create and update an account. The account's settings are stored in a Json file. For the idea to work, I need JavaScript to be able to update this json file with a new version whenever changes are made. However, to prove it's the account holder making them, it needs to check a password or key.
<Taoki>
How close is this to being possible?
<lgierth>
whyrusleeping: ah true. still want a custom multicodec for that though, right?
rendar has quit []
<lgierth>
"custom"
<whyrusleeping>
lgierth: nah, you don't need it
<Taoki>
What I believe I need is to create and update an IPNS address, pointing to the user's json database. However there are two problems: First is how browser-side JS can manipulate an IPNS in the first place! Second is how this IPNS can be secured with a password or key, so only the owner can edit it regardless of browser or location.
<Taoki>
Can the JS IPFS AIP do any of those things?
<jesse22>
whyrusleeping: LAN hosts have 5ms pings, 'close' WAN has around 25ms, 'far' WAN has around 125ms
<whyrusleeping>
jesse22: hrm...
<whyrusleeping>
jesse22: could you try setting up two hosts with the new experimental datastore?
<whyrusleeping>
ipfs init --profile=badgerds
jkup has quit [Remote host closed the connection]
<whyrusleeping>
i'm wondering if youre hitting a dumb disk bottleneck
jkup has joined #ipfs
<whyrusleeping>
frist: ^ this is the sort of thing I want to be able to debug with the tracing stuff
Vaelatern has joined #ipfs
<jesse22>
whyrusleeping: yeah that's something that i've noticed: if a single close host starts hosting the file, transfers are pretty darn quick - it's just when it that file is hosted all over the network throughput drops like a rock
<jesse22>
but, yes - i'll give that a go
<jesse22>
and report back
<lgierth>
when transferring a ton of stuff the other week, i noticed bitswap constantly being at 512 wantlist entries. might be a bottleneck too
chjk6x_ has quit [Quit: Leaving]
<lgierth>
didn't ever get above 20MBps between two hosts that can otherwise do 120MBps
<whyrusleeping>
jesse22: another metric to check on the host doing the fetch is `ipfs bitswap stat` and look for duplicate blocks
<Taoki>
Okay... I take it no one knows about my question.
<jesse22>
yeah we are definitely seeing duplicate blocks - but i don't know what to make of the number
<jesse22>
Taoki: unless someone pipes up about knowledge with js-ipfs, you might be best served by diving into their code & docs
<jesse22>
ipns may not be supported in js-ipfs yet? worth pining the issue
<Taoki>
Still not figuring out how to get js-ipfs functions working in Firefox. The instructions involve a NPM command, but I assume I need something in FF itself.
<Taoki>
ok
<whyrusleeping>
Taoki: yeah, i don't know much about js-ipfs, sorry
<whyrusleeping>
jesse22: how much duplicate blocks?
<jesse22>
whyrusleeping: i have 13 GB of dup data/51k blocks
<whyrusleeping>
well shit
<jesse22>
this is for a 8GB that file that i've been testing over & over again (doing a ipfs repo gc between each test)
<whyrusleeping>
yeah, so we have some work to do on reducing duplicate sends
l2d has quit [Read error: Connection reset by peer]
l2d has joined #ipfs
<jesse22>
but - that wouldn't explain the slowness would it? i mean its wasted bandwidth, sure, but the speed should be better than a megabyte, no?
<whyrusleeping>
yeah, it doesnt explain the slowness
<whyrusleeping>
i'm still curious to see how the different datastore works
<jesse22>
yep getting stuff uploaded now :)
sz0_ has joined #ipfs
sz0_ has quit [Client Quit]
sz0_ has joined #ipfs
sz0_ is now known as sz0
paralin has joined #ipfs
zautomata has quit [Read error: Connection reset by peer]
chriscool2 has joined #ipfs
chriscool1 has quit [Quit: Leaving.]
zautomata has joined #ipfs
zautomata has quit [Read error: Connection reset by peer]
<jesse22>
yeah, okay LAN host<->LAN host (single node serving file) getting >50 MB/s and often 90, 100 MB/s
zautomata has joined #ipfs
<jesse22>
bbiab
Mateon1 has quit [Ping timeout: 248 seconds]
Mateon1 has joined #ipfs
zautomata has quit [Read error: Connection reset by peer]
zautomata has joined #ipfs
James_Epp has quit [Quit: Leaving]
}ls{ has joined #ipfs
<natewalck>
@whyrusleeping I think you are on to something here
<natewalck>
dup blocks received: 119
<natewalck>
dup data received: 31 MB
<natewalck>
and its balooning
<natewalck>
ballooning*
<natewalck>
dup blocks received: 313
<natewalck>
dup data received: 82 MB
<natewalck>
thats while transferring an 8gb file
<whyrusleeping>
jesse22: so things are much faster with the new datastore?
<natewalck>
dup data received: 384 MB
<natewalck>
dup blocks received: 1463
<whyrusleeping>
yeah, we definitely need to work on that...
<natewalck>
Is there any way to tweak settings for that right now ?
<natewalck>
Is it bitswap strategy related?
<jesse22>
whyrusleeping: not exactly suer what you mean by datastore - it's the same repo and everything just a new file that's uploaded to one single node and downloaded from one other node
<jesse22>
the slowness we see is with a file that 6 or 7 hosts have
ljhms has quit [Ping timeout: 264 seconds]
<natewalck>
So to break this down to stats
<whyrusleeping>
jesse22: oh, I thought the fast speeds you were getting were when you used the --profile=badgerds option
<natewalck>
blocks received: 3394
<natewalck>
data received: 889764252
<natewalck>
data sent: 0
<natewalck>
blocks sent: 0
<natewalck>
dup blocks received: 2139
<natewalck>
dup data received: 561 MB
<natewalck>
~ 63% duplicate block rate
<natewalck>
which also means 63% of the data received is just tossed because it already has it, correct?
Neomex has joined #ipfs
<jesse22>
whyrusleeping: no, same running daemons, same repos, and everything - its just that we have multiple hosts hosting a file speeds tank
ljhms has joined #ipfs
<whyrusleeping>
jesse22: very interesting...
<whyrusleeping>
I think I know how to go about making that better
<MikeFair>
whyrusleeping: thanks for the tip on where to register; I'll look into it
<jesse22>
whyrusleeping: lemme try and get this ipfs-benchmark stuff and we can get this visualized :)
<whyrusleeping>
great :)
<MikeFair>
(And yes, I actually want/require both --- command line ipns --- and link ipns ----- but the latter seems more difficult than the former)
<MikeFair>
But it still seems wierd to me that "Qm" can't just be made to "do the right thing" ;)
<MikeFair>
(I can envision the reasons for why it "shouldn't" be made to ; but not how the code would make it difficult)
matoro_ has joined #ipfs
matoro has quit [Read error: Connection reset by peer]