<z0mfg>
hi, i've heard of your project, but after reading a bit on it i'm still not sure of what the intended purpose of ipfs is. i mean, it's a nice protocol, but like on https://ipfs.io/#why you explain why it is better than http, not what it should/could be used for. i'd be very glad if someone would like to discuss this with me or explain it a little more.
spilotro has quit [Ping timeout: 250 seconds]
<z0mfg>
at the moment i see it like the "bittorrent+dht+magnet" combo with added versioning.
<z0mfg>
thanks for the video links! i've read the faq, but found no helpful information there. i mean, i see the difference between filesystem and download system, but i wonder about real-life usecases
zbyte64 has joined #ipfs
<z0mfg>
i don't see it in use for a website, where the person who puts up a website wants to also control it and its server.
<z0mfg>
and if i understood it corretly, ipfs also doesn't solve the problem of disappearing seeds, it even makes them harder to find.
zbyte64 has quit [Ping timeout: 260 seconds]
spilotro has quit [Ping timeout: 252 seconds]
spilotro has joined #ipfs
zbyte64 has joined #ipfs
anonymuse has quit [Remote host closed the connection]
<haad>
z0mfg: as for the website use case you describe above, and you understood correctly that IPFS in itself doesn't solve the problem of disappearing seeds, see Filecoin http://filecoin.io/
espadrine has joined #ipfs
<haad>
hope those will help
jholden has joined #ipfs
spilotro has joined #ipfs
jholden has quit [Ping timeout: 245 seconds]
<Yatekii>
haad: hmm doesn't ipns solve the disappearing seeds problem?
Akaibu has quit [Ping timeout: 258 seconds]
Boomerang has joined #ipfs
<z0mfg>
haad: thanks, will look into this!
Akaibu has joined #ipfs
maxlath has quit [Ping timeout: 258 seconds]
ralphtheninja has quit [Quit: leaving]
<haad>
Yatekii: in a way yes, you still need to make sure whatever you point IPNS to that the content exists whereas in Filecoin network takes care of that for you (ie. makes sure that the content is stored on at least n nodes)
<Yatekii>
ahh haad you are tolking about actual loss of data! yeah I asked that one time in here but I forgot the answer tbh
<Yatekii>
guess the answer was: it exists in cache and if cache gets full it dies, not sure
chris613 has joined #ipfs
pfrazee has joined #ipfs
Encrypt has joined #ipfs
dmr has joined #ipfs
tmg has quit [Ping timeout: 256 seconds]
dignifiedquire has joined #ipfs
anonymuse has quit [Remote host closed the connection]
Boomerang has quit [Remote host closed the connection]
<victorbjelkholm[>
richardlitt: your submission on r/ipfs have disappeared for some reason. The post is still there but it's not visible in the subreddit
kulelu88 has joined #ipfs
<richardlitt>
Huh. I have no idea why that might have happened
<richardlitt>
victorbjelkholm[: I still see it
<richardlitt>
both on while logged in and in aprivate session
<victorbjelkholm[>
hm, I can see it in private session but not logged in
<victorbjelkholm[>
weird stuff
maxlath has joined #ipfs
atrapado_ has quit [Ping timeout: 265 seconds]
atrapado_ has joined #ipfs
zbyte64 has joined #ipfs
zbyte64 has quit [Ping timeout: 260 seconds]
jager has quit [Ping timeout: 246 seconds]
herzmeister has quit [Quit: Leaving]
herzmeister has joined #ipfs
anewuser has joined #ipfs
geemili has quit [Ping timeout: 268 seconds]
jager has joined #ipfs
horrified has quit [Quit: weechat memory leak]
jedahan has joined #ipfs
hylas has joined #ipfs
zbyte64 has joined #ipfs
hylas has quit [Ping timeout: 268 seconds]
zbyte64 has quit [Ping timeout: 260 seconds]
herzmeister has quit [Quit: Leaving]
herzmeister has joined #ipfs
Aranjedeath has joined #ipfs
gts has joined #ipfs
<gts>
hey guys
gts has quit [Client Quit]
dignifiedquire has quit [Quit: Connection closed for inactivity]
jedahan has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Encrypt has quit [Ping timeout: 246 seconds]
jedahan has joined #ipfs
cemerick has quit [Ping timeout: 258 seconds]
iczero is now known as wlp1s1
Encrypt has joined #ipfs
Encrypt has quit [Client Quit]
maxlath has quit [Ping timeout: 258 seconds]
brendyn has joined #ipfs
rendar has quit [Ping timeout: 252 seconds]
ylp has joined #ipfs
sdelic has joined #ipfs
jedahan has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
UukGoblin has joined #ipfs
<UukGoblin>
hi
<UukGoblin>
it should be possible to convert a .torrent file to an ipfs hash, right? Has someone made a tool?
<UukGoblin>
my thinking: AFAIR .torrent files are just some hashes of content, and since ipfs work on multihashes...
<Kubuxu>
UukGoblin: as ipfs chunks data so it can be transferred easily (and links the chunks with hashes) you need to know whole file to create ipfs hash of given file
<Mateon1>
UukGoblin: Not really, the way Bittorrent transfers data is wildly different from IPFS, but on second thought it might be possible. We would have to have a special type of node that fetches the torrent data, though.
<Mateon1>
Kubuxu: It could work like how we want to get the Ethereum blockchain to work, as a special IPLD type
<Kubuxu>
yes, but not IPLD but CID
<Kubuxu>
you could link torrent data
<UukGoblin>
interesting :-)
<Kubuxu>
but someone would need to write torrent resolver (which would fetch the data).
<Kubuxu>
or link some other
<UukGoblin>
how about the other way round? Say you have an IPFS link, and a .torrent file, and you want to check if that .torrent matches the IPFS link (only downloading from IPFS, not from .torrent)
<Kubuxu>
it should be, if torrent file includes hash of the whole file
<Mateon1>
I need to look up what .torrent metadata actually contains. Definitely contains title, filename and blocks that belong to them, and things like blocksize
<UukGoblin>
yeah, I haven't looked either, it may be useful to know that :-)
vtomole has joined #ipfs
<Mateon1>
Kubuxu: a magnet hash basically hashes all metadata of a torrent (such that it can be looked up in DHT to find peers), and the metadata hashes all pieces.
<Kubuxu>
but really, once you have file on disk, you can validate if it belongs to this IPFS hash or this torrent file
<UukGoblin>
yeah
<UukGoblin>
my original thought would be downloading .torrent files directly from IPFS though
<Mateon1>
That would only be possible if either torrent support is put into every node, or if someone puts that specific torrent you're looking for into IPFS
<UukGoblin>
yeah, someone'd have to put that data into IPFS first I guess
<Mateon1>
Also, note that a lot of torrents have incredibly large piece length, reaching 8 MiB (possibly more, but haven't seen that in practice). That is incompatible with IPFS.
<Mateon1>
Or rather wherever we put the 2MB packet limit at (in libp2p probably)
<UukGoblin>
oh. hm.
jdloft has joined #ipfs
<UukGoblin>
so if I have a large file X, say 100MiB, I can't address it in IPFS as something like /ipfs/MULTIHASH(SHA1(X)) ?
<Mateon1>
Nope
rendar has joined #ipfs
<UukGoblin>
ah. ok.
<Mateon1>
IPFS chunks data, currently just naively at 256k intervals
<UukGoblin>
right, thanks for the explanation :-)
<UukGoblin>
I have one more question, having difficulty finding it: where (locally) does IPFS store the data of a file I'm trying to put into the network? Is it all copied to ~/.ipfs?
<UukGoblin>
also, does it store all the blocks it got there? Can I place a limit on the size of it?
<Mateon1>
UukGoblin: All block data is stored in $IPFS_PATH/blocks/ , I'm not personally sure what the $IPFS_PATH/datastore is used for
<Mateon1>
You can set a limit on the space used in the config, as well
<Mateon1>
I'm not perfectly sure that config var actually works, you might need to run the daemon with --enable-gc, but I think we had some issues with GC being really slow
<UukGoblin>
ok, thanks! so if I `ipfs add` a large file, my $IPFS_PATH/blocks will grow by that size too, right?
<UukGoblin>
or is it possible to make it use the file itself straight from the filesystem?
<Mateon1>
Yep, we are currently implementing a different way to store the blocks, referencing data from the existing files in the filesystem to avoid duplication.
<mib_kd743naq>
Kubuxu: ipfs.io seems to be according to that version link
<Kubuxu>
ok, I will look through your notes
zbyte64 has quit [Remote host closed the connection]
zbyte64 has joined #ipfs
<mib_kd743naq>
Kubuxu: I am clearly missing something very very minor, likely at how I specified the links...
<UukGoblin>
this https://ipfs.io gateway looks pretty interesting, by the way. Aren't you scared of being raided by police if someone hosts some nasty illegal stuff there?
<Kubuxu>
UukGoblin: it can't be worst than any possible upload site
<Mateon1>
Oh my god, we still need to fix all the "access denied" errors in gx while fetching packages
<Kubuxu>
echo -n 'f' | ipfs add --raw-leaves is probably what caused it.
<mib_kd743naq>
I actually add the leaves via ipfs add block
<Kubuxu>
Mateon1: are they because of too deep dir structure on windows
<Kubuxu>
hmm
<Kubuxu>
that might also work
<Kubuxu>
as type is not stored in blockstore
<Mateon1>
Kubuxu: No, they are random, happen multiple times when doing gx install.
<mib_kd743naq>
Kubuxu: let me try something fresh (different file / chunking), and show you the entire console
<Kubuxu>
sure
<mib_kd743naq>
brb 5mins
<Mateon1>
Kubuxu: Probably a FS race condition?
<Kubuxu>
no idea, I would love to help you as I know this position
<Mateon1>
Anyway, this is an annoyance, only a blocking bug for windows CI
<Mateon1>
It only happens to packages it doesn't already have, and since it doesn't happen all the time the gx install finishes after 5 - 20 tries
mguentner has joined #ipfs
jonnycrunch1 has joined #ipfs
jonnycrunch has quit [Read error: Connection reset by peer]
jonnycrunch1 is now known as jonnycrunch
<mib_kd743naq>
Kubuxu: so... I guess this is a bug in `ipfs block put` of some sort: here is a complete reproduction of my problem with a different set of chunks (even simpler)
<Kubuxu>
how does AllKeysChan work with blockstore
<Kubuxu>
CID
<Kubuxu>
whyrusleeping: are you around
<Kubuxu>
hmm
<footslogger>
hmm
<Kubuxu>
wait
b100d has quit [Ping timeout: 256 seconds]
<Kubuxu>
we are storing CID in blockstore?
<Kubuxu>
we are
<Kubuxu>
ok
<Kubuxu>
I didn't know that
<Kubuxu>
I thought we didn't/
<whyrusleeping>
Yeap, we are
<whyrusleeping>
we basically have to, otherwise lots of things get really hard (tm)
<whyrusleeping>
i didnt want to
<Kubuxu>
I didn't get that memo.
<whyrusleeping>
but jbenet and daviddias held me down and forced me to
<Kubuxu>
:/
<Kubuxu>
would be good if I got occasion to review this: https://github.com/ipfs/go-ipfs/pull/3290 PR (if it was tagged need review) and if it described this, quite a big, change.
<Kubuxu>
mib_kd743naq: that explains everything, use `ipfs block put -f raw`
<Kubuxu>
whyrusleeping: very sorry for accusing you of breaking something.
<Kubuxu>
I had no idea about this change.
<Kubuxu>
and I thought it was only a caching issue.
<whyrusleeping>
ha, no worries. Sorry for not being more clear about storing cids
<Kubuxu>
if you have some time: why blockstore has to be aware of CID? Doesn't it just need to know how to hash given block?
<Kubuxu>
but I can understand as then we loose some data
<whyrusleeping>
gc gets really hard
vtomole has quit [Ping timeout: 260 seconds]
<Kubuxu>
you have all the pins which you can iterate to mark all block that should stay, rest is removed.
<whyrusleeping>
how do you iterate through a graph if you don't know what type of node it is?
<Kubuxu>
you have root node
<Kubuxu>
you have link to root node (that has CID)
<Kubuxu>
and in that root node you have links with CID
<whyrusleeping>
so, the problem is when you have to match up the same object with different formats
<whyrusleeping>
and then things like 'ipfs refs local'
<Kubuxu>
true, there would be problem with iterating your whole storage and understanding what you got there
<whyrusleeping>
I just figured it would be easier if the data on disk was self describing :P
apiarian has quit [Ping timeout: 265 seconds]
apiarian has joined #ipfs
jedahan has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
lamarpavel has joined #ipfs
apiarian has quit [Ping timeout: 245 seconds]
apiarian has joined #ipfs
yoosi has quit [Remote host closed the connection]
yoosi has joined #ipfs
<lamarpavel>
Are any IPFS gatherings/events planned for this years CCC? I think there was a lightning talk last year, but I can't find anything regarding IPFS in the list of this years talk or the congress-wiki.
<whyrusleeping>
lamarpavel: we were trying to have a talk, but it got rejected
<Kubuxu>
lamarpavel: we sent one talk but it got rejected
<whyrusleeping>
and we failed at getting tickets this morning
<whyrusleeping>
:/
<lamarpavel>
oh, I see
<Kubuxu>
we also having hard time getting tickets :/
<Kubuxu>
llaaag
<lamarpavel>
Yeah, tickets are hard to get this time
biot has left #ipfs [#ipfs]
<lamarpavel>
If some of you get tickets, you could create an assembly
<lamarpavel>
It would be really nice to have a place to talk about IPFS and the things that happened over the past year. You know, for people who are interested but had no time following the development closely.
<whyrusleeping>
lamarpavel: Yeah, we definitely plan on doing that
<whyrusleeping>
if we can get tickets :/
<lamarpavel>
yeah
<lamarpavel>
Next ticket sale is on the 25th, I think
<whyrusleeping>
lol, black friday
<lamarpavel>
10 am UTC+1, I think
<whyrusleeping>
i'll be up, and having a nice internet connection
<whyrusleeping>
and multiple computers
<lamarpavel>
It's crazy how hard it is to get a ticket this year. Last year I got one about two weeks after the sale opened.
<Kubuxu>
ouch, go-datastore is deep in the dep tree :p
<Kubuxu>
lgierth: your tool (if it works) will be so useful for that.
<kevina>
Kubuxu: tell me about it, I had a hard time setting things up so I could hack on it and see the results in go-ipfs
herzmeister has quit [Quit: Leaving]
<Kubuxu>
you can always try using `gx update --with-deps` tales are that it worked for someone
herzmeister has joined #ipfs
mildred1 has joined #ipfs
<whyrusleeping>
Yeah, ideally changes to go-datastore are very infrequent
mildred has quit [Ping timeout: 256 seconds]
mib_kd743naq has joined #ipfs
<mib_kd743naq>
hi again
<mib_kd743naq>
so I am... not entirely sure what you guys are saying
<mib_kd743naq>
echo -n "abc" | ipfs block put -f raw - vs echo -n "abc" | ipfs block put -
<mib_kd743naq>
^^ are these two different things on the low level?
<mib_kd743naq>
because the hashes they return ( the merkledag-like Qm and the raw-like z... ) point to the same sha256 when decomposed
<mib_kd743naq>
Kubuxu, whyrusleeping: ^^
<Kubuxu>
yes
<Kubuxu>
in storage they have different identifiers
kulelu88 has quit [Quit: Leaving]
<Kubuxu>
In general, most of the things should be using CID now
<Kubuxu>
if you have choice CID vs no-CID
<Kubuxu>
choose CID
<mib_kd743naq>
well... a "strict constraint" of what I am trying to do is that https://ipfs.io/ipfs/<thing here> works from day one and works essentially forever
<mib_kd743naq>
my impression is that the only thing satisfying this today is a unixfs mdag