aschmahmann changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.7.0 and js-ipfs 0.50.2 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of Conduct: https://git.io/vVBS0
fogWraith has joined #ipfs
AgenttiX has joined #ipfs
{MelissaMinerva} has joined #ipfs
enricomarinoDisc has joined #ipfs
chicoxyzzy has joined #ipfs
BPC has joined #ipfs
Mai-HsuanKevinCh has joined #ipfs
nickjohnson has joined #ipfs
zolk3ri[m] has joined #ipfs
blzDiscord[m] has joined #ipfs
IonutMatisDisco4 has joined #ipfs
ruko[m] has joined #ipfs
LandrylDiscord[m has joined #ipfs
katakotoDiscord[ has joined #ipfs
}Discord[m] has joined #ipfs
tooetooDiscord[4 has joined #ipfs
manfredDiscord[m has joined #ipfs
JordanKrageDisco has joined #ipfs
dermicha[m]1 has joined #ipfs
TristanDiscord[m has joined #ipfs
theuyserDiscord[ has joined #ipfs
arndot[m]1 has joined #ipfs
Panos[m] has joined #ipfs
M9672Discord[m] has joined #ipfs
moul has joined #ipfs
nofwayyDiscord[m has joined #ipfs
peddie has joined #ipfs
xlcDiscord[m] has joined #ipfs
captain-nemoDisc has joined #ipfs
test123Discord[m has joined #ipfs
obernardovieiraD has joined #ipfs
null_radix[m] has joined #ipfs
prettymuchbryce has joined #ipfs
Miaourt has joined #ipfs
georgyo has joined #ipfs
imdeni has joined #ipfs
indra[m]1 has joined #ipfs
chmanieDiscord[4 has joined #ipfs
zwelsternDiscord has joined #ipfs
zaibon[m] has joined #ipfs
brewskiDiscord[m has joined #ipfs
cepmee_69[m] has joined #ipfs
Jassu has joined #ipfs
paulcowgillDisc4 has joined #ipfs
mZDiscord[m]1 has joined #ipfs
sernaDiscord[m]4 has joined #ipfs
ShinyRice[m] has joined #ipfs
verum_alpha[m] has joined #ipfs
thatguyDiscord[m has joined #ipfs
hubaDiscord[m] has joined #ipfs
gamma-zro[m] has joined #ipfs
pandaru[m] has joined #ipfs
jojojoDiscord[m] has joined #ipfs
obensourceDiscor has joined #ipfs
william_shakesDi has joined #ipfs
RichardLittDisco has joined #ipfs
RomainDiscord[m] has joined #ipfs
im-37[m] has joined #ipfs
silwol has joined #ipfs
Devon[m] has joined #ipfs
dynodeio[m] has joined #ipfs
bmiller59Discord has joined #ipfs
anthony-albertor has joined #ipfs
cannabysDiscord[ has joined #ipfs
JakeHemmerle[m] has joined #ipfs
testAccount63Dis has joined #ipfs
FeNiXDiscord[m]2 has joined #ipfs
JohnnyMilkshakes has joined #ipfs
gmelodieDiscord[ has joined #ipfs
elisa2020[m] has joined #ipfs
zenoxygen[m] has joined #ipfs
yakkleDiscord[m] has joined #ipfs
jackmajackma[m] has joined #ipfs
sachaDiscord[m] has joined #ipfs
dataDiscord[m]1 has joined #ipfs
RupersiaDiscord[ has joined #ipfs
cereal_killerDi4 has joined #ipfs
The_8472 has joined #ipfs
bekoDiscord[m] has joined #ipfs
codynhatDiscor10 has joined #ipfs
swills has joined #ipfs
Newami has quit [Quit: Leaving]
joocain2 has quit [Ping timeout: 240 seconds]
joocain2 has joined #ipfs
ipfs-stackbot1 has quit [Remote host closed the connection]
ipfs-stackbot1 has joined #ipfs
koo5_ has quit [Quit: No Ping reply in 180 seconds.]
koo5 has joined #ipfs
ctOS has quit [Quit: Connection closed for inactivity]
Ecran10 has quit [Quit: Going offline, see ya! (www.adiirc.com)]
zeden has joined #ipfs
reit has quit [Quit: Leaving]
_whitelogger has joined #ipfs
user_51_ has joined #ipfs
user_51 has quit [Ping timeout: 240 seconds]
<JCaesar>
Hm. What can I do if ipfs refs -r QmNgAS66HpHrZHaDJ4Mzy6nM3LzVseUJ3Z9RfmT5r9ANgs prints Error: merkledag: not found?
jrt is now known as Guest87953
Guest87953 has quit [Killed (rothfuss.freenode.net (Nickname regained by services))]
jrt has joined #ipfs
Adbray has quit [Quit: Ah! By Brain!]
Belkaar has quit [Ping timeout: 240 seconds]
Belkaar has joined #ipfs
Belkaar has joined #ipfs
reit has joined #ipfs
Nact has quit [Quit: Konversation terminated!]
john2gb has quit [Ping timeout: 240 seconds]
john2gb has joined #ipfs
zopsi has quit [Max SendQ exceeded]
zopsi has joined #ipfs
fogWraith has quit [*.net *.split]
AgenttiX has quit [*.net *.split]
enricomarinoDisc has quit [*.net *.split]
chmanieDiscord[4 has quit [*.net *.split]
indra[m]1 has quit [*.net *.split]
{MelissaMinerva} has quit [*.net *.split]
zaibon[m] has quit [*.net *.split]
zwelsternDiscord has quit [*.net *.split]
Miaourt has quit [*.net *.split]
imdeni has quit [*.net *.split]
moul has quit [*.net *.split]
prettymuchbryce has quit [*.net *.split]
nickjohnson has quit [*.net *.split]
chicoxyzzy has quit [*.net *.split]
georgyo has quit [*.net *.split]
BPC has quit [*.net *.split]
cepmee_69[m] has quit [*.net *.split]
brewskiDiscord[m has quit [*.net *.split]
mZDiscord[m]1 has quit [*.net *.split]
Mai-HsuanKevinCh has quit [*.net *.split]
paulcowgillDisc4 has quit [*.net *.split]
sernaDiscord[m]4 has quit [*.net *.split]
zolk3ri[m] has quit [*.net *.split]
blzDiscord[m] has quit [*.net *.split]
ruko[m] has quit [*.net *.split]
IonutMatisDisco4 has quit [*.net *.split]
LandrylDiscord[m has quit [*.net *.split]
Jassu has quit [*.net *.split]
katakotoDiscord[ has quit [*.net *.split]
}Discord[m] has quit [*.net *.split]
manfredDiscord[m has quit [*.net *.split]
dermicha[m]1 has quit [*.net *.split]
tooetooDiscord[4 has quit [*.net *.split]
JordanKrageDisco has quit [*.net *.split]
theuyserDiscord[ has quit [*.net *.split]
TristanDiscord[m has quit [*.net *.split]
Panos[m] has quit [*.net *.split]
arndot[m]1 has quit [*.net *.split]
M9672Discord[m] has quit [*.net *.split]
null_radix[m] has quit [*.net *.split]
nofwayyDiscord[m has quit [*.net *.split]
peddie has quit [*.net *.split]
captain-nemoDisc has quit [*.net *.split]
xlcDiscord[m] has quit [*.net *.split]
obernardovieiraD has quit [*.net *.split]
test123Discord[m has quit [*.net *.split]
thatguyDiscord[m has quit [*.net *.split]
verum_alpha[m] has quit [*.net *.split]
bmiller59Discord has quit [*.net *.split]
anthony-albertor has quit [*.net *.split]
dynodeio[m] has quit [*.net *.split]
obensourceDiscor has quit [*.net *.split]
jojojoDiscord[m] has quit [*.net *.split]
hubaDiscord[m] has quit [*.net *.split]
cannabysDiscord[ has quit [*.net *.split]
pandaru[m] has quit [*.net *.split]
RomainDiscord[m] has quit [*.net *.split]
william_shakesDi has quit [*.net *.split]
Devon[m] has quit [*.net *.split]
silwol has quit [*.net *.split]
gamma-zro[m] has quit [*.net *.split]
im-37[m] has quit [*.net *.split]
sachaDiscord[m] has quit [*.net *.split]
RichardLittDisco has quit [*.net *.split]
codynhatDiscor10 has quit [*.net *.split]
ShinyRice[m] has quit [*.net *.split]
JakeHemmerle[m] has quit [*.net *.split]
testAccount63Dis has quit [*.net *.split]
yakkleDiscord[m] has quit [*.net *.split]
elisa2020[m] has quit [*.net *.split]
FeNiXDiscord[m]2 has quit [*.net *.split]
zenoxygen[m] has quit [*.net *.split]
bekoDiscord[m] has quit [*.net *.split]
gmelodieDiscord[ has quit [*.net *.split]
RupersiaDiscord[ has quit [*.net *.split]
JohnnyMilkshakes has quit [*.net *.split]
jackmajackma[m] has quit [*.net *.split]
cereal_killerDi4 has quit [*.net *.split]
The_8472 has quit [*.net *.split]
dataDiscord[m]1 has quit [*.net *.split]
VoiceOfReason has quit [Quit: Free ZNC ~ Powered by LunarBNC: https://LunarBNC.net]
17SAAVVG7 has quit [Read error: Connection reset by peer]
rjknight has joined #ipfs
jhiesey has quit [Ping timeout: 240 seconds]
^iverson has quit [Ping timeout: 240 seconds]
imdeni has quit [Ping timeout: 240 seconds]
xinbenlv has quit [Ping timeout: 240 seconds]
jhiesey has joined #ipfs
noresult has quit [Ping timeout: 240 seconds]
imdeni has joined #ipfs
noresult_ is now known as noresult
xinbenlv has joined #ipfs
iverson has joined #ipfs
Taoki has joined #ipfs
LHLaurini has joined #ipfs
Taoki has quit [Read error: Connection reset by peer]
Taoki has joined #ipfs
<swedneck>
CIDs don't contain the chunking algorithm used, right?
<aschmahmann1>
Nope
<aschmahmann1>
Chunking is specific to UnixFS. I guess theoretically if UnixFS had metadata support you could choose to include that in a file/directory root... but there's a huge potential variety in even what valid UnixFS files look like so it'd likely be pretty complicated
mindCrime_ has joined #ipfs
<swedneck>
you say specific to UnixFS, isn't that a part of the ipfs standard? What other options are there?
<aschmahmann1>
IPFS is more than happy to transport arbitrary IPLD DAGs
<aschmahmann1>
generally applications that are looking to move around their application-specific data via IPFS can use IPLD structures specific to their application instead of just serializing everything to a file and then using a chunking algorithm.
<aschmahmann1>
IPFS provides UnixFS to deal with a large number of use cases around moving around pre-existing files, and generally making things easy to work with.
<aschmahmann1>
Then you'll start realizing that actually you want to subdivide and reference parts of files, get better benefit from deduplication of data within your applications structures, or want people who serialize the same data to always end up with the same CID (e.g. protobuf does not always serialize the same way) and so you'll start looking at building a more custom IPLD schema on top of an existing codec like DagCbor.
<aschmahmann1>
For example, you're familiar with using Protobufs to transport data in your language so you might decide to just serialize the data with Protobufs and turn it into a UnixFS object to share over IPFS.
<swedneck>
alright
<swedneck>
i feel that unixfs (or more specifically files) is a very very important usage of IPFS though, so wouldn't it make sense to find some way to include all the vital data in the CID?
<swedneck>
otherwise how can you verify data?
<aschmahmann1>
What are you trying to verify? Is this something like wanting untrusted gateways?
teaman has quit [Ping timeout: 260 seconds]
<swedneck>
like making sure the data a peer sends is actually what you want
<swedneck>
you can't check the hash of the data without all the settings used to add it to ipfs, since if you get the wrong hashing algorithm the hash will end up different, and thus you would erroneously conclude the data is incorrect
<swedneck>
* you can't check the hash of the data without all the settings used to add it to ipfs, since if you get the wrong chuking algorithm the hash will end up different, and thus you would erroneously conclude the data is incorrect
<swedneck>
* you can't check the hash of the data without all the settings used to add it to ipfs, since if you get the wrong chunking algorithm the hash will end up different, and thus you would erroneously conclude the data is incorrect
<aschmahmann1>
If I ask you for CID bafyabc you cannot send me any data other than what I've asked for and convince me it's bafyabc, the hashes just won't match.
<aschmahmann1>
This is so strict that even if you send me the same file, but chunked differently I'll reject it.
<aschmahmann1>
Where "send me" is defined as using `ipfs get bafyabc`
<swedneck>
right, so it's important for the reciever to know all the settings that were used when adding the data to IPFS
<swedneck>
but currently you just have to presume the default go-ipfs settings were used..
<aschmahmann1>
You can download Qmf... without me telling you what the settings are
<dnl[m]>
Is the default hashing algorithm still SHA256?
<swedneck>
uh, surely a peer could just pretend to have that file and send something completely different?
jcea has quit [Remote host closed the connection]
jcea has joined #ipfs
<swedneck>
you could just wait for someone to request a CID and respond as if you had it, and just send a completely different block of data
ib07 has joined #ipfs
<aschmahmann1>
If you send me a different block of data then the one I asked for then I won't accept it since when I hash the received block of data I'll notice that it does not match the hash that is embedded in the CID
<swedneck>
or am i missing something?
<nimaje>
but ipfs would just assume that sended data got corrupted somewhere as the hashes don't match or did I understand something wrong?
<aschmahmann1>
> Is the default hashing algorithm still SHA256?
<aschmahmann1>
yes
<swedneck>
right, but how do you know the correct settings to check the hash?
<swedneck>
you could use the wrong chunking algorithm or similar, and get a false negative
<swedneck>
oh right, hmm, i guess a false positive isn't possible
ib07 has quit [Ping timeout: 246 seconds]
<swedneck>
but still, a false negative seems possible?
<aschmahmann1>
You mean like my 2 panda pictures above with different CIDs?
<swedneck>
yes
<aschmahmann1>
Yep, that's a bit of a complex problem that requires really drilling down to concrete use cases to figure out what type of solution to look for.
<aschmahmann1>
For example, there might be good reasons why two people might want to encode the same file differently (e.g. large chunks vs small chunks)
<aschmahmann1>
This makes it quite difficult without some fancy cryptography for me to be able to download some of the data from both of the uploaders since they're using different structures to store the data. This is true even if I manage to get an identifier for some canonical representation of the file (e.g. SHA256 of all the bytes)
<aschmahmann1>
Generally speaking it's not so bad that two users could get different CIDs out of chunking the same file since data that's shared with CIDs doesn't experience this problem. e.g. Alice adds a picture P and gets CID_A, she publishes CID_A on her blog Bob downloads CID_A then he tweets out a link to CID_A. Bob isn't separately downloading and then re-adding the data into IPFS.
<swedneck>
oh right i completely forgot that CIDs are composed of either individual blocks or a list of other CIDs..
<swedneck>
and you don't need extra data to get the same CID from those things, right?
<aschmahmann1>
Each block points to other blocks via their CIDs forming a tree of blocks linked by CID
Taoki has joined #ipfs
<rvklein[m]>
"Yo dawg I heard u liek secure hashsums"
arthuredelstein has joined #ipfs
<swedneck>
hmm, would be neat if gateways had an option to download the full CID chain, and the corresponding end blocks
<rvklein[m]>
That's a Merkle tree right? Those are kinda fun at face value. Very useful for this problem space however.
<swedneck>
that seems like a simple solution to being able to verify stuff downloaded via gateways
<rvklein[m]>
Isn't a gateway HTTP?
jessehattabaugh has joined #ipfs
<swedneck>
<rvklein[m] "Isn't a gateway HTTP?"> the current ones are yeah, but i'm thinking they could just serve the DAG as a text file or something
<aschmahmann1>
basically, why fight with gateway shenanigans when you could just run IPFS
<swedneck>
backwards compatibility?
<rvklein[m]>
The thing is about HTTP being on TCP traditionally is that web pages are supposed to load and be parsed and rendered in an ordered stream.
<swedneck>
i certainly understand making this a low priority, but i see no reason to exclude the feature
<rvklein[m]>
The gateway has to download the whole files just for you to do them again.
<rvklein[m]>
*download them afain
* rvklein[m]
headdesks
<swedneck>
it could also be a nice educational feature, seeing the individual blocks and the structure of the DAG (i hope i'm using the right terms)
<rvklein[m]>
IPFS is wildly powerful. I'm almost glad that the dat protocol Beaker browser had broken for me since it forced me to attempt the IPFS command line.
<swedneck>
yeah i was about to say, it's basically the explorer but as an API
ctOS has quit [Quit: Connection closed for inactivity]
sympt_ has joined #ipfs
jessehattabaugh_ has joined #ipfs
<rvklein[m]>
I mean it's daunting and all, and some things are confusing in a way that could be explained better inside the util help strings.
LHLaurini has quit [Ping timeout: 258 seconds]
<aschmahmann1>
> and some things are confusing in a way that could be explained better inside the util help strings.
<aschmahmann1>
I'd certainly bet this is true. If you have any suggestions feel free to file a PR.
<rvklein[m]>
It's good though. I think I might like it better than dat now.
jrgcombr__ has quit [Quit: Leaving]
cheetypants has joined #ipfs
sympt has quit [Ping timeout: 272 seconds]
cheet has quit [Ping timeout: 260 seconds]
KeiraT has quit [Remote host closed the connection]
KeiraT has joined #ipfs
<rvklein[m]>
aschmahmann: thank you. I suppose my primary concern regarding such is that associating a content hash to an IPNS address was a bit of a guessing game as to what data it wants in the command and where/which args. This is all anecdote of course.
<rvklein[m]>
But hey at least when you figure it out then it is quite solid.
<rvklein[m]>
Or at least seems that way. I know it's not magic. But it can feel that way.
<aschmahmann1>
The offline functioning stuff definitely feels pretty magical when compared to most web experiences
Pikipeanut has quit [Quit: A bientôt!]
<rvklein[m]>
You can technically download bittorrent chunks somewhat in-order and maybe buffer them a bit so they can be streamed through that way to the client.
mindCrime_ has quit [Ping timeout: 260 seconds]
<swedneck>
i think this is a decent example of what i'm thinking of: ipfs://bafybeibadtvn2biynu6tm6dx7h7a4ubjfst52j4i43pteglvcjkzaxsvaa
<rvklein[m]>
But I don't consider that very polite behaviour to load balancing and the model of the thing.
imdeni has quit [Read error: Connection reset by peer]
imdeni has joined #ipfs
<swedneck>
oh wait, the go-ipfs gateway already shows you named links, couldn't this just be extended to show linked blocks?
KempfCreative has quit [Ping timeout: 260 seconds]
<aschmahmann1>
Swedneck: you sure your node is online and reachable? I'm getting unreachable errors trying to download that CID (finding the provider record just fine, but can't connect to the node).
<swedneck>
oh crud my server is offline
<swedneck>
any way to make 2 nodes on the same network publically reachable?
<aschmahmann1>
I can definitely see the argument behind wanting a more "IPLD-like" view like explore.ipld.io available via the Gateway (e.g. the blocks and links). I think the counter argument is that the gateway is mostly for backwards compatibility with files/folders.
<aschmahmann1>
I'd do some looking for issues on GitHub as I think this may have come up before, but definitely feel free to suggest in an issue.
<aschmahmann1>
> any way to make 2 nodes on the same network publically reachable?
<aschmahmann1>
If you have UPnP on your router and your ISP isn't super obnoxious it should just work. Alternatively you could assign them different ports for manual port-forwarding (e.g. public-ip:4001 -> machine1:4001 and public-ip:5555 -> machine2:4001)
<swedneck>
hmm, my router runs openWRT so maybe that doesn't support upnp
<swedneck>
i'll try changing the port
<swedneck>
can you reach me now?
<aschmahmann1>
hmmm... actually I'm realizing if you change the ports you might need to also change the publicly announced addresses in the config due to https://github.com/ipfs/go-ipfs/issues/7621
<rvklein[m]>
Swedneck: it should support upnp
<aschmahmann1>
Check that the addresses announced by `ipfs id` look dialable to you
<swedneck>
yeah i thought ahead and changed the announced port :D
<rvklein[m]>
Yeah I'd assign the different machines alt ports
<rvklein[m]>
Having a node to sync to over a LAN sounds like a good strategy though. Especially if you have a lot of content.
<swedneck>
i've been using my server to proxy stuff from my desktop, basically
<rvklein[m]>
Yeah not bad
bakuzan has joined #ipfs
zeden has quit [Quit: WeeChat 2.9]
dethos has quit [Ping timeout: 260 seconds]
koo5 has quit [Ping timeout: 272 seconds]
mowcat has quit [Remote host closed the connection]
LHLaurini has joined #ipfs
<dnl[m]>
What are you guys using IPFS for nowadays?
<swedneck>
primarily sharing files and screenshots
<dnl[m]>
Was to be expected, any use for local networks?
<dnl[m]>
for example, can it replace Synthing, rsync, etc.
bakuzan has quit [Quit: Leaving]
<swedneck>
you'd have to build an application ontop of IPFS for something like that
<dnl[m]>
Yeah.
<swedneck>
sadly the syncthing devs don't seem willing to make it use IPFS for some reason
<dnl[m]>
Is it easy to limit connections from your IPFS client, if you only want to share files locally and not have it accessible through the internet.
<dnl[m]>
Swedneck: Odd, unless this is too complex.
<swedneck>
<dnl[m] "Is it easy to limit connections "> i think for that you'd configure the node to only advertise a local IP, such as 192.168.X.X
<dnl[m]>
Right.
<swedneck>
<swedneck "i think for that you'd configure"> so that running `ipfs id` doesn't show you any IP that's reachable on the internet
<dnl[m]>
Well if you have a few computers on your local network, where you share files with, but not all of them have this configuration then ultimately it will be accessible through the internet, no?
<swedneck>
not sure
<swedneck>
oh right, might be better to use a private IPFS network
<dnl[m]>
Hm, sorry for asking, I should just try out IPFS ;)