notduncansmith has quit [Read error: Connection reset by peer]
therealplato has quit [Ping timeout: 272 seconds]
Wallacoloo has joined #ipfs
Daboloskov has joined #ipfs
headbite has quit [Ping timeout: 255 seconds]
inconshreveable has quit [Ping timeout: 265 seconds]
inconshreveable has joined #ipfs
notduncansmith has joined #ipfs
<whyrusleeping>
we have weird pinning issues if a pin operation fails midway
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
its fairly unavoidable, perhaps we should have a 'pin clean' command that removes all indirect pins, and recounts them based on the recursively pinned objects
rschulman has quit [Remote host closed the connection]
rschulman1 is now known as rschulman
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
patcon has quit [Ping timeout: 244 seconds]
sharky has joined #ipfs
<jbenet>
substack: signatures should be a different standard, not bundled into multihash. haven't needed to specify them yet.
notduncansmith has joined #ipfs
<jbenet>
whyrusleeping: what's not finished?
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
substack: multihash is a multi-representation format. you dont want to store hashes unpacked. and you want to pack these codes as much as possible when you're storing millions of them.
<jbenet>
hence binary and human readable, depending on what you want.
<substack>
maybe for hashes
hellertime has joined #ipfs
www1 has quit [Ping timeout: 264 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
jbenet: was referring to the list of hashes mutlihash supports
<whyrusleeping>
jbenet: also, did you have an issue open for an ipfs tray application?
kallaballa has joined #ipfs
<kallaballa>
whyrusleeping: feel like discussing service discovery?
lgierth_ has quit [Quit: Ex-Chat]
<jbenet>
whyrusleeping: dont have an issue open yet
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
kallaballa: sure thing
<whyrusleeping>
so, the ideas about service discovery are fairly straightforward
pfraze has joined #ipfs
<whyrusleeping>
we will want to include a list of services supported by each peer during the handshake
<whyrusleeping>
that way you know the capabilities of each peer youre connected to
<whyrusleeping>
but that only helps you know that info for your immediate peers
Wallacoloo has quit [Ping timeout: 240 seconds]
<whyrusleeping>
for discovery peers who support a given service globally, i was planning on abusing our providers system
<whyrusleeping>
right now, any peer can announce that they have a given block through the dht
<whyrusleeping>
we could make a block out of the name of the service in question
<whyrusleeping>
(or maybe a protobuf/json structure including that service and some other info maybe)
www has joined #ipfs
<whyrusleeping>
and any node that 'provides' that block, supports the service in question
<whyrusleeping>
its not a perfect method, but it could work with no changes to the existing system
Wallacoloo has joined #ipfs
<whyrusleeping>
jbenet: also, mars is in the state caused by that go-peerstream bug again
Wallacoloo has quit [Read error: Connection reset by peer]
Wallacoloo has joined #ipfs
<kallaballa>
whyrusleeping: i understand
<kallaballa>
but you also mentioned the findprovs doesn't list all possible peers. why is that? or why is that no problem in your scenario?
<whyrusleeping>
kallaballa: findprovs doesnt list all possible peers because we set a maximum number to return, and because we set a timeout
<kallaballa>
makes sense. thx :)
<whyrusleeping>
if we set both of those parameters higher, it should return all nodes in the network
<whyrusleeping>
(that provide the given value)
<kallaballa>
ic
<whyrusleeping>
no problemo!
<kallaballa>
just to make my case more clear. i have a number of embedded devices with a playlist. i would like to enable decentralized playlist sharing via ipfs. but in order to do that they need to find each other.
<kallaballa>
sounds like a good use case?
<whyrusleeping>
for sure!
<whyrusleeping>
you could also use corenet for communication
<whyrusleeping>
allows you to get a tcp like socket to another peer just using their PeerID
<krl>
the problem now with the current geoip, is that you will probably not get much structural sharing on updates
<zignig>
krl: indeed , we need a way to _merge_ ipns entries
<krl>
would be nice to have something that diffs the current tree with the new data, and tries to preserve as much as possible of the old
<zignig>
krl: with multile sources you can check that at least 2 nodes have the same answer and merge that.
<krl>
you could probably calculate a diff from the csv files and have something that more cleverly updates the root hash
<zignig>
you can diff merkel dags directly.
<cryptix>
zignig: thanks for the link ( or /ipfs/QmT8tNW8a5L5gXDd3qwhtii5gAg3oTqgCvASiS4bcEShMn :)
<cryptix>
or ipfs:obably calculate a diff from the csv files and have something that more cleverly updates the root hash
<cryptix>
lol sorry
<krl>
yes, but say you add an entry in the middle of one 32-bucket, then the rest of the tree will be the same data but skewed off by one, leading to no deduplication
<zignig>
haha ! , add it to the porridge that is ipfs at the moment ! BRING ON THE NAMESYS !
<cryptix>
pastafail.. can we get the firefox and chrome addon guys to use the same hook? firefox does ipfs:$hash while chrom* does /ipfs/$hash to redirect to local gw
<zignig>
krl: the blocking kind of borks the diff. if would work on a file level.
<zignig>
krl: oh yes for data like you geoip, quite right.
<krl>
you could add some wiggle-room to the leaves, allowing them to have between 16 and 48 entries, for example
<zignig>
krl: can i have the latest geoip hash.
marklock has joined #ipfs
<zignig>
that should be pinned. ;)
<krl>
yes, i think (hope) jbenet managed to pin the whole thing already
<krl>
cryptix: how do you get your own addres in this format?
<zorun>
"ipfs id", I guess
<cryptix>
krl: ipfs id
<krl>
ah thx
<cryptix>
ipfs refs -r doesnt seem to have a timeout - also mirroring it locally and it was blocked for much longer than the vps process and now it also started growing again
<cryptix>
seems more practical to mirror it with refs and pin it later currently
<krl>
oh, refs actually also pulls the data?
<krl>
aaaah, wish i knew this before. ok cool
<cryptix>
krl: just noticed it :) wondering why the local 'refs -r' didnt timeout
<cryptix>
and yea - a 2nd run of refs prints them instantly - so i fetches i assume
<krl>
it's super slow though
<cryptix>
i wonder if bitswap already has bandwith sharding smarts (bittorrent like giving 1/n to n peers)
<krl>
lots of small objects is slower than fewer large objects it seems?
<cryptix>
krl: yea - i think thats related to figuring out the next refs on each tree node recursivly. a archived dump would be easier to spread i assume
<krl>
yeah, like ipfs import geoip.mdag
<cryptix>
cc whyrusleeping transfer/bw sharding? /\
<krl>
i'm getting one object literally every 20s, how can this be?
<krl>
and i have a direct connection between my laptop and my vhost
<cryptix>
krl: i'm stealing them from you :P
<cryptix>
16402 and counting ^^
<krl>
i'm at 34k @ vhost
<krl>
and i'm stuck at 34199 for some reason
<krl>
i wonder if you're pulling from my laptop or my vhost, or both
<krl>
or from jbenet
<krl>
hmm. could i just manually copy the blocks?
<krl>
will try.
<cryptix>
krl: yea init a 2nd node on your laptop - get all the geoip data and transfer the whole ipfs dir to the vhost
<cryptix>
or transfer all your data tot he vhost.. depends :)
<krl>
will pack up a tar of geoip data + deprecated geoip data + random crap
<krl>
:)
<krl>
if it works i'll do a tar with only the relevant stuff
<krl>
should be able to pipe refs -r into tar paths
<krl>
unix voodoo to the rescue again!
<cryptix>
:) go for it
<krl>
who made the .ipfs/blocks storage btw? is there a reason why the directory split is so large?
<krl>
git uses /a7/rest of hash afaik
<krl>
hmm. i don't understand the format
tilgovi has joined #ipfs
<krl>
ok it seems taring will not work, if i understand this correctly
<krl>
i would assume the format was something like .ipfs/blocks/QQ/3BUpPjgYiTdhp4H9YWSCtoFXs8t91njhpvXNNLd3yB.block
<krl>
or maybe even .ipfs/blocks/QQ/3B/UpPjgYiTdhp4H9YWSCtoFXs8t91njhpvXNNLd3yB.block
<krl>
or actually even .ipfs/blocks/Qm/QQ/3B/UpPjgYiTdhp4H9YWSCtoFXs8t91njhpvXNNLd3yB.block if we want to be multihash future proof
<krl>
at the moment we have blocks/122008f9/122008f9189d8a7a36e779472fe2fb7d2a3860b1dcab7b5dddd1237cff5560a68b3e.data
<krl>
i should research this
<cryptix>
krl: the Qm* hashes are base58 encoded - you dont want to work with that directly
<krl>
ah, i see
<cryptix>
the blocks stuff is homegrown by whyrusleeping, i think - backed by metadata in leveldb
<krl>
so the format is probably like that
<krl>
so 12200 is probably Qm..
<cryptix>
krl: i think you can use the multihash tool to convert them iirc
<jbenet>
krl: the slowness you're seeing is the same I've been encountering. whyrusleeping and I are on it
<jbenet>
(Mostly whyrusleeping )
<ehmry>
there is more than one way to base58 encode though, some encoders use blocks and padding
inconshreveable has quit [Read error: Connection reset by peer]
nickemery has joined #ipfs
ehmry has quit [Read error: Connection reset by peer]
nickemery is now known as ehmry
mildred has joined #ipfs
Taek has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
patcon has joined #ipfs
inconshreveable has joined #ipfs
ehmry has quit [Quit: Leaving]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
hellertime has quit [Quit: Leaving.]
pfraze_ has joined #ipfs
Wallacoloo has joined #ipfs
Wallacoloo has quit [Client Quit]
Wallacoloo has joined #ipfs
zooko has quit [Ping timeout: 256 seconds]
marklock has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Wallacoloo has left #ipfs [#ipfs]
zooko has joined #ipfs
therealplato1 has joined #ipfs
marklock has quit [Ping timeout: 246 seconds]
marklock has joined #ipfs
therealplato has quit [Ping timeout: 245 seconds]
<marklock>
Bouncing Idea: IPFS leveraged for an education system(alread mentioned this in breif to jbenet). Every concept is represented by a file containing its description and then links to other files it connects to as a concept. Using those links users can build networks of ideas. If a user wants to learn something they host the file and they are naturally connected to other users with it. Making sense somewhat?
patcon has quit [Ping timeout: 250 seconds]
<Sonarpulse>
sounds like the web's original use
pfraze_ has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
<marklock>
Sonarpulse: pretty much but in this case optimized for people learning I guess, and finding other peers to work with
notduncansmith has quit [Read error: Connection reset by peer]
zooko has left #ipfs ["#tahoe-lafs"]
octalberry has joined #ipfs
octalberry has quit [Client Quit]
<headbite>
marklock: isn't what you're talking about just social networking at it's core?
<marklock>
headbite: pretty much I guess, but social networking not based around personal connections but connections of something else that those parties are brought together over if that makes sense
<headbite>
so reddit?
<techgrin>
marklock: I get it, like it.
domanic has joined #ipfs
<whyrusleeping>
krl: yeah, my highest priority right now is fixing that slowness
<whyrusleeping>
i'm working on a refactor and cleanup of bitswap first
<whyrusleeping>
it will be able to give me a much better view of whats going on
mildred has quit [Ping timeout: 255 seconds]
Daboloskov has joined #ipfs
<techgrin>
headbite: marklock: Figuriatively, think of a shelf in your house with lots of books. You pick out a book that you like. Whoah someone else is looking at that same book, and you're connected directly to them as long as you're holding the book. Not quite reddit because reddit requires going to the figuritive library.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Daboloskov has quit [Client Quit]
Daboloskov has joined #ipfs
<headbite>
techgrin: I could see that as an app that opens a chatroom/forum next to each file/hash. Might even be able to leave notes like warning file contains virus or whatever.
Daboloskov has quit [Client Quit]
<techgrin>
headbite: yeah! that would be a handy app
<headbite>
but if it's not moderated then anyone could just flood it to the point of being useless.... I don't know... there might be something there... but it would have to be thought out more.
<marklock>
techgrin: Thats the idea, and your books all have a connected section which lead you to other books that you can pull into your own library
<marklock>
headbite: since it would be p2p how would you flood it? I can see it being relatively easy to set up a white/black-list or specific trusted nodes or anything on top of it that could assure safety
<marklock>
techgrin: How would that work if someone could just rehost the file differently?
<headbite>
I think this idea of being able to send pins to other nodes might be how to get comments together. I haven't been following that all that close.
<whyrusleeping>
jbenet: i'm still seeing some sizeable address bombs
<whyrusleeping>
nodes with >20 addresses broadcasted
<marklock>
In general I can see it being useful for communities springing up around files and data. Though some sort of balance with permanance and scale, are all users connected together, are only a few, do logs stay somehow, etc etc
mildred has joined #ipfs
<techgrin>
marklock: You could have some md5sum of the content... hmm no, that would be too easy to alter a bit and rehost. Bittorrent style. If it isn't popular, it dies.
<marklock>
I think it could be more useful making the content completely impermanent so nothing is there to keep it in the system. So everytime a user connects based on a file its a new interaction.
<jbenet>
whyrusleeping: all different ports? We should handle symmetric NATs better :/
<whyrusleeping>
yeah, its the different ports thing
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
patcon has joined #ipfs
grncdr has joined #ipfs
<whyrusleeping>
jbenet: it might be nice if you could inform a peer which addresses you failed to dial for them
<jbenet>
We should probably not use an "observed address" until 2 peers have seen it be the same
<jbenet>
Hmmm maybe. Not even sure about that.
<whyrusleeping>
ooooh, yes
<whyrusleeping>
connectivity is really hard
<jbenet>
(It's heuristics after heuristics, we're fumbling between use cases based on what's more probable to happen, as there is no solution without relay)
<whyrusleeping>
in my opinion, getting two computers talking to eachother is the hardest problem in distributed computing
grncdr has quit [Ping timeout: 252 seconds]
<jbenet>
In the end relaying across WAN is less of a big deal than relaying on backbone to reach someone in my LAN
<jbenet>
(So relay to fight symmetric nat is more ok)
marklock has quit [Ping timeout: 246 seconds]
marklock has joined #ipfs
<marklock>
So any other input/ideas on a social network built around files themselves?
blocktechceo has quit [Ping timeout: 246 seconds]
<whyrusleeping>
idk man, i just write code
<headbite>
might sound like a stupid question, but can I set a cron job to restart ipfs if it crashes?
<techgrin>
headbite: might be more suitable to use upstart or systemd
<marklock>
whyrusleeping: "just" writing code seems like an understatement
<marklock>
a code question then, could it be technically feasbile?
chriscool has joined #ipfs
<whyrusleeping>
i beleive our gateways are set up to autorestart when they die
<whyrusleeping>
i think it could be as simple as a shell script that does 'while !ipfs daemon`
<okket>
and then max out the cpu when a process fails repeatedly? e.g because oom or disk full? ;)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<techgrin>
marklock: I have an idea for a pinterest type thingy. some ipfs daemon downloads every hash it can find and keeps it if it's a picture. then the picture gets published on some pinterest-looking website.
<marklock>
whyrusleeping: I'll look into that approach, thanks!
<techgrin>
buut feel free to take the idea because I don't plan on doing it :P
<marklock>
techgrin: To what purpose? Couldn't it be set up with the external publishing?
<techgrin>
external publishing as in an ipfs gateway?
<techgrin>
The purpose would be... to aggregate pictures hosted on ipfs. not much use other than being a museum or something
<okket>
a museum of captchas ;)
<techgrin>
okket: haha that sounds funny
<techgrin>
marklock: it could be made social with comments? an art museum that you can do graffiti on
<marklock>
Wouldn't that be tied into again chat or permanent records around the files application? it could approach the graffiti thing with files you link to modified version of the same file etc. and then maybe one person could host a mueseum or whatever just pulling the files and storing logs.
marklock has quit [Ping timeout: 246 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has quit [Ping timeout: 245 seconds]
Daboloskov has joined #ipfs
Daboloskov has quit [Client Quit]
pinbot has quit [Ping timeout: 256 seconds]
Sonarpulse has quit [Remote host closed the connection]
sindresorhus has quit [Ping timeout: 256 seconds]
mafintosh has quit [Read error: Connection reset by peer]
jabroney has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
jabroney has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
ipfs name resolve doesnt expect a path though
<whyrusleeping>
it expects a peerID
<wking>
right, but I want it to take a path
<wking>
seems simpler than having a parallel 'ipfs name resove-a-path' command ;)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<wking>
anyhow, thanks for unsticking me on ParsePath. I'll contemplate pathresolver.go and see if this approach makes sense ;)
<jbenet>
wking whyrusleeping: treat "ipfs name" as "ipns" (i actually plan to make a standalone `ipns` tool that's exactly the same as `ipfs name`.
<wking>
jbenet: why require the hard separation?
<wking>
And if you're sure about the separation, what would you call commands that are IP*S?
<wking>
I don't see a point to having an IPNS name resolver if you already have a generic IP*S path resolver.
<jbenet>
wking whyrusleeping: so `ipfs name publish [<ipns-name>] <ipfs-or-ipns-path>` -- the `<ipns-name>` means either: <ipns-hash> or /ipns/<ipns-hash>
patcon has quit [Ping timeout: 256 seconds]
<wking>
that <ipns-name> is not what I'm talking about. It's really <key-hash>, which is fine with me
<jbenet>
wking whyrusleeping: the `<ipfs-or-ipns-path>` means (as whyrusleeping mentioned above): either /ipfs/<hash><path>, /ipns/<hash><path>, or <ipfs-hash><path> (today it assumes /ipfs to be consistent with rest of tools. we could give this up and force user to specify a full path).
<wking>
All sounds good^
<jbenet>
wking: "I don't see a point to having an IPNS name resolver if you already have a generic IP*S path resolver." because ipns has larger scope than ipfs.
<wking>
I'm looking at 'ipfs name resolve [<name>]'
<jbenet>
wking: many people exist that are actually much more interested in ipns than on ipfs.
<wking>
But IP*S encompasses both
<jbenet>
wking: modularit.
<jbenet>
modularity.
<wking>
I want it to be 'ipfs name resolve [<ipfs-or-ipns-path>]', with the usual semantics (assumes /ipfs where we're missing a prefix)
<jbenet>
wking: a full operating system encompasses a kernel and packages. but useful to separate the two sometimes.
ei-slackbot-ipfs has quit [Remote host closed the connection]
<jbenet>
wking: <ipfs-path> is not a name. it does not belong under `ipfs name`.
ei-slackbot-ipfs has joined #ipfs
<jbenet>
wking: specifically, a name here is a pki bound mutable pointer.
<wking>
so where does it belong?
<jbenet>
wking: `ipfs resolve <ipfs-or-ipns-path>` could exist.
<wking>
ok
<wking>
and 'ipfs name resolve <key-hash>' could just be an alias for 'ipfs resolve /ipns/<key-hash>'
<jbenet>
wking: many tools have clashes like this, where there's some very specific plumbing commands and some similarly named porcelain commands that do a bit more.
<wking>
so why bother defining the former?
<wking>
it's even the same number of characters ;)
<jbenet>
because manipulating names at a low level is very useful.
<jbenet>
(names and only names)
<jbenet>
again, modularity.
<wking>
but all the name manipulation you can do with 'ipfs name resolve <key-hash>' you can do with 'ipfs resolve <ipfs-or-ipns-path>'
<whyrusleeping>
modularize all th things!
<wking>
I agree that modularity is good for the *implementation*
<jbenet>
remember that other implementations will exist, and someone may want to only implement ipns in language X.
<wking>
but I don't think it's useful here at the interface level
<whyrusleeping>
having tools dedicated to a given task is nice
<jbenet>
they shouldnt need to write a full `ipfs resolve` tool.
<whyrusleeping>
git has it all over the place
<whyrusleeping>
you can do things a number of different ways at different levels of the git tool
<wking>
so just write your 'ipfs resolve' and error out if you don't get an /ipns/ prefix?
<jbenet>
wking. small things, that do one thing well.
<wking>
which is one reason folks have trouble learning the Git interface ;)
<jbenet>
wking: again, they dont want "ipfs resolve" say they dont care at all about "ipfs"
<jbenet>
wking: yeah the git cli interface (used to) have a very steep learning curve, but it is way more expressive and hackable than any other vcs. more
<wking>
are you saying just an 'ipns' tool that holds stuff from 'ipfs name' but doesn't support the 'ipfs' syntax for those commands?
<wking>
My suggestion doesn't restrict the expressivity or hackability ;)
<jbenet>
wking: the git plumbing makes up in spades for the original authors' lack of time in making pretty ramps for new users. the pretty ramps were made eventually anyway, so it's all good now.
<wking>
But it does decrease the interface size
<jbenet>
no, it actually increases the complexity of the interface. instead of two small simple things, you have one more complex thing.
<jbenet>
it's harder to reason about from a systems perspective.
<wking>
No, instead of one marginally-complex thing and one simple thing, you only have one marginally-complex thing ;)
<wking>
or are you suggeting 'ipfs resolve' only take IPFS paths?
<wking>
anyhow, I'll think about it and post to 1082 ;)
<jbenet>
this is a matter of abstraction, right? think about it like separating code into functions
<jbenet>
conceptual abstraction helps understand things by breaking it into distinct pieces that work together.
mildred has quit [Quit: Leaving.]
Daboloskov has joined #ipfs
<jbenet>
though the overall "size" of the system may get larger, abstractions often make it easier to understand the multiple pieces. modularity hangs on to that, and makes it much more powerful because the abstractions become pieces of code and protocol and tools that anyone can use and implement in portions-- not needing to think about or even understand the
<jbenet>
whole.
Daboloskov has quit [Client Quit]
<jbenet>
wking: i wish i could communicate all this better. the unix modularity philosophy is hyper useful to design all these protocols and tools.
<jbenet>
someday i hope to write or give a talk on it-- cause it's easy to miss. i think it was even lost upon the go team :( :( :(. i think node/npm/iojs gets the unix ideals much better than the go teamm even though some of those guys are direct disciples of the unix team. (ken even worked on go)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
lgierth has joined #ipfs
feross_ has joined #ipfs
anshukla_ has joined #ipfs
okket_ has joined #ipfs
zrl_ has joined #ipfs
atrapado has quit [Quit: Leaving]
whyrusleeping1 has joined #ipfs
feross has quit [Ping timeout: 252 seconds]
okket has quit [Max SendQ exceeded]
whyrusleeping has quit [Ping timeout: 252 seconds]
vonzipper has quit [Ping timeout: 252 seconds]
zrl has quit [Ping timeout: 252 seconds]
anshukla has quit [Ping timeout: 252 seconds]
feross_ is now known as feross
vonzipper has joined #ipfs
<mafintosh>
jbenet: whats your take on multiplexing pbs streams?
<substack>
jbenet: only some parts of the node ecosystem are like that
<substack>
a rather small part, but it's big enough that you mostly don't need to care that there are other parts when you're building
zrl_ is now known as zrl
<jbenet>
mafintosh: i made the protocol-stream definition (upgrading with the header, encapsulation, etc). i need to push it up
<jbenet>
(it's just a def atm, not impl)
<whyrusleeping1>
impls are easy
<mafintosh>
jbenet: does that mention multiplexing?
<jbenet>
substack: yeah, fair, but i think the package manager help a lot because it's helping create this sort of culture, where in other cultures the tools get in the way.
<jbenet>
mafintosh: yeah, has an example of a protocol to mux multiple streams in one.
<jbenet>
mafintosh (basically multiplex)
<mafintosh>
jbenet: i have an idea but i'm not sure its good
<mafintosh>
jbenet: i realized you can basically multiplex to applications that both use pbs by just "merging" the schemas
<mafintosh>
*two
<mafintosh>
jbenet: assuming there is no naming clashes
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has quit [Read error: Connection reset by peer]
Wallacoloo has joined #ipfs
<jbenet>
mafintosh: yeah like just embedding the messages in either without extra wrapping, right? it's dangerous territory, but you can pretty much get away with it if you control all the protocols. (i probably wouldn't do it cause I don't, i'd wrap for safety)
<jbenet>
i think the wrapping you have with multiplex is minimal anyway
<mafintosh>
jbenet: yea. thats what i meant by "not sure if its good"
Daboloskov has joined #ipfs
therealplato1 has quit [Ping timeout: 256 seconds]
lgierth has quit [Ping timeout: 246 seconds]
lgierth has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping1>
jbenet: im 90% certain on that bitswap theory
<jbenet>
whyrusleeping1 yeah it makes sense to me
<whyrusleeping1>
why am i whyrusleeping1?
therealplato has joined #ipfs
whyrusleeping1 is now known as whyrusleeping
<jbenet>
whyrusleeping we're starting the cloning program.
<whyrusleeping>
you never informed me that my dna was sampled
<whyrusleeping>
i did not consent to this
<whyrusleeping>
>.>
<jbenet>
whyrusleeping: bitswap should try to be impervious to those types of failures-- i.e. sucks that its workers could be stuck doing anything other than sending data out.
<jbenet>
whyrusleeping: no dna. it's all virtual.
<whyrusleeping>
o.o
<jbenet>
whyrusleeping: but not sure how to achieve that, given the network layer obscures a lot of this from the clients.
<whyrusleeping>
yeah...
<whyrusleeping>
my idea
<jbenet>
whyrusleeping: what type of interface would help here? can check if there is a connection before sending
<whyrusleeping>
is to have the latency tracker on the peers also keep track of 'last interaction'
<whyrusleeping>
and we can use that to grab peers that are likely to be active
<jbenet>
yeah, thats how i want to do the timeouts too
<jbenet>
so works well.
<jbenet>
but its probably the case that right now the bug we're seeing isnt to peers "we think" are connected, but to peers we've found that aren't connected anymore
<whyrusleeping>
yeah
<jbenet>
though not sure. maybe it is dead connections
<whyrusleeping>
thats likely it
<jbenet>
"anymore" or "yet", too.
<jbenet>
"yet" = likely to be new providers we found
<jbenet>
"anymore" = old connections that've died silently
<jbenet>
whyrusleeping: go sets tcp keepalives, right?
<whyrusleeping>
apparently pinbot doesnt handle netsplits very well
<whyrusleeping>
ill have to work on that
rschulman has joined #ipfs
<whyrusleeping>
rschulman: heyo!
<whyrusleeping>
hows life at 10^100?
<G-Ray>
whyrusleeping: so that is amazing. It is then possible to replace Tahoe-lafs with IPFS ?
<whyrusleeping>
G-Ray: tahoe lafs has a different purpose than ipfs
<rschulman>
whyrusleeping: hey hey
<whyrusleeping>
we've actually talked in depth with them and i think that our systems are complimentary in a few good ways
<okket_>
actually, a googol is 10^100^100 ;)
<whyrusleeping>
okket_: no, thats a googolplex
<okket_>
right
<whyrusleeping>
lol, the wikipedia article for googol, "not to be confused with google"
<G-Ray>
whyrusleeping: maybe, but IPFS could give a better bandwidth than tahoe-lafs IIUC. (on a testnet network)
notduncansmith has joined #ipfs
<whyrusleeping>
G-Ray: yeah, but ipfs isnt about storing the data in a redundant manner
<whyrusleeping>
you could have ipfs/tahoe crossover nodes
notduncansmith has quit [Read error: Connection reset by peer]
<domanic>
jbenet, whyrusleeping hey! in the interest of learning more crypto I have been investigating how various secure channel protocols work and summarizing their design (and how it gives the security properties)
<domanic>
If you don't have a description of the ipfs secure channel i would like to contribute one - can you point me to where/how you want it?
<whyrusleeping>
domanic: that would be super cool
<whyrusleeping>
ours is in p2p/crypto/secio
<whyrusleeping>
(the code at least)
<G-Ray>
tahoe seems rather slow. If we can pin contents with IPFS, we could store content in a redoundant manner right ?
<whyrusleeping>
yeap
<whyrusleeping>
tahoe could easily be setup as a backend to ipfs
<whyrusleeping>
where multiple ipfs nodes in different locations share the same backing tahoe 'drive'
<domanic>
whyrusleeping, reading the code I see a number of comments that are just wrong
<whyrusleeping>
G-Ray: because links do not change
<whyrusleeping>
a link to given content will always link to that content
<whyrusleeping>
so long as someone cares enough about an object to keep it, it will be accessible easily
<domanic>
whyrusleeping, okay I'll send a pull request to fix the comments
<whyrusleeping>
domanic: awesome, thanks!
<whyrusleeping>
and any other writeup/docs you contribute would be awesome
<whyrusleeping>
if we get it documented nicely then its easier to audit
<domanic>
whyrusleeping, another question, I notice that you use a multihash to decide the cipher suite selection order
<G-Ray>
I don't see the advantage to set up tahoe behind IPFS rather than using IPFS and pin contents between some nodes
<domanic>
and that the cipher suite negioates a hash... is that an issue? you can never change that hash, otherwise the peers probably might not agree on the suite
<whyrusleeping>
domanic: that is true
<whyrusleeping>
hrm
<whyrusleeping>
i guess i could think of attacks that would benefit from controlling selection order
<whyrusleeping>
although, its not a huge advantage
<domanic>
no, it's a small thing, but it is a thing
<whyrusleeping>
thats a good thing to note
<domanic>
and a good primitive doesn't have leaks
<domanic>
the intention here seems to make cipher selection order fair
<domanic>
but i think this could be improved
<domanic>
the calculation is cmp(hash(alice_key + bob_nonce), hash(bob_key+alice_nonce))
<domanic>
but if alice expects to be talking to bob (pretty likely, if alice initiates the connection)
<whyrusleeping>
jbenet: a good interface for the dialing issue is to allow NewStream to take a boolean to not dial and fail if there are no open connections
<domanic>
then she can mine for a nonce where hash(bob_key+nonce) is a very high value
<whyrusleeping>
hrm...
<domanic>
what would be fair though: cmp(hash(alice_nonce + bob_nonce), hash(bob_nonce + alice_nonce))
<whyrusleeping>
so both sides could potentially mine
<whyrusleeping>
thats a good point
<domanic>
I'm not sure, but if you could replace the hash at that point with an xor, then you wouldn't have to worry about upgrading the internal hash
<whyrusleeping>
because it honestly doesnt need to be a hash
<whyrusleeping>
it just needs to be something both sides agree on and know how to compute
<whyrusleeping>
reliably
<domanic>
whyrusleeping, what about take the nonce which is closest to xor(alice_nonce, bob_nonce) ?
<domanic>
(in edit distance?)
<whyrusleeping>
i am not a crypto expert, but that sounds good to me
<whyrusleeping>
i cant think of any way to abuse that