jbenet changed the topic of #ipfs to: IPFS - InterPlanetary File System - https://github.com/jbenet/ipfs -- channel logged at https://botbot.me/freenode/ipfs/ -- code of conduct at https://github.com/ipfs/community/blob/master/code-of-conduct.md -- sprint at https://github.com/ipfs/pm/issues/7
<lgierth> cat foo.json | json2ipfs | ipfs object put
martinBrown has joined #ipfs
Bioblazin has joined #ipfs
Bioblaze has quit [Ping timeout: 264 seconds]
<jbenet> lgierth yeah, and it sort of works today, it's just a bit clunky. the current merkledag format is a bit clunky to work with.
<jbenet> lgierth though maybe you're right, maybe there's a "mdag-lite" that is a simple json doc, that gets properly formatted on impot
<jbenet> import
<jbenet> kbala are you around/
<jbenet> ?
<kbala> yes!
<kbala> would you like to talk now?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet> yeah, read the irc backlog re the sync?
<kbala> got it
<jbenet> lgierth: does that answer the questions? (i'll take a look at issues shortly)
<lgierth> jbenet: i might a have another question or two once i get to it, but that definitely helped
livegnik has joined #ipfs
<jbenet> lgierth: ok sounds good.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<kbala> jbenet: should i make a hangout?
hellertime has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet> anyone want to talk about ipfs datastructures? (i'm ok to skip it, but happy talk about things)
<jbenet> this is things like "unixfs" "trickledag" "rabin fingerprinting" "merkledag format", etc.
<lgierth> i'm about to go to bed, but i had an idea for monitoring and metrics. ipfs add them, and let anyone interested fetch them. that would be timeseries in ipfs or so
<lgierth> although it's probably a bad idea to ship metrics over the thing that we're trying to monitor
<lgierth> could be a separate daemon and network of course
<jbenet> lgierth yeah that's great
<jbenet> lgierth it may be ok to do it after the fact, or to do it on another daemon, yeah.
<jbenet> lgierth i've been wanting to play with little tools that make ephemeral single-purpose nodes
<jbenet> lgierth so instead of using the local daemon, they create a node inside without a local fs repo (all in memory) and just run a set of simple tasks
<jbenet> lgierth could do `curl` or `wget` equivalents this way, and if ipfs is working the right way, it should find locally running daemons if the content is cached locally.
<lgierth> yes!
<jbenet> it would also put all our protocols to the test-- optimizing these use cases will get ipfs working really well for nodes in embedded devices that mostly delegate the hard parts (routing, etc) to other nodes.
<lgierth> the community network people will love it
<lgierth> because on of the big problems is bandwidth :)
<lgierth> so you want to have caches in the neighbourhood
<jbenet> yeah exactly
<lgierth> bright times ahead
<jbenet> i've been thinking about a creative use of network coding
<jbenet> where we store a bunch of encrypted data in the network, and only the user knows the linear combinations that reveal their data
<jbenet> i guess this is just sss (shamir secret sharing)
<lgierth> would that be any different from usual symetric crypto?
ianopolous has quit [Ping timeout: 246 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet> yeah, data is split into chunks, which are then combined using network coding algorithms into single blocks, so store the data with redundancy and only user knows how to extract their piece (fountain, raptor codes, etc).
<jbenet> lgierth: want to add the infra issue links to https://etherpad.mozilla.org/Fo3cmIE6jk ?
<lgierth> ah got it
<lgierth> jbenet: already added a few
<lgierth> lgierth: or do you mean add to the backlog?
<jbenet> lgierth: ah cool. no you're good
* lgierth out
<lgierth> good night
<jbenet> o/ night!
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
meadman has quit [Ping timeout: 252 seconds]
rht__ has joined #ipfs
notduncansmith has joined #ipfs
meadman has joined #ipfs
notduncansmith has quit [Ping timeout: 258 seconds]
hellertime has quit [Quit: Leaving.]
<jbenet> hey rht__ are you around?
<rht__> yes here
<jbenet> cool! did you read backlog for today?
<jbenet> just wanted to sync-- how did last week's stuff go, and what's next for this week
<rht__> last week: with the get progressbar, either unixfs or tar size doesn't match the reader data transport when compressed flag is enabled.
<rht__> s/data transport/pipe/
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet> hm, may be useful to make a little program that outputs all the relevant sizes for a given root, so we can see what's not matching up?
<rht__> it only doesn't match when compressed flag is enabled
<rht__> I figure I could just add size for each format (tar, tar.gz), but there could be other compressed flag like bz2, etc.
<jbenet> rht__ right-- not sure that the best thing to do is. for compressed could cheat and just base it on the total (uncompressed).
<jbenet> rht__ though that's not great.
<rht__> and tar size can be precomputed, but tar.gz stream can't
<jbenet> rht__ yeah. not sure what the best thing to do there is-- maybe just adjust the total size once it's known.
<jbenet> rht__ a progress bar bug, by the way, is that it doesnt get cleared after exit-- sometimes the process exits and the progressbar stays in the terminal. should probably clear (`\b` or something) the line before exiting.
<rht__> oh right, will fix that
anshukla has joined #ipfs
<rht__> also the option to put the pb earlier in the pipe doesn't work. there is already fetchgraph function to get all child of a node recursively (instead of getdag). still I don't know how much exposing getblock transfer to dagservice buffer will affect memory usage / perf.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
anshukla has quit [Remote host closed the connection]
anshukla has joined #ipfs
inconshr_ has quit [Ping timeout: 265 seconds]
<rht__> unless there is other use for exposing the block transfer pipe. looks like for now, only show get pb for tar output
<rht__> (tar or extracted tar)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
jbenet has quit [Remote host closed the connection]
Luzifer has quit [Remote host closed the connection]
mappum has quit [Remote host closed the connection]
vonzipper has quit [Remote host closed the connection]
lohkey has quit [Remote host closed the connection]
insanity54 has quit [Remote host closed the connection]
richardlitt has quit [Remote host closed the connection]
henriquev has quit [Remote host closed the connection]
kumavis has quit [Write error: Broken pipe]
mquandalle has quit [Remote host closed the connection]
hosh has quit [Read error: Connection reset by peer]
kbala has quit [Remote host closed the connection]
daviddias has quit [Remote host closed the connection]
mikolalysenko has quit [Remote host closed the connection]
Muis has quit [Write error: Broken pipe]
bret has quit [Remote host closed the connection]
otherchas_ has quit [Remote host closed the connection]
kyledrake has quit [Remote host closed the connection]
jonl has quit [Remote host closed the connection]
zrl has quit [Write error: Broken pipe]
ehd has quit [Quit: X_X]
anshukla has quit [Remote host closed the connection]
anshukla has joined #ipfs
tso has quit [Ping timeout: 244 seconds]
anshukla has quit [Ping timeout: 264 seconds]
anshukla has joined #ipfs
zabirauf has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
anshukla has quit [Remote host closed the connection]
anshukla has joined #ipfs
anshukla has quit [Read error: Connection reset by peer]
anshukla has joined #ipfs
<Tv`> whyrusleeping: is there a simpler test than TestCorrectPinning for what the dagmodifer does? it's quite noisy to debug
<Tv`> and it's not even deterministic so i can't print keys from one round and look for them later :(
zabirauf has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Tv`> okay wtf is there a blockstore and a blockservice and how many layers of abstraction can you pile on top of this problem, each hiding error details
anshukla has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot> [go-ipfs] tv42 force-pushed pin from 397fbcf to 2a453f2: http://git.io/vIRMq
<ipfsbot> go-ipfs/pin c71a6c9 Tommi Virtanen: pin: Guard against callers causing refcount underflow...
<ipfsbot> go-ipfs/pin 64d01af Tommi Virtanen: sharness: Use sed in a cross-platform safe way...
<ipfsbot> go-ipfs/pin 7b71cfc Tommi Virtanen: sharness: `fusermount -u` is the documented way to unmount FUSE on Linux
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
anshukla has joined #ipfs
anshukla has quit [Remote host closed the connection]
sharky has quit [Ping timeout: 245 seconds]
sharky has joined #ipfs
anshukla has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
<ipfsbot> [go-ipfs] wking created tk/ls-trailing-tabs (+1 new commit): http://git.io/vIu1k
<ipfsbot> go-ipfs/tk/ls-trailing-tabs 3b244d2 W. Trevor King: core/commands/ls: Remove trailing tabs...
<ipfsbot> [go-ipfs] wking opened pull request #1346: core/commands/ls: Remove trailing tabs (master...tk/ls-trailing-tabs) http://git.io/vIu1Z
<ipfsbot> [go-ipfs] wking force-pushed tk/ls-trailing-tabs from 3b244d2 to 97119da: http://git.io/vIu1c
<ipfsbot> go-ipfs/tk/ls-trailing-tabs 97119da W. Trevor King: core/commands/ls: Remove trailing tabs...
<ipfsbot> [go-ipfs] wking force-pushed tk/ls-trailing-tabs from 97119da to 2ec0ba4: http://git.io/vIu1c
<ipfsbot> go-ipfs/tk/ls-trailing-tabs 2ec0ba4 W. Trevor King: core/commands/ls: Remove trailing tabs...
elima has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has quit [Remote host closed the connection]
ehd has joined #ipfs
hosh has joined #ipfs
jbenet has joined #ipfs
insanity54 has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
brab has joined #ipfs
mildred has joined #ipfs
bret has joined #ipfs
marklock has joined #ipfs
kumavis has joined #ipfs
anshukla has quit [Remote host closed the connection]
zrl has joined #ipfs
mafintosh has quit [Ping timeout: 265 seconds]
lohkey has joined #ipfs
Tv` has quit [Ping timeout: 255 seconds]
robmyers has quit [Ping timeout: 252 seconds]
niran has quit [Ping timeout: 256 seconds]
tibor has quit [Ping timeout: 276 seconds]
bigbluehat has quit [Ping timeout: 265 seconds]
oleavr has quit [Ping timeout: 265 seconds]
sindresorhus has quit [Ping timeout: 265 seconds]
prosodyContext has quit [Ping timeout: 256 seconds]
ffmad has quit [Ping timeout: 276 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Muis has joined #ipfs
warner has joined #ipfs
zootella has quit [Remote host closed the connection]
otherchas_ has joined #ipfs
mikolalysenko has joined #ipfs
henriquev has joined #ipfs
mquandalle has joined #ipfs
mappum has joined #ipfs
rht__ has quit [Remote host closed the connection]
feross has quit [Remote host closed the connection]
ogd has quit [Remote host closed the connection]
rubiojr has quit [Remote host closed the connection]
ladekjaer has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshr_ has joined #ipfs
inconshreveable has quit [Ping timeout: 265 seconds]
bigbluehat has joined #ipfs
jonl has joined #ipfs
rubiojr has joined #ipfs
niran has joined #ipfs
gunn has joined #ipfs
gunn has quit [Client Quit]
gunn has joined #ipfs
sindresorhus has joined #ipfs
gunn has quit [Client Quit]
zabirauf has joined #ipfs
richardlitt has joined #ipfs
oleavr has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Luzifer has joined #ipfs
ogd has joined #ipfs
vonzipper has joined #ipfs
<Luzifer> ah well. irccloud users are slowly recovering… :/
ladekjaer has joined #ipfs
mafintosh_ has joined #ipfs
feross has joined #ipfs
prosodyContext has joined #ipfs
tibor has joined #ipfs
<jbenet> yep
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
kbala has joined #ipfs
kyledrake has joined #ipfs
ffmad has joined #ipfs
daviddias has joined #ipfs
zabirauf has quit [Ping timeout: 250 seconds]
<cryptix> if time passing faster is a symptom of getting older, i need a way to stop this - right meow
<jbenet> cryptix: it only gets worse
Tv` has joined #ipfs
<cryptix> jbenet: i feared so... i feel like xmas was yesterday
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
rht__ has joined #ipfs
robmyers has joined #ipfs
rht___ has joined #ipfs
<Luzifer> wow. still not every irccloud user back… O_o
notduncansmith has joined #ipfs
brab` has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
brab` has quit [Read error: Connection reset by peer]
brab` has joined #ipfs
brab has quit [Ping timeout: 264 seconds]
Wallacoloo has quit [Quit: Leaving.]
williamcotton has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
rht___ has quit []
<kyledrake> Well, hopefully I did this right https://github.com/neocities/ruby-multihashes
<kyledrake> jbenet whyrusleeping FYI https://github.com/neocities/ruby-multihashes
<jbenet> kyledrake super cool. PR to https://github.com/jbenet/multihash ?
<kyledrake> on it
<kyledrake> Just changed the repo name to https://github.com/neocities/ruby-multihash
<jbenet>
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
nemik has quit [Read error: Connection reset by peer]
nemik has joined #ipfs
<kyledrake> Done.
pfraze has joined #ipfs
pfraze has quit [Ping timeout: 256 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<kyledrake> That library will likely change as I figure out the tooling. The base58 decode puts out an integer. Working with binary in ruby is.. not very fun.
<ipfsbot> [go-ipfs] rht opened pull request #1347: Make sure to stop progressbar on exit (master...master) http://git.io/vIzrY
hellertime has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
kbala has quit [Quit: Connection closed for inactivity]
zabirauf_ has joined #ipfs
zabirauf_ has quit [Ping timeout: 265 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
brab` has quit [Ping timeout: 244 seconds]
Tv` has quit [Quit: Connection closed for inactivity]
zabirauf_ has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<kyledrake> I think I'll be making a lot of ruby gems this week https://github.com/lian/bitcoin-ruby/blob/master/lib/bitcoin.rb#L123
JasonWoof has quit [Ping timeout: 246 seconds]
inconshr_ has quit [Remote host closed the connection]
JasonWoof has joined #ipfs
<jbenet> kyledrake: is that one wrong?
notduncansmith has joined #ipfs
<jbenet> ohhh it's not a ge
<jbenet> gem
notduncansmith has quit [Read error: Connection reset by peer]
marklock has quit [Ping timeout: 246 seconds]
zabirauf_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mafintosh_ is now known as mafintosh
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has quit [Ping timeout: 256 seconds]
tso has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
brab has joined #ipfs
daedalus316 has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
flugsio has quit [Quit: ctrl-l]
flugsio has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has joined #ipfs
williamcotton has quit [Ping timeout: 265 seconds]
cjb has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot> [go-ipfs] rht closed pull request #1347: Make sure to stop progressbar on exit (master...master) http://git.io/vIzrY
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot> [go-ipfs] wking force-pushed tk/ls-trailing-tabs from 2ec0ba4 to 15135e1: http://git.io/vIu1c
<ipfsbot> go-ipfs/tk/ls-trailing-tabs 15135e1 W. Trevor King: core/commands/ls: Remove trailing tabs...
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
therealplato has quit [Ping timeout: 258 seconds]
williamcotton has joined #ipfs
Blame has joined #ipfs
williamcotton has quit [Ping timeout: 244 seconds]
therealplato has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pjz has joined #ipfs
nessence has joined #ipfs
nessence has quit [Remote host closed the connection]
inconshreveable has joined #ipfs
nessence has joined #ipfs
nessence has quit [Remote host closed the connection]
nessence has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has joined #ipfs
domanic has quit [Ping timeout: 265 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has joined #ipfs
brab` has joined #ipfs
brab has quit [Ping timeout: 272 seconds]
brab` has quit [Ping timeout: 258 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has quit [Read error: Connection reset by peer]
williamcotton has joined #ipfs
cjb82 has joined #ipfs
inconshreveable has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mildred has quit [Quit: Leaving.]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> https://www.youtube.com/watch?v=4XpnKHJAok8 <- ive probably watched this 10 times...
nessence has quit [Remote host closed the connection]
williamcotton has quit [Ping timeout: 256 seconds]
daedalus316 has quit [Ping timeout: 246 seconds]
<daviddias> Oh, I remember watching that for the first time :)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has joined #ipfs
Blame has quit [Quit: Connection closed for inactivity]
<rmk0> jbenet: have gone with the dumb option for now: still have the central server which controls which package groups users can publish into, but now it only stores signatures and URIs, so the files themselves can be placed in ipfs
<rmk0> i'll look into decentralizing more when ipfs has matured a bit!
<lgierth> From: info@go.digitalocean.com Subject: Introducing Team Accounts
<lgierth> jbenet: whyrusleeping ^
atrapado has joined #ipfs
Tv` has joined #ipfs
nessence has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
grawity has joined #ipfs
crest has quit [Remote host closed the connection]
crest has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
nessence has quit [Remote host closed the connection]
elima has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
elima has joined #ipfs
cjdmax has joined #ipfs
Encrypt has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> lgierth: ooOoo, neat
Encrypt has quit [Quit: Quitte]
nessence has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mildred has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
chriscool has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
elima has quit [Ping timeout: 256 seconds]
warner has quit [Quit: ERC (IRC client for Emacs 24.5.1)]
<sprintbot> Sprint Checkin! [whyrusleeping jbenet cryptix wking lgierth krl kbala_ rht__]
<whyrusleeping> sprint checkin: working on an ipfs packaging tool, its going pretty well
<whyrusleeping> also checked out a place this morning that may serve as a good seattle office
williamcotton has quit [Ping timeout: 250 seconds]
<ogd> would you call it a.... seattleite office?
* ogd sees himself out
<headbite> what's the function of the offices?
<headbite> is it something like paid support?
elima has joined #ipfs
<whyrusleeping> ogd: so punny
<whyrusleeping> headbite: just a place for the devs to work together
<headbite> yeah.. get back to work. :)
<wking> checkin: Some progress on UnixFS-friendly command interfaces for the Docker-registry driver (e.g. 'ipfs unixfs ls ...' that returns the UnixFS size.)
Encrypt has joined #ipfs
<whyrusleeping> wking: sweet
<whyrusleeping> i pushed a few more commands to ipfs-shell, namely Get AddDir and NewObject
<wking> ah, good
<jbenet> checkin: making an fs stress test thing, to create thousands of files and add them
williamcotton has joined #ipfs
<jbenet> (want to make sure everything's good)
Blame has joined #ipfs
<wking> Thinking over the API levels, I think we want 'ipfs object ...' for Merkle-DAG level access, 'ipfs <something> ...' for generic fanout-aware access, and 'ipfs unixfs ...' for Unix-filesystem access
<wking> But "generic fanout-aware" is currently stuck (at least for me) on https://github.com/ipfs/specs/issues/10
<whyrusleeping> wking: i dont think we need a generic fanout thing
<wking> And this all ties up with <ipfs-path> being interpreted as a fanout-aware path (e.g. you can't use an IPFS path for a fanout directory node besides /ipfs/<hash-for-the-fanout-node>)
www has quit [Ping timeout: 250 seconds]
<wking> whyrusleeping: The Docker-registry driver needs to be fanout-aware (for performance), but it doesn't need to be a Unix filesystem
<wking> I imagine there will be other objects in this category of non-Unix-FS things where fanout is useful
notduncansmith has joined #ipfs
<jbenet> yep, fanout is useful. not sure that the best way to represent this is
notduncansmith has quit [Read error: Connection reset by peer]
<wking> And I think a generic distinction between "I've fanned out the data via chunking, and here's how to reassemble", "I've fanned out the link array via, and here's how to lookup a given link by name", and other situtations like that would be useful
<wking> * link array via <scheme>, and ...
inconshreveable has joined #ipfs
<wking> jbenet: yeah, https://github.com/ipfs/specs/issues/10 has my head-scratching to date ;)
www has joined #ipfs
<lgierth> we already have a berlin office btw :P
<whyrusleeping> lol, ipfs-berlin?
www1 has joined #ipfs
<lgierth> yeah i have a desk in a shared office
<lgierth> :)
<lgierth> i don't know where krl works
<lgierth> anyhow, i can organize a sign on the bell and postbox
<jbenet> lgierth: awesome!
<jbenet> krl o/
www has quit [Ping timeout: 264 seconds]
<lgierth> i now have tags in thunderbird for assigned, mentioned, closed, merged. order to my github notifications!
www1 has quit [Quit: Leaving.]
www has joined #ipfs
<jbenet> yeah, that really helps
<jbenet> or, it can.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
kbala has joined #ipfs
<nell> file:///home/avi/Desktop/earth3.webm https://u.teknik.io/jAlie5.webm
<nell> couple of google earth videos i took with janus
* kyledrake is about 3 more hex and/or binary and/or integers away from building a Buffer for ruby
<nell> those little programmer joys of things coming together and working
<nell> often celebrated alone... the world unknowingly aware of the implications you've created through code
<nell> (´•ω•`)
<jbenet> nell: oh man. that's awesome
<whyrusleeping> nell: that looks pretty sweet :)
<whyrusleeping> i want one
<nell> i like how its running in linux
notduncansmith has joined #ipfs
<whyrusleeping> thats awesome :)
cow_2001 has quit [Quit: ASCII Muhammad - @o<-<]
domanic has joined #ipfs
nessence has quit [Read error: Connection reset by peer]
nessence has joined #ipfs
<whyrusleeping> jbenet: question on multistream, pretend i'm the server, someone connects to me and sends me a different multistream version than i'm running
<whyrusleeping> what do i do?
anshukla has joined #ipfs
<kyledrake> Are base58 and base58check the same thing?
mafintosh has quit [Ping timeout: 256 seconds]
ladekjaer has quit [Ping timeout: 256 seconds]
mikolalysenko has quit [Ping timeout: 256 seconds]
ladekjaer has joined #ipfs
<kyledrake> My multihash works right, then I put it into Bitcoin.encode_base58 and get a different encoding than I'm expecting.
mikolalysenko has joined #ipfs
mafintosh has joined #ipfs
<kyledrake> I believe I have been working under the mistaken impression that IPFS has been using SHA256 for the current hashes
<kyledrake> Wait, yes it is. Gah. Why is my sha256 hasher arriving at a different conclusion
nessence has quit [Read error: Connection reset by peer]
nessence has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
cjdmax has quit [Remote host closed the connection]
nessence_ has joined #ipfs
nessence_ has quit [Client Quit]
<kyledrake> Okay, the light just turned on, I get it now. The hash IPFS provides includes the PBNode data structure the data is housed in. Sorry for the channel noise. :)
<jbenet> kyledrake no worries about noise
<jbenet> and no, base58check is not the same thing exactly
<jbenet> it uses a few bits at the end to verify the hash (like a small error checksum)
<kyledrake> jbenet I saw you were using base58-native which I know was made for check (I worked briefly with it's author). IPFS is using check then?
<jbenet> kyledrake: no, we're using raw base58
<kyledrake> ok.
<jbenet> (we're using that module, but it's not check)
<jbenet> i may be wrong, but i thik this is the case.
nessence has quit [Ping timeout: 265 seconds]
<_fil_> what is this? API?
<kyledrake> ah, it provides both. got it.
<kyledrake> thx
Encrypt has quit [Quit: Quitte]
rht__ has quit [Quit: Connection closed for inactivity]
williamcotton has quit [Ping timeout: 256 seconds]
<whyrusleeping> jbenet: lets change all our hashes to base1024 using utf8
<whyrusleeping> base58 is so outmoded
<jbenet> haha right?
<jbenet> (on the phone) "ok, look, just go to /ipfs/åå´œ©œ´∫©∑†®´©åߘ∫©©ßœœ´√´®/my-site"
atrapado is now known as atrapa
* _fil_ can't add go-ipfs to ipfs
<whyrusleeping> i cant see any problems with that
<_fil_> there's a dead symlink apparently
<_fil_> Error: open Godeps/_workspace/src/github.com/jbenet/go-multihash/test/sharness/bin: no such file or directory
<whyrusleeping> _fil_: we fail on broken symlinks :/
<_fil_> ok
<_fil_> kinda funny not to be able to ipfs add -r .
<_fil_> just after make install
<whyrusleeping> yeaahhh....
atrapa is now known as atrapado
<jbenet> whyrusleeping: we should fix that finally.
<jbenet> it's been plaguing us for a while.
<kyledrake> lol
<whyrusleeping> no. i never want to fix it. it will stay broken forever
<whyrusleeping> #dealwithit
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ianopolous has joined #ipfs
therealplato has quit [Ping timeout: 245 seconds]
www has quit [Ping timeout: 244 seconds]
therealplato has joined #ipfs
<kyledrake> ipfs patch is awesome. I'm going to give it a spin right now.
<whyrusleeping> kyledrake: :D
<_fil_> http://ipfs.io/blog/ <= needs some love
<ipfsbot> [go-ipfs] wking created tk/unixfs-ls (+2 new commits): http://git.io/vIwsE
<ipfsbot> go-ipfs/tk/unixfs-ls d0ce146 W. Trevor King: core/commands/unixfs: Add 'ipfs unixfs ls ...'...
<ipfsbot> go-ipfs/tk/unixfs-ls bdefbbb W. Trevor King: core/commands/unixfs/ls: Don't recurse into chunked files...
<Luzifer> oh there is a blog…
* Luzifer hmm I should fill my blog with words…
williamcotton has joined #ipfs
<whyrusleeping> Luzifer: help fill our blog with words :)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Luzifer> but my blog… I don't have the motivation to fill my own blog… xD
<headbite> everyone just post pet pictures to the blog for now
<Luzifer> cat :3
<Luzifer> oh… to the blog… whoops…
williamcotton has quit [Ping timeout: 255 seconds]
<_fil_> I had no time to look at ipfs for the last weeks, and don't know where to go for fresh news
<_fil_> I end up reading the commits on go-ipfs :)
<whyrusleeping> _fil_: thats not a *bad* place to get news
<whyrusleeping> lol
<Luzifer> I tried to read the pull-requests… Gave up because I got not enough time to cover that all xD
<whyrusleeping> we're due to push a new version bump and changelog
<Luzifer> changelog++ :)
<Blame> what is with the "Sec-WebSocket-Accept" sillyness in rfc6455?
<Luzifer> hmpf. reminder to self: discussions with some people are just a waste of time… (maybe I'll learn that one day…)
<Luzifer> s/one/some/
www has joined #ipfs
cow_2001 has joined #ipfs
<ipfsbot> [go-ipfs] wking opened pull request #1348: Add 'ipfs unixfs ls …' (master...tk/unixfs-ls) http://git.io/vIwWz
<jbenet> remember kids, exponentials grow fast. dont hose your harddrives.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
nell has quit [Read error: Connection reset by peer]
<whyrusleeping> jbenet: hows that ssd?
<whyrusleeping> jbenet: also: https://github.com/whyrusleeping/datagen
<jbenet> oh dammit
<jbenet> oh well mine's cooler.
<whyrusleeping> yeah, i didnt put much effort into it
williamcotton has joined #ipfs
mildred has quit [Quit: Leaving.]
williamcotton has quit [Ping timeout: 256 seconds]
<Blame> I am suddenly hating websockets.
<jbenet> kyledrake: im writing large tests
chriscool has quit [Ping timeout: 256 seconds]
<ianopolous> hi guys, is anyone aware of any instructions for running a "private" ipfs network. i.e. within a company? Obviously I'd need to remove all the public bootstrap nodes, and bootstrap with one of our own nodes. Any pointer to where to read would be appreciated.
<Blame> the simple solution is to replace the bootstrap nodes and run in in a VPN/LAN
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Blame> however, if a member of the main DHT attempts to peer with one of yours (or the other way around) the networks would pretty elegantly merge automagically.
<ianopolous> yep, that's what I'm trying to prevent :-)
<Blame> ipfs is not really meant for this use case (right now), the only thing I could think of that would not involve too much hacking is backlisting/whitelisting
<Blame> I'd make a rule at a firewall/nat level that would only allow connections on your whitelist to the IPFS ports
<ianopolous> we're behind a decent firewall in terms on incoming connections, but I just need to make sure there aren't any stray outbound connections to the real DHT
<Blame> inbound connections are the problem
<Blame> and if you are solidly NAT'ed it shoudl be fine
<ianopolous> ok cool
<Blame> hmm. Maybe not. the NAT would prevent "outside" nodes from peering with you, but not you from peering with outside nodes.
<ianopolous> essentially I hope to use it with fuse to give a shared folder across all office machines, where we can just dump stuff and have it sync
<ianopolous> but is the only source of outside nodes the bootstrap nodes list?
Wallacoloo has joined #ipfs
<Blame> It should be, you might need to worry about bitswap, but you are now wandering outside of "comfort zone" to advise you on.
<Blame> I konw alot about how DHTs work in theory, but I'm worried their implementation differs from theory...
<ianopolous> i guess I could easily set up a canary by storing a file locally and setting up a daily curl of the gateway asking for its hash :-)
<Blame> oh
<Blame> do you use ipv6 on you internal network?
<whyrusleeping> we are going to be able to have a dial blacklist soon
<whyrusleeping> you'll be able to say "don't dial addresses in this range"
<whyrusleeping> it will make it pretty easy to isolate nodes with a firewall
<ianopolous> cool
<ianopolous> our LAN is only ipv4
<Blame> it looks like most of the network only supports ipv4, so that would also be a sneaky way to prevent peering
<jbenet> Blame ianopolous: hang on-- it is very much meant for this use case. we have some tooling to build (firewall rules etc)
<jbenet> but you can very much do it by removing the bootstrap list + adding iptables rules today.
domanic has quit [Ping timeout: 276 seconds]
<jbenet> that said, right now ipfs doesnt have auth-- so anybody who manages to talk to the node + requests things will get them
<jbenet> we will be adding auth tooling and so on, but need people with use cases to help us spell out all the details / needs
<ianopolous> no auth is fine for our usecase for now :-)
<ianopolous> quite excited actually to have a legitimate use for IPFS at work :-)
<jbenet> Blame: the problem with saying "ipfs is not really meant for this use case (right now)" is that it's not correct, even with the temporal qualifier. it implies that we're not targetting that at all (though we may change our minds). "it is _meant_ for the use case (i.e. we are thinking about it and building tooling for it already) but it lacks tooling for
<jbenet> certain things right now, so YMMV" is much more accurate. i dont mean to be stickler, but messaging is really important. we dont want to incorrectly turn away people because of subtle misunderstandings.
inconshreveable has quit [Ping timeout: 265 seconds]
<jbenet> ianopolous yay.
<jbenet> ianopolous just bug us here to fix things
<jbenet> as you need them.
notduncansmith has joined #ipfs
<Blame> Sorry. I am bit bit behind on what y'all have been up to (honestly at least a month).
notduncansmith has quit [Read error: Connection reset by peer]
<Blame> I'll try to keep that in mind next time I try to sound like I know what I am doing.
<jbenet> whyrusleeping: go-ipfs needs a "ipfs init --testing" flag that inits with no bootstrap nodes, randomized ports, etc.
<ianopolous> awesome, shall do
<jbenet> Blame: np, thanks :)
<jbenet> thanks for help, too, much appreciated.
<jbenet> helping*
<whyrusleeping> jbenet: use iptb
<jbenet> whyrusleeping does iptb randomize nicely?
<jbenet> whyrusleeping: no actually, iptb clashes remember?
<jbenet> it's not yet working
<whyrusleeping> uhm.. it let's you pick ports
<whyrusleeping> randomizing ports is a hard problem
<whyrusleeping> what is your usecase?
<whyrusleeping> picking random ports is very racy. especially when you have to set up bootstrapping on multiple modes
www1 has joined #ipfs
<whyrusleeping> only good way is disabling bootstrapping, using port 0, and relying on multicast
www has quit [Ping timeout: 252 seconds]
<whyrusleeping> which isn't even a good way
<jbenet> i dont even want it to connect to anthing, just have a daemon open.
<whyrusleeping> a single daemon?
<whyrusleeping> iptb -n=1 -p=0 init
<whyrusleeping> jbenet: question on multistream, pretend i'm the server, someone connects to me and sends me a different multistream version than i'm running
<whyrusleeping> what happens?
<jbenet> for now, close it
<jbenet> we can think about semver
headbite has quit [Quit: Leaving.]
<whyrusleeping> okay
<wking> dPow: I'm poking at your multihash-to-short issue from go-ipfs#1350
<whyrusleeping> jbenet: alright, do you want to provide an official 'multistream' version string?
<jbenet> whyrusleeping: yeah we should. im not sure what specs to put there, whether the git repo itself, or an ipfs add -r of the files, or what yet.
<whyrusleeping> okay, just let me know
<kyledrake> jbenet awesome! That looks like it'll do.
atrapado has quit [Quit: Leaving]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet> random-files halts, not sure why
<whyrusleeping> want me to try and figure it out?
<whyrusleeping> repro steps?
<jbenet> oh it may be hashing, ipfs is pegging the cpu
<whyrusleeping> hahaha, 'it hung!'
<jbenet> whyrusleeping follow steps in https://github.com/jbenet/ipfs-stress-fs
<whyrusleeping> jbenet: i assume you'll want me to run the 1TB tests for you at some point
<jbenet> yeah at some point
<whyrusleeping> i just scrubbed my raid array last night
<whyrusleeping> 540MB/s read, 480MB/s write :D
<jbenet> lol
<whyrusleeping> 9TB
<jbenet> maybe this is a problem solved by concurrent add
<whyrusleeping> shes my baby.
<jbenet> it's crawling atm.
<jbenet> this must be what kyledrake was seeing
<whyrusleeping> how big of an add is it?
<jbenet> it's not even going over 100% cpu, so it's not using multiple cores.
<ipfsbot> [go-ipfs] wking created tk/resolve-fixes (+1 new commit): http://git.io/vIwyc
<ipfsbot> go-ipfs/tk/resolve-fixes 9db4b5e W. Trevor King: core/commands: Catch ErrResolveRecursion in non-recursive resolution...
<jbenet> ok it finished
<whyrusleeping> lol, good
<whyrusleeping> i only have 50GB free space on my laptop
<jbenet> only a measly 365MB, but on tons of little files.
<whyrusleeping> ah
<jbenet> 102641 files.
<whyrusleeping> are your tests reproducible?
<jbenet> yeah, we suck at small files.
<jbenet> whyrusleeping: yeah, you can pass in a seed.
<jbenet> it doesnt atm.
<whyrusleeping> does each test generate the same data across runs?
<whyrusleeping> okay
<ipfsbot> [go-ipfs] wking opened pull request #1351: core/commands: Catch ErrResolveRecursion in non-recursive resolution (master...tk/resolve-fixes) http://git.io/vIwSf
williamcotton has joined #ipfs
<jbenet> lol rm is pegging the cpu now, hah.
<jbenet> poor little macbook.
<whyrusleeping> lol, HFS+ is the worlds shittiest filesystem
<whyrusleeping> daviddias: i updated the go-multistream
<whyrusleeping> it does the protocol echo on connect now
<whyrusleeping> but it will disconnect if it doesnt get "/multistream/version1.0.0" (placeholder until jbenet gets us a real version string)
williamcotton has quit [Ping timeout: 264 seconds]
<kyledrake> jbenet whyrusleeping I didn't reperform, but when I tried to do an ipfs add -r all_sites I believe it did lock up on me
<kyledrake> There wasn't an obvious bottleneck I could find.
<whyrusleeping> kyledrake: interesting...
<kyledrake> I think I did start it on a screen, let me go check what happened there
* whyrusleeping is very curious
<kyledrake> It seems to behave better when you throw in smaller amounts
<kyledrake> so, a site with <1000 files, for example.
<kyledrake> per add
<whyrusleeping> youre running latest master?
<whyrusleeping> (or recent < 1week old)
<kyledrake> started 6/1
<whyrusleeping> jbenet: we need to release 0.3.5 and do a changelog
<kyledrake> What's the stack dump command
<whyrusleeping> ctrl + \
<kyledrake> I've got it hanging on a screen
<kyledrake> got it.
<kyledrake> one second
<whyrusleeping> this is the daemon, right?
<whyrusleeping> unless you ran it offline, the commands stack dump isnt super useful
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Evermore> Hey IPFS is taking like 200 MB of RAM on my VPS, and I only have 512 MB
<Evermore> So I might shut it down
<Evermore> I don't know if anyone else is seeding Homestar
<lgierth> yeah there's a memleak
<Evermore> :/
<lgierth> it's supposed to eat far less
<jbenet> Evermore we'll fix this asap. try rebooting it
<kyledrake> That may be the "online" version. If it is, let me know how I can run it "offline" and i'll re-run the import
<Evermore> Tor is using a lot too, I don't even remember having Tor on there
<whyrusleeping> kyledrake: make sure a daemon isnt running on that machine
<whyrusleeping> and run ipfs add -r like you normally would
<kyledrake> ok
<Evermore> jbenet: I'm gonna compile the latest version, I'm building a new desktop tonight that will have Go installed and everything
<jbenet> kyledrake: can also stack dump the daemon
<jbenet> Evermore nice.
<whyrusleeping> ^ that too
<jbenet> whyrusleeping: +1 on releasing 0.3.5,
<Evermore> I've been between computers for about a week, using an old nettop and I don't think either of my Debians have Go
<jbenet> shall we fix the memleak or no?
inconshreveable has joined #ipfs
<whyrusleeping> it's all about what you want me focused on
<kyledrake> I'm going to pull the latest ipfs and run it again directly
<wking> jbenet: can you pull go-ipfs#1351 into 0.3.5?
<kyledrake> IPFS is using 4GB of res memory on our production node, but I didn't think much of it just because I'm used to Bitcoin eating about that much memory. The machine has 15GB so it's not a problem, my assumption was that it was just using some extra memory for caching.
<jbenet> wking commented about the UX
<jbenet> kyledrake: 4GB is quite a bit. it's not super smart on caching yet-- we can do this easily on top (have some LRU cache in front of the fs sized from the config), we just have to do it.
<lgierth> i'd like that for the gateways too
<kyledrake> The unix FS cache is doing most of the caching, which is good.
<lgierth> oh i see what you mean, nevermind my comment
<kyledrake> We're at 95% with the FS cache thrown in
<kyledrake> whyrusleeping okay, I've restarted the import offline, on the latest from the zipball download from the web site. Let me know if that's not right.
<kyledrake> My assumption is that the zipball is a nightly from master, which may be an incorrect assumption.
<whyrusleeping> kyledrake: i beleive that assumption is correct
<whyrusleeping> fairly sure a build is triggered on every push to master
<kyledrake> gobuilder doesn't show the commit it bases from, but it said it built it a day ago
<kyledrake> from master
<whyrusleeping> sounds about right
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<wking> jbenet, whyrusleeping: 'ipfs files ...' and not 'ipfs file ...'? go-ipfs#1348 is a lot like 'ipfs object ...', and that's singular.
<whyrusleeping> oh, i meant file
<wking> maybe filesystem?
<whyrusleeping> my point was to only make the change to the command
<wking> fs?
<whyrusleeping> too much typing
<whyrusleeping> fs...
* whyrusleeping wouldnt complain
<wking> jbenet: Just drop 'unix' to be less scary? ;)
<whyrusleeping> scurreh
<jbenet> `ipfs fs` sounds dumb.
<jbenet> `ipfs file` or `ipfs files` is best i think.
<jbenet> (by "dumb" i mean: "why am i specifying "fs" twice? isn't "ipfs" already an "fs"? wat?"
<wking> we'll, maybe that means the unixfs commands should be root-level, and the generic Merkle-DAG stuff should be pushed down into a lower namespace?
<jbenet> no, unixfs is _one_ -- even though very common -- application.
* jbenet this kind of promotion is what gets us stuck deeper into the unix file swamp, and further away from better models of computation.
<wking> right. But it's part of the project name, so it can get some special treatment ;)
<wking> anyhow, 'ipfs file' it is, although I wish there was a more evocative term. I'm never sure where 'file' lands on the scale between "filesysytem", "everything is a file", "a single chunk of data with an associated inode" :p
<jbenet> lgierth: those metrics/graphs on the gateway would be super cool to get.
www1 has quit [Ping timeout: 265 seconds]
<jbenet> wking: i hear you. it's optimizing for the average user, for whom "file" means `a thing i can "cat <file>"`
<jbenet> (average user of the cli)
<jbenet> (not average user of ipfs in general)
<wking> it also needs to mean "a thing I can ls"
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> jbenet: i want a command like "ipfs diag cli"
<ianopolous> jbenet: I'd love to have a chat with you sometime about mapping Peergos's data structures onto IPNS. Let me know if you get a chance.
<whyrusleeping> that will run through all the commands and verify that their marshaling works properly
<lgierth> jbenet: yeah i'm actually not sure why i didn't add them to my list!
<lgierth> wanna add me to the org and i'll edit them in?
<ipfsbot> [go-ipfs] wking pushed 1 new commit to tk/unixfs-ls: http://git.io/vIrvt
<ipfsbot> go-ipfs/tk/unixfs-ls 7c91e3a W. Trevor King: core/commands/unixfs: Rename 'ipfs unixfs' to 'ipfs file'...
<jbenet> ianopolous: yeah sounds good. i think i owe you an email, sorry will get to it soon.
<jbenet> lgierth: yep yep! just request access anywhere where relevant
<ipfsbot> [go-ipfs] wking force-pushed tk/unixfs-ls from 7c91e3a to b5e4fac: http://git.io/vIrva
<ipfsbot> go-ipfs/tk/unixfs-ls b5e4fac W. Trevor King: core/commands/unixfs: Rename 'ipfs unixfs' to 'ipfs file'...
<lgierth> thnx i'll add that
williamcotton has joined #ipfs
<ianopolous> jbenet: no worries. I get that you're super busy.
<jbenet> gitcop should set an indicator in the PR.
<jbenet> (i.e. shouldnt be "green")
<lgierth> jbenet: can't edit sprint k, but somehow can edit your comment in that issue
<lgierth> jbenet: fixed
<lgierth> i don't know if that was github acting up or you being quick :)
<jbenet> moved fs tests to https://github.com/ipfs/fs-stress-test
<jbenet> lgierth \o/ who knows
<jbenet> i think me? not sure
<lgierth> :]
williamcotton has quit [Ping timeout: 246 seconds]
<jbenet> teams as ONLY acl for orgs is a bit broken
<jbenet> wish could invite individuals to repos inside an org.
pfraze has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]