jbenet changed the topic of #ipfs to: IPFS - InterPlanetary File System - https://github.com/jbenet/ipfs -- channel logged at https://botbot.me/freenode/ipfs/ -- code of conduct at https://github.com/ipfs/community/blob/master/code-of-conduct.md -- sprint at https://github.com/ipfs/pm/issues/7
lgierth_ has quit [Quit: Ex-Chat]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<zignig> is anyone else getting Error: can't Lock file "/root/.ipfs/repo.lock": has non-zero size , on the latest master ?
<zignig> command line commands started failing.
<cryptix> zignig: yup
<cryptix> but i think i was the first one
<zignig> wassup with that ?
<cryptix> no idea yet
<whyrusleeping> zignig: thats a bug, probably my fault
<whyrusleeping> ill patch
<zignig> whyrusleeping: ok, was about to add a issue , but if it's known.
<ipfsbot> [go-ipfs] whyrusleeping created fix/lock-check (+1 new commit): http://git.io/vkIwv
<ipfsbot> go-ipfs/fix/lock-check 270b477 Jeromy: catch other error that means the file is locked
<zignig> !botsnack
<whyrusleeping> try that out
<ipfsbot> [go-ipfs] whyrusleeping opened pull request #1295: catch other error that means the file is locked (master...fix/lock-check) http://git.io/vkIwu
<cryptix> yey!
<cryptix> whyrusleeping: works
<cryptix> well.. that was nasty though
<whyrusleeping> yeah. tell me about it
<whyrusleeping> they dont return a typed error
<zignig> confirmed ! , merge that bad boy into master.
<whyrusleeping> and the error returned contains a path
<whyrusleeping> so i cant full text match it
<cryptix> oh well..
<zignig> typed errors are good.
<cryptix> yea.. they would be.. gotta sleep, guys. cya
<Tv`> eww what
<Tv`> oh bad library
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Tv`> okay that camli lib you're using definitely stinks
<Tv`> looks like it's from before the time things were added to syscall, and never updated
<cryptix> kill it with fire
<whyrusleeping> Tv`: suggestions?
<Tv`> syscall.Flock?
<whyrusleeping> oh, well thats nice
<whyrusleeping> is that supported cross platform?
<Tv`> fuck windows
<whyrusleeping> lol
timgws has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
pfraze has joined #ipfs
guest756 has joined #ipfs
timgws has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshr_ has quit [Remote host closed the connection]
anshukla has quit [Remote host closed the connection]
<whyrusleeping> i love how well the go authors document code
anshukla has joined #ipfs
anshukla has quit [Remote host closed the connection]
guest756 has quit [Ping timeout: 265 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
headbite has quit [Remote host closed the connection]
<Tv`> syscalls are not really their responsibility
hellertime has quit [Quit: Leaving.]
pfraze has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
rht_ has quit [Quit: Connection closed for inactivity]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
timgws has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
headbite has joined #ipfs
timgws has joined #ipfs
<whyrusleeping> fair
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
headbite has quit [Read error: Connection reset by peer]
headbite has joined #ipfs
kyledrake has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
timgws_ has joined #ipfs
timgws_ has quit [Client Quit]
timgws_ has joined #ipfs
timgws has quit [Ping timeout: 245 seconds]
timgws_ has quit [Client Quit]
kyledrake has quit [Ping timeout: 252 seconds]
timgws has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze_ has joined #ipfs
inconshreveable has joined #ipfs
tilgovi has quit [Read error: Connection reset by peer]
tilgovi has joined #ipfs
guest756 has joined #ipfs
timgws has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has quit [Remote host closed the connection]
anshukla has joined #ipfs
timgws has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
timgws has joined #ipfs
timgws has quit [Client Quit]
rht_ has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
EricJ2190 has quit [Ping timeout: 272 seconds]
anshukla has quit [Remote host closed the connection]
anshukla has joined #ipfs
tilgovi has quit [Ping timeout: 246 seconds]
kyledrake has joined #ipfs
timgws has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> the peer ID of my mobile pi: QmYUoju4vrs3eTskwRGJzNEUvKKBWMyDggt5M8JZ5JsN5E
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
kyledrake has quit [Ping timeout: 252 seconds]
bren2010 has quit [Ping timeout: 255 seconds]
bren2010 has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
wallet42 has joined #ipfs
guest756 has quit [Ping timeout: 245 seconds]
pfraze_ has quit [Remote host closed the connection]
sharky has quit [Ping timeout: 258 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
sharky has joined #ipfs
pfraze_ has joined #ipfs
pfraze_ has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Tv`> good collection of gifs for your CI integration: http://imgur.com/a/YET5a
timgws has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
wallet42 has quit [Quit: Leaving.]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
timgws has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mildred has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
timgws has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
kyledrake has joined #ipfs
inconshreveable has joined #ipfs
inconshreveable has quit [Remote host closed the connection]
inconshreveable has joined #ipfs
cjdmax has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
urtz has joined #ipfs
urtz has quit [Quit: That's Miss t7DS to you   [www.t7ds.com.br]]
therealplato has quit [Ping timeout: 246 seconds]
flugsio has joined #ipfs
neoteo has quit [Quit: Lingo: www.lingoirc.com]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
wallet42 has joined #ipfs
<cryptix> gmorning
notduncansmith has joined #ipfs
wallet42 has quit [Quit: Leaving.]
notduncansmith has quit [Read error: Connection reset by peer]
<rht_> what kind of dac is peerchemist building? how is the cpu resource distributed?
chriscool has joined #ipfs
lgierth has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
chriscool has quit [Ping timeout: 264 seconds]
domanic has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
chriscool has joined #ipfs
cjdmax has quit [Remote host closed the connection]
cjdmax has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
wallet42 has joined #ipfs
wallet42 has quit [Client Quit]
mildred has quit [Ping timeout: 276 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mildred has joined #ipfs
octalberry has joined #ipfs
octalberry has quit [Client Quit]
domanic has quit [Ping timeout: 256 seconds]
octalberry has joined #ipfs
lgierth has quit [Quit: Ex-Chat]
atomotic has joined #ipfs
Tv` has quit [Quit: Connection closed for inactivity]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ekarlso has quit [Quit: WeeChat 0.4.3]
Wallacoloo has quit [Ping timeout: 265 seconds]
inconshreveable has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
infinity0 has quit [Remote host closed the connection]
wallet42 has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
octalberry has quit [Quit: Leaving]
octalberry has joined #ipfs
octalberry has quit [Client Quit]
wallet42 has quit [Quit: Leaving.]
hpk has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<hpk> what's the best doc/blog post on ipns?
kyledrake has quit [Quit: Leaving]
<cryptix> hey hpk - not sure why this isn't on the examples at ipfs.io, its pretty short and hands on, though
<cryptix> gateway.ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsNTn2aDxdXBFca7D/example#/ipfs/QmThrNbvLj7afQZhxH72m5Nn1qiVn3eMKWFYV49Zp2mv9B/ipns/readme.md
<cryptix> hpk: i think we met at dtnconf btw. i'm still trying to find the crypto hippies on the canaries.. :))
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
wallet42 has joined #ipfs
daviddias has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
atomotic has quit [Quit: Textual IRC Client: www.textualapp.com]
hellertime has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
chriscool has quit [Ping timeout: 272 seconds]
daviddias has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
wallet42 has quit [Quit: Leaving.]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
atomotic has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
therealplato has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
nell has quit [Quit: WeeChat 0.4.2]
void has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix> void: thanks :)
ehmry has joined #ipfs
emery__ has joined #ipfs
emery__ has quit [Quit: Leaving]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ehmry has quit [Quit: Leaving]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
nessence has joined #ipfs
nessence has quit [Remote host closed the connection]
nessence has joined #ipfs
domanic has quit [Quit: Leaving]
domanic has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
therealplato has quit [Ping timeout: 265 seconds]
domanic_ has joined #ipfs
domanic has quit [Ping timeout: 265 seconds]
therealplato has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic_ is now known as domanic
peerchemist has joined #ipfs
pfraze has joined #ipfs
atomotic has quit [Quit: Textual IRC Client: www.textualapp.com]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> i find myself frustrated that viewing files on my computer and browsing the internet are such different tasks
<mmuller_> whyrusleeping: me too. I just want one path that presents me a unified view from wherever.
<whyrusleeping> mmuller_: exactly!
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mildred has quit [Ping timeout: 272 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Tv` has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
wallet42 has joined #ipfs
void has quit [Quit: ChatZilla 0.9.91.1 [Firefox 38.0/20150511103818]]
wallet42 has quit [Client Quit]
headbite has quit [Quit: Leaving.]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
rht_ has quit [Quit: Connection closed for inactivity]
<peerchemist> can someone please guide me through listing and getting a file from another node? I have issues with this one, now I run nodes on LAN, and they see each other, and I can ping properly. But I can not list files on the node nor I can obtain the file. Both "ls" and "cat" fail with "Error: context deadline exceeded"
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> you cannot say "show me all files on this other node"
<whyrusleeping> you can request a given file from the network
<whyrusleeping> and if that node happens to have it, it may give it to you
<whyrusleeping> using the bitswap protocol you can directly request a file from a given node as well
<cryptix> tl/dr; 'node1$ ipfs add fileA' then 'node2@ ipfs cat $hashOfFileA' should work
<cryptix> peerchemist: maybe you could copy/paste together what you did so we can eliminate wrong assumptions?
domanic_ has joined #ipfs
domanic has quit [Ping timeout: 255 seconds]
domanic_ has quit [Ping timeout: 272 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<peerchemist> okay
<peerchemist> so $node1 has file hello.txt added
<peerchemist> that works
<peerchemist> then I ping $node1 from $node2 to see if it works
<peerchemist> it does, they are connected
<peerchemist> then I do ipfs cat /ipfs/$node1ID/hello.txt >hello.txt
<cryptix> yea, that cat is wrong
<cryptix> just 'ipfs cat $hashofhello.txt'
<cryptix> ipfs doesnt care where your content is, it will figour out the location itself
<peerchemist> okay
<peerchemist> so how to get a hash of the file?
<whyrusleeping> have a pi running ipfs on a 10000mah battery, been up for 14 hours
<whyrusleeping> peerchemist, ipfs add tells you
<peerchemist> okay, got it
<peerchemist> this was not clear from your intro documentation :)
<peerchemist> and how to list files on a specific node?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
<cryptix> peerchemist: there currently is no nice way of doing this, i fear... content you added yourself will be pinned and you can list that with 'ipfs pin ls'
<cryptix> whyrusleeping: do you have something that needs review?
daviddias has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<whyrusleeping> cryptix: uhm... if i could get your thoughts on the locking issue
<whyrusleeping> and
<whyrusleeping> the bitswap PR
<whyrusleeping> theres something weird with it
<cryptix> the relevance of camlistore/lock? in short: there should be a more modern impl of that functionality
<cryptix> i'll respond on gh in full
<whyrusleeping> cryptix: okay, i started working on one
<cryptix> i'd check confd/docker/syncthing/... etc to see if they have something solid. i have only vague clues about how difficult it is to strike a good balance between portability and ease of impl
<whyrusleeping> well, ive got it working on linux, osx and bsd
<whyrusleeping> windows doesnt have a syscall.Flock
notduncansmith has joined #ipfs
<whyrusleeping> i'm going to use a very similar method to camlistore locks fallback for every other OS that doesnt have a real locking mechanism
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix> not invented here, huh? ;)
<whyrusleeping> lol
<whyrusleeping> trying to avoid reinventing the wheel as much as i can
<whyrusleeping> makes maintaining things easier
<cryptix> 'the bitswap pr' is #1290?
<cryptix> re lock: no, sure - go for it :)
<whyrusleeping> yeah, 1290, feat/bitswap-speed
<whyrusleeping> the primary motivation for that PR is getting findproviders out of the main request flow
<whyrusleeping> currently, all requests have to wait for the previous one to finish its find providers call
<whyrusleeping> which is made worse by the dht not being as good as it could be right now
<cryptix> aaah, i think i understand
<peerchemist> one more question, if node is behind the firewall... which port do I need to open to allow communication to outside world?
<cryptix> so you basically go 'i want x -> how has x? -> <waiting for prividers>' and still have lots of other 'i want y,z,q' queued up?
<cryptix> peerchemist: that's the 'swarm' address in config
<cryptix> by default its 4001
<whyrusleeping> cryptix: yeah, thats why its kinda slow
<whyrusleeping> mostly, find providers hangs on bad dials
<whyrusleeping> otherwise its pretty snappy
<whyrusleeping> but still not snappy enough to warrant having it in the main flow path for no reason
wallet42 has joined #ipfs
pfraze has quit [Remote host closed the connection]
<peerchemist> @cryptix, thx
notduncansmith has joined #ipfs
<peerchemist> how do you implement gateway? by placing datastore in webserver root?
mildred has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix> peerchemist: like http://gateway.ipfs.io/ ? the 'ipfs daemon' just comes with one at localhost:8080
<cryptix> the host/port is configured with Addresses.Gateway in the config
<peerchemist> that is simple, okay
<cryptix> whyrusleeping: ah - now i get it, before you waited with the WantBlocks() call until you know that a peer has it, now you just threw them on the want manager and then start searching for providers
<whyrusleeping> cryptix: yeah, that way we can move on with requests
<cryptix> yup got it :)
<cryptix> wow.. swap on an ssd still frightens me sometimes...
pfraze_ has joined #ipfs
mildred has quit [Ping timeout: 246 seconds]
<whyrusleeping> rpi running ipfs has 15 hours of uptime on a 10ah battery so far
<whyrusleeping> no real estimate of how long its going to last though...
<peerchemist> @cryptix; zramwsap ;)
<jbenet> impressive. considering ipfs is currently a hog.
pfraze_ has quit [Remote host closed the connection]
cjdmax has quit [Remote host closed the connection]
<jbenet> hello everyone o/ i'm back online with reasonably fast internets for many hours at a time.
<jbenet> sorry if i'm slow on returning email-- i'm on that today.
<cryptix> hey jbenet :)
<jbenet> and if you have high prio items for me to take a look just drop me links here. i'll try to get to all outstanding PRs needing CR / merging in next 2-3 days.
<cryptix> peerchemist: well.. i meant more the feeling of impending doom when your swap dev fills up with 500megs a second
notduncansmith has joined #ipfs
<jbenet> cryptix: o/ -- i miss berlin already.
<cryptix> jbenet: tell me about it..
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> i missed seattle for a long time
<whyrusleeping> i'm happy where i'm at for a few more months at least
<whyrusleeping> :P
daviddias has joined #ipfs
<whyrusleeping> rfcr
mildred has joined #ipfs
<peerchemist> last night @whyrusleeping told me about yet to be implemented "keypool" feature. Can you tell me (in short) how it will work?
<whyrusleeping> peerchemist, it will allow you to create as many different keys as you like, encrypt content with those keys, and share the keys with others so they can seamlessly decrypt content encrypted by those keys
atomotic has joined #ipfs
<whyrusleeping> youll also be able to have a different ipns record per key you generate
<peerchemist> is there an integrated way to share the key to a specific node, or that is to be done outside IPFS
<whyrusleeping> that will be done through ipfs, securely
<whyrusleeping> (securely as long as you trust the peer id you type)
<peerchemist> I take it that "revoking" the key from node you gave it before is done by simply changing the encryption?
<whyrusleeping> yeah, there wouldnt be an easy way to revoke someones access to content
<whyrusleeping> unless someone knows of more advanced crypto than i do
<whyrusleeping> but afaik, crypto is the only way to do distributed permissions
<peerchemist> yep
<peerchemist> and revoking in this way I've just explained is quite simple and effective
<jbenet> peerchemist: yes, re-keying is the only way to revoke permissions if you hand out the key directly.
<jbenet> peerchemist: more sophisticated setups might use decryptor nodes (i.e. a node you use as a decryptor that has access to the right keys), but this is much less flexible.
<whyrusleeping> yeah, and that breaks decentralization
<peerchemist> okay
<peerchemist> I have some more questions.... underlying database is leveldb. Now, is it possible to use shared leveldb as key:value store instead of emulated layer above it?
<peerchemist> or would that break all kind of stuff?
<whyrusleeping> peerchemist: you could do that
<jbenet> peerchemist: we're actualy moving out of leveldb, right into the raw filesystem. but-- we have a "repo" abstraction that lets you write different backends for it. (so yes).
<jbenet> peerchemist: we have s3 on the way.
wallet42 has quit [Quit: Leaving.]
<peerchemist> I am quite interested in using IPFS as p2p database, so key:value store instead of "files"
<jbenet> peerchemist: yep, that would work :)
<peerchemist> splendid
G-Ray has joined #ipfs
<peerchemist> So following setup can be implemented > have decentralized key:value store, and define which nodes can write and which can read. Others are just storage nodes
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> jbenet: can you explain what decapsulate is used for in multiaddr?
<jbenet> whyrusleeping: i think decapsulate( /A/a/B/b/C/c, /A/a) -> /B/b/C/c (or a signature close to that)
<whyrusleeping> okay
<jbenet> the idea is to easily do: encapsulate(A, B) -> A/B decapsulate(A/B, A) -> B
<jbenet> the reason for "-capsulate" instead of "-wrap" is that it's directly related to packet en/de-capsulation.
<jbenet> (but naming-- meh)
<jbenet> peerchemist: that's correct.
<tperson> Github pages should use IPFS.
<jbenet> peerchemist: in the future we will make it super super easy to do that. it's a good amt of work atm because you have to define the write/read perm stuff yourself. but ideally we can use capabilities in the future to do all that.
<jbenet> tperson agreed. we need to get the perf up to match before we can make the claim for real though.
cjdmax has joined #ipfs
<peerchemist> okay, np. I now have enough data to merge this into my idea. When I finish the paper I will present it here. I hope we can work together on this
<jbenet> perchemist sgtm.
<jbenet> peerchemist*
<jbenet> cjb: i'm in NYC today/tonight. i leave tomorrow, but should be back in june.
<vaelys> arude.
<jbenet> (if we wanted to hack on git things)
<jbenet> vaelys ?
<vaelys> Are you going to be there early jun?
<vaelys> june*
<jbenet> vaelys possibly
<vaelys> This is early June: http://www.nycbug.org/index.cgi
<jbenet> vaelys dont have return planned atm, but must be here within Jun 5-20
<vaelys> ah
<vaelys> I'll be there in July
<vaelys> missu
<jbenet> i may be here july too. i'll be around nyc more often now.
<vaelys> neat
<jbenet> whyrusleeping: lmk what's next for CR on that. i'm not looking at PRs page today until i whittle this email stack down.
<whyrusleeping> that one is RFCR, but not RFM, it needs feedback
notduncansmith has joined #ipfs
<whyrusleeping> not "finalized" yet
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet> ok on it
<whyrusleeping> thats the final one before bitswap perf is in my acceptable range
<jbenet> i think we're far from that :/
<whyrusleeping> you might be surprised
<whyrusleeping> if it werent for that weird network latency crap, transfers would be pretty snappy
<whyrusleeping> (and dht resolution issues)
wallet42 has joined #ipfs
<peerchemist> do you have plans to improve privacy of the nodes? Integration with I2P and/or Tor?
<jbenet> peerchemist: that is a very long subject. in short, yes, but that comes later.
tilgovi has joined #ipfs
<peerchemist> ok thx
hellertime has quit [Quit: Leaving.]
<jbenet> whyrusleeping: yeah the RTT stuff is a problem. right now we waste a ton of times to RTTs. we look like http://gateway.ipfs.io/ipfs/QmRXgLAtuzPJswEaNuEjgijc6z2ytYXCpdHp9V3abHaVgA/bitswap-rtts.png
<whyrusleeping> thats assuming perfect, unlimited bandwidth, but yes
<jbenet> whyrusleeping: yep, agreed. well-- technically the sender stopped at some point ;) maybe that's the bw cap, or the end of the file.
<jbenet> end of the dag*
<jbenet> mafintosh: i think we need a repliconf
<jbenet> mafintosh: a conf all about dag replication algorithms, perf, strategies, benchmarks, applications, etc.
* whyrusleeping votes to have it in seattle
pfraze_ has joined #ipfs
<jbenet> whyrusleeping: try as i might, the bitswap implementation won't stick to my head. not sure what feedback to give in this PR.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> oh.. alright.
<jbenet> whyrusleeping: could you write up how this PR changes the model maybe?
peerchemist has left #ipfs [#ipfs]
<jbenet> whyrusleeping: like a paragraph or two about what changes here, and why. we should be doing much more of that in general, but specially in changes like these
lgierth has joined #ipfs
<jbenet> (and perhaps it should go into the commit messages)
<cryptix> whyrusleeping: could blockstore.Has be an s3 instance there? if so the err def needs to bubble up at some point
pfraze_ has quit [Ping timeout: 265 seconds]
<jbenet> cryptix: yeah possibly.
mildred has quit [Ping timeout: 246 seconds]
<cryptix> jbenet: btw were you able to visit the travis office on monday? i thought they might be closed due to holiday
<jbenet> cryptix: i figured they would be and i didnt have much time anyway so didnt try
<cryptix> kk
<jbenet> wking: lmk if CR is much better earlier today, or later evening / tomorrow ok.
neoteo has joined #ipfs
<neoteo> Hi guys. Don't know if this is useful for anyone but I've made a Swift version of Multihash. -> github.com/NeoTeo/SwiftMultihash
<jbenet> neoteo: that's great! mind adding it to https://github.com/jbenet/multihash ?
<cryptix> we need to increase the 'go test -timeout' on travis
<neoteo> Sure thing.
<whyrusleeping> cryptix: i was under the opinion we needed to lower it
<whyrusleeping> travis's timeout is also 10 minutes, and if both timeout at the same time, we lose out on the stack dump
<cryptix> ah.. then the 'no output' kicks in, huh?
<whyrusleeping> yeah, exactly
<cryptix> sigh.. yea.. than we might need to slip up the tests some more
notduncansmith has joined #ipfs
<whyrusleeping> which ones are taking too long?
<wking> jbenet: review for what?
<jbenet> FWIW travis timeout is "10min without output" and it can be disabled with a hack (that outputs something every minute) though it's not recommended
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix> i can trigger timeouts on bitswap with my machine using the stress tool
<jbenet> wking: for the adder api or whatever
<whyrusleeping> ideally, the entire 'go test ./...' shouldnt take more than a minute, bitswap tests should only take ~30s on a busy machine
<wking> jbenet: I don't have any secret local changes for the adder API ;). Review whenever.
<jbenet> cryptix: yeah i think no output for 5min is alraedy bad times. no need to wait 10min for a hang
<whyrusleeping> there is something weird going on in the large swarm test that randomly makes it take much longer
<whyrusleeping> but i think thats to do with goroutine scheduling
<jbenet> cryptix: although --- is the 10min for all of `go test` ??
<jbenet> whyrusleeping: yeah i wonder.
<cryptix> jbenet: no, just the exchange/bitswap. using x/tool/stress, it usually kicks in after ~600 runs that tests start to fail
<wking> jbenet: Should I open a branch with the current version so folks don't have to read the comments to pickup trickle -> layout and such (https://github.com/ipfs/go-ipfs/issues/1291#issuecomment-106011606)
<jbenet> wking: sounds good-- just dont mean to block others. i've a ton of stuff to get through so want to order the PQ accounting for others' expectations
<cryptix> and dont run that combo with -race or your life will oom out
<wking> yup, I'm in no hurry for the adder API, so long as we have a consensus in a week or so
<jbenet> wking: not sure-- haven't taken a look.
mildred has joined #ipfs
<jbenet> wking: maybe whyrusleeping can comment on that
<wking> And I don't mind if we land stuff that doesn't match my currrent recommendation in the meantime (e.g. #1274)
daviddias has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
daviddias has joined #ipfs
<jbenet> ffs. i really dislike that github doesnt let me rebase a PR from the webui
<jbenet> it makes out history very convoluted and hard to read.
<jbenet> maybe i should suck it up and rebase + merge things manually
<cryptix> hehe maybe :)
<whyrusleeping> daviddias: you around?
<daviddias> still on the meeting
<daviddias> will need more 30 mins
<whyrusleeping> daviddias: cool, gonna grab lunch
daviddias has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
anshukla_ has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
anshukla has quit [Ping timeout: 240 seconds]
anshukla_ has quit [Ping timeout: 244 seconds]
nessence has quit [Remote host closed the connection]
lgierth has quit [Quit: Ex-Chat]
G-Ray has quit [Remote host closed the connection]
atomotic has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mildred has quit [Ping timeout: 256 seconds]
mildred has joined #ipfs
<mafintosh> jbenet: sounds like a good idea. i'd be there if one were to happen
nessence has joined #ipfs
notduncansmith has joined #ipfs
nessence has quit [Ping timeout: 264 seconds]
notduncansmith has quit [Ping timeout: 258 seconds]
lgierth has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
lgierth has quit [Client Quit]
daviddias has joined #ipfs
daviddia_ has joined #ipfs
zorun has quit [Ping timeout: 276 seconds]
zorun has joined #ipfs
daviddias has quit [Ping timeout: 265 seconds]
G-Ray has joined #ipfs
<G-Ray> Hi, can we take part of the filecoin development ?
<G-Ray> Hi, can we take part of the filecoin development ?
www1 has quit [Ping timeout: 276 seconds]
<jbenet> G-Ray short answer is yes, but longer answer is when timing is right. PM me and let's setup a time to talk.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
www has joined #ipfs
ei-slackbot-ipfs has quit [Ping timeout: 255 seconds]
EricJ2190 has joined #ipfs
nessence has joined #ipfs
wallet42 has quit [Ping timeout: 240 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
nessence has quit [Ping timeout: 272 seconds]
wallet42 has joined #ipfs
mildred has quit [Ping timeout: 256 seconds]
Wallacoloo has joined #ipfs
inconshreveable has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Wallacoloo has quit [Ping timeout: 250 seconds]
kyledrake has joined #ipfs
pfraze_ has joined #ipfs
<kyledrake> jbenet whyrusleeping http://i.imgur.com/YowuK7g.png
<jbenet> kyledrake: that looks awesome
<kyledrake> Seems to work!
<kyledrake> Not perfect, but it'll do for now.
<jbenet> kyledrake: we havent fixed your json problem-- tperson what do we need to fix? maybe krl can help too
<kyledrake> Next step: nginx proxy so I can do MULTIHASH.ipfs.neocities.org
<whyrusleeping> kyledrake: :D
<jbenet> kyledrake: that would be super sweet
<kyledrake> And bake it into the site interface.
<kyledrake> jbenet It's really the only way for us to make it work. It needs that root domain and origin to work correctly.
<kyledrake> This won't be hard to do, I can probably bang it out today.
<kyledrake> Then I'll just throw an update order for every site, and it'll finish up sometime this week.
<whyrusleeping> kyledrake: are you slowed down at all by ipfs?
<jbenet> super awesome
<whyrusleeping> anything you want me to hit with the perf stick?
<jbenet> whyrusleeping: we need to fix the json problem
<jbenet> oh perf-wise--
<whyrusleeping> jbenet: we needed to make a decision on tpersons PR
<kyledrake> whyrusleeping The bottleneck here is the sshfs mount I'm using to isolate this from our main fileserver.
<whyrusleeping> kyledrake: ah, okay
<whyrusleeping> well if ipfs gives you any trouble, please let us know :)
<kyledrake> whyrusleeping yeah I'm not sure yet. I haven't tried to just dump the whole site right in with the current FS to test performance. I could run that on our replica server as an experiment though.
<kyledrake> This is going to be a Sites.all.each {|s| ArchiveWorker.perform_async s} basically. So I guarantee this is going to take a while, but once it catches up it should be fine.
<whyrusleeping> okay
<kyledrake> (if it does ;) )
notduncansmith has joined #ipfs
<kyledrake> whyrusleeping Independently, I'll throw the latest ipfs on our replica and order an all-site dump via time ipfs add -r so you can get some metrics on performance.
notduncansmith has quit [Read error: Connection reset by peer]
wallet42 has quit [Quit: Leaving.]
<kyledrake> This can take a week if it wants to - it won't cause any problems.
<kyledrake> The IPFS hashes will show up on the site profiles as they come online
<whyrusleeping> okay, i know we're around 40-50GB/hr on adds of largeish files
<whyrusleeping> i'm not sure how performance would look on neocities
<jbenet> ooof that's still slow. much better than before, but still got lots of room to go
inconshreveable has joined #ipfs
<whyrusleeping> yeah, 10x what it was before :P
<kyledrake> whyrusleeping tons of small files.
<kyledrake> Well, 30-50GBish of small files in many directories.
<kyledrake> If it can do it in an hour, that's more than good enough for us.
ei-slackbot-ipfs has joined #ipfs
<kyledrake> Starting on UI now, bbiab
kyledrake has quit [Ping timeout: 255 seconds]
inconshreveable has quit [Ping timeout: 256 seconds]
nessence has joined #ipfs
therealplato has quit [Ping timeout: 264 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
nessence has quit [Ping timeout: 272 seconds]
therealplato has joined #ipfs
neoteo has quit [Quit: Be back later ...]
kyledrake has joined #ipfs
www1 has joined #ipfs
www has quit [Ping timeout: 252 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
headbite has joined #ipfs
flugsio has quit [Quit: WeeChat 1.2]
G-Ray has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
therealplato has quit [Ping timeout: 245 seconds]
pfraze_ has quit [Remote host closed the connection]
<whyrusleeping> my battery pi has 20 hours of uptime now, with 77 active peer connectins
<whyrusleeping> ~3% idle cpu load from ipfs
<Evermore> whyrusleeping: Is this one of the new pis or one of the old ones
<whyrusleeping> this is the quad core
<Evermore> I want to know what hopes I have of running ipfs on the Pandora
<Evermore> oh :(
<whyrusleeping> oh, you can easily run it on the older ones
<Evermore> What's the RAM usage look like?
<Evermore> I think I got 512 but Firefox takes up a whole pile if it's running
<whyrusleeping> 57MB
<Evermore> hm I can live with that
<whyrusleeping> it could be better
<whyrusleeping> i think with a little bit of effort, i could drop it to 20MB
<whyrusleeping> without changing the perf
nessence has joined #ipfs
<cryptix> whyrusleeping: do you see any sense in this? /ipfs/QmPHSpem6CjnczSBxHs2SM9FSoQ9Juk7sAbA4S6gNV7HYA
<whyrusleeping> cryptix: no, thats weird...
<cryptix> it feels like there is some global state getting clogged up.. the bitswap tests are really finicky
<whyrusleeping> yeah, ive noticed
<whyrusleeping> its weird that youre getting outright failures though
<whyrusleeping> i normally get testLargeSwarm timing out
notduncansmith has joined #ipfs
<cryptix> i disabled the largeSwarm tests, they cause the lockups/timeouts
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> oh, those are all timeouts
nessence has quit [Ping timeout: 258 seconds]
<cryptix> no, those arnt timeouts
<cryptix> timeout would be a sigabort kind of kill
<cryptix> this is the test suite exiting
<whyrusleeping> those are context timeouts
<whyrusleeping> note the precise timing
<whyrusleeping> each one almost right at 60s
<cryptix> ah, right - this must be the first first masked context: timeout error i found
<cryptix> this still feels weird.. why would it time out with 0 delay
<whyrusleeping> hrm?
<whyrusleeping> something is definitely weird
<cryptix> oh wait...
<cryptix> i'll let it stew for some more
<whyrusleeping> okay
<whyrusleeping> i'm really happy youre helping to look into this
<cryptix> you can at least use the stress runner to kick a test into zombieland pretty easily and inspect it with gdb.. i wish there was a nicer way to work with these huge stack traces
<whyrusleeping> its hard for me since i wrote most of it
<whyrusleeping> (hard to find issues in your own code compared to in somebody elses)
<whyrusleeping> does gdb work for go?
<whyrusleeping> i didnt think the symbols were right
<cryptix> oh no
<whyrusleeping> Evermore: i'm also going to experiment with an odroid w, and an intel edison soon
<cryptix> gdb 7.1
<whyrusleeping> oh hot
<cryptix> its limited but you can say 'goroutine 12 bt' for instance
<cryptix> but
<cryptix> not right now
<cryptix> they broke the python bridge between the runtime and gdb - thats where i left of before..
<cryptix> its fixed on go master but.. thats a mind field to walk in right now, i think
<cryptix> whyrusleeping: o/ dont get over excited
Wallacoloo has joined #ipfs
<whyrusleeping> :(
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix> i can't wait for a rc1 cut
<whyrusleeping> i'm not too excited for 1.5 honestly
<whyrusleeping> compilation is taking a decent perf hit
<cryptix> well.. we'll see - the gc perf also looked horrible at first
<whyrusleeping> true
<cryptix> and personally, i'm allready liking gb more
kyledrake has quit [Ping timeout: 272 seconds]
<whyrusleeping> hmmmmmm
<whyrusleeping> thats the package manager thingy?