notduncansmith has quit [Read error: Connection reset by peer]
inconshr_ has joined #ipfs
ehmry has quit [Quit: Leaving]
inconshreveable has quit [Ping timeout: 265 seconds]
martinBrown has quit [Quit: -]
martinBrown has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
<whyrusleeping>
jbenet: something weird is going on in our network stack
notduncansmith has quit [Read error: Connection reset by peer]
timgws_ has joined #ipfs
<whyrusleeping>
its taking up to 1 second to send a 256k block over LAN
timgws has quit [Ping timeout: 246 seconds]
<jbenet>
Sure it's isolated at network?
<whyrusleeping>
yeap, no bootstrapped peers
<whyrusleeping>
and i'm wrapping timers around the SendMessage call
timgws_ has quit [Quit: Computer has gone to sleep.]
timgws has joined #ipfs
<jbenet>
Hm I'm always happy to hear of problems found down there
<jbenet>
Usually increases perf dramatically :)
timgws has quit [Client Quit]
<whyrusleeping>
lol
* whyrusleeping
gets his network layer diving gear and sighs
<jbenet>
I'm on phone so I can't manipulate code effectively but let's try
<jbenet>
Or rather can I say things to try?
<whyrusleeping>
sure
<jbenet>
Log when writes go out and reads come in of manet.Conn
<whyrusleeping>
i did, 'ipfs ping' the peer reports 2.3 ms
<whyrusleeping>
thats round trip
<jbenet>
Would love to see the difference between the writes and see if some happen to take longer
<whyrusleeping>
some sends take ~1s
<whyrusleeping>
some are nearly instantaenous
<jbenet>
What do you mean by take 1s? Between what and what?
<whyrusleeping>
between me calling 'SendMessage' and 'SendMessage' returning
domanic has joined #ipfs
<jbenet>
So am not sure whether yamux does any waiting
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Taek is now known as Taek42
<whyrusleeping>
hrm...
<jbenet>
Another thing to try is to disable secio
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has quit [Ping timeout: 264 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
lgierth_ has quit [Quit: Ex-Chat]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Taek42 is now known as Taek
williamcotton has joined #ipfs
vitzli has joined #ipfs
<vitzli>
morning
<vitzli>
whyrusleeping, which language should I use for the benchmark test? sh/bash?
notduncansmith has joined #ipfs
<whyrusleeping>
vitzli: sh is what we normally use, chriscool and jbenet have a grudge against bash
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
in a perfect world i would use fish shell scripts for everything
<jbenet>
Not a grudge it doesn't work _everywhere_
<jbenet>
There's install overhead
<whyrusleeping>
>.> "grudge"
<jbenet>
(Which I guess we could incur, but best not to?)
<jbenet>
Haha
<vitzli>
sh in POSIX environment? so I could assume bc is present?
<whyrusleeping>
bc?
<whyrusleeping>
i dont think so... its not normally installed on my machines
<whyrusleeping>
but maybe
<vitzli>
sh is not an issue, I think I could do this
timgws has joined #ipfs
timgws has quit [Changing host]
timgws has joined #ipfs
<whyrusleeping>
vitzli: awesome :)
williamc_ has joined #ipfs
williamcotton has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
EricJ2190 has quit [Ping timeout: 252 seconds]
vitzli has quit [Quit: Leaving]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
weeeeird network stuff jbenet
<whyrusleeping>
put timers around the swarm stream writes
<whyrusleeping>
logging size of the write, and the time it took
<whyrusleeping>
theyre taking a long time there
<whyrusleeping>
even on really small (1 or 3 byte) writes
<whyrusleeping>
outgoing writes might need to be buffered
<whyrusleeping>
maybe thats the issue?
<whyrusleeping>
but no... tcp should buffer for us, right?
<whyrusleeping>
"QmWmaYimYHtjHvKHp9A7M7vfQ8gRxfkr6TLRMyDBrpijWZ// Write writes bytes to a stream, flushing for each call.
<whyrusleeping>
whoops, ignore the hash
patcon has quit [Ping timeout: 272 seconds]
pfraze has joined #ipfs
timgws has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ebarch has joined #ipfs
timgws has joined #ipfs
domanic has quit [Ping timeout: 250 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
inconshr_: hey, you around?
<inconshr_>
o/
<whyrusleeping>
any ideas on techniques for debugging network latencies?
<inconshr_>
unpack that further?
<whyrusleeping>
i'm on a LAN, with ~2ms actual latency between the peer
<whyrusleeping>
s
<whyrusleeping>
but writes are taking a long time, up to a second
<whyrusleeping>
and the writes arent large, or super numerous
<whyrusleeping>
i have the window size set plenty large enough on the stream muxer
<inconshr_>
there are probably two good options for this
<inconshr_>
1. application level metrics
<inconshr_>
insert a whole bunch of timing points into the code that writes stuff out wherever
<inconshr_>
so that you can analyze it later
<inconshr_>
or 2. try network monitoring tools like tcpdump
<whyrusleeping>
hrm, okay. i might have to take a look at tcpdump
<inconshr_>
it only gets you so much visibility
<inconshr_>
it may or may not help
<whyrusleeping>
alright
<whyrusleeping>
i'm already outputting timing statistics from a few different points on the layer
<whyrusleeping>
thanks!
<whyrusleeping>
inconshr_: one other thing, when is the new muxado gonna be merged?
<inconshr_>
good question
<inconshr_>
not sure. i can tell you that it works really well though!
<inconshr_>
it's running all of ngrok 2.0
<whyrusleeping>
oh shit, okay
<inconshr_>
without any issues (that i've seen)
timgws has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
timgws has joined #ipfs
timgws has quit [Read error: Connection reset by peer]
<inconshr_>
whyrusleeping: i've actually been thinking
<inconshr_>
what if the right way to right these low-level networking libraries is to actually use non-blocking sockets and a polling loop all from one goroutine
<inconshr_>
just like the good-ol' C days
<whyrusleeping>
ooooo, i'd love to try that
<inconshr_>
you avoid the whole overhead of goroutine switching and the goroutine scheduler
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot>
[go-ipfs] whyrusleeping created muxado2-test (+1 new commit): http://git.io/vTtKx
<ipfsbot>
go-ipfs/muxado2-test c23a5b9 Jeromy: testing out muxado 2
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamc_ has quit [Ping timeout: 272 seconds]
sharky has quit [Ping timeout: 276 seconds]
sharky has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
hm
<krl>
jbenet: i'm getting into pretty random sleep habits as well it seems..
<jbenet>
haha
<jbenet>
the internet does that
<krl>
but fixed up the ui things you pointed out and fpushed
<krl>
when do you arrive here btw?
<jbenet>
friday
<jbenet>
need me to take a look at anything?
<17SACLI7U>
[webui] jbenet pushed 1 new commit to master: http://git.io/vTt7n
<17SACLI7U>
webui/master b5c2de8 Juan Batiz-Benet: Merge pull request #50 from ipfs/object-view-extras...
pfraze has quit [Remote host closed the connection]
<jbenet>
oh that's great-- could work very well. i asked about one space, but maybe can email them too.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mildred has joined #ipfs
domanic has quit [Ping timeout: 240 seconds]
Tv` has quit [Quit: Connection closed for inactivity]
<ipfsbot>
[webui] krl opened pull request #52: Set gateway based on window hostname (master...gateway-link-quickfix) http://git.io/vTtA3
domanic has joined #ipfs
<krl>
jbenet: just texted the space org to find out if it's free, i'll let you know
<jbenet>
awesome, thanks!
<jbenet>
we should hang out friday and monday regardless -- if you've time
<krl>
yeah i'm free
patcon has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Wallacoloo has quit [Quit: Leaving.]
domanic has quit [Ping timeout: 272 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
RzR has quit [Excess Flood]
RzR has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
timgws has joined #ipfs
patcon has quit [Ping timeout: 245 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
timgws has quit [Read error: Connection reset by peer]
timgws has joined #ipfs
timgws has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has quit [Ping timeout: 246 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
atomotic has joined #ipfs
www1 has quit [Quit: Leaving.]
lohkey has quit [Ping timeout: 276 seconds]
lohkey has joined #ipfs
jbenet has quit [Ping timeout: 276 seconds]
mafintosh has quit [Ping timeout: 276 seconds]
mikolalysenko has quit [Ping timeout: 276 seconds]
jbenet has joined #ipfs
mafintosh has joined #ipfs
insanity54 has quit [Ping timeout: 276 seconds]
insanity54 has joined #ipfs
mikolalysenko has joined #ipfs
insanity54 has quit [Ping timeout: 276 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
insanity54 has joined #ipfs
ehmry has joined #ipfs
chrisr_ has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix>
gmorning ipfss
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
crest_ is now known as crest
chrisr_ has quit [Ping timeout: 246 seconds]
m0ns00n has joined #ipfs
<m0ns00n>
Hoi!
<cryptix>
hi m0ns00n
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<m0ns00n>
Hoi!
<m0ns00n>
:)
<m0ns00n>
cryptix: You one of the devs here?
<cryptix>
kinda :)
<cryptix>
hit me
<crest>
cryptix: any idea when file system commands will be implemented? adding everything again for every tiny change sucks even if dedup limits the storage overhead
<crest>
the fuse /ipns mount implements most of the logic but is painfully slow
<crest>
ipfs add + ipfs name publish is faster if i want to add a 200MB file to a 5GB directory
<cryptix>
crest: i'm sure there are bottlenecks we can tackle to make things more tollerable
<crest>
cryptix: the problem is that i can't link to exiting sub-trees in building a new file system tree
ehmry has quit [Quit: Leaving]
<crest>
currently i have to add the whole file hierarchy and publish it as new root
<cryptix>
one thing baffling me currently is that in daemon mode, when adding a file, it is uploaded in total to the daemon which then adds its in total again. imho we should move the chinking and hash checking into the client
<crest>
cryptix: that would remove the ipc overhead but change nothing in terms of complexity
<cryptix>
crest: hmm - i think you need to get into the core/unixfs stuff and do that yourself (adding new items to a dir) if you want it more efficient right now
<crest>
cryptix: it looks like the code exists and just the tooling is missing
<cryptix>
crest: it definetly is. i saw it when working on the writable http gateway. it allows PUT methods to add hashes to an existing tree
<cryptix>
it got broken by reworking the path resolving stuff though :X
<m0ns00n>
:)
<cryptix>
but the plumbing is ther just no porcelain
<m0ns00n>
cryptix: We're working on a distributed, cloud OS
* crest
takes his shit to the outhouse
<m0ns00n>
cryptix: Kindof like you are git+bittorrent, we're a kernel+bittorrent
<m0ns00n>
cryptix: reason why I'm hanging around here, is because I would very much like to include ipfs in our OS default setup
<m0ns00n>
crest: for storage (like "My computer")
<m0ns00n>
sorry crest
<m0ns00n>
cryptix: If using our OS behind Tor, f.ex., you'd have a totally encrypted user environment
<crest>
m0ns00n: don't combine ipfs with tor an expect privacy
<crest>
m0ns00n: ipfs publishes all your local ip addresses to the DHT
<crest>
even rfc1918 ips
<cryptix>
crest: sorry if that was discouraging
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix>
m0ns00n: i have to agree with crest right now - you would have to rip out the current multiaddr stuff and make sure to just use tor transports (and idially only swarm with nodes that publish tor hidden service entry points)
<crest>
m0ns00n: this bit me when my hoster complained because ipfs tried to connect to peers with martian ip addresses. their stupid IDS complained and triggered and automatic warning + threat
<cryptix>
crest: hetzner?
<crest>
cryptix: correct ;)
<cryptix>
yea - i got that one, too..
<crest>
cryptix: maybe i should get a cloak...
<crest>
i just added a bogon filter to my pf.conf and ignored the problem
<cryptix>
hehe yea, sounds reasonable
<cryptix>
i opend an issue about the stupid dials if you want to track it
<crest>
maybe once in a blue moon someone will find a valid peer via rfc1918 ips
<m0ns00n>
crest: Used an example, point is, web browser + crypto + encrypted filesystem = very safe user environment
<m0ns00n>
:)
<cryptix>
m0ns00n: i think it's agreat idea though btw - its just that you have to be very careful with it. i'd like add /i2p/$pubkey addresses to multiaddr someday - but like crest said - ipfs currently assumes it's okay to advertise ALL of its known adresses, which is not tolerable for privacy
<m0ns00n>
cryptix: Yes - but still, I think the ideals of ipfs aligns well with FriendUP.
<m0ns00n>
cryptix: We're gonna try to launch our website this week - friendos.com
<m0ns00n>
cryptix: right now it's still the "stupid quick thing for investors" :)
<cryptix>
m0ns00n: i'll keep an eye open ;)
<m0ns00n>
Working hard now to demonstrate in amsterdam and california
inconshr_ has quit [Remote host closed the connection]
tilgovi has quit [Ping timeout: 265 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mildred has quit [Quit: Leaving.]
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
atomotic has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
RzR has quit [Excess Flood]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
RzR has joined #ipfs
<m0ns00n>
cryptix: Have you ever used the AmigaDOS?
mildred has joined #ipfs
<cryptix>
m0ns00n: not that i know of
<m0ns00n>
cryptix: It was the DOS implementation of MorphOS, AmigaOS and AROS.
<m0ns00n>
cryptix: It's known for it's clean implementation of partitions and expressive shell.
hellertime has joined #ipfs
<m0ns00n>
cryptix: Right up there with Linux, but with an easier to understand partition implementation.
<m0ns00n>
cryptix: This is the one we're using in our OS.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<m0ns00n>
cryptix: It has also got some cool things like AREXX
<m0ns00n>
cryptix: Modified, this concept allows us to connect applications as smart disks.
chrisr_ has left #ipfs [#ipfs]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
atomotic has joined #ipfs
atomotic has quit [Ping timeout: 264 seconds]
<krl>
cryptix: commented on your metrics pull
Blame has quit [Quit: Leaving]
Blame has joined #ipfs
Blame_ has joined #ipfs
Blame_ has quit [Client Quit]
lgierth has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
HastaJun has quit [Ping timeout: 245 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
HastaJun has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
u7654dec has joined #ipfs
ur5edgb has quit [Ping timeout: 255 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
torpor has joined #ipfs
<xelra>
It's something I'm also looking forward to. Additionally to the global ipfs net, have the ability to only publish and connect to a very defined set of private nodes.
<xelra>
With authentication and authorization.
williamcotton has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has quit [Ping timeout: 264 seconds]
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
gatesvp has joined #ipfs
tilgovi has joined #ipfs
ur5edgb has quit [Ping timeout: 265 seconds]
rht___ has quit [Quit: Page closed]
fleeky has quit [Remote host closed the connection]
ur5edgb has joined #ipfs
torpor has quit [Quit: Leaving.]
notduncansmith has joined #ipfs
torpor has joined #ipfs
amiller- has quit [Ping timeout: 252 seconds]
williamcotton has quit [Ping timeout: 246 seconds]
fleeky has joined #ipfs
notduncansmith has quit [Ping timeout: 250 seconds]
Guest11561 has joined #ipfs
<alusion>
yo jbenet ya here?
<whyrusleeping>
i dont beleive so, but he may prove me wrong
torpor1 has joined #ipfs
<Luzifer>
but according to the current time it might be possible :D
<whyrusleeping>
he should be here at noon for the sprint meeting
<Luzifer>
should be around 11:30 at yours right?
<whyrusleeping>
yeah, its 11:@3
torpor has quit [Ping timeout: 252 seconds]
<Luzifer>
yay I'm getting better at guessing other TZs times :D
<whyrusleeping>
must be a side effect of none of us being in the same TZ :P
<whyrusleeping>
jbenet and i arent even in the same timezone right now
<Luzifer>
hmm he's at the east-coast? ;)
lgierth has quit [Quit: Ex-Chat]
<whyrusleeping>
yeap, he decided to go be weird and visit faraway lands
<whyrusleeping>
everyone knows west coast best coast...
torpor1 has quit [Quit: Leaving.]
<Luzifer>
nah east-coast isn't far away... image he'd stay in germany currently... thats way more distant ;)
<Luzifer>
8:28 pm distant :D
<jbenet>
speak of the devil
<Luzifer>
and he'll appear \o/
<whyrusleeping>
and jbenet shows up instead?
<jbenet>
and he will get an irc notification :D
<Luzifer>
ohai jbenet
<jbenet>
yeah he sent me instead-- he was busy.
<jbenet>
cryptix krl wking whyrusleeping: reminder sprint mtg in 30min
<jbenet>
err 45min
<Luzifer>
hmpf. need more desk space... have to set up another mac... weird company stuff going on :(
<whyrusleeping>
i had no idea
<Luzifer>
and a better screen... *grumbles*
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
whyrusleeping: maybe we should have a bot that reminds people about sprint stuff + asks people to sync daily.
<whyrusleeping>
Luzifer: sounds like you should come work with us instead :D
<whyrusleeping>
jbenet: i can do that
<jbenet>
whyrusleeping: did you TAL at chrischool's PR?
<whyrusleeping>
yeah, i did
<whyrusleeping>
it seems like it should work
<whyrusleeping>
and the tests pass
<whyrusleeping>
although, i havent pulled it down to play with yet
<whyrusleeping>
i can do that now
<Luzifer>
whyrusleeping: you remember the discussion a few months ago about that intellectual property stuff at google here in ipfs? looks like we're going that way now :(
<jbenet>
thoughts on the XOR scenario with stdin and args? do we have commands that need both?
<whyrusleeping>
Luzifer: that sucks...
alusion has quit [Quit: WeeChat 0.4.2]
<jbenet>
to be clear though -- google's very good about IP and open source, it's workflows are just slower than "commit away".
<Luzifer>
yeah. absoluteley. currently were above 90% of ppl who denies to sign that :D
<whyrusleeping>
jbenet: i can imagine a command that would need both
<whyrusleeping>
echo "my content" | ipfs add --encrypt=mykey
nell has joined #ipfs
<jbenet>
that needs some reform, but the actual clearing of people to work on OSS is really good.
<whyrusleeping>
but thats an option, not an argument
<Luzifer>
jbenet: might be... the document we had in discussion was not nice... :(
<jbenet>
whyrusleeping: right-- think arg.
<whyrusleeping>
uhm... in that case
<whyrusleeping>
i dont think so
Wallacoloo has joined #ipfs
<whyrusleeping>
our requirement could be to just structure it as an option if you might need both
<whyrusleeping>
although we should look at the source code for cat to see how it handles both
<whyrusleeping>
echo "test" | cat myfile
<whyrusleeping>
oh, it doesnt
<whyrusleeping>
cool
<whyrusleeping>
if cat has arguments, it ignore stdin
<whyrusleeping>
and i assume other unix commands behave the same way
<Luzifer>
if a filename is required and none is provided many tools take "-" as a substitute which is stdin...
<whyrusleeping>
yeah, that too
void has quit [Quit: ChatZilla 0.9.91.1 [Firefox 38.0/20150511103818]]
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
patcon has quit [Ping timeout: 256 seconds]
<krl>
jbenet: btw, what about friday?
<jbenet>
hey krl! still need to go through email-- thanks for setting it up. the ethereum folks also offered space for friday -- which is better in your opinion? (benefit of ethereum is that there's other people interested in this stuff)
<jbenet>
whyrusleeping: on bitswap-- i think the want manager and the engine should be together, yes.
<jbenet>
whyrusleeping: one thing to note here is that there needs to be a "decider" that has full view into everything-- the metrics, the bandwidth, the lag sending, the cancels, etc.
<jbenet>
whyrusleeping: i.e. all the information aggregated into one "thing" (even if that thing has per-peer sub-structs)
<whyrusleeping>
okay, thats what i was thinking too
<jbenet>
whyrusleeping: otherwise decisions happening in multiple places can run against each otehr
gozala has quit [Read error: Connection reset by peer]
<whyrusleeping>
well, the wantmanager doesnt make any decisions
<jbenet>
whyrusleeping: (and can't be integrated in an ML model easily ;) )
<whyrusleeping>
but i agree
<whyrusleeping>
ill make that change
<jbenet>
whyrusleeping: it could-- it could cancel a send ctx if it got a Cancel msg.
<whyrusleeping>
also, did you even try the branch out??
<jbenet>
whyrusleeping: not yet i can do that now finally.
<whyrusleeping>
do itttttt
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
whyrusleeping: -- and many cancels in a row without us finishing might tell the engine that we should either send ahead or send something else because our peer is having more luck with someone else
gozala has joined #ipfs
chriscool has joined #ipfs
<whyrusleeping>
other peers cancels go to the decision engine anyways
<whyrusleeping>
the wantmanager is only for the blocks that we ourselves want
<jbenet>
whyrusleeping ah ok.
* jbenet
tries it out.
<jbenet>
whyrusleeping: you added a readme.md -- there's a README.md -- osx can't even.
<jbenet>
whyrusleeping: also github's style is to use README.md
<whyrusleeping>
>.>
<whyrusleeping>
how does OSX even handle that?
<jbenet>
whyrusleeping: it overwrites to both files.
<jbenet>
err
<jbenet>
into one file.
<whyrusleeping>
dudewhat
<whyrusleeping>
how is that even acceptable in a real operating system?
<jbenet>
also "ipfs add -r go-ipfs" = 163MB and a 2.5 hr eta.
<jbenet>
this is crap.
<jbenet>
ok now it's a 10min eta.
<jbenet>
(past .git apparently)
<jbenet>
we really do not handle small files well.
<jbenet>
(many small files*)
<okket_>
well, most people think that readme.md and README.md are the same files, only computer distinguishes between them differently...
<jbenet>
okket_ yeah i'd agree.
<jbenet>
okket_ the problem is the mix with other use cases for filesystems, like hashes as filenames.
<jbenet>
(with encodings that use both upper + lower cases)
<ipfsbot>
[webui] krl opened pull request #53: Support un-named links (master...object-link-fix) http://git.io/vT3ym
<whyrusleeping>
krl: wanna will out stuff youre working on in that eterpad?
<tperson>
Almost... just almost moved into my new place.
<tperson>
Computer is getting setup now.
therealplato has joined #ipfs
<whyrusleeping>
tperson: jbenet just asked about you :P
<tperson>
Ya.. moving sucks. I'll be online a lot more come this afternoon. Going to pretty much spend all my time on the webui helping out now.
<tperson>
Did not think graduating and moving was going to take me off line so much. Thought I would actually have more time!
ehmry has joined #ipfs
williamcotton has quit [Ping timeout: 264 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has joined #ipfs
<jbenet>
thanks everyone for joining. sorry for my crap audio. things remaining to scope out:
<jbenet>
wking: mind adding whatever issues relevant to the etherpad?
<jbenet>
chriscool: if you want to add issues + PRs to a heading, i just made one for you.
<chriscool>
jbenet: ok thanks I will have a look
williamcotton has quit [Client Quit]
<jbenet>
krl: we put electron shell and https://github.com/ipfs/go-ipfs/issues/1153 under yours. go-ipfs/1153 is just picking a nicer looking index page. im happy to find something that works and provide all the html + css needed, it'll then just be wiring it up on go-ipfs.
void has joined #ipfs
<jbenet>
krl: anything else that you cant to grab would be great for webui or node land of things
<Blame>
jbenet: thanks. At this point I randomly watch people do group meetings just to learn more about how to run them. It is something my advisor (and normal source of such education) is self admittedly bad at.
<jbenet>
krl: and lmk if you want to publish the webui or want me to do it. i dont mind either way.
<jbenet>
Blame: oh dont take cues from me-- i'm not good at it either.
<Blame>
I got about half of what you said about the DHT record spec. I'd love to see your draft when you are ready.
<jbenet>
Blame: btw, thanks for stackstream -- very cool thing. we should think about writing an implementation at some point :)
<krl>
jbenet: i could do it later, but when i get home in a few hours
<jbenet>
Blame: ok sounds good will tag you when i post the DHT record spec.
<jbenet>
krl: yep, no worries
<krl>
feel free to do it, but check out my last-minute fix that i found demoing it to some people here
<jbenet>
@everyone feel free to modify the etherpad, we'll move it to an issue in ~5 hrs
<krl>
i just got 5 people to install and play around :)
<Blame>
I've already started hacking on one
<jbenet>
krl: yay :)
<Blame>
I'm digging out my big-int stuff I did for nim
<jbenet>
krl: write down any hurdles they run into
<jbenet>
krl: UX is super important.
<wking>
jbenet: posted my issues for this week to the Etherpad
<jbenet>
wking: thanks!
<krl>
biggest hurdle was performance issues unfortunately
inconshreveable has quit [Ping timeout: 265 seconds]
<whyrusleeping>
krl: perormance in regards to what?
void___ has joined #ipfs
void has quit [Ping timeout: 264 seconds]
void___ is now known as void
<krl>
bitswap i'd say
<krl>
sharing a video was pretty slow, even with peers in local network
<whyrusleeping>
krl: cool, that should be better in 1218
<whyrusleeping>
Blame: i did that once. got my friends to take it, went better than expected
<whyrusleeping>
Blame: how goes work on urDHT?
inconshreveable has joined #ipfs
<whyrusleeping>
jbenet: what is your availability like for coding this week?
<jbenet>
not great-- i shouldn't take on more stuff than i've got
<Blame>
whyrusleeping: I'm haking out the python version right now
<whyrusleeping>
jbenet: alright
<Blame>
im hoping it have it essentially done by this weekend
<whyrusleeping>
Blame: nice! which implementation?
<jbenet>
i may sink in to fix things if there's some UX (or perf, which is UX) thing pressing for dtn talk
<whyrusleeping>
jbenet: alright, you have time to CR 1218 better?
<jbenet>
whyrusleeping: yes still on it. i'm trying out the branch next.
<Blame>
whyrusleeping: none of them really. I wanted it to be simple. each node is assigned a point on a 2d plane, is responsible for it's voronoi regions and connects to it's delaunay peers.
<jbenet>
whyrusleeping: we should have a test case with iptb
<Blame>
with a smattering of longer links
<whyrusleeping>
jbenet: like an iptb file transfer?
<jbenet>
whyrusleeping: generate a random hierarchy of files up to 100MB and then use iptb to get them.
notduncansmith has quit [Read error: Connection reset by peer]
<Blame>
so, sanity check: all you need for ipns is a DAG object stored at "ownerid" in the DHT that is signed by a public key matching the id?
<whyrusleeping>
Blame: theres a format for the object
<whyrusleeping>
but thats how ipns roughly works, yes
<Blame>
why would it not work when your node goes offline?
therealplato has quit [Ping timeout: 245 seconds]
<whyrusleeping>
it works when your node goes offline
<whyrusleeping>
until the DHT churns your records out that is
<Blame>
ok thats when I thought
<Blame>
is teh DHT churning un-supported records out a feature?
<Blame>
because that could be avoided
Guest11561 is now known as amiller
amiller has joined #ipfs
amiller has quit [Changing host]
<whyrusleeping>
Blame: yeah, its been thought about
<whyrusleeping>
basically have nodes rebroadcast as they die
<Blame>
well, you never know when they are going to die. so once every hour or two they re-broadcast
<whyrusleeping>
oh, that too
<whyrusleeping>
we could do that
<Blame>
you could even it it stochastically or spread out over time. Essentially, ask "given my peers, who would take this if I died" and store it there.
<Blame>
*do it
<Blame>
partition you records by their new owners and send a store.
<Blame>
no lookups
<Blame>
and you could rate limit, pick active records first (take your pick of equally bad caching stratagies)
<Blame>
at the end of the day, even with backups, every record has a half-life
<Blame>
unless you go to heroic means to preserve it (like the webui and a few other default pins right now)
<Blame>
huh, there is a useful measure that I need to start using in papers "record half-life"
<Blame>
a part of the issues is that we don't really have terms for discussing the robustness qualities of a distributed system
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
eyebloom has joined #ipfs
<whyrusleeping>
Blame: yeah, its hard to to talk about robustness on something with so few concrete guarantees
<Blame>
well the reality is that ANY computer system has a chance of failure, we just normally ignore it. When you are building massive decentralized systems, it suddenly becomes a more meaningful number.
<whyrusleeping>
yeah
<Blame>
and something like a "record-half-life" would let me compare topologies and backup/caching stratagies
<Blame>
at your current network size (or at least since I last checked) it would be as simple as periodically getting the list of active nodes
<Blame>
we could get measures out of that in post-analysis
<niran>
at the risk of a tangent, what was the reason for not using a blockchain for IPNS? permanent names that can be rewound to any point in time. the DHT approach is free, but for names that people want to live forever, will there be a layer on top where they can do that?
<gatesvp>
@jbenet: possibly, if there's another "easy" you want looked at you can throw it down as my "stretch goal" for the week
<Blame>
niran: honestly, other people are solving that one better. pass the buck
<jbenet>
niran: blockchain is a way to organize consensus around a dataset. IPNS avoids consensus entirely by having cryptographically assigned names.
<jbenet>
no need for a central authority.
<jbenet>
it's just a need to disseminate the records, but these don't even need to be accessible in one place.
notduncansmith has joined #ipfs
<jbenet>
you can have subnetworks with their own names that are never broadcast outside.
<whyrusleeping>
we may have a variant of ipns that can utilize a blockchain
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
down the road
<jbenet>
(right, as a "routing system"
<niran>
hm... good point. so if you wanted snapshots of names, someone could build a snapshot, publish the data on IPFS and share the link
<niran>
so if entries churned out of the DHT, anyone could put them back
<jbenet>
yep
<niran>
neat.
<Tv`>
wow the desire for s3 really ends up being a tangled web of many features
<jbenet>
(this is why validity is important-- you'll see in the record spec)
<ipfsbot>
go-ipfs/master 9c4135f Juan Batiz-Benet: Merge pull request #1240 from vijayee/master...
<lgierth>
ipn.io/join says If you are interested in joining us, but are not sure you fit any of these roles, or your timing is difficult, reach out anyway!
<jbenet>
bret: i scoped it out but i havent made iot
<bret>
i've only used childprocess for short running external commands
<bret>
when you say node here, you mean an ipfs node
<jbenet>
bret: sorry, let me clarify.
<jbenet>
bret: "https://github.com/ipfs/node-ipfsd-ctl" is a node/iojs module for controlling _any_ `ipfs` implementation (the goal is to make the bins have the same API). or at least go-ipfs.
<jbenet>
bret: yeah that's the idea :) -- would you mind describing how you would go about this in a note over at https://github.com/ipfs/notes/issues ?
<jbenet>
bret: then we can coordinate about doing it