<ipfsbot>
go-ipfs/tk/unixfs-ls ebdd948 W. Trevor King: core/commands/unixfs/ls: Explicitily record stat in LsObject...
inconshreveable has quit [Read error: Connection reset by peer]
tilgovi has quit [Ping timeout: 252 seconds]
dandroid has joined #ipfs
<jbenet>
spikebike: if we can figure out a good flow for self assembling the network, we can design a hack of a pretty cheap pi + HDD self-assembling netcaches
<cryptix>
jbenet: cat show similar behavior
<jbenet>
would just need a nice UI to show netcaches around, secure them on first interaction, and configure them as sinks for data.
<cryptix>
deaddrop2.0
<spikebike>
heh
<spikebike>
include a solar panel and wifi for each one ;-)
<jbenet>
spikebike: no joke. i've been wanting this for years.
<spikebike>
I'm sad to say p2p physical networks suck.. it's maddening that there's 100s of millions of smartphones around, yet for the most part they can't directly communicate
<cryptix>
spikebike: +9000 yes
<jbenet>
spikebike: if you know a lot about good solar panels + mesh nodes, come hang out with us a weekend in july and let's hack stuff together :)
<jbenet>
spikebike: yeah its really annoying. that's how the ISP want it-- nice and locked down so that all the bw goes through them.
<cryptix>
everytime i'm on the train/sub and everybody looks angry when the celular drops out, i'm crying a little
<spikebike>
jbenet: Me as well. I've been tracking mesh networks, DHTs and related for years. My current p2p backup widget is on it's 3rd language/platform.
<spikebike>
java = no file stat... really?
<spikebike>
python = closer, but the crypto libs are horrifying, the GiL is annoying as well
<Blame>
Its that inverse square radio strength that gets you.
<spikebike>
Got the furthest with go, been sweet so far
<jbenet>
Blame: yeah. but you;d be surprised how good the mesh networks have gotten.
<spikebike>
Blame: not so bad, especially if you are latency tolerant
<Blame>
I'm a fan of cjdns
<jbenet>
Blame: get your lab to fund a trip to Battlemesh + CCC this summer to "check on state of the art mesh networks for locality aware dhts" or something
<Blame>
But if you are latency tolerant, it is more power efficent to use moving people as a medium.
<spikebike>
most cities, university campuses, highways, etc have plenty of density for phone <-> phone meshes if everything wasn't so tower centric
<jbenet>
Blame: i'm going with lgierth and cjd and other friends
<spikebike>
I'm hoping wifi direct helps things
<spikebike>
(and becomes popular/standard on phones/tablets/watches)
<Blame>
jbenet: I already asked, no chance in hell unless I fund the try myself.
<jbenet>
lgierth: oh another possible reason the wifi at the cinema sucked: they probably had a microwave in there! >.<
<uhhyeahbret>
:[ my daemon keeps crashing after a number of hours
<jbenet>
uhhyeahbret what time, usually?
<lgierth>
jbenet: heh!
compleatang has joined #ipfs
<jbenet>
and yeah i keep mine managed with systemd initd dockerd or whatever
<jbenet>
so they reboot automatically.
<jbenet>
uhhyeahbret: also specs on machine? and are you on latest?
<spikebike>
I run 2, they've been stable for days
<uhhyeahbret>
jbenet: not sure exactly just did it a second time
<Blame>
seriously though, there is starting to be an interesting field of research around using people's movement as a method of moving packets.
<uhhyeahbret>
jbenet: its a raspi2
<jbenet>
uhhyeahbret: whyrusleeping has made a bunch of fixes/optimizing improvements in the last week
<spikebike>
Blame: yeah, especially for cars.
<Blame>
the necessary creepy is that you need to have a solid predication of people's future movements.
<uhhyeahbret>
jbenet: safe to just go get with the update flag?
<Blame>
VANets are going to get a lot more funding soon
<uhhyeahbret>
or do I need to nuke something
<jbenet>
Blame: yeah all of that is actually used by telcos for intra-network routing
<jbenet>
Blame: they use your cellphone to route packets a ton
<spikebike>
for many things latencies involved 100s of hops with help from human movement, cars, etc would be quite good... doubly so in the 3rd world
<jbenet>
Blame: but they wont let _you_ route them.
<jbenet>
Blame: cause you know, that wouldn't be fair at all.
<jbenet>
uhhyeahbret: yeah that should be safe. can also get a new bin from ipfs.io/docs/install
<Blame>
jbenet: I've not heard about that. Got places where I can read about it?
<uhhyeahbret>
are tehre crash logs somewhere?
<jbenet>
Blame: sorry dont remember where i read it, but probably can turn up somewhere. they're probably pretty secretive about this sort of thing
<jbenet>
uhhyeahbret: when the daemon crashes, it will usually dump out a stack trace
<spikebike>
jbenet: heard of CERNIO or iocoop.net? They are a tech co-op in the bay area that had some people/projects involved with building solar/wifi mesh networks. I think they did one in berkeley
<spikebike>
unfortunately my hardware triumphs so far was getting GPIO pins to make a buzzer buzz. ;-)
therealplato has quit [Quit: Leaving.]
<lgierth>
jbenet: since i only replied "yes!" to that mail, i had an idea about something practical for battlemesh. the cjdns people in freifunk leipzig want to get the local collective radio's archive on the mesh. a little website with ipns and ipfs within the battlemesh wifi would be a cool proof-of-concept
<lgierth>
plus everybody would immediately be sold
<kbala>
jbenet: ready?
<jbenet>
lgierth: yeah that'd be awesome
<lgierth>
i'll write it down somewhere
<jbenet>
lgierth: github.com/ipfs/notes maybe -- we can spec all the things we need there
<jbenet>
whyrusleeping: thoughts on a --disable-transport-encryption cli flag that just removes secio on the daemon? (it would be totally incompatible with default encrypted ones, and allow for easy debugging of the protocol)
<ipfsbot>
[go-ipfs] jbenet created fix-nil-conn-bug (+2 new commits): http://git.io/vLalK
<ipfsbot>
go-ipfs/fix-nil-conn-bug 6bbe575 Juan Batiz-Benet: update go-multiaddr-net with a nil check...
<ipfsbot>
go-ipfs/fix-nil-conn-bug 1363790 Juan Batiz-Benet: remove duplicate Godeps.json entry...
<ipfsbot>
[go-ipfs] jbenet opened pull request #1395: update go-multiaddr-net with a nil check (master...fix-nil-conn-bug) http://git.io/vLal1
therealplato has joined #ipfs
<whyrusleeping>
jbenet: flags that are longer than one word scare me
temet has quit [Ping timeout: 265 seconds]
hellertime has quit [Quit: Leaving.]
timgws has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<jbenet>
whyrusleeping: that's why it's long, to be very sure you really mean it
dread-alexandria has joined #ipfs
<alu>
jbenet: I hope an API becomes available for it, I want to talk to OpenBCI and having biometric inputs like respiration, pulse, EEG and influence the subjective matter of our sensory stimuli
<alu>
really wish I had a 360 camera so I can get into more augmented reality side of things.
pfraze has quit [Remote host closed the connection]
tilgovi has joined #ipfs
nessence has quit [Remote host closed the connection]
nessence has joined #ipfs
pfraze has joined #ipfs
nessence has quit [Ping timeout: 265 seconds]
patcon has quit [Ping timeout: 252 seconds]
dread-alexandria has quit [Quit: dread-alexandria]
temet has quit [Ping timeout: 250 seconds]
headbite has quit [Ping timeout: 245 seconds]
headbite has joined #ipfs
pfraze has quit [Remote host closed the connection]
williamcotton has quit [Ping timeout: 265 seconds]
Wallacoloo has joined #ipfs
temet has joined #ipfs
* alu
is still looking at the artificial neural networks research
<alu>
fucks sake jbenet
<jbenet>
alu: ?
<alu>
ITS SO COOL.
<alu>
I can't take my mind off it.
<alu>
this stuff that your friend worked on
tilgovi has quit [Quit: No Ping reply in 180 seconds.]
<alu>
its like looking at the computers dreams
<alu>
kinda like that research awhile ago where people were looking into how the brain sees images
tilgovi has joined #ipfs
sharky has quit [Ping timeout: 255 seconds]
<jbenet>
alu: not dreams yet, but we're close. This is a lot like looking at the representation of brain segments
<jbenet>
There's large brain structures that form similar styles of classifieds
<jbenet>
And yeah ML/AI is one of the most exciting areas.
<jbenet>
alu: it's not that hard to learn either, just a lot of knowledge to get down. Recommend the coursera and edx courses
<jbenet>
alu and I think Geoff Hinton made a couple of online classes posted on YouTube
<jbenet>
It's a lot of stats, linear algebra first
sharky has joined #ipfs
nessence has joined #ipfs
<alu>
>:)
<spikebike>
I did a fair bit of work helping to map the visual cortex
<jbenet>
alu: but seriously, you can set aside a few months to a year and by the end be at the cutting edge of ML research if you work hard
<spikebike>
google has a pretty similar approach.
* alu
laughs
<alu>
jbenet I can be at the cutting edge of anything if I work hard for a year
<alu>
thats what I decided for 2015 and VR
<spikebike>
jbenet: then he'll need as many cycles as google has ;-)
<alu>
but halfway into the year
<alu>
still no funding ;_;
<spikebike>
heh, I built a cluster for $1.5M CAVE
<alu>
I have a problem of communicating or publishing my work to get funding
<spikebike>
immersive 3d is surprisingly cool, I thought it would be a toy for VIPs
* alu
has never really had more than 10k to his name
<jbenet>
spikebike: don't need as much to repro some of the smaller results. And then can go play at Google / FB / etc
<alu>
I missed my chance with bitcoin
<alu>
I tried hard to convince people to listen to me
<alu>
should of just taken a loan
<nessence>
Where's best place (in code or otherwise) to find out more about network/query-like operations w/IPFS?
<nessence>
I can see how the DAG works. I'd like to learn what specific interfaces there are available for it over the network.
dread-alexandria has joined #ipfs
zabirauf has joined #ipfs
<jbenet>
nessence: what you get is to be able to pull pieces of the dag out from by requesting them individually (that's good enough for 80% of file transfer use cases) -- that's what the DagService uses the "Exchange" for. right now you can request hash by hash, but soon will be able to request using a query path thing (think XPath but for ipfs dags)
<ipfsbot>
[go-ipfs] wking created tk/public-name-commands (+1 new commit): http://git.io/vLwvW
<ipfsbot>
go-ipfs/tk/public-name-commands 40c6ffd W. Trevor King: core/commands: Make IpnsCmd and PublishCmd public...
<ipfsbot>
[go-ipfs] wking opened pull request #1397: core/commands: Make IpnsCmd and PublishCmd public (master...tk/public-name-commands) http://git.io/vLwvz
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Wallacoloo has quit [Quit: Leaving.]
kbala has quit [Quit: Connection closed for inactivity]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
martinBrown has quit [Quit: -]
<ipfsbot>
[go-ipfs] jbenet created disable-secio-option (+2 new commits): http://git.io/vLwqw
<ipfsbot>
go-ipfs/disable-secio-option 9618b6d Juan Batiz-Benet: daemon option to optionally disable secio...
<ipfsbot>
go-ipfs/disable-secio-option 756166a Juan Batiz-Benet: daemon output includes swarm addresses...
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot>
[go-ipfs] wking created tk/publish-local-peer-id (+1 new commit): http://git.io/vLwWW
<ipfsbot>
go-ipfs/tk/publish-local-peer-id e700c02 W. Trevor King: core/commands/publish: Allow explicit local node ID...
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
nessence_ has joined #ipfs
nessence_ has quit [Remote host closed the connection]
nessence_ has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
www has quit [Ping timeout: 264 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Blame>
daviddias: If you want/need a crash course on how a DHT works in general, I'm available.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
hellertime has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot>
[go-ipfs] jbenet force-pushed disable-secio-option from 756166a to b0ac49b: http://git.io/vLwHW
<ipfsbot>
go-ipfs/disable-secio-option a098cab Juan Batiz-Benet: t0060-daemon: test transport is encrypted...
<ipfsbot>
go-ipfs/disable-secio-option 7c2a38f Juan Batiz-Benet: ipfs swarm addrs local - show local addrs...
<ipfsbot>
go-ipfs/disable-secio-option 60a53fc Juan Batiz-Benet: ipfs id -f=<addrs> and \n \t...
www has joined #ipfs
<ipfsbot>
[go-ipfs] jbenet force-pushed disable-secio-option from b0ac49b to f0d3d19: http://git.io/vLwHW
<ipfsbot>
go-ipfs/disable-secio-option 62f3169 Juan Batiz-Benet: ipfs id -f=<addrs> and \n \t...
<ipfsbot>
go-ipfs/disable-secio-option 767b2b1 Juan Batiz-Benet: ipfs swarm addrs local - show local addrs...
<ipfsbot>
go-ipfs/disable-secio-option 0b2f1dd Juan Batiz-Benet: daemon output includes swarm addresses...
<ipfsbot>
[go-ipfs] jbenet created gpe-a098cabc40f2d5732b7fd99fa916ef09576bf6b2 at a098cab (+0 new commits): http://git.io/vLw5W
<ipfsbot>
[go-ipfs] jbenet created gpe-f0d3d19d9dcb6c793a251d1c7836229de08b8440 from disable-secio-option (+0 new commits): http://git.io/vLw5z
williamcotton has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Guest96 has joined #ipfs
therealplato has quit [Ping timeout: 264 seconds]
<daviddias>
Thanks Blame:) Currently I feel pretty confident algorithm wise when it comes to DHT and p2p in general as I've been studying them for quite a while. My current questions right now go more towards how it was coded and structured in the go-ipfs impl, so I can make a good node version that maps the same ideas and structure
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Encrypt has joined #ipfs
therealplato has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Blame>
That is actually a service I'll actively offer to anybody who wants it. I have a background in teaching and DHTs. So if you want to send people my way for such a lesson, feel free.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
temujin has joined #ipfs
Tv` has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
m0ns00n has quit [Remote host closed the connection]
pfraze has joined #ipfs
mildred has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
<daviddias>
do you work at an University Blame ?
notduncansmith has quit [Ping timeout: 256 seconds]
Encrypt has quit [Quit: Quitte]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
tilgovi has quit [Ping timeout: 276 seconds]
<Blame>
Yeah. I'm working on my phd
<whyrusleeping>
jbenet: i'm trying to avoid complicating things in go-peerstream
<whyrusleeping>
but i have a problem
<whyrusleeping>
theres a lock taken to 'set up' the connections, and the transport.NewConn is called inside the lock
domanic has quit [Ping timeout: 256 seconds]
<whyrusleeping>
this means that if a swarm is dialing itself, both sides of the connection cant create a connection at the same time
martinBrown has joined #ipfs
<whyrusleeping>
for me, this sucks because i was making the connections negotiate which transport to use
<whyrusleeping>
so i have two options, make the multistream transport do some weird 'deferred negotiation' or change the locking scheme
dread-alexandria has joined #ipfs
<Blame>
sounds like changing the locking scheme is wiser.
<whyrusleeping>
thats where i'm hacking right now
tilgovi has joined #ipfs
<whyrusleeping>
i wish go had a readers/writer lock that supported upgrading
<whyrusleeping>
so i could upgrade from the read lock that i'm holding to a write lock
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<JasonWoof>
daviddias: you know that there is a node impl started?
<JasonWoof>
daviddias: I think it's very incomplete, but just wanted to make sure you knew about it
<daviddias>
Blame nice! Which university?
<daviddias>
JasonWoof right now the node impl is pretty outdated, a lot has changed since it was last touched. But that's my focus now, so we should getting something new soon :)
<daviddias>
Right now the goal is to make sure both ( go and node ) can talk to each other, and that is through multistream and spdy
<JasonWoof>
daviddias: you going to update the existing node impl or start over?
<daviddias>
JasonWoof hybrid. There isn't really a way to tell of what's there that is usable or what it isn't, I'm "redoing" but at the same time, looking what was there, what is in the go-impl and also what is new and has to be added to both
<JasonWoof>
cool. I wish I could help right now
<JasonWoof>
I've got too much on my plate right now
<daviddias>
got it :)
<daviddias>
well, when you have time and if you want to help, ping me :)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
dread-alexandria has quit [Quit: dread-alexandria]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
dread-alexandria has joined #ipfs
patcon has joined #ipfs
<JasonWoof>
daviddias: noted! (in my notes file)
<cryptix>
suddenly: spare time \o/
nessence_ has quit [Ping timeout: 246 seconds]
nessence has quit [Ping timeout: 246 seconds]
tilgovi has quit [Ping timeout: 256 seconds]
nessence has joined #ipfs
nessence_ has joined #ipfs
mildred has joined #ipfs
notduncansmith has joined #ipfs
jb55 has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
inconshreveable has quit [Read error: Connection reset by peer]
dread-alexandria has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
dread-alexandria has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
tilgovi has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
dread-alexandria has quit [Ping timeout: 250 seconds]
dread-alexandria has joined #ipfs
<whyrusleeping>
cryptix: wooo!
williamcotton has quit [Ping timeout: 256 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
temet has quit [Ping timeout: 276 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
warner has quit [Read error: Connection reset by peer]
warner has joined #ipfs
temet has joined #ipfs
Wallacoloo has joined #ipfs
domanic has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has joined #ipfs
<ipfsbot>
[go-ipfs] whyrusleeping created feat/mss-mux (+1 new commit): http://git.io/vLKZ7
<ipfsbot>
go-ipfs/feat/mss-mux 5eadfac Jeromy: WIP: using multistream muxer...
<whyrusleeping>
sprint checkin, worked on go-multistream a bit more, added some tests and client code
<whyrusleeping>
wrote a basic multistream transport for go-peerstream, but something isnt quite working
<whyrusleeping>
and i also started implementing all of that into go-ipfs, some things work
<whyrusleeping>
some things dont
jb55_ has joined #ipfs
Encrypt has joined #ipfs
jb55 has quit [Ping timeout: 252 seconds]
<wking>
checkin: Pushed a number of minor PRs to go-ipfs and ipfs-shell last night (cc whyrusleeping)
<wking>
Docker-registry storage driver now almost works. Just have a registry error that I haven't figured out yet: unexpected error getting blob HTTP handler: unsupported method
<wking>
I'm also getting occasional IPFS daemon errors: http: Accept error: accept tcp4 127.0.0.1:5001: too many open files; retrying in ...
<wking>
But a Docker-image push with an ipfs-shell-based backend hits the IPFS API pretty heavily ;). I haven't looked at the headers yet, maybe we can persist the connection?
<wking>
whyrusleeping: ^ (for ipfs-shell PRs and persistent ipfs-shell connections)
<whyrusleeping>
hrm... persist the connection?
<whyrusleeping>
how do you mean?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
tilgovi has quit [Read error: Connection reset by peer]
tilgovi has joined #ipfs
<wking>
I expected the "too many open files" error was due to a cap on TCP connections to the API, so if all ipfs-shell requests go through a single TCP connection we won't have that problem
<wking>
but maybe that's not what's going on and I'm just misreading the error
<jbenet>
wking: interesting, that makes sense-- we may want to either try http2 (so muxing over one connection) or even putting it over a unix domain socket
<jbenet>
not sure which is easiest, there may be something to enable on the go http1 client
<whyrusleeping>
wking: interesting... i didnt know that was possible
<whyrusleeping>
wking: what calls is it making a ton of?
<whyrusleeping>
jbenet: i think it might be in our best interests for you to take over on the multistream integration
<cryptix>
i could use that as well for the broken PUT handler *caugh*
<jbenet>
wking: btw do let me know before PRing to docker with any ipfs stuff
<wking>
jbenet: sure
<jbenet>
wking: i want to review all of that beforehand because at that point it's project<-->project PR.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
jbenet: re 1266, sure thing.
<wking>
jbenet: The commit I just linked to is probably fairly close to what we'll end up with in case you want to glance over it before I've got through a full push/pull cycle ;)
<wking>
but I'll ping you again after I get through that push/pull cycle
<jbenet>
<wking>
and that commit depends on a few in-flight PRs on go-ipfs and ipfs-shell
<whyrusleeping>
wking: i'm thinking of adding a new subcommand to patch
<whyrusleeping>
something like 'add-path'
<whyrusleeping>
and it will let you specify a path and a hash
<whyrusleeping>
or maybe just change the functionality of add-link to do that
tilgovi has quit [Ping timeout: 264 seconds]
kbala has joined #ipfs
<wking>
whyrusleeping: to handle the bubbling?
<whyrusleeping>
yeah
<pjz>
where is add-link?
<whyrusleeping>
so ipfs patch <root> add-link a/b/c <hash>
<whyrusleeping>
would add <hash> at a/b/c under root
<wking>
pjz: 'ipfs object patch ...'
<whyrusleeping>
and return the modified hash of root
<pjz>
hmm
<whyrusleeping>
the question i have is, do we require a/b to exist already?
<wking>
whyrusleeping: works for me. I'd put it in 'ipfs file patch ...' though, since I want it to take Unix-FS paths (with e.g. implicit fanout traversal)
<pjz>
whyrusleeping: I think so
<whyrusleeping>
wking: does file patch exist?
<wking>
whyrusleeping: the addBubblePublish I linked doesn't require a/b to exist
<whyrusleeping>
okay
<wking>
whyrusleeping: no, bit 'ipfs file ...' is the place for stuff on the UnixFS layer
<pjz>
needs fuse support for ln
<whyrusleeping>
pjz: yeah, symlinks are something we need to get to
<wking>
*but
<whyrusleeping>
but its hard to do them in the context of merkledags
<pjz>
whyrusleeping: isn't that what add-link is?
<wking>
pjz: add link if for a Merkle-object's Links[] array
<pjz>
I figured: ln -s /ipfs/Qm.... /ipns/local/foo
<pjz>
if you want real fun, consider whether there's a difference between hard and soft links in IPFS
<wking>
pjz: there is. Hard-links are metadata-1 -> payload-1 and metadata-2 -> payload-1
<jbenet>
pjz: there is when you jump into ipns
<jbenet>
(but that's out of scope i guess)
<wking>
or dir-1 -> payload-1 and dir-2 -> payload-1
<wking>
but hard-links and bubbling updates is tricky
<wking>
since we don't have back-references
<pjz>
what's a bubbling update?
<wking>
say you create a new payload-1' Merkle object. To insert it into your Merkle DAG (really "create a new Merkle DAG using the new payload-1' object"), you update the dir-1 link
<wking>
Now you have dir-1' -> payload-1'
<pjz>
ah, right
<pjz>
..but that's what IPNS is for, no?
<wking>
Etc., etc., as far rootward as you like
<wking>
you could just do it in IPFS and keep the hashes on a piece of paper (poor-man's IPNS?)
<jbenet>
there's real uses, think of how a git updates its tree when you commit stuff
<wking>
but somebody has to bubble the changes up, and that's separate from publishing the new hash to IPNS
<jbenet>
and only rebinds the root
<pjz>
right, or you could do it out of IPFS and 'ipfs add -r' stuff
inconshreveable has quit [Read error: Connection reset by peer]
<wking>
yeah, that would do the same thing. But sometimes (often?) it's cheaper to not shift the whole tree in and out of the filesystem ;)
<pjz>
jbenet: but work on a git tree is done _out of tree_, in the checkout-space. commit is approx. like 'ipfs add -r'.
<wking>
whyrusleeping: I think <roo> should be optional
<wking>
*<root>
<wking>
so: ipfs file patch add-link a/b/c <hash>
<jbenet>
pjz: there are many operations one would do with ipfs that are like that. "ipfs add" is only one interface
<wking>
creates a new b' with a c link pointing at <hash>, and then creates a new a' with a b link pointing at b'
<wking>
and it keeps going until it hits a parent it doesn't understand
<pjz>
sure, my point is just that trying to make changes - even to an existing DAG - requires reference to a state that's _not_ in the DAG.
<whyrusleeping>
wking: i'm gonna veto that
<wking>
e.g. /ipfs/ or /ipns/<one-segment>
<wking>
then you can optionally add --root <root> to cut that chain short when you want to
<wking>
why veto?
<wking>
I think most cases are going to be: ipfs file patch /ipns/local/bla/bla/bla add-link QmNewLeafHash
<whyrusleeping>
making the <root> in patch optional? that makes things needlessly complicated
<wking>
and you'll want to bubble all the way back to the new value that you want to publish to /ipns/local
<wking>
why needlessly complicated? The default <root> is /ipfs or /ipns/<one-segment>. That seems well defined
<wking>
you'll only want to break off early for use cases that are optimizing a meta-bubble with many leaf changes
<jbenet>
wking i also dislike that interface. it's not "patch <root> add-link" at that point. this -- i think -- is a different subcmd.
<jbenet>
wking maybe "patch set <path> <hash>"
<jbenet>
and always requires an absolute path.
<wking>
you want to keep the add-link / remove-link distinction (and bubbling happens the same way in both cases)
<wking>
so I don't want to squash into 'set' (if I understand what you're suggesting?)
<wking>
what sort of non-absolute paths could this command possibly accept?
<wking>
Or is it: ipfs file patch <root> <relative-root-to-leaf> add-link <hash>
<whyrusleeping>
wking: yes. the path is relative to the root
therealplato has quit [Ping timeout: 252 seconds]
<wking>
hrm
<whyrusleeping>
just like the name in the current add link is relative to the root
<wking>
Do we use relative paths like that anywhere else in the UI (besides single-segment names)?
<wking>
ok, I'm coming around to the "just like 'ipfs object patch ..." but with multi-segment names" UI
<whyrusleeping>
wooo!
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
jbenet: ping
<whyrusleeping>
'http: Hijack is incompatible with use of CloseNotifier'
<jbenet>
ah
<jbenet>
lol
<whyrusleeping>
*sigh*
<whyrusleeping>
we need the closenotifier code...
<whyrusleeping>
i'll see if i can work around hijack
inconshreveable has joined #ipfs
<ipfsbot>
[go-ipfs] jbenet pushed 1 new commit to master: http://git.io/vLKdn
<ipfsbot>
go-ipfs/master 36a5281 Juan Batiz-Benet: Merge pull request #1395 from ipfs/fix-nil-conn-bug...
<ipfsbot>
[go-ipfs] whyrusleeping force-pushed close-notify from 85685bd to afd9d58: http://git.io/vLuMZ
<ipfsbot>
go-ipfs/close-notify 14bda02 Jeromy: cancel contexts if http connections closes...
<ipfsbot>
go-ipfs/close-notify 2e5329c Jeromy: select with context on closenotify signal...
<ipfsbot>
go-ipfs/close-notify afd9d58 Jeromy: hijack and closenotify dont work together...
Guest96 has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<whyrusleeping>
jbenet: use test_should_fail instead of !
<whyrusleeping>
jbenet: also, i'm using my desktop to run tests
<whyrusleeping>
travis and circleCI arent all that useful
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
tilgovi has joined #ipfs
Encrypt has quit [Quit: Quitte]
<jbenet>
cryptix: oh is it? i thought it was not, is that a bashism?
<jbenet>
whyrusleeping i could use test_should_fail but that would make me break it up into multiple tests no? (take a look at the test)
<jbenet>
cryptix: note that the one on https://github.com/ipfs/go-ipfs/pull/1399/files#diff-4e87752f38e9b707e89413267509dce0R103 works just fine. if you run the test, the problem is not with the `!`, it's with the fact that the `nc` doesnt output anything into the file. BUT if you add a long sleep there, you can try `nc` yourself to the daemon. (you'll want to do
<jbenet>
"echo $PORT_SWARM && go-sleep 10s" or something, so that you see the port number you need to "nc localhost $PORT_SWARM"
<jbenet>
but anyway, if you do, your manual netcat will get the right stuff.
<jbenet>
im not sure why.
dread-alexandria has quit [Ping timeout: 265 seconds]
therealplato has quit [Read error: Connection reset by peer]
dread-alexandria has joined #ipfs
therealplato has joined #ipfs
<whyrusleeping>
jbenet: you can use test_should_fail on its own
<whyrusleeping>
inside your test
<whyrusleeping>
so 'test_should_fail grep X Y'
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
dread-alexandr-1 has joined #ipfs
dread-alexandria has quit [Ping timeout: 264 seconds]
<wking>
^ with a trailing '&&' to keep it in the chain
<ipfsbot>
[go-ipfs] dylanPowers opened pull request #1400: "object" added to the descriptions of object patch and object data (master...object-help) http://git.io/vL6O3
<whyrusleeping>
jbenet: this http stuff is weird...
<whyrusleeping>
i'm getting different hashes for files if the progress bar shows
<whyrusleeping>
i didnt think the progress bar actually sent anything over the wire
<jbenet>
whyrusleeping: woah, different hashes of files? what do you mean? on add?
<whyrusleeping>
yeah, on the close-notify branch
<whyrusleeping>
ipfs add and ipfs add -q produce different hashes for the same file
<whyrusleeping>
(with the daemon running)
<wking>
what's different about the added objects?
therealplato has quit [Read error: Connection reset by peer]
<whyrusleeping>
idk, theyre large binary blobs right now
<whyrusleeping>
still testing
<whyrusleeping>
on small files they are the same
therealplato has joined #ipfs
williamcotton has quit [Ping timeout: 250 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
jbenet: it looks like its just getting truncated somehow...
* whyrusleeping
suspects an error somewhere is being hidden
<jbenet>
whyrusleeping: im pretty sure about my comments there
<jbenet>
whyrusleeping the request or response is ending early.
<whyrusleeping>
jbenet: well, when you were looking at it nothing was actually happening, it was erroring out because you cant call hijack and closenotify on the same connection
<wking>
It seems to work fine locally, but you probably won't be able to serve read-only registrys on other nodes off that IPNS path until we get validity comparison for records
<wking>
To run the IPFS storage backend locally, you'll also need go-ipfs#1348, go-ipfs#1397, go-ipfs#1398, ipfs-shell#4, and ipfs-shell#5. ipfs-shell#4 depends on go-ipfs#1348
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
patcon has quit [Ping timeout: 272 seconds]
nessence_ has quit [Read error: Connection reset by peer]
<cryptix>
i'm trying to fix the back links now following 1365 and then this should be a PR
<cryptix>
i could also mix in humanize to make 18381238812381923182398bytes understandable
tperson has quit [Read error: Connection reset by peer]
<spikebike>
prepare for religios wars debating 2^10 vs 1000
<spikebike>
religious
<cryptix>
hehe yea
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
okay, ive got it narrowed down to a single bool, if the bool is true, it breaks
<whyrusleeping>
it its false, things work
<whyrusleeping>
but i've disabled everything that the bool affects
<whyrusleeping>
so as far as i can tell, the bool does nothing
<cryptix>
now i'm interested
<cryptix>
whyrusleeping: this is add i presume?
<whyrusleeping>
yeah, in add
<whyrusleeping>
pull the close-notify branch
<whyrusleeping>
and on line 94 or 95, set progress to true or false
<whyrusleeping>
true, makes the hash output wrong
<whyrusleeping>
false, makes it right
<ipfsbot>
go-ipfs/nicerIndex 07b4e9d Henry: http/gateway: first tab at integrating @krl's new index page
<ipfsbot>
[go-ipfs] cryptix created nicerIndex (+1 new commit): http://git.io/vL6Ka
<whyrusleeping>
make sure you add a large enough file though
<cryptix>
large like 1g or 100meg?
<whyrusleeping>
100MB
<whyrusleeping>
or 50MB is probably fine too
<whyrusleeping>
add it on master so you know what hash to expect first
dread-alexandr-1 has quit [Quit: dread-alexandr-1]
<cryptix>
yup - confirmed
nessence has quit [Remote host closed the connection]
<whyrusleeping>
weird, eh?
nessence has joined #ipfs
<cryptix>
yea..
<cryptix>
actually.. all of that looks really convoluted.. the Run func with the go func() inside.. the progressReader{} ...
nemik has quit [Ping timeout: 244 seconds]
pfraze has quit [Read error: Connection reset by peer]
pfraze has joined #ipfs
nessence has quit [Ping timeout: 244 seconds]
<cryptix>
the custom printing is mostlikly used so that you can print the 'added $hash $name' lines without breaking the bar?
<whyrusleeping>
cryptix: probably?
<cryptix>
:)
<whyrusleeping>
it looks like the only thing that the progress variable changes is the 'progress reader' thing
<whyrusleeping>
and if you comment that out
<whyrusleeping>
it still fails
pfraze has quit [Remote host closed the connection]
<cryptix>
whyrusleeping: it fails with the bar disabled, too? oO
<whyrusleeping>
yeap
<whyrusleeping>
as far as i can tell
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix>
i'll check again but it looked like it only failed with the bar disabled
Aaron1011 has quit [Ping timeout: 252 seconds]
* whyrusleeping
needs to work on something else :/
<cryptix>
whyrusleeping: close-notify, with progress:=false hardcoded works afaict
<whyrusleeping>
cryptix: yeap
afdudley has quit [Ping timeout: 256 seconds]
<whyrusleeping>
but with it set to true, it breaks
afdudley has joined #ipfs
<whyrusleeping>
if you go into postrun and set 'show progress bar' to false, it still breaks
<cryptix>
okay, than i misunderstood. i thought it breaks in both cases
Aaron1011 has joined #ipfs
Aaron1011 has quit [Changing host]
Aaron1011 has joined #ipfs
<whyrusleeping>
cryptix: just curious, whats the 'wrong' hash you get?
Xe has quit [Ping timeout: 265 seconds]
jb55 has joined #ipfs
<cryptix>
trying to come up with a good test pattern to see the corruption.. so far i've just thrown 200megs of zeros at it - the size changes
shurcooL has quit [Ping timeout: 252 seconds]
jb55_ has quit [Ping timeout: 265 seconds]
<cryptix>
Qmf4fR63KvJS5eRbdUNj9F6WA2eNx3so6nodL3ZK1w41qS < some shell output comparing 'object get' of the two
Xe has joined #ipfs
jb55 has quit [Ping timeout: 252 seconds]
<cryptix>
msf had this tool to generate binary patterns to throw in your overflows to find memory offsets.. i wonder if there is something similar to this.. basically just a need a non repeating pattern
cmars has quit [Excess Flood]
notduncansmith has joined #ipfs
cmars has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
hellertime has joined #ipfs
<whyrusleeping>
cryptix: i lied.
<whyrusleeping>
it *is* the progress reader somehow
www1 has joined #ipfs
nessence has joined #ipfs
www has quit [Ping timeout: 276 seconds]
<cryptix>
the custom counting irked me a little already
<whyrusleeping>
custom counting?
<cryptix>
the completness book-keeping of the reader and the files.. might be correct but..
<cryptix>
i just threw a word list in it and there was a piece missing in between it
<whyrusleeping>
so a chunk just gets missed somehow?
<cryptix>
the progressReader sends addedObject's out even if there was an error on the file read.. i'm not sure if that's really what you want
dread-alexandria has joined #ipfs
<whyrusleeping>
maybe...
<whyrusleeping>
is an error happening?
sbruce has joined #ipfs
<cryptix>
not sure.. this beeing async i thought it might bubble up to late.. doesnt explain data missing in the middle though - add is sequential after all
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix>
there are also some unchecked type assertions
<cryptix>
sorry - i cant make heads or tails of this yet
<whyrusleeping>
yeah...
<whyrusleeping>
meh
<whyrusleeping>
i'm going to do something else and come back to it later