<whyrusleeping>
yeap, fails on go 1.5 because testing/quick changed to allow nil fields for nested struct pointers
<whyrusleeping>
and since some of those fields were listed as 'required' it failed
<daviddias>
jbenet: thanks, this is looking good
<daviddias>
now I know (or at least feel confident) of what I have to do
gordonb has quit [Quit: gordonb]
<daviddias>
should the serialization to CBOR be part of the IPLD implementation? or will be a separate thing that we just attach when it is time to send things through the wire?
<daviddias>
(like protobuf stream)
<jbenet>
daviddias: it's part of it, because we need to be able to deterministically compute a hash.
<jbenet>
daviddias: i think the cbor encoded version should be the "canonical form". others may exist, and people may use hashes to them (i.e. so we dont break links to protobuf things), but we should endeavor to make sure people use cbor to gen the hashes, so things dedup correclty.
patcon has quit [Ping timeout: 246 seconds]
ygrek has joined #ipfs
<daviddias>
Oh, good point, I was thinking in CBORing the whole object, not the value first, then calculate the hash and then the whole object
<jbenet>
i didnt understand that o/
voxelot has quit [Ping timeout: 260 seconds]
<daviddias>
I was thinking that a value would be always a bytearray
<daviddias>
If I understood well what you said, you meant that since the value of the object can be several things of several types
<daviddias>
We need to serialize it first, so we don't get different hashes when using different languages
<daviddias>
So, 1) we serialize the value, 2) calculate the hash, 3) serialize the object ( with value , hash, context and respective links, if any) 4) put it on the wire
<mg->
I haven't used golang gitrepo imports for anything but upstream stuff, how can I most easily build stuff so that it uses the code in my checkout? I.e. I want to build the ipfs command with my own core/commands/object.go
<whyrusleeping>
jbenet: i dont know if we can use quick anymore for testing
<whyrusleeping>
protobuf apparently cant handle nil elements in an array of things
<whyrusleeping>
im getting " repeated field foos has nil element"
<jbenet>
daviddias: im not understanding step 3
<daviddias>
jbenet when we have to transfer a DAG node from peer to peer
<jbenet>
from object in memory ---- serialize with multicodec (canonical: cbor) ---> Buffer. hash = multihash(Buffer). normally we just send the Buffer over the wire. the otherside calculates the hash always (to check)
<jbenet>
P1 ----"i want content that hashes to X"---> P2; P1 <--- "here's content C that hashes to X"--- P2; P1 assert(hash(C) == X)
<daviddias>
Got it :)
<daviddias>
I thought we sent the hash too and therefore the need to serialize again for the wire
<daviddias>
Thanks jbenet !
<jbenet>
ahhhhhh ok, yeah, we dont trust other people's hashes
<jbenet>
daviddias:
<jbenet>
whyrusleeping: bummer re: quick :(. other ways?
<xelra>
Just a quick question. Is it already possible to make my personal syncing space with the current ipfs, only accessible by me?
<xelra>
Not participating in the global namespace?
<ipfsbot>
[go-ipfs] jbenet pushed 1 new commit to master: http://git.io/vsjYh
<ipfsbot>
go-ipfs/master 7abebb1 Juan Benet: Merge pull request #1611 from ipfs/quick-fix/ipns-err...
gordonb has joined #ipfs
<whyrusleeping>
jbenet: well... i'm not sure
<whyrusleeping>
we could post process in this one case to remove all nil elements from the generated struct
<whyrusleeping>
jbenet: i pushed a fix
<jbenet>
xelra: yeah, with address filters. we should probably add a key-check (a way of adding a key-check to the handshake and ensure only a cluster of trusted nodes connect)
<jbenet>
and an important one to fix, particularly if it's in refs, or pin.
<whyrusleeping>
yeah
<whyrusleeping>
i wish github was better at organizing tasks for you
<jbenet>
right!?
<whyrusleeping>
it seems like something they could do so well
<whyrusleeping>
i would pay good money for it too
<mg->
I did some digging into that json-encoding issue, can someone that knows more about the encoding of the data than I do have a look at the comment I wrote in the issue?
<jbenet>
whyrusleeping: oh right we haven't pulled in the new thing. here, why dont we pull it in optimistically, and if we cant use it we remove it. we wont ship it anyway for a while. just massaging code.
<jbenet>
(whyrusleeping the author hasnt responded, right?)
<whyrusleeping>
author hasnt responded, correct
<whyrusleeping>
and yeah, i'm fine just pulling it and removing later if we have to
<jbenet>
whyrusleeping: ok wanna pr it up?
<jbenet>
mg- thanks for looking into it btw
<mg->
jbenet: I find that looking into bugs is the best way to learn a code base, so no worries. I guess the correct fix is to proto.Unmarshal the data that goes into the JSON/XML then?
<jbenet>
mg- this may change soon once we land the IPLD changes. unixfs wont need to be a protobuf anymore
<jbenet>
though we'll have to keep it to handle older files
<mg->
IPLD?
therealplato has quit [Quit: Leaving.]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
montecarlo has joined #ipfs
<whyrusleeping>
jbenet: PR it into multicodec?
<jbenet>
whyrusleeping: yeah dont we have to change go-multicodec/cbor ?
<jbenet>
to import your codec?
<whyrusleeping>
right
montecarlo has quit [Quit: leaving]
ygrek has joined #ipfs
* zignig
does a happydance.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ygrek has quit [Ping timeout: 255 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
gordonb has quit [Quit: gordonb]
<jbenet>
whyrusleeping: cbor pr?
ygrek has joined #ipfs
Eudaimonstro has joined #ipfs
amstocker has joined #ipfs
domanic has quit [Ping timeout: 246 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
I haven't used the json-ld module, so not sure about the quirks there
<pjz>
daviddias: so, like, stupid question, perhaps, but have you tried inlining ipld.context.merkleweb in that test file, just to make sure there's no issue pulling it in correctly?
<jbenet>
whyrusleeping: it diverged, mind fixing and i'll play with it locally? (would be good if you had a short example of how to use it)
<whyrusleeping>
jbenet: sure
Encrypt has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
gordonb has joined #ipfs
<daviddias>
krl check be npm-release module. It's simply but does the release nicely
therealplato1 has joined #ipfs
Eudaimonstro has joined #ipfs
therealplato has quit [Ping timeout: 246 seconds]
mkarrer_ has quit [Ping timeout: 260 seconds]
mkarrer has joined #ipfs
akhavr1 has joined #ipfs
akhavr has quit [Ping timeout: 260 seconds]
therealplato1 has quit [Read error: Connection reset by peer]
therealplato has joined #ipfs
akhavr1 has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
ygrek has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
gordonb has quit [Quit: gordonb]
marianoguerra has quit [Quit: leaving]
akhavr has joined #ipfs
Eudaimonstro has quit [Ping timeout: 244 seconds]
akhavr1 has joined #ipfs
akhavr has quit [Ping timeout: 252 seconds]
gordonb has joined #ipfs
silotis_ has quit [Quit: No Ping reply in 180 seconds.]
silotis has joined #ipfs
Eudaimonstro has joined #ipfs
akhavr1 has quit [Quit: akhavr1]
mildred has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Remote host closed the connection]
akhavr has joined #ipfs
Encrypt has quit [Quit: Eating time!]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
akhavr1 has joined #ipfs
ladekjaer has quit [Quit: Connection closed for inactivity]
<akhavr1>
ipfs add -r hangs for me, how I can debug it?
<akhavr1>
ipfs add -r go-ipfs doesn't complete, even if I wait for several days :)
akhavr has quit [Ping timeout: 260 seconds]
akhavr1 is now known as akhavr
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
akhavr: yeah... this is a known issue. It *should* error though.
<whyrusleeping>
the issue is that it cant handle broken symlinks
<akhavr>
Well, it doesn't error for me. Is there a github issue?
inconshreveable has joined #ipfs
<whyrusleeping>
uhm... let me check
<whyrusleeping>
jbenet: i think we had a regression on the mock_notifications test
patcon has joined #ipfs
mildred has quit [Ping timeout: 244 seconds]
patcon has quit [Ping timeout: 244 seconds]
mkarrer_ has joined #ipfs
<lgierth>
davidar: how did you do it in the end? the best would be if you just run your own ipfs node there
mkarrer has quit [Ping timeout: 264 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
dignifiedquire has quit [Quit: dignifiedquire]
pfraze has quit [Remote host closed the connection]
gordonb has quit [Quit: gordonb]
gordonb has joined #ipfs
mildred has joined #ipfs
chriscool has quit [Read error: No route to host]
chriscool has joined #ipfs
pfraze has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ianopolous has joined #ipfs
gordonb has quit [Quit: gordonb]
domanic has joined #ipfs
<Bat`O>
isn't 'ipfs repo gc' supposed to clear the datastore if i have no pin ? (ipfs pin ls return nothing)
akhavr has quit [Ping timeout: 244 seconds]
chriscool has quit [Ping timeout: 244 seconds]
akhavr has joined #ipfs
gordonb has joined #ipfs
gordonb has quit [Client Quit]
<whyrusleeping>
Bat`O: there are multiple types of pins, 'ipfs pin ls' will just show direct pins (which are actually rarely used)
<whyrusleeping>
'ipfs pin ls -r' shows recursive pins
notduncansmith has joined #ipfs
domanic has quit [Ping timeout: 250 seconds]
notduncansmith has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
akhavr has quit [Ping timeout: 265 seconds]
<pjz>
I just did an 'ipfs add' a few minutes ago, and now a do 'ipfs pin ls' and nothing shows up... is that right?
gordonb has joined #ipfs
<pjz>
how do I see what blocks I have in my store?
<whyrusleeping>
pjz: 'ipfs pin ls -r'
<pjz>
Error: Unrecognized option 'r'
<whyrusleeping>
jbenet: i keep saying this, but we really should have recursive pins be the default
<whyrusleeping>
oh, right: ipfs pin ls --type recursive
<whyrusleeping>
its even more obnoxious >.>
akhavr has joined #ipfs
<pjz>
whyrusleeping: just one small pull request away... ?
tjgillies__ has joined #ipfs
<whyrusleeping>
pjz: not really, its a bit more than that
<whyrusleeping>
its about semantics
<pjz>
how is changing default behavior about semantics?
<pjz>
I mean, the default for 'pin add' is recursive, right?
<pjz>
what is the default type? 'direct' ?
* pjz
reads code
<pjz>
oh, direct. okay.
<pjz>
buuuut.... then it should show recursive too
<pjz>
b/c direct = 'that specific object' and recursive = 'that specific object + indirect descendants'
<pjz>
and, actually... given that the trees don't mutate once stored, why does 'recursive' even exist as a 'type'? It seems to me to be a way of adding direct/indirect pins, not a kind of pin.
<Bat`O>
whyrusleeping: thanks
amstocker has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Bat`O>
pjz: are you sure direct is the default for ipfs add ?
<Bat`O>
i have a lot of recursive pin that i only can explain by ipfs add i did
<whyrusleeping>
the default for 'ipfs pin add' is direct
<whyrusleeping>
but when adding files, they are broken into chunks
<whyrusleeping>
so we pin the root recursively so those chunks stick around
<Bat`O>
whyrusleeping: so, direct pin if the data is small enough, recursive otherwise ?
<Bat`O>
it's especially ugly for now, but it start to look like something :)
<whyrusleeping>
Bat`O: personally, i would just recursive pin all the time, unless youre trying to do something weird
<whyrusleeping>
directly pinning in the wrong scenario might result in you losing data
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Bat`O>
whyrusleeping: my question was about what actually happen in ipfs, because you said that ipfs add use direct pin, but i have only recursive pin in my datastore
<whyrusleeping>
ipfs add uses recursive pin
<whyrusleeping>
'ipfs pin add' (note, its the 'pin' command, not the 'add' command) uses direct by default
<Bat`O>
ho !
<Bat`O>
ok, understood :)
<whyrusleeping>
woo!
<Bat`O>
.. re-reading the conversation, i shall read more carefully :-\
<whyrusleeping>
lol
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<drathir>
there will be update and upgrade commands? somethin which only will inform if updates are available w/o download them?
<lgierth>
jbenet: ping, i have something on a usb stick for ye
<whyrusleeping>
jbenet: krl daviddias what is the current status of the webui?
<whyrusleeping>
could we push for a new release soon?
<daviddias>
whyrusleeping: there is a new version that includes the working version of add -r on the ipfs-api
<daviddias>
There is also a new version with iframes, which will be changed to webcomponents, but we did agree that we could use the iframe version meanwhile
pfraze has quit [Remote host closed the connection]
<daviddias>
I can point you at the right commits, let me just get home ( coming from Capoeira)
<lgierth>
whyrusleeping: interesting discussions regarding search/indexing, tor integration, ipns
<lgierth>
very few people, max 10 or so, but that didn't hurt at all
<whyrusleeping>
daviddias: no worries, i'm just trying to get a feel for the current state of things
gordonb has quit [Quit: gordonb]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has joined #ipfs
inconshreveable has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
gordonb has joined #ipfs
mildred has quit [Ping timeout: 245 seconds]
zml has joined #ipfs
ygrek has joined #ipfs
notduncansmith has joined #ipfs
<drathir>
daviddias: how the tooth?
notduncansmith has quit [Read error: Connection reset by peer]