<ipfsbot>
[go-ipfs] chriscool created binary-data-and-json (+2 new commits): http://git.io/vswWm
<ipfsbot>
go-ipfs/binary-data-and-json a5beacb Christian Couder: t0051: add UTF-8 test file...
<ipfsbot>
go-ipfs/binary-data-and-json 7fd2123 Christian Couder: t0051: check json i/o...
<ipfsbot>
[go-ipfs] chriscool opened pull request #1599: [DO NOT MERGE] Binary data and json (master...binary-data-and-json) http://git.io/vswWl
zabirauf has quit [Ping timeout: 255 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
patcon has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
chriscool has quit [Ping timeout: 246 seconds]
<M-davidar>
mg__: yay, another ML'er :)
<M-davidar>
jbenet: sorry late to the party, but re that discussion several hours ago, would it be possible to have the hypothetical ipfsd work offline too?
<M-davidar>
That way you could also use the API for offline stuff
<jbenet>
M-davidar: yeah it should definitely do that
<M-davidar>
jbenet: in that case, +1 for splitting ipfsd into a separate binary
<M-davidar>
also would mean the go client isn't privileged over the other language bindings :)
<ipfsbot>
go-ipfs/master 11a66b3 Juan Benet: Merge pull request #1558 from rht/cleanup-get...
<ipfsbot>
go-ipfs/master 9f0c813 rht: Decompose DagArchive from unixfs tar...
<ipfsbot>
[go-ipfs] jbenet closed pull request #1569: Notify when daemon is ready (master...daemon-msg) http://git.io/v3VLP
qqueue has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<M-davidar>
jbenet: lgierth: so, arxiv's PDFs are in a bunch of big tarballs on s3 (~500GB total), but only about 2% of those are licensed to allow redistribution, any thoughts on the best way to get them?
<jbenet>
M-davidar: probably wget. i'll add your key to sirius
<M-davidar>
jbenet: so, just pull it all and throw out what we can't use?
<jbenet>
M-davidar: if we can't discern beforehand, probably?
<M-davidar>
I know which articles they are, but they're likely distributed across most of the tarballs
patcon has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
inconshreveable has quit [Ping timeout: 244 seconds]
domanic has quit [Ping timeout: 246 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
simonv3 has quit [Quit: Connection closed for inactivity]
Eudaimonstro has quit [Ping timeout: 246 seconds]
bedeho_ has quit [Ping timeout: 250 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
sharky has quit [Ping timeout: 250 seconds]
sharky has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has quit [Remote host closed the connection]
inconshreveable has joined #ipfs
amstocker has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
besenwesen has quit [Quit: ☠]
besenwesen has joined #ipfs
besenwesen has quit [Changing host]
besenwesen has joined #ipfs
patcon has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Leer10 has joined #ipfs
bedeho_ has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
amstocker_ has joined #ipfs
amstocker has quit [Ping timeout: 265 seconds]
<prosodyContext>
M-davidar: Would Aaron Swartz have used IPFS or er Gittorrent, given that limfitting factor? Sad to cee self-censoring.
inconshreveable has quit [Remote host closed the connection]
<M-davidar>
I'm very much in favour of open-access, and I admire Aaron's motivations, but I don't think violating copyright is the right way to go about it
<M-davidar>
yes, there's problems with the current copyright model, which I think really need to be fixed, but you can't just go throwing out the baby with the bathwater
<M-davidar>
there's a difference between self-censoring and respecting other people's rights
<M-davidar>
/rant
<prosodyContext>
Um, it's an honest question. 500GB of important data that Aaron Swartz was suicidal for not being able to repeat, and they were treated like a troll and now after death are idolized. :/ I said 1 line, thank you for the quick instant hate. ∫me runs away to quiet again.
<prosodyContext>
I don't know you, but you know instantly my 3 usernames? Interesting.
amstocker_ has quit [Ping timeout: 265 seconds]
<M-davidar>
prosodyContext: given that you were having a conversation with yourself the other day between them, it's not too hard to figure out
<prosodyContext>
I do not talk to myself, I don't know what you're referring to. But I'm amazed you are so confident in your hate speech.
<prosodyContext>
(Otherwise what is Gittorrent for?)
chriscool has quit [Ping timeout: 244 seconds]
<M-davidar>
look, you're obviously looking to provoke someone into an argument (it's not the first time I've seen you do it), so I'm not going to buy into it
<prosodyContext>
Your first line is calling me names, I asked about diff between IPFS and Gittorrent for sensitivee data, you could have saved your hate and just said a simple answer.
<M-davidar>
ok, I apologise for saying the word "troll" (and no, I don't hate you), but please don't phrase your questions in a provocative manner
<prosodyContext>
Um, they call that ableism and tone-policing in ##feminism, im sure they'd ban u back. ;) Thank you for apologizing. I'll end now, I clearly triggered you long before asking an honest question still unanswered here.
okket has quit [Remote host closed the connection]
okket has joined #ipfs
patcon has quit [Read error: Connection reset by peer]
Encrypt has joined #ipfs
<cryptix>
hellow
keroberos has quit [Max SendQ exceeded]
<cryptix>
Luzifer: re build failure. i've seen that before - 1.5 doesnt allow imports of main pkgs. we can't use the 'sadhack' package anymore to trick godeps to import stuff for the sharness helpers (cc jbenet)
<Luzifer>
ahhhh
<cryptix>
i just dont get why sadhack is imported to build cmd/ipfs
<cryptix>
what is the cmd ran after 'Found Godeps. Restoring them' ?
<cryptix>
Luzifer: godep restore doesn't fail - its from 'go fmt ./...'
<cryptix>
Luzifer: we don't import sadhack to build cmd/ipfs
<Luzifer>
oh…
<cryptix>
i like the sanity test fo go fmt, though
<Luzifer>
hm yeah sometimes `set -x` would be useful to see what's going on but mostly its cluttering the logs… :/
<cryptix>
yea i belive that :)
<cryptix>
jbenet: imho we should drop the main helpers from utils/sadhack and just pull a specific version from gobuilder or pin them on ipfs?
dignifiedquire has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
akhavr1 has quit [Read error: Connection reset by peer]
ei-slackbot-ipfs has quit [Remote host closed the connection]
dignifiedquire has quit [Quit: dignifiedquire]
ei-slackbot-ipfs has joined #ipfs
Encrypt has quit [Quit: Quitte]
akhavr has joined #ipfs
dignifiedquire has joined #ipfs
dignifiedquire has quit [Quit: dignifiedquire]
<M-davidar>
ouch, disk i/o on sirius
<M-davidar>
ETA 110h
klarmann has quit [Quit: Connection closed for inactivity]
<M-davidar>
is it possible to ipfs add directly from a tar archive?
* daviddias
wonders what we've decided on Relay stuff
<daviddias>
whyrusleeping: when you can, can you hit me up and explain me how relay is being built into go-ipfs?
G-Ray has joined #ipfs
<daviddias>
Not 100% how it is at the moment, but I had one idea of having 'relay records' similar to provider records, but where a Peer publishes which peers are accessible to him and which of those have been around for a longer period
chriscool has joined #ipfs
<dawuud>
ugh
<dawuud>
anonymouse
chriscool has quit [Ping timeout: 244 seconds]
<daviddias>
jbenet: do you have multicodec pushed?
G-Ray has quit [Quit: Konversation terminated!]
G-Ray has joined #ipfs
<G-Ray>
If I pin a block from an object, people requesting this object can download the block I store ?
richardsmith has joined #ipfs
G-Ray has quit [Quit: Konversation terminated!]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
richardsmith has left #ipfs [#ipfs]
<daviddias>
jbenet: around?
<drathir>
mornin...
<drathir>
bret: thanks a lot, checkin...
<drathir>
and pinning is some kind of bookmarking?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
vitzli has joined #ipfs
<vitzli>
hi, here is the page http://www.archiveteam.org/index.php?title=INTERNETARCHIVE.BAK/ipfs_implementation - as far as I understand IA.BAK project uses git-annex now, but if I would write a description for that kind of storage system based on IPFS, could somebody fact-check me?
<cryptix>
vitzli: somebody did ipfs for git-annex iirc
notduncansmith has quit [Read error: Connection reset by peer]
Encrypt has joined #ipfs
JasonWoof has joined #ipfs
pfraze has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<drathir>
which one port is used for peering?
<cryptix>
4001 by default
<drathir>
cryptix: thanks lets see what happen if i open it at vps...
<drathir>
cryptix: btw it have hardcoded some ips to peer wihth right?
<cryptix>
drathir: not hardcoded but .ipfs/config has some bootstrap peers by default after init
<cryptix>
'semi hardcoded' :)
<dvn>
any docs on node (server) configuration?
<drathir>
cryptix: thats good move than ++
<cryptix>
dvn: are you looking for something specific? not much more than whats in .ipfs/config
<cryptix>
drathir: the default list could be a bit more diverse :) i've been to networks where 4001 was just blacklisted
<dvn>
okay, figured that was the case. Essentialy wanting to peer two systems only with each-other
<cryptix>
dvn: you need to be very careful - there is currently no option to enable restrictions
<cryptix>
if somebody knows your ip and peerid they can still dial you
<cryptix>
i purposed something like ssh authorized_keys but its not in yet
<drathir>
looks like workin ^^
<cryptix>
drathir: :) nice
<drathir>
cryptix: need pass some time i discover "all" how thats workin...
<dvn>
cryptix, good to know. I was wondering if that was the case... Wanting to experiment with using over a vpn, so if I can bind ipfs to only the vpn ip, it should be okay
<dvn>
okay = theoretically secure
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<drathir>
dvn: in theory creating separate user could be good idea too...
<drathir>
even if somethin happen only that user rights left for attacker...
<mg__>
has refs-to-seed been updated with the correct refs yet?
G-Ray has joined #ipfs
<drathir>
is planned traffic shapeing?
<drathir>
-e*
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<drathir>
dvn: make cjdns beter...
<drathir>
dvn: link with both nodes i guess better than vpn...
<dvn>
drathir: yea, using cjdns as the vpn
<dvn>
works just fine so far
mg__ is now known as mg-
substack has quit [Ping timeout: 255 seconds]
<jbenet>
daviddias will be around in ~30
bedeho_ has joined #ipfs
substack has joined #ipfs
pfraze has quit [Remote host closed the connection]
<cryptix>
Jbenet: if you need some CR for the 0.4 push, ping me - I have some time on my hand next week
<jbenet>
You can just filter all connections from IPs not within your vpn
<jbenet>
Thanks cryptix
<jbenet>
dvn o/
<drathir>
dvn: using for long cjdns at 2 vps and all home devices not problem at all (not counting once problem with config, but was changed a little generated new and movrd data helped)...
<daviddias>
have to leave now, but I'll be back later in the evening
G-Ray has quit [Quit: Konversation terminated!]
G-Ray has joined #ipfs
elwisp has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
voxelot has joined #ipfs
ladd has joined #ipfs
ladd has quit [Remote host closed the connection]
atrapado has joined #ipfs
vitzli has quit [Quit: Leaving]
Tv` has joined #ipfs
amstocker has joined #ipfs
caseorga_ has joined #ipfs
caseorga_ has quit [Remote host closed the connection]
caseorga_ has joined #ipfs
Encrypt has quit [Quit: Quitte]
G-Ray has quit [Ping timeout: 240 seconds]
<jbenet>
p/
<jbenet>
daviddias: sorry for delay. am back.
therealplato has joined #ipfs
<jbenet>
cryptix Luzifer: wait, what broke with 1.5?
<jbenet>
oh what-- hah. ffs. the program-lib distinction is a sad decision. a program and a package should only differ by the existence of a `func main() {}`
<jbenet>
ideas on how to fix it?
<Luzifer>
cryptix pointed out something some hours ago…
<jbenet>
i dont think grabbing the binaries only is a good idea. many platforms arent supported yet.
<jbenet>
it's "just go" we should be able to include "just go" binaries that are not used to build the main cmd/ipfs
<jbenet>
we'll just have to change the import graph
<jbenet>
(and maybe ditch godeps once and for all)
<jbenet>
M-davidar: yeah, you'll be able to add from a tarball soon-- it's a feature WIP
<jbenet>
M-davidar: oh writing on sirius? did you get the tarballs? (btw, we can try adding all the contents to an ipfs node that's _not_ connected to the network (add filters, use it offline, etc.) to see what dedup we get. one of the reasons some of these groups might adopt ipfs is (a) huge reduction in space, (b) cryptographic integrity checks, (c) distribution of
<jbenet>
encrypted archives (which are a copyright compatible way to get replication-- turns into a key management problem)
kord has joined #ipfs
pfraze has joined #ipfs
caseorga_ has quit [Remote host closed the connection]
Leer10 has quit [Remote host closed the connection]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
caseorga_ has joined #ipfs
G-Ray has quit [Ping timeout: 272 seconds]
<jbenet>
krl: drop a link here to starlog when you get a chance?
caseorga_ has quit [Remote host closed the connection]
caseorga_ has joined #ipfs
* daviddias
arrives home
<daviddias>
jbenet: re: thought we would be using a polished multistream spec for multicodec and multistream-select for multistream (encoded stream negotiation)
<daviddias>
jbenet: re: relay - How does a peer know what is the best peer to ask to relay a connection?
zabirauf has joined #ipfs
<whyrusleeping>
apparently theres a bounty on us implementing symlinks now
<whyrusleeping>
lol
<jbenet>
daviddias: yeah we can do that. go-multicodec basically pulled all of whyrusleeping/go-multistream
<jbenet>
--woah **type** of the devil. even before enter
<daviddias>
Oh, I see. What I was thinking is having, similar to 'block records', have a 'relay records' were each peer keeps providing to a given key (e.g hash(relay+peerId)) which are the peers it knows best (for long period of time and with lower latencies), so that others nodes can use those frontier peers as relays
<daviddias>
we can go through it in tomorrows discussion, I guess :)
<jbenet>
yeah, let's. sounds like an interesting proto. (the dht case was merely to leverage existing connections which may be correlated with "good peering" if the dht starts taking latency into account, etc). i think ultimately, (long term, not now) we want a packet switched internet on top. it's the "data packets vs voice streams" over again, and paul baran
<jbenet>
already taught us what to do many decades ago. (it's just harder!)
<daviddias>
:D
atrapado has quit [Quit: Leaving]
pfraze has quit [Remote host closed the connection]
zabirauf has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
amstocker has joined #ipfs
<amstocker>
Hi all
<amstocker>
I was wondering if someone could help me understand ipns; I get that the point of the self-certifying path name is so that you don't have to pass public keys around, but how do you actually check that the signature matches without the public key?
<amstocker>
I guess im not too hot on how public key cryptography works
<whyrusleeping>
amstocker: the public key can be requested through the dht by the hash in the path
<amstocker>
oh ok
<amstocker>
so when you lookup a certain ipns url, it certifies itself and then through the dht it fetches the actual ipfs object that the url points to?
<jbenet>
amstocker: you look up the value of the ipns name in the dht. and you know it's correct because it carries a signature, and by what key? the key that hashes to the name.
<whyrusleeping>
yeap!
<jbenet>
amstocker: think about it like DNS but where there's no central authorities at all
<jbenet>
amstocker: (and thats also why it isn't a pretty name like in DNS or Namecoin)
<amstocker>
ok that makes sense, thanks
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<amstocker>
and the proper api for this is > ipfs name publish/resolve ?
<jbenet>
amstocker: yep.
<amstocker>
ok cool
<jbenet>
amstocker: it's still very alpha though, beware
<amstocker>
haha no worries I understand
<amstocker>
I'm just wondering because I was thinking of working on a simple web forum built on IPFS, and I realized that you would need some sort of head pointer that is mutable
<jbenet>
amstocker: yep! and that's what IPNS is for :)
<amstocker>
which points to the newest post or something
<amstocker>
ok great
<jbenet>
amstocker: think of IPNS like a git branch
<jbenet>
(asmtocker: also-- you currently can use DNS too for the same purpose, but DNS is much slower)
<amstocker>
ok, i honestly have no clue how DNS works anyways so im fine not using it
<jbenet>
(amstocker: or even a single host on the network, for now)
<amstocker>
but yeah my plan was to write a simple node app and then host it on my server
<jbenet>
if you build this, we'd love to help you get there. there's a lot we can improve it would help us find the holes
<jbenet>
you could deliver the mutable head that way
<amstocker>
thats awesome! thanks for the encouragement
<amstocker>
yeah the node app would just act as a gateway
<amstocker>
to fetch the head or submit a post
<jbenet>
http://<host>/latest just gives you an /ipfs/<hash> or something
<jbenet>
yep, exactly
<jbenet>
+1
<M-hash>
jbenet, where should be my mostest favoritest place to look for ipns docs right now? i don't see a ton of links from ipfs.io leading out to it
<mg->
jbenet: there's no in-band way to *send* information through ipfs right? So something like a forum needs to do (at least) POSTing of new threads oob?
<M-hash>
i have a very similar "resolve name -> hash pin" situation in mind and am wondering how much i can jump on your bandwagon / reuse stuff
inconshreveable has quit [Ping timeout: 244 seconds]
<jbenet>
mg- ipns works. the dht works (you'll be able to add records too). we'll have pub sub eventually, and you can mount your own protocol on top of our streams
<jbenet>
so there's too many ways.)
<jbenet>
M-hash paper, code, github issues?
<mg->
jbenet: neat!
<voxelot>
can haz whitepaper?
<jbenet>
ipfs.io has a link
<Bat`O>
humm, in a command, the marshaler cmds.Text is used only for the CLI, right ?
<jbenet>
Bat`O right
<zorun>
jbenet: the main difficulty with ipns is that objects are not cacheable, right?
<Bat`O>
\o/
<zorun>
because records are mutable
<Bat`O>
somehow, golang start to fit in my brain
<jbenet>
right
<Bat`O>
jbenet: and for the http api, it's the mashaller defined in response.go:54 that is used ?
<zorun>
so, either you try to cache things a bit anyway (maybe controlled by the publisher, a bit like DNS TTL), or you need a high-performance DHT (or any "routing engine")
<voxelot>
great paper, good way to spend my sunday, thanks!
<jbenet>
voxelot: salt: some has changed, i need to write DRAFT 4.
<jbenet>
zorun: exactly.
<jbenet>
on all count.
<jbenet>
s
<zorun>
\o/
<zorun>
what is the current direction?
pfraze has joined #ipfs
<M-hash>
err, jbenet, i hope you don't mind if i call you out on some degree of linkrot? :3
<jbenet>
M-hash: please do. i need to put everything on ipfs and then get an ipfs cluster to back up everything.
<M-hash>
or if not outright linkrot, like, "ehh, i wish this was easier to navigate for a new user" sort of stuff
<ipfsbot>
[ipfs] heavenlyhash opened pull request #84: Direct attention to paper in this repo. (master...patch-1) http://git.io/vsK1i
<M-hash>
sure! I don't know if that's exactly the tone you aim for, but I gave it a shot :)
<jbenet>
M-hash: wait, ipfs/ipfs/ipfs-cap2pfs/ipfs-p2p-file-system.pdf is more up to date than ipfs/papers/ipfs-cap2pfs/ipfs-p2p-file-system.pdf ?
<jbenet>
it should be the same
<M-hash>
oh! perhaps that one should be deleted entirely then..?
<M-hash>
i drilled down into the "ttpw" paper in the other repo, saw something draftier than i was looking for, and i guess i backedtracked too many links up at once
<M-hash>
the pdfs are identical
<jbenet>
M-hash yeah it should be deleted. or replaced by a link? (the thing with deleting is you break a link (**gasp**)
<M-hash>
right? :D <rant about the 'L' in URL goes here>
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
M-hash: then maybe add a note in that directory saying that paper is outdated at best, or has moved to another location? im not sure, whatever you think helps people the most
<M-hash>
yeah, i'll change it to a note about moved/dated
voxelot has quit [Ping timeout: 244 seconds]
<M-hash>
and then i'm thinking a quick little TOC in the papers repo? just to point people mainly to the finished one, and note the draft/wip status of the other
<M-hash>
it's a shame cross-repo linking in git without itself exacerbating linkrot problems is a near impossibility
<jbenet>
M-hash yep, we dont have that problem :D
<M-hash>
so, now that i'm paper in hand...
<M-hash>
i see that you have a plan for `commit` objects for historical records, which is awesome
<M-hash>
ahheh, there's a sentence that just trails off at the end of 3.7.1
<jbenet>
yeahhhhhh im not touching paper for a while. i know there's bugs sry too much to do :(
<jbenet>
mg- damn. i'm stuck replicating refs-to-seed as well-- we may have lost some
<M-hash>
mmmm okay will read issue, thanks
<jbenet>
mg- what ref were you stuck at?
<jbenet>
last one i see is QmPHPs1P3JaWi53q5qqiNauPhiTqa3S1mbszcVPHKGNWRh
inconshreveable has quit [Read error: No route to host]
inconshr_ has joined #ipfs
<M-hash>
i'm mulling on the intersection of what ipns wants and what, say, tuf does for liveness, and these seem possibly convergent... i'll spend some more time reading out from your pointers, thanks jbenet
<M-hash>
that system also describes signing lists that map $name to []$hash, more or less, plus a bunch of stuff around timestamping and liveness which is pretty well thought out
<jbenet>
mg- nvm, progressing now. found on the network \o/
Leer10 has quit [Remote host closed the connection]
<M-hash>
p.s. jbenet i'll be in the seattle area for a bit over the next week, if you can be cajoled into coffee sometime..? :3
<jbenet>
M-hash am not in seattle. was only visiting there.
<jbenet>
M-hash am in berlin atm :)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<M-hash>
heh. ccc and/or aftermath?
<jbenet>
yep
<M-hash>
is that still going? i thought all the main track wrapped up this last week
<M-hash>
i've failed to pay enough attention to their calendar for that obvs, or i'd probably be over there too haha
Leer10 has joined #ipfs
<jbenet>
M-hash oh no it's done. i'm hanging out in berlin between things
<jbenet>
mg- now stuck at QmTgHU1LYeLK9CngF6EdwNGz5nCWX1UHpRuRhbFwEXqubx
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
whyrusleeping found a weird problem. ipfs refs -r halts, but there's nothing on the wantlist... didnt we have something like this in the past?
<jbenet>
needs a "find parent of <ref>
dignifiedquire has quit [Quit: dignifiedquire]
mildred has joined #ipfs
mildred has quit [Ping timeout: 260 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
krl: i'm impressed the geoip tree is out there. turns out the node i was backing it up with ... doesn't have all of it! \o/ -- but it seems to be getting the pieces!
<mg->
jbenet: stuck at QmUzorLM61MJzKaPtX2jGceKuYBdeXitgNim4AjdrMYGBw
<daviddias>
to reuse leftover space from encoded data?
<daviddias>
this is like omega optimization
<jbenet>
its also fast decoding.
Leer10 has quit [Remote host closed the connection]
patcon has quit [Read error: Connection reset by peer]
Leer10 has joined #ipfs
notduncansmith has joined #ipfs
amstocker has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<daviddias>
jbenet: thinking through it again. why would \n in multicodec break ndjson? it should only become a ndjson stream after multicodec is interpreted
<jbenet>
daviddias: oh, maybe that's fine. i guess depends how you use it. if you had a sequence of items, and each item was one entry (with its own header)
<jbenet>
i'm thinking of a stream that sends {protobuf, json, cbor} objects randomly, each object with a header.
<jbenet>
but yeah i guess all bets are off and newline is not really a big deal
<daviddias>
but that would mean that a thing like ndjson would have to speak multicodec or you always have a multicodec transform stream in front of the ndjson
<jbenet>
my concern maybe is uses of multicodec in single-line situations
<daviddias>
if we have a ndjson thing in the midle
<daviddias>
/ndjson/"{"hello":"world"}\n"
<daviddias>
the final (invisible) \n would be for multicodec
<daviddias>
and the one inside quotes is for ndjson
<daviddias>
Maybe I'm missing something that you are seeing
<daviddias>
but I would say "put it on the boat!"
<jbenet>
oh maybe. not sure, i dont know enough about how people might use this. like the base encoding ones, potentially, it's annoying to have a newline there? not sure.
<jbenet>
yeah sure.
<daviddias>
we can always put the decision on the developer, of using a new line or not (default to new line), if we come to find a protocol where it breaks things
<daviddias>
then the dev, just has to know that for that protocol, it also has to do the reading without the new line
<jbenet>
yep +1 to that idea
simonv3 has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshr_ has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
Leer10 has quit [Remote host closed the connection]
<jbenet>
daviddias: lmk what else you want me to do in multistream prs
<daviddias>
I'll move the multicodec stuff to the other repo. Once you merge the PR on multistream, it would be good also to make lpm into its own repo and that would be all :)
inconshreveable has quit [Ping timeout: 246 seconds]