jbenet changed the topic of #ipfs to: IPFS - InterPlanetary File System - https://github.com/jbenet/ipfs -- channel logged at https://botbot.me/freenode/ipfs/ -- code of conduct at https://github.com/ipfs/community/blob/master/code-of-conduct.md -- sprint at https://github.com/ipfs/pm/issues/7
pfraze has quit [Remote host closed the connection]
Wallacoloo has quit [Quit: Leaving.]
inconshreveable has joined #ipfs
therealplato has quit [Ping timeout: 276 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has joined #ipfs
pfraze has joined #ipfs
Wallacoloo has joined #ipfs
nessence has joined #ipfs
nessence has quit [Remote host closed the connection]
alexandria-devon has joined #ipfs
pfraze has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Bioblaze has quit [Remote host closed the connection]
anshukla has quit [Quit: Leaving...]
tilgovi has quit [Remote host closed the connection]
lgierth has quit [Quit: Ex-Chat]
Wallacoloo has quit [Quit: Leaving.]
hellertime has quit [Quit: Leaving.]
<Confis> I'm trying to write a simple consensus protocol on top of IPFS. I was wondering:
<Confis> - MakePutRecord in record.go accepts a `sign` boolean. If true, a signature is added to the record.
inconshreveable has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
<Confis> - PutValue in dht/routing.go lets dht.Validator.isSigned() decide if the record is signed, with the record key as parameter.
williamcotton has quit [Ping timeout: 272 seconds]
<Confis> - dht.Validator is instantiated on DHT creation (dht/dht.go), so that isSigned returns true on keys with prefix "pk".
<Confis> Now for the actual question:
notduncansmith has quit [Ping timeout: 258 seconds]
<Confis> When a new IPNS record is published, two things happen:
<Confis> (meanwhile my rubber duck is quacking back at me)
nessence has joined #ipfs
nessence has quit [Ping timeout: 240 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Confis> Goooooooddddd. Question gone :)
<whyrusleeping> Confis: was getting food, you need anything still?
williamcotton has joined #ipfs
thorax has joined #ipfs
<Confis> I was pondering the fine differences between en /pk/ key signed on the Record level, and an IPNS record being signed inside its own protobuf.
<whyrusleeping> Confis: all that is going to change really soon
<Confis> Alright, I'll pull my head out of the Go-sand and read that instead.
<whyrusleeping> lol, its not set in stone yet
<Confis> I'm really getting an idea of how a consensus could be made on top of an immutable datastore.
<Confis> Of course not :)
<whyrusleeping> so keep that in mind
<Confis> But I really need trust to be anchored on the peerId, and I was trying to see how that could fit
Bioblaze has joined #ipfs
<Confis> Also, in the current implementation I can use the IPNS record as a historic attestation.
<Confis> But all resources need to be scoped by path under the global IPNS namespace, and that seems less convienent.
<Confis> (well, global.. I mean the user's IPNS root)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has quit [Ping timeout: 265 seconds]
nessence has joined #ipfs
zrl has joined #ipfs
nessence has quit [Ping timeout: 240 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
EricJ2190 has quit [Ping timeout: 246 seconds]
guest965 has quit [Ping timeout: 240 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has joined #ipfs
nessence has joined #ipfs
<JasonWoof> I've put a significant amount of thought into how to implement p2p backups (before reading about ipfs). I'm thinking of doing the "3 of 10" thing (where you split your data hunks into 10 pieces that are each about 1/3 the size of your hunks, and you can reconstruct your data hunk if you can retrieve _any_ 3 pieces)
<JasonWoof> so then what you want to do, is give a different piece to different people on p2p, and have them stored in different locations
nessence has quit [Ping timeout: 256 seconds]
pfraze has joined #ipfs
<JasonWoof> then you have your node periodically check that the pieces are still being stored on the p2p network
<JasonWoof> with untrusted peers, you can have them verify that the pieces are stored (eg by asking for a checksum of the data prefixed by some random data supplied in the query)
<JasonWoof> but you cannot verify that the unstrusted peer is storing the data in a separate location, or (besides timing analisys) that they are storing it themselves. You can only verify that it is stored somewhere out there
<JasonWoof> So I decided that it is valueble to have your pieces stored on trusted peers (ie those of your friends.)
<JasonWoof> maybe there's some cool thing I haven't thought of that would make it so a peer would need to have the data stored locally in order to prove it had the data
notduncansmith has joined #ipfs
<headbite> sound like par2
notduncansmith has quit [Read error: Connection reset by peer]
<JasonWoof> but even so, it doesn't seem possible for a node to prove that it has my piece stored on a different physical device than the other pieces
<JasonWoof> brb
pfraze has quit [Ping timeout: 256 seconds]
<whyrusleeping> JasonWoof: look up proof of retreivability
guest965 has quit [Read error: No route to host]
guest965 has joined #ipfs
williamcotton has quit [Read error: Connection reset by peer]
williamcotton has joined #ipfs
<JasonWoof> whyrusleeping: I just thought it was interesting that in my use case, it's not just retreivability that I want. I also want assurance that my pieces are unlikely to drop off the network at the same time
<JasonWoof> seems to me that the best way to get promises for the future, and especially promises that won't be likely to be broken simultaniously, is to have different friends make those promises
headbite has quit [Ping timeout: 240 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has quit [Ping timeout: 245 seconds]
headbite has joined #ipfs
<JasonWoof> oop, I'm off to bed. I'll read the backlog in the morning
lidel has quit [Ping timeout: 276 seconds]
lidel has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has joined #ipfs
guest965 has quit [Ping timeout: 246 seconds]
Wallacoloo has joined #ipfs
nessence has joined #ipfs
Blame has quit [Quit: Connection closed for inactivity]
nessence has quit [Ping timeout: 256 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has joined #ipfs
bren2010 has joined #ipfs
<jbenet> whyrusleeping: "i dont think any are missing, i just think you ate a commit that i rebased/ammended" that counts as missing. that's at least one missing commit. which one was it? and are you sure there weren't others? (how?)
<jbenet> whyrusleeping: "jbenet: ping" ICMP_ECHO_REPLY
<jbenet> kyledrake: "Is it possible to distinguish between an IPFS and IPNS hash?" not the hash, but yes the path. "/ipfs/..." "/ipns/...". wherever absolutely possible use _paths_ not _hashes_. in most cases, using hashes instead of paths is the wrong thing to do.
<jbenet> kyledrake: "I was thinking of making building a proxy that uses subdomains of hashes to deal with the Origin issue" yeah, we'll have to do that too until https://www.chromium.org/developers/design-documents/per-page-suborigins lands.
<jbenet> whyrusleeping: "subdomains? like <hash>.ipfs.io?" -- if doing this, i'd also support resolving the full path. (so make both "<hash>.ipfs.io/foo/bar" and "<hash>.ipfs.io/ipfs/<hash>/foo/bar" work)
<jbenet> whyrusleeping "it would upset jbenet" not in this case, i alraedy know we have to have a facility for this until chrome + FF land per-page suborigins. otherwise we blow up the security model of the web.
<jbenet> kyledrake: "The other solution is of course to use the TXT record. BTW that's going to be a problem too, the TXT record is usually used for SPF filtering for mail servers." AFAIK, domains can have multiple TXT records. not all providers implement this.
<jbenet> kyledrake: can we move this discussion over on github.com/ipfs/notes ? this is stuff that would be relevant to others too. And it's a thing we can help with-- we can make (a) gateway binaries, (b) example server configs, and (c) run actual servers that make all this easier. part of gateway stuff.
<jbenet> whyrusleeping: "jbenet: youre not allowed to not be on irc" sorry :c
<jbenet> rht_: "whyrusleeping: tried swapping crypto/rand with math/rand. no noticeable change" there is a noticeable change for me in my computer. a reader with math/rand is _way_ faster than crypto/rand. that's why many tests use a math-based rand reader instead.
<jbenet> whyrusleeping can you merge + deploy: https://github.com/whyrusleeping/pinbot/pull/1 ? also add other relevant people.
<jbenet> Confis: "I'm really getting an idea of how a consensus could be made on top of an immutable datastore." this is what bitcoin is. + a huge multi party computation.
<jbenet> Confis: "But I really need trust to be anchored on the peerId, and I was trying to see how that could fit" yep sounds right.
<jbenet> Confis: "But all resources need to be scoped by path under the global IPNS namespace, and that seems less convienent." for consensus, i'd use IPNS to point to a long chain / commit log ("blockchain minus the work").
<jbenet> Confis: how serious are you about implementing this? I want to make an implementation of Raft that logs to ipfs.
<jbenet> JasonWoof: "So I decided that it is valueble to have your pieces stored on trusted peers (ie those of your friends.)" yes-- this is a peering or web-of-trust model of replication.
<jbenet> JasonWoof: "I also want assurance that my pieces are unlikely to drop off the network at the same time" you can get a promise, and a probabilsitic expectation using game theory models (like reputation systems and economic incentives bitcoin) but you cannot get _certainty_ over this question. in the end, there's a tradeoff between increasing the probability
<jbenet> JasonWoof: "...pieces..." look up erasure coding, fountain codes, raptor codes, etc. don't do this ad-hoc-- standard methods exist with excellent properties and many libraries that implement them. use those.
<jbenet> your content is retrievable and how much resources ($) you're willing to trade off for doing so.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet> whyrusleeping: what do you need merged asap?
<zignig> jbenet: wow , that was a backlog dump .... ;)
<zignig> jbenet: good to see some new voices asking serious questions. That means that the ipfs word is spreading.
<jbenet> zignig: indeed! :)
<zignig> astral boot is still plodding along, I managed to get coroes units ( rkt , docker and stuff ) running out of IPFS.
<zignig> so in theory.... you should be able to deploy a cluster from two hashes ( one for boot , one for services )
<zignig> whyrusleeping: o/
<whyrusleeping> zignig: hey!
<zignig> whyrusleeping: as you are sitting around doing nothing ;P , can you fix one thing for me.
<whyrusleeping> !friends
<pinbot> my friends are: whyrusleeping jbenet tperson krl kyledrake
<whyrusleeping> zignig: sure thing
<whyrusleeping> lol
<jbenet> zignig: that sounds awesome
<jbenet> zignig: how can i try it?
sprintbot has quit [Remote host closed the connection]
<whyrusleeping> oh whoops
sprintbot has joined #ipfs
<zignig> jbenet: will check in the relevant code when I get home, ( and fix a bug ).
<jbenet> zignig: just through things out there
<jbenet> zignig less polish, more usability.
<jbenet> :]
<zignig> whyrusleeping: when you ask for an expire ipns record , it just goes BURP! and does not give you the existing hash , just an error.
<zignig> can you mod the code so it says EXPIRED!!! , but still gives you the old ref.
<jbenet> zignig: this will change with the record validity.
sharky has quit [Ping timeout: 245 seconds]
<zignig> jbenet: indeed, old habbits die hard for me. with my system and networking jobs, get it going , polish and then release.
<zignig> jbenet: thousands of cranky customers makes for angry bosses.
<jbenet> zignig: also, i don't think records known-to-be-invalid should not be returned by the standard API. we could have another call that returns all found records, regardless of validity, but that's plumbing not porcelain. porcelain requires stronger guarantees.
<jbenet> zignig: in open source, your users help you fix it :)
<zignig> yes they do ;).
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
sharky has joined #ipfs
<zignig> jbenet: on the ipns , perhaps being able to specify the TTL , currently 24 hours ( i think ) on the commands line.
<zignig> ipfs name publish <HASH> --ttl= 1 month for example.
<whyrusleeping> jbenet: i'll get to other code things in the morning
<whyrusleeping> but i did a lot of testing and benchmarking different things today
<whyrusleeping> and found out that the random weird network latencies i was seeing was actually wifi
<whyrusleeping> tcp does *not* have good performance over wifi
nessence has joined #ipfs
* zignig hands whyrusleeping two cans and a piece of string.
guest965 has quit [Ping timeout: 264 seconds]
guest965 has joined #ipfs
nessence has quit [Ping timeout: 265 seconds]
<jbenet> whyrusleeping im going to merge that o/
<jbenet> unless you have feedback?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Tv` has quit [Quit: Connection closed for inactivity]
<zignig> jbenet: wow , long read. need to digest , but I like the jist of it.
guest449 has joined #ipfs
thorax has quit [Ping timeout: 256 seconds]
<jbenet> whyrusleeping adding that disabled iptb test?
<jbenet> can be commit on top in the same pr branch
<jbenet> aaaaaand what's next after that
guest4491 has quit [Ping timeout: 272 seconds]
<tperson> I vagely remember krl doing something with multi file upload. Does anyone know if he worked on that?
<tperson> Either way, been working on rewriting node-ipfs-api to use streams instead of just buffers/strings and file names. For named files it uses vinyl.
guest4491 has joined #ipfs
<tperson> Given an array of vinyl files I can now construct a multipart stream that should work (haven't fully tested) with ipfs to due folder uploads.
guest449 has quit [Ping timeout: 272 seconds]
mildred has joined #ipfs
nessence has joined #ipfs
nessence has quit [Ping timeout: 246 seconds]
alexandria-devon has quit [Quit: alexandria-devon]
warner has quit [Quit: ERC (IRC client for Emacs 24.5.1)]
warner has joined #ipfs
mildred has quit [Ping timeout: 276 seconds]
williamcotton has quit [Ping timeout: 258 seconds]
<cryptix> gmorning ipfolks
mildred has joined #ipfs
<tperson> heyo
zabirauf has joined #ipfs
<zabirauf> Hi
<zabirauf> Is there any good resource to understand more about Merkle DAG?
<tperson> Probably the best place besides the source: https://github.com/jbenet/random-ideas/issues/20
<zabirauf> Went over that, but its more of a discussion on hash collision than on the details of merkle dag
<richardlitt> Anyone have suggestions for a good gulp // browserify // jasmine set up?
<tperson> richardlitt: Like an example?
<richardlitt> yeah.
<richardlitt> I should really have more tests going on in my code, and now that I'm building a schema I figure this would be a good place to start.
<tperson> The webui used to use gulp/browserify
<tperson> I worked heavily with jasmine this summer though, so if you have any questions feel free to ask.
<richardlitt> What's it using now?
<tperson> bygg and browserify, just moved away from webpack
<daviddias> Good morning everyone :)
<tperson> gulp might make it's way back in, not sure yet (as we don't know if we really need it)
<tperson> Heyo
machrider_ has quit [Ping timeout: 265 seconds]
<tperson> zabirauf: I had some example code written up a bit ago, I'll see if I can find it.
machrider has joined #ipfs
<tperson> It might help you, not sure exactly what your looking for in terms of details though.
<jbenet> daviddias morning o/
guest965 has quit [Ping timeout: 264 seconds]
<richardlitt> Hmmm
<richardlitt> Just found gulp-jasmine
<richardlitt> Might just use that, may not need to integrate with brfs right now
<tperson> Note: that probably won't compiler anymore.
<jbenet> zabirauf: i want to clean up the merkledag description there into something better. actually mafintosh gave a good exaplanation in a talk at DTN recently. he may have slides
<richardlitt> thanks tperson. not familiar with jasmine at the moment, been a while since I used it
<tperson> I mostly used jasmine in conjunction with karma
<daviddias> zabirauf slides from mafintosh talk are here: http://mafintosh.github.io/slides/dtn-2015/ :)
guest965 has joined #ipfs
<zabirauf> Thanks a lot @tperson, want to understand the implementation details of the Merkle DAG.
<zabirauf> Thanks @jbenet @daviddias for the slides, will take a look at them. I did take a look at the merkle dag implementation in IPFS but it was dependent on BlockService and Blocks so couldn't go much deeper yet as i'm not much aware of golang
nessence has joined #ipfs
inconshreveable has joined #ipfs
<mafintosh> jbenet: i can contribute some drawings if needed :)
guest449 has joined #ipfs
guest4492 has joined #ipfs
guest4491 has quit [Read error: Connection reset by peer]
<daviddias> jbenet this is awesome stuff :)
guest965 has quit [Ping timeout: 276 seconds]
guest449 has quit [Ping timeout: 265 seconds]
guest449 has joined #ipfs
guest4492 has quit [Read error: No route to host]
guest4491 has joined #ipfs
guest449 has quit [Ping timeout: 265 seconds]
machrider has quit [Ping timeout: 256 seconds]
guest449 has joined #ipfs
guest4491 has quit [Ping timeout: 272 seconds]
guest965 has joined #ipfs
machrider has joined #ipfs
inconshr_ has joined #ipfs
domanic has joined #ipfs
guest4491 has joined #ipfs
inconshreveable has quit [Ping timeout: 256 seconds]
guest449 has quit [Read error: Connection reset by peer]
<ipfsbot> [webui] jbenet pushed 1 new commit to master: http://git.io/vkFQy
<ipfsbot> webui/master ba05a29 Juan Batiz-Benet: Merge pull request #58 from ipfs/feat/fixed-globe...
<ipfsbot> [webui] jbenet pushed 1 new commit to master: http://git.io/vkF7N
<ipfsbot> webui/master b74b7e0 Juan Batiz-Benet: Merge pull request #59 from ipfs/hot-reload...
guest449 has joined #ipfs
guest965 has quit [Ping timeout: 276 seconds]
guest4491 has quit [Ping timeout: 265 seconds]
guest4491 has joined #ipfs
guest449 has quit [Ping timeout: 255 seconds]
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
guest965 has joined #ipfs
lgierth has joined #ipfs
zabirauf has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
inconshr_ has quit [Remote host closed the connection]
tso has quit [Ping timeout: 265 seconds]
Wallacoloo has quit [Quit: Leaving.]
Bioblaze has quit [Remote host closed the connection]
<jbenet> mafintosh: when are you in PA again?
<ipfsbot> [go-ipfs] rht opened pull request #1326: Test repeat in 10 sec: increase check frequency tenfold (master...master) http://git.io/vkbsk
<mafintosh> jbenet: i fly to the bay on monday
<jbenet> cool.
e-lima has joined #ipfs
zigguratt has joined #ipfs
<ipfsbot> [go-ipfs] jbenet pushed 2 new commits to master: http://git.io/vkbB2
<ipfsbot> go-ipfs/master 9bdc1d6 rht: Test repeat in 10 sec: increase check frequency tenfold
<ipfsbot> go-ipfs/master 18297e2 Juan Batiz-Benet: Merge pull request #1326 from rht/master...
inconshreveable has joined #ipfs
inconshreveable has quit [Remote host closed the connection]
inconshreveable has joined #ipfs
inconshreveable has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest449 has joined #ipfs
guest4491 has quit [Ping timeout: 255 seconds]
<lgierth> switched my servers over to ed25519 ssh keys too
<lgierth> feels so good
gurst has joined #ipfs
<jbenet> lgierth i should too :)
<lgierth> regenerating the ssh moduli took a while on these little digitalocean slices
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest4491 has joined #ipfs
guest449 has quit [Read error: Connection reset by peer]
Blame has joined #ipfs
www has joined #ipfs
hellertime has joined #ipfs
guest965 has quit [Ping timeout: 244 seconds]
zabirauf has joined #ipfs
guest449 has joined #ipfs
guest965 has joined #ipfs
guest4491 has quit [Ping timeout: 272 seconds]
<ipfsbot> [go-ipfs] rht opened pull request #1327: Swap all 'crypto/rand' rng in tests with 'math/rand' (master...cleanup-rand) http://git.io/vkbKh
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
zabirauf has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
reit has joined #ipfs
brab has joined #ipfs
www has quit [Ping timeout: 264 seconds]
guest965 has quit [Ping timeout: 258 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has joined #ipfs
williamcotton has joined #ipfs
williamcotton has quit [Ping timeout: 276 seconds]
slothbag has joined #ipfs
guest965 has quit [Read error: Connection reset by peer]
guest965 has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<zigguratt> hey all! I don't know whether this is the right place for this but on a fresh install of IPFS (once on debian sid and once on centos 6) I get 'Path Resolve error: no link named "fontawesome-webfont.woff2"' from the daemon. I know it's not a catastrophe, but I'd like to resolve it if possible.
guest965 has quit [Read error: Connection reset by peer]
guest9651 has joined #ipfs
<slothbag> I get that as well when loading the new webui
<slothbag> not a big issue
guest9651 has quit [Read error: Connection reset by peer]
guest965 has joined #ipfs
<rht_> if you replace woff2 with woff in style.css, it should work
guest965 has quit [Ping timeout: 240 seconds]
guest965 has joined #ipfs
guest965 has quit [Ping timeout: 256 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has joined #ipfs
_vanmount has joined #ipfs
_vanmount has quit [Client Quit]
gurst has quit [Ping timeout: 246 seconds]
<rht_> zigguratt: just wait for people to wake up, I think it's just browserify bundling. need more fontawesome files in static/fonts
<zigguratt> thanks everyone! I'll look at style.css in the short term. as I mentioned, I know it's not a huge issue but it's nice to have clean output.
* jbenet zigguratt pls file an issue over at github.com/ipfs/webui
<zigguratt> will do
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has quit [Read error: Connection reset by peer]
guest9651 has joined #ipfs
williamcotton has joined #ipfs
williamcotton has quit [Ping timeout: 246 seconds]
slothbag has quit [Quit: KVIrc 4.2.0 Equilibrium http://www.kvirc.net/]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<JasonWoof> jbenet: yes :) I know not to implement my own crypto/etc (re erasure encoding)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has quit [Ping timeout: 240 seconds]
williamcotton has joined #ipfs
williamcotton has quit [Max SendQ exceeded]
williamcotton has joined #ipfs
domanic has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has quit [Ping timeout: 245 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has joined #ipfs
guest9651 has quit [Read error: Connection reset by peer]
williamcotton has joined #ipfs
zigguratt has left #ipfs [#ipfs]
zigguratt has joined #ipfs
guest9651 has joined #ipfs
guest965 has quit [Read error: No route to host]
guest965 has joined #ipfs
guest965 has quit [Read error: No route to host]
guest9651 has quit [Ping timeout: 264 seconds]
guest9653 has joined #ipfs
guest965 has joined #ipfs
guest9653 has quit [Read error: Connection reset by peer]
guest9651 has joined #ipfs
guest9652 has joined #ipfs
guest9651 has quit [Read error: Connection reset by peer]
guest965 has quit [Ping timeout: 264 seconds]
guest9652 has quit [Ping timeout: 256 seconds]
guest965 has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
lgierth has quit [Quit: Ex-Chat]
pfraze has joined #ipfs
guest965 has quit [Read error: No route to host]
<Confis> jbenet: I was only trying to build a basic consensus algorithm, so I could kick off (a basic version of) the application I'm really building: DC-nets on elliptic pairings.
guest965 has joined #ipfs
<Confis> I'm taking a modification of the Verdict algorithm, which is already somewhat arbitrary-fault tolerant.
<Confis> Raft is cool, but it introduces so much abstraction that I can barely see the trees in the forest.
<Confis> To keep an historic track of the consensus, I could:
<Confis> - Require the IPNS namespace (where the signature goes) to be append-only
williamcotton has quit [Ping timeout: 244 seconds]
<Confis> - Copy the signature from the IPNS root and put it deeper in the DAG
<Confis> But I'm probably rambling within my own concepts, and I haven't thought this out completly :)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has quit [Ping timeout: 244 seconds]
williamcotton has joined #ipfs
guest965 has joined #ipfs
guest965 has quit [Ping timeout: 265 seconds]
brab has left #ipfs ["ERC (IRC client for Emacs 24.5.1)"]
guest965 has joined #ipfs
guest965 has quit [Ping timeout: 252 seconds]
guest965 has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has quit [Ping timeout: 258 seconds]
guest965 has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Tv` has joined #ipfs
<whyrusleeping> jbenet: i realize my test for the 400GB add cat is just xt0130-multinode.sh with a larger number
<whyrusleeping> whoops, not 400GB, 400MB
<whyrusleeping> >.>
mildred has quit [Ping timeout: 272 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has quit [Ping timeout: 264 seconds]
patcon has joined #ipfs
williamcotton has joined #ipfs
guest965 has quit [Ping timeout: 264 seconds]
<ipfsbot> [go-ipfs] whyrusleeping pushed 1 new commit to revert-godeps-update: http://git.io/vkAzF
<ipfsbot> go-ipfs/revert-godeps-update db98e77 Jeromy: add a test for issue repro
guest965 has joined #ipfs
guest965 has quit [Read error: No route to host]
guest965 has joined #ipfs
therealplato has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
tilgovi has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
lgierth has joined #ipfs
emery has joined #ipfs
<ipfsbot> [go-ipfs] whyrusleeping pushed 1 new commit to feat/patch: http://git.io/vkAAw
<ipfsbot> go-ipfs/feat/patch e37fbae Jeromy: add two new subcommands to object patch and clean up main Run func
guest965 has quit [Ping timeout: 276 seconds]
<ipfsbot> [go-ipfs] whyrusleeping force-pushed feat/patch from e37fbae to b70dd34: http://git.io/vkWAt
<ipfsbot> go-ipfs/feat/patch a9ebf8d Jeromy: WIP: object creator command
<ipfsbot> go-ipfs/feat/patch 1d1544a Jeromy: implement an ipfs patch command for modifying merkledag objects
<ipfsbot> go-ipfs/feat/patch 1737c0f Jeromy: better docs
guest965 has joined #ipfs
zigguratt has left #ipfs [#ipfs]
<emery> did you guys ever find alternatives to rabin fingerprinting for splitting blocks?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> emery: no, we are suing constant sized block splitting
<whyrusleeping> rabin is still planned, just havent gotten around to it yet
<emery> ok, I have a non-ipfs situation where I might want to split blocks with deduplication in mind, I'll let you guys know if I find something better
<whyrusleeping> emery: are you going to be working in go?
<whyrusleeping> also, i dont know if theres much better than rabin if you are optimizing for dedup
<emery> this will not be in go ultimately, but it might be easy to test new algos in go :)
emery has left #ipfs ["Leaving"]
emery has joined #ipfs
<emery> I was looking at the paper for the venti filesystem and it mentioned rabin fingerprinting, which is why I thought of it
<whyrusleeping> emery: what is your use case?
guest449 has quit [Ping timeout: 276 seconds]
<emery> I'm making a file system server for a microkernel OS that sits between a client and a backend server and transparently hashes files for deduplication and to make directory trees and file immutable
<emery> which has me thinking about ipfs again
<emery> the requirements are actually a subset of ipfs features but I don't have the go runtime available
<emery> so this is just something a little hacky for now
<whyrusleeping> emery: you dont need a go runtime, go compiles statically.
<whyrusleeping> you can build for that system on another machine and just copy the binary over
<emery> the problem is the threads and the thread local storage stuff is different
<whyrusleeping> oooh, missed the 'different OS' part
bren2010 has quit []
<emery> yea that stuff is beyond me
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has quit [Ping timeout: 256 seconds]
<krl> tperson: doing streams seems like a very good idea
<krl> tperson: i did not do multi file, just directory wrapping
<krl> tbh, i think an easier/better way to go is to use unixfs directly from js
<krl> instead of making api accept folders with special syntax, just construct the unixfs objects locally
domanic has quit [Ping timeout: 264 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
guest449 has joined #ipfs
thorax has joined #ipfs
guest449 has quit [Ping timeout: 265 seconds]
www1 has joined #ipfs
rmk0 has joined #ipfs
guest449 has joined #ipfs
<whyrusleeping> alright, serializing blockstore writes doesnt improve perf
* whyrusleeping checks another box off of his list
www1 has quit [Ping timeout: 250 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
thorax has quit [Ping timeout: 255 seconds]
fleeky has quit [Read error: Connection timed out]
fleeky has joined #ipfs
williamcotton has quit [Ping timeout: 264 seconds]
williamcotton has joined #ipfs
<krl> tperson: yeah, it's cool :)
williamcotton has quit [Ping timeout: 264 seconds]
guest4491 has joined #ipfs
nessence has quit [Remote host closed the connection]
guest449 has quit [Ping timeout: 240 seconds]
fleeky has quit [Read error: Connection timed out]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
warner has quit [Quit: ERC (IRC client for Emacs 24.5.1)]
Bioblaze has joined #ipfs
<krl> jbenet: regarding protobuf/mpack/etc
<krl> maybe the best way would be to go to streamstack objects directly
<krl> but just limit them to resolving links and providing data
<krl> then build from there
<bigbluehat> jbenet: you are likely already well aware of this...but sharing just in case :) https://tools.ietf.org/html/draft-ietf-ppsp-peer-protocol-12
<bigbluehat> PPSP
<bigbluehat> Peer-to-Peer Streaming Peer Protocol (PPSPP)
<bigbluehat> in erlang http://www.swirl-project.org/
<bigbluehat> by dch (of CouchDB contribution "fame") ;)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> bigbluehat: note the date on that :P
<whyrusleeping> jbenet: where would a good place for questions on the records spec implementation be?
<whyrusleeping> there are a few issues that need to be worked out before i can begin on it
guest756 has joined #ipfs
<bigbluehat> whyrusleeping: yeah...RFC dates are tricky
<bigbluehat> JSON Schema's expired years ago
<bigbluehat> but people still use and code for it
<bigbluehat> ...and that's unlikely to stop
<bigbluehat> doesn't mean it's not useful stuff :)
<bigbluehat> swirl's fairly new, afaik
<whyrusleeping> well, i'm saying it's dated a ways after we started
<bigbluehat> if it ain't interesting, though, ignore it ;)
<whyrusleeping> its definitely interesting
<whyrusleeping> i remember seeing it a few months ago
grncdr has joined #ipfs
<bigbluehat> ah. :)
guest965 has joined #ipfs
<ipfsbot> [go-ipfs] whyrusleeping force-pushed builder-default from ba18dac to 8cb34bc: http://git.io/vkx9M
<ipfsbot> go-ipfs/builder-default 8cb34bc Jeromy: make the default repo for corebuilder work
<tperson> PPSPP Sounds so much like IPFS.
<whyrusleeping> a little too much like ipfs IMO
<whyrusleeping> but ipfs is open source so...
pfraze has quit [Remote host closed the connection]
meadman has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot> [webui] krl opened pull request #61: rewrite referer to make api happy (master...api-proxy-fix) http://git.io/vkxdq
grncdr1 has joined #ipfs
guest965 has quit [Ping timeout: 265 seconds]
grncdr has quit [Ping timeout: 245 seconds]
guest965 has joined #ipfs
guest965 has quit [Ping timeout: 246 seconds]
guest965 has joined #ipfs
pfraze has joined #ipfs
inconshreveable has joined #ipfs
guest965 has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has joined #ipfs
inconshreveable has quit [Remote host closed the connection]
<sprintbot> Sprint Checkin! [whyrusleeping jbenet cryptix wking lgierth krl]
inconshreveable has joined #ipfs
<krl> ping
<lgierth> pong
<lgierth> checkin: gateway infrastructure cleanups & stability #funwithpython
patcon has quit [Ping timeout: 252 seconds]
guest965 has joined #ipfs
guest9651 has joined #ipfs
guest965 has quit [Read error: Connection reset by peer]
<lgierth> it's really amazing
<lgierth> yesterday i had ansible working fine against all 8 managed hosts
<lgierth> today two of them fail, and it's not the two which are out-of-space
<krl> checkin: removed webpack in favor of bygg/browserify
harlan_ has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
flugsio has quit [Quit: WeeChat 1.2]
<tperson> krl: I don't believe the proxy works anymore.
<lgierth> i'm still waiting for a hanging gateway to happen
<tperson> proxy-middleware does not mask origins. Is there a flag we need to run on the daemon?
<tperson> lgierth: Hanging gateway?
<krl> tperson: proxy works?
<krl> oh sorry
<krl> i already have a pull request for this
<ipfsbot> [webui] travisperson pushed 1 new commit to master: http://git.io/vkpsJ
<ipfsbot> webui/master 816bdc1 Travis Person: Merge pull request #61 from ipfs/api-proxy-fix...
<lgierth> tperson: spike in response times and boostrap times
e-lima has quit [Ping timeout: 256 seconds]
guest9652 has joined #ipfs
guest9651 has quit [Ping timeout: 272 seconds]
guest9652 has quit [Read error: Connection reset by peer]
alexandria-devon has joined #ipfs
rht_ has quit [Quit: Connection closed for inactivity]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<wking> checkin: Responded to spec discussions in go-ipfs#1307, go-ipfs#1320, specs#7. Working on response for go-ipfs#1291
<wking> I'll hopefully get time to work on the Docker-registry storage driver this afternoon
<krl> tperson: got it working?
e-lima has joined #ipfs
<tperson> krl: Ya, I just merged that in.
guest965 has joined #ipfs
tilgovi has quit [Ping timeout: 256 seconds]
tso has joined #ipfs
guest965 has quit [Ping timeout: 258 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot> [go-ipfs] wking pushed 1 new commit to add-api: http://git.io/vkpBl
<ipfsbot> go-ipfs/add-api e6b7dea W. Trevor King: Revert "Replace single callbacks with slices of function references"...
nessence has joined #ipfs
guest965 has joined #ipfs
fleeky has joined #ipfs
tilgovi has joined #ipfs
hellertime has quit [Quit: Leaving.]
hellertime has joined #ipfs
alexandria-devon has quit [Quit: alexandria-devon]
<ipfsbot> [go-ipfs] wking pushed 1 new commit to add-api: http://git.io/vkp2O
<ipfsbot> go-ipfs/add-api 820b3d8 W. Trevor King: Rename Add -> AddFile and AddFromReader -> Add...
notduncansmith has joined #ipfs
hellertime has quit [Ping timeout: 265 seconds]
chriscool has joined #ipfs
notduncansmith has quit [Ping timeout: 272 seconds]
patcon has joined #ipfs
alexandria-devon has joined #ipfs
inconshr_ has joined #ipfs
cjdmax has quit [Remote host closed the connection]
inconshreveable has quit [Ping timeout: 265 seconds]
<barnacs> whyrusleeping: your benchmarks for add are...interesting
<barnacs> i can't say i understand what's happening there
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
anshukla has joined #ipfs
guest965 has quit [Ping timeout: 276 seconds]
guest965 has joined #ipfs
<jbenet> Confis: makes sense. Raft is geared for fast datacenters.
<jbenet> whyrusleeping: ohhh interesting. 400MB is easier to test with :)
<jbenet> emery: i want to put chunking algos here: https://github.com/jbenet/go-chunk -- maybe should make a lang agnostic repo and focus on algos + link to impls in various langs.
<jbenet> bigbluehat: thanks! yep, aware but always good to check.
<jbenet> whyrusleeping: put questions that may clarify or improve the specs in that repo itself. if it's very general "how to use" maybe in ipfs/notes? (hard to classify human discussion scoping :) )
<jbenet> PPSPP is a good thing and we can probably use it as a transport eventually. we should start a conversation with the team. (and, i think they've been around for a while, it started as academic work so probably predates IPFS? it's mostly inspired by classic bittorrent + some papers improving on it, i think)
<jbenet> lgierth: :( re ansible. Sometimes i have to run the ./clear_logs.sh script just to be able to run ansible commands-- maybe ansible attempts to write to disk?
<jbenet> checkin: pushed out multistream multistream-select and ipfs wire specs. CR + merges. next up is allocating the ipfs wire work. pinning migration + S3 CR.
<jbenet> anshukla: are you around Stanford / PA these days? i'll be here this week + next.
<anshukla> jbenet: yes, let's meet.
<anshukla> I want to talk to you about some tools to make smart people more effective
<whyrusleeping> jbenet: well, my first question is: since the records are now merkledag objects, that means the dht will have to import merkledag, which creates a circular dependency
<whyrusleeping> barnacs: yeah, its kinda weird...
<jbenet> anshukla sgtm. around philz in dt PA today. also, make it "tools to make people smarter and more effective". people tend to feel less smart than they are, and tend to feel put off by "smart things"
<jbenet> whyrusleeping: why circular? merkledag should depend only depend on blocks / blockstore / blockservice, bitswap (which depends on dht) should be wired in as an exchange in the core ?
<anshukla> hm. there's some intentionality behind that phrasing, but I understand what you mean. I'm pretty busy until Friday evening -- I'll be on the channel after and we can figure out a time to drop by
<jbenet> sounds good. sunrise is so good.
notduncansmith has joined #ipfs
<whyrusleeping> jbenet: merkledag depends on bitswap, which depends on routing, which depends on the records which depends on the merkledag
notduncansmith has quit [Read error: Connection reset by peer]
<bigbluehat> jbenet: cool. :) keep up the greatness!
<jbenet> bigbluehat: we haven't caught up! have time today?
<jbenet> whyrusleeping: right, merkledag should not depend on bitswap. it should use an exchange interface, not bitswap specifically.
<jbenet> (or not even an exchange, just a blockservice or blockstore or whatever)
<anshukla> whyrusleeping: forgot to send this, but continue a long-ago conversation about the failure of web browsers: http://www.drdobbs.com/architecture-and-design/interview-with-alan-kay/240003442?pgno=2
<anshukla> you might find it worth filing away under "to read later"
<bigbluehat> jbenet: sadly I'm wrapping up for today
<bigbluehat> let's chat soon though
<bigbluehat> tomorrow AM (EST)?
<bigbluehat> got some meetings tomorrow afternoon, but tomorrow or (possibly) Friday AM my time could work
<jbenet> bigbluehat: various times after 12PDT work this week, but not earlier
Xe has quit [Ping timeout: 265 seconds]
<whyrusleeping> jbenet: oh look, blockservice imports bitswap because of the mock thing i was trying to fix..
* whyrusleeping sighs
<bigbluehat> jbenet: hm...how's next week look :)
<bigbluehat> email? ;)
<jbenet> bigbluehat: yeah, email :)
<jbenet> whyrusleeping: in tests? package tests can always be packaged into `package <pkg>_test` packages. ... package.
<whyrusleeping> jbenet: the problem is that other package import it too...
<whyrusleeping> which sucks
<jbenet> whyrusleeping: \o/ dependency graph problems! -- once we rip things apart we'll be much more judicious in our imports, because we'll write tools that we wont want to import half of ipfs for.
<whyrusleeping> yeah...
<whyrusleeping> i really hate go sometimes
Xe has joined #ipfs
<whyrusleeping> its claiming that i have two different packages in a certain folder
notduncansmith has joined #ipfs
<whyrusleeping> when i dont
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot> [go-ipfs] chriscool created implement-test-seq (+1 new commit): http://git.io/vkpNN
<ipfsbot> go-ipfs/implement-test-seq 4bbe06e Christian Couder: ipfs-test-lib: implement test_seq()...
<ipfsbot> [go-ipfs] chriscool opened pull request #1329: ipfs-test-lib: implement test_seq() (master...implement-test-seq) http://git.io/vkpAY
harlan_ has quit [Quit: harlan_]
EricJ2190 has joined #ipfs
<whyrusleeping> (also see line 17)
<jbenet> lolwut.
<jbenet> sigh.
<jbenet> whyrusleeping: should i merge the revert of cryptix's stuff?
<ipfsbot> [go-ipfs] jbenet pushed 2 new commits to master: http://git.io/vkppk
<ipfsbot> go-ipfs/master db98e77 Jeromy: add a test for issue repro
<ipfsbot> go-ipfs/master 898863d Juan Batiz-Benet: Merge pull request #1323 from ipfs/revert-godeps-update...
<whyrusleeping> jbenet: yeah
<ipfsbot> [go-ipfs] whyrusleeping created fix/test-deps (+1 new commit): http://git.io/vkpp2
<ipfsbot> go-ipfs/fix/test-deps c2b7456 Jeromy: fix up some dependencies to avoid circular record deps
<tperson> I feel like it shouldn't let you do that...
<tperson> Is that the compile error you were getting?
<whyrusleeping> i wasnt getting any compiler errors on that
<tperson> What issue brought it up? Above you said it was complaing about having two different packages.
<tperson> Does it treat them as different objects?
<whyrusleeping> no, those are two separate issues
<whyrusleeping> i just discovered that import thing while debugging the two package issue
guest965 has quit [Read error: Connection reset by peer]
<tperson> ah
guest9651 has joined #ipfs
<tperson> For how picky go is, I would assumed they'd yell at you for importing something twice lol
Wallacoloo has joined #ipfs
notduncansmith has joined #ipfs
<whyrusleeping> me too!
alexandria-devon has quit [Quit: alexandria-devon]
chriscool has quit [Ping timeout: 272 seconds]
elima_ has joined #ipfs
notduncansmith has quit [Ping timeout: 240 seconds]
patcon has quit [Ping timeout: 256 seconds]
<tperson> Electron is amazingly simple to work...
<whyrusleeping> tperson: oh yeah?
<tperson> No complicated build process for development
guest9651 has quit [Read error: Connection reset by peer]
<tperson> krl: Do you know if there is a way to attach the chrome debugger?
guest965 has joined #ipfs
e-lima has quit [Ping timeout: 258 seconds]
<ipfsbot> [go-ipfs] whyrusleeping opened pull request #1330: fix up some dependencies to avoid circular record deps (master...fix/test-deps) http://git.io/vkhJ5
chriscool has joined #ipfs
guest9651 has joined #ipfs
guest965 has quit [Read error: Connection reset by peer]
<krl> tperson: hmm, to what?
<tperson> electron app
<tperson> I played around with enabling the debug port but it fails to load, just curious if you got it to work.
<krl> mainWindow.openDevTools()
<krl> i still get a weird crash where the tray icon dies
<krl> looking at it now
<ipfsbot> [go-ipfs] chriscool force-pushed implement-test-seq from 4bbe06e to 59cac0a: http://git.io/vkhkD
<ipfsbot> go-ipfs/implement-test-seq 3c2e0ba Christian Couder: ipfs-test-lib: implement test_seq()...
<ipfsbot> go-ipfs/implement-test-seq 59cac0a Christian Couder: test-lib: use test_seq()...
guest9651 has quit [Ping timeout: 264 seconds]
warner has joined #ipfs
<jbenet> tperson: i usually find that cmd+alt+j works just fine in all electron apps
notduncansmith has joined #ipfs
<jbenet> but this may be setup manually
notduncansmith has quit [Read error: Connection reset by peer]
guest965 has joined #ipfs
chriscool has quit [Quit: Leaving.]
chriscool has joined #ipfs
pfraze has quit [Remote host closed the connection]
<jbenet> whyrusleeping: your ctx cancels-- https://github.com/ipfs/go-ipfs/pull/1299/files#r31675032
chriscool has quit [Read error: Connection reset by peer]
chriscool has joined #ipfs
<whyrusleeping> jbenet: youre picking my nits. it'l time out eventually! :P
<jbenet> no it wont-- ctx cancels _always_ need to be called or it leaks
<whyrusleeping> thats what the 30 second timeout is for.
<whyrusleeping> it makes it time out
<whyrusleeping> i'm changing the code anyways, but context.WithTimeout calls the cancel func after the given timeout
<jbenet> cancel must be called to release resources. even if this happens for us under the hood, the API they provide is to call cancel.
<jbenet> the problem is that someone will come along and do a one-line change to make it not timeout
<jbenet> and then leaks.
<jbenet> as discussed in #822 both cryptix and i think not cancelling is error prone
<jbenet> ("not canceling explicitly"
<whyrusleeping> ah, this is an argument of convention, not of actual behaviour
<whyrusleeping> gotcha
guest9651 has joined #ipfs
guest9651 has quit [Read error: Connection reset by peer]
<ipfsbot> [go-ipfs] whyrusleeping pushed 1 new commit to feat/patch: http://git.io/vkhm9
<ipfsbot> go-ipfs/feat/patch f155f2e Jeromy: cancel contexts in early returns
guest965 has quit [Ping timeout: 245 seconds]
<ipfsbot> [go-ipfs] jbenet pushed 1 new commit to master: http://git.io/vkhYY
<ipfsbot> go-ipfs/master 238fac5 Juan Batiz-Benet: Merge pull request #1330 from ipfs/fix/test-deps...
<ipfsbot> [go-ipfs] jbenet deleted fix/test-deps at c2b7456: http://git.io/vkhYn
<ipfsbot> [go-ipfs] jbenet pushed 1 new commit to master: http://git.io/vkhYz
<ipfsbot> go-ipfs/master 0397bdf Juan Batiz-Benet: Merge pull request #1329 from ipfs/implement-test-seq...
<ipfsbot> [go-ipfs] jbenet closed pull request #1327: Swap all 'crypto/rand' rng in tests with 'math/rand' (master...cleanup-rand) http://git.io/vkbKh
<jbenet> whyrusleeping patch LGTM. not to travis: https://github.com/ipfs/go-ipfs/pull/1299#issuecomment-108630266 want to rebase + squash?
<whyrusleeping> yeap, on it
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot> [go-ipfs] whyrusleeping force-pushed feat/patch from f155f2e to 1c7d478: http://git.io/vkWAt
<ipfsbot> go-ipfs/feat/patch cd1e31e Jeromy: implement an ipfs patch command for modifying merkledag objects
<ipfsbot> go-ipfs/feat/patch 225ffd7 Jeromy: WIP: object creator command
<ipfsbot> go-ipfs/feat/patch f2dc177 Jeromy: better docs
<whyrusleeping> forgot to commit the fixes...
<whyrusleeping> meh
<lgierth> jbenet: pretty sure PPSPP predates ipfs... i've met dch on the couchhack in december 2013 and it was already concrete
<lgierth> dch is great btw
<jbenet> wking thanks so much for the email to uw :)
elima_ has quit [Ping timeout: 244 seconds]
<whyrusleeping> squash it all!
<ipfsbot> [go-ipfs] whyrusleeping force-pushed feat/patch from 1c7d478 to 40e423d: http://git.io/vkWAt
<ipfsbot> go-ipfs/feat/patch 40e423d Jeromy: implement an ipfs patch command for modifying merkledag objects...
<jbenet> lgierth: yeah i thought so too i rememeber people referencing it when discussing ipfs in feb 2014, which means it had already been around. in any case, temporal precedence is less important than logical / derivation precedence (i.e. what's the flow of ideas). i think PPSPP could use more ipfs concepts in it-- like their use of hash trees is very limited, and
<jbenet> only used in the way that bittorrent uses them, not the way we or git uses them.
<jbenet> lgierth would be awesome to open an email/github stream with them. i'll ping
pfraze has joined #ipfs
<lgierth> yeah i agree
<jbenet> also want to see how we can best make use of PPSPP, if people will implement it, it could be a really nice underlying transport for ipfs.
<lgierth> doesn't matter that you're first when someone comes along, picks up your ideas and does them the right way
<lgierth> (i'm not even sure what ppspp does in detail :P)
<jbenet> yeah exactly, like most of ipfs is a rehash of older ideas (primarily mazieres (or mazieres derived) ideas)
<Luzifer> yay. now i'm in too. just got an abuse message with a thread to block my server tomorrow after an outgoing net-scan...
<whyrusleeping> Luzifer: yay
<whyrusleeping> Luzifer: mind sending us your email?
<jbenet> Luzifer damn-- did you try the iptables fix? we should get that proper fix into ipfs.
<jbenet> and the simple hack to not advertise addresses observed only once.
<jbenet> i can fix that today.
<Luzifer> jbenet: just got back from bed to search it
<Luzifer> (the iptables-fix)
<jbenet> whyrusleeping: we should also fix the "dont dial a LAN addr if you're not on that LAN". do we have a list of good heuristics for that?
<Luzifer> whyrusleeping: interested in the netscan-log or the whole mail (obviously in german)
<whyrusleeping> the netscan log mostly
<whyrusleeping> Luzifer: whats the LAN addr of your node?
<whyrusleeping> 188.40 is a weird subnet
<Luzifer> hetzner (german low-cost server hoster) owns 188.40
<warner> jbenet: hm, what if you compare the non-RFC1597 addresses (between yourself and the peer in question), and only try to connect to the yes-RFC1597 LAN ones if the external ones are "close"?
<whyrusleeping> Luzifer: ah, okay. so thats entirely public
<warner> you could use "the same" instead.. that would fail to use the local connection for large NATs with multiple external addresses
<whyrusleeping> and there arent actually any internal LAN addrs associated with that node
<Luzifer> whyrusleeping: yep. no private networking except docker
<warner> but effectively use the external addrs to scope the internal ones
<jbenet> warner: yeah close makes more sense. have a good heuristic for this? (maybe just comparing subnet masks...)
<warner> hm
<warner> oh, right, compare the subnet masks of the local interface too
<Luzifer> jbenet: comes that close to your iptables fix? http://gateway.ipfs.io/ipfs/QmQNjyU8wFKPFNKeU9bCnJi7KjargvLLjrrBzXnJ5xVufj (just built that from mind)
<jbenet> warner: the tricky case is stuff like a 192.168.x.x AP connected to a 10.1.x.x ethernet behind a 172.16.x.x
<warner> (if I'm 10.0.0.1/8, and you're 10.0.0.1/16, then we're probably not on the same subnet)
<jbenet> yep, that too
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<warner> eh, I don't think multiple NAT is effectively weirder than single-NAT: it's not like you could detect or use the in-between addresses very well
<jbenet> it may be useful to have a heuristic list on how best to classify these.
<jbenet> sometimes it's not fully NATed, just different subnets not well configured-- thinking of houses with ethernet + wireless.
guest965 has joined #ipfs
<warner> I mean, if you tried to connect to the middle one, and succeeded, then they could tell you what your middle-NAT not-quite-"external" address was, but only someone on that middle network would be able to use it
<jbenet> routers do all sorts of different stuff by default-- thinking of some cisco routers that use 10.x, with most wireless APs using 192.168.x etc
<jbenet> warner: yeah that makes sense
<warner> so I think we could just use 1: the local en0/etc address, and 2: the external non-RFC1597 address
<jbenet> interestingly, in the "non-NATed" case is harder to tell apart cause both sides will see addresses in different local networks
<lgierth> do you actually scan subnets?
<jbenet> we don't scan them, we just dial lots of addresses that are local
<jbenet> so it _looks like_ a scan.
<lgierth> :P
chriscool has quit [Ping timeout: 264 seconds]
<lgierth> i.e. deliberately dial whatever we can come up with locally
<warner> you can learn your (local) subnet mask by looking at your interface, but you can't learn (in fact it's not really useful or meaningful) anything about the subnet mask on the exterior of your NAT box
<jbenet> well it's not random, this follows the ICE WebRTC way of connecting peers.
<Luzifer> okay. put iptables online on both servers and triggered a re-check at my hoster. lets see if they are happy for now and let me explain the reason without blocking the server
<lgierth> oh i see
<jbenet> ICE was designed for browsers in local networks where it's usually one or two attempts at dialing and mostly lax enviroments, not large p2p networks with lots of peers churning, and sysadmins carefully monitoring traffic.
<warner> ah, but IPFS doesn't advertise the subnet mask to peers
<jbenet> in any case, we can do things better
<jbenet> warner: yeah we should probably do that.
guest965 has quit [Read error: Connection reset by peer]
<warner> so maybe the best initial rule would be: if the peer has an RFC1597 address which is a member of our local subnet, and if their non-RFC1597 address is the same as ours, then attempt to use the ("internal"/"LAN") RFC1597 address too. Otherwise ignore it.
guest9651 has joined #ipfs
<warner> later, if we advertise the subnet, the rule expands to require that the subnet be the same (same prefix+length)
<warner> and a different later change would be to allow the external/WAN/RFC1597 address to be slightly different, instead of requiring it to be exactly the same
<jbenet> well, sometimes the external addresses may not even match-- there's lots of weird nat topologies.
<jbenet> yeah
<warner> maybe the same 24-bit prefix
<warner> yeah, that's why it's a heuristic :)
<whyrusleeping> ^
<jbenet> btc once suggested using an ML classifier to train the local node on what addrs are likely to work or not, so it can just discard addrs it learns aren't going to.
<whyrusleeping> i'm going to put that idea on the 'overkill' pile
<lgierth> do you consider this stuff part of routing too? i'm just not very clear on some terms yet
<warner> the two nodes may in fact be running on the same machine, in different containers, with entirely unrelated IP addresses forwarded from some external place.. there's no way to discover connectivity like that without actually trying it
<jbenet> would need careful attention to changes in the topology to reset the classifier-- including changes that are not reflected in local interfaces.
<krl> who was it that was interested in machine learning for ipfs?
<krl> jbenet:
<krl> in the sprint
<krl> it raised my eyebrows for sure
<whyrusleeping> krl: kbala_
<lgierth> or is routing only when you wanna get data to a node that you're not peered with?
<jbenet> whyrusleeping: it may not be overkill and may actually yield amazing results. simple ml classifiers like naive-bayes arent very cpu intensive.
<jbenet> oh yeah kbala_ this may be an interesting side-quest
<jbenet> may distract from bitswap, but might be a good "get feet wet" thing to do.
<Luzifer> ah my provider stated the problem is solved
<jbenet> Luzifer: with the iptables fix?
<jbenet> lgierth: it's sort of part of routing and part of network.
<Luzifer> jbenet: yep. with that one I posted
<lgierth> that's a good thing to have completely abstracted away by a transport layer
<jbenet> lgierth: it's more network, but we use routing to find each other. (network has so far been point-to-point.) we've discussing moving the "networking forming + peer finding " parts of the "routing interface" into the "network (libp2p) interface"
<lgierth> yeah nice
<whyrusleeping> jbenet: i still want 1187
<lgierth> nat-busting is and webrtc-style finding is new to me, in cjdns everything is one big lan
<lgierth> i mean i'm familiar with the concept, just have never really done it
<jbenet> lgierth: yeah, maybe we should just do that, rip that part out of routing and put it in network. maybe split network into "(a) peer discovery, (b) establushing point to point links, and (c) achieving maximal connectivity (what's now in routing)
<jbenet> lgierth: yeah i want to expose a cjdns-style world to the higher layers, where it doesn't matter at all.
alexandria-devon has joined #ipfs
<whyrusleeping> lgierth: ipfs feels like one big LAN if you use corenet
<jbenet> lgierth the issue is discovery of new peers, who could be behind nats, and who randomly change addresses, etc.
<whyrusleeping> every peer is dialable by their peerID
<jbenet> Luzifer: would you mind posting in the related go-ipfs issue that this happend + got solved by that script?
<jbenet> !pin QmQNjyU8wFKPFNKeU9bCnJi7KjargvLLjrrBzXnJ5xVufj
<pinbot> now pinning QmQNjyU8wFKPFNKeU9bCnJi7KjargvLLjrrBzXnJ5xVufj
<pinbot> pin QmQNjyU8wFKPFNKeU9bCnJi7KjargvLLjrrBzXnJ5xVufj successful!
<jbenet> pinbot <3
<whyrusleeping> hes getting really good
<Luzifer> jbenet: sure. do you have a link for the issue?
<jbenet> pinbot i should've never doubted you.
<jbenet> yeah
<jbenet> Luzifer no, i can look for it, but my bw isn;t great atm
<lgierth> whyrusleeping: sure but after finding its network addresses you still need to figure out whether you can actually connect to them, right? and otherwise use routing through other nodes
<Luzifer> jbenet: I'll look after I finish my statement for my hoster
<whyrusleeping> lgierth: ideally, the corenet package has no idea of whats going on beneath
<jbenet> whyrusleeping: on 1187, not sure. i dont like the name. but i do think that fancier functionality (like bubbling as wking mentions) is definitely desired
grncdr1 has quit [Quit: Leaving.]
<jbenet> whyrusleeping: maybe we can get over the hump of making some ipfs commands be shell scripts and use the one wking provided?
<whyrusleeping> jbenet: i dont like the name either, and the bubbling isnt really what i'm worried about
<jbenet> Luzifer thank you for fixing this, sorry about the trouble.
<whyrusleeping> its the live modification of the ipnsfs system
<jbenet> oh interesting
<whyrusleeping> that 'save' method will live modify the fuse fs at /ipns/local
<Luzifer> jbenet: that's the risk being involved in new software. ¯\_(ツ)_/¯
therealplato has quit [Ping timeout: 265 seconds]
<jbenet> lgierth: "routing" is unfortunately overloaded. there's "routing in the network" (the cjdns style routing) and "routing to find content" (dht-style "routing of queries to find content)
<lgierth> "net routing" "dht routing"
<ipfsbot> [go-ipfs] jbenet closed pull request #1299: implement patch command (master...feat/patch) http://git.io/vkWAS
<jbenet> ipfs object patch's in!
<wking> whyrusleeping: why not leave the IPNS update to a separate (post-bubble) call?
notduncansmith has joined #ipfs
<lgierth> what do you folks use to get a grasp of a big go codebase? sublime and gosublime don't seem to help a lot in grasping a big codebase
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet> lgierth well sometimes it's not a dht, but a gossip protocol. some academic papers call it "content discovery", but this is clunky. maybe we can call it "record routing"
<jbenet> cause that's what it really is.
<kyledrake> jbenet how is the IPFS http gateway determining content-type?
<jbenet> it's routing to find bits and pieces of information that tell you where to go next.
<jbenet> kyledrake i think it MIME sniffs atm.
<jbenet> kyledrake which i know isn't great practice, but std go http server does
<lgierth> jbenet: i.e. get closer in keyspace to the destination
<lgierth> ?
<whyrusleeping> wking: because ipnsfs handles the bubbling in a way that doesnt break the fuse interface
<wking> And 'ipfs name publish ...' does break the FUSE interface?
<jbenet> lgierth: we break it apart into smaller modules until it isn't hard to grasp. whyrusleeping is making a tool to use ipfs for our deps (instead of godeps) once that's in we can shard the repo into components that are well scoped.
<kyledrake> jbenet cool, thanks. there's no better solutions, unfortunately. IIRC S3 makes you tell it the mime type
<jbenet> lgierth: that's one of the things, but this also means putting IPNS record onto the dht, so then we use "record routing" to (a) find the record, (b) get the record, (c) use the record to find something else.
<whyrusleeping> wking: ipfs name publish breaks it
<jbenet> (and here "the dht" can be replaced by "the gossip protocol", "mdns", ...)
<jbenet> whatever (what we've been calling) "the (content and peer) routing system" us
<jbenet> is*
lgierth_ has joined #ipfs
<jbenet> this could use a better name
<lgierth_> last was <jbenet> (and here "the dht" can be replaced by "the gossip protocol", "mdns", ...)
<lgierth_> yeah that sounds good
<ipfsbot> [go-ipfs] whyrusleeping force-pushed builder-default from 8cb34bc to aa16d6f: http://git.io/vkx9M
<ipfsbot> go-ipfs/builder-default aa16d6f Jeromy: make the default repo for corebuilder work
<wking> whyrusleeping: that seems like an important bug (I guess it's https://github.com/ipfs/go-ipfs/issues/1186), but it sounds independent of a save UI to me.
<jbenet> lgierth_: '<jbenet> whatever (what we've been calling) "the (content and peer) routing system" is'
anshukla has quit [Remote host closed the connection]
<Luzifer> added my details to #1226 and giving my hosters staff the hint if they got questions they might come here and ask
<jbenet> kyledrake: we'll have a metadata dag object soon, which will allow mime types.
<jbenet> (its a small object that points to another, adding things like mime types, unix perms, etc)
<jbenet> Luzifer: thanks so much :)
<kyledrake> Yeah, that seems reasonable.
patcon has joined #ipfs
<whyrusleeping> jbenet: we have a metadata dag object already
<whyrusleeping> just no CLI to play with them
<jbenet> whyrusleeping: ah
<Luzifer> okay statement is put into the ticket... lets see what they say about it.
guest756 has quit [Ping timeout: 264 seconds]
<kyledrake> I'll have a quick node.js subdomain proxy within the hour
<jbenet> i think a possible problem here is the current structure of the merkledag object. its clunky to do the links + data as arrays of objects and so on. if there was a nicer, more succinct, more json-y expression, that may be a thing to look into.
<Luzifer> and now back to bed... gn8 everyone!
<whyrusleeping> Luzifer: sleep well!
anshukla has joined #ipfs
lgierth has quit [Ping timeout: 245 seconds]
lgierth_ is now known as lgierth
<jbenet> (one idea that's corssed my mind is make it one dictionary, with a well-known magic key for the data segment, and the constraint that all link names must be different (so we number them when names arent important, which makes them resolvable in a path, which now they are not...)
anshukla has quit [Remote host closed the connection]
<jbenet> the well known magic data key is annoying.
<jbenet> but if we go the JSON-LD @type or @context route, we could use @data
anshukla has joined #ipfs
guest756 has joined #ipfs
<whyrusleeping> jbenet: we could just enforce that the first character of a merkledag link name has special meaning
<jbenet> interesting idea
<wking> ^ I like this better than making it impossible to have files named @context, but I don't see the point of keeping the information in one string
therealplato has joined #ipfs
<wking> why not just add more explicit fields to the Link type?
<whyrusleeping> because we can!
<jbenet> well i would still like a "mounted merkledag" to give me the spec when i `cat <hash>/@type`
<ipfsbot> [go-ipfs] whyrusleeping force-pushed builder-default from aa16d6f to 6b16981: http://git.io/vkx9M
<ipfsbot> go-ipfs/builder-default 6b16981 Jeromy: make the default repo for corebuilder work
<wking> This is basically the "extensible Link" discussion from https://github.com/ipfs/ipfs/issues/36
<wking> jbenet: how do you access the file named '@type' (that's not a spec)?
<whyrusleeping> wking: 'cat @type'
<jbenet> wking: right, you don't in the raw merkledag.
<jbenet> wking: unixfs could transparently fix this
<whyrusleeping> its okay to have random binary data
<whyrusleeping> jbenet and i discussed a little while ago allowing unixfs file leaf nodes to not have unixfs formatting
<whyrusleeping> it would make some metadata stuff awkward, but we could do it pretty easily
guest756 has quit [Ping timeout: 264 seconds]
<wking> hmm, so some UIs will use the @type link that points at a type, and other UIs use the @type link that points at a file/subdir?
<jbenet> whyrusleeping: the concern Tv` and wking share is that making the links be @context and @type and so on is that we then lose the ability to have _other_ things called those names, so it dirties up the namespace. FWIW, i think reaching 1:1 mapping to JSON and in particular JSON-LD is a really, really powerful thing. more powerful than breaking other files
<jbenet> named @type or @context, but ideally would like to find a good fix.
<jbenet> wking: that's one possibility, i don't particularly like it.
<wking> me neither ;). I'm just trying to understand what you're saying ;)
<jbenet> ideally, we could use Blame's StackStream and make all the things play in the same object space without problems.
<whyrusleeping> well if you have an @type, i would question whether or not you really need another file/folder named @type
<wking> maybe it's "you can't have files named @type" ;)
<Tv`> i don't understand why people thought that slapping @type into arbitrary json was safe in the first place
<jbenet> (except maybe, one UI for the raw merkledag data, and one UI for the "executed" merkledag data)
<jbenet> (insead of N UIs per object ontology) (like unixfs, git, bitcoin, ....)
<Tv`> does http://www.w3.org/TR/json-ld/ talk at all about json objects containing keys that start with @ already?
<jbenet> Tv` it's actually a brilliant idea. i know you dont like it, but it is amazingly useful and it's the most pragmatic way to fix the massive unlinked API problem. we shouldn't discuss this, we fundamentally disagree.
<Tv`> i don't like namespace collisions
<Tv`> it's pragmatic for the usual cases where the top-level json doesn't contain unknown keys
<Tv`> -> make the top-level of your ipfs json representation behave like that, then
<Blame> well, that leads to an interesting question. How would I encode the idea of a "directory" into bytecode? Should I just cough up the UnixFS equivalent?
<jbenet> i don't like non-linked data and i dont like forcing people to change their formats _too much_. forcing _everyone_ to wrap an object is IMO worse than forcing a few people to chose a different key.
<Tv`> {"@type": "...", "links": [ .... ], "data": "base64'ed stuff" }
guest9651 has quit [Ping timeout: 264 seconds]
<jbenet> Blame: yeah! \o/
<jbenet> Tv` yeah that's a possibility, which krl favors strongly
<wking> jbenet: And you don't have to set the explicit "type" entry if you don't want to use it
<wking> so it seems compatible with the existing Merkle object spec
<Blame> the git merkel dag approach would work for files.
<jbenet> what i was discussing above is an attempt to gett to _a single_ dictionary per object, without needing to nest.
<jbenet> (but it may be fraught)
<Tv`> hell, you can intermingle "links" and the data, already
<Tv`> what key would you put the data under?
<jbenet> Blame: unixfs is that
<Tv`> same collision
<jbenet> Blame: https://github.com/ipfs/go-ipfs/blob/master/unixfs/pb/unixfs.proto -- but it's not a nice program inside the merkledag itself.
<Tv`> erf, i meant "you can't" above
guest965 has joined #ipfs
<Tv`> doing too many things
<jbenet> right
<jbenet> :)
<jbenet> the loss of universality is a pain, so wrapping may be the only way. but i want to think through this all now.
<Blame> that uint64 will catch up with you someday. How future proofed do you want to get?
<jbenet> if we don't i'll regret it massively in 10 years.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Tv`> Blame: file size greater than 2**64?
<jbenet> Blame: unixfs format can be replaced entirely by something programmed within StackStream
<jbenet> Tv`: yeah, like PI.
<Tv`> PI?
<jbenet> yeah, PI, some merkledag interfaces will compute files out.
<jbenet> low kolmogorov complexity representations
<jbenet> compression, encryption, etc.
<Blame> One of the ideas going into stackstream was future proofing against increases in storage space, memory, and speed in a forever sort of sense.
<jbenet> but also, computation of infinite files.
<jbenet> Blame: yes, we must future proof completely
<jbenet> so ignore that uint64 :)
<jbenet> scratch work! \o/
<Blame> rather than picking a high enough constant that 'should be good forever'
<wking> jbenet: one problem with an explicit "type" field on par with Links and Data is that you're stuck when someone else asks for other metadata from http://www.w3.org/TR/json-ld/#h3_syntax-tokens-and-keywords ;)
<Tv`> wking: map<string,string> // done!
<Blame> because the 'constant that should be good forever' has been historically disproven.
<wking> Tv`: is that {namespace},{name}?
<jbenet> wking: can actually use an indirection there, so one key would point to an object that has all the metadata people ask for. IIRC, json-ld allows you to _only_ put the @context without dragging the semantic kitchen sink in
<jbenet> but still have the power to add it from the context spec
<Tv`> wking: you asked for what json-ld can do
<jbenet> (or other linked objects)
<jbenet> Blame: thank you. somehow i feel we'll never stop having to remind people of that
<wking> jbenet: if you offload both the metadata and the metadata-spec into "type" link, you're going to have trouble untangling them
<Tv`> note the weight calculation
<Tv`> also, if you want to argue about protobuf extensibility, learn the wire format; that uint64 doesn't mean what you think it means
<Tv`> future-protobuf could perfectly happily expand that into uint512, for all the wire format cares
<jbenet> Tv` yep, varints are one reason i like protobuf.
guest4491 has quit [Read error: Connection reset by peer]
<wking> jbenet: but maybe you mean: {"type": "{hash-for-metadata-spec}", "links": {"language": "...", ..., "data": "{hash-for-data}"} and then the data object just has {"type": "hash-for-data-spec", ...}
<wking> so "add a wrapping metadata object if you need more metadata than an explicit-type hash"
<Blame> essentially, the fundamental issue with Content Based Addressing is that until you can safely address many orders of magnitude more than the space of potential content, you WILL have a collision.
<jbenet> Tv` hahaha lovely blog post
<jbenet> wking yeah something like that could work
<jbenet> Blame: indeed
<Blame> Its _Inter-Planetary_ file system
<jbenet> exactly.
<Blame> take all senses of previous scale you had and chuck them out the window.
<Blame> the ZFS blog post talks about 128-bit being good enough until the author dies. This stops making sense when you have a non-zero odds of becoming effectively immortal in your lifetime (I would sadly not rate them too good though...)
<ipfsbot> [go-ipfs] jbenet closed pull request #1317: make the default repo for corebuilder work (master...builder-default) http://git.io/vkDIE
<jbenet> Blame: hahahhahahhahah should write an equally hilarious rebuttal.
<jbenet> <3
<Blame> We can design something that would work for a grandkids, or for a little more effort, we can design for civilizations to come.
<jbenet> Blame: exactly. it's often astounding how small the delta actually is.
<Tv`> i sure hope my non-existent grandkids will not boil my oceans
<jbenet> Blame: what would it take to make an impl of StackStream we can test it? and how do we evaluate whether this is the best language for the future?
<jbenet> Blame: someone recently suggested having something like a "multibytecode" where one a prefix determines the language, just in case.
<jbenet> multicode
<jbenet> or something.
<Blame> that would make a lot of sense. In the #!/bin/bash sense
<jbenet> Blame yeah, cause then we can adopt StackStream and worry about it later.
<jbenet> (and can always have code in StackStream that implements the other languages, just in case no native bytecode impl exists)
<jbenet> (so that people can write native impls or hardware to make things better, but can always default back to stackstream in software)
<jbenet> or whatevr.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> jbenet: so, the record spec is a little confusing...
<whyrusleeping> also, what happens when two records have different validity schemes?
<whyrusleeping> how do they get compared?
<whyrusleeping> should each 'record validity schema type thing' be its own package?
<Blame> jbenet: actually, I am going to write that rebuttal...
<jbenet> hahahah sgtm. careful about the hornet nets you may poke though :)
domanic has joined #ipfs
<wking> whyrusleeping: records including signatures by the node they apply to can have sane handoffs between validity schemes. I don't think that's written up in the spec yet though.
<jbenet> whyrusleeping: ah damnit, two records with different validities-- we need to totally order that. it's not as easy as a hack like order them from the byte representations of the validity scheme field, because then may find a lower hash and invalidate other records forever if the record sticks around. perhaps the simplest ordering hack there is to have a last
<jbenet> recourse of an incrementing varint, so only compare records with same varint, or if validity schemes differ, use the higher varint. (could be "version" or something)
<jbenet> krl thoughts o/
<jbenet> wking: yeah though one use case i want to guard against is "i did something stupid but dont want to burn my name forever"
<jbenet> wking: e.g. i publish a record with validity scheme A, but i really meant to do B.
<jbenet> wking thoughts on the above, too o/ ?
<wking> so you sign and publish a new message saying that.
<wking> and before it gets out to clients you just look silly ;)
<whyrusleeping> wking: what if my previous record is valid forever
<whyrusleeping> and i change validity types
<wking> your new record (that says "oops, please ignore that other record and it's scheme change") will be more valid
<whyrusleeping> why?
<wking> because it references the broken record
<wking> so it's unambiguously more recent
<wking> of course, a censor could maliciously hide your fixup record and continue only serving the broken record forever
<wking> but avoiding censorship is a separate issue
<whyrusleeping> so, now say that ive updated my record
<whyrusleeping> a few times after the bad one
<whyrusleeping> and now someone takes that bad record and re-broadcasts it
<whyrusleeping> nobody has any reason not to think its good
<wking> you have to keep the "please ignore bad records ..." stuff for as long as those bad records could be valid
<wking> ^ good reason to not set infinite lifetimes ;)
<jbenet> wking: what you say means specifically a record that has an ancestry chain, implying all records should have it (which krl favors)
<whyrusleeping> and i dont mind
<kyledrake> jbenet whyrusleeping much to my frustration, I have just learned that browsers downcase domain names. What a wonderful feature!
<jbenet> wking: i'd like to allow creation of records without an ancestry chain
<whyrusleeping> kyledrake: o.o
<jbenet> kyledrake: oh man, forgot about that.
orzo has quit [Ping timeout: 240 seconds]
<kyledrake> So this one's off the table ;)
<wking> revocation lists, not ancestry chains
<jbenet> kyledrake: yeah it's going to bite us someday in HDF osx. i silently hope they'll fix it.
<jbenet> kyledrake: no, use a base32 encoded multihash.
<jbenet> kyledrake: have been thinking about prefixing our strings with the encoding, too. not sure what TRTTD is because it can get out of control fast with the self description, but encoding may make sense.
<jbenet> kyledrake: "this is a base58 encoded multihash of sha2-512 clipped to 256 bits: <digest>" vs "this is a base16 encoded multihash of sha3 clipped to 256 bits: <digest>"
<whyrusleeping> wking: you cant guarantee that 'revocation lists' will stick around
<jbenet> yeah revocation lists have the same problem.
<wking> whyrusleeping: why not? Just publish your list of revocations with every new IPRS entry until the revoked records expire.
<jbenet> and cant even explicitly revoke to peers. the record may stick around because rogue peers may continue to serve it
* whyrusleeping prefers a sequence number on records
<wking> jbenet: does it matter if they still serve revoked records? This is for tie-breaking when a client gets the old and new records
<jbenet> i want to design IPRS so that malicious nodes can serve whatever records they want and as long as you can reach _a few_ honest nodes with the "most valid" records, you're fine.
<wking> jbenet: so you get the old record from a malicious/stale node, and the new record that revokes it from an honest node, and can easily see to use the new record
<jbenet> whyrusleeping: do you want seqno to replace most validity ordering, or have seqno as i described (to mean sort of a "shell" or "latest version" of validity)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet> this is vaguely reminiscent of paxos / pbft view stamps.
<kyledrake> jbenet there is a specific subset of base64 encoding "url safe base64" that may be what is needed
<jbenet> wking yeah that makes sense but then revocation lists cant be thrown out.
<jbenet> wking never know when a bad record claiming to be valid forever will be served
<wking> not until the broken records expire, but just set shorter expiration times
<wking> it's a risk you take when you decide to publish a valid-forever record ;)
<jbenet> wking: some records dont expire
<kyledrake> "Base 64 Encoding with URL and Filename Safe Alphabet" in RFC 4648. The alphabet uses ‘-’ instead of ‘+’ and ‘_’ instead of ‘/’.
<jbenet> wking: yeah but people will screw up
<jbenet> wking: its like adding a handbrake or not to a car. an int seqno that's always there helps.
<wking> and a few "oops" entries in a revocation list aren't going to kill anyone
<jbenet> kyledrake: yeah sadly we didnt have 1 more letter in english.
<whyrusleeping> but thats ugly
orzo has joined #ipfs
<wking> jbenet: folks who are comfortable with a sequence number can use (and stick with) a validity scheme that includes a sequence number.
<jbenet> kyledrake makes me want to go back in time and convince the anglosaxons to adopt ñ or ö or keep ß