<jbenet-on-plane>
sometimes things arent broken -- im a bit disappointed ASN.1 is not as popular anymore. it still has improvements on modern counterparts
bsm117532 has quit [Read error: Connection reset by peer]
<lgierth>
jbenet-on-plane: and it's also what you would end up with resolving /dns addresses -- a list of /ip4 and /ip6 addresses
<lgierth>
<- big fan of a resolve step
<lgierth>
jbenet-on-plane: you could probably also model ACLs with a "maddr set"
<Ralith>
jbenet-on-plane: the really modern stuff like flatbuffers and cap'n proto is all about forwards-compatibility
<Ralith>
which is very good stuff to have
Peeves has joined #ipfs
ygrek has quit [Ping timeout: 264 seconds]
clownpriest has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
clownpriest has joined #ipfs
clownpriest has quit [Client Quit]
cwahlers_ has joined #ipfs
cwahlers has quit [Ping timeout: 265 seconds]
wallacoloo has joined #ipfs
dignifiedquire has quit [Quit: Connection closed for inactivity]
anonymuse has joined #ipfs
<jbenet-on-plane>
lgierth: know where the "<keytype> <base64-keybits> <comment>\n" ssh pubkey file format is defined?
<achin>
the authorized_keys file? there's a useful section in the man page for sshd(8)
fleeky__ has quit [Quit: Ex-Chat]
zat0 has joined #ipfs
<jbenet-on-plane>
thanks achin! just what i was looking for
<achin>
cool
<jbenet-on-plane>
achin: im defining multikey and thinking through what the encoded bits will be
<jbenet-on-plane>
achin: trying to avoid pem/der/ber and all that madness if i can. and looks like SSH is one of the few systems that does.
<achin>
i assume the keybits will be encoded all the same way, but there'd be additional data attach to the encoded bits to provide the necessary metadata?
<jbenet-on-plane>
achin: yeah-- so, what is the necessary metadata here?
<jbenet-on-plane>
achin: as i see it, there's two levels: encoding keys, and encoding values and relationships of cryptographic artifacts
<jbenet-on-plane>
so "encoding keys" fits into the latter, but it seems to be a wide enough use case to merit a separate format-- and ssh's format is a good example of this-- they dont use full PEM or X.509 certs
<jbenet-on-plane>
these two levels-- as i see it -- are basically "mulikey" (<key-type><key-len><key-data>) and "keychain" (IPLD objects linking keys to signatures to algos, to other keys, ...
<jbenet-on-plane>
but im wondering if i'm missing anything in multikey
<achin>
there my be subtly in defining keytype (thinking about futureproofing, if new keytypes get created in the future)
<Ralith>
storing cryptographic keys in text seems super weird
<jbenet-on-plane>
Ralith: im not proposing text-- im proposing binary encoding. but ssh and most HTTPS systems store keys in simple text files
<jbenet-on-plane>
achin: example?
<Ralith>
jbenet-on-plane: then you can understand why saying things like "ssh's format is a good example" might make people think otherwise :p
<Ralith>
but indeed, metadata aside, there's no need for a key file to be anything but a block of raw binary data
anonymuse has quit [Remote host closed the connection]
<Ralith>
afaik openssh keys basically are that except in base64 or whatever for some reason
<achin>
well, ssh public keys are copy/pastable over ssh links, which turns out to be useful in practice
<Ralith>
that's a failure of tooling, more than anything
<achin>
perhaps, but still useful i argue
<Ralith>
you could easily write a ssh-authorize-key script or something that did the same thing but binary-safe
* Ralith
shrug
<achin>
in the past, i've certainly done things like received public keys from friends via gists, and copy/pasted them into authorized_keys files
Aranjedeath has quit [Quit: Three sheets to the wind]
<Ralith>
it is a practical reality that a long string of poor design decisions have made it often easier to juggle base64 than binary
<Ralith>
this is not unfixable and probably should not be a guiding principle of new development
<achin>
so i confess i'm not sure what kind of keys are targeted for a "Multikey" format
<jbenet-on-plane>
Ralith: completely disagreed, we have to design to deal with the environment we
<jbenet-on-plane>
we're in, hence base encodings to maximize readability, copy pasting, etc.
<Ralith>
that's a very strange notion of readability
<Ralith>
and in any case, you did just say you were proposing a binary encoding...
<jbenet-on-plane>
Ralithy: fine, "visibility"
<jbenet-on-plane>
achin: ed25519, rsa, for starters
<Ralith>
is non-ascii data invisible? I don't get it
<jbenet-on-plane>
Ralith: which will be encodable and often represented in some other base encoding, eg base58
<jbenet-on-plane>
Ralith: yes, many ascii control characters do not render
<Ralith>
everything's representable in textual encodings, that's not really relevant except insofar as that it means you don't need to build systems to use such encodings internally (thank god)
<jbenet-on-plane>
-------
<Ralith>
(not that this stops people)
<Ralith>
not sure what control characters have to do about anything either, we're talking about data...
<jbenet-on-plane>
Ralith: im going to stop discussing this because we're way to far away from understanding, sorry.
<Ralith>
that's your prerogative
<jbenet-on-plane>
--------------
<jbenet-on-plane>
achin: just those two key types for now, others possible
<achin>
jbenet-on-plane is now speaking on morse code
<jbenet-on-plane>
multikey is looking just like multihash, actually.
<achin>
yeah
<jbenet-on-plane>
perhaps this structure (<varint-type><varint-len><value>) should be a more general data encoding thing
<jbenet-on-plane>
its the minimal amount you need for "multiformats style" self description
* Ralith
imagines jbenet-on-plane trying to make sense of a jpg in a text editor and being very frustrated
<achin>
a few of the formats like asn.1 can encode arbitrary structures, right?
<jbenet-on-plane>
achin: yep. asn.1 is an unfortunate failure. lots of good ideas, but huge scope creep
<jbenet-on-plane>
protobuf is a swing too far the other direction-- too inflexible
<achin>
why do you call protobuf inflexible?
<jbenet-on-plane>
imagine something right in the middle would do pretty well.
<Ralith>
I'm pretty sure the main lesson learned from protobuf was that it was *too* flexible
<Ralith>
lots of ways to shoot yourself in the foot long-term
<Ralith>
you might be interested to read Kenton's retrospective thoughts
<achin>
but if you were careful about planning for the future, you could have a pretty reasonable forward compatiblly story (perhaps with a lot of work, though)
<achin>
do you have a link to that handy, Ralith ?
<jbenet-on-plane>
achin: inflexible in a different way-- its hard to use if for self-describing protocols, the streams part is left ambiguous, and it's hard to use for humans
<jbenet-on-plane>
Ralith: im very familiar with kenton and his thoughts on it. i agree with some and disagree with some.
<achin>
for sure the decoding side needs to know what format to expect. i think that's where the compact encoding comes from
<Ralith>
what do you disagree with in particular?
<jbenet-on-plane>
achin: sure, but you can achieve both-- you can have systems that can fetch context when it is not known
<jbenet-on-plane>
achin: imagine a protobuf endpoint that begins the stream with some compact repr of the schema (protobuf doesnt have such a thing, but imagine it)
<jbenet-on-plane>
achin: then programs and "tool-assisted humans" can easily make sense of data and endpoints they've just found for the first time
JesseW has joined #ipfs
<achin>
yes, i see there are some usecases when a self-describing format is useful
<jbenet-on-plane>
you dont have to sacrifice that for compression
<Ralith>
achin: cap'n proto is fiendishly clever about forwards-compat, it's very nice; messages are *just* self-describing enough that they can be manipulated safely by outdated (or even entirely zero-knowledge) code
<jbenet-on-plane>
the web uses JSON, despite protobuf's existence, for a reason: humans manipulate it all the time
<Ralith>
e.g. a cap'n proto message has a normal form that is defined independently of its schema
<Ralith>
the web uses JSON because JSON is already built into every web browser :P
<jbenet-on-plane>
Ralith: yep that's a cool feature.
<Ralith>
jbenet-on-plane: I'm interested in where you disagree with Kenton's principles
<achin>
i like fiendishly clever things
<jbenet-on-plane>
Ralith: no, it doesn't. it is quite easy to ship encoders in javascript. the web uses json because people ran away screaming from xml
<jbenet-on-plane>
but still wanted a human readable format, that self-described and allowed easy, standalone manipulation
<Ralith>
"quite easy" is a gross oversimplification, and at any rate there's not much incentive to bother when you already have something that works
<jbenet-on-plane>
Ralith: too many to mention, honestly, i have other things to do to write that out. also not very useful to, kenton and co value things very differently so this won't achieve synthesis of efforts, and that's fine. (for context, i've had many conversations with kenton and others in the protobuf/capnp school)
<Ralith>
the web is full of stories to the effect of "we replaced JSON in our haphazardly-scaled system with something else and suddenly everything didn't suck anymore"
<achin>
in general, it seems like it's rare when things don't suck :P computers are hard
<jbenet-on-plane>
achin: computers are hard indeed. made harder by historical development, but still.
<achin>
and a corollary: software development is hard :P
<achin>
but headed in the right direction, i think!
<Ralith>
by process of natural selection if nothing else
<achin>
sort of. there are some people who wish that things like c++ die more quickly than they might naturally
<Ralith>
then they should hurry up and build something that can replace it :P
<achin>
they are!
<achin>
the rust team is applying an impressive amount of resources to push things in this direction
<achin>
maybe the same could be said for the swift team, but i don't know their goals as well
<Ralith>
and they are heroes for it
<Ralith>
I hope it works out
jbenet-on-plane has quit [Read error: Connection reset by peer]
<achin>
i hope so soo
<achin>
too*
jbenet-on-plane has joined #ipfs
herzmeister has quit [Quit: Leaving]
PrinceOfPeeves has quit [Quit: Leaving]
herzmeister has joined #ipfs
zat0_ has joined #ipfs
zat0 has quit [Ping timeout: 260 seconds]
<achin>
i'm writing some code to interact with a json api, and i'm a bit annoyed that this nice self-describing json file doesn't actually self-describe the expected schema
structuralist has quit [Remote host closed the connection]
ygrek has joined #ipfs
<JesseW>
achin: which one?
<achin>
well, no json does this, but i'm working with the gitlab API at the moment
<JesseW>
ah
<jbenet-on-plane>
achin: graphql is pretty sweet for apis. i would imagine gitlab may adopt it soon.
zat0_ has quit [Read error: Connection reset by peer]
zat0 has joined #ipfs
<Ralith>
"self-describing" APIs are the worst, because it just means you have to guess what the schema is and since the author will almost ever have looked at a formal description in one place it's usually a mess
<achin>
id' just be happy with accurate docs
<Ralith>
the tooling associated with e.g. json encourages people to develop without any conformance checking, so even docs tend to be approximate at best
<Ralith>
"friendly to a human reader" is much too close to "requires a human reader" for comfort
<achin>
what format has tooling that encourages docs and schemas and all that?
<Ralith>
pretty much anything schemaful (asn.1, protobufs, flatbuffers, capnp, ...)
<achin>
i've not used it personally, but there are json-schema validators
<Ralith>
they exist, but if you write a program that speaks a schemaful encoding, then you're generally encoding/decoding it through code generated to the schema
<Ralith>
you can't e.g. put a string in a field meant for a number because the "setFoo" accessor only takes a string, or whatever
<achin>
in theory. not all langaguages, projects, or build systems are that sophisticated
<Ralith>
I'm not aware of any way to interact with a protobuf, flatbuffers, or capnp system that doesn't inherently enforce that level of correctness, unless you're going to vastly more effort to use their respective reflection APIs
<Ralith>
I mean, if you're in javascript or something you might crash or do a surprising implicit conversion, but at least you won't generate non-conforming data
<achin>
yes, probably all of those systems have that type of enforcement, even if it's jsut runtime checks
<achin>
but if your API returns, say, XML? you could easily have an impl that doesn't do any validation
<Ralith>
yeah, XML's no better in that regard
<achin>
but it's an interesting case, though
<achin>
because there *are* established tools to describe schemas, and do validation
<Ralith>
you get the good properties as a result of the convention of relying on code generated from a formal specification for encoding/decoding
<achin>
btw, do you know of any web-based API that actually returns protobuf/capnp ? i'm curious
<Ralith>
because the code generator becomes a platform for translating the specification into the programming language
<Ralith>
where it can be statically checked or even correct by construction
<Ralith>
what does "web-based" mean?
<Ralith>
HTTP transport?
<Ralith>
no, most applications I'm aware of don't see any benefit from screwing around with that gigantic pile of complexity :P
<achin>
yeah, http
<achin>
the standard protobuf interface (in the generated code) creates stuff that'll serialize and deserialize to streams, seems like it'd be fairly trivial to plug them in
<Ralith>
I'm working on a capnp encoding for matrix.org which may end up going over http in the short/mid term for legacy reasons
<Ralith>
but that's definitely not ideal
<Ralith>
(unless something like QUIC works out, in which case it may end up being good enough to not be worth screwing with)
<achin>
what are your clients? browsers?
<Ralith>
matrix clients can be anything
<jbenet-on-plane>
Ralith: if you can, define your protocol to go over any transport, including but not limited to http, from the get go
<Ralith>
"go over any transport" is kind of a ridiculous proposition because different transports have wildly different semantics
<Ralith>
certainly there is value in supporting both http and something more specialized, though
<jbenet-on-plane>
sure, i guess unix was a ridiculous idea
<achin>
some apis that are only web are mostly transport agnostic already
<jbenet-on-plane>
amazing how embedded streams like didn't work at all for interprocess and internet communication, thankfully we had complicated specialized protocols to the rescue
<achin>
as input they take some formatted data, as output they produce formatted data
<achin>
if your API uses http specific things like headers, then that's (hopefully) a fairly small shim on either end
<Ralith>
jbenet-on-plane: hm? unix sockets/pipes are great
<jbenet-on-plane>
and i guess i should be thankful that we now have huge bloated like http2 and websockets only to go back to the same ideas
<jbenet-on-plane>
Ralith: i was being sarcastic-- "go over any transport" is precisely the unix idea
<Ralith>
that's certainly an interpretation
<Ralith>
it sounds like you might be using "transport" to mean "stream"
structuralist has joined #ipfs
<jbenet-on-plane>
no i do mean transport-- a duplex connection with specialized semantics
<jbenet-on-plane>
(well not always duplex)
<jbenet-on-plane>
but the point im making is: don
<achin>
unix pipes don't really have specialized semantics, do they?
<jbenet-on-plane>
don
<achin>
they are just sinks and sources for bytestreams
<Ralith>
well, I assure you that if you blindly stuff something that requires stream semantics into UDP you'll have a bad time :P
<jbenet-on-plane>
don't use the HTTP specialized stuff, keep it general, to be overlaid on http but also any other transport
<jbenet-on-plane>
Ralith: not if you use any kind of reliable transport on top of UDP (uTP, UDT, SCTP, DCCP, RUDP, ...)
<jbenet-on-plane>
the point is all of those should be swappable by an application without incurring any protocol hit
<Ralith>
those still aren't streams.
<Ralith>
some of them can implement streams, which is great if that's what you want
<Ralith>
but streams are very rarely the right answer to a specific application
<Ralith>
at least, monolithic streams
<Ralith>
head-of-line blocking is a serious problem
mgue has quit [Quit: WeeChat 1.5]
<Ralith>
for example, matrix clients (and servers) typically have many overlapping, independent requests going simultaneously
<jbenet-on-plane>
i'm well aware of HOL blocking, how stream muxing works, and so on
<jbenet-on-plane>
i think you're too opinionated to see value where we're coming from, but you would find our approaches interesting
<Ralith>
then you appreciate that it doesn't make sense to consider transport and protocol as completely independent
<jbenet-on-plane>
obviously we all want QUIC to deliver us from evil, but to reach all devices and all networks, no single transport or network is enough, and must be flexible-- meaning yes sometimes your protocol will have to be piped over a stream with head of line blocked (eg on the browswer with webrtc or websockets) and you have to deal with it.
<Ralith>
what devices are you worried about that can't be reached by either http or ip?
<jbenet-on-plane>
a server running inside of a browser tab. try dialing http to that
<Ralith>
that's not a use case I have any interest in, but I'm morbidly curious how would structure such a thing
ligi has joined #ipfs
<Ralith>
WebRTC I guess?
<jbenet-on-plane>
WebRTC or Websockets, depends on what the other peers you wnat to talk to are running
<achin>
i didn't realize websockets could accept inbound connections
<Ralith>
neither WebRTC nor websockets alone are sufficient, you'd need some sort of intermediary to negotiate setup
<jbenet-on-plane>
achin: it can't-- you have to dial out, but an inbound connection may be setup via relaying (like webrtc or all the nat traversal stuff)
<Ralith>
I feel like websocket security policies might be an issue too but that's getting outside my expertise
<achin>
with webrtc, after you negotiate the setup, everything is true peer-to-peer, right?
<Ralith>
yeah, WebRTC's pretty nice
<Ralith>
featurewise anyway
<Ralith>
the implementation situation is kind of a mess
<achin>
as long as the concept is solid, i guess i have faith that impl messes will sort themselfs out
<Ralith>
yeah, things are definitely looking promising in that respect
<Ralith>
there's already one standalone implementation available, and I know of at least one other that may be open sourced in the mid term
structuralist has quit [Remote host closed the connection]
ygrek has quit [Remote host closed the connection]
ygrek has joined #ipfs
keks has joined #ipfs
Kane` has joined #ipfs
zorglub27 has quit [Quit: zorglub27]
lidel has quit [Ping timeout: 248 seconds]
lidel has joined #ipfs
ianopolous has quit [Remote host closed the connection]
ygrek has quit [Ping timeout: 276 seconds]
computerfreak has joined #ipfs
Bat`O has quit [Ping timeout: 276 seconds]
lidel has quit [Ping timeout: 272 seconds]
Bat`O has joined #ipfs
rendar has joined #ipfs
KozakIvan[m] has joined #ipfs
computerfreak has quit [Remote host closed the connection]
KozakIvan[m] has left #ipfs ["User left"]
ligi has joined #ipfs
lidel has joined #ipfs
edrex has quit [Ping timeout: 265 seconds]
lidel has quit [Ping timeout: 240 seconds]
chriscool has left #ipfs [#ipfs]
em-ly has quit [Quit: Connection closed for inactivity]
pfrazee has quit [Read error: Connection reset by peer]
chriscool has joined #ipfs
pfrazee has joined #ipfs
Oatmeal has joined #ipfs
chkm8 has quit [Ping timeout: 240 seconds]
abbaZaba has joined #ipfs
pfrazee has quit [Read error: Connection reset by peer]
pfrazee has joined #ipfs
palkeo has joined #ipfs
palkeo has joined #ipfs
palkeo has quit [Changing host]
Encrypt has joined #ipfs
zorba has joined #ipfs
Encrypt has quit [Quit: Quitte]
cyberwolf has joined #ipfs
cyberwolf has quit [Client Quit]
cyberwolf has joined #ipfs
cyberwolf has quit [Client Quit]
zz_r04r is now known as r04r
<victorbjelkholm>
Good morning! Or rather good afternoon maybe
chadoh has joined #ipfs
chadoh has quit [Quit: leaving]
chadoh has joined #ipfs
A124 has quit [Quit: '']
rgrinberg has joined #ipfs
A124 has joined #ipfs
<panicbit-M>
vi
<panicbit-M>
victorbjelkholm: Good day :)
chadoh has left #ipfs [#ipfs]
chadoh has joined #ipfs
chriscool has quit [Ping timeout: 250 seconds]
<chadoh>
Hey all! New to IPFS and IRC. I'm giving a talk at the NYC React meetup in a couple weeks titled "Online/Offline Equivalence: what our interplanetary tomorrow can teach us about building apps today". I'm going to use the thought experiment of how a functional internet on another planet could work to talk about emerging tech today. I'd like to cover Offline First stuff (service workers, PouchDB) as well as IPFS. The whole tal
<chadoh>
ly understand how I'd make shared todo lists, like a distributed Wunderlist. Probably use Ethereum as the data store? Not sure how that works!
<chadoh>
k is only going to be half an hour, so I doubt I'll get too deep into any particular thing, but I would love to be able to show off some cool demos! What are some cool open source dapps y'all know of that are built on top of ipfs? A simple todo list is all I'm really looking for. I'm willing to jump in and build it myself if it doesn't exist. I know how I'd make an app that saves all state to local storage, but don't current
chriscool has joined #ipfs
m0ns00n has joined #ipfs
chriscool has quit [Ping timeout: 248 seconds]
captain_morgan has joined #ipfs
chadoh has quit [Remote host closed the connection]
WhiteWhaleHolyGr has quit [Ping timeout: 265 seconds]
chriscool has joined #ipfs
Peeves has joined #ipfs
clownpriest has joined #ipfs
ppham has joined #ipfs
clownpriest has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
chris613 has quit [Quit: Leaving.]
captain_morgan has quit [Ping timeout: 265 seconds]
clownpriest has joined #ipfs
PrinceOfPeeves has joined #ipfs
chris613 has joined #ipfs
chadoh has joined #ipfs
chadoh has quit [Ping timeout: 250 seconds]
ppham has quit [Remote host closed the connection]
Encrypt has joined #ipfs
Peeves has quit [Quit: Going offline, see ya! (www.adiirc.com)]
computerfreak has joined #ipfs
ljhms has quit [Ping timeout: 264 seconds]
ljhms has joined #ipfs
<keks>
is the gx team in contact with the people working on an official package managing standard in go by any chance?
<keks>
it seems related
Encrypt has quit [Quit: Going to test i3]
Encrypt has joined #ipfs
JesseW has joined #ipfs
chrisg_ has quit [Ping timeout: 265 seconds]
ppham has joined #ipfs
chrisg_ has joined #ipfs
ppham has quit [Remote host closed the connection]
<keks>
I have trouble building go-libp2p-transport. It uses the type "github.com/anacrolix/utp".Addr, which, according to github, never existed.
<victorbjelkholm>
daviddias, dignifiedquire you have any small example on how to use js-ipfs as a lib? Tried just new IPFS() but haven't none of that success...
Encrypt has quit [Remote host closed the connection]
Encrypt has joined #ipfs
ppham has quit [Remote host closed the connection]
ppham has joined #ipfs
pinkieval has joined #ipfs
ppham has quit [Remote host closed the connection]
<pinkieval>
hi
<pinkieval>
I am trying to understand how multistream works
<pinkieval>
in the example at https://github.com/multiformats/multistream#protocol , one of the peers sends “/ipfs/QmVXZiejj3sXEmxuQxF2RjmFbEiE9w7T82xDn3uYNuhbFb/ipfs-dht/0.2.3” followed by DHT traffic. But how does it know the other peer is able to understand this protocol?
Foxcool has quit [Ping timeout: 260 seconds]
<dignifiedquire>
victorbjelkholm have you looked at the factory methods in the tests folder?
chadoh has quit [Remote host closed the connection]
kitcambridge has quit [Remote host closed the connection]
quotidien_afk has quit [Read error: Connection reset by peer]
gozala_ has quit [Remote host closed the connection]
ppham has joined #ipfs
ppham has quit [Remote host closed the connection]
ppham has joined #ipfs
ppham has quit [Remote host closed the connection]
<deltab>
pinkieval: aiui, it doesn't, but it'll get either "na\n" or a valid protocol response
ppham has joined #ipfs
ppham has quit [Remote host closed the connection]
<pinkieval>
deltab: but it starts transmiting stuff before getting the 'na', right?
<pinkieval>
(the “<dht-traffic>” in the example)
<pinkieval>
how does the other peer make the difference between that inner-protocol-stuff and a new multistream header?
<deltab>
that seems unspecified
ppham has joined #ipfs
ppham has quit [Remote host closed the connection]
<victorbjelkholm>
dignifiedquire, I did quickly, yeah, but thought that was if you wanted to specify the repo yourself, not for normal usage
<victorbjelkholm>
I'll take a look again and see if I can get it to work
<victorbjelkholm>
ah, I think what I missed was "load" and "goOnline" as well
<victorbjelkholm>
somehow thought init would be enough...
<victorbjelkholm>
richardlitt, I wrote a quick scripts that extracts all links from a markdown file and makes sure they are 200, is that something you are interested in having for ipfs/awesome-ipfs?
ppham has joined #ipfs
matoro has quit [Ping timeout: 272 seconds]
<richardlitt>
There's a bot that does that!
<richardlitt>
Awesome_bot. Should be enabled
<victorbjelkholm>
richardlitt, oh, but then we should have had a error when my nodejs link stuff went offline, no?
ppham has quit [Remote host closed the connection]