<gozala>
lgierth: daviddias another thing that would also help is to have a well reasoned explanation from diverging existing standards
<gozala>
like there are ton of assumptions both in firefox & chromium about scheme + hostPort being an origin
<lgierth>
hah right :)
<gozala>
but then for ipfs origin needs to be root path after the host
<lgierth>
mh?
A124 has quit [Remote host closed the connection]
<gozala>
it’s not only implementation issue, but also user expectation concern
<gozala>
lgierth: fs:/ipfs/hash/
<lgierth>
ah, you mean if it's ipfs-over-http it should only look at the path and not the host
xelra has joined #ipfs
<lgierth>
oh yep
<gozala>
lgierth: presumably you want origin to be fs:/ipfs/hash
<lgierth>
that one doesn't even have a host
<daviddias>
all right, now that I'm on my keyboard, my throughput is way higher (sorry my phone is borked)
<lgierth>
hey david p/
eocarragain has quit [Quit: Page closed]
Nycatelos has quit [Excess Flood]
<daviddias>
As you said, yes, there are several ways we can go about this, and from my current understanding, actually having dags for origin policies is a way safer model than domains all together
neurrowcat has quit [Ping timeout: 264 seconds]
<lgierth>
gozala: for suborigins we're making the suborigin identifier a base32 version of the hash
<daviddias>
speciall since we have cryptographic naming into the mix
<gozala>
not to mention that with multihash hash1 and hash2 could be same origin if they are just different encodings of the same thing
Nycatelos has joined #ipfs
<lgierth>
the same site could be under different hashes too
<lgierth>
the mutable /ipns/example.com, and then the individual /ipfs/hash versions
<daviddias>
but even before jumping into that territory, there is one thing I would like to ensure we are in the same page CIDv0 vs CIDv1
<gozala>
lgierth: daviddias you could have achieved that without necessarily placing hash as after /
fleeky_ has joined #ipfs
<gozala>
not saying you should
jkilpatr has quit [Ping timeout: 240 seconds]
<gozala>
but having a good argument explaining why this is important would be very helpful
<daviddias>
CIDv0 is what people have been calling in 2016 as 'a IPFS Hash', the base58 encoded sha256 hash of the content
<daviddias>
CIDv0 had some shortcomings, one of them being that it doesn't tell us the type of the content (it has no multicodec), which was our motivation to move to CIDv1
xelra has quit [Remote host closed the connection]
<daviddias>
However, in addition to adding a multicodec to the Identifier, we also added a 'multibase' header, to fit specifically these cases, where base58 won't work
<daviddias>
So now, we can represent any Content, using a panoply of different bases
xelra has joined #ipfs
<daviddias>
And CIDv0 can be easily migrated to CIDv1, because it is just adding information
fleeky__ has quit [Ping timeout: 245 seconds]
<daviddias>
CIDv0, the IPFS Hash, doesn't have the information of which base is it using, that is why we can't do 'QmdbMBaYk8s9Ah312pWPLfN8Da2NnRB7tDxWntsD4zJWxz' to be encoded to base16 or base32 directly
<gozala>
daviddias: so is CIDv1 is still an encoding of sha256 hash of the content ?
<daviddias>
However, we can do (new CID('QmdbMBaYk8s9Ah312pWPLfN8Da2NnRB7tDxWntsD4zJWxz')).toV1().toBaseEncodedString('base16')
<daviddias>
CIDv1 is multibase + cid version + multicodec + multihash + hash of the content
<gozala>
daviddias: :) problem is I don’t know what each of these components mean :)
<daviddias>
ah
<daviddias>
Ok, so multibase, is a code that we prefix a string to signal which base it is encoded
<gozala>
daviddias: my understanding is:
<daviddias>
multicodec is a byte that we prefix to tell its data tipe
<daviddias>
multihash is a byte (well, actually a varint) to tell the hash function used
<daviddias>
Creating a full self describable Identifier that lets you pick the things that work for your app or your resolver
<gozala>
daviddias: thanks for explaining, that helps
<daviddias>
:) happy to help
<gozala>
daviddias: so when you explain multicodec
<gozala>
what do you mean by it’s data type ?
<gozala>
data type of what ? in the context of he CID at least ?
<daviddias>
It is the data type of the data that CID is pointing to
<daviddias>
It is 'lifting the file extension to the link'
<daviddias>
Giving us a canonical way to know before hand which data we are fetching from the network, so that the IPLD Resolver (query engine), can traverse through graphs that have different data types
<gozala>
daviddias: right, I read IPLD page yesterday, which also said it’s being redrafted :)
<daviddias>
Can be text, images, video, sound and also, serialization formats, such as protobuf, cbor, ion and others
Nycatelos has quit [Quit: Party like it's December 31st, 1983]
vandemar has quit [Read error: Connection reset by peer]
<gozala>
daviddias: got it
vandemar has joined #ipfs
vandemar has joined #ipfs
vandemar has quit [Changing host]
<gozala>
daviddias: so I guess I still have one question
<daviddias>
Yeah, it is one of our focus of this quarter, lgierth is in charge of making sure we update all our specs :)
<daviddias>
With multicodec on the CID, we are now able to have a `CID/cid-of-git-object/cid-to-eth-block/some-transaction` and execute this as a query, traversing different types of data that exist today, but also it is prepared for new types of data that exist in the future
<gozala>
daviddias: what I find confusing there is that output of cid.toBaseEncodedString() will be different based of the version
<gozala>
meaning v0 or v1
<daviddias>
I see, we need to make it more clear why CIDv0 exists in the first place, essentially, we kept that so that we continue to resolve 'old IPFS hashes'
<daviddias>
When the next version of IPFS gets released
<daviddias>
you will start getting CIDv1 instead of CIDv0 when you add a file
<gozala>
daviddias: hmm.. not sure I do follow that
<daviddias>
The Qm58 Hashes will become more rare, unless someone forces it to use CIDv0
<daviddias>
ah, ok, I believe I understood a different question from your sentence
<gozala>
daviddias: what I mean why not make toBaseEncodedString() return output in some canonical encoding regardless of version of CID it was parsed from
<daviddias>
cid.toBaseEncodedString() - results ina different string for v0 and v1 because:
<daviddias>
v0 is always base58.encode(multihash('sha2-256', 'some content'))
john2 has quit [Ping timeout: 240 seconds]
chris613 has joined #ipfs
<gozala>
daviddias: so if I understand what you’re saying is since v0 does not allow you to specify encoding same as v1 does you just does not allow it
<daviddias>
v1 is multibase(<the base you want>,'v1' + multicodec('<type of data being hashed'>) + multihash('<the hash func you want>', 'your data that gets hashes with the hash func you want'))
<gozala>
daviddias: so from a outsider perspective I’d still expect that cidv0.toMultibaseString() would just return string of version 1
<daviddias>
gozala: exactly. I probably should have written in the code to instead of defaulting to base58 for V0, just throw an error if someone does a cidV0.toBaseEncodedString('base16'), it is clear to me now that should be the case
<lgierth>
yeah it should return version 1, or just refuse to return anything. toMultibaseString() should return a multibase string :):)
jkilpatr has joined #ipfs
<gozala>
and maybe for backwards compatibility you could have cid.base56has() or something
<daviddias>
ok, let's fix that, cidv0.toBaseEncodedString('base16') will throw an Error saying ('Only base58 is supported in CIDv0, migrate to CIDv1 to support any base with .toV1') (<- or something along these lines)
<gozala>
daviddias: this is kind of odd thing to ask but I’d honestly suggest to do something like
<daviddias>
lgierth: there is no toMultibaseString, there is toBaseEncodedString specifically to avoid that assumption :)
<gozala>
if you ever want to have typed version of that stuff or allow usage of this from the say flow-typed project
<gozala>
it’s going to be very type-checker unfriendly
<gozala>
daviddias: well alternative could be
<gozala>
cid.toBaseEncodedString(encoder)
<daviddias>
Understood
<gozala>
where encoder is some type other than string
<kythyria[m]>
Why base 58 of all things?
<gozala>
so you could define type Encoder = Base16Encoder | Base64Encoder ….
<gozala>
daviddias: obviously feel free to ignore this
<gozala>
this kind of feedback maybe too late in the process
<lgierth>
kythyria[m]: it seemed like a sensible choice at one point -- and we only came up with '"it's never gonna change" considered harmful' after that
<gozala>
daviddias: to be clear what that would allow you is to get compile time error when trying to use incompatible encoding with a legacy types
<gozala>
vs having runtime erros
<daviddias>
gozala: It is super valuable, the only question in my head is "when will I find the time to allocate to change every single module that there is that uses patterns like this one throughout the code base", they are everywhere in multiformats
<daviddias>
I'm a fan of types and it would make dev so much nicer, but also, requires everyone to buy into them
<gozala>
daviddias: yeah that’s why I said feel free to ignore
<daviddias>
and we've been balancing between what we can do to be more productive, without creating barriers for new contributors
<gozala>
daviddias: although one way you could do it is to then wrap typed version with helper function
<gozala>
and flow for instance would be able to infer it properly I believe
<gozala>
daviddias: anyway feel free to ignore this point
maxlath has quit [Quit: maxlath]
<gozala>
daviddias: partially the reason I was pointing out this is the protocol implementation I was writing uses flow typed js
structuralist has quit [Read error: Connection reset by peer]
<gozala>
and given that I’d need to use CID there is why I was thinking of it
structuralist has joined #ipfs
dryajov has joined #ipfs
<gozala>
daviddias: would you be willing to take contributions that adds flow types / integration to some of these libs ?
<gozala>
daviddias: that might be a nice way for me to learn some internals
<daviddias>
Is there a way today to add flow types without needing to add transpilation to the code before usage/publish?
<daviddias>
Someone gave me the idea that it is possible to add flowtypes with jsdoc?
<daviddias>
which flow compilers would capture, but someone not using any flow typechecker would use the module just fine?
<daviddias>
Is this doable? (It would be so useful)
<gozala>
@daviddias there is a way to add types through comments, but honestly it’s not worth it
<gozala>
daviddias: We did that in the project I’m working on and ultimately end up moving to just transpiring
<daviddias>
The reason why I'm wary of having a transpilation pipeline for prepublish/prerun/preusage is because we were there in the past by babel transpiling everything to ES5 when Node.js didn't support all the Node.js features in ES6 we wanted and everyone hated the experience of having the source code and the transpiled thing
arkimedes has joined #ipfs
<gozala>
daviddias: I think there is possibly a way to do transpiring live kind of like coffee if you really don’t want to do it during development
<gozala>
daviddias: point is if you add flow to workflow you would have to run typechecker
<gozala>
at which point transpiring is not a big deal
<gozala>
if you don’t type check than what’s the point
<daviddias>
gozala for things that might involve best practices, API design and other things that involve major refactors (which consume a lot of time), we have opted to go into this 'exploratory' discussions, where everyone brings their experience, thoughts, ideas and so forth so that us and the whole community can evaluate if something is good for the health of the
<gozala>
daviddias: thanks so much, that’s awesome!
<daviddias>
no, thank you for reviewing all of this, testing it out and coming with loads of questions and feedback :D
bastianilso_ has quit [Quit: bastianilso_]
<daviddias>
hope that CID is more clean now :) we should strive to present users to CIDv1 from now on
<daviddias>
as they offer all the flexibility browsers need
<daviddias>
as for CIDv0, we can have a service in like cid.io/convert or something
<daviddias>
or a browser extension
Aranjedeath has quit [Quit: Three sheets to the wind]
<daviddias>
that helps users get their CIDS in different bases or migrated to the version
<dryajov>
daviddias: I'm giving this a shot https://github.com/ipfs/js-ipfs/issues/127, but was wondering if you guys have any preference as to what website/example would make sense to use for this...
<daviddias>
So, in terms of a concrete app, the idea is to make it clear that the idea of lazy loading an application from a content addressed network is a thing that is super useful
<daviddias>
So, an app where users can load different pieces themselves
<daviddias>
And app that leverages progressive enhancement
<dryajov>
reason being to save time... but if there is a better use case in the context of the IPFS project I would use that instead... I really don't have a preference there...
<dryajov>
so.. in that sense, the voice memo demo fits the bill nicely
<daviddias>
will make a fantastic demo
<daviddias>
yeah, voice memos are cool, because then each voice memo can be sync'ed through browsers on demand
cemerick has quit [Ping timeout: 248 seconds]
<dryajov>
yep...
<dryajov>
My current approach is to do it over service workers and possibly integrate that into sw-toolbox/sw-precache as a POC...
<daviddias>
I'll be going AFK now, getting later here
<daviddias>
See everyone tomorrow :)
<dryajov>
diasdavid: one question I have is for storing voice messages in IPFS ... I might have to run two instances of IPFS nodes in the browser... one in the SW another in the main app... becuase it might not make sense to handle storing messages in the SW itself...
<daviddias>
Would the SW have access to the same IndexedDb?
structuralist has quit [Remote host closed the connection]
<dryajov>
it should...
<daviddias>
so you don't need both
<dryajov>
well... SW is more or less intended to be a resource proxy, not sure if app logic would make sense to go there... its certainly possible but it might end up being hacky?
structuralist has quit [Ping timeout: 255 seconds]
zopsi has quit [Ping timeout: 240 seconds]
chris613 has quit [Quit: Leaving.]
zopsi has joined #ipfs
lacour has quit [Quit: Leaving]
dryajov has joined #ipfs
dryajov has quit [Client Quit]
wallacoloo____ has quit [Quit: wallacoloo____]
dryajov has joined #ipfs
dryajov has quit [Client Quit]
pfrazee has quit [Remote host closed the connection]
dryajov has joined #ipfs
rendar has joined #ipfs
dryajov has quit [Client Quit]
Caterpillar has joined #ipfs
dryajov has joined #ipfs
dryajov has quit [Client Quit]
wallacoloo____ has joined #ipfs
dryajov has joined #ipfs
dryajov has quit [Client Quit]
dryajov has joined #ipfs
dryajov has quit [Client Quit]
dryajov has joined #ipfs
dryajov has quit [Client Quit]
pfrazee has joined #ipfs
pfrazee has quit [Ping timeout: 252 seconds]
structuralist has joined #ipfs
structuralist has quit [Ping timeout: 248 seconds]
mildred3 has quit [Ping timeout: 255 seconds]
john2 has joined #ipfs
mildred3 has joined #ipfs
maxlath has joined #ipfs
tabrath has joined #ipfs
tabrath has quit [Ping timeout: 248 seconds]
espadrine_ has joined #ipfs
maxlath has quit [Ping timeout: 248 seconds]
pfrazee has joined #ipfs
pfrazee has quit [Ping timeout: 256 seconds]
structuralist has joined #ipfs
maxlath has joined #ipfs
ajsantos has joined #ipfs
wallacoloo____ has quit [Quit: wallacoloo____]
mo has joined #ipfs
mo is now known as Guest85113
ajsantos has quit [Remote host closed the connection]
Guest85113 is now known as moeviz
Encrypt has joined #ipfs
mo_ has joined #ipfs
maxlath has quit [Ping timeout: 264 seconds]
maxlath has joined #ipfs
moeviz has quit [Ping timeout: 260 seconds]
mo_ has quit [Client Quit]
mo_ has joined #ipfs
mo_ is now known as moeivz
moeivz is now known as moeviz
steefmin has quit [Ping timeout: 240 seconds]
Encrypt has quit [Quit: Quit]
ajsantos has joined #ipfs
ajsantos has quit [Remote host closed the connection]
Mizzu has joined #ipfs
moeviz has quit [Ping timeout: 240 seconds]
bastianilso_ has joined #ipfs
pfrazee has joined #ipfs
pfrazee has quit [Ping timeout: 245 seconds]
Encrypt has joined #ipfs
<Polychrome[m]>
Looks like Neocities still does IPFS even though it's not publically exposed right now. They're going to release a big related update soon.
structuralist has quit [Remote host closed the connection]
mrpoopyb1 has joined #ipfs
shizy has quit [Ping timeout: 240 seconds]
arpu has joined #ipfs
spectrophonic has joined #ipfs
UukGoblin has quit [Ping timeout: 248 seconds]
UukGoblin has joined #ipfs
mildred1 has joined #ipfs
espadrine_ has quit [Ping timeout: 256 seconds]
shizy has joined #ipfs
spectrophonic has quit [K-Lined]
chris613 has joined #ipfs
arkimedes has joined #ipfs
shizy has quit [Quit: WeeChat 1.6]
structuralist has joined #ipfs
espadrine_ has joined #ipfs
Encrypt has joined #ipfs
pfrazee has quit [Read error: Connection reset by peer]
pfrazee has joined #ipfs
ianopolous has quit [Ping timeout: 240 seconds]
eater has joined #ipfs
ashark has joined #ipfs
structuralist has quit [Remote host closed the connection]
structuralist has joined #ipfs
ashark has quit [Ping timeout: 240 seconds]
mildred1 has quit [Ping timeout: 240 seconds]
seagreen has quit [Quit: WeeChat 1.6]
seagreen has joined #ipfs
google77 has joined #ipfs
<google77>
hi guys
<google77>
just a heads up...
<google77>
Installing ipfs on the latest mac os is apparently not that easy...One has to go all the way to some boot menu to disable that gatekeeper or whatever it's called...
Mizzu has quit [Ping timeout: 255 seconds]
mo has joined #ipfs
mo is now known as Guest11087
<dignifiedquire>
google77: thanks, I never see this as its one of the first things I turn off when settting up a mac
<dignifiedquire>
but I agree we need to mention it in the docs and should investigate if we can get a certificate to sign ipfs such that there is no warning
<dignifiedquire>
(I think this is a matter of throwing money at apple but I'm ot a 100% sure)
ianopolous has joined #ipfs
G-Ray has quit [Quit: G-Ray]
ygrek has joined #ipfs
<google77>
also while I am not experienced with BSD... on BSD I have to run ipfs as ./ipfs from /usr/bin, not just ipfs from anywhere...IDK, is this normal behaviour on IPFS?
<google77>
*on BSD
ivo_ has joined #ipfs
<FrankPetrilli>
google77: You can add /usr/bin to your path
<FrankPetrilli>
Which flavor of BSD are you running?
<google77>
FrankPetrilli Thx, that's probably what I forgot to do. It is PC-BSD.
mildred3 has quit [Ping timeout: 264 seconds]
iav has joined #ipfs
<FrankPetrilli>
Okay, can you tell me the output of echo $SHELL and echo $PATH?
<FrankPetrilli>
Okay, so your path looks fine. Try running ipfs now.
iav has quit [Client Quit]
<google77>
still nothing
<google77>
it doesn't even tell 'command not found'
<FrankPetrilli>
Interesting. Can you run 'which ipfs'
<google77>
/usr/bin% which ipfs
<google77>
/sbin/ipfs
shizy has joined #ipfs
<FrankPetrilli>
Okay, interesting. What is /sbin/ipfs? Can you run 'md5sum /sbin/ipfs' and 'md5sum /usr/bin/ipfs'
<google77>
There is no md5sum installed
<FrankPetrilli>
Okay, let's do 'mv /sbin/ipfs ~/'
<FrankPetrilli>
Once that's done, run 'which ipfs' again, and if it returns /usr/bin/ipfs, go ahead and run just 'ipfs'
<google77>
Now it runs fine :)
<FrankPetrilli>
Perfect, glad I could help. :)
<google77>
This should be added to documentation.
<google77>
Or somewhere so that people can see it.
<FrankPetrilli>
This is from a fresh install? How did you go about doing the install on BSD?
mildred3 has joined #ipfs
<google77>
still nothingNormal fresh install of PC-BSD on a VM
<google77>
*Normal
<FrankPetrilli>
Sorry, I should clarify; how did you perform the install of IPFS?
<google77>
I installed ipfs as it was in the instructions for Mac and Linux. I installed from the install script.
<FrankPetrilli>
Okay, none of the install scripts should touch /sbin, so I'm downloading PC-BSD to see if I'm missing something. I'll make changes if any are needed.
<google77>
FYI mine was PC-BSD desktop.
<google77>
Meanwhile I will post our solution on an imageboard I frequent' so that it is at least somewhere. It's a rather technical and nice imageboard.
pfrazee has quit [Read error: Connection reset by peer]
<daviddias>
nicolagreco: I think that, for the sake of argument, we should restrain ourselves from having IPLD discussions through IM and only use Github issues
<daviddias>
nicolagreco: it is impossible to get a proper discussion going
<nicolagreco>
daviddias: alright!
<nicolagreco>
my strategy in the past was making my opinion over here and post an issue
<nicolagreco>
but I can post an issue straight away
<daviddias>
I see the value in that, it gets some things resolved right away
<daviddias>
but since there are so many people involved, it is really better for us to have everything down in threads that people can quickly catch up and respond to
<chrono[m]>
well, I'm often finding myself building things I'd like to become "Mars ready"
<chrono[m]>
and veryoften I sadly fail at even basics like reconciling timestamps
<lgierth>
heh -- time in distributed systems is super tricky and we try to avoid it at all costs. there's a couple of time-based timeouts all across the stack that we'd like to see removed rather sooner than later
<lgierth>
i.e. connection timeouts and stuff like that
<chrono[m]>
timeouts are ok
<chrono[m]>
I'd say
<lgierth>
a 30s timeout does get you to mars, but it might not be graceful enough for the subway
<chrono[m]>
but time/datetime will run away pretty soon when combining data from earth days and mars days
<lgierth>
yup you'll have to choose earth time as the canonical time, and suddenly you're centralizing :)
<chrono[m]>
that won't work
<chrono[m]>
most people are having trouble living on UTC
<SchrodingersScat>
stardate
<chrono[m]>
+1
<lgierth>
interesting, thanks SchrodingersScat, i'll pass that along
<SchrodingersScat>
thank star wars captain janeway for inventing it
shizy has quit [Quit: WeeChat 1.7]
<chrono[m]>
I don't think janeway invented it
<SchrodingersScat>
i don't think that was star wars
<chrono[m]>
stardate -> star trek
<chrono[m]>
exists from at least TNG
<chrono[m]>
but when we're really thinking about mars as a target we'll have to come up with something to reconcile this problem
<SchrodingersScat>
thought kirk said it, but I'm no friday the 13th expert
<SchrodingersScat>
thought mars /was/ a target
<chrono[m]>
I don't know enough about original series to be sure, might even have been back then
<SchrodingersScat>
lgierth: and then once it's added I simply make sure I don't delete my .ipfs/blocks/ ?
<SchrodingersScat>
pin it on other machines for redundancy?
ivo_ has quit [Quit: Leaving]
neurrowcat has quit [Ping timeout: 255 seconds]
<lgierth>
an non-pinned object will be removed by garbage collection runs -- they're off by default for now, but in the future we'll probably enable the by default
<SchrodingersScat>
aw, but I love the idea of garbage collection. That's half of why I love tahoe-lafs.
<mguentner>
how well does IPFS perform with small files in general? I am trying to figure out why 30K files @ 4K each (~ 130 MB) amounts to a repo size of 85+ G
<SchrodingersScat>
merkels?
<mguentner>
lgierth: for mars you need a 30 min timeout if you expect an answer :)
vapid has quit [Quit: No Ping reply in 180 seconds.]
vapid has joined #ipfs
Caterpillar has quit [Quit: You were not made to live as brutes, but to follow virtue and knowledge.]
<lgierth>
:))
<lgierth>
mguentner: how did you add those files? i know this can happen if you construct very large directories
<lgierth>
whenever you add/remove a link from that directory, it'll copy that large directory node and applies the change
<lgierth>
we've got directory sharding coming to improve large directories
<lgierth>
that's the last thing blocking npm-on-ipfs
<kythyria[m]>
I guess that also makes a way in which IPFS is subtly different from a standard unixy FS.
realisation has quit [Max SendQ exceeded]
<kythyria[m]>
It's one where you really want to be able to write to it as huge transactions whereupon it applies them in bulk.
maxlath has quit [Quit: maxlath]
<mguentner>
each file: ipfs --api /ip4/x.x.x.x/tcp/5001 add --raw-leaves $f
<mguentner>
lgierth: ^
suttonwilliamd has quit [Ping timeout: 264 seconds]
chungy has quit [Ping timeout: 258 seconds]
chungy has joined #ipfs
<lgierth>
and to the 85G survive GC?
<lgierth>
daviddias: you've changed my mind -- i think we should go with /dns4 and /dns6 for now, these are straightforward and don't need much thought about how to resolve, and we can keep /dns for the future. what do you think? i should probably carry it over to multiformats/multiaddr
<mguentner>
lgierth: will try in a minute
<daviddias>
lgierth: sounds good, I can migrate the code to use /dns4 for now (and support for dns6 too) :)
<lgierth>
and /dns4 /dns6 do address a host unambigiuously (while with /dns there's probably going to be something like RFC6555)