notduncansmith has quit [Read error: Connection reset by peer]
<alu>
jbenet you know what spooked me today
<alu>
i been reading a lot about copyright, IP wars, information society, pirate party, etc
<alu>
im about to move to west coast
<alu>
but just for a moment i realized
<alu>
that ipfs can sort of become sometihing like an evolved wikileaks.
<alu>
and for a moment I thought about those caught in the ashley madison dumps
<alu>
all their embarassing secrets will follow them for the rest of their lives..
<alu>
^_^
<jbenet>
mmmm yes and no. i mean, ipfs doesn't change the situation _much_ on that regard. leaked stuff like that already proliferates and gets served using today's tools. and then those people have the solace that information is only maintained on the internet through the continued expenditure of energy. so there's a total amount of energy they can spend to prevent
<jbenet>
it from being served. (though luckily for freedom of speech, the multipliers are on the side of the information broadcaster. we've come a long way since spoken-only language)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot>
webui/master 243b076 Juan Benet: Merge pull request #75 from ipfs/build-fix...
<ipfsbot>
[go-ipfs] jbenet pushed 2 new commits to master: http://git.io/vstrF
<ipfsbot>
go-ipfs/master d1dd53b Juan Benet: Merge pull request #1587 from rht/feat/read-only-api-gateway...
<ipfsbot>
go-ipfs/master 90b6490 rht: Add test for readonly gateway api sanitation...
<ipfsbot>
[go-ipfs] jbenet pushed 3 new commits to master: http://git.io/vstoI
<ipfsbot>
go-ipfs/master a3a10a4 Lars Gierth: gateway: bring back TestGatewayGet test...
<ipfsbot>
go-ipfs/master 09d7501 Lars Gierth: gateway: make IPNSHostname complete...
<ipfsbot>
go-ipfs/master 3dfe02a Juan Benet: Merge pull request #1577 from lgierth/gateway-host-header...
<whyrusleeping>
jbenet: know when we are going to push the next webui version out?
<jbenet>
whyrusleeping we could do it now
<whyrusleeping>
is it at a good state for that?
<jbenet>
mmm not sure. i can check in a bit
vitzli has joined #ipfs
thomasreggi has joined #ipfs
substack has quit [Ping timeout: 250 seconds]
<M-davidar>
hmph, hdd seems to be failing, total uptime suspiciously close to 10k hrs
substack has joined #ipfs
thefinn93 has quit [Ping timeout: 264 seconds]
thefinn93 has joined #ipfs
<whyrusleeping>
M-davidar: swap it out for a 16TB ssd
<M-davidar>
whyrusleeping: haha, don't tempt me :)
<M-davidar>
any recommendations that won't cost as much as a used car?
<whyrusleeping>
M-davidar: can you wait 6 months?
<ipfsbot>
[go-ipfs] whyrusleeping pushed 1 new commit to node-construct-v2: http://git.io/vstM7
<ipfsbot>
go-ipfs/node-construct-v2 373d334 Jeromy: break up node construction into separate function...
<M-davidar>
whyrusleeping: at the rate I'm getting disk errors, I suspect not
<whyrusleeping>
if not, get a samsung evo 850 500GB
<M-davidar>
whyrusleeping: what's the lifetime of ssd's these days?
<vitzli>
I have 840 EVO 120GB, Power_On_Hours is 5400
<vitzli>
oh, hi all
vitzli has quit [Quit: Leaving]
<whyrusleeping>
M-davidar: a long long time
sharky has quit [Ping timeout: 250 seconds]
<M-davidar>
cool, last time I looked it was much worse than for hdd's (except hitachi hdd's apparently...)
felixn_ is now known as felixn
kbala has quit [Quit: Connection closed for inactivity]
goutham has joined #ipfs
sharky has joined #ipfs
Tv` has quit [Quit: Connection closed for inactivity]
goutham has quit [Ping timeout: 244 seconds]
thefinn93 has quit [Ping timeout: 260 seconds]
thefinn93 has joined #ipfs
goutham has joined #ipfs
Leer10 has joined #ipfs
<M-davidar>
whyrusleeping: ordered, thanks
simonv3 has quit [Quit: Connection closed for inactivity]
mildred has joined #ipfs
thomasreggi has quit [Remote host closed the connection]
pfraze_ has quit [Remote host closed the connection]
goutham_ has joined #ipfs
goutham has quit [Read error: Connection reset by peer]
goutham_ has quit [Remote host closed the connection]
dignifiedquire_ has joined #ipfs
dignifiedquire_ is now known as dignifiedquire
fleeky has quit [Quit: Leaving]
fleeky has joined #ipfs
goutham has joined #ipfs
ZioFork has quit [Ping timeout: 252 seconds]
ZioFork has joined #ipfs
goutham has quit [Ping timeout: 244 seconds]
pfraze has joined #ipfs
goutham has joined #ipfs
voxelot has quit [Ping timeout: 255 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
goutham_ has joined #ipfs
goutham has quit [Read error: Connection reset by peer]
domanic has quit [Ping timeout: 265 seconds]
Eudaimonstro has quit [Ping timeout: 265 seconds]
goutham_ has quit [Remote host closed the connection]
plyp has joined #ipfs
Eudaimonstro has joined #ipfs
zabirauf has joined #ipfs
domanic has joined #ipfs
zabirauf has quit [Max SendQ exceeded]
marianoguerra has joined #ipfs
LennyLinux has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
goutham has joined #ipfs
bedeho has quit [Ping timeout: 244 seconds]
goutham has quit [Ping timeout: 246 seconds]
<plyp>
hi
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
goutham has joined #ipfs
goutham has quit [Ping timeout: 246 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
zabirauf has joined #ipfs
<daviddias>
lgierth ansuz up for lunch?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has quit [Remote host closed the connection]
Encrypt has joined #ipfs
silotis has quit [Quit: No Ping reply in 180 seconds.]
domanic has quit [Ping timeout: 244 seconds]
silotis has joined #ipfs
oleavr has quit [Ping timeout: 246 seconds]
karissa has quit [Ping timeout: 246 seconds]
daviddias has quit [Ping timeout: 246 seconds]
karissa has joined #ipfs
daviddias has joined #ipfs
oleavr has joined #ipfs
compleatang has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<lgierth>
daviddias: hey just got up
<lgierth>
there's a nice burger place at ostkreuz
<daviddias>
lgierth: sounds good. Let me see where ostkreuz is :P
<daviddias>
lgierth: oh super close
<daviddias>
would 1:30pm be nice time for lunch? (so we can warn more people )
<lgierth>
daviddias: yes!
<daviddias>
lgierth: what is the name of the place (to get the addr)
<lgierth>
Burgers
<daviddias>
lgierth: what is your twitter handle?
Encrypt has quit [Quit: Lunch time!]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<lgierth>
daviddias: i don't have one
<daviddias>
lgierth: oh so that is why I was unable to find you. Can I induce you in getting one? :)
<daviddias>
or maybe educe instead
<lgierth>
nopenopenope.jpg
<lgierth>
i might get a solarlog thing
<lgierth>
but that's the maximum
<cryptix>
ssb maybe? :)
<lgierth>
maybe!
<lgierth>
daviddias: we're heading out
<daviddias>
cool, meet you there, we might arrive just on time (1:30pm)
zabirauf has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Eudaimonstro has quit [Ping timeout: 256 seconds]
LennyLinux has quit [Ping timeout: 250 seconds]
slothbag has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
M-staplemac has quit [Remote host closed the connection]
M-prosodyContext has quit [Remote host closed the connection]
M-hash has quit [Remote host closed the connection]
M-davidar has quit [Remote host closed the connection]
M-mistake1 has quit [Remote host closed the connection]
reit has quit [Ping timeout: 260 seconds]
Encrypt has joined #ipfs
plyp has quit [Quit: Leaving]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
eyebloom has quit [Quit: eyebloom]
thomasreggi has joined #ipfs
<ehmry>
are you guys going to meet up tonight in berlin?
mkarrer_ has quit []
mkarrer has joined #ipfs
<lgierth>
ehmry: they're zebrano in lenbachstrasse right now
<lgierth>
*at zebrano
<lgierth>
+ ansuz + cjd
slothbag has quit [Read error: Connection reset by peer]
reit has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
M-staplemac has joined #ipfs
M-hash has joined #ipfs
M-prosodyContext has joined #ipfs
M-mistake has joined #ipfs
M-davidar has joined #ipfs
chriscool has joined #ipfs
chriscool has quit [Client Quit]
chriscool has joined #ipfs
domanic has quit [Ping timeout: 240 seconds]
<rschulman>
lgierth: They’re hanging out with cjd now?
<rschulman>
worlds colliding. :)
pfraze has joined #ipfs
retsej has joined #ipfs
therealplato has quit [Ping timeout: 256 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Encrypt has quit [Quit: Quitte]
therealplato has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
reit has quit [Ping timeout: 246 seconds]
border has joined #ipfs
JasonWoof has quit [Ping timeout: 244 seconds]
JasonWoof has joined #ipfs
Encrypt has joined #ipfs
JasonWoof has quit [Ping timeout: 260 seconds]
JasonWoof has joined #ipfs
JasonWoof has quit [Changing host]
JasonWoof has joined #ipfs
vijayee has joined #ipfs
JasonWoof has quit [Ping timeout: 256 seconds]
JasonWoof has joined #ipfs
JasonWoof has quit [Changing host]
JasonWoof has joined #ipfs
border_ has joined #ipfs
border has quit [Ping timeout: 246 seconds]
voxelot has joined #ipfs
Tv` has joined #ipfs
warner has quit [Read error: Connection reset by peer]
<mg__>
Is IPFS development funded by some{one,thing}, or is it all volunteer work?
mildred has quit [Ping timeout: 250 seconds]
reit has joined #ipfs
PayasR has joined #ipfs
<PayasR>
how stable are we with the IPFS API?
<voxelot>
I'm using the API, they put in a lot of work last week, adding and retrieving files is stable atm
<voxelot>
can run it server or client side
<voxelot>
ohh haven't tried ls yet, that looks new
<PayasR>
but why exactly do we require IPFS API implemented in other languages?
<PayasR>
how would they be useful?
<jbenet>
PayasR it is bindings to the API, so you can use IPFS from other languahes
<jbenet>
mg__ i started a company called Protocol Labs (ipn.io) to drive development of IPFS and other protocols.
<jbenet>
rschulman yeah, right?
<jbenet>
ehmry: i might-- pretty tired though.
<rschulman>
jbenet: Hehe, how is cjd?
<PayasR>
But do you expect IPFS to be running as a background application on the client side?
<mg__>
jbenet: aha, that's cool. Are you funding it yourself or? I was looking at the jobs-section on the ipn.io site, and was wondering who paid for those :)
amiller has quit [Read error: Connection reset by peer]
<jbenet>
mg__ companies typically have funding to pay for development. some raise money before making revenue. i dont particularly want to discuss all this at the moment (i'm busy with other things), but happy to do QA on protocol labs here another time, or by email. :)
<mg__>
jbenet: I totally understand :) I'll send you an email
PayasR has quit [Ping timeout: 260 seconds]
<voxelot>
PayasR you need the daemon running for local api port 5001
<pjz>
voxelot: what languages doyou have bindings for so far?
<voxelot>
currently just nodejs and javascript
<voxelot>
i haven't used the http api
PayasR has joined #ipfs
bedeho has joined #ipfs
Encrypt has quit [Quit: Quitte]
<ehmry>
jbenet, ok, I just want to talk once more before you leave
<jbenet>
ehmry: yeah for sure. i'm here for two weeks, we'll def hang out. i'll ping you here later tonight if i'm feeling up to hacking outside
<ehmry>
cool
atrapado has joined #ipfs
gunn has quit [Ping timeout: 250 seconds]
Leer10 has quit [Ping timeout: 256 seconds]
simonv3 has joined #ipfs
<jbenet>
Hey berliners (ha!) -- i'm talking at http://berlinjs.org/ tomorrow night. event starts 7pm, talks at 7:30pm
<whyrusleeping>
i am a donut.
<jbenet>
cc lgierth mappum krl ehmry ehd ansuz cjd h1jinx and whoever else is around
<ehmry>
got it
dignifiedquire has quit [Quit: dignifiedquire]
<jbenet>
karissa is also speaking there about dat
<rschulman>
ich bin ein… wait.
<rschulman>
:)
Gaboose has joined #ipfs
warner has joined #ipfs
Encrypt has joined #ipfs
marianoguerra has quit [Ping timeout: 265 seconds]
<whyrusleeping>
jbenet: have time today for a go-ipfs hangout?
<jbenet>
whyrusleeping yeah, give me a couple of hours
<whyrusleeping>
sounds good
Encrypt has quit [Quit: Eating time!]
chriscool has quit [Quit: Leaving.]
chriscool has joined #ipfs
<lgierth>
jbenet: got a meeting at 8pm tomorrow... :*
<whyrusleeping>
but yeah, thats why it was removed
<whyrusleeping>
we didnt want our users who run fedora to not be able to build easily
<bren2010>
makes sense
<bren2010>
your decision makes sense
<bren2010>
Redhat is mental
<whyrusleeping>
yeah
Eudaimonstro has quit [Ping timeout: 252 seconds]
simonv3 has quit [Quit: Connection closed for inactivity]
<whyrusleeping>
huh, if you remove all of the sync calls in flatfs, adding files becomes as fast as using '-n'
Eudaimonstro has joined #ipfs
<jbenet>
hey whyrusleeping: sorry dinner took long while
<jbenet>
hey bren2010 o/
<bren2010>
hey jbenet :)
<whyrusleeping>
jbenet: no worries
<jbenet>
bren2010 havent seen you in a while. have been traveling europe for a while :) -- catch up next time i'm over in PA.
<bren2010>
wat :o
<bren2010>
Where'd you go?
<jbenet>
whyrusleeping: i can discuss go-ipfs now. it may be better to discuss by irc than hangout-- mind a bit all over the place.
<whyrusleeping>
alrighty then
<jbenet>
bren2010 seattle to hangout with whyrusleeping, then battlemesh.org (slovenia) -- then events.ccc.de/camp/2015/ -- now in berlin next: zurich then london. long trip!
<whyrusleeping>
how long are you gonna be in berlin?
<jbenet>
whyrusleeping until the 28/29th.
<whyrusleeping>
sounds like fun! all the german beer to try :)
<jbenet>
(and you thought the records were stalled... i was busy pestering random hackers!)
<whyrusleeping>
lol...
<whyrusleeping>
and here i was just gonna implement records using protobufs and see what happened :P
<jbenet>
i'm so closssse
* stick`
o/
<jbenet>
stick` \o
<stick`>
jbenet: still getting deeper and deepr into the gory technical details, but i must admit that ipfs is a cool project!
<whyrusleeping>
:D
Gaboose has quit [Quit: No Ping reply in 180 seconds.]
<jbenet>
thanks stick`! welcome to #ipfs -- a space of interplanetary hackery for the win of all
<jbenet>
whyrusleeping, the problem with the encoder is it doesnt follow the "4 rules" i listed there.
<jbenet>
not sure which
<whyrusleeping>
hrmm...
<whyrusleeping>
well, i know one encoder that works ;)
Gaboose has joined #ipfs
<jbenet>
if you want to take a stab at it, it may be a big deal. we could also just fork the code and have our own implementation. i think cbor is going to be very popular in IoT (a bunch of people from bigcos are pushing the standard)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
huh, alright
<whyrusleeping>
i can take a look
<stick`>
is there any estimate how many nodes are there?
<jbenet>
ok, so that's it for multicodec. i'll push the changes needed to go-ipld tonight
<jbenet>
stick` there's many clusters/networks we dont know about, but ours oscillates between 80-120 currently connected nodes atm. (i've sene it go way higher, and i've seen lots of nodes come up and disappear)
<jbenet>
(there's other known clusters, more than i expected. i met several people at CCC that use their own and never connect them to our gateways)
joshbuddy has joined #ipfs
<stick`>
jbenet: are you going to 32c3?
<jbenet>
stick` tiny small numbers, but it roughly correlates with that many people using it enough to run it constantly in a vps or laptop.
<jbenet>
once we do node-ipfs + ipfs.js (browserified) it will shoot way up, as websites have no adoption problem)
<jbenet>
stick` yeah thinking about presenting ipfs there
<stick`>
you should!
<stick`>
:)
<jbenet>
will have to polish a ton before then, i'd rather avoid djb ripping me to shreds :)
<stick`>
haha
<jbenet>
stick` am in berlin atm, if you happen to be in this side of the planet
<jbenet>
(a few of us here)
<stick`>
i am based in prague
<jbenet>
stick` we may do a meetup here next week
<jbenet>
and speaking at berlinjs.org tomorrow night
<stick`>
cool!
<stick`>
tomorrow is impossible, but next week seems fine
<jbenet>
whyrusleeping: ok ipld is basically ready, i'll push out the changes tonight and then (assuming go-multicodec fixes) it's ready to be plugged into go-ipfs
<whyrusleeping>
jbenet: still skeptical of ipld...
<jbenet>
yeah everyone was skeptical about html too
<jbenet>
;P
<whyrusleeping>
not really
<jbenet>
at the beginning tim got a load of crap like "ughhhhhhh yet another hypertext format? not again...."
<jbenet>
it might as well been named YAHTML
bedeho has quit [Ping timeout: 245 seconds]
<jbenet>
anyway neither here nor there
<jbenet>
json + ld friendliness will go a long way
<jbenet>
(web standards easy to use)
<jbenet>
so, i think we should pull out all the dag utilities (editor and so on) into a package -- either go-mdag or even within go-ipld, or go-ipld-util
<jbenet>
though not sure if that depends on unixfs -- i guess it does
<jbenet>
whyrusleeping: btw, we dont have to break any links
<whyrusleeping>
but it has to be an ugly hack to make that work...
<jbenet>
whyrusleeping we can just hash protobuf encoded things without the header
<whyrusleeping>
we've mostly avoided ugly hacks so far...
joshbuddy has quit [Quit: joshbuddy]
therealplato1 has quit [Ping timeout: 240 seconds]
<jbenet>
well there's no free lunch, i cant break cryptographic hash functions atm, nor enumerate the set of ipfs hashes that exist but i've never seen before :(
<jbenet>
we can do the wrapping if you want as a first step of intergration, but that _will_ increase the total size of the data
<jbenet>
(want to be able to sign cbor buffers and verify them easily)
<jbenet>
(oh btw the cose ietf wg is about to define a bunch of stuff for this too, and we stand to provide a ready made solution they could just take http://datatracker.ietf.org/wg/cose/documents/ )
<jbenet>
cc daviddias o/ good to get involved methinks
joshbuddy has joined #ipfs
atrapado has quit [Quit: Leaving]
<whyrusleeping>
alright, you should show me how unixfs looks when rewritten using ipld
<jbenet>
these are not final examples, but roughly in this direction. see
<jbenet>
just a map of names to links, where the links can have extra data in them as needed
<stick`>
n
<jbenet>
(here we can discuss whether the mode should be a metadata wrapper etc, that was there for illustration. but it may very well work to put it there now that links are extensible easily)
<jbenet>
(the big problem before was that links were difficult to extend cleanly/generally without shipping around protobuf schemas)
joshbuddy has quit [Quit: joshbuddy]
<jbenet>
whyrusleeping btw, we avoid 1 whole copy here becuase cbor marshals into the datastructs directly
<jbenet>
currently we do unmarshal to the protobuf datastruct, then copy over the things we want
marianoguerra has joined #ipfs
marianoguerra has joined #ipfs
Eudaimonstro has quit [Ping timeout: 246 seconds]
<whyrusleeping>
its not much of a copy saved, but yeah. a bit nicer
<whyrusleeping>
how do i get from bytes on disk to this 'File' struct?
therealplato has joined #ipfs
<jbenet>
whyrusleeping: i want to get it to be just mc.Unmarshal(codec, buf, file) (or even ipld.Unmarshal(buf, file) since it can guess the codec from the buf + registered handlers)
<whyrusleeping>
but you dont have that yet?
<jbenet>
(though admittedly it may be easier at first to do what we do with protobuf and decode to an ipld.Node, which is generic)
<whyrusleeping>
yeah... changing that base generic 'Node' object is one of my larger worries
<jbenet>
oh, i think that should work now--
<jbenet>
why is it a worry?
<whyrusleeping>
its just going to complicate things a lot
Eudaimonstro has joined #ipfs
<whyrusleeping>
theres also a lot more mental load on potential new users
<whyrusleeping>
sooooo many terms they need to learn
<whyrusleeping>
i need a ipld with a cbor and a json-context field to my datastructures definition
<whyrusleeping>
personally, i would just go find something simpler to build on top of...
<jbenet>
whyrusleeping actually it's the other way around. the ipld change is to _drop_ the cognitive load. it becomes: "hey, look, it's json on the wire. just make something that works with json and you're good"
<daviddias>
<whyrusleeping>
daviddias: wooo!
<jbenet>
right now everyone i talk to, their eyes glaze over and they sign, when i say "you need a protobuf with your own schema and you need to wrap that in ours"
<jbenet>
sigh*
<whyrusleeping>
jbenet: but thats not how simple its looking right now
joshbuddy has joined #ipfs
<jbenet>
whyrusleeping: yep, that is, where are you seeing the complication?
<jbenet>
the example links i sent are literally meant to be that small
<jbenet>
i.e. Unmarshal and Marshal should work with that struct.
<whyrusleeping>
where do people need to have their own protobufs?
<jbenet>
whyrusleeping: when they make their own datastructures
<jbenet>
whyrusleeping like unixfs
<whyrusleeping>
i could have just put json in unixfs instead of protobufs
<whyrusleeping>
that was just a choice we made because protobufs are faster and more compact than json
<jbenet>
oh but then you need to either extract the links and put them elsewhere, and know the node format well enough, or duplicate data
<whyrusleeping>
huh?
<jbenet>
so you end up with sharded links, part of it in the wrapper and part of it in your json
<whyrusleeping>
could i get an example?
<jbenet>
unixfs, it keeps data in the data segment serialized about each of the links
<whyrusleeping>
okay
<jbenet>
it's that "datastructure sharding" that's very painful to think about, and this kills it completely.
<jbenet>
(particularly painful if you don't understand protobuf, or our format well)
<jbenet>
having it "just work" and ipfs look into your datastructure and know what to do is really, really nice
<whyrusleeping>
so will this file struct have a 'Links' method on it?
<jbenet>
this is one thing to consider. we can do it without, when it's marshaled. (or _as we marshal_). because we can recognize what links are.
<jbenet>
on the marshal step, we can catch either ipld.Link objects by type, or we can catch map[string]interface{} which have a "@type": "mlink" entry
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
forcing everything that just wants to work on a generic dag node to manually parse out links of every given object will be annoying
<jbenet>
(that's what the ld lets us do).
<whyrusleeping>
like, say i want to traverse a dag
<jbenet>
whyrusleeping i'm talking about a: func Links(o interface{}) map[string]ipld.Link {} function
<jbenet>
that works generically on everything.
<whyrusleeping>
using reflection?
<jbenet>
no
<whyrusleeping>
would you want map[string]Link? or just []Link?
<whyrusleeping>
i dont think your file struct can work with the map
<jbenet>
casting, like all the marshaling stuff already works. we _can_ define a type Linked interface { Links() map[string]ipld.Link } id you want, so that things can be done way faster, but the magic working is very nice for prototyping and users
<jbenet>
yeah map[string]Link
<whyrusleeping>
how would you do that with casting? what would you cast to?
<jbenet>
so the map[string]Link already works with ipld.Node, but yeah, the File may require tags or reflection? (how do encoding/gob, encoding/json, and proto do it?
<whyrusleeping>
json and gob use reflection
<whyrusleeping>
proto (depending on the impl) uses codegen
<jbenet>
well, codegen is certainly possible, reflection is costly but easy (we should _measure_ before blowing up its impact, my guess is the hashing at the end of marshal dominates the marshaling anyway, well beyond using reflection), and the other option is defining the Linked interface i mentioned to make the user return things. it's more annoying but lets you
<jbenet>
make things really fast if you want them to be.
<jbenet>
(please remember also that this is not just for Go, this is a multiplatform, multilanguage format, whether go uses reflection or not is an implementation detail that we should figure out. the bigger problem is the conceptual overhead on the user, which is reduced when you show them a json document they can just create)
<whyrusleeping>
i know its not just for go, but i'm going to have to deal with it in go :P
<whyrusleeping>
and reflection isnt really all that slow in go anyways
<whyrusleeping>
i'd be fine with an optional 'Links' method/interface that falls back to using reflection
<whyrusleeping>
like the go std lib does it
<jbenet>
yeah
<jbenet>
let's do that then
<whyrusleeping>
now, youre sure about map[string]Link ?
<jbenet>
so take a look at how map[string]Link in go-ipld
<jbenet>
it uses path notation for levels in the datastructure
<jbenet>
it's a way of returning one map for all the links in the object
<jbenet>
(no matter how deep it is)
<jbenet>
it's like flattening the thing
<whyrusleeping>
how will that work with embedding arrays of links?
<whyrusleeping>
just have the key be fieldname-# ?
<whyrusleeping>
or, fieldname/# ?
<jbenet>
this _does_ mean that either (a) slashes are outlawed in link names, or (b) slashes are outlawed when there it is ambiguous, or (c) we always take top levels first. (--- but they're currently outlawed as far as we've been concerned. i mentioned the b + c options in IETF style flexibility.)
<jbenet>
btw, the glorious thing about all this is that an ipld node is just a unix-path-thing (i.e. a "pseudo-filesystem")
<jbenet>
so if i have an object like: {"foo": [ {"bar": "baz"} ] }, and i link to it with quux -> hash(obj)
<jbenet>
i can ask for: .../quux/foo/0/bar
<jbenet>
to get "baz"
<jbenet>
storing large amounts of data in ipfs becomes really nice
<jbenet>
becuase i can query it out with standard tooling + interfaces
Encrypt has quit [Quit: Quitte]
<jbenet>
without having to write my own tooling for it
<jbenet>
(and equivalently, ../quuz/foo/0 would give us {"bar":"baz"}
<whyrusleeping>
as long as {"bar":"baz"} is an object that is understood by the system
notduncansmith has joined #ipfs
<whyrusleeping>
it has no way of knowing that its a link
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
otherwise
<jbenet>
no, not a link, just raw cbor
<jbenet>
it can walk your cbor-encoded datastructure
<lgierth>
whyrusleeping: no notifications or anything like that for crashes on the gateway, just `docker logs ipfs`, which is basically stdout+stderr of IPFS_LOGGING=debug ipfs daemon
<jbenet>
(this is what made json so popular, btw)
<jbenet>
(or one of the things)
<lgierth>
whyrusleeping: we have https://gateway.ipfs.io, but with a self-signed cert until startssl get their shit together and reply to jbenet's request :)
<jbenet>
(the total simplicity + universality of the thing, everything understood foo.bar.baz without custom parsers.
<jbenet>
lgierth: i'll email them again now
<lgierth>
jbenet: if there's anything i can take off your shoulders there, let me know
<lgierth>
i'm about to go to bed and will take care of my sprint todo lis and denylist tomorrow
<jbenet>
lgierth: ok sounds good.
<pjz>
jbenet: did you have some special request for startssl?
<jbenet>
pjz: no, just following their process
<ipfsbot>
[go-ipfs] prusnak opened pull request #1592: Add --empty-repo option for init (#1559) (master...empty-repo) http://git.io/vsGGu
<pjz>
ah
<jbenet>
something failed
<jbenet>
hey whyrusleeping we have more to discuss \o/
<jbenet>
anything else on this ipld stuff?
<jbenet>
cause we should really plan out dev 0.4.0
<jbenet>
and the remaining things
Eudaimonstro has quit [Remote host closed the connection]
<jbenet>
(lgierth we have some infra things to think about re dist.ipfs.io and the other outstanding things, but if you have spare bandwidth for go-ipfs these next 2 weeks, would be very useful)
rawtaz has quit [Ping timeout: 260 seconds]
<whyrusleeping>
oh, okay
<whyrusleeping>
uhm, i guess i'll stop complaining about ipld for today
<whyrusleeping>
what else on 0.4.0 are you thinking?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
jbenet: on the ugorji thing, its not sorting the keys of embedded map[string]interfaces
rawtaz has joined #ipfs
therealplato has quit [Read error: Connection reset by peer]
<lgierth>
jbenet: i'm sure i could take on a go-ipfs task or two
therealplato has joined #ipfs
<lgierth>
in addition to finally getting denylist and discovery.cjdns out the door
<jbenet>
whyrusleeping: oh is it the int things?
<jbenet>
lgierth: sounds good
<whyrusleeping>
jbenet: no, it seems to have something to do with nested maps
<whyrusleeping>
'if we dont have a case for it, just return'
<jbenet>
lol yeah i saw that
<jbenet>
i was like wat
<jbenet>
whyrusleeping ok
<jbenet>
lets keep discussing
* whyrusleeping
gets bagel bites out of the oven
vijayee has joined #ipfs
<jbenet>
assuming ipld merges, and we move a bunch of the dag utils out into another package (modularization with gx or worst case godeps)
<jbenet>
we can then make records + keystore easily on top of those
<whyrusleeping>
hrm, okay
pfraze has quit [Remote host closed the connection]
pfraze has joined #ipfs
<jbenet>
that means ideally we should have pub/priv (can we rename these "nodestate/storage" or "internal/external" ? "pub/priv" are more about sharing models.
<whyrusleeping>
fine with renaming
<jbenet>
the internal node state is not shared, but that's a different thing. i dont want people thinking we should put "things we dont what to share" in the "nodestate or internal" dagstore
<jbenet>
and can we call these things dagstores instead of dagservice ? (either just local or online)
<whyrusleeping>
you named it in the first place, lol
<jbenet>
i know, im not infallible.
<jbenet>
don't quote me on that, i'm probably wrong about it
<jbenet>
oh no, paradox. i'm surprised freenode didnt boot me.
<whyrusleeping>
lol
<whyrusleeping>
pinbot: set paradox-kick=true
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
god, when dot stars seeming attractive to map task deps, run
<lgierth>
hrm openwrt mirroring still running, is at 640G now
<lgierth>
2.5 days now
<jbenet>
lgierth: oh wow.
<lgierth>
i wonder if we could mirror it to a fuse mount
<jbenet>
lgierth: yeah i bet we can
<lgierth>
would be bound by net i/o so it might stress the daemon less
<jbenet>
whyrusleeping take a look at the etherpad, see the "ordering" at the bottom. that's just the ipld/records/dev0.4.0 stuff
<jbenet>
the batches are roughly dependent on the prior ones
<lgierth>
bed, will write stuff down first thing tomorrow. today nada happened
<whyrusleeping>
jbenet: okay
* lgierth
zzz
<jbenet>
lgierth: is the daemon already running? thought just downlaoding atm
<jbenet>
lgierth: night!! o/
<lgierth>
yeah just downloading
<jbenet>
ah yep
<jbenet>
whyrusleeping seem good?
<jbenet>
whyrusleeping: <3
<jbenet>
dat dot
Gaboose has quit [Quit: No Ping reply in 180 seconds.]
<jbenet>
once harlan's dagviz stuff is in we can use ipfs files and graphmd to visualize our issues
<jbenet>
the objects can be the issues themselves!! zmong
<jbenet>
zomg*
<whyrusleeping>
lol
<whyrusleeping>
i dont want to think about that
<whyrusleeping>
and yeah, seems good to me
Gaboose has joined #ipfs
<jbenet>
Ok sounds good. matters because we can put the batches per week.
<whyrusleeping>
maybe batches per month...
<whyrusleeping>
or every two weeks if we dont sleep
<jbenet>
whyrusleeping: any help on all my remaining stuff here will be super useful as im also shipping dist.ipfs.io, shipping electron app with daviddias, handling lots of org stuff, and moving to make starship, the tool i'm demoing at container.camp
<whyrusleeping>
yeah, i'm looking at the cbor stuff
<jbenet>
whyrusleeping: batch1 is doable in 1 week i think. the hardest thing is the cbor thing i think, and the blockstore split
<whyrusleeping>
hmm... we will see
<jbenet>
batch 2 could also be done in 1 week. batch 3 idk. the dht naming stuff im not so sure about. should just be tweaks to the dht logic but ....
<jbenet>
ok, for the rest of the stuff
<jbenet>
we should divide it into batches, even though it's not very ordered.
<whyrusleeping>
you think so?
<whyrusleeping>
youre talking about the stuff under general, right?
<jbenet>
i'd like to see if some arrangement makes sense -- of related things
<whyrusleeping>
i'm gonna go get some caffiene real quick