<whyrusleeping>
kevina: a little busy, but i can spare a few minutes
<lgierth>
we have a few different yaks to shave right now haha
<whyrusleeping>
by the time i finish shaving one yak, the others have grown more hair!
<whyrusleeping>
lgierth: build me a yak shaving machine
<lgierth>
i'm the yak barber
<kevina>
whyrusleeping: if you are too busy it can wait, basically I would like that to go in as is (after I disallow adding multiple directories in PreRun).
<kevina>
my thinking is we can clean up the add command later so find a better solution
<whyrusleeping>
i dont think that is a good idea, Disallowing the adding of two directories at a time seems like such an arbitrary limitation...
ylp has quit [Ping timeout: 252 seconds]
<kevina>
there is not an easy fix for that unfortunately, right now it is not really being handled correctly
kvda has joined #ipfs
<kevina>
It has to do with the how commands are sent to the server
ylp has joined #ipfs
<kevina>
adding directories works okay with "-w", and I will still allow that
<whyrusleeping>
is there any way to tell from the resultant structure of the mfs root what the input was?
<whyrusleeping>
like, for adding two directories recursively, does the final mfs root have two top level entries?
Oatmeal has joined #ipfs
<kevina>
It might. But I can't be sure.
<kevina>
Something like this might not work: ipfs add -r dir1/ dir2/subdirA dir2/subdirB.
gmcquillan__ has quit [Ping timeout: 265 seconds]
<kevina>
If that works, than this likely won't: ipfs add dir1/subdirA dir2/subdirA as it will likely try to add two entries to the MFS root named subdirA.
<kevina>
Even though they are not the same directory.
<whyrusleeping>
ah, fair point
<lgierth>
subdirA~1 subdirA~2
<lgierth>
fat16 ftw
<whyrusleeping>
kevina: go ahead and disallow that for now, but also file an issue describing the problem
<whyrusleeping>
so we can make sure to fix it soon
<kevina>
Will do! Thanks.
<whyrusleeping>
the rest of this year will be paying off technical debt
edubai__ has quit [Read error: Connection reset by peer]
<whyrusleeping>
ah, gotcha
<whyrusleeping>
i should order some food...
<lgierth>
go for it -- i'm gonna do some assets stuff and coreapi stuff
cmeik has quit [Ping timeout: 265 seconds]
<whyrusleeping>
okay, i'll report back
<lgierth>
bed in 1h
<whyrusleeping>
lgierth: no worries, i can manage
<lgierth>
:)
<whyrusleeping>
hopefully
wa7son has joined #ipfs
voldyman has joined #ipfs
bret has joined #ipfs
cemerick has quit [Ping timeout: 248 seconds]
henriquev has joined #ipfs
jryans has joined #ipfs
mikolalysenko has joined #ipfs
Muis has joined #ipfs
edubai__ has joined #ipfs
<whyrusleeping>
lgierth: you got myrepos into apt?
<whyrusleeping>
or is that not your program
<lgierth>
whyrusleeping: it's in ubuntu 16.04
<lgierth>
a gx workspace tool can completely replace it
<lgierth>
all we need is "run command on set of directories"
yoshuawuyts has joined #ipfs
<whyrusleeping>
ah, gotcha
<lgierth>
and "clone all these repos"
kulelu88 has quit [Quit: Leaving]
richardlitt has joined #ipfs
cmeik has joined #ipfs
kyledrake has joined #ipfs
<whyrusleeping>
lgierth: 'gx test' is now a thing
<lgierth>
i updated a couple of PRs too :)
rsync has quit [Ping timeout: 252 seconds]
<A124>
Sorry to interupt again. Got two things. One, if I unpin, defeaults to recursive? and it unpins everything? And if so, if I have another directory that resolves to those files, does it preserve?
<A124>
Two, B/W limiting. Some people cannot afford unlimited nodes, but can preserve content that would else die.
xavivives has quit [Read error: Connection reset by peer]
rsync has joined #ipfs
<ploop>
when ipfs devs get stuck on a topological problem, do they say "DAG nabbit"?
rsync has quit [Ping timeout: 264 seconds]
JesseW has quit [Ping timeout: 264 seconds]
rgrinberg has quit [Ping timeout: 264 seconds]
Aeon has quit [Read error: Connection reset by peer]
se3000 has joined #ipfs
<whyrusleeping>
ploop: i am so disappointed that i never thought of that
rgrinberg has joined #ipfs
gmcquillan__ has joined #ipfs
gmcquillan__ has quit [Client Quit]
gmcquillan__ has joined #ipfs
rsync has joined #ipfs
anewuser_ has joined #ipfs
anewuser has quit [Ping timeout: 248 seconds]
ygrek_ has quit [Ping timeout: 248 seconds]
gmcquillan__ has quit [Quit: gmcquillan__]
bonesso has quit [Quit: Leaving]
herzmeister has quit [Quit: Leaving]
herzmeister has joined #ipfs
PrinceOfPeeves has quit [Quit: Leaving]
Qwertie- has quit [Ping timeout: 272 seconds]
wallacoloo has joined #ipfs
JesseW has joined #ipfs
mgue has quit [Quit: WeeChat 1.5]
gmcquillan__ has joined #ipfs
Qwertie has joined #ipfs
anewuser_ has quit [Quit: anewuser_]
dmr has joined #ipfs
bsm117532 is now known as Guest26311
mgue has joined #ipfs
se3000 has quit [Quit: My iMac has gone to sleep. ZZZzzz…]
vmatekol_ has joined #ipfs
voldyman has quit [Quit: Connection closed for inactivity]
vmatekole has quit [Ping timeout: 264 seconds]
ylp has quit [Ping timeout: 252 seconds]
ylp has joined #ipfs
Aeon has joined #ipfs
rsync1 has joined #ipfs
rsync has quit [Ping timeout: 244 seconds]
gmcquillan__ has quit [Quit: gmcquillan__]
gmcquillan__ has joined #ipfs
jooni has joined #ipfs
<ploop>
does having lots of things pinned (but without anyone downloading them) increase the amount of RAM/CPU IPFS uses?
gmcquillan__ has quit [Quit: gmcquillan__]
<JosiahHaswell[m]>
@ploop I haven't noticed that it does
Foxcool has joined #ipfs
kvda has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Akaibu has quit [Ping timeout: 265 seconds]
Tv` has quit [Ping timeout: 265 seconds]
ulrichard has joined #ipfs
Tv` has joined #ipfs
kvda has joined #ipfs
Akaibu has joined #ipfs
kvda has quit [Client Quit]
corvinux has joined #ipfs
herzmeister has quit [Ping timeout: 252 seconds]
captain_morgan has quit [Ping timeout: 265 seconds]
Tv` has quit [Quit: Connection closed for inactivity]
<damongant>
this looks like you want to store it in a blockchain
<damongant>
sure, you can make a makeshift chain with IPFS, but why?
anonymuse has joined #ipfs
<vmatekol_>
damongant: only state changes made every 60mins are stored on-chain, essentially the current head of the chain.
<vmatekol_>
damongant: storing all state changes on-chain would be expensive
rgrinberg1 has joined #ipfs
pawnbrokering has joined #ipfs
rgrinberg has quit [Ping timeout: 244 seconds]
<vmatekol_>
damongant: can you elaborate more on minimizing metadata exposure?
<damongant>
vmatekol_ just be aware you're putting this up on a public network, and it'll be hard to get this data out
<vmatekol_>
damongant: you mean from a privacy perspective or data format? If the latter that is why we are not storing information that can change. We only store a hash of the postal address just to prove that was the address of the member at that time of registration. Any subsequent changes are stored in a private datastore.
<vmatekol_>
damongant: and as for privacy we are only storing anonymised data.
<damongant>
just be aware of correlation attacks, if two addresses have the same hash, if they sign up in the same timezone, and so on
<damongant>
you should definitely not store raw hashes, but hmac(hash+salt)
<damongant>
"anonymized data" is hard
<vmatekol_>
damongant: Perfect advice. Thank you for that.
pfrazee has quit [Remote host closed the connection]
<Kubuxu>
but instead of git branch -d
<Kubuxu>
git push origin --delete
vmatekol_ has quit [Ping timeout: 252 seconds]
kaotisk has joined #ipfs
taw00 has quit [Client Quit]
ylp1 has quit [Quit: Leaving.]
corvinux has joined #ipfs
Tv` has joined #ipfs
<hsanjuan>
richardlitt: Im trying to use the standard-readme generator and I go through the questions but it doesn't write any file at the end :S any ideas?
<richardlitt>
hmmm
<richardlitt>
Interesting. Have you tried using it with Yeoman?
<hsanjuan>
yes, same in both cases
<richardlitt>
instead of `> standard-readme`, `> yo standard-readme`
<richardlitt>
Can you show me a gist of the output?
<hsanjuan>
theres no output other than the questions
anonymus_ has joined #ipfs
anonymuse has quit [Read error: Connection reset by peer]
flyingzumwalt_ has joined #ipfs
<hsanjuan>
richarlitt: it exits with 0 too... maybe its putting the readme somewhere i dont expect it to
corvinux has quit [Remote host closed the connection]
PseudoNoob has joined #ipfs
shizy has joined #ipfs
pfrazee has joined #ipfs
<hsanjuan>
richarlitt: found the problem pull req incoming...
JesseW1 has joined #ipfs
JesseW has quit [Read error: Connection reset by peer]
<richardlitt>
hsanjuan: sorry, got sidetracked by another issue
<richardlitt>
thanks for finding that, sorry!
Guest26311 is now known as bsm117532
tomoaki_ has joined #ipfs
<keks>
okay, after a few weeks of woking on libp2p, now a total ipfs noob question: i want to use the library functions in ipfs/go-ipfs (i.e. core, path, repo, fsrepo, ...) but get go build errors all the time because 1/2 the code base is gx'd and the other isn't. Is there a standard procedure or is it random gx-go rewrite --undo everywhere?
<lgierth>
keks: yes rewrite --undo for now, we'll have all of go-ipfs properly gx'd by the end of the year
<keks>
but that also means gx'ing libp2p right?
<lgierth>
libp2p is gx'd
<keks>
hm okay
<tomoaki_>
hi, is there any ipfs as video streaming CDN experiments ?
<lgierth>
note that for stuff in go-ipfs, we can't give you any interface stability guarantees
<keks>
yup, i'm looking at orbit in go atm
<hsanjuan>
lgierth: Ithink go-libp2p is not really gx'ed.. at package.json is there but libs are not rewritten as in go-ipfs
<hsanjuan>
(as of yesterday)
<lgierth>
not right now at least -- we aim to have stable interfaces when everything is gx'd. and i'm also working on go-ipfs-core-api right now
<lgierth>
hsanjuan: yes the rule of thumb is: if it's a lib, don't commit gx'd imports. if it produces an executable, commit the gx'd imports
<lgierth>
so that libs can also be used without gx. we don't wanna force gx on people
<keks>
that was super helpful
<keks>
i didn't know that
<hsanjuan>
lgierth: cool thanks
<Kubuxu>
also the main reason why go-ipfs is gx rewritten by default is we don't want people to build it with go get without deps being pinned by gx
<lgierth>
it's gonna be so amazing when go get works with go-ipfs again
taaem has quit [Ping timeout: 265 seconds]
Oatmeal has quit [Ping timeout: 272 seconds]
<keks>
lgierth: but e.g. in go-ipfs/core/core.go all the libp2p stuff is embedded using gx, even though core is more of a library
<keks>
but then again it is needed by ipfs and it would fire build errors...but that is the problem
<keks>
i don't think "only gx executables" is a working solution because you force gx on everybody using the ipfs API
<lgierth>
core is not technically a lib right now, since it's tightly coupled with the rest of go-ipfs.git
JesseW1 has quit [Ping timeout: 252 seconds]
<keks>
yes i know
<lgierth>
it is definitely going to be a proper nice lib in the next few months, but not quite yet
<keks>
okay
<Kubuxu>
yup, in future we should have core-interface
<lgierth>
i'd love to be able to just flip a switch, but it's an awful lot of work :)
<keks>
i know
<Kubuxu>
but go-ipfs itself will be always bound to gx
<keks>
i just don't see how gx is ever supposed to work seamlessly when it is super fragile with the go import paths and you need one format all along the tree
<keks>
*dep tree
<Kubuxu>
We are thinking about it, you could have for example some package say: I need dep X to work, it was tested with version A of X
<Kubuxu>
but allow importing package to override: I want dep X to be version B
<richardlitt>
hsanjuan: merged, patched,pushed, published
<Kubuxu>
for f in **/*.go; do sed 's|jbenet/go-multi|multiformats/go-multi|g' -i $f; done
<whyrusleeping>
A124: could I also get your ipfs binary?
<whyrusleeping>
Kubuxu: yes, or any number of different ways of doing it
Encrypt has quit [Quit: Quit]
musicmatze[m] has quit [Quit: node-irc says goodbye]
s_kunk has quit [Ping timeout: 244 seconds]
keks_ has joined #ipfs
<whyrusleeping>
lgierth: i have a task i need your help with
<lgierth>
whyrusleeping: shoot
<whyrusleeping>
go to: Rigaer Straße 8, 10247 Berlin, Germany and ask for whoever is in charge, then tell them to give us more build runners. especially on the OSX side
<whyrusleeping>
if they don't want to do it, throw money at them, preferably coins
<lgierth>
i'm in that neighbourhood tomorrow lol
<lgierth>
haad lives closer though ;)
<whyrusleeping>
haad: you up for a little adventure?
<lgierth>
Kubuxu: it's spinning disks and sometimes other people do stuff on it too :)
edubai__ has quit [Ping timeout: 252 seconds]
cketti has joined #ipfs
cketti has joined #ipfs
cketti has quit [Changing host]
<Kubuxu>
It might have unchocked
edubai__ has joined #ipfs
<haad>
richardlitt: reviewed your PR
<Kubuxu>
whyrusleeping: do you have branch with go-ipfs updated for libp2p after extraction?
<whyrusleeping>
i'm working on it
<Kubuxu>
kk, I will just revert myself to 3.6
<whyrusleeping>
yeah, i needed to also update floodsub, and the dht, and iptb and a few other libs
<Kubuxu>
ok, I can't figure out this mess
<whyrusleeping>
give me a minute, nearly done
captain_morgan has joined #ipfs
<whyrusleeping>
erk, context lib differences
<whyrusleeping>
+15min
<Kubuxu>
no reason to hurry
<Kubuxu>
and I can review the pin sharder
<whyrusleeping>
yeah, go do that
<whyrusleeping>
review mah code pls
Guest16955 has quit [Quit: Iovoid has quit!]
<Kubuxu>
now I am thinking about an attack where you make the other party pin DAG that has awful fanout with this alg.
<Kubuxu>
ie. no fanout at all
<whyrusleeping>
you only ever add one entry per pin to that
<Kubuxu>
ahh, right, forgot about htat
<Kubuxu>
and there is random seed
<whyrusleeping>
its a solid algorithm
<whyrusleeping>
might not be the best thing for our usecase
<whyrusleeping>
but good nonetheless
<Kubuxu>
yeah the sharing you implemented would be incremental, I don't think this is.
ashark has quit [Ping timeout: 265 seconds]
<whyrusleeping>
i'm pretty sure its not
<Kubuxu>
it shouldn't, it uses random seed every time
cemerick has quit [Ping timeout: 265 seconds]
<whyrusleeping>
thus me being pretty sure
<whyrusleeping>
Kubuxu: thoughts on adding a method to go-datastore:
<whyrusleeping>
NewKeyFromBinary(...)
<whyrusleeping>
that will base32 encode the input
<kevina>
whyrusleeping: is this a method or a function?
<whyrusleeping>
function
<kevina>
so just a convenience fun than?
<whyrusleeping>
yeap
captain_morgan has quit [Read error: No route to host]
<whyrusleeping>
to keep it in one place
<kevina>
off hands, sounds like a good idea to me :)
iovoid has joined #ipfs
iovoid is now known as Guest75921
captain_morgan has joined #ipfs
mildred has quit [Ping timeout: 272 seconds]
<kevina>
whyrusleeping: BTW: I am strongly leaning to not rebasing my filestore code, but instead cleaning up the commits some and pushing each one on a new PR to make sure each commit pass the test...
<kevina>
There is just too much code, and much of the earlier code depends on too many things that have changed.
<kevina>
There may be some duplicate commits, but they may be unavailable
<kevina>
do you think you could live with that?
<kevina>
unavailable = unavoidable
<whyrusleeping>
kevina: yeah, that seems fine. Its a pretty absurdly massive PR at this point :P
<kevina>
thanks, will do!
<Kubuxu>
Any recommendation of what I should use for storage of benchmark results, I would like us to be able to plot something like that: https://areweslimyet.com/ per commit bases
<lgierth>
prometheus? :P
<Kubuxu>
Not for something like this, I would rather think about some db.
<Kubuxu>
or just ndjson but filtering this will be nightmare
<kevina>
sqlite?
<lgierth>
ah you're looking for more than just exactly the kind of data on that site? ^
<Kubuxu>
yeah, I will have teamcity runners collect the data, maybe produce it as an artifact, then you can use it to join it into one big db or benchmarking data
<kevina>
I am a big fan of using SQL for data like this.
<kevina>
Makes getting exactly what you want really easy
<kevina>
although I am not 100% sure what you are after :)
rgrinberg has quit [Ping timeout: 256 seconds]
<Kubuxu>
yeah, sqllite seems like a good choice for the runners
<Kubuxu>
use UUDIs for the keys so everything can be merged at whim
soloojos has quit [Remote host closed the connection]
wallacoloo has quit [Quit: wallacoloo]
soloojos has joined #ipfs
fleeky_ has joined #ipfs
iovoid42 has quit [Quit: Iovoid has quit!]
fleeky__ has quit [Ping timeout: 244 seconds]
pfrazee has quit [Remote host closed the connection]
ianopolous_ has joined #ipfs
anonymus_ is now known as anonymuse
ianopolous has quit [Ping timeout: 256 seconds]
iovoid has joined #ipfs
iovoid is now known as Guest73858
Guest73858 has quit [Changing host]
Guest73858 has joined #ipfs
Guest73858 is now known as iovoid
anonymuse has quit [Remote host closed the connection]
anonymuse has joined #ipfs
<richardlitt>
haad: thanks
<A124>
How do I diagnose network? Aka what stuff is being transfered to whom at what speed?
Kane` has quit [Quit: Leaving]
shizy has quit [Ping timeout: 252 seconds]
anonymuse has quit [Ping timeout: 264 seconds]
<lgierth>
whyrusleeping: yay!
<whyrusleeping>
lgierth: that was pretty itnense
<whyrusleeping>
intense*
<lgierth>
:):)
iovoid has quit [Quit: Iovoid has quit!]
ianopolous has joined #ipfs
ianopolous_ has quit [Ping timeout: 272 seconds]
<whyrusleeping>
lgierth: yeah, lots of weird dependency stuff. I made extensive use of sed
<lgierth>
btw, what's the relationship of the string type and the String() func on an a struct
<lgierth>
i know the fmt package is aware of String(), but i figure the type system probably isn't
<whyrusleeping>
you cant pass a Stringer into a function that takes a string
<whyrusleeping>
unfortunately
<whyrusleeping>
but otherwise you can use them interchangeably in things like fmt calls
<whyrusleeping>
weirdly enough, the builtin string type doesnt implement Stringer
<lgierth>
cause then nobody would use the string type for argument or return types, hehe
<lgierth>
string -- the glorified byte array
bmpvieira has quit [Read error: Connection reset by peer]
sugarpuff has quit [Read error: Connection reset by peer]
iovoid has joined #ipfs
iovoid is now known as Guest11425
robmyers has quit [Ping timeout: 265 seconds]
oleavr has quit [Ping timeout: 272 seconds]
Guest11425 has quit [Changing host]
Guest11425 has joined #ipfs
Guest11425 is now known as iovoid
bmpvieira has joined #ipfs
sugarpuff has joined #ipfs
matoro has joined #ipfs
oleavr has joined #ipfs
robmyers has joined #ipfs
anonymuse has joined #ipfs
kvda has joined #ipfs
se3000 has quit [Quit: My iMac has gone to sleep. ZZZzzz…]
<kevina>
whyrusleeping: the build servers are having a hard time getting old gx deps: hanging on - running post install for gogo-protobuf: /home/ubuntu/.go_workspace/src/gx/ipfs/QmZ4Qi3GaRbjcx28Sme5eMH7RQjGkt8wHxt2a65oLaeFEV