<jbenet>
zignig_ yeah, really can add it in a bunch of places
<zignig_>
so check nodes discovered with mdns first ?
<wking>
I need a non-blocking Open() to handle named pipes without data in them
<jbenet>
wking: no policy yet.
<wking>
ok. I'll just keep the existing syscall imports then
<cryptix>
wking: just call stat on it not open?
<whyrusleeping>
wking: any time we have to use syscall, we have to have separate files per linux/windows
<wking>
yeah, but "stat and then open" is racy
<cryptix>
whyrusleeping: i'm just testing windows again actually - there is something about iptb/main using a field in syscall.SysProcAttr that doesn compile
<wking>
and serialFile.NextFile() needs to return a file instance anyway (so I'd still need a non-blocking open?)
<wking>
whyrusleeping: Does Windows have named pipes?
<cryptix>
wking: the open shouldnt block to begin with. the read should
<wking>
cryptix: that's what I thought, but a dummy test program calling Open("/path/to/named/pipe") blocks for me
<cryptix>
uhm.. i might be wrong.. waiting for the other side to open might make more sense actually
williamcotton has joined #ipfs
<cryptix>
like create is Listen and both opens is the accept
<whyrusleeping>
wking: i dont think it does
<whyrusleeping>
cryptix: which field?
<cryptix>
havnt checked but i also thought it would open immediatly
<cryptix>
whyrusleeping: Setsid is unknown
<cryptix>
btw - travis maybe doesnt run windows but we could trick it into at least building for it
<cryptix>
bootstrapping might be abit me but it will get much easier with 1.5
<ipfsbot>
[go-ipfs] wking pushed 1 new commit to tk/check-for-non-file-or-dir-modes: http://git.io/vLS5e
<ipfsbot>
go-ipfs/tk/check-for-non-file-or-dir-modes a739709 W. Trevor King: commands/files/serialfile: Non-blocking open...
<whyrusleeping>
cryptix: yeah, once 1.5 drops we should do cross compile builds to windows
<whyrusleeping>
and *bsd
<whyrusleeping>
although, probably not on travis
<cryptix>
commands/cli also fails quite hard right now
<whyrusleeping>
because its slower than my parents elderly pug taking a dump after eating too much peanut butter
<cryptix>
whyrusleeping: yea ..
<cryptix>
whyrusleeping: you are always good for a goodnight story
<whyrusleeping>
lol
<ipfsbot>
[go-ipfs] wking pushed 1 new commit to tk/check-for-non-file-or-dir-modes: http://git.io/vLS5n
<ipfsbot>
go-ipfs/tk/check-for-non-file-or-dir-modes 5baa0c8 W. Trevor King: commands/cli/parse: Non-blocking open...
<wking>
gendale__, cryptix: Ok, tk/check-for-non-file-or-dir-modes seems to die cleanly when it hits an empty named pipe now.
<wking>
I'll go back through, polish it up, add tests, etc. And I guess split these openers out to avoid syscall stuff that's not supported on Windows...
<zignig_>
whyrusleeping: cheers , for the merge. now for some testing.
<whyrusleeping>
zignig_: good luck to you :)
headbite has joined #ipfs
<cryptix>
just got a major wat moment..
<cryptix>
i think i will end testing window now
<dPow>
Does the api port only allow certain whitelisted ipfs content to be loaded? I'm getting a 403 on /ipfs/<hash-to-my-cool-app-that-talks-to-the-api>
<cryptix>
dPow: yup - we added a flag to make dev easier
<cryptix>
--unrestricted-api
<cryptix>
on daemon
patcon has joined #ipfs
<dPow>
Thanks cryptix
<cryptix>
very welcome :)
<dPow>
We're going to have to come up with a way for apps to get permission to talk directly to the api in the future
reit has quit [Ping timeout: 264 seconds]
dandroid has joined #ipfs
<dandroid>
hey @jbenet you around? i jsut talked to Lars as well :)
notduncansmith has joined #ipfs
<jbenet>
hey dandroid! yeah i am. give me a few min. I'm behind on email, i'll get a response to ideas in the next hour
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix>
dPow: yup - espc the readonly stuff. i think harlan is onto something there currently
<jbenet>
whyrusleeping we should get pinbot managed in the infrastructure repo like the gateways. we want reliable uptimes on things, and we should all be able to manipulate the infra as needed. (+ having it in one place will make it easier to have things like devops dashboard + setup downtime alerts)
inconshreveable has quit [Read error: Connection reset by peer]
inconshr_ has joined #ipfs
williamcotton has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has quit [Remote host closed the connection]
inconshr_ has quit [Remote host closed the connection]
inconshreveable has joined #ipfs
bret has quit []
bret has joined #ipfs
dandroid has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
tilgovi has quit [Ping timeout: 264 seconds]
dandroid has quit [Ping timeout: 246 seconds]
tilgovi has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
chriscool has quit [Ping timeout: 264 seconds]
Wallacoloo has quit [Quit: Leaving.]
avalo has quit [Ping timeout: 255 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has joined #ipfs
hellertime has quit [Quit: Leaving.]
williamcotton has quit [Ping timeout: 264 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has quit [Remote host closed the connection]
inconshreveable has joined #ipfs
inconshreveable has quit [Remote host closed the connection]
Wallacoloo has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
patcon has quit [Ping timeout: 255 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
kbala has quit [Quit: Connection closed for inactivity]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
dread-alexandria has quit [Quit: dread-alexandria]
sharky has quit [Ping timeout: 272 seconds]
<zignig_>
booting a 16 machines coreos cluster, 12 min 53 seconds. no tftp failures.
<zignig_>
w00t
sharky has joined #ipfs
<jbenet>
zignig_: cool! :) -- 12min? What was the hold up? Shouldn't it be 1x os boot time?
<spikebike>
jbenet: where you chatting about google's AI image recognition?
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot>
[go-ipfs] wking force-pushed tk/check-for-non-file-or-dir-modes from 5baa0c8 to 9b8e737: http://git.io/vLyfs
<ipfsbot>
go-ipfs/tk/check-for-non-file-or-dir-modes 38ce1bc W. Trevor King: commands/files: Make Stat() part of the File interface...
<ipfsbot>
go-ipfs/tk/check-for-non-file-or-dir-modes 65f538b W. Trevor King: commands/files/serialfile: Non-blocking open...
<ipfsbot>
go-ipfs/tk/check-for-non-file-or-dir-modes c789d9a W. Trevor King: commands/cli/parse: Non-blocking open...
inconshreveable has joined #ipfs
<zignig_>
jbenet: serving all the images ( 160 mb ) via http at the same time tends to max out the webserver.
<wking>
For the mode branch I've been working on, dummy FileInfo for stuff we get over multipart API pushes isn't right (we need to pass the mode from the client? Or something). But other than that it's getting pretty close to working ;)
<zignig_>
re ran , actually 6m40s - 7m58s.
notduncansmith has joined #ipfs
<zignig_>
need to do some experiments to see if tftp is faster....
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
spikebike: sigh that post is pretty dumb. the datasets involved are pretty specific about what they're testing. if you trained convnets with tons of data including sofas with various prints and leopard-print things (minus specifically leopard-print sofa), and with more dimensions (video, audio, 3D, larger light spectrum, whole-scene (other, related objects,
<jbenet>
like "dining room" vs "jungle")) they'd handle this case trivially well too. the post does not acknowledge what convnets are actually doing-- they're learning features intrinsic to the data, they gauge "thing-ness" (like "leopardness") in the dimensionality of the dataset (in this case small 2D snapshots of light in human eye wavelengths). And that sofa
<jbenet>
certainly ranks highly on "leopardness" or "leopard-like-ness" in my brain. particularly when only considering 2D static representations. . Training simple convnets on simple images will obviously produce skewed results like this. it's such a well-known thing that researchers often joke about the weird, tricky edge cases yielded by their datasets+algos.
<jbenet>
Also, the post builds up to the point of "oh it's as if we're actually just training to recognize visual data patterns instead of deriving conceptual patterns with logic!" when that is an obvious and trivial assumption of the models. _of course_ we're doing that.
<jbenet>
the datasets human brains are exposed to over our lives dealing with sofas and dealing with leopards teach us that a sofa's _shape_ is a much bigger predictor of sofa vs leopard, but that doesn't come from interaction via single 2D images.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mildred has joined #ipfs
inconshr_ has joined #ipfs
inconshreveable has quit [Ping timeout: 248 seconds]
Wallacoloo has quit [Quit: Leaving.]
<dPow>
I forked a live markdown editor to save directly into ipfs. It's fun to play around with if you have the --unrestricted-api flag set on your daemon.
<jbenet>
rht__ because if those repos move, are taken down, etc it's a huge mess. vendoring makes sure all the code travels together and nothing breaks unexpectedly because, say, someone decided to change their github username today, ... etc.
<cryptix>
gmorning
<jbenet>
wait... why is Paris and Berlin _both_ GMT+2
<cryptix>
jbenet: i think paris goes into daylight saving later
<cryptix>
jbenet: cryptix: you're missing a command? < yea i know. it was the argument path expension that frightend me
<jbenet>
timezones and the fact they change at different times of year everywhere is annoying.
<jbenet>
geopolitics. sigh.
<cryptix>
daylight saving is pointless to begin with
<cryptix>
it only makes sense if you work outside and depend on natural light
<jbenet>
once IoT is in full swing, it will be ... time ... for a coup de temps: lock the entire globe onto UTC. Change all digital time mentions to UTC everywhere, and make it a _bigger_ hassle for people to still use timezones than to adjust their mental model of when it's acceptable locally to be awake.
<rht__>
but DST, timezone, ... reflect the lifecycle of local ecosystem. even if everyone is using UTC, there will still be a need to take the effects of dst/timezone into account
notduncansmith has joined #ipfs
<jbenet>
rht__ yeah that's fine, so things can open at whatever times make sense locally,
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
Ok what? madrid is _west_ of london.
<rht__>
(on godep) i c, that's a different argument than the one I saw in https://github.com/tools/godep/issues/131 ("because if heroku sees Godeps/ then it will only `godep go install`"). but still unexpected compared to e.g. npm
pfraze has joined #ipfs
pfraze has quit [Ping timeout: 248 seconds]
bedeho has quit [Ping timeout: 272 seconds]
<cryptix>
rht__: i dont know what the purpose of 'go get -d' is, but i only had problems with it
<cryptix>
when updating with -d packages might not be reinstalled and weird things happen (something that gb solves btw *caugh*
<cryptix>
anyways..
<cryptix>
rht__: big thangs for the logging and context cleanup - always wanted to do that as well. i'm glad somebody beat me to it :)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
bedeho has joined #ipfs
kbala has quit [Quit: Connection closed for inactivity]
<rht__>
cryptix: can't think of the equivalent of `go get -d` with other package managers, or its use case
<cryptix>
i think it makes sense to update repos with no go code (or where you know a build error occurs)
<cryptix>
still.. a bit weird
<rht__>
in either case `go install` of the deps is still necessary
<cryptix>
i think, without the -d he would have been fine. on some cases go tool doesnt realize stale packages, then a 'go install -a' might be necessary but in my experiance its mostly the -d
<rht__>
but 'go get' knows that github.com/ipfs/go-ipfs is a git repo, Godeps/Godeps.json is updated via git
<rht__>
'-d' should mean, "like just '-u', but don't 'go install -a'"
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
<rht__>
godep: "@jbenet> rht__ because if those repos move, are taken down, etc it's a huge mess" -> another solution is to ref to a pinning service (this is npm's design choice)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix>
finally came around to watch his AEAD talk at yahoo - quite interesting the follow up questions and stuff about synthetic IVs etc
mildred has joined #ipfs
<cryptix>
jbenet: looks like i have some free time coming up today and tomorrow - anything special you want to hand over? otherwise i'll take a look at my old unresolved sprint issue and/or git-remote-ipfs
<jbenet>
cryptix: git-remote-ipfs would be awesome to get working right
<jbenet>
cryptix: otherwise will see tomorrow at the sprint
<jbenet>
"tomorrow" :d
<cryptix>
tomorrow?
<cryptix>
ah yea ^^
m0ns00n_ has quit [Quit: Leaving]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix>
i'll try to get the pull working with my nasty bare checkout - i think its better to get the full picture before starting to 'do it right' (ie ipfs git-import .. )
<jbenet>
very much agreed
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
hellertime has quit [Ping timeout: 255 seconds]
hellertime has joined #ipfs
williamcotton has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mildred has quit [Ping timeout: 256 seconds]
williamcotton has quit [Ping timeout: 255 seconds]
mildred has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
williamcotton has joined #ipfs
temujin has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has quit [Read error: Connection reset by peer]
Encrypt has quit [Ping timeout: 276 seconds]
<cryptix>
i think it's time to move my node dir outside of my copies=4 zfs home..
<cryptix>
add of small files is painfully slow
<cryptix>
33k files, 40megs size. running for more than 5mins (75% done)
Encrypt has joined #ipfs
domanic has quit [Ping timeout: 272 seconds]
dread-alexandria has quit [Ping timeout: 264 seconds]
Tv` has joined #ipfs
<whyrusleeping>
ugh
<whyrusleeping>
we need to fix our datastore interface
dread-alexandria has joined #ipfs
domanic has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
jibber11 has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
domanic_ has joined #ipfs
domanic has quit [Ping timeout: 277 seconds]
Encrypt has quit [Quit: Eating time!]
<whyrusleeping>
cryptix: thoughts about a '--dry' option for ipfs add?
<whyrusleeping>
where it doesnt write anything to disk
<whyrusleeping>
just gets the hash
domanic_ has quit [Ping timeout: 256 seconds]
jibber11 has joined #ipfs
<ipfsbot>
[go-ipfs] wking opened pull request #1413: WIP: Improved Stat() and Mode() handling for added files (master...tk/check-for-non-file-or-dir-modes) http://git.io/vL5zC
<cryptix>
whyrusleeping: could make sense.. dont see a use in it right now
<wking>
whyrusleeping: filed a PR ;)
<whyrusleeping>
wking: cool. want me to close mine out?
<whyrusleeping>
cryptix: i was thinking to be able to quickly check the hash of a file you know youve added
<whyrusleeping>
also, to check if a file has changed since adding it
<whyrusleeping>
kinda like an ipfs shasum
<wking>
whyrusleeping: sure
<wking>
sorry I was late to open mine. I thought I'd been chatty enough about it here to wait until I thought it was merge-able ;), but there's no harm in WIP PRs
<ipfsbot>
[go-ipfs] whyrusleeping closed pull request #1412: ignore and refuse to add pipes (master...pipes-are-bad) http://git.io/vL5vi
<whyrusleeping>
wking: nope! and i've been spotty this weekend
<whyrusleeping>
so i must have missed it
<cryptix>
whyrusleeping: oh okay. i'd be more in favor of 'ipfs checksum <file>' though. add is already quite complex and --dry from other tools implies to me that it could change/break something without it
<whyrusleeping>
wking: hows progress on docker stuff?
notduncansmith has joined #ipfs
<whyrusleeping>
cryptix: i could live with checksum
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix>
at first i thought 'why not add this to the multihash tool' but the resulting hash depends on the importer used, right?
<whyrusleeping>
yeap
<whyrusleeping>
it would essentially be an add call with a mocked out datastore
<whyrusleeping>
wking: sweet. need anything from me on it?
* whyrusleeping
is trying to find meaning in life
<wking>
review? jbenet wanted to look it over too before I PR the Docker folks
<whyrusleeping>
mmkay, will do
<whyrusleeping>
also, what do you think of a checksum command?
<cryptix>
the docker reg only uses the ipfs-shell? awesome
dread-alexandria has quit [Quit: dread-alexandria]
dread-alexandria has joined #ipfs
<wking>
whyrusleeping: ah, I think there are still some outstanding PRs that the Docker-storage driver depends on (e.g. 'ipfs file ls' and the associated ipfs-shell PR
<wking>
)
<whyrusleeping>
wking: link to the file ls?
<wking>
I think go-ipfs#1348 is also just blocking on jbenet's stamp of approval ;).
<wking>
whyrusleeping: ^ that's the 'file ls' PR
sandymac has joined #ipfs
<wking>
whyrusleeping: ipfs-shell#5 is not blocking on jbenet though, so maybe you can merge that ;)
<whyrusleeping>
wking: merged
<wking>
:)
<cryptix>
need to un-f** my brain.. bb in an hour
avalo has quit [Ping timeout: 246 seconds]
<whyrusleeping>
cryptix: drinkies?
<whyrusleeping>
i had this really awesome peach wheat beer last night
<cryptix>
nope
<cryptix>
global string beeing empty after it was set and checked in main
<cryptix>
head>table iteration cound exceeded
<cryptix>
count*
<whyrusleeping>
lol
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<wking>
so workable options for passing FileInfo include:
<wking>
A new File-Info header that sticks something (a version string, e.g. 'v1'?) before a colon: v1; mode=0777; size=123; mod-time=0
<wking>
Attaching some of the parameters to Content-Distribution
<wking>
Attaching some of the parameters to Content-Type
<wking>
Do folks have preferences there? I think I'm leaning towards the File-Info header, but maybe the mode belongs with Content-Type and the size and mod-time belong with ContentDisposition?
<lgierth>
I mentioned this in previously meetings, by just realized I never wrote it down. I'm during today and tomorrow on MediterraneaJS, which means I won't be able to do much. Also attending the Sprint meeting will be hard as the Wifi is pretty much broken (currently on 3G Roaming).
<jbenet>
Anyway, let's start with whyrusleeping
<lgierth>
-- daviddias
<whyrusleeping>
alright
<jbenet>
lgierth thanks!
<wking>
jbenet: yeah, somehow I fell off last week's list, but since I was still focused on the Docker-storage driver (like the week before), I didn't worry about it :p
<whyrusleeping>
this week i pushed a bunch of go-multistream code into go-ipfs, and hit a bunch of test failures
<whyrusleeping>
so i deferred it to jbenet
<whyrusleeping>
the go-multistream library has a bunch more code in it now, client stuff too
<whyrusleeping>
i implemented a multistream transport for go-peerstream
<whyrusleeping>
i pushed eventlogs out to an http endpoint
<whyrusleeping>
and made them not render while nobody is connected
<whyrusleeping>
mars is very happy about that
<lgierth>
nice
<jbenet>
yeah so is celebdil (my node)
<whyrusleeping>
i fixed a daemon shutdown bug, and also added bubbling stuff to object patch
<jbenet>
anything to add to that issue or what's the next step?
<whyrusleeping>
1222, basically needs a refactor on the importer
<jbenet>
makes sense cool
<whyrusleeping>
we need to use a buffer pool for incoming blocks from disk
<jbenet>
want to write up thoughts on that issue?
<whyrusleeping>
and we need to implement the batch datastore operations
<whyrusleeping>
*before* we do the pipeline IMO
<whyrusleeping>
i'll post there
<jbenet>
sgtm
<jbenet>
ok-- anything else?
<krl>
jbenet: LOTR reference? :3
<whyrusleeping>
uhm, i'm babysitting a cat
rht__ has quit [Quit: Connection closed for inactivity]
notduncansmith has joined #ipfs
<whyrusleeping>
oh!
<jbenet>
(krl: yep :) all my machines are named after lotr places-- i'm on lorien atm)
notduncansmith has quit [Read error: Connection reset by peer]
<lgierth>
:3
<jbenet>
ok next up
<whyrusleeping>
i worked on implementing close-notify on the API stuff
<whyrusleeping>
and it broke everything
<jbenet>
ahh yeah
<jbenet>
that's a sad story, want to write up findings too?
<jbenet>
ok, next up-- krl
<krl>
okaii, did say some things above
<krl>
basically, did the html menu and init guide stuff
<krl>
tsall working, but needs designer loving
<jbenet>
<krl>
and some performance tweaks it seems, thx jbenet
<krl>
cool!
<jbenet>
electron app works really, really well
<jbenet>
it's replaced my usage of `ipfs daemon`
<jbenet>
(because i no longer lose it on a random terminal somewhere)
<krl>
it might have replaced mine as well if i used a tray :D
<jbenet>
haha
<krl>
well, i could just run it on my ctrl-alt-f2 gnome :)
<jbenet>
krl you were also going to work on starlog proposal-- any progress on that?
<krl>
jbenet: not yet, apart from the things i showed you already
<krl>
but i think it ties in well also with the modularization of webui
<jbenet>
sounds good, no rush on that, and electron app is higher prio i think
<krl>
still have some floating thoughts about auth and identity stuff
<jbenet>
yeah! let's talk about that today (or tomorrow if better)
<jbenet>
(btw moving node-ipfs to another time/day, so can probably use that slot if you want, or talk tmrw)
<krl>
ah, yes that would be cool also
<krl>
and, what's the progress on them curves?
<jbenet>
-- that reminds me -- in https://github.com/ipfs/pm/pull/14 i proposed moving up sprint planning by 2 hours so that people in germany dont have really crap time slots
<jbenet>
krl: yeah agreed, we could try and get ECC in this week or next-- it was waiting on keystore stuff and whyrusleeping just made lots of progress on that last week
mildred has quit [Ping timeout: 264 seconds]
<jbenet>
krl: lets maybe discuss in go-ipfs chat or over gh)
<dread-alexandria>
hey @jbenet, do you have 5 minutes to help me trouble shoot a users ipfs install? he didn’t have a /local/bin/ folder on his mac, so i’m not sure how to properly set that up for him and none of my guys are available at the moment
<krl>
that's cool
<krl>
and
<whyrusleeping>
dread-alexandria: pm me, i'll help out
<dread-alexandria>
i had him make the folders manually, but their permissions aren’t properly set up im sure
<jbenet>
dread-alexandria: i cant, we're in the middle of our sprint meeting. maybe someone else can, but not dure.
<krl>
instead of having the user choose key-size
<krl>
we could have the user setup a 'default identity'
<krl>
kind of like, associate a nickname with the key
<krl>
and then provide things for setting up multiple identities
<whyrusleeping>
krl: yeap, thats what i want too
<jbenet>
krl: yep sounds good-- let's discuss in discussions as other people are waiting to checkin
<krl>
and connecting these with the different apps that
<krl>
-that
<lgierth>
will that be go-ipfs hangout or gh?
<krl>
ok. over and out
<dread-alexandria>
no prob, understood - and thanks whyrusleeping :)
<jbenet>
(would go well with sdsi/friend naming, too)
<jbenet>
lgierth: depends on whether you and krl want to be part of that discussion and cant make go-ipfs hangout :)
<jbenet>
(am down to discuss it wherever you want)
* krl
prefers async communication mostofthetime
<jbenet>
ok, thanks krl, solid progress on electron
<jbenet>
next up, lgierth \o/
<lgierth>
i'll get back to the curves stuff in a bit
<lgierth>
yeah
williamcotton has joined #ipfs
<lgierth>
i'm about to peer the gateway nodes into hyperboria (the cjdns network). that's cool on its own, but i'm actually doing it to be able to authenticate metric scrapers. i went with prometheus for now, it's a little lib that exposes metrics as a net/http handler for the actual prometheus daemon to collect
<lgierth>
we'd have the prometheus daemon on two or three machines, scraping ipfs ans host metrics from the gateway hosts.
<lgierth>
i'll open a PR against go-ipfs in a few, which integrates prometheus, but doesn't gather ipfs-specific metrics yet, just general stuff like number of goroutines, cpu time, such stuff
<jbenet>
sounds really good
<lgierth>
bootstrapping from cjdns peerings hasn't seen progress since i didn't finish the monitoring/metrics yet (i want to this night though)
<cryptix>
lgierth: check if you can expose expvar, too (we already have metrics for blockstore iirc)
<jbenet>
i know kbala is interested in go-ipfs metrics as well, so i'll tag him in that issue-- we can discuss how to collect useful ipfs specific metrics throughout the system (had a design in mind, i'll write it up)
<lgierth>
non-code-wise, i'm still trying to get into a good work habit, going to the office, etc, to not get distracted all day long. it's getting better
<jbenet>
haha yeah, when working distributed/remote that can be hard.
<cryptix>
b4 i forget: cjdns transport reminds me to ask about status of relaying ipv4<>ipv6 node
<lgierth>
long story short, none of my tasks finished yet EOF
<jbenet>
(maybe can discuss this in go-ipfs/community, but working hours (which i've so far sucked at) and things like https://sqwiggle.com/ can help
<jbenet>
lgierth: no worries-- they're large, etc. maybe can break it down more as you go through a sprint? so take a large thing like "Monitoring for gateway.ipfs.io" and break it into component issues/PRs
therealplato has quit [Ping timeout: 248 seconds]
<lgierth>
yeah i should do that
<lgierth>
monitoring could easily be split into 3+ smaller chunks
<lgierth>
oh yeah! a small handful. i realized i don't understand the current network stack well enough yet
<jbenet>
but only if useful to you
<lgierth>
so i'd like to do more go-ipfs, now that the infrastructure is largely wrapped up
<jbenet>
lgierth: the current network stack is a poor man's world without cjdns. let's maybe discuss that over gh, i've been wanting a full p2p overlay network for many years,
<lgierth>
anyhow, later
<jbenet>
sounds good
<lgierth>
col
<jbenet>
lgierth: ok anything else
<lgierth>
EOF :)
<lgierth>
i don't babysit a cat :X
<jbenet>
haha
<jbenet>
ok next up, chrischool
notduncansmith has joined #ipfs
<jbenet>
chriscool*
notduncansmith has quit [Read error: Connection reset by peer]
<chriscool>
yeah, I have been very busy last week
<chriscool>
and especially last week end
<jbenet>
no worries :)
<chriscool>
but I have just pushed a PR 1414 to fix a test crash
<jbenet>
you got all the gitcop stuff done, which is great
<jbenet>
chriscool: ah nice! will take a look
<chriscool>
and I also just force pushed community/pull/34
<jbenet>
chriscool: great! will add both of those to the issue
<chriscool>
I think it takes cares of the comments you and wking made
<krl>
btw regarding gitcop, should we sign off js stuff this way too?
<jbenet>
krl: yeah i think so. maybe we can make a script that sets up (a) gitcop, (b) CI (travis/circleci) (c) README with links to community repo (d) license, etc
<jbenet>
so it's trivial to bootstrap a repo. there's some good examples of this out there, will dig them up
jibber11 has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<jbenet>
chriscool: great, thanks! anything else?
<chriscool>
no
<chriscool>
I will have a look at circleci soon
<jbenet>
great, thank you!
<whyrusleeping>
jbenet: real quick, what do you think of having an option to add that doesnt store anything in the datastore?
<whyrusleeping>
or just an 'ipfs checksum' command
<jbenet>
yeah i've been wanting that
<whyrusleeping>
alright, 'ipfs add --checksum' or 'ipfs checksum' ?
<jbenet>
maybe `ipfs chunk --hash` or `ipfs file [chunk|hash]`
<jbenet>
ok
<jbenet>
next up -- cryptix
<jbenet>
you didnt have anything on the sprint doc, but i know you've been hacking on something really sweet, want to add it there?
<jbenet>
and i think you pushed another thing but i dont recall what
<cryptix>
sure
<cryptix>
2 things: git-remote-ipfs and the krl designed a new index for the gateway a while back
<cryptix>
git-remote-ipfs is a git helper that gets triggered when you use ipfs:// protocl in any git command
<cryptix>
so you can 'git clone ipfs://$hash..' and it triggers that helper - right now it only does fetch/clone but i want to develop it further so that it can push to an ipns entry for instance
<wking>
I pushed some fixes to bubbling in ipfs#1407, although I think we're still waiting for consensus on auto-creating intermediate nodes
<wking>
*go-ipfs#1407
<jbenet>
sounds good will take a look
hellertime1 has quit [Quit: Leaving.]
<wking>
and then yesterday I started putting in explicit mode handling with go-ipfs#1413
<jbenet>
idea: if you're waiting for comments on an issue/pr from me, just add it to the etherpad under my name, that will make sure i get to it asap
<wking>
this week I plan on landing #1413, and then adding switches so folks can use that new mode information to attach metadata Merkle nodes in front of their directory and file nodes
<wking>
jbenet: or we could assign the issue/PR to you?
<jbenet>
wking: that should work too
<wking>
hmm, maybe doesn't scale to folks that don't have push access to the repo in question
<wking>
etherpad notes may be better ;)
<dPow>
******
<dPow>
jbenet I'd like to complete our email discussion before getting into too much here
<dPow>
Everything else is up to date in terms of checking things off in the sprint issue
<dPow>
****
<wking>
or Waffle for "ready for review"?
<jbenet>
dPow roger that, sounds good
<jbenet>
ok
<jbenet>
ok lgtm, i think that concludes the check in part \o/
<jbenet>
maybe we can speed it up if everyone writes up their update beforehand, or at the beggining, and can paste them in like dPow just did
<lgierth>
yeah that's a good idea
<jbenet>
and maybe it will help too if i write a list of the order, maybe we can use the sprint order + whoever else at the end
<jbenet>
oh wait
<jbenet>
i forgot, last and least, jbenet
bengl has quit [Ping timeout: 252 seconds]
<jbenet>
ok, i didnt get to setting up mss muxer, whyrusleeping got to a lot of it, and i'll be taking a look at problems
<lgierth>
what's mss?
<whyrusleeping>
multistream-select
<jbenet>
i merged https://github.com/ipfs/go-ipfs/pull/1399 which added `ipfs swarm addrs local, ipfs id -f=<addrs>, and --disable-transport-encryption for node-ipfs interop
<cryptix>
jbenet: just fly into a black hole - everything will be fine
<jbenet>
wking yeah, it will. i want to finish this modules listing first, but maybe can get started? if we pick a good format for it we can both collab on it (i'd suggest tikz, but maybe a dot dag may work out well)
<lgierth>
jbenet: yeah we should talk about identity and p2p in that gist ^ re: cjdns
<lgierth>
iprs == ip record system?
<jbenet>
lgierth: yeah absolutely
<jbenet>
lgierth yeah it is. i realized iprs is actually a pretty subtly important module.
<wking>
tikz or dot work for me
<lgierth>
jbenet: checking EOF?
<lgierth>
*checkin
<jbenet>
yep, checkin EOF
<jbenet>
thanks everyone, sorry for the length, maybe next week we can be twice as fast if we all write them up beforehand/at the beginning.
<lgierth>
the curves discussion plays into too i guess
<whyrusleeping>
jbenet: nonsense, optimization?
<lgierth>
yeah lets make sure to do that
<jbenet>
yeah
<wking>
jbenet: Do you have anything for the modules diagram now, or shall I PR a tikz-based sketch?
<jbenet>
krl: lmk if you want to take that spot to discuss electron, or want to discuss it tomorrow.
<krl>
jbenet: tomorrow at 20:00 utc maybe? would work for me
<jbenet>
also how does everyone feel about moving up the sprint planning discussion by 2 hrs for our friends in deutschland https://github.com/ipfs/pm/pull/14 ?
<jbenet>
(i'll be recording a talk tonight which will be played, and i'll video call in for Q/A)
<krl>
sorry i was being confused
<krl>
i have a weekly meeting with my collective at 17:00 tuesdays
<krl>
that usually goes on for 2-3h.
<krl>
so, actually today might work better
<jbenet>
(if anyone wants me to cover anything in particular on that lmk-- i'll mostly talk about various datastructure use cases of ipfs, putting databases on the open web as "open databases" or "open datastructures"
<jbenet>
krl: ok sounds good
<krl>
but within ~1h, or it will be bit too late
<whyrusleeping>
jbenet: goprocess is taking up 100MB of ram on mars
<cryptix>
jbenet: please include the rotating rocket animation :P
<krl>
:D
<jbenet>
wking: i dont have anything for the modules diagram. feel free. my only worry with tikz is i suck at memorizing pdfs that tell me how to do things.
<jbenet>
cryptix: oh man, i want that dtn talk to be posteddddd
<krl>
jbenet: yep, it was great
<jbenet>
whyrusleeping: :(
<cryptix>
yea - i wondered too when that will happen
<cryptix>
ccc really spoils you..
<krl>
:)
<wking>
jbenet: me too, so I write the first few nodes with the manual open, and then match-and-replace for subsequent revisions ;)
chilicuil has joined #ipfs
<jbenet>
wking: sgtm :) i'm really good at copypasta. been training for years.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
krl ok lets use the node-ipfs time then?
<krl>
jbenet: which is?
<chilicuil>
hi, sorry if this question is too dump, I couldn't find it on the documentation, I'm trying to setup a ipfs node to act as a ftp like site, I've a bunch of files under a root directory which I'd like to distribute after I give the hash of the root directory, however ipfs get and ipfs cat doesn't seem to get the files recursivery, is there any way to get the whole directory in one go?
<spikebike>
ipfs ls
<whyrusleeping>
chilicuil: 'ipfs get <hash>' will download an entire directory
<spikebike>
oh download
<chilicuil>
oh funny, ipfs get actually gets the directory recursive, I must have mix-ups things
<whyrusleeping>
chilicuil: no worries! did that work for you?
<chilicuil>
whyrusleeping: yep, thank you =)
<jbenet>
chilicuil: we should have an `ipfs pull <ref>` that does what you want. `ipfs get` also downloads them to the local directory
<jbenet>
chilicuil: you can use `ipfs refs -r <ref>` as a stunted version of `ipfs pull <ref>`
<jbenet>
chilicuil: and if you want to provide them do `ipfs pin add -r <ref>`
<ipfsbot>
[go-ipfs] chriscool force-pushed fix-verify-file from 4e4285d to d182960: http://git.io/vLdXe
<ipfsbot>
go-ipfs/fix-verify-file d182960 Christian Couder: ipns_test: fix slice bounds out of range...
<chilicuil>
jbenet: ok, just to clarify a little bit more, I want this files to change eventually, so I should look to use somehow ipns on top of ipfs, right?
<JasonWoof>
chilicuil: yes
<jbenet>
@everyone, please weigh in here: https://github.com/ipfs/pm/pull/16 PROPOSAL: Move sprint planning to start at 17:15 UTC (mondays)
<jbenet>
chilicuil: yes, but we need to fix ipns-- it's not robust atm
<JasonWoof>
chilicuil: /ipfs/* is immutable (because the * is the hash of the content)
<jbenet>
whyrusleeping: lets discuss iprs/dht/ipns stuff in go-ipfs discussion today
<whyrusleeping>
sure
<whyrusleeping>
when is that gonna be?
<chilicuil>
JasonWoof, jbenet ok, good to know =), I'll then update the hash publication after every update
<whyrusleeping>
trying to decide if i wanna go get pie
<JasonWoof>
chilicuil: /ipns/<hash of public key>/foo can be updated by whoever has the private key
Encrypt has quit [Quit: Sleeping time!:]
<whyrusleeping>
chilicuil: if you use ipns you dont need to update the hash
<jbenet>
lgierth: "gateway.ipfs.io runs cjdns, peers into hyperboria" very excited about that.
<krl>
jbenet: hey, wheyn you say node-ipfs time, what do you mean?
<whyrusleeping>
lgierth: i'm excited about the cjdns stuff too
<jbenet>
krl: 14:30PT 21:30Z 23:30GT
<JasonWoof>
cjdns is very cool
<krl>
jbenet: ok half an hour
therealplato has quit [Quit: Leaving.]
* krl
be there
chriscool has quit [Ping timeout: 256 seconds]
<jbenet>
whyrusleeping: would love a bot that does this:
<jbenet>
hey daviddias: wasnt sure if you were going to make it to node-ipfs today + krl asked to switch electron in for now, is that ok with you? we could also talk fast and split the slot in two
<jbenet>
there's probably not 30min worth of efficient discussion
<uhhyeahbret>
o/ hi jbenet
<uhhyeahbret>
at work gonna mute
<lgierth>
jbenet: damn sorry... i forgot the clock while digging through code... haven't figured out desktop notifications for weechat yet :/
<uhhyeahbret>
or here
<daviddias>
I can wait, no worries jbenet. Sorry if I didn't communicate that effectively. So I'm MediterraneaJS conf, spoke today during the evening, so now I'm mentally and physically free. Tried to make sure to post a note on that here https://github.com/ipfs/pm/issues/13#issuecomment-114175272 maybe it was not the best place as I see some mentions here of you
<daviddias>
wondering where I was, apologies for that.
<krl>
it worked like last time!!!
<krl>
what is this
<jbenet>
daviddias: oh no worries about that i think i read it but figured i'd tag you in the stream anyway :)
<lgierth>
jbenet: we can discuss infra in chat later or tomorrow
<lgierth>
there's lots of movement in the codebase
<jbenet>
but maybe he still uses it for Tahoe LAFS
jibber11 has joined #ipfs
<jbenet>
whyrusleeping thoughts on the test on https://github.com/ipfs/go-ipfs/pull/1399 ? on how to get it to output on travis? not sure what tool will make it output correctly
<jbenet>
maybe a "pollEndpoint + nc + grep" where `nc-grep --timeout 10s "/ipfs/identify"` would connect, read from connection until it gets /ipfs/identify or 10s passes.
pfraze has joined #ipfs
pfraze has quit [Client Quit]
<lgierth>
ping rht__
ruby32 has quit [Quit: ruby32]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
* cryptix
hibernates
<jbenet>
lgierth: rht__ just sent me: "Having issue with irccloud (ISP problem). So I missed this week's sprint meeting once again!"
<lgierth>
ok thank you
<lgierth>
wanna sync with them regarding metrics
<jbenet>
lgierth: should also discuss with kbala
<lgierth>
yeah i'll open this prometheus PR now and cc them there. it's only 9 additions anyhow... :)
<whyrusleeping>
jbenet: that test is weird...
<whyrusleeping>
does it get any sort of output on travis?
<lgierth>
cryptix: i looked up expvar just now, it's a go-specific format for exposing metrics, right?
<lgierth>
what kind of tools are there for it?
<lgierth>
i want to end up with graphs and alerts, the monitoring will have to expand in the not-so-distant feature
notduncansmith has quit [Read error: Connection reset by peer]
dread-alexandria has joined #ipfs
<dPow>
I use init.d rather than systemd because it's more widely compatible.
<dPow>
Currently I'm unhappy with my init daemon because things broke recently to force running sudo -E ipfs <command> to actually interact with the daemon
m3s has quit [Ping timeout: 256 seconds]
jibber11 has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<bret>
running archlinux which comes with systemd by default
<bret>
so far, super duper duper happy with it
<dPow>
Between upstart and systemd, I'm glad everyone's finally on board with systemd. Now we can finally stop writing init scripts :D
<dPow>
I just wanted everyone to pick one
m3s has joined #ipfs
m3s has quit [Changing host]
m3s has joined #ipfs
<dPow>
I use Ubuntu, which recently announced systemd support, but it's going to be awhile
<jbenet>
whyrusleeping: what reboots it?
www1 has joined #ipfs
<lgierth>
i can't believe i didn't notice ./metrics/ until now...
www has quit [Ping timeout: 244 seconds]
williamcotton has quit [Ping timeout: 276 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]