jbenet changed the topic of #ipfs to: IPFS - InterPlanetary File System - https://github.com/jbenet/ipfs -- channel logged at https://botbot.me/freenode/ipfs/ -- code of conduct at https://github.com/ipfs/community/blob/master/code-of-conduct.md -- sprint at https://github.com/ipfs/pm/issues/7
<jbenet> spikebike oh interesting
<jbenet> spikebike we should turn on ipv6 on our gateways
<spikebike> or at least ship a config file that enables ipv6
<spikebike> 18% or so of google clients in the usa are ipv6
<spikebike> sounds reasonable to turn it on by default
<spikebike> Generally I'd expect any p2p app to behave much more efficiently without IP Masq/NAT.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping> jbenet: i'm working on writing up a changelog for 0.3.5
<wking> whyrusleeping: I can write up the resolution command changes later tonight
<wking> but feel free to beat me to it ;)
<whyrusleeping> wking: that would be great
<whyrusleeping> read the CHANGELOG.md to see the format we've been doing
<jbenet> wking: did we ... resolve ... the resolution problems?
<jbenet> wking: also, what was it that was originally fixed?
<wking> jbenet: https://github.com/ipfs/go-ipfs/issues/1307 is still open
<wking> and https://github.com/ipfs/go-ipfs/pull/1351 is unmerged (although the now-default non-recursive UI didn't exist before)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<wking> the changes started in https://github.com/ipfs/go-ipfs/pull/1208 were more about shifting the resolver to a clearer interface, so they don't actually resolve anything in particular ;)
<wking> But things like 'ipfs dns ...' and the distinction between the all-protocol resolver and the DHT-based resolver are now possible
Wallacoloo has joined #ipfs
<ipfsbot> [go-ipfs] rht opened pull request #1359: Cleanup logging 1 (master...cleanup-logging) http://git.io/vIQES
<mafintosh> whyrusleeping: you just pipe it to a tcp socket :)
<mafintosh> whyrusleeping: socket.pipe(plex).pipe(socket)
<whyrusleeping> mafintosh: why are there two pipe calls?
<jbenet> duplex
<whyrusleeping> aaah, okay
<jbenet> whyrusleeping: mind writing changelog for last one too?
<jbenet> i never did it :(
<whyrusleeping> 0.3.4?
<whyrusleeping> yeaaahhhhh... i gues i can
<jbenet> <3
<kbala> jbenet: ready?
<jbenet> kbala sorry can i get 15 more min? finishing something up
<mafintosh> jbenet: you should head up here. we need more routers
<jbenet> mafintosh srs?
<kbala> jbenet: sure np
<jbenet> mafintosh okay i'll head up after hanging out with kbala
<jbenet> feross did you leave already? else can hitch a ride with me
<jbenet> " <wking> the changes started in https://github.com/ipfs/go-ipfs/pull/1208 were more about shifting the resolver to a clearer interface, so they don't actually resolve anything in particular ;)" well, they broke things :( -- of course we need more tests, and we found some of that this way, but we've to be very careful about this. I should dev have done
<jbenet> closer CR. Though CR is not going to catch many of these problems. must get more tests
meadman has quit [Read error: Connection reset by peer]
<jbenet> " <wking> But things like 'ipfs dns ...' and the distinction between the all-protocol resolver and the DHT-based resolver are now possible" yeah that's very cool. +1
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
meadman has joined #ipfs
<wking> jbenet: yeah, I should have added more tests from the start of that PR :/. I have a much better feel for the resolution test coverage now ;)
Wallacoloo has quit [Quit: Leaving.]
<jbenet> wking: no worries. we should've had more from the start
<jbenet> kbala: kbala: ok im ready
<jbenet> sec
<wking> And part of the difficulty was that the initial goal of the PR was to de-clobber the protocol namespaces, but then we nixed that change part way through, and some things I'd tested before the rebase weren't covered in the test suite... :p
<jbenet> yeah we need way more tests
<jbenet> we should have a tests sprint
<Tv`> jbenet: oh btw just to ensure you've seen it, it seems go1.5 will largely solve your vendoring pains, at the cost of needing to run "go get -vendor" (for now, to be default later if things go well)
inconshreveable has quit [Read error: Connection reset by peer]
<Tv`> whyrusleeping: ^
<whyrusleeping> Tv`: yeahh........... its okay
<whyrusleeping> we will see how it works out
JasonWoof has quit [Ping timeout: 264 seconds]
nham has quit [Quit: Lost terminal]
JasonWoof has joined #ipfs
stackmutt has quit [Ping timeout: 258 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
stackmutt has joined #ipfs
<whyrusleeping> jbenet: updated my comment
<feross> jbenet: still sick believe it or not :(
<feross> jbenet: doc now thinks it's strep throat
<jbenet> feross: oh no! sucks :(
<jbenet> feross: so no camp?
* whyrusleeping feels better about not being the only sick one
<jbenet> i'd join you, but i rather not.
<whyrusleeping> lol
<whyrusleeping> stay healthy
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<spikebike> Ah, nice, I was worried about IPFS handling large numbers of objects. I just found the bitswap want_list and have_list which is a perfect fit.
<spikebike> kudos
<whyrusleeping> spikebike: i try to keep the words 'exabyte scale' in mind when designing things
<spikebike> heh, my goals were trying to support laptops/desktops on 1Mbit class home connections up to servers with infiniband and 36x6TB disks.
<spikebike> but I wasn't attempting filesystem, just backups, so trying to keep up with churn rate not instantaneous demands for I/O
<spikebike> but yeah exabytes aren't as big as they used to be ;-)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
hellertime has joined #ipfs
<spikebike> wow, ipfs is pretty much a perfect fit for my p2p backup client.
Blame has quit [Quit: Connection closed for inactivity]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
timgws has joined #ipfs
<wking> whyrusleeping: branch for the go-ipfs v0.3.5 changelog?
<wking> ah, I see it's comments in go-ipfs#1360
<wking> I'll create a branch, see about adding issue numbers for related bugs and pull requests, and write up the resolver UI changes
<wking> Also add a v0.3.6 section for collecting notes about unreleased changes as they land
<wking> I'm not sure how seriously we're taking semver at the moment, but maybe 0.4.0, since 'ipfs name resolve <domain-name>' no longer works (use 'ipfs dns <domain-name>' or 'ipfs resolve /ipns/<domain-name>', see https://github.com/ipfs/go-ipfs/issues/1307).
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
* jbenet wking: don't worry about extensive release notes at the moment. `git lg v0.3.4...v0.3.5` is not hard to see at the moment for those very interested. I'd rather include a pre-filled link to a github compare view which gives access to all the related PRs and Issues etc.
<jbenet> wking: we aren't taking semver seriously yet for the go-ipfs version. to take it seriously would mean to up the major AND it would "mean something" from a outsider perspective. i'd feel more ok if we used vanity-semver. <vanity>.<semver>
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<wking> jbenet: 63 PRs landed in v0.3.4...v0.3.5 with 129 non-merge commits. There's a reasonably large gap between 18 bullet points and that ;)
<whyrusleeping> wking: thats okay, we just try to highlight the major changes in the changelog for now
<jbenet> we'll get more descriptive as we get towards a beta
<whyrusleeping> jbenet: comitting now!
<jbenet> \o/
<jbenet> whyrusleeping: can you add a `latest` {branch or tag}, and switch gobuilder to point to `latest`
<ipfsbot> [go-ipfs] whyrusleeping pushed 1 new commit to master: http://git.io/vIQhQ
<ipfsbot> go-ipfs/master 952dc9c Jeromy: go-ipfs version 0.3.5...
<jbenet> probably have to change https://github.com/ipfs/website
<ipfsbot> [go-ipfs] whyrusleeping tagged v0.3.5 at master: http://git.io/vIQhp
<whyrusleeping> did 'git diff HEAD~1' and got really scared because i saw changes that i didnt mean to push
<jbenet> whyrusleeping: sorry, `stable` or `release`
<whyrusleeping> then i realized it was showing me un'added changes
<jbenet> maybe `release` as `master` is not unstable.
* whyrusleeping prefers release
<jbenet> sgtm
<jbenet> i'd like to make gifs like these: https://github.com/ipfs/examples/pull/14#issuecomment-109250406 programmatically.
<jbenet> there's probably some gif tool out there
<ipfsbot> [go-ipfs] jbenet deleted fix/bitswap-leak at e014a66: http://git.io/vIQjx
<wking> jbenet: for animated GIFs there's gifsicle, ImageMagick's convert (see http://www.imagemagick.org/Usage/anim_basics/)
<wking> and I'm sure many others ;)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
reit has quit [Quit: Leaving]
reit has joined #ipfs
<ipfsbot> [go-ipfs] wking created tk/changelog (+1 new commit): http://git.io/vI7vw
<ipfsbot> go-ipfs/tk/changelog 37d9c7e W. Trevor King: CHANGELOG.md: Start tracking user-visible changes for 0.3.6...
<ipfsbot> [go-ipfs] wking force-pushed tk/changelog from 37d9c7e to d6ed089: http://git.io/vI7vN
<ipfsbot> go-ipfs/tk/changelog d6ed089 W. Trevor King: CHANGELOG.md: Start tracking user-visible changes for 0.3.6...
inconshreveable has joined #ipfs
Bat`O has quit [Ping timeout: 265 seconds]
Bat`O has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet> wking: no shit, gifsicle is my eddie kohler, haha
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
sugarpuff has joined #ipfs
hellertime has quit [Quit: Leaving.]
<ipfsbot> [go-ipfs] spikebike opened pull request #1362: Add ip6 interface to default config (master...master) http://git.io/vI7Il
<ipfsbot> [go-ipfs] spikebike closed pull request #1362: Add ip6 interface to default config (master...master) http://git.io/vI7Il
<ipfsbot> [go-ipfs] jbenet reopened pull request #1362: Add ip6 interface to default config (master...master) http://git.io/vI7Il
<ipfsbot> [go-ipfs] spikebike closed pull request #1362: Add ip6 interface to default config (master...master) http://git.io/vI7Il
<whyrusleeping> jbenet: want me to write a blog post about the 0.3.5 release?
<jbenet> whyrusleeping: yeah go for it, kick it over so i can expand on it too. i've been meaning to write posts
<jbenet> do we have a `fork` call for humans yet?
<whyrusleeping> uhm... i think it requires two humans
<whyrusleeping> and its a very expensive call
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot> [go-ipfs] spikebike opened pull request #1363: Resubmit with license signed off. (master...master) http://git.io/vI7La
<spikebike> gah, I'm really botching this.
<ipfsbot> [go-ipfs] spikebike closed pull request #1363: Resubmit with license signed off. (master...master) http://git.io/vI7La
<whyrusleeping> i'll do that tomorrow...
<jbenet> spikebike: can squash it into one commit. the cop will complain if any commit is not signed.
<jbenet> spikebike: use `git commit --amend` and so on.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<spikebike> ah, tried to force a git commit -a -m without luck
sharky has quit [Ping timeout: 258 seconds]
<spikebike> didn't know about amend
<spikebike> then I made new one and of course there's a complaint about the first
<spikebike> I deleted the fork, sorry for wasting your time
<jbenet> spikebike: --amend and `git rebase` are the best -- learn them! will take you an hour and save you thousands.
<jbenet> spikebike: no no no, please contribute that fix. i want to merge it from you.
<jbenet> spikebike: there's a script to setup the commit hook here: https://github.com/ipfs/community/blob/master/dev/tools/hooks/setup_commit_msg_hook.sh
sharky has joined #ipfs
inconshreveable has quit [Ping timeout: 256 seconds]
headbite has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Tv` has quit [Quit: Connection closed for inactivity]
pfraze has quit [Remote host closed the connection]
<ipfsbot> [go-ipfs] spikebike opened pull request #1364: Added unspecified/any address "::" to enable IPv6 by default. (master...master) http://git.io/vI78h
zabirauf_ has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
zabirauf_ has quit [Ping timeout: 250 seconds]
bedeho has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
tilgovi has joined #ipfs
rht__ has joined #ipfs
<jbenet> do we have a "click to archive whole website" extension yet?
tilgovi has quit [Ping timeout: 256 seconds]
mildred has joined #ipfs
domanic has joined #ipfs
brab has joined #ipfs
nemik has quit [Read error: Connection reset by peer]
nemik has joined #ipfs
timgws has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
krl has quit [Quit: WeeChat 0.3.8]
konubinix has quit [Quit: ZNC - http://znc.in]
hellertime has joined #ipfs
konubinix has joined #ipfs
domanic has quit [Ping timeout: 256 seconds]
<spikebike> wow, that's an impressive amount of activity
anshukla has quit [Ping timeout: 265 seconds]
<spikebike> oh, ha, same patch from weeks ago
rht__ has quit [Quit: Connection closed for inactivity]
<spikebike> kinda complicated
<spikebike> seen numerous solutions
<spikebike> ping and ping6
<spikebike> ssh -4 and ssh -6
<spikebike> making a ~/.ipfs without any users queries for a complete p2p app is nice of course, but maybe a question or two would be appropriate?
<spikebike> ipfs init could ask things like take all defaults (generate key, assume ipv4 and ipv6)
<spikebike> then if the user says no ask enable v4? Enable v6? Skip temporary privacy addresses?
<spikebike> autostart daemon?
inconshreveable has joined #ipfs
williamcotton has joined #ipfs
<spikebike> zorun: just commented on that thread
pinbot has quit [Ping timeout: 264 seconds]
sprintbot has quit [Ping timeout: 264 seconds]
jbenet_ has joined #ipfs
mildred has quit [Quit: Leaving.]
jbenet_ has left #ipfs [#ipfs]
mildred has joined #ipfs
marklock has joined #ipfs
mildred has quit [Client Quit]
<jbenet> ei-slackbot-ipfs casey: i'd love to hear what happened on that demo + why it wasn't seeding
<jbenet> lgierth: nice dashboards: http://keen.github.io/dashboards/
krl has joined #ipfs
kbala has quit [Quit: Connection closed for inactivity]
domanic has joined #ipfs
tso has quit [Ping timeout: 244 seconds]
mildred has joined #ipfs
<lgierth> jbenet: it looks like one has to host the timeseries with keen :/
<jbenet> lgierth :(
mildred has quit [Quit: Leaving.]
<lgierth> prometheus on the other hand makes it surprisingly simple to export timeseries to other graphing systems
<lgierth> ... or to ipfs
<jbenet> :D
<jbenet> maybe we can take the nice js UI and remap it to prometheus?
<jbenet> oh that looks great too
inconshreveable has quit [Remote host closed the connection]
<lgierth> bbiab, need coffee
nicknikolov has joined #ipfs
www has quit [Ping timeout: 256 seconds]
compleatang has joined #ipfs
<krl> lgierth: where are you working in general? i'm at c-base atm
<lgierth> mostly at home, although i'm trying to go to the office space more often, which is near goerlitzer park
<lgierth> krl: will you be there a bit longer?
<lgierth> i can come over in ~1 hour
zorun has quit [Ping timeout: 252 seconds]
zorun has joined #ipfs
<krl> lgierth: cool, i'll be here
<daviddias> jbenet just making sure I didn't miss any comm, any love on the DHT spec/draft?
williamcotton has quit [Ping timeout: 255 seconds]
Encrypt has joined #ipfs
Blame has joined #ipfs
therealplato has quit [Ping timeout: 265 seconds]
therealplato has joined #ipfs
headbite has joined #ipfs
brab` has joined #ipfs
<krl> lgierth: ping
<lgierth> krl: pong
<lgierth> i'm about to head out
<krl> i'll be here until ~18:30
brab has quit [Ping timeout: 258 seconds]
<lgierth> yeah i have to leave then too
<lgierth> see you in a bit
<whyrusleeping> goood morning friends
brab` has quit [Ping timeout: 256 seconds]
pfraze has joined #ipfs
guest449 has joined #ipfs
domanic has quit [Remote host closed the connection]
<wking> jbenet: I think we may want a discussion channel with lower latency than GitHub-issue posts to sort out the 'ipfs file ls ...' UI (go-ipfs#1348)
<wking> Feel free to ping me here or via a direct message if you agree and have time ;)
williamcotton has joined #ipfs
domanic has joined #ipfs
rmk0 has left #ipfs [#ipfs]
besenwesen has quit [Quit: ☠]
besenwesen has joined #ipfs
besenwesen has joined #ipfs
williamcotton has quit [Ping timeout: 255 seconds]
<whyrusleeping> jbenet: dont try adding a massive directory and then view it on the gateway
<whyrusleeping> its kinda slow right now, lol
<lgierth> krl: didn't see you inside or outside
<krl> upstairs!
<krl> where are you? :)
<krl> lgierth: ^
<krl> lgierth: behind the replicator
<krl> lgierth: :/
<lgierth> asd
<whyrusleeping> my 1080p wallpaper collection :)
<lgierth> krl: ok wow wifi outside is really bad
cjdmax has joined #ipfs
rht__ has joined #ipfs
<Encrypt> Hi o/
<JasonWoof> whyrusleeping: cool. this is /ipfs/ so it's immutable right? why the ".." link at the top?
<whyrusleeping> JasonWoof: lol, thats just a side effect of the generic directory listing http handler
<JasonWoof> is that html generated by your client before uploading? or is it a directory tree that's stored at QmXVQM[...] and the HTTP proxy is turning it into that html?
<whyrusleeping> the latter
<JasonWoof> are there ever multiple path segments on ipfs? I mean like /ipfs/QmXVQM[...]/foo/bar
<whyrusleeping> JasonWoof: yeap
<JasonWoof> whyrusleeping: cool. I'll have to read up on how that works. I assumed each dir would have it's own /ipfs/[^/]+
<JasonWoof> whyrusleeping: btw, the ".." links in that go up two directories for me
<JasonWoof> probably because the urls do not end with / even though they are directories as far as ipfs is concerned
<JasonWoof> goes to [...]/b/ not [...]/b/c/
<JasonWoof> the ".." link goes to [...]/b/c/ as you'd expect
<JasonWoof> seems like ipfs doesn't notice/care if there's a / at the end, but the browser does (in how it interprets a href="./..")
tilgovi has joined #ipfs
<JasonWoof> this seems like the sort of thing where I should try to fix it myself
<whyrusleeping> JasonWoof: good catch. ccould you file a bug?
<JasonWoof> but not today, I'm going to feel bad about myself if I don't get a bunch of work done today. [where] should I file this bug?
<JasonWoof> http proxy is part of https://github.com/ipfs/go-ipfs ?
<whyrusleeping> yeap
<JasonWoof> k
<whyrusleeping> first world problems: the coffee shop with the comfiest chairs has the worst wifi
<whyrusleeping> JasonWoof: thanks!
<whyrusleeping> every little bit helps
<JasonWoof> I wish people filed bug reports to me
<JasonWoof> well, I'd like it if the were explicit enough :)
<JasonWoof> I'm co-developer of vor, which is in several distributions. Packagers of multible distros fixed a fairly bad bug, and I didn't hear a peep
<whyrusleeping> thats... scary
<JasonWoof> I didn't even know it was packages for over a year I think
<JasonWoof> I forget how I found out
<JasonWoof> the game wouldn't run if you had a directory named "data" in the current directory
<JasonWoof> (unless it was the game's "data" directory)
<whyrusleeping> lol
<JasonWoof> right? silly mistake
<JasonWoof> now I have ~/data but I guess I didn't at the time
Encrypt has quit [Quit: Quitte]
bren2010 has quit []
mildred has joined #ipfs
Tv` has joined #ipfs
atrapado has joined #ipfs
zabirauf has joined #ipfs
williamcotton has joined #ipfs
<whyrusleeping> Luzifer: is there a story on gobuilder builds for windows?
<Luzifer> mh?
<Luzifer> story?
<Luzifer> whyrusleeping: what do you mean with that question?
<whyrusleeping> oh, there arent windows builds for go-ipfs
<whyrusleeping> i was wondering if there was a reason for that
<whyrusleeping> or just that we havent set it up yet
* JasonWoof hugs mxe
<Luzifer> yep there is a reason: https://github.com/Luzifer/gobuilder/issues/12
williamcotton has quit [Ping timeout: 276 seconds]
<Luzifer> build tags… I still have not a real clue how to do them and tbh I neglected that topic a bit in favor of other projects
<whyrusleeping> Luzifer: ah, gotcha
<Luzifer> in the meantime I'm sure its more complicated than what I wrote in that issue… it has to have different build-tags for different platforms…
<Luzifer> on windows without fuse, on linux with fuse and so on…
<whyrusleeping> yeah... it would have to be a map of GOARCH/GOOS to build tags
<Luzifer> ack
<Luzifer> I feel a bit guilty that two features I wanted to build are still not in… commit signing support / artifact signing and build tags…
<whyrusleeping> go build them! :D
<whyrusleeping> i'll sit here and cheer you on :P
<Luzifer> but all the other stuff on my list…
<Luzifer> /o\
<Luzifer> and in 20min I have to go to the fire dept to get a briefing on the first of all those trucks… xD
cjdmax has quit [Remote host closed the connection]
<pjz> is tehre something that should be pruning events.log ?
<whyrusleeping> pjz: there is supposed to be
<whyrusleeping> but it picks and chooses who it likes
<whyrusleeping> and if it doesnt like you, it will eat your disk
* pjz frowns at 174M events.log
<whyrusleeping> yeah.. thats not supposed to happen
<pjz> so I need to periodically do shutdown/nuke-logs/start up?
* whyrusleeping grimaces
<whyrusleeping> yeah...
<whyrusleeping> i'll bump priority on fixing that
<pjz> if you want to encourage people to run long-lived nodes that would seem to be a priority
<pjz> owait
<pjz> for some reason the default config says maxlogsize is 250M
<Luzifer> total 644144
<Luzifer> -rw-r--r-- 1 luzifer staff 250M 11 Jun 18:31 events-2015-06-11T16-31-42.192.log
<Luzifer> -rw-r--r-- 1 luzifer staff 65M 12 Jun 19:31 events.log
<Luzifer> nkay…
<whyrusleeping> oh, so it still might be working for you
<Luzifer> 315M logs O_o
<whyrusleeping> lol
<pjz> but it rolled okay for him
<pjz> hm, I just set it to 10M
* pjz wonders how/if it will fix itself
<Luzifer> yeah… the first line of the rotated log is weirdly truncated but yeap.
<Luzifer> hm the last one too :D
inconshreveable has joined #ipfs
<whyrusleeping> we're going to switch that to only outputting to an http endpoint when connected
<whyrusleeping> so no events will be logged unless you curl localhost:5001/eventlogs or something
<Luzifer> and off to the fire dept. laters!
<whyrusleeping> Luzifer: peace
<pjz> okay, changed setting, bouncedit and it rolled the log
<pjz> whyrusleeping: smaller default would do fine, It hink
<whyrusleeping> pjz: okay, sounds good
<pjz> a quarter gig of log files seems excessive unless you're debugging or something
<pjz> oh, and it's actually twice that b/c ther's the active one and then the one allowed backup file (by default)
<whyrusleeping> yeah, that is really excessive
atrapado has quit [Quit: Leaving]
<whyrusleeping> does anyone else not trust /tmp ?
inconshreveable has quit [Remote host closed the connection]
<pjz> "trust" how?
<pjz> I trust it to hold my files... temporarily
domanic has quit [Ping timeout: 264 seconds]
inconshreveable has joined #ipfs
<whyrusleeping> i'm writing data to a file
<whyrusleeping> and i think i'm getting different data back
rht__ has quit [Quit: Connection closed for inactivity]
<pjz> that seems... bad
<pjz> how long between write/read ?
<whyrusleeping> oh, i guess i'm just dumb
<pjz> ceramic squirrel effect
<whyrusleeping> i was using O_WRONLY and O_TRUNC as my open flads
<whyrusleeping> flags*
<whyrusleeping> but i also needed O_TRUNC
<whyrusleeping> for some reason
<whyrusleeping> even though each call was made in an empty os.TempDir()
* pjz remembers there's subtleties between tempdir and tempfiles, but forgets what they are - but I know they exist so I can look them up when needed!
<whyrusleeping> meh
<pjz> the wrong one can lead to security holes via race conditions
<whyrusleeping> not worried about security holes in my test code, lol
<whyrusleeping> but thanks for the notice
<daviddias> folks, leaving for today, going to local Lisbon traditional festivities. Have an awesome weekend (I'll be around tomorrow :)) Cheers!
<whyrusleeping> daviddias: have fun!
williamcotton has joined #ipfs
williamcotton has quit [Ping timeout: 252 seconds]
<JasonWoof> I've been reading the debian security mailing list. There have been several vulnerabilities about /tmp. I think the usual insecurity is that if an attacker can predict what path in /tmp you're going to open, they can put a symlink there, and trick you into doing stuff to other files on the system as your user
<ipfsbot> [go-ipfs] whyrusleeping created wantmanager-cleanup (+1 new commit): http://git.io/vIbzy
<ipfsbot> go-ipfs/wantmanager-cleanup b12cc09 Jeromy: select with context when sending on channels...
<JasonWoof> thus a sort of privilege escelation
<ipfsbot> [go-ipfs] whyrusleeping opened pull request #1367: select with context when sending on channels (master...wantmanager-cleanup) http://git.io/vIbzA
<whyrusleeping> JasonWoof: ah, that makes sense
<whyrusleeping> i almost feel like /tmp should be isolated per user
<JasonWoof> whyrusleeping: yeah, a more systematic fix to this would be nice. I use ~/tmp in my scripts
<JasonWoof> that's less practical for system daemons though
<whyrusleeping> yeah
<whyrusleeping> system daemons should really avoid /tmp
<whyrusleeping> /var/run is a good place for them
<whyrusleeping> interesting side effect
<whyrusleeping> naming branches X/Y
<whyrusleeping> causes git to make directories in the .git/refs/heads folder
<pjz> yeah
<pjz> don't conflict with your remotes
<pjz> they get branch-dirs by default
<whyrusleeping> yeap
<pjz> so what happens right now if I change a file in my ipfs dir?
<whyrusleeping> hrm?
<whyrusleeping> in .ipfs?
<pjz> er, in *an* ipfs dir
<whyrusleeping> uhm.. through the fuse interface?
<pjz> yes
<whyrusleeping> /ipfs/QmBlah/X?
<whyrusleeping> its readonly
<pjz> right
<pjz> oh
<pjz> really?
<whyrusleeping> yeap
anshukla has joined #ipfs
anshukla has quit [Client Quit]
<pjz> I thought anything you added via 'ipfs add' got pinned by default?
<whyrusleeping> pjz: that is true
<pjz> so I added some stuff then did 'ipfs pin ls' and got... nothing
<whyrusleeping> ipfs pin ls shows (by default) 'direct pins'
<pjz> ah
<whyrusleeping> direct pins are almost never used (we didnt think this through way back when)
<pjz> got it
<whyrusleeping> ipfs pin ls -r
<whyrusleeping> er
<whyrusleeping> ipfs pin ls --type=recursive
<pjz> I see
<whyrusleeping> Sprint Checkin! cc jbenet cryptix wking lgierth krl kbala_ rht__ daviddias dPow chriscool gatesvp
<whyrusleeping> (sprintbot and pinbot appear to have been disconnected in a netsplit
pinbot has joined #ipfs
sprintbot has joined #ipfs
www has joined #ipfs
Encrypt has joined #ipfs
williamcotton has joined #ipfs
<whyrusleeping> my sprint checkin: recovering from being sick, looking for memory leaks and other fun things in bitswap, and researching spdy/http2 muxers
<whyrusleeping> and other options in that area
williamcotton has quit [Ping timeout: 276 seconds]
guest4491 has joined #ipfs
guest449 has quit [Ping timeout: 265 seconds]
fleeky has quit [Ping timeout: 256 seconds]
guest4491 has quit [Ping timeout: 256 seconds]
fleeky has joined #ipfs
<whyrusleeping> jbenet: ping
<pjz> did the netsplit eat them somehow?
<whyrusleeping> yeah, i dont have them able to reconnect currently
<whyrusleeping> i should work on that
hellertime has quit [Ping timeout: 256 seconds]
inconshr_ has joined #ipfs
inconshreveable has quit [Ping timeout: 265 seconds]
alu has quit [*.net *.split]
gwillen has quit [*.net *.split]
infinity0 has quit [*.net *.split]
alu has joined #ipfs
infinity0 has joined #ipfs
marklock has quit [Ping timeout: 246 seconds]
williamcotton has joined #ipfs
williamcotton has quit [Ping timeout: 246 seconds]
kbala has joined #ipfs
domanic has joined #ipfs
<wking> checkin: iterating on intended behavior in specs#12, go-ipfs#1348, go-ipfs#1351. Pushed seed changelog for our next release in go-ipfs#1361.
Bioblazin has joined #ipfs
Bioblaze has quit [Ping timeout: 256 seconds]
<whyrusleeping> wking: thanks!
<whyrusleeping> anything blocking you?
<wking> whyrusleeping: I have a number of open issues / PRs in go-ipfs that are blocked on deciding on a spec ;)
pfraze has quit [Remote host closed the connection]
<wking> e.g. Docker driver needs to return file size in its stat results, but access to those sizes is blocked on go-ipfs#1348 landing and then that command being exposed in ipfs-shell
<wking> the Docker driver is currently just catting the file, reading the whole stream, and returning the count of bytes read.
<whyrusleeping> wking: what decision needs to be made on 1348?
<wking> Which works, but is less than ideal ;)
<wking> I think we want some hand-holding from you on the Context handling ;). See https://github.com/ipfs/go-ipfs/pull/1348#issuecomment-111340373
guest449 has joined #ipfs
<wking> And jbenet is trying to reach enlightenment on the type handling, but I'm trying to get that postponed until after #1348 lands ;) https://github.com/ipfs/go-ipfs/pull/1348#issuecomment-111523827
<whyrusleeping> hrm... in regards to the context
<whyrusleeping> we should probably keep that context in there
<whyrusleeping> if the directory object exists, but the children are not accessible, that may hang
<wking> why not stick with req.Context()?
<whyrusleeping> i dont know that req.Context() actually gets cancelled reliably when the user ctrl+c's
<whyrusleeping> although, that may just be my paranoia
<wking> that seems like a separate issue that should be solved in a separate PR ;)
<wking> and we should setup our own local replacement Context that we use everywhere inside commands until that is solved (or tested to be already solved)
<wking> I've be very surprised if the minute-timeout context is cancelled reliably when the user ctrl+c's ;)
<jbenet> whyrusleeping: pong! I can look at things in ~30min
<whyrusleeping> wking: the minute just ensures things dont hang forever
<whyrusleeping> we want to ensure that *something* cancels the context eventually
<whyrusleeping> thats why we would use a timeout derived from the requests context
<wking> *What
<whyrusleeping> wking: being wrong
<wking> this probably ties in with https://github.com/ipfs/go-ipfs/issues/1325
<whyrusleeping> that would be nice to have as well
<whyrusleeping> it should set a timeout on the req.Context
<wking> So I think #1325 gives you a global timeout, but we're still missing ideas for how to have that scale (e.g. for pinning huge trees)
<wking> Currently we're just hard-coding these sub-timeouts
<wking> And there are lots of sub-timeouts that we might want to control, but maybe we can get that down to a single per-resolution timeout as an optional replacement for the per-command timeout?
<whyrusleeping> yeah, the global timeout and sub-timeouts are separate
<whyrusleeping> wking: as far as 1348 goes, just make it a timeout child of the req.Context, and that SGTM
<wking> whyrusleeping: ok, sounds good for now
<whyrusleeping> i think with the commands stuff, we just need to keep chipping away at making it better
Blame has quit [Quit: Connection closed for inactivity]
williamcotton has joined #ipfs
williamcotton has quit [Ping timeout: 244 seconds]
tilgovi has quit [Ping timeout: 265 seconds]
domanic has quit [Ping timeout: 272 seconds]
tilgovi has joined #ipfs
therealplato has quit [Read error: Connection reset by peer]
<whyrusleeping> jbenet: poke
domanic has joined #ipfs
<jbenet> ok here finally
<jbenet> whyrusleeping what's up?
<whyrusleeping> uhm, 1367
<whyrusleeping> also, wanted to ask about some encryption stuff i was thinking about
tilgovi has quit [Ping timeout: 256 seconds]
mildred has quit [Quit: Leaving.]
* spikebike waits for encryption discussion
<whyrusleeping> basically, we have two ways we can encrypt data
<whyrusleeping> we can either encrypt the data within each dag node, or encrypt the entire block of every node in the tree
<whyrusleeping> i prefer the latter as it hides a lot of metadata
<jbenet> oh both have uses.
<jbenet> we need both.
<whyrusleeping> what is the use of the first one?
<jbenet> and a third kind: serialize a dag, encrypt _that_, and re-import / chunk
therealplato has joined #ipfs
<jbenet> the first means the dag can be replicated without knowing what's in the data
<jbenet> it's a bit similar to the third-- actually.
<jbenet> and the third is the real security thing-- everything else leaks.
<wking> the drawback is that with good crypto, you'll have no overlap after rehashing a slightly-tweaked DAG
<wking> ^ for the full-security (3)
<wking> so like always, it's going to be a balance between usability and security ;).
grawity has quit [Quit: "i found my savior in these shards of shattered glass"]
grawity has joined #ipfs
Encrypt has quit [Quit: Quitte]
<ipfsbot> [go-ipfs] jbenet deleted wantmanager-cleanup at b12cc09: http://git.io/vINAQ
<jbenet> which is why we want to support the different ones.
williamcotton has joined #ipfs
<whyrusleeping> alright, and what i'm thinking is we generate a random key of some large length and use that as the key for AES (or similar) encryption
<spikebike> my solution for low security was to encrypt with a files checksum. So sure if your attacker has a copy of the file they can tell if you have it. But if you encrypt your plan to take over the world your attacker can't feasible read it until they already have a copy of the plaintext.
<whyrusleeping> we then encrypt that key with one of the selected keypairs
<spikebike> for low security and deduplication
<jbenet> whyrusleeping yep and pre-pend it.
www1 has joined #ipfs
<whyrusleeping> and put it into a special unixfs dagnode that points to the root of the encrypted dag
<jbenet> so it's <symmetric-key-encrypted-with-keypair><data-encrypted-with-symmetric-key>
Wallacoloo has joined #ipfs
<jbenet> not sure if it should be unixfs node-- long term i want to get decryption on stackstream
<jbenet> so "ipfs ss cat <node>" could decrypt.
<whyrusleeping> i want reading an encrypted file that i have the key for to be seamless
<jbenet> agreed
<whyrusleeping> 'ipfs cat <node>' would decrypt
www has quit [Ping timeout: 252 seconds]
<spikebike> whyrusleeping: with a ipfs loadkey <key> or somesuch?
<jbenet> whyrusleeping: anything else on this?
williamcotton has quit [Ping timeout: 265 seconds]
<whyrusleeping> just thinking about the dag format
<whyrusleeping> i'm still thinking it should be a special dag node
<wking> generic encryption should live above generic sharding/fanout and below UnixFS
<jbenet> wking: wanted to discuss 1348 ?
<wking> sure
<wking> whyrusleeping: does this sound like it corresponds to reality? https://github.com/ipfs/go-ipfs/pull/1367#discussion_r32360308
<jbenet> whyrusleeping yes it should be-- take a look at the diagrams i began on keychain-- there's a datastructure in mind there, where the keys and cyrptographic artifacts (proofs, ciphertexts) are objects.
<whyrusleeping> wking: no. those functions get called
<wking> hrm
<wking> do you know where from?
<wking> jbenet: on 1348, whyrusleeping cleared up the Context issue 30 minutes ago here
<wking> On POSIX-ls interop, I'm all for it. Which POSIX options do you want?
<wking> Because I like POSIX-ls interop, I don't like your -q option, and I couldn't find references to it stripping trailing newlines in a grep through the Git docs
<wking> Do you still need -q? I think the POSIX-specified -q (hiding non-printable characters) is low enough priority to skip it for now
<whyrusleeping> wking: ls by default just prints names
<whyrusleeping> adding '-p' will include the / on dirs
<wking> And in the face of our current lack of explicit type information (ipfs/ipfs#36), do we prefer enum ints or strings in the JSON output now?
<wking> whyrusleeping: unless you pass in several directories (for just printing names)
<wking> I'm fine mimicking that
<wking> but that doesn't match 'ipfs ls'
<wking> But 'ipfs ls' is probably different enough that trying to match POSIX-ls doesn't matter
<wking> I'd consider dropping all the UnixFS stuff from 'ipfs ls' anyway, and just have it be a list of an objects links (possibly hiding fanout if we resolve ipfs/specs#10)
<wking> but that's getting a bit off topic for go-ipfs#1348 ;)
<jbenet> wking: -q is already in many ipfs commands and i want to keep it that way. git does have -q in some commands too. e.g. man git-pull | grep -- -q
<wking> jbenet: right, but it always means "hide unneccessary data". You'd mentioned something about newlines https://github.com/ipfs/go-ipfs/pull/1348#issuecomment-111354847
<jbenet> wking: i dont necessarily need the "/" on names, i dontn care much for it, but do want to do what people expect there.
<wking> And I couldn't find a reference for that
<jbenet> wking: yes it hides "unnecessary output", including extra newlines in some cases.
<wking> So POSIX-ls has -p and -F for trailing slashes and related markers
<wking> We can add options mimicking that
<jbenet> no, i'd like to keep using -q across the ipfs toolchain please.
<wking> We can't match POSIX for 'ipfs file ...'?
<jbenet> i'd like consistency with ourselves first
<jbenet> we can add aliases
<jbenet> to improve what other people expect
<wking> That -q interpretation clashes with POSIX
<wking> do we want an option for replacing non-printable characters in names?
<jbenet> we won't match all of the kitchensink of options that the standard tools habe.
<jbenet> have*
<wking> Is consistency with the rest of the IPFS CLI worth breaking compatibility with POSIX ls?
<whyrusleeping> screw posix.
<wking> ;)
therealplato has quit [Ping timeout: 256 seconds]
<nell> this cheers me up https://u.teknik.io/s07vF2.png
<jbenet> yes, worth it. it's already very different.
<wking> ok
<wking> so if POSIX-compat is at least debatable, what do you want the UI to look like here?
<wking> do we write our own options for suffix chars or use POSIX?
<jbenet> which suffix chars?
<wking> of course, we don't have executables, FIFOs, or symlinks at the moment, but we likely will have at least executables and symlinks in the nearish future
therealplato has joined #ipfs
<spikebike> wking: sure, but many other aspects of posix are pretty much incompatible with a performant distributed filesystem
<jbenet> we can use some of those if (and when) we need them, good point to keep in mind.
<wking> sure, but knowing the types you're dealing with doesn't seem like one of those things ;)
<jbenet> but it's not _needed_ now.
<jbenet> or is it?
<whyrusleeping> its not
<wking> the current 'ipfs ls ...' has slash appending on by default (and maybe not configurable at all)
Wallacoloo has quit [Ping timeout: 276 seconds]
<jbenet> yeah and i don't particularly like it, though some do and requested it. it's a standard enough thing (apparently osx ls has it on by default :/)
<wking> Knowing whether something is a file or directory for example probably helps here: https://github.com/ipfs/go-ipfs/issues/1365
<jbenet> so i was fine merging it
<jbenet> wking: but in that we're not in the shell... we're in ipfs where we have datastructures
<wking> I think in situations like this, we will do pretty well mimicking POSIX
<wking> jbenet: ah, I wasn't sure. Thought it might be driving the command API
<jbenet> wking i agree, but _when_ we actually need it. definitely good point to bring it up and keep it in mind (so, let's keep the suffix symbols in mind for the future)
<whyrusleeping> wking: 1365 is a trivial fix
<whyrusleeping> types arent needed
<wking> jbenet: so no suffixes or option to enable them in 'ipfs file ls ...'?
<jbenet> wking: do we need them _today_ ?
<jbenet> wking: if we try to build everything we'll never finish.
<jbenet> wking: it's good to think things through and make sure we dont build the wrong thing
<jbenet> wking: and it's good to signpost the direction we'll go
<jbenet> wking: do you need that type in the ls output?
<jbenet> wking: it looks to me like that's just on the code / datastructure side
<wking> no, I could grab it from a stat call if we had that
<jbenet> wking: is it going _through the cli_ ?
<wking> so it's going through the JSON command
<jbenet> so-- you have the full objects, with types, right?
<jbenet> not POSIX type suffixes?
<wking> yeah, no need to worry about the text output for the Docker-registry driver, other than we have to land a text output with the PR ;)
<wking> jbenet: so no suffixes or option to enable them in 'ipfs file ls ...'? ;)
<jbenet> no, we don't need them.
<wking> good
<wking> So for what type information comes out in the JSON: strings or enum ints?
<wking> Or none, since I guess I can add 'ipfs file stat ...' or some other single-entry endpoint to get an object's type....
<jbenet> i'd use strings for types.
<wking> Looks like Coreutils ships a 'stat' call if we wanted a template UI...
silotis has quit [Ping timeout: 264 seconds]
silotis_ has joined #ipfs
<wking> Ok, with types in the ls output I don't need a separate stat call
<wking> Moving on to argument names
<wking> to separate out raw arguments from their resolved hashes
<wking> if so (and it sounds good to me), please say yes ;)
<wking> And I'll PR to update 'ipfs ls ...' accordingly
<wking> (as well as adjusting the 'ipfs file ls ...' implementation
<wking> )
williamcotton has joined #ipfs
<jbenet> wking: why do you need "argument"
<wking> you send in a list of arguments to ls, and want to grab the result from a particular argument from the result set
<jbenet> and can't do it based on order?
<wking> We need that in the text results to separate various directory listings (e.g. 'ls /bin /etc')
<jbenet> also, may want to deduplicate in that case
<wking> you could grab based on order, but the text output needs the raw path
<wking> ah, hadn't thought of checking to dedup, that's a good idea
<wking> not sure how POSIX handles it, or whether it dedups unequal synonyms
<jbenet> i'm still not convinced the argument thing is needed. the "Hash" injected property is enough.
<jbenet> i just mean to dedup on the transport
<wking> so 'ipfs file ls /bin /etc' would print:
<wking> <hash-for-bin>:
<jbenet> if A -> B -> C
<wking> <contents-of-bin>
<wking> <blank-line>
<jbenet> and i do "ipfs ls A B C" i only need each once.
<wking> <hash-for-etc>:
<wking> <contents-of-etc>
<wking> ?
<jbenet> no, it should print what you entered:
<wking> so if that info is going to be in the returned object, it's going to get rendered in the JSON
<jbenet> /bin:
<jbenet> <contents-of-bin>
<jbenet> <blank-line>
<wking> unless we jump through some hoops to keep that from happening
<jbenet> the _text_ output on the cli does not need to be the exact same json output.
<jbenet> there are things we do in the cli for cli convenience
<jbenet> and dont need to encumber json consumers for it.
<wking> no, but the text output has to be build from the same Go object
<jbenet> the cli called the command, the cli knows the arguments.
<wking> and the JSON marshaller is currently just serializing the structure returned by the output-agnostic command
<jbenet> but sure.
<jbenet> it's fine
<wking> so it's fine as a map? And I need to figure out the deduping/order-preservation?
<wking> or it's fine as an array, and we can keep it like it currently is? Or we need to add the object's Hash to how it currently is?
meadman has quit [Quit: WeeChat 0.4.2]
<jbenet> well the map should not be { path : <object> } it should be { hash : <object> }
<wking> with an array of arguments inside the <object>?
tso has joined #ipfs
<wking> so ls A/B/C B/C C would have a map with <hash-C>: {"Arguments": ["A/B/C", "B/C", "C"], ...}?
<wking> ah, have to run. That gives me enough for a reasonable reroll anyway ;)
<wking> Feel free to post further specs to the issue
<wking> (obviously ;)
<jbenet> result of calling: `ls Foohash/bar Barhash`
www1 has quit [Ping timeout: 276 seconds]
Guest33644 has joined #ipfs
Guest33644 has quit [Client Quit]
gwollon has joined #ipfs
hellertime has joined #ipfs
compleatang has quit [Quit: Leaving.]
gwollon has quit [Quit: leaving]
sbruce has joined #ipfs
sbruce has quit [Quit: WeeChat 0.4.2]
williamcotton has quit [Ping timeout: 255 seconds]