<ipfsbot>
go-ipfs/fix/iptb-fails ed55efc Jeromy: update iptb dependency again, and pick different ports for each iptb cluster
<ipfsbot>
[go-ipfs] whyrusleeping opened pull request #1194: increase iptb test stability (master...fix/iptb-fails) http://git.io/vJa2c
<cryptix>
whyrusleeping: how about a vendorfail scoreboard? ;)
<whyrusleeping>
cryptix: did i fuck up? lol
<cryptix>
whyrusleeping: only nagging because the end is near :PP
<cryptix>
but yea, you did ˆˆ
<whyrusleeping>
lol, the end??
<whyrusleeping>
mehhh
<cryptix>
Godeps/_workspace/src/github.com/whyrusleeping/iptb/main.go:21:2: cannot find package "github.com/jbenet/go-multiaddr-net/Godeps/_workspace/src/github.com/jbenet/go-multiaddr" in any of:
<whyrusleeping>
why is this so hard
<whyrusleeping>
like
<whyrusleeping>
i
<whyrusleeping>
cant
<whyrusleeping>
even
<jbenet>
hahahaha
<cryptix>
somebody should have spared mappum from implementing chunked encoding btw
<whyrusleeping>
thats it
<cryptix>
poor man
<whyrusleeping>
cryptix: hahahaha
<jbenet>
yeah chunked encoding is hard.
<jbenet>
also-- i think we're super close
<whyrusleeping>
alright, im done working on ipfs, gonna go write my package manager
<jbenet>
whyrusleeping: if you get pinbot to run on its own and working, we can probably do it.
<jbenet>
whyrusleeping: noooo we need pinning first
<jbenet>
whyrusleeping: i've been staving off that very same urge for months now.
<jbenet>
we're so close!
* whyrusleeping
scratches his neck like a crack addict
<cryptix>
sysly though, dont let him see this: QmXGwPNexMzaKQhxo3Y59jG3VGyZXGFkB8J63mFZvxUAsj
<whyrusleeping>
cryptix: wow
<cryptix>
and i'm not even sure that is necassary - if you don't set Content-Length the http server does the exact same thing by itself
<cryptix>
which is most likely why you did it, to get both
<jbenet>
cryptix: with all these things i'd make sure to ask mappum why he made certain choices. i dont think he would've _wanted_ to implement chunked encoding, but figured he had to along the way for some reason. of course things can be missed, but other times, there's silly complications. -- like https://github.com/golang/go/issues/9661
<cryptix>
i'm not saying it was ill advised or his fault or anything... i missed obvious stuff in the stdlib, too
<mappum>
did that exist when i wrote the chunk stuff? i remember looking for it and not finding it, i was even thinking about making a PR
<ehmry>
writing your own package manager eh?
<mappum>
oh wait, i knew that the http server automatically handled chunking. i believe the problem was that their implementation doesn't let us start writing until the request is completed
<krl>
doing javascript package management on ipfs would be cool
<mappum>
and we need to be able to write as the client is still writing their request (e.g. when uploading files with the add command)
<krl>
there's no reason for ipfs browsers to cache 100 different jqueries in 100 different minified blobs
<whyrusleeping>
jbenet: 1181
<whyrusleeping>
ehmry: yea, i'm tired of gos package management
<jbenet>
krl: agreed. we should make PRs to browserify and webpack to do this right.
<whyrusleeping>
mappum: that makes sense
<jbenet>
krl: one complication is how requests are handled the first time (one request for 1MB tends to perform better than 10 requests for 0.1MB)
<cryptix>
mappum: yea, the httputil was there. i see what you mean with the not waiting for the body to be read though.
<jbenet>
cryptix: see mappum's comments
<jbenet>
kk
<krl>
jbenet: first time webui usage is already quite slow
<krl>
i don't think this is a big problem
<ehmry>
go does need an alternative package utility
<whyrusleeping>
ehmry: ive been thinking about it and doing 'UX test runs' for a bunch of different workflows
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ehmry>
when I got nix to deal with go packages I really wanted to rewrite 'go' so it could be driven with environmental variables the way scripts are in nix but I found better things to do
<ehmry>
i am curious how hard it would be to pick apart the go command
<krl>
regarding pinning, i'm trying to pin -r about 200mb of data from my laptop to my vhost, and it's constantly timing out. i think a better approach would be to return directly, and have the daemon expose pinning state (how much is fetched) as a progress bar-thing
<ehmry>
jbenet, ok, I'll give them a look
<jbenet>
krl: interesting points. yeah there's a few different desires-- one problem with that is we'd like the semantics of "ipfs pin -r <hash>" to be that it exits 0 iff <hash> and all descendants are fetched and pinned.
<jbenet>
krl: that way can use it in scripts.
<jbenet>
krl: but this could be done by other commands or options
<jbenet>
_lots_ of tooling to make.
<krl>
it could return a promise, for example
<krl>
or make it a command switch, pin --synchrounous
<jbenet>
krl: i wouldn't necessarily want a promise if i want to automate this in a bunch of shell scripts. sometimes i just want it to do the simple + dumb thing: hang till you're done
<jbenet>
(yeah that's my suggestion re being an option)
<krl>
i'm just running it in a loop for 3 hours now, and still getting blocks, but it has to re-check everything each time
<jbenet>
whyrusleeping: i responded to 1181. need anything else from me atm?
<jbenet>
ok-- i'm going to go back to processing important email / todos. if you have something pressing you need from me poke. (cc whyrusleeping cryptix krl wking)
<jbenet>
(sorry -- super behind these last few days. weddings + family are not very precise on time constraints)
<cryptix>
jbenet: if you have the time, take a look if you are fine with 1189, i addressed the comments, if its merged i can continue with the http gw
ei-slackbot-ipfs has quit [Remote host closed the connection]
<krl>
no prob, i'll be on tomorrow
ei-slackbot-ipfs has joined #ipfs
<krl>
btw, current geoip root hash is QmQQ3BUpPjgYiTdhp4H9YWSCtoFXs8t91njhpvXNNLd3yB
<jbenet>
krl: btw been meaning to sink into geoip! props on getting it to work
Daboloskov has joined #ipfs
<krl>
it could need some pinning love!
<jbenet>
krl: pinning now
<ehmry>
jbenet, cryptix I don't have much to say about go these days, its all c++ for me lately :\
<jbenet>
whyrusleeping: how do we run a permanent pinbot ?
<jbenet>
our DO boxes? heroku?
<ehmry>
but I am working towards a lazy package manager that can install on demand using a VFS
<ehmry>
but it won't be for unix :/
<jbenet>
:\
Daboloskov has quit [Client Quit]
<cryptix>
ok ppl - taking off. cya
<ehmry>
I will be keeping my eye on the Hyper Modular Programming System
<jbenet>
cryptix: ok will TAL at 1189
<jbenet>
cryptix: night!
jabroney has joined #ipfs
<whyrusleeping>
jbenet: i put it on a machine that i dont turn off at night
<jbenet>
whyrusleeping: it doesn't appear to be in here
<whyrusleeping>
uno segundo
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
clever has joined #ipfs
pinbot has joined #ipfs
<clever>
is there a way to add files to an existing folder within ipfs?, or is the folder name a hash of all of its files, and by nature depends on the file contents?
<jbenet>
guys-- this is pretty bad! `ipfs refs -r <hash>` can exit 0 WITHOUT pulling all refs.
<whyrusleeping>
jbenet: i thought we knew about that one for a while now
<jbenet>
there's some nasty bug here.
<jbenet>
i thought it happened with get only-- not with everything
<jbenet>
clever: right-- you have it right. we'll have some tooling to make it easier to add content to dirs and get the new root (see issues on github re "mv" and "more git like commands")
<whyrusleeping>
could have /block and /object endpoints on the gateways
<jbenet>
i think this is because `ipfs ls` assumes unixfs in children or wat?
<jbenet>
whyrusleeping: yeah that'd be nice. also maybe /files or whatever we want to use unixfs strictly for.
<whyrusleeping>
no, it tries to parse the data as unixfs
<jbenet>
/ipfs route should do whatever we end up defined to be whatever we make the data segment output -- i.e. if we go the "embedded language" route
<jbenet>
ah right
<jbenet>
god-- github should make issues and CR be distributed-- should be files inside the repo-- on specific branches.
<jbenet>
"git CR that doesn't suck"
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
* whyrusleeping
shrugs
<jbenet>
i want to CR on a plane tonight-- and i wont be able to because all comments have to go over the network.
<whyrusleeping>
jbenet: pay $5 for gogo
<krl>
jbenet: sad times?
<jbenet>
whyrusleeping: not all flights have it :(
<krl>
it's not a unixfs object
<whyrusleeping>
or buy a chromebook and get it for free
<whyrusleeping>
:(
<jbenet>
krl: yeah i know-- the gateway is being silly. it should be smart enough to handle non-unixfs things
<whyrusleeping>
jbenet: what do you want as the larger fix for the OOM bandaid?
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
<jbenet>
whyrusleeping: i think either making it so all the handshake has a size limit (right now the handshake will read a protobuf from the stream which may be arbitraily big-- or may have arbitraily big byte buffers in it).
<jbenet>
whyrusleeping: or adding a fixed size header string at first and close the connection if no match. (something like "/ipfs/secio/0.1.0"). then only willing adversaries can screw it over, not random net probes
<whyrusleeping>
but this bug is happening after the handshake completes
<clever>
hmmm, i cant seem to run ipfs mount as non-root, and when i do run it as root, non-root users cant access /ipfs/
<whyrusleeping>
clever: you need to configure fuse properly
<jbenet>
whyrusleeping: ohhh writeMsgCtx appears to use msgio-- maybe cryptix fixed it
<whyrusleeping>
we have a guide... searching
<jbenet>
yeah but it was also vulnerable before the handshake too
<jbenet>
brb reloc
<whyrusleeping>
mmkay
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
krl: can you make the sprint mtg tomorrow?
<clever>
wasnt reading it character for character, didnt expect a second folder to come out of nowhere
<clever>
mounts just fine once i chown both
<whyrusleeping>
i kinda want an 'ipfs setup' type command that will do a bunch of these things automatically
<whyrusleeping>
i hate having to fix fuse mountpoints on every new machine i touch
<clever>
the gentoo ebuild made them both, and created the directories and init scripts to manage the daemon&mount for me
<whyrusleeping>
jbenet: i really wish i could define a git branch as the combination of a number of other branches
<clever>
but it stores the data in /root/.ipfs/ and runs the daemons as root
<clever>
not really where i would want to leave them long-term
<whyrusleeping>
you can change that
<whyrusleeping>
just move it and set IPFS_PATH to match
<clever>
what is /ipns/ for?
<whyrusleeping>
its a mutable filesystem backed by the naming system on top of ipfs
<clever>
all i can see in there is the id for my node and a symlink to that id
<whyrusleeping>
yeap
<clever>
what can i do with it?
<whyrusleeping>
so you can write whatever you want inside /ipns/local
<whyrusleeping>
and once you do, i can see it under /ipns/<your peer id>
<clever>
ah
<whyrusleeping>
its your signed public keyspace
pfraze has joined #ipfs
<clever>
so /ipns/ is to list a writtable folder, identified by the peerid, and ipfs is for read-only folders, where the folder name is a hash of its contents
<whyrusleeping>
yeap
domanic has joined #ipfs
<clever>
i'm guessing the ipns stuff is also public, since anybody with my public id can read it?
<clever>
and that would be visible on the dht
<whyrusleeping>
yeap!
<clever>
and internaly, ipns is just using ipfs to store the actual files?
<whyrusleeping>
exactly
<whyrusleeping>
ipns is just a mutable pointer to an ipfs object
<clever>
ah
<clever>
so when you modify the folder, it just re-creates the directory object, with the new contents, and posts it up under my peer id
<whyrusleeping>
correct!
* whyrusleeping
thinks its pretty slick
<clever>
yep
<clever>
i just get a 404 error when i open localhost:8080
<clever>
what can i do to fix that?
<krl>
clever: webui is at port 5001 /webui
pfraze has quit [Remote host closed the connection]
<clever>
ah
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has quit [Ping timeout: 272 seconds]
<clever>
krl: i'm guessing /webui is an http redirect to a hard-coded ipfs object, the latest version of the html page?
jabroney has quit [Quit: Leaving.]
<krl>
clever: correct
<krl>
this will however be moved to ipns when it's stable
<whyrusleeping>
and ipns will be stable as soon as i get the dht records spec
<clever>
and does my wifi ever suck, its been loading /webui for the past 15 minutes
<krl>
whyrusleeping: (y)
<krl>
clever: hmm, is your daemon running?
<krl>
eh. sorry, of course
<clever>
krl: yep
<krl>
no errors in console?
<clever>
correct
<krl>
should not take too long, could be a network hiccup
<clever>
my wifi just sucks horidly for some reason, if i try to move any data, it drops to under 1kb/sec
<krl>
clever: if you do ipfs swarm peers
<krl>
do you get a list of peers?
<krl>
could be that you're disconnected and cannot fetch any data
<whyrusleeping>
i need to write a tool that works with expvarmon called picknchoose
<cryptix>
picknchoose what? nodes?
<whyrusleeping>
lol, it was supposed to be a pokemon pun...
<whyrusleeping>
but it wasnt quite obvious enough
<cryptix>
ohlol
<cryptix>
i see
<whyrusleeping>
i wish i knew why my iptb tests are randomly failing...
<whyrusleeping>
the issues i'm seeing are ones that i'm pretty sure i just fixed
<whyrusleeping>
and i cant repro locally
<cryptix>
it just fails horribly on travis?
<whyrusleeping>
yeah, travis hates it for some reason
<whyrusleeping>
i even changed the code to wait for the api endpoint on all the nodes to be up, and have it check the id's of each through the endpoint to verify
<cryptix>
do they speak to the outside world or is it local testing only?
<cryptix>
gotta admit i didnt look into iptb at all yet
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
<whyrusleeping>
i think its local only
<whyrusleeping>
iptb manages a group of ipfs nodes for you
<whyrusleeping>
it sets their configs each on different ports and has them bootstrap to eachother
<whyrusleeping>
and makes it easy to bring them all down afterwards
<cryptix>
sounds nice so far :)
ei-slackbot-ipfs has quit [Read error: Connection reset by peer]
ei-slackbot-ipfs has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix>
whyrusleeping: can you link me to a failed travis log? maybe i get a sudden inspiration
<okket>
maybe jbenet is thinking along the lines that everything another nodes send should be treated as potential malicious/harmful and everything should be checked/sanitized before it gets processed?
<cryptix>
whyrusleeping: not sure what he meant either..
<cryptix>
okket: yea could be
* cryptix
takes a note to fuzz all the protocoll layers
<whyrusleeping>
cryptix: that would be pretty handy
<whyrusleeping>
fuzz all the things
<whyrusleeping>
might be good to fuzz yamux and muxado as well
<cryptix>
yup :) i could need some help figuring out connection setup and stuff
<cryptix>
and its good to get knowledge about how that code actually works
<cryptix>
i think i never looked at the swarm stuff at all
<whyrusleeping>
yeah...
<whyrusleeping>
thats not a bad thing to admit
notduncansmith has joined #ipfs
ei-slackbot-ipfs has quit [Remote host closed the connection]
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
i wonder if jbenets flight has landed yet....
ei-slackbot-ipfs has joined #ipfs
<cryptix>
flightracker? :)
<whyrusleeping>
haha, if only i knew what flight it was
<whyrusleeping>
although, i have a good idea
<whyrusleeping>
huh, looks like he wont arrive until around 8am?
<whyrusleeping>
i didnt think america was that wide
<cryptix>
lol
<whyrusleeping>
oh wait
<whyrusleeping>
time zones
<whyrusleeping>
i need to stop staying up so late
<whyrusleeping>
i miss mornings...
<cryptix>
lol i dont
<cryptix>
nearly got hit by a car this morning when i fetched bread
<whyrusleeping>
o.o
<cryptix>
and i just need to cross two streets to get to the backery
<whyrusleeping>
wow, are drivers crazy over there?
<whyrusleeping>
or just hungover?
<cryptix>
mostlikly my fault - traffic accidents are pretty rare in my perception
<cryptix>
i just shouldnt leave the house until 3pm and/or before the third coffee sank in
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
lol
<whyrusleeping>
thats fair
<ipfsbot>
[go-ipfs] whyrusleeping pushed 1 new commit to fix/bitswap-multisend: http://git.io/vJwke
<ipfsbot>
go-ipfs/fix/bitswap-multisend 6f04302 Jeromy: remove logging of dup blocks, move to counters for bitswap stat
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix>
hmpf.. i broke the putHandler...
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<mafintosh>
jbenet, whyrusleeping: https://twitter.com/mafintosh/status/595208972447195136 - we need this to make webrtc a truly useable transport protocol. if you could get involved in the issues it would be appreciated
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<clever>
hmmm, how do you delete something you added with 'ipfs add' ?
<okket>
i guess the current way is unpin it and then garbage collect
<clever>
weird, since its not in pin ls
<clever>
ah, its under recursive
<clever>
and it cant unpin because of that
<clever>
Error: QmauRta7LrHeVZf23E45wj1d3Kopx2BnQkMp83gehAsGZY is pinned recursively
<okket>
ipfs pin rm -r
<okket>
and then the root hash
<clever>
root hash?
<crest>
it would be nice of the command to prevent cycles of length 1
<okket>
the directory hash
<clever>
i didnt add a directory, just a single file
notduncansmith has quit [Read error: Connection reset by peer]
<clever>
what kind of client would actualy look for such TXT records on a domain and then be able to switch to ipfs?
<crest>
clever: a browser with the right plugin?
<clever>
ah
<clever>
but what benefit would it have over just using http?
<crest>
the load is redirected from a http gateways to your local ipfs instance
pfraze has quit [Remote host closed the connection]
<crest>
*a set of
<okket>
you can access the content of the site even when it the main server is not available (given that other ipfs nodes have the content, either by previously requesting it or pinning)
<okket>
or if the content is removed from the main server
<clever>
ah, so ipfs is just acting as a mirror, with potentialy many backups
<clever>
though i cant see it working for most sites ive managed, they have a lot of server side code and databases
<okket>
switch to single page application instead of server rendered and use the DAG as a database ;)
<clever>
ok, where shal i store the users passwords? lol
hellertime has quit [Read error: Connection reset by peer]
hellertime has joined #ipfs
<okket>
also searching / indexing would be interesting, maybe as a value added external service?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<clever>
there is also server logic, where you dont really want to leave the users in control, for cheating reasons
vijayee has joined #ipfs
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
torpor has joined #ipfs
chrisr_ has left #ipfs [#ipfs]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has joined #ipfs
mildred has quit [Quit: Leaving.]
lgierth has quit [Quit: Ex-Chat]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Daboloskov has joined #ipfs
<wking>
cryptix: rebasing #1136 doesn't seem to hit any conflicts. Was there something in particular I should be adjusting to?
Daboloskov has quit [Client Quit]
<cryptix>
wking: try running the tests - i think you used debugerr in some testutil
<cryptix>
otherwse lgtm :) and i wanted to include the core/unixfs AddFromReader in my http gw work
Daboloskov has joined #ipfs
<wking>
cryptix: ok, I'll run through the tests and see. I guess I should add a Context argument to AddFromReader? Or should that wait on #1189?
pfraze has quit [Remote host closed the connection]
<cryptix>
no, leave it as is for now :) 1136 is large enough already imho
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
torpor has left #ipfs [#ipfs]
<wking>
cryptix: I just don't want to have to document another public-API migration once #1189 lands ;)
<wking>
I'd rather not land the AddFromReader command until we feel like it's got a fair shot at being our forever-API :p. I'm happy with the earlier parts of #1189 landing while we think about that though
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has joined #ipfs
pfraze has quit [Remote host closed the connection]
Tv` has joined #ipfs
pfraze has joined #ipfs
<cryptix>
wking: i think, unixfs.AddFromReader() is fine without a context - there is nothing down from that call we need to pass a context to be able cancel processing properly. if you want to stop adding from a network connection, you can use a pipe and close it if you need to (cc jbenet)
notduncansmith has joined #ipfs
<jbenet>
cancellation probably needs to make sure it closes with an error. EOF likely won't cut it.
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix>
sure
<cryptix>
pipe.CloseWithError ftw
<cryptix>
jbenet: re1189: i dont get how osx is failing and linux isnt
<cryptix>
i wonder if there is something like a post-hook in shareness that could be used to upload the trash directories...
<techgrin>
vitzli, whyrusleeping: I like Vultr VPSs, they have a storage series, accept btc. Not as cheap as ramnode though.
G-Ray has joined #ipfs
G-Ray has quit [Remote host closed the connection]
<cryptix>
yea.. i was starring at that in disbelive as well
<cryptix>
jbenet: how could it be intact on linux though? i dont see any platform specifics there
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<cryptix>
lets see what my macbook has to say about that
<wking>
cryptix: rebased #1136 to respond to the removal of debugerr in 7a6febbe (godeps: following up on PR #1098 to drop the facebookgo code, 2015-04-28, #1156). Running through the tests now...
<wking>
cryptix: cancelling by closing the pipe wouldn't let you cancel the pinning. I think that's the only thing that happens after we finish reading...
besenwesen has joined #ipfs
besenwesen has joined #ipfs
<wking>
pinning is unlikely to hang or to take very long, but covering all the activity with a Context seems more consistent
<wking>
You also don't have to remember which functions you cancel with Contexts and which you cancel by erroring readers if everything takes a Context
nham has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
cjb has quit [Remote host closed the connection]
cjb has joined #ipfs
RzR has quit [Excess Flood]
RzR has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has quit [Remote host closed the connection]
atomotic has joined #ipfs
gordonb has joined #ipfs
<gordonb>
has anyone played around with ipfs on mobile devices?
cjb has quit [Remote host closed the connection]
cjb has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Daboloskov has joined #ipfs
Daboloskov has quit [Client Quit]
<krl>
wking: pinning recursively takes ages when the tree is large, and times out
<wking>
krl: even more reason for a cance-lable Context then. Do we know why there's the delay? Are we queuing up all the pins and then flushing them at the end?
<wking>
*cancel-able
<krl>
i'
<krl>
i'd like to have an async pinning feature
<krl>
preferably with progress reporting
<wking>
krl: async pinning sounds good to me, except when you want synchronous pinning ;). I think there's an issue talking about that...
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has quit [Remote host closed the connection]
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
<krl>
wking: yes, it should be an option
<krl>
at the moment i'm having to resort to r-pinning small pieces at a time
<wking>
Are your slow recursive pins from a local addition, or when you pin objects that you don't currently have locally? I'd expect recursively pinning remote objects to be slow, because you have to fetch all the pinned objects.
<krl>
this is remote, i'm trying to pull the geoip tree to my vhost
<krl>
from my laptop, where it currently lives
<krl>
i think reorganizing pinning into an ipfs object is the right thing
<krl>
which is being worked on
<wking>
ah, maybe no way around that being slow then ;). But there should be an option that says "I don't really care if/when this completes. Just work on this in the background"
<wking>
yeah
<krl>
exactly
<krl>
and provide pretty progress bars :)
<krl>
%, or bittorent style chunk visualization
<wking>
where would you display the progress bars if you weren't blocking the pin request?
<krl>
make it an api function
<wking>
Or are you suggesting a follow up 'ipfs pin status ...' command?
<krl>
you could show it in the webui for example
<krl>
"pins in progress"-tab
<wking>
yeah
<krl>
or, call it something else, since pinning is instant
<newipfsusert>
quick question, i installed ipfs and it was working fine on friday, now when i do "ipfs init" and it doeswnt generate a key pair. Is that normal? How do i find the hash to get started?
<grawity>
reuse the old one, from `ipfs id`?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mildred has quit [Ping timeout: 240 seconds]
mildred has joined #ipfs
<newipfsusert>
hi thanks very much for the tip, ipfs id gives me the ID & Pub Key. Is there a way to get a fresh instance instead of reusing the old one? I installed on a VM, then copied the VM as I am trying to get the 2 talking to each other. But they are both using the same public key
<newipfsusert>
A: remove the ipfs data directory, ~/.ipfs
<newipfsusert>
thanks grawity
mildred has quit [Ping timeout: 250 seconds]
mildred has joined #ipfs
inconshreveable has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Daboloskov has joined #ipfs
<dPow>
I'm ready for the meeting whenever. Will we be starting on time today?
m0ns00n has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Daboloskov has joined #ipfs
hellertime has quit [Ping timeout: 256 seconds]
Daboloskov has quit [Client Quit]
patcon has quit [Ping timeout: 265 seconds]
<whyrusleeping>
would be cool to have a community webpage in ipfs linking to the pages of other people
<whyrusleeping>
so a good starting point for browsing the distributed web
<cryptix>
the wordpress idea is really nice but a lot of ui too - i wonder if there is something like an open source tumblr clone or similiar we could fork for that
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has joined #ipfs
sindresorhus has quit [Read error: Connection reset by peer]
crossdiver has quit [Ping timeout: 265 seconds]
vonzipper has quit [Read error: Connection reset by peer]
mafintosh has quit [Read error: Connection reset by peer]
warner has quit [Quit: ERC (IRC client for Emacs 24.5.1)]
mquandalle has quit [Read error: Connection reset by peer]
otherchas_ has quit [Read error: Connection reset by peer]
mitchty has quit [Quit: No Ping reply in 180 seconds.]
mitchty has joined #ipfs
<cryptix>
or.. not :)
vonzipper has joined #ipfs
crossdiver has joined #ipfs
zootella has joined #ipfs
patcon has joined #ipfs
jbenet has quit [Ping timeout: 265 seconds]
sindresorhus has joined #ipfs
mquandalle has joined #ipfs
pfraze has quit [Remote host closed the connection]
otherchas_ has joined #ipfs
<cryptix>
wking: i think you are right. having a context as first param is more consistent
jbenet has joined #ipfs
mafintosh has joined #ipfs
saebekassebil has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<cryptix>
btw not sure if everybody saw it: since Tv`s /debug/vars pr landed we can use tools like this to monitor ipfs internals https://github.com/divan/expvarmon
<cryptix>
whyrusleeping: did you hack up the -vars= flag for the datastore metrics maybe?
nham has quit [Ping timeout: 244 seconds]
saebekassebil has joined #ipfs
inconshreveable has joined #ipfs
<whyrusleeping>
TV was working on that I thought
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ipfs-newbie has joined #ipfs
<cryptix>
whyrusleeping: no, they are exposed - i was just to lazy to type the flag for expvarmon to show them
<ipfs-newbie>
I have installed ipfs on 2 machines, on 2 netorks and am trying to get them to connect. Both can see swarm peers and plenty of them, but in the peers list of one i cant see the other one. I do an ipfs id on both to get the addresses and try to connect manually with ipfs swarm connect <addr>.
<ipfs-newbie>
my addresses are: /ip4/192.168.101.135/tcp/4001/ipfs/QmajGVJzQ2ZTEsYzEZDBRoF7aWCx51JJCBYSanw5mNgYWC and /ip4/192.168.101.135/tcp/4001/ipfs/QmajGVJzQ2ZTEsYzEZDBRoF7aWCx51JJCBYSanw5mNgYWC
<ipfs-newbie>
second one should be /ip4/192.168.70.128/tcp/4001/ipfs/QmXr3ZAhDqb86y5AD78dAWNQSgpwbLPq1awM5ExkA3ZnMc
<cryptix>
ipfs-newbie: currenyly, you need to open a port on your router and use their external IP to connect them to each other
<cryptix>
there are plans to add relay routing (using a third node of the network) so you dont have to do that, but its not implemented yet
<ipfs-newbie>
i see, ummm that might be tricky where i am. i assume if i put them on the same network it will be ok ?
* zorun
points at ipv6
<cryptix>
zorun: :) yea
<cryptix>
ipfs-newbie: same network works. you just have to expose one nodes swarm port to get them connected
<cryptix>
hrm... i guess, the save choice is do open individual ip4 and ip6 ports and not rely on dualstack beeing present?
<cryptix>
or you open the ipv6 port first, connect yourself with ipv4 to see if it is dualstack and open ipv4 too if it is not?
<cryptix>
the latter is messy but the situation already is if we want to cover all compat basis
<ipfs-newbie>
cryptix: sorry for n00b question, but what do u mean expose one nodes swarm port?
<cryptix>
ipfs-newbie: ipfs uses the 'swam port' for its p2p communication. its 4001 by default
<cryptix>
and you just need one node with that port open from the outside to build a connection so that they can talk
<zorun>
cryptix: yes, what we don't want is the second bind to fail (because the IPv6 bind is actually dual-stack)
<wking>
\me thinks "swam port" sounds like a good pirate activity ;)
* wking
forgets how to talk in the third person, too
<cryptix>
zorun: yea - if it is dualstack the self connect works and you dont have to open another - if it does, the 2nd bind should work - if it doesnt.. well.. somebody else is on it..? not sure if we want to exit at that point
<zorun>
what puzzles me, on my Debian node, is this:
<zorun>
- telling go-ipfs to bind on tcp6/4001 creates a dual-stack listening port
<zorun>
yeah, I totally understand you're not confortable with it
<zorun>
cryptix: self-connect sounds a bit ugly to me
Daboloskov has joined #ipfs
Daboloskov has quit [Client Quit]
patcon has quit [Read error: Connection reset by peer]
warner has joined #ipfs
<jbenet>
hey guys-- if someone wants to work with me on making the ipfs blog (and coming up with good guidelines for static blogs, like hugo, jekyll, etc on ipfs, lmk
<wking>
jbenet: is "static blogs on IPFS" just build into an IPNS namespace? And do you want me to open an issue (and where?) on IPNS / local object with browsable names? Or were you going to open that issue?
<whyrusleeping>
on that same note, if anyone has a website or page in ipfs, let me know
<whyrusleeping>
im going to start building a directory
<jbenet>
wking: not quite-- this is just using `ipfs add` to publish. what i mean is guidelines for making sure the blog has all relative links, etc.
<cryptix>
zorun: i know its messy but def better than parsing netstat :) if you have a better idea for testing dualstack, lmk :)
<jbenet>
wking: on the ipns / local objects with browsable names-- maybe worth discussing somewhere. i think there are multiple use cases here to disambiguate.
<luzifer>
jbenet: on my tasklist there is a go-build for generating static blog from markdown files to be published into S3... Maybe one could use that attempt to publish into IPFS. It's not that hard (I've done the generator stuff without markdown in python for google appengine) but it has to be done...
<jbenet>
krl cryptix tperson -- i'll look at webui today/tonight. post here any issues/PRs you want me to prioritize
<jbenet>
would be nice to publish a new version in next few days
<cryptix>
jbenet: my vars plotting is far from done sadly
<Tv`>
cryptix: fwiw as far as linux is concerned, netstat is as dead as ifconfig and route; the iproute tools are "ip" and for seeing socket states, "ss"
<jbenet>
wking: yeah at least for the webui use case. may also be worth doing it in ipfs/notes since it may overlap
<Tv`>
ss has built-in filtering support
<whyrusleeping>
Tv`: i knew about ip, but didnt know about ss. cool stuff
<jbenet>
Tv` btw lmk if you need something from me? (ctx switching a lot right now so reminders are good interrupts)
<Tv`>
jbenet: nothing currently
<jbenet>
<cryptix>
yea thanks Tv` :) new tools every day
<whyrusleeping>
jbenet: can you elaborate on 1181?
<whyrusleeping>
you mentioned wanting a larger fix, but i still am unsure what you mean
<luzifer>
(sadly my task list is just too long to get finished within years... :'()
<Tv`>
for values of "new" about 10 years old ;)
<whyrusleeping>
also, 1164 is RFM
<jbenet>
luzifer: i've a makefile that does it :) <3
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<luzifer>
jbenet: nice too :) I wanted to have an online editor which pubishes to S3 for infinite scale... But yeah. In the meantime I don't see the requirement anymore. It could be a repo with .md files and a go binary to generate it all...
<cryptix>
jbenet: publishing to ipns with hugo is also dead simple - just need to put in the prefix as the baseUrl
frumpyEwok has left #ipfs [#ipfs]
<whyrusleeping>
anyone know of a good vps that has really low specs except for large disk?
<wking>
^ I'd expect that holds for most static generators. Or to your native filesystem and then 'ipfs add -r ...'
<jbenet>
whyrusleeping: the bitswap bug is killing me-- the "apparent stalls" are super disconcerting-- what is the fix?
<Tv`>
whyrusleeping: afaik there's no such thing as large cheap directly-attached disk in the cloud, they all want you to go through object stores because that's easier to manage
<whyrusleeping>
jbenet: i cant repro this issue, if i can get a solid repro, i can fix it
<whyrusleeping>
but it doesnt stall for me, even between peers with > 120ms latency
<jbenet>
using metalsmith but that's its own problem orthogonal to ipfs
<whyrusleeping>
jbenet: can you please buy the domain ipfs.cat ?
<Tv`>
whyrusleeping: even http://www.dreamhost.com/cloud/computing/ which is using Ceph uses fixed block sizes because that lets them fix the ratio of compute to storage per row; the object store is a separate cluster
<jbenet>
whyrusleeping :( try adding really large trees? like 200MB+ of files ~ N(0.5MB, 2MB)
<whyrusleeping>
jbenet: okay, ill try that
<jbenet>
whyrusleeping: yeah actually, i can. cause i'm catalan
<jbenet>
(or my family is)
<whyrusleeping>
wooo!
<whyrusleeping>
buy it naow!
<Tv`>
right meow
<whyrusleeping>
i want to change gateway.ipfs.io -> ipfs.cat
<whyrusleeping>
easier to type
<jbenet>
anyone know a goo provider?
<whyrusleeping>
(at least an alias)
<whyrusleeping>
i like gandi
<jbenet>
it will be `ipfs.io` -- we should probably switch to thayt soon
<zorun>
cryptix: cool, thanks for the finding
<whyrusleeping>
oh, yeah, ipfs.io would be good too
inconshreveable has quit [Ping timeout: 250 seconds]
<jbenet>
whyrusleeping: find me the best domain registrar for .cat and i will get it.
EricJ2190 has joined #ipfs
<jbenet>
i just... dislike them.
<whyrusleeping>
you dont like gandi?
<whyrusleeping>
i personally use marcaria for everything since they are pretty good about not disclosing your info to anyone
<jbenet>
i like gandi's tagline.
<jbenet>
we'll see though...
<whyrusleeping>
lol
inconshreveable has joined #ipfs
<jbenet>
websites should stop making me register
<whyrusleeping>
i love 'sign in with google'
<whyrusleeping>
its the best thing ever
<whyrusleeping>
could also get ipfs.sexy
<whyrusleeping>
ipfs.works
<whyrusleeping>
ipfs.rocks
<whyrusleeping>
ipfs.technology
<whyrusleeping>
man, these are great
pfraze_ has joined #ipfs
<jbenet>
"Our contract with ICANN requires that we sell our WHOIS database to any party that requests it. However, you have the right to refuse the sending of your personal data in such an event. Only click on YES and proceed to the next form if you accept that your personal data may be sold." what the fuck
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
lol
<cryptix>
ipfs.horse
<zorun>
AFAIK, all registrars I've used seems to be forced to do that
<Tv`>
jbenet: sure but then you put a proxy entity in WHOIS..
<Tv`>
google domains has that as standard, name.com charges $3.99/year
<Tv`>
.io rules say you can't do that, but just about everything else allows it
<whyrusleeping>
lol, just try doing a whois on jero.my
<whyrusleeping>
tried*
<jbenet>
yeah am familiar with guards-- just didnt know it was that stark ("forced to sell" vs, "this info is likely to propagate to icann who will make it freely available to all")
<whyrusleeping>
half of the whois lookups fail for some reason
<jbenet>
also, lols: "You will publish a website on the internet, within 6 months following Your .CAT domain name registration, which must be fully or partly in Catalan language."
ei-slackbot-ipfs has quit [Remote host closed the connection]
<Tv`>
jbenet: WHOIS info is considered public
ei-slackbot-ipfs has joined #ipfs
<Tv`>
for good rasons
<Tv`>
*reasons
<jbenet>
Tv` am down with public-- just, in that case, give it away, not sell it :]
<cryptix>
jbenet: i wanted.. now i want to get a bigger picture first. how and where are they created and waited on. how can we better do that. i think the work on the exit codes and the CloseNotifier should help with that
<jbenet>
whyrusleeping: what's weird about it? (afaict, it's bootstrapping, then dialing, then resolving best local addr to dial someone)
<whyrusleeping>
yeah, but look closely
<jbenet>
(kinda unideal it ctx switches)
patcon has quit [Ping timeout: 264 seconds]
<whyrusleeping>
syscall, 38 minutes
<cryptix>
jbenet: re 1189: yea.. me too. havnt investigated deeper yet- i think there is a subtle difference between how we resolved before
<jbenet>
oh 0x0's everywhere?
<jbenet>
ohhhhh wow
<jbenet>
cryptix: yeah-- didn't spot it on the PR. maybe should look closely again.
mildred has quit [Quit: Leaving.]
<ipfsbot>
[go-ipfs] jbenet pushed 1 new commit to master: http://git.io/vJ6pg
<ipfsbot>
go-ipfs/master 9b12e02 Juan Batiz-Benet: Merge pull request #1193 from ipfs/improve_parse_test...
<whyrusleeping>
cryptix: with your PR do gateway ipns links work?
<cryptix>
whyrusleeping: yup - core.Resolve does it
<jbenet>
@whyrusleeping: you did " save memory profile + binary to ipfs" right?
<cryptix>
jbenet: iirc the test failures where only in the writable :8080 gateway
<pinbot>
now pinning QmavE42xtK1VovJFVTVkCR5Jdf761QWtxmvak9Zx718TVr
<jbenet>
wking: an example of when to "try forver" is how torrents actually work for everyone.
<whyrusleeping>
oh, it appears to have crashed?
<jbenet>
wking: a torrent is downloaded and run, even if there's no seeds the client will retry periodically until it works
<pinbot>
failed to pin QmavE42xtK1VovJFVTVkCR5Jdf761QWtxmvak9Zx718TVr: pin: context deadline exceeded
<whyrusleeping>
the ipfs node on that machine is acting strangely
<jbenet>
wking: it's very much the desired UX for torrent users -- otherwise not all torrent clients would have continued to do this for over a decade.
pinbot has quit [Remote host closed the connection]
pinbot has joined #ipfs
xaiki has joined #ipfs
<whyrusleeping>
jbenet: its having trouble pulling the key QmSNZeGrW1YzNdPpGUzsuxW7XmPDyQCFN16MSPPJW5hrNq
<jbenet>
stalled?
<jbenet>
(maybe there's the repro.)
<jbenet>
or the dht records may be inaccessible.
notduncansmith has joined #ipfs
<whyrusleeping>
yeah, it appears stalled
<whyrusleeping>
your machine has all of it, right?
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
like-- there's a lot going on, and none of it's 100% production quality, so a lot of little edge cases can be compounding to surface as these sort of bugs
<jbenet>
whyrusleeping: have 3 machines with all of it now
<whyrusleeping>
can you check the wantlist of QmcafGHtJjpkG7XraU1SKeDFrNJ4RkZR3XruiKFeuCGPAH on any of them?
<www1>
quick question: how is going with the JS implementation?
<jbenet>
www1: need to make an interoperable wire format-- it's almost there-- but i keep getting distracted. want to help?
<wking>
there are multiple values for you key in the DHT. How long should a "block as long as you need" pinner wait to declare each lookup resolved?
<wking>
it sounds like torrents are using "block until you get any valid response for that key". That that seems overly optimistic to me, but it will work (it just might return stale results)
<jbenet>
wking: ahh yeah if a lookup yields results, use them. but if the lookup yields no results-- i want there to be a mode where it waits a while and tries again later.
<jbenet>
wking: dht results for content addressed objects can be used right away-- it's non-content-addressed things -- like ipns names -- that fall into the consistency question.
<jbenet>
(and yeah agreed on this being important to get right)
<wking>
I'm fine with that^, but I just wanted to make sure "take as long as you need" didn't accidentally pull in the DHT-result-aggregation step
<jbenet>
agreed!
<whyrusleeping>
can we please stop using manners
<whyrusleeping>
when i ctrl+c my daemon i really dont care about it waiting for the http server to shut down
<cryptix>
+1
<www1>
jbenet: thanks. Will think about it!
<jbenet>
wking: one thing on this too would be to look at what other systems do on cases like this-- systems like dns, ospf, coral, etc. on the bittorrent dht -- one signs onesself up for the key, so you will be pinged by newcommers who have the torrent. some ipfs routing systems could potentially have a similar mechanism -- "tell me if you have X"
<jbenet>
(instead of only "who has X?")
<jbenet>
whyrusleeping: sold.
<wking>
jbenet: lists of nodes wanting an object sound like part of a filecoin-broker, although it's been a while since I've looked at any filecoin stuff...
<jbenet>
wking: also bitswap.
<jbenet>
wking: (a more advanced strategy)
<whyrusleeping>
i'm starting to think my computer is broken
<whyrusleeping>
i keep getting that weird goroutine being stuck in a syscall for multiple minutes
<cryptix>
whyrusleeping: platform?
<whyrusleeping>
arch linux
<whyrusleeping>
its stuck querying for interface addresses
<jbenet>
cryptix: wat do about that osx gateway thing? did you check it on osx? or need me to try something?
<cryptix>
ok - was about to guess full zvol but thats more weird
<whyrusleeping>
if osx had a nice package manager (which brew is not) and a nice tiling WM, i would probably use it
<cryptix>
jbenet: would be nice to get a 2nd confirmation - i'm still not sure whats up
<cryptix>
i'm currently digging through the error code stuff
<whyrusleeping>
sorry, i guess my definition of a nice tiling WM is "pretty much just i3"
<cryptix>
it's more of a fact finding mission - i'm to tired for disecting the resolve stuff right now
<whyrusleeping>
dPow: i think that tag is no longer used
<okket>
brew is ok, actually more complete and up to date than most linux distributions / package managers, for tiling WM: there are addons like BetterSnapTool and Cinch
<jbenet>
also, brew is not so bad. imo better than rpms, yum, pip, and ... go get *zing*
<whyrusleeping>
its failed atrociously on me every time ive attempted to use it
<jbenet>
dPow yeah i think that tag is no longer used
<whyrusleeping>
and at one point, it screwed up so bad i had to uninstall it completely
<jbenet>
whyrusleeping: what did you do to the poor thing?
<whyrusleeping>
tried to install mercurial
<cryptix>
whyrusleeping: i've been there too
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<dPow>
whyrusleeping, jbenet: Was it being used like we use backlog now?
<jbenet>
hmmm i've never had _problems_ with it. i've had "not founds", but nothing like that.
<jbenet>
dPow yeah i think so
<whyrusleeping>
i guess i just expect things to work as nicely as pacman
<okket>
if you accept the rules on the mac, brew is at least as good as rpm/dpkg, no experience with arch/pacman though
<Xe>
no
<Xe>
Brew is a user level package manager
<Xe>
dpkg/rpm is a system level one
<Xe>
please don't confuse the two
<okket>
that are the rules, you can't mess with the system on the mac
<substack>
blocktechceo: that patch gives you some of what ipfs can do (ipfs can do a lot of things)
<substack>
but specifically creating per-user feeds addressed at the hash of a public key
<substack>
it was just easier right now to hack that patch into bittorrent-dht than to build an entire ipfs stack in pure js
<blocktechceo>
aha, that makes sense
<substack>
because I'm making a bunch of p2p serverless browser apps
<www1>
per-user feeds.. do you have some expamples on how-to-use?
<www1>
substack:
copernicus has quit [Read error: Connection reset by peer]
<substack>
you can just create a content-addressed merkle dag to make a feed
<jbenet>
btw substack: i've the wire format finally figured out. i'll be making a build for you this week
<substack>
and the mutable content only needs to point at the hash of the most recent document
<blocktechceo>
IPNS addressed?
<substack>
jbenet: fantastic!
patcon has joined #ipfs
copernicus has joined #ipfs
<jbenet>
no it isn't ipns addressed -- bittorrent is a separate world entirely. we could actually write an ipfs-routing-system module on top of bittorrent dht, but it's actually really mean to do to torrent clients, cause we'll be dumping tons of stuff onto the dht.
<jbenet>
substack: are you back in the bay or traveling?
copernicus has quit [Read error: Connection reset by peer]
<blocktechceo>
@jbenet - that makes sense, gracias
copernicus has left #ipfs [#ipfs]
<wking>
dPow: No opinion ;). I've removed the 'ready' labels from all my stuff, so as far as I'm concerned you can remove that label
copernicus has joined #ipfs
<wking>
dPow, jbenet: So I could move stuff on waffle.io/ipfs/go-ipfs, but waffle.io/ipfs/ipfs is giving me "Only collaborators can update cards", even for go-ipfs issues/PRs.
<wking>
Any idea why that would be?
notduncansmith has joined #ipfs
copernicus has quit [Read error: Connection reset by peer]
notduncansmith has quit [Read error: Connection reset by peer]
copernicus has joined #ipfs
<jbenet>
wking: hm not sure. maybe people all need to be part of ipfs/ipfs ? sigh acls.
<jbenet>
wking: can you assign the label in go-ipfs?
<dPow>
wking: weird, usually it only denies on a per-repo basis of whether you have write permissions. It all comes down to whether or not you can assign labels to issues.....
<dPow>
That would suck if everyone needed access to ipfs/ipfs
<dPow>
wking: Ah, so it doesn't allow you to touch a single thing then. hmm. I've contacted their support before and they're pretty friendly. I also didn't see anything about this mentioned in https://github.com/waffleio/waffle.io/wiki/FAQs#multirepo