<martinklepsch>
jbenet: maybe not just IPFS but also other efforts towards a more resilient distributed architecture for global information sharing/communication (and that kind of stuff if you know what I mean
<jbenet>
martinklepsch oh there are many out there. take a look at Tahoe LAFS, Ethereum, all the bitcoin stuff going on. various end-to-end messaging services, etc.
<martinklepsch>
jbenet: can you give some names for the end-to-end messaging services?
<spikebike>
tox and bitmessage spring to mind
<martinklepsch>
I was also thinking about Ethereum when reading the paper, is there anything written relating those two?
<spikebike>
If writing an app on top of IPFS I'm still pondering how to manage the need for messaging.
<martinklepsch>
the demo video for IPFS is really nice
<martinklepsch>
spikebike: actually just thought about this on the way to work today, what problems do you see?
<spikebike>
well IPFS is a nice distributed filesystem, but as soon as you write an app for it you want to talk to other peers
<spikebike>
communicating through a filesystem is painful.
<spikebike>
Ideally you could use the same publci key so you can identify your peers
<spikebike>
ideally you could fine them in the DHT
<spikebike>
ideally you could use some network layer to send them messages
<spikebike>
er find not fine ;-)
<spikebike>
so even my simple use case of trading storage to allow p2p backups, you need some kind of message layer
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<martinklepsch>
spikebike: my naive thinking was just that you'd have directories (threads) and files (messages) and would somehow watch those for changes, using that as a messaging layer, but I see that this might get cumbersome pretty fast
notduncansmith has quit [Read error: Connection reset by peer]
<reit>
i'm thinking perhaps ideally the solution might be an ipfs service that transparently delivered something like standard POST messages from clients to servers using peer IDs
<reit>
but i can't for the life of me think how one might implement that in the daemon without requiring some sort of alteration to the browser as well
<reit>
to address to an ipfs peer id instead of the standard approach
<jbenet>
reit: could be done like: $ipfshostname:5001/api/v0/rest/$remote-peerid
<jbenet>
just need to mount a route on the api
<jbenet>
it's a very interesting idea
<reit>
oh right i see, you could use the api, so perhaps something like <form method="POST" action="127.0.0.1:5001/api/v0/rest/<remote-peerid>/script.php">
kallaballa has joined #ipfs
<reit>
and then the php server on the other end gets a message from its own local ipfs api through the matching interface
<reit>
not that i wish to say you could only use php there
<jbenet>
:p
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
compleatang has quit [Ping timeout: 264 seconds]
hellertime has joined #ipfs
wedowmaker has quit [Ping timeout: 240 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<spikebike>
jbenet: very interesting, thanks.
williamcotton has quit [Ping timeout: 255 seconds]
therealplato1 has joined #ipfs
therealplato has quit [Ping timeout: 240 seconds]
compleatang has joined #ipfs
domanic has quit [Ping timeout: 264 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<spikebike>
jbenet: where would I file a bug against that URL?
<spikebike>
at least it's not specific to my go environment
<jbenet>
huh interesting
<jbenet>
spikebike: it works for me in a new env
<jbenet>
mkdir /tmp/foo && GOPATH=/tmp/foo go get golang.org/x/net/context && ls /tmp/foo/src/golang.org/x/net/context
<cryptix>
hello guys
<jbenet>
krl: let's try meeting up today at some point. i should go to sleep now, but maybe in a few hours.
<jbenet>
cryptix: heyo o/
<cryptix>
hey jbenet :) i hope i have more time for you guys in the following week. there is a festival next week and i have a metric f**ton of stuff to solve before
<spikebike>
jbenet: ah, heh, ya that works, I didn't realize that context wouldn't be downloaded as needed
<rht__>
(or currently just `proc.(*process).teardown = tf` works too)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<rht__>
(oh doesn't work because process is not exported)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
hellertime has quit [Quit: Leaving.]
williamcotton has quit [Ping timeout: 276 seconds]
PayasR has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<lgierth>
jbenet: regarding monitoring, prometheus uses a pull model, where instances expose metrics at an http endpoint, and one or more prometheus instances collect them. for golang, that'd look like this: https://godoc.org/github.com/prometheus/client_golang/prometheus
<lgierth>
i haven't used it before, but it looks a lot more lightweight than the usual statsd+ganglia+graphite dance
domanic has quit [Ping timeout: 245 seconds]
<lgierth>
also more composable. a prometheus instance can pull from other prometheus instances
williamcotton has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Blame has joined #ipfs
<jbenet>
lgierth: that looks good to me. maybe we can try an example?
williamcotton has quit [Ping timeout: 256 seconds]
tilgovi has joined #ipfs
<jbenet>
ok bbiab. ZzZz
williamcotton has joined #ipfs
<Luzifer>
gn8 jbenet
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<reit>
if i ipfs add something, does that also permanently pin it?
therealplato1 has quit [Ping timeout: 264 seconds]
pfraze has joined #ipfs
therealplato has joined #ipfs
jibber11 has joined #ipfs
williamcotton has quit [Ping timeout: 265 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
reit: it pins it, yes
<whyrusleeping>
you can unpin it with 'ipfs pin rm -r <hash>'
<reit>
it's weird, when i go ipfs pin ls, nothing shows up
kallaballa has quit [Ping timeout: 256 seconds]
<whyrusleeping>
reit: 'ipfs pin ls --type recursive'
<whyrusleeping>
i agree its weird
jibber11 has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<whyrusleeping>
but jbenet wanted 'direct' pins to show by default
<whyrusleeping>
and recursive pins to be specified
* whyrusleeping
shrugs
<reit>
uh wait, if i'm adding a file directly why would it be recursive?
jibber11 has joined #ipfs
<Luzifer>
60 keys with type recursive… O_o I'm storing a lot of stuff…
<reit>
i mean, what's it recursing *from*?
<whyrusleeping>
Luzifer: those are just indirect blocks
<whyrusleeping>
reit: files are broken up into chunks
<whyrusleeping>
and made into a DAG
<whyrusleeping>
you pin the root of the dag recursively
<whyrusleeping>
and all the other child blocks are pinned indirectly as a result
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
vijayee_ has joined #ipfs
<reit>
so wait, each ipfs set up has a hidden root node, and every time you add something the hidden node is updated based on all other things you've added? does that sound right?
<whyrusleeping>
reit: ehm... kinda
jibber11 has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
jibber11 has joined #ipfs
jibber11 has quit [Client Quit]
<reit>
uhh, is there by any chance a way to grab that node and publish it? it sounds like a really neat way to back up all your stuff, to be able to go to your other server and ipfs pin <root-hash>
jibber11 has joined #ipfs
<whyrusleeping>
yeah.. thats why i said kinda
<whyrusleeping>
lol
<nessence>
rand q. Are ya'll using priorities w/the spdy framing? And is there a wiki/spec/code for ipfs wireline protocol? :)
<whyrusleeping>
after 0.4.0, youll be able to grab all the pins easily
<whyrusleeping>
nessence: not sure, we're just using dockers spdystream lib
<whyrusleeping>
reit: but the pins arent all stored as dag links, so you cant just 'pin' those blocks..
<whyrusleeping>
we will probably have a mechanism to do so though
<nessence>
I saw a chat the other night about duplexing. Does IPFS need to send/receive out of band data while tx/rx other data?
<whyrusleeping>
nessence: all data transfers happens over a two way spdy pipe
<whyrusleeping>
(or currently, a two way yamux pipe)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<reit>
whyrusleeping: i see, well it's nice you can go ipfs pin ls -q --type=all to grab everything, backups through ipfs are still possible with a simple script
<lgierth>
might check its other dependencies for removal too
<rht__>
and Godeps
<whyrusleeping>
hrm...
<whyrusleeping>
theres an s3 datastore PR in the works
<whyrusleeping>
its something we want
<lgierth>
ok
<lgierth>
rht just shot me that question yesterday
marklock has quit [Ping timeout: 246 seconds]
www has joined #ipfs
<ipfsbot>
[go-ipfs] whyrusleeping pushed 1 new commit to feat/filter: http://git.io/vL8kA
<ipfsbot>
go-ipfs/feat/filter 1c2fb48 Jeromy: broke filters out into a struct...
<cryptix>
lgierth: i personally dont bring any electronics to fusion and dislike seeing cameras there but in general i agree that we need this tech. :)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
jibber11 has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
notduncansmith has quit [Read error: Connection reset by peer]
<lgierth>
cryptix: yeah the cameras on parties and festivals... so annoying
<lgierth>
on the other hand fusion is such a great testbed, when compared to actual actions in the streets
dread-alexandria has joined #ipfs
warner has quit [Quit: ERC (IRC client for Emacs 24.5.1)]
domanic has quit [Ping timeout: 245 seconds]
jibber11 has joined #ipfs
nessence has joined #ipfs
domanic has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Encrypt has quit [Quit: Eating time!]
<ipfsbot>
[go-ipfs] whyrusleeping closed pull request #1379: Wired up the trickle dag flag for the add command (master...use-trickle-dag) http://git.io/vLZNE
dandroid has joined #ipfs
gozala has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot>
[go-ipfs] whyrusleeping pushed 1 new commit to http-eventlog: http://git.io/vL8wF
<ipfsbot>
go-ipfs/http-eventlog 45d45ad Jeromy: clean up unused log options...
Blame has quit [Quit: Connection closed for inactivity]
gozala has joined #ipfs
<ipfsbot>
[go-ipfs] whyrusleeping pushed 1 new commit to http-eventlog: http://git.io/vL8iL
<ipfsbot>
go-ipfs/http-eventlog 92a7a0a Jeromy: skip logs when no writers connected...
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
something is sniping my ssh connections at this coffee shop
<whyrusleeping>
mosh is doing fine
<whyrusleeping>
but any normal ssh connection gets dropped within 20 seconds
<whyrusleeping>
otherwise the internet is great
compleatang has quit [Quit: Leaving.]
Encrypt has joined #ipfs
compleatang has joined #ipfs
compleatang has quit [Client Quit]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
dandroid has quit [Ping timeout: 246 seconds]
<daviddias>
whyrusleeping: you know that day would come
<cryptix>
whyrusleeping: oh wow.. interesting that they aloow the initial handshake than..
* cryptix
would like to have every transport behave like mosh
<krl>
yep.
<cryptix>
hey krl :) how is berlin these days?
<krl>
ah just fine
<krl>
summer has kind of begun
<krl>
and i've been running again!
<krl>
cryptix: i might come one day for fusion as well, not sure yet how it will work out
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<krl>
got ipc working between the menu and the host app, visual indication of node running
<lgierth>
checkin: giving prometheus a try for metrics
<krl>
the menu is a little react app by itself
<krl>
also, working on specs for ipfs 'apps' (or what we want to call them), and the blogging interface
mildred has joined #ipfs
kbala has joined #ipfs
<cryptix>
sprint: only some feedback on gh and a new node in the freifunk hamburg network (long overdue..)
rektide has joined #ipfs
<lgierth>
nice!
<lgierth>
i flashed one of two nanostations for my neighbourhood on the weekend :)
<cryptix>
lgierth: i took a look at prometheus a while back. can you add auth to the http endpoints yet?
sharky has joined #ipfs
<lgierth>
it'd be localhost
<lgierth>
then a prometheus instance on each host that collects them
<lgierth>
i'm very open to other suggestions! :]
<cryptix>
i dont like to have a collector for each node :)
<cryptix>
im still not sure about the pull or push model (influx)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<daviddias>
I worked on my node-spdystream side impl from scratch, while the node-spdy whole refactor isn't done. However, leveraging node-spdy will be the way because there are way too many things going on to be able to ship a solid version in timely
<lgierth>
would rather push to a configurable statsd instance?
<cryptix>
lgierth: i'm not sure but i'd like to collect the info of all of my nodes together.
<daviddias>
I can continue looking at it, but probably would get more stuff done if shifted focus to the other main task for this sprint, that is, the DHT. Any updates on that end jbenet ?
<lgierth>
my initial idea with prometheus is an instance on each host, which collects stats from the local ipfs, and general stats about the local node. then one or two instances which aggregate all hosts' metrics, and serve the graphs and ui
dread-alexandria has quit [Quit: dread-alexandria]
<lgierth>
one single prometheus instance would work too
<lgierth>
we should have some kind of private network between the gateway hosts anyhow
<wking>
checkin: Some work on another 'ipfs file ls ...' reroll (go-ipfs#1348), but I haven't finished or pushed the reroll yet. Hopefully will get through that this afternoon
<lgierth>
right now we expose the gateway directly on :8080
<lgierth>
what i like about the pull model is that the process which produces the metrics, doesn't have to know anything
<cryptix>
lgierth: i think thats soundclouds assumption, too. would be great if you could just enable https client cert auth on their endpoints though - much easier for opt-in metrics collection, too
<lgierth>
or have a $vpn between the hosts and bind the metrics only to that interface
<lgierth>
i'm about to head out to the cinema
<lgierth>
but i'll have a look at what net/http has to offer
<lgierth>
the metrics handlers are simple net/http handlers
<cryptix>
lgierth: what are you watching?
<lgierth>
victoria
<lgierth>
can't wait to see it
<cryptix>
aah - need to watch that one too
<lgierth>
140 min, one shot
<cryptix>
yup yup
<lgierth>
and not fake ux single-shot like birdman
<cryptix>
i've seen a lot about it already but didnt find the time yet.. and now i need to wait until i'm back in hh with my girlfriend after fusion.. gaarrww ^^
<jbenet>
sprintbot: working on Wire diagram and ipfs interfaces spec
domanic_ has joined #ipfs
<lgierth>
cryptix: no spoilers from me :)
<cryptix>
<3
jibber11 has quit [Ping timeout: 265 seconds]
domanic has quit [Ping timeout: 245 seconds]
<jbenet>
daviddias: don't have the dht spec yet, I can try to do that today too.
<lgierth>
cryptix: a simple way of getting the authentication would be putting the gateways on hyperboria (or a private cjdns network)
<lgierth>
then we can have a simple ip address whitelist
alu has joined #ipfs
<lgierth>
h.gateway.ipfs.io <3
<cryptix>
yup that sounds good too
<lgierth>
dns round robin with ipv6 is broken though, for backwards compat reasons :/
<cryptix>
btw can you hook me up on hyperboria? last time i checked you need to know somebody to get in?
<lgierth>
there are public peers now
<lgierth>
they're crappy :)
<lgierth>
i'll send you peering credentials for my amsterdam vps later tonight
<lgierth>
have you met kpcyrd yet? hh freifunker
<cryptix>
nope - i know kantorkel fairly well
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<lgierth>
i have no idea if he's still in hamburg tbh. last thing i heard was he started working for an obscure vpn provider :)
<lgierth>
anyhow, gotta run! you'll get a mail
<jbenet>
daviddias: so node-spdystream after indutny's refactor? The issue would be we couldn't test any dht (or other protocol) progress against go-ipfs yet
<cryptix>
lgierth: have fun! :)
<lgierth>
:)
<daviddias>
jbenet: true, but I can still work in implementing all the logic to make DHT work
tilgovi has quit [Ping timeout: 264 seconds]
<daviddias>
nevertheless, if you have something else in mind I can/should put my energy on, let's shoot for it. Just want to make sure I'm not doing something for the sake of doing something which soon enough will be deprecated
<jbenet>
Yeah understood. Ok let's get going on the rest of the protocol then. (Btw I thought indutny's rewrite was to http2? Maybe I'm misremembering)
alu has quit [Changing host]
alu has joined #ipfs
<daviddias>
From what I understand, the goal was to add http2, but then he realised how to make it better overall. First I thought of using just what's there, but he did ask me to wait for that branch to be merged
<whyrusleeping>
sprint checkin: have open PRs for the dial blocking issue and for the eventlog http stuff
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
i also started a dev0.4.0 branch to stage changes for the 0.4.0 release
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
krl: yeah, give me 15min to get caught up on some notifs? or if you have to rush can shorten to 5
<krl>
no rush
williamcotton has joined #ipfs
Wallacoloo has joined #ipfs
williamcotton has quit [Read error: Connection reset by peer]
williamc_ has joined #ipfs
Aaron1011 has quit [Quit: Goodbye!]
Aaron1011 has joined #ipfs
Aaron1011 has quit [Changing host]
Aaron1011 has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot>
[go-ipfs] whyrusleeping closed pull request #1333: Do not merge - protocol change discussion (master...proto-fix) http://git.io/vIUOA
<whyrusleeping>
jbenet: would you be in favor of having a global timeout on gateway file requests?
williamc_ has quit [Ping timeout: 255 seconds]
<whyrusleeping>
like, ten minutes or something?
<whyrusleeping>
we're running into the situation where a client requests a file that doesnt have all of its blocks available
<whyrusleeping>
so the request hangs halfway, and without writing more of the file back to them we have no way of knowing they disconnected.
domanic_ has quit [Ping timeout: 245 seconds]
<jbenet>
whyrusleeping: no, imagine downloading a 100GB file.
<jbenet>
it will take a while.
<whyrusleeping>
well, i guess we have to deal with leaking memory then...
<jbenet>
what we should be doing is timing out if we cant find the stuff in a given amount of time
<jbenet>
the difference is that a connection making forward progress would not be disconnected prematurely.
<whyrusleeping>
yeah, i guess i could make a timeout writer thing
<jbenet>
like if next block is not found in X amount of time, time out.
<whyrusleeping>
that would cancel a context if a write doesnt happen within a given time
<jbenet>
timeout writer?
<cryptix>
whyrusleeping: have you seen my comment on this?
<whyrusleeping>
cryptix: must have missed it
<jbenet>
no, i think that's the opposite. a write to the http client will fail right away. it's a timeout in the other side, the ipfs side, that's needed.
<cryptix>
closeNotifier is what you want
<whyrusleeping>
cryptix: oooOoo, that is exactly what i want
<cryptix>
whyrusleeping: feel free to take over that branch from me
<cryptix>
or redo it.. its not much anyway
<whyrusleeping>
will do. thanks :)
<jbenet>
krl: ok am good to go
<krl>
ok i'll relocate, minute
Encrypt has quit [Quit: Sleeping time!]
notduncansmith has joined #ipfs
<krl>
jbenet: o/
<krl>
so we wanted to talk about electron distribution and general 'ipfs apps'
<krl>
if we want to call them that
notduncansmith has quit [Ping timeout: 264 seconds]
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
inconshr_ has joined #ipfs
inconshreveable has quit [Ping timeout: 265 seconds]
<ipfsbot>
[go-ipfs] whyrusleeping created close-notify (+1 new commit): http://git.io/vL4bX
<ipfsbot>
go-ipfs/close-notify 5e91c87 Jeromy: cancel contexts if http connections closes...
<ipfsbot>
[go-ipfs] whyrusleeping opened pull request #1388: cancel contexts if http connections closes (master...close-notify) http://git.io/vL4NG
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshr_ has quit [Read error: Connection reset by peer]
therealplato has quit [Ping timeout: 252 seconds]
therealplato has joined #ipfs
timgws has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
whyrusleeping dPow: pls add a test case for trickledag stuff https://github.com/ipfs/go-ipfs/pull/1379 or remove it entirely. we cannot keep adding untested crap
tilgovi has joined #ipfs
uhhyeahbret has quit [Quit: WeeChat 1.2]
uhhyeahbret has joined #ipfs
patcon has quit [Ping timeout: 244 seconds]
dandroid has quit [Ping timeout: 246 seconds]
uhhyeahbret has quit [Quit: WeeChat 1.2]
domanic_ has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
hellertime has quit [Quit: Leaving.]
tilgovi has quit [Ping timeout: 265 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]