<jbenet>
whyrusleeping, lgierth: i recommend NOT using mars for gateway service. this is an official high availability service we need to run. we're already relied upon. we do DNS load balancing right now. if a user clicks a link to "ipfs.io/..." and happened to get a DNS entry to mars, and mars crashed and waiting for whyrusleeping to wake up or something, that user
<jbenet>
is completely hosed. mars is not "devops managed". so please leave it out. whyrusleeping, if you want to see traffic, log into the other machines and attach to the containers to inspect running processes. don't kill them, as people _do depend_ on this service.
<jbenet>
every time an ipfs.io/ link 404s or 403s or 500s or anything like that, it's a huge "THIS THING SUCKS" sign. please take it seriously.
<whyrusleeping>
'ipfs add -r doesnt work, this thing sucks'
<jbenet>
(( perhaps we should load balance differently, to avoid the geographic locality issue ))
<jbenet>
whyrusleeping: of course, that too!
anshukla has joined #ipfs
<whyrusleeping>
aaaaany idea why ipfs would want to dial my router 800 or so times on port 5555?
<jbenet>
(did you bring it up for another reason beyond "we must fix this"?)
<whyrusleeping>
davidar: not to doubt your ability at all, but youre sure that you rebuilt and installed the new code, and you restarted your daemon?
rjeli has joined #ipfs
<jbenet>
whyrusleeping the next step on the IPLD saga is to plug it into go-ipfs dev 0.4.0. which wont be trivial. it may be better to draw the line, land 0.4.0 first, and do ipld after. i hate to do two migrations but at this point, it's hard.
<davidar>
whyrusleeping: haha, had the same thought myself. Built from a clean working directory, no daemon running, installed it under a different name, so reasonably sure
<whyrusleeping>
davidar: huh.
<whyrusleeping>
what kind of setup do you have?
<whyrusleeping>
SSD? raid?
<whyrusleeping>
lots of ram?
<jbenet>
i think he's on pollux. probably HDD.
<whyrusleeping>
beefy cpu?
<jbenet>
large disks.
<whyrusleeping>
okay
<jbenet>
whyrusleeping you should be able to go in there and inspect the i/o. (davidar may want to guide you)
notduncansmith has quit [Read error: Connection reset by peer]
atomotic has joined #ipfs
<mappum>
alu: wow, that project has grown a lot since i first saw it
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
chriscool has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
yosafbridge has quit [Ping timeout: 256 seconds]
SebastianCB has joined #ipfs
yosafbridge has joined #ipfs
atrapado has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
atomotic has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
voxelot has joined #ipfs
mildred has joined #ipfs
ygrek has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
mildred has quit [Ping timeout: 250 seconds]
t3sserakt has quit [Read error: Connection reset by peer]
mildred has joined #ipfs
voxelot has quit [Ping timeout: 244 seconds]
Encrypt has quit [Quit: Quitte]
<SebastianCB>
Hi, I started playing with IPFS and love it so far. What is right now the best option to mirror/backup files via IPFS except for having it just on my own machine(s)? I could put my block-directory into dropbox,storj etc but it all doesnt seem very elegant.
atomotic has joined #ipfs
<mappum>
SebastianCB: we're working on a service for that! you'll be able to pay to pin your files on a server
Wkter_ has left #ipfs ["Leaving"]
Leer10 has joined #ipfs
mildred has quit [Ping timeout: 255 seconds]
mildred has joined #ipfs
<blame>
cjdns as a p2p vpn-ish replacement is not bad.
<blame>
Only 1 system needs to be open to via nat, and it figures out how to route from there.
<blame>
I have my home workstation + vps + laptop networked pretty seamlessly.
<SebastianCB>
mappum: are you talking about filecoin? Cool, looking forward to whatever it is.
<SebastianCB>
Alternatively I was wondering if we could setup some service ourselves, e.g. IT department of a university would probably very willing to commit to this
<blame>
SebastianCB: How would you feel about an IRC-bot that traded pinning for bitcoin at a flat btc/mb rate?
<SebastianCB>
sounds good. I'd have to check how a uni would feel about paying btc/mb but I'm willing to find out
<blame>
I know my department would not be up for something like that
<SebastianCB>
Blame: checking cjdns, dont see how this could store a backup, just routing, do i get something wrong?
chriscool has quit [Ping timeout: 255 seconds]
<blame>
cjdns does not really do backups. Sorry, the messages were not intended as response to you
<SebastianCB>
ah thanks!
<blame>
folks in the channel had asked me to report back on how well that worked out for me
<blame>
ipfs is not really meant for backups either
<blame>
but it could be used for that
<blame>
ipfs is meant to create a robust public internet
<blame>
not private backups
<SebastianCB>
ok, got that, i guess im waiting for filecoin
<blame>
since everything you put on ipfs is viewable
<blame>
by anybody
<blame>
and even easy to find
<SebastianCB>
I'm interested in your IRC bot service I guess
<SebastianCB>
if we could deploy that also, e.g. locally in a department and students could commit stuff into that local IPFS-based backup it could be quite useful and I guess we'd use it
<blame>
Honestly I am not even sure I like the idea of such a bot
<blame>
but I wanted to see what people tought
<blame>
It is the sort of thing that is going to happen someday
<daviddias>
whyrusleeping: jbenet around? I need a quick clarification on bitswap
Encrypt has joined #ipfs
mildred has joined #ipfs
jamescarlyle has joined #ipfs
chriscool has joined #ipfs
<blame>
new experiments in ipfs+cjdns madness
<blame>
My DNS points a wildcard subdomain to my vps, my vps is connected via a private cjdns network to my workstation, my workstation is running ipfs. I abused nginx to path_foward to my workstation via cjdns to resolve the site via ipns.
amstocker has joined #ipfs
jamescarlyle has quit []
mildred has quit [Ping timeout: 272 seconds]
jb55_ is now known as jb55
jb55 is now known as jb55_
ygrek has quit [Ping timeout: 272 seconds]
mildred has joined #ipfs
mildred has quit [Ping timeout: 260 seconds]
t3sserakt has joined #ipfs
SebastianCB has quit [Ping timeout: 246 seconds]
patcon has joined #ipfs
bedeho has quit [Ping timeout: 255 seconds]
amstocker has quit [Ping timeout: 244 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
amstocker has joined #ipfs
<blame>
whyrusleeping: what does the dns -> ipns integration look like right now?
bedeho has joined #ipfs
simonv3 has joined #ipfs
amstocker has quit [Ping timeout: 246 seconds]
border has joined #ipfs
border_ has quit [Ping timeout: 250 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
chriscool has quit [Ping timeout: 240 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
t3sserakt has quit [Quit: Leaving.]
reit has quit [Ping timeout: 244 seconds]
reit has joined #ipfs
domanic has joined #ipfs
<whyrusleeping>
Blame: i think you put a TXT record that contains 'dnslink=/ipns/QmBlah' on your domain
<whyrusleeping>
run dig on ipfs.io
inconshreveable has quit [Remote host closed the connection]
<fleeky>
blame , i understood some of those words
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<blame>
fleeky: me too... I don't do web-dev so this is all very exciting to me.
<fleeky>
what is the use case ?
<fleeky>
was confused why you needed to jump through so many hoops
<whyrusleeping>
daviddias: i'm around
<daviddias>
I figured it out, thank you anyway :)
<whyrusleeping>
lol, fine!
<daviddias>
it was about how the mapping between blocks being provided and actual records is done
<daviddias>
since records are MDAG objs too therefore, can have their own hash
<whyrusleeping>
daviddias: yeah... thats gonna get weird
ygrek has joined #ipfs
<daviddias>
Sorry for not having it explained right away :)
<daviddias>
What I'm doing in the DRS, inspired by the current DHT impl in go-ipfs is making a record being looked up by the hash of block it is refereeing
<daviddias>
and so when a DRS gets a request for key->k, I check a table for which Records are linked to key k, that table will have the hashes of the Signature of the Records (which link to the Record itself) and that are stored inside the MerkleDAG Store
<whyrusleeping>
makes sense
<daviddias>
I say inspired by the current DHT impl, because right now every Record is looked up by the block hash (which will remain the same), although the DHT doesn't need to have this mapping as the 'records' are just simple objs
<blame>
fleeky: mostly for the fun of it. I wanted my website hosted on ipfs, I don't have ipfs running on my vps yet, I have it running on my workstation, I was learning nginx and I just setup a secure cjdns virtual network between my workstation, vps, and laptop. So I wrote a nginx config that forwarded my website to my workstation's ipfs node via the secure
<blame>
virtual network.
<daviddias>
Blame, you can always host your website locally and let the gateways do the magic of finding and caching your website for you as clients request it
<fleeky>
ahhh
<fleeky>
heh nice
<daviddias>
IPFS will know how to traverse the network and get to your machine in your LAN
<blame>
daviddias I plan on doing that too
<daviddias>
however, your solution sounds like a very cool hack, please do it and document it :)
<blame>
but should I just use nginx to forward my domain to a public gateway?
<fleeky>
i really want to run my sites off of ipfs soon, but i dont have enough time to get serious about ipfs atm
mquandalle has joined #ipfs
<blame>
daviddias: right now I have my nginx config setup to forward requests to ipfs.io's gateway iff my ipfs node goes down. Whats best practice here?
warptangent has quit [Read error: Connection reset by peer]
<blame>
daviddias: The ideal way would be something like a cdn style dns hack that points you at the closest "public" gateway
<daviddias>
do you want to have the resources on the webpage being loaded directly by ipfs.io/ipfs/... or you want to have your site served entirely by your domain (using nginx as a proxy)?
warptangent has joined #ipfs
notduncansmith has joined #ipfs
<blame>
both?
notduncansmith has quit [Read error: Connection reset by peer]
<fleeky>
im confused on the part where your href's are normal , but it links to the proper ipfs hash
<daviddias>
Blame: on that, lgierth will be the best person to provide with more creative ways. But I wouldn't say that looks bad
* fleeky
looks at your page source
<blame>
fleeky: I'm missing somethign in your question
<fleeky>
i probably dont understand ipfs enough
<fleeky>
so in your page code it links to say index.html
<fleeky>
but then it goes to ipfs/hashhh
<blame>
the last server in the list is my site
<blame>
it does manually link to a given hash
<fleeky>
so nginx is doing the translation
<blame>
the "ipns_hosts" is a hack to let me use ipfs.io as a backup
<blame>
yeah
<fleeky>
nicee
<fleeky>
so how does nginx do the translation ?
<fleeky>
i have a server at my house that i was going to setup apache and such with that would be my new self hosted server but this sounds much more interesting
ygrek has quit [Ping timeout: 250 seconds]
<blame>
nginx was obtuse at first
<blame>
but essentially it a pattern match + transform
<blame>
My goal was to have more controll over subdomains
<blame>
so I could forward sub-domains to services I have running on odd ports.
<blame>
Now I just need a static site generate that I do not hate
<blame>
I could write all the html myself, but I loath it.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<fleeky>
yeah writing that html manually sounds.. painfull
notduncansmith has quit [Read error: Connection reset by peer]
chriscool has quit [Ping timeout: 256 seconds]
<mappum>
Blame: nice GoL
<blame>
Its kinda a theme for me
<ipfsbot>
[go-ipfs] whyrusleeping created fix/debug-err (+1 new commit): http://git.io/vZezO
<ipfsbot>
go-ipfs/fix/debug-err 308ab9a Jeromy: remove error log i forgot to remove from another PR...
<ipfsbot>
[go-ipfs] whyrusleeping opened pull request #1657: remove error log i forgot to remove from another PR (master...fix/debug-err) http://git.io/vZezG
Encrypt has quit [Quit: Quitte]
<lgierth>
Blame: reading backlog
<lgierth>
jbenet: are the ipfs presentations online somewhere? someone's asking for the layers slide
<ipfsbot>
[go-ipfs] whyrusleeping pushed 1 new commit to master: http://git.io/vZegi
<whyrusleeping>
thats why the gateways have */ipfs/XXXX
<whyrusleeping>
instead of just */XXXX like you have
<blame>
no, I mean the relative links look right, but instead they get forwarded to ipns.blamestross.com/ipns/blah rather than ipns.blamestross.com/blah
<whyrusleeping>
hrm...
<whyrusleeping>
lgierth: any idea?
* blame
is sorry he breaks things so regularly
<lgierth>
Blame: yeah well ipfs sees a request for /ipfs/...
<lgierth>
it has no clue you're messing with the URI
<lgierth>
you're rewriting on the way in, so you also need to rewrite on the way out :)
<lgierth>
i had to do that in the gatewayhandler too
<lgierth>
for directory listings and redirects to work correctly
Leer10 has quit [Excess Flood]
<border>
ok gentlemans, I have some bandwith aviable, how can I help ?
Leer10 has joined #ipfs
reit has quit [Ping timeout: 272 seconds]
notduncansmith has joined #ipfs
thomasreggi has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<blame>
border: the short answer is just run an ipfs node? Am I missing context?
<whyrusleeping>
border: yeap, just run the ipfs daemon :)
<whyrusleeping>
you can pin popular content for other people if you like as well
<border>
and if i don't query stuff, it gonna contribute to the network ?
<border>
ok so I need to do query in order to "seed"
<border>
?
<lgierth>
yeah
<lgierth>
query or pin
<lgierth>
whyrusleeping: maybe we should have a pinlist
<whyrusleeping>
border: just running a node will help out with the dht, hosting lookup information and provider records
<whyrusleeping>
lgierth: that would be a good idea 'pin these things to help us out'
<border>
is there a discovery process, or it's like the dark net
<border>
ah ok good cool
<lgierth>
border: there are bootstrap nodes specified in the default config
<lgierth>
whyrusleeping: can be a simple object in ipns
<lgierth>
linking to all pinned objects (non-recursive)
<lgierth>
s/pinned/to-be-pinned/
* blame
now has the same dns -> nginx -> ipns -> dns -> ipfs loop ipfs.io has
<blame>
interestingly, if you have a public gateway, we can use that to convince a bunch of gateways to soft-pin content
simonv3 has quit [Quit: Connection closed for inactivity]
inconshreveable has quit [Remote host closed the connection]
Leer10 has quit [Ping timeout: 240 seconds]
<border>
whats a gateway ?
<whyrusleeping>
border: a gateway is a public node serving content over http from ipfs
<border>
like tor2web ?
<blame>
I have my domain + ipfs node setup so you can get anything on ipfs via blamestross.com/ipfs/hashhere