pfrazee has quit [Remote host closed the connection]
wallacoloo____ has joined #ipfs
apiarian has joined #ipfs
vtomole has joined #ipfs
apiarian has quit [Read error: Connection reset by peer]
apiarian has joined #ipfs
jkilpatr has joined #ipfs
apiarian has quit [Ping timeout: 240 seconds]
apiarian has joined #ipfs
apiarian has quit [Read error: Connection reset by peer]
apiarian has joined #ipfs
apiarian has quit [Read error: Connection reset by peer]
apiarian has joined #ipfs
mildred2 has quit [Ping timeout: 256 seconds]
rcat has quit [Remote host closed the connection]
apiarian has quit [Read error: Connection reset by peer]
apiarian has joined #ipfs
apiarian has quit [Read error: Connection reset by peer]
apiarian has joined #ipfs
apiarian has quit [Ping timeout: 256 seconds]
mildred2 has joined #ipfs
apiarian has joined #ipfs
wallacoloo____ has quit [Quit: wallacoloo____]
wallacoloo____ has joined #ipfs
cyanobacteria has quit [Ping timeout: 255 seconds]
cyanobacteria has joined #ipfs
userydfkumydfjyt has joined #ipfs
MDude has joined #ipfs
JayCarpenter has joined #ipfs
user_htrdfhytfg has joined #ipfs
crossdiv1r has quit [Quit: leaving]
userydfkumydfjyt has quit [Ping timeout: 256 seconds]
crossdiver has joined #ipfs
ianopolous has quit [Remote host closed the connection]
wkennington has joined #ipfs
tmg has quit [Ping timeout: 248 seconds]
cemerick has joined #ipfs
cemerick has quit [Ping timeout: 240 seconds]
ianopolous has joined #ipfs
shizy has joined #ipfs
tilgovi has joined #ipfs
wallacoloo____ has quit [Quit: wallacoloo____]
ygrek has quit [Ping timeout: 248 seconds]
anonymuse has quit [Remote host closed the connection]
tmg has joined #ipfs
reit has joined #ipfs
shizy has quit [Ping timeout: 260 seconds]
JayCarpenter has quit [Quit: Page closed]
tilgovi has quit [Ping timeout: 255 seconds]
mguentner has quit [Quit: WeeChat 1.6]
user_htrdfhytfg has quit [Ping timeout: 245 seconds]
mguentner has joined #ipfs
tilgovi has joined #ipfs
tilgovi has quit [Ping timeout: 245 seconds]
kvda has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
wallacoloo____ has joined #ipfs
tilgovi has joined #ipfs
mguentner2 has joined #ipfs
mguentner has quit [Ping timeout: 256 seconds]
ygrek has joined #ipfs
vtomole has quit [Ping timeout: 260 seconds]
tmg has quit [Ping timeout: 252 seconds]
kulelu88 has quit [Quit: Leaving]
<sneak>
Is there any plan or discussion to integrate an option for compression at any point in the object storage layer?
<sneak>
Or is that out of scope
<sneak>
It would be nice to be able to ipfs add -z and have the blocks themselves be comprised of compressed data that gets decompressed on dereference
<sneak>
Or maybe that is a job for the file system and p2p layer
<sneak>
Compression on the wire would solve most of the use case I am thinking of.
<sneak>
Is that done now?
chris613 has quit [Quit: Leaving.]
lidel has quit [Ping timeout: 252 seconds]
mildred2 has quit [Ping timeout: 240 seconds]
lidel has joined #ipfs
cyanobacteria has quit [Ping timeout: 255 seconds]
ike_ has joined #ipfs
iczero has quit [Quit: jeffl35 has quit (Quit: jeffl35 has quit (Quit: jeffl35 has quit (Quit: jeffl35 has quit ...]
iczero has joined #ipfs
ygrek has quit [Remote host closed the connection]
ulrichard has joined #ipfs
diffalot_ has joined #ipfs
diffalot_ has joined #ipfs
diffalot_ has quit [Changing host]
diffalot has quit [Ping timeout: 240 seconds]
cyanobacteria has joined #ipfs
Guest47221 has quit []
mildred2 has joined #ipfs
akkad has quit [Ping timeout: 240 seconds]
mildred has joined #ipfs
mildred1 has joined #ipfs
akkad has joined #ipfs
Caterpillar has joined #ipfs
kevina has quit [Quit: Leaving]
rendar has joined #ipfs
wrouesnel2 has joined #ipfs
ylp has joined #ipfs
A124 has quit [Ping timeout: 245 seconds]
wkennington has quit [Read error: Connection reset by peer]
<daviddias>
Good morning planet :)
ike_ has quit [Quit: Connection closed for inactivity]
slothbag has joined #ipfs
slothbag has left #ipfs [#ipfs]
s_kunk has quit [Quit: Read error: Connection reset by beer]
tmg has joined #ipfs
cyanobacteria has quit [Ping timeout: 258 seconds]
tilgovi has quit [Remote host closed the connection]
mguentner2 is now known as mguentner
krs93 has joined #ipfs
wrouesnel2 has joined #ipfs
maxlath has quit [Quit: maxlath]
maxlath has joined #ipfs
Admin__ has joined #ipfs
Admin__ is now known as Hory
Boomerang has joined #ipfs
meehow has joined #ipfs
cemerick has joined #ipfs
<victorbjelkholm_>
Hello World!
espadrine has joined #ipfs
infinity0_ has joined #ipfs
infinity0 has joined #ipfs
infinity0 is now known as Guest78056
infinity0_ is now known as infinity0
Guest78056 has quit [Killed (adams.freenode.net (Nickname regained by services))]
infinity0 has quit [Remote host closed the connection]
bastianilso____ has joined #ipfs
infinity0 has joined #ipfs
rcat has joined #ipfs
infinity0 has quit [Remote host closed the connection]
infinity0 has joined #ipfs
infinity0 has quit [Remote host closed the connection]
infinity0 has joined #ipfs
rcat has quit [Ping timeout: 240 seconds]
Hory has quit [Ping timeout: 240 seconds]
zanadar has joined #ipfs
Hory has joined #ipfs
jkilpatr has quit [Ping timeout: 255 seconds]
<Kubuxu>
lgierth: this trace is not fully available
zanadar has quit [Ping timeout: 255 seconds]
<kpcyrd>
is anybody running ipfs on digital ocean and, if yes, which size is the droplet? OOM killer is really trigger happy and it kills other applications occasionally
<kpcyrd>
woke up today and my project was down :/
<victorbjelkholm_>
kpcyrd, I run IPFS on the cheapest droplet but with swap, not the best solution but stops from OOM kill
<kpcyrd>
victorbjelkholm_: how large is your swap?
cwahlers has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
corvinux has joined #ipfs
nunofmn has quit [Ping timeout: 240 seconds]
zanadar has joined #ipfs
meehow has quit [Quit: Leaving]
mildred1 has quit [Ping timeout: 252 seconds]
mildred_ has quit [Ping timeout: 248 seconds]
espadrine has quit [Ping timeout: 252 seconds]
cwahlers has joined #ipfs
maxlath has quit [Ping timeout: 240 seconds]
zanadar has quit [Ping timeout: 258 seconds]
espadrine has joined #ipfs
maxlath has joined #ipfs
Boomerang has joined #ipfs
zanadar has joined #ipfs
ashark has joined #ipfs
zanadar has quit [Ping timeout: 240 seconds]
corvinux has quit [Ping timeout: 240 seconds]
shizy has joined #ipfs
Boomerang has quit [Ping timeout: 255 seconds]
pfrazee has joined #ipfs
Boomerang has joined #ipfs
mildred_ has joined #ipfs
mildred1 has joined #ipfs
cemerick has quit [Ping timeout: 240 seconds]
<pjz>
victorbjelkholm_: I just run it with a restart-on-kill process watcher (supervisord)
<pjz>
victorbjelkholm_: so it dies OOM but gets restarted
<pjz>
also: if anyone uses netdata, I just put in a PR that adds a datastore-low-on-space alarm
<Mateon1>
Cool, was it merged or have you just created it?
<pjz>
just created it
<Mateon1>
Ah, I'll let cron do its job then
martinBrown has joined #ipfs
<pjz>
took a tweak to the ipfs usage graph as well
<pjz>
I'm trying to get alarms/alerts set up so I can launch pinbits.io and be mostly hands-off
<pjz>
next I rewrite the coupon code workings and then... reddit, I guess
<pjz>
oh, i should re-run my financials to make sure I'm at least covering costs :)
Guest75149[m] has joined #ipfs
ebarch has quit [Remote host closed the connection]
ebarch has joined #ipfs
Boomerang has quit [Ping timeout: 256 seconds]
Boomerang has joined #ipfs
ulrichard has quit [Remote host closed the connection]
mildred1 has quit [Ping timeout: 240 seconds]
mildred_ has quit [Ping timeout: 240 seconds]
ylp has quit [Quit: Leaving.]
mildred2 has quit [Ping timeout: 256 seconds]
Hory has quit [Ping timeout: 256 seconds]
apiarian has quit [Ping timeout: 258 seconds]
apiarian has joined #ipfs
cemerick has joined #ipfs
Jackneilll has joined #ipfs
Jackneill has quit [Ping timeout: 248 seconds]
krs93 has quit [Read error: Connection reset by peer]
Boomerang has quit [Ping timeout: 260 seconds]
G-Ray_ has quit [Quit: G-Ray_]
MDude has quit [Ping timeout: 248 seconds]
Hory has joined #ipfs
Hory has quit [Read error: Connection reset by peer]
G-Ray_ has joined #ipfs
Boomerang has joined #ipfs
G-Ray_ has quit [Quit: G-Ray_]
mildred2 has joined #ipfs
<Mateon1>
pjz: At 1$ per terabyte hour, definitely. I can find storage at 30$ per terabyte month, and bandwidth isn't terribly expensive
espadrine has quit [Ping timeout: 252 seconds]
Encrypt has joined #ipfs
bronger has joined #ipfs
rcat has joined #ipfs
<frood>
for cloud stores bandwidth is typically $50 - $90/TB
strixx has joined #ipfs
iczero is now known as wlp1s1
voldz1tic3n has joined #ipfs
s_kunk has quit [Ping timeout: 256 seconds]
kevina has joined #ipfs
Mizzu has joined #ipfs
espadrine has joined #ipfs
Boomerang has quit [Ping timeout: 258 seconds]
jkilpatr has quit [Ping timeout: 252 seconds]
jkilpatr has joined #ipfs
Boomerang has joined #ipfs
<pjz>
Mateon1: It's not the storage, it's keeping the server running
<pjz>
Mateon1: also, well, my software dev time isn't free :)
voldz1tic3n has quit [Ping timeout: 258 seconds]
ygrek has joined #ipfs
<kevina>
whyrusleeping: around?
palkeo has joined #ipfs
palkeo has joined #ipfs
palkeo has quit [Changing host]
Encrypt has quit [Quit: Quit]
<pjz>
Mateon1: though the storage is cheaper than the servers for sure, so as I get more customers I can lower the price some
<Mateon1>
Yeah, still, a 20$/mo VPS can easily handle IPFS
maxlath has quit [Ping timeout: 240 seconds]
strixx has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<pjz>
sure, but you only get ~ 30-40GB of storage for that $20/mo
<pjz>
$20/mo for 40GB total storage ( take off 1-2GB for OS )
<pjz>
...works out to ~0.75 per Tb*hr
Boomerang has quit [Quit: leaving]
jonnycrunch has joined #ipfs
<pjz>
so if I charge $1/TB*hr, I end up making something like $10/mo ...if it's full to capacity
arpu has quit [Ping timeout: 245 seconds]
<pjz>
...which I of course can't count on
tilgovi has joined #ipfs
maxlath has joined #ipfs
arpu has joined #ipfs
jkilpatr has quit [Ping timeout: 255 seconds]
<Mateon1>
True. There do exist providers that allow you to attach cheap HDD backed storage to your VPS
s_kunk has joined #ipfs
<pjz>
So anyway
<pjz>
I'm going to raise it to $2 to start with
<Mateon1>
Ah, you changed it
kulelu88 has joined #ipfs
<Mateon1>
Also, how do you measure the size of the pin? If you use "CumulativeSize", that can be forged
<Mateon1>
Also, it doesn't count deduplication
<pjz>
hmm, I went around with whyrusleeping on this :)
<pjz>
I think I went with CumulativeSize
<pjz>
let me check the code :)
<pjz>
yeah, I did that
<pjz>
I used to have code to like traverse the object graph and dedup and etc
<pjz>
but the nice thing about CumulativeSize is that it's a single block fetch from a remote to find out
<Mateon1>
Yeah, both approaches have problems
<pjz>
though I'll note that I start the timer as soon as I do the pin
<Mateon1>
Traversing the graph can DOS if you are given a large graph (like a zipbomb)
<pjz>
sure
<Mateon1>
Well...
<pjz>
so worst case all the storagetime gets eaten up by download time
<pjz>
at which point it gets unpinned
<Mateon1>
You can be smart about the graph traversal, marking hashes as "visited"
rcat has quit [Ping timeout: 248 seconds]
<pjz>
but you have to download them first
<pjz>
to be able to do anything fancier than look at CS
voldz1tic3n has joined #ipfs
jkilpatr has joined #ipfs
jonnycrunch has quit [Ping timeout: 255 seconds]
arkimedes has joined #ipfs
rcat has joined #ipfs
pfrazee has quit [Remote host closed the connection]
cemerick has quit [Ping timeout: 255 seconds]
cemerick has joined #ipfs
rendar has quit [Ping timeout: 240 seconds]
Boomerang has joined #ipfs
Encrypt has joined #ipfs
<lgierth>
Kubuxu: mh ok that explains my trouble using tools with it. it might a case of daemon-crashes-when-dumping-too-many-goroutines
<lgierth>
Kubuxu: i'll try to get a new one
<lgierth>
smaller, but complete
<lgierth>
edsilv[m]: hey sorry the storage access has taken a while, we were all a bit busy. could you tell me an ssh key of yours? i'll get it done tonight
cemerick has quit [Ping timeout: 255 seconds]
cemerick has joined #ipfs
<edsilv[m]>
lgierth: hey, no worries - definitely fine to post a public key here?
voldz1tic3n has quit [Ping timeout: 260 seconds]
anonymuse has quit [Remote host closed the connection]
cemerick has quit [Ping timeout: 240 seconds]
rendar has joined #ipfs
onabreak has quit [Quit: Page closed]
warner has quit [Quit: ERC (IRC client for Emacs 25.1.1)]
onabreak has joined #ipfs
clavi has quit [Ping timeout: 260 seconds]
galois_dmz has joined #ipfs
anonymuse has joined #ipfs
clavi has joined #ipfs
cemerick has joined #ipfs
anonymuse has quit [Remote host closed the connection]
step21 has quit [Ping timeout: 256 seconds]
seg_violation has quit [Ping timeout: 258 seconds]
step21_ has joined #ipfs
seg_violation has joined #ipfs
step21_ is now known as step21
superluser[m] has joined #ipfs
<substack>
adding peermaps data v2 to ipfs now so I can see if tilegen works on my dev machine
jkilpatr has quit [Ping timeout: 256 seconds]
<Kubuxu>
lgierth: I will take a look at it once more
<Kubuxu>
I might be able to recover it
<lgierth>
Kubuxu: here, ovenfresh from uranus: Qme1hTZkdzPLuUgjAP889vuUa3vowBfCyttjwPh7Lpu1KZ
<lgierth>
and jupiter is ramping up too. here's the command i use: curl localhost:5001/debug/pprof/goroutine\?debug=2 | docker exec -i ipfs ipfs add
<lgierth>
i gotta run now
<lgierth>
mmmmmh it might not be complete either. it's exactly 64MB again
<lgierth>
here's the one from jupiter: QmSZ1fURUFqzMSyDfNKsMLESckYGq2rUs2QSzXLFxKpouJ
gsf has quit [Changing host]
gsf has joined #ipfs
rendar has quit [Quit: std::lower_bound + std::less_equal *works* with a vector without duplicates!]
<whyrusleeping>
do we want to follow the ipld spec?
<whyrusleeping>
lol
<whyrusleeping>
(js-ipfs isnt either, so i feel better)
step21 has quit [Ping timeout: 240 seconds]
seg_violation has quit [Ping timeout: 240 seconds]
pfrazee has joined #ipfs
step21_ has joined #ipfs
<bronger>
Does the key "link" mean something special for IPLD? On https://github.com/ipld/specs/tree/master/ipld, some examples suggest that one has to traverse through it with an explicit "/link/" component in the path, while others suggest that it is implicit.
seg_violation has joined #ipfs
step21_ is now known as step21
<bret>
whyrusleeping: any plans to try to use gx with node_modules?
<kevina>
whyrusleeping: still around?
voldz1tic3n has joined #ipfs
<whyrusleeping>
bret: Yeah, i really want to make a gx-js. ask victorbjelkholm_ about it, hes done some stuff with that in the past
<whyrusleeping>
kevina: for a few minutes yeah
<whyrusleeping>
in a hostel in iceland at the moment, internet seems stable
<bret>
sweet
<kevina>
whyrusleeping: just want to move forward with the conversion code
<whyrusleeping>
kevina: ah, yeah. i'll take a look at that
<whyrusleeping>
I think keeping sharding named as in is fine
jkilpatr has joined #ipfs
shizy has joined #ipfs
<kevina>
okay
cemerick has quit [Ping timeout: 255 seconds]
wrouesnel has joined #ipfs
wrouesnel has quit [Remote host closed the connection]
wrouesnel has joined #ipfs
Encrypt has quit [Quit: Quit]
vasile has left #ipfs [#ipfs]
Mizzu has quit [Ping timeout: 252 seconds]
<wrouesnel>
has anyone running ipfs nodes noticed a large amount (~1mbit) of traffic from the process that does not seem to be reflected in the actual datain/out of the ipfs stats?
<Mateon1>
wrouesnel: The `ipfs stats bw` report is greatly inaccurate, probably due to various overhead
anonymuse has joined #ipfs
<whyrusleeping>
wrouesnel: i havent really noticed. But i havent paid too close attention
<whyrusleeping>
Mateon1: ipfs stats bw is much closer now than it used to be
<whyrusleeping>
its still off by a bit, but not a factor of 10x
<Mateon1>
whyrusleeping: On current master, the stats bw report was about 10 times lower than actual
<Mateon1>
Uh
<whyrusleeping>
>.>
<whyrusleeping>
hm
<wrouesnel>
^^
* whyrusleeping
goes to sleep
<wrouesnel>
that's the problem I'm having. well, part of it
<wrouesnel>
the other being the whole "uses all my upstream just speaking dht"
<Mateon1>
That's odd, IPFS only uses about 10kB/s (as reported by stats bw...) when idle
<wrouesnel>
Mateon1: with dhtclient mode, ipfs itself reports 3-5kB/s - which would be fine. but my router reports ~1mbit of upload.
<wrouesnel>
Mateon1: confirmed because when i kill the the ipfs container on my server, suddenly my connection is usable again.
<Mateon1>
Just to make sure, are you getting the same traffic without IPFS?
<Mateon1>
Ah...
<Mateon1>
Well, that's weird
<Mateon1>
Let me check my server
<wrouesnel>
I haven't run wireshark yet to see what it is (i.e. it could just be some truly staggering number of dns requests or something)
<Mateon1>
Well, I'm getting 200-500 kbps right now, across all applications
<Mateon1>
For reference, IPFS disabled: 100-120/250-300 kbps
shizy has quit [Ping timeout: 240 seconds]
* Mateon1
waits for a bunch of peers to connect
<Mateon1>
Okay, with IPFS enabled: 150-180/250-350 kbps
Jeon has joined #ipfs
robattila256 has joined #ipfs
Jeon has quit [Quit: Page closed]
espadrine has quit [Ping timeout: 240 seconds]
ashark has quit [Ping timeout: 240 seconds]
mib_kd743naq has quit [Ping timeout: 260 seconds]
shizy has joined #ipfs
chris613 has joined #ipfs
strixx has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
MDude has joined #ipfs
maxlath has quit [Quit: maxlath]
rcat has quit [Quit: leaving]
galois_dmz has quit []
drathir has quit [Remote host closed the connection]
voldz1tic3n has quit [Ping timeout: 260 seconds]
galois_dmz has joined #ipfs
galois_d_ has joined #ipfs
galois_dmz has quit [Ping timeout: 245 seconds]
Caterpillar has quit [Quit: You were not made to live as brutes, but to follow virtue and knowledge.]
galois_d_ has quit [Ping timeout: 245 seconds]
<lgierth>
bronger: no, there's simply an element called "link" in that example document. it's unfortunate naming :)
<lgierth>
a link itself is just an object "/":"Qmhash"
<lgierth>
and the forward-slash there is not a path, it's a reserved character with special meaning in that case