jbenet changed the topic of #ipfs to: IPFS - InterPlanetary File System - https://github.com/ipfs/ipfs -- channel logged at https://botbot.me/freenode/ipfs/ -- Code of Conduct: https://github.com/ipfs/community/blob/master/code-of-conduct.md -- Sprints: https://github.com/ipfs/pm/ -- Community Info: https://github.com/ipfs/community/ -- FAQ: https://github.com/ipfs/faq -- Support: https://github.com/ipfs/support
Ape has joined #ipfs
<mildred> See also PR #63 https://github.com/ipfs/specs/pull/63 for precisions about the canonical format.
<Ape> Is anybody using ipfs as a backup utility? I'm thinking of combining ipfs with something like zbackup, then just pinning my data on multiple servers.
<mildred> and I'm going to sleep soon. It's 1 am in France
<Ape> zbackup for encryption and pre-encryption deduplication
<Ape> Is there a way to verify that my data is being pinned on a number of servers without necessarily trusting those servers?
<Ape> E.g. by making random queries to check that a random piece of my data is available
NightRa has quit [Quit: Connection closed for inactivity]
<jbenet> mildred: will see that too. thank you so much for all the help here, and again sorry for not having a clearer resolution sooner
<Not_> >Is there a way to verify that my data is being pinned on a number of servers without necessarily trusting those servers?
<Not_> this would be sweet
<Not_> knowing how many pins it had
<Not_> like seeds
<mildred> jbenet: thank you for looking up these PR.
<Not_> Ape, you can open a github request
<achin> if a gateway returns "context deadline exceeded" does it keep the hash in its wantlist?
<ion> jbenet: (I'm sure you're sick of hearing variants of this joke.) Q: How many IPFS developers does it take to change a lightbulb? A: Just Juan.
anticore has quit [Ping timeout: 260 seconds]
jager_ is now known as jager
<Ape> Not_: I will
<mildred> I'll leave you to get some sleep. Good night :)
<Ape> Also, does anybody have experience in using ipfs for this kind of personal backup? Is there any tools already available to make it easier?
Soft has quit [Read error: Connection reset by peer]
<feross> whyrusleeping: you should join #arcticjs !
mildred has quit [Ping timeout: 250 seconds]
ygrek_ has quit [Ping timeout: 240 seconds]
ygrek has joined #ipfs
longguang has quit [Ping timeout: 245 seconds]
Gaboose has joined #ipfs
longguang has joined #ipfs
uhhyeahbret has quit [Quit: WeeChat 1.3]
uhhyeahbret has joined #ipfs
joshbuddy has quit [Quit: joshbuddy]
Soft has joined #ipfs
pfraze has joined #ipfs
reit has joined #ipfs
pfraze has quit [Remote host closed the connection]
joshbuddy has joined #ipfs
rendar has quit [Quit: std::lower_bound + std::less_equal *works* with a vector without duplicates!]
joshbuddy has quit [Quit: joshbuddy]
voxelot has quit [Ping timeout: 240 seconds]
rektide has quit [Ping timeout: 246 seconds]
warptangent_ has quit [Ping timeout: 240 seconds]
devbug has quit [Ping timeout: 245 seconds]
rektide has joined #ipfs
warptangent has joined #ipfs
<jakoby> Ape: afaik nothing will pin unless you tell it to
<lgierth> ipfs add pins
<jakoby> so, we don't have the ability to pin on servers we don't have access to
<achin> correct
<lgierth> ipfs add pins
<lgierth> oops
<jakoby> could one just periodically load the data via the gateway to have the gateway hold on to the cache
<lgierth> yeah
<jakoby> cool
hashcore has quit [Ping timeout: 240 seconds]
<jakoby> so, is it a known thing that the fuse mounts are slow and cpu intensive currently?
<achin> there is no promises that the gateway will hold on to the cache for any period of time, though
<jakoby> or should I submit an issue? :)
<achin> best to submit an issue
Kalon has joined #ipfs
joshbuddy has joined #ipfs
pfraze has joined #ipfs
disgusting_wall has quit [Quit: Connection closed for inactivity]
Pria has quit [Ping timeout: 272 seconds]
Kalon has quit [Quit: Kalon]
autoxeny has joined #ipfs
<Ape> jakoby: I can request somebody to pin my data. Then I want to verify if they did so. I can't make anybody to pin my data, but perhaps they are friendly or I could pay them.
<noffle> see also http://filecoin.io if you haven't heard of it
_whitelogger_ has joined #ipfs
ralphtheninja has joined #ipfs
pfraze has quit [Remote host closed the connection]
ralphtheninja has quit [Quit: Lost terminal]
ralphtheninja has joined #ipfs
ralphtheninja has quit [Quit: Lost terminal]
ralphtheninja has joined #ipfs
pfraze has joined #ipfs
ScoreUnder has joined #ipfs
jager_ has joined #ipfs
joshbuddy has quit [Quit: joshbuddy]
infinity0_ has joined #ipfs
infinity0 has quit [Killed (asimov.freenode.net (Nickname regained by services))]
infinity0_ is now known as infinity0
zen|merge_ has joined #ipfs
Not_ has quit [*.net *.split]
AtnNn has quit [*.net *.split]
M-giodamelio has quit [*.net *.split]
nekomune has quit [*.net *.split]
Confiks_ has quit [*.net *.split]
jager has quit [*.net *.split]
Score_Under has quit [*.net *.split]
sbruce has quit [*.net *.split]
zen|merge has quit [*.net *.split]
guybrush has quit [*.net *.split]
riveter has quit [*.net *.split]
Not_ has joined #ipfs
AtnNn has joined #ipfs
M-giodamelio has joined #ipfs
nekomune has joined #ipfs
Confiks_ has joined #ipfs
guybrush has joined #ipfs
riveter has joined #ipfs
sbruce has joined #ipfs
feross has quit [Ping timeout: 260 seconds]
zmanian_ has quit [Ping timeout: 260 seconds]
feross has joined #ipfs
zmanian_ has joined #ipfs
jager_ is now known as jager
<brimstone> herm, any tips for when outbound tcp/4001 is blocked?
{prosody} is now known as prosody
voxelot has joined #ipfs
simonv3 has quit [Quit: Connection closed for inactivity]
<jakoby> Ape: I think you can use this: https://ipfs.io/docs/commands/#ipfs-dht-findprovs
voxelot has quit [Ping timeout: 256 seconds]
<jakoby> brimstone: you can change ports in ~/.ipfs/config
<brimstone> jakoby: for my local daemon, but not the bootstrap peers :(
<jakoby> oh, outbound is blocked? yeah, you're screwed :(
<brimstone> yeah :(
<jakoby> afaik. I'm not an ipfs dev, but I'm pretty sure.
<brimstone> and the swarm is all tcp right?
<brimstone> so no udp tricks
<Not_> any of you run ipfs through cjdns?
<jakoby> brimstone: you could just make your own network with your own peers :)
<jakoby> change bootstrap to just your own
<Not_> we have a group of peers with cjdns and are looking for some more that run ipfs with it to see how much faster we can get ipfs through it
<brimstone> jakoby: i'm trying to do that now
patcon has joined #ipfs
dignifiedquire has quit [Quit: Connection closed for inactivity]
Gaboose has quit [Remote host closed the connection]
voxelot has joined #ipfs
cemerick has joined #ipfs
jaboja has joined #ipfs
sivachandran has joined #ipfs
jaboja has quit [Remote host closed the connection]
hartor has quit [Ping timeout: 245 seconds]
_whitelogger has joined #ipfs
jaboja has joined #ipfs
akkad has quit [Excess Flood]
akkad has joined #ipfs
grahamperrin has joined #ipfs
r04r is now known as zz_r04r
sivachandran has quit [Remote host closed the connection]
Soft has quit [Ping timeout: 245 seconds]
<whyrusleeping> brimstone: on ipfs 0.4.0 you can use udp
prim1 has quit [Ping timeout: 250 seconds]
prim1 has joined #ipfs
Soft has joined #ipfs
patcon has quit [Ping timeout: 255 seconds]
pfraze has quit [Remote host closed the connection]
reit has quit [Ping timeout: 256 seconds]
reit has joined #ipfs
ygrek has quit [Ping timeout: 245 seconds]
patcon has joined #ipfs
joshbuddy has joined #ipfs
grahamperrin has left #ipfs ["Leaving"]
Soft has quit [Ping timeout: 265 seconds]
Tv` has quit [Quit: Connection closed for inactivity]
Not_ has quit [Ping timeout: 276 seconds]
Soft has joined #ipfs
joshbuddy has quit [Quit: joshbuddy]
joshbuddy has joined #ipfs
jaboja has quit [Remote host closed the connection]
neurosis12 has joined #ipfs
neurosis12 has quit [Changing host]
neurosis12 has joined #ipfs
ygrek has joined #ipfs
cemerick has quit [Ping timeout: 256 seconds]
ygrek has quit [Ping timeout: 272 seconds]
joshbuddy has quit [Quit: joshbuddy]
jfis has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
Soft has quit [Ping timeout: 264 seconds]
<The_8472> brimstone, if you have a server somewhere... tunnel.
Soft has joined #ipfs
grahamperrin has joined #ipfs
grahamperrin has left #ipfs ["Leaving"]
jhulten has quit [Ping timeout: 250 seconds]
Oatmeal has quit [Read error: Connection reset by peer]
pskosinski has quit [Read error: Connection reset by peer]
dignifiedquire has joined #ipfs
Oatmeal has joined #ipfs
patcon has quit [Ping timeout: 265 seconds]
<dignifiedquire> daviddias: npm is just waiting for ipfs to be ready :) https://twitter.com/othiym23/status/684477169154666496
Soft has quit [Quit: WeeChat 1.4-rc1]
pskosinski has joined #ipfs
ugjka has quit [Quit: Leaving]
jhulten has joined #ipfs
ugjka has joined #ipfs
jhulten has quit [Ping timeout: 250 seconds]
<ehd> huh
<ehd> npm is a beast
<M-davidar> ehd: if you're not careful it'll remove a limb
<ehd> really, it should come with safety instructinos
<ehd> *instructions
<patagonicus> Calling ipfs files stat for every file in this mirror is … kinda slow. I think I'll have to switch to ipfs files ls + parsing for "production" use.
ygrek has joined #ipfs
<ipfsbot> [go-ipfs] jbenet force-pushed v0.4.0-dev from 63a8e75 to 1c1f9c6: http://git.io/vRLhG
ygrek has quit [Ping timeout: 246 seconds]
pfraze has joined #ipfs
Senji has joined #ipfs
hoony has joined #ipfs
ygrek has joined #ipfs
hoony1 has joined #ipfs
chriscool has quit [Quit: Leaving.]
hoony has quit [Ping timeout: 255 seconds]
hoony1 is now known as hoony
chriscool has joined #ipfs
sivachandran has joined #ipfs
hartor has joined #ipfs
chriscool has quit [Read error: Connection reset by peer]
rendar has joined #ipfs
jhulten has joined #ipfs
<ipfsbot> [ipfs] jbenet pushed 2 new commits to master: http://git.io/vu7sa
<ipfsbot> ipfs/master 567e912 Eric Myhre: Explain relationship of remaining documents to papers repo....
<ipfsbot> ipfs/master a02e760 Juan Benet: Merge pull request #84 from heavenlyhash/patch-1...
jhulten has quit [Ping timeout: 260 seconds]
hoony has quit [Ping timeout: 264 seconds]
chriscool has joined #ipfs
ygrek has quit [Ping timeout: 245 seconds]
zorglub27 has joined #ipfs
the193rd has joined #ipfs
zorglub27 has quit [Quit: zorglub27]
ilyaigpetrov has joined #ipfs
pfraze has quit [Remote host closed the connection]
sivachandran has quit [Remote host closed the connection]
devbug has joined #ipfs
devbug has quit [Ping timeout: 240 seconds]
zorglub27 has joined #ipfs
dignifiedquire has quit [Quit: Connection closed for inactivity]
sivachandran has joined #ipfs
grahamperrin has joined #ipfs
grahamperrin has quit [Client Quit]
dignifiedquire has joined #ipfs
chriscool has quit [Ping timeout: 240 seconds]
voxelot has quit [Ping timeout: 260 seconds]
grahamperrin has joined #ipfs
sivachandran has quit [Remote host closed the connection]
grahamperrin has left #ipfs ["Leaving"]
prim1 has quit [Ping timeout: 276 seconds]
prim1 has joined #ipfs
zorglub27 has quit [Ping timeout: 256 seconds]
zorglub27 has joined #ipfs
hartor1 has joined #ipfs
hartor has quit [Ping timeout: 246 seconds]
hartor1 is now known as hartor
jhulten has joined #ipfs
hoony has joined #ipfs
jhulten has quit [Ping timeout: 264 seconds]
anon3459 has joined #ipfs
reit has quit [Ping timeout: 246 seconds]
<yangwao> who is going there http://arcticjs.club/
<yangwao> :D
mildred has joined #ipfs
Oatmeal has quit [Ping timeout: 260 seconds]
hoony has quit [Ping timeout: 265 seconds]
hashcore has joined #ipfs
Oatmeal has joined #ipfs
hashcore has quit [Ping timeout: 255 seconds]
Matoro has quit [Ping timeout: 256 seconds]
hashcore has joined #ipfs
jaboja has joined #ipfs
jaboja has quit [Ping timeout: 256 seconds]
<patagonicus> Has anyone put the 32C3 videos into IPFS yet?
Gaboose has joined #ipfs
m0ns00n has joined #ipfs
m0ns00n has quit [Quit: undefined]
m0ns00n has joined #ipfs
jfis has joined #ipfs
<dignifiedquire> patagonicus: I don't think so, but if you do please post the hashes to github.com/ipfs/archives
jhulten has joined #ipfs
voxelot has joined #ipfs
jhulten has quit [Ping timeout: 245 seconds]
Tv` has joined #ipfs
cSmith_ has joined #ipfs
Encrypt has joined #ipfs
mildred has quit [Ping timeout: 260 seconds]
kerozene has quit [Ping timeout: 260 seconds]
cemerick has joined #ipfs
voxelot has quit [Remote host closed the connection]
snormett has joined #ipfs
kerozene has joined #ipfs
mildred has joined #ipfs
Not_ has joined #ipfs
zz_r04r is now known as r04r
jfis has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
jfis has joined #ipfs
vijayee_ has joined #ipfs
snormett has quit [Quit: leaving]
SpX has joined #ipfs
pfraze has joined #ipfs
<dignifiedquire> whyrusleeping: also quite a few binaries fail to build
lempa has quit [Ping timeout: 250 seconds]
<patagonicus> dignifiedquire: Don't think I have the space right now to do so, was just gonna grab one or two videos.
<dignifiedquire> whyrusleeping: also ipfs-update and fs-repo-migrations don't build anything for me
<dignifiedquire> patagonicus: even two videos help :)
<patagonicus> Hmm. All the HD ones seem to be 80G (just MP4 though, no WebM). Maybe I can find some space on one of my servers for that, but I want to get my Alpine mirror up first. rsync'ing to /ipns/local is very slow, unfortunately.
mildred has quit [Ping timeout: 255 seconds]
jhulten has joined #ipfs
m0ns00n has quit [Quit: undefined]
jhulten has quit [Ping timeout: 272 seconds]
<ansuz> daviddias: pewpew
<daviddias> ansuz: check this gist, pasted the commands I ran plus links to the js code that does it in the registry-mirror https://gist.github.com/diasdavid/f42e557c6d898f6047b6
<ansuz> QmTjHX7dx3CKg4woQFoWKvcYjZ7mSc5Rmmn5D8VkgsmVft
<ansuz> (get)
<whyrusleeping> dignifiedquire: ipfs-update doesnt build anything for you?
<whyrusleeping> what happens?
<dignifiedquire> pasted log in github
<whyrusleeping> (also, its normal that a lot of the binaries don't build, i want that so we can see what we need to fix)
hashcore has quit [Quit: Leaving]
<dignifiedquire> (ok)
libman has joined #ipfs
chriscool has joined #ipfs
<whyrusleeping> (freedole)
<ansuz> (pewpew)
<lgierth> oh hey daviddias is registry-mirror still crunching??
<lgierth> that was one question mark to o much
<ansuz> ohai lgierth
<ansuz> are you planning to do that irc bot or should I start hacking on it?
<ansuz> (for caleb)
<lgierth> ansuz: yeah go for it
<lgierth> i don't have the focus for it
<ansuz> word
pfraze has quit [Remote host closed the connection]
<lgierth> i also hope the /mode #hyperboria -v thing goes somewhere
<lgierth> oh, discussion for another irc network :)
<whyrusleeping> ansuz: QmYeVNgi9tS5K4vKdDF38BbTa6wet26PAqPDqDzThyfHit
<ansuz> thx
<daviddias> lgierth: yes, but almost over
<daviddias> did you reset the node?
<daviddias> seems it is processing some very large modules (150Mb ++)
<patagonicus> Just noticed that cdn.media.ccc.de has an index which lists all files (including timestamp + size). All of their media seems to be just a bit more than 3TB. :D
<daviddias> lgierth have you rebooted the node or something? Almost feels it is kind in a deadlock state for some reason (it was working just fine today's morning)
cSmith_ has quit [Changing host]
cSmith_ has joined #ipfs
cSmith_ is now known as cSmith
<dignifiedquire> whyrusleeping: any thoughts on those logs?
<lgierth> daviddias: didn't touch it
<lgierth> it looks like it's still doing *something* though
<daviddias> still has some space available too, must have locked for some other reason then
<lgierth> well, no api requests though for the past 40 minutes
<daviddias> pinged whyrusleeping to take a peek at it
<daviddias> registry-mirror waits for ipfs to finish the write before trying to add more
<lgierth> and a rise in goroutines just a minute before it stopped doing api stuff
<dignifiedquire> lgierth: how is biham holding up? pinning is still running
<daviddias> dignifiedquire: pinning of what?
<lgierth> dignifiedquire: no idea at the moment :)
<dignifiedquire> 180gb stackexchange archives
<daviddias> when did that pinning started?
<dignifiedquire> 2 days ago, but it was stopped over night as my machine was shutdown, somaybe 28h or so in total now
<dignifiedquire> but it's veeery slow ipfs is uploading super slow from my machine :/ not even 10% of the capacity of my connection
pskosinski_ has joined #ipfs
pskosinski has quit [Read error: Connection reset by peer]
pfraze has joined #ipfs
chriscool has quit [Quit: Leaving.]
pfraze has joined #ipfs
zorglub27 has quit [Ping timeout: 245 seconds]
<daviddias> so, that means that registry-mirror on biham is not only a couple of orders of magnitude faster
<daviddias> but it was even fetching and verifying your stack exchange archive too?
<daviddias> sweet! :D
<daviddias> whyrusleeping check that ^^, o/ yeah
zorglub27 has joined #ipfs
<lgierth> patagonicus: i'm going to mirror that to biham
<daviddias> lgierth: biham seems to be locked now
<daviddias> do you know what is the issue?
<lgierth> no
<lgierth> but i'm gonna fetch that goroutine dump for why
yosafbridge` has quit [Ping timeout: 250 seconds]
<patagonicus> lgierth: Are you going to add that to IPFS yourself or will you just pin it once someone has done so?
chriscool has joined #ipfs
<lgierth> patagonicus: gonna mirror and add it to ipfs
<patagonicus> Nice.
<lgierth> it doesn't make much sense fetch to 3 TB on a normal dsl connection haha
<lgierth> or what have you got over there?
libman1 has joined #ipfs
libman has quit [Read error: Connection reset by peer]
gingeropolous has quit [Quit: Leaving]
<patagonicus> Got a server with 100MBit and my dormitory got fibre to the uni which is our uplink. Not sure how fast it can be as I'm sharing with a lot of other people, but I've had ~60MByte/s down in the middle of a night once. :D
<patagonicus> Upload is less of a problem as it's not used as much, but my problem is mostly that of storage. I could use some old drives, but that would mean losing some backups I have on there.
<whyrusleeping> lgierth: don't reboot biham without telling daviddias first
<The_8472> 60MB/s on a 100Mbit link? impressive
<The_8472> ah, silly me, dormitory and server being separate things
yosafbridge has joined #ipfs
<patagonicus> Yeah. It's very easy to max out the servers bandwidth making it unresponsive for anything else. :D
<dignifiedquire> daviddias: lgierth biham is still downloading refs for me
<lgierth> ok wow i get some steady 90MB/s from https://ftp.fau.de/cdn.media.ccc.de
<lgierth> on biham
<lgierth> that's a mirror run by a university in the same city as biham (nuremberg)
<lgierth> not even sure if the bottleneck is network or disk io lol
anticore has joined #ipfs
<patagonicus> lol
<Kubuxu> lgierth: I my case (60MiB/s) it is the HDD, if I /dev/null/ data I get upto 95MiB/s :P
<Kubuxu> HDD + cache
<Kubuxu> it dropped then when cache filled
<lgierth> yeah you're nearby too eh
<lgierth> .pl?
<lgierth> ok me -> supermarket
<lgierth> whyrusleeping: i'm putting the dumps in /root
chriscool has quit [Ping timeout: 246 seconds]
<Kubuxu> Yup.
<Kubuxu> From my friend's business internet. I don't have that good one in my flat :/.
hartor1 has joined #ipfs
fsl has joined #ipfs
hartor has quit [Ping timeout: 245 seconds]
hartor1 is now known as hartor
The_8472 has quit [Ping timeout: 240 seconds]
jhulten has joined #ipfs
<dignifiedquire> lgierth: niiiice
<dignifiedquire> biham is getting pretty used these days :D
fsl has quit [Quit: leaving]
jhulten has quit [Ping timeout: 246 seconds]
<ScoreUnder> I've got another query about IPFS 0.4. It seems like it leaks memory (it's currently using 1600MB, when it started around 400MB. I'd restarted it after a crash, which I can only assume must have been the OOM killer). Is this usage normal, and if not is there a way I can trace it back to the allocation like with valgrind or something? (I don't know Go enough to determine this kind of thing)
<ScoreUnder> curently up to 1750M
ygrek has joined #ipfs
libman1 has quit [Ping timeout: 260 seconds]
anon3459 has quit [Ping timeout: 264 seconds]
ilyaigpetrov has quit [Quit: Connection closed for inactivity]
NightRa has joined #ipfs
joshbuddy has joined #ipfs
The_8472 has joined #ipfs
zorglub27 has quit [Ping timeout: 260 seconds]
libman has joined #ipfs
<lgierth> ScoreUnder: if it runs out of memory on its own, it'll print a stacktrace
Soft- has joined #ipfs
<ScoreUnder> lgierth: even taking into account linux's overcommit and such?
<ScoreUnder> [1] 28003 killed ipfs -D daemon
<ScoreUnder> ipfs -D daemon 26966.50s user 4755.62s system 53% cpu 16:31:09.19 total
<ScoreUnder> ↑ how it died
prosody is now known as prsdy
<ScoreUnder> it's currently at 2158MB
<lgierth> yeah i've seen it eat lots of ram too
<lgierth> you might be seeing high numbers of goroutines too
<lgierth> check :5001/debug/pprof
<lgierth> the "full goroutine dump" or so is useful
<ScoreUnder> 0 block, 149061 goroutine, 2117 heap, 38 threadcreate
<ScoreUnder> I have a feeling the full goroutine stack dump is going to send a few GB to my browser
<lgierth> na not soo much
<lgierth> a 100 mb though or so
jhulten has joined #ipfs
<ScoreUnder> 64MB that's not so bad
prsdy is now known as Prosody
jhulten has quit [Ping timeout: 245 seconds]
<ScoreUnder> Would it mean much to you if I showed you this: https://0x0.st/oFk.png
<ScoreUnder> it looks like 22118 goroutines are inside readHandshake ther
pfraze has quit [Remote host closed the connection]
warner has joined #ipfs
<dignifiedquire> whyrusleeping: ping re distributions
joshbuddy has quit [Ping timeout: 256 seconds]
cemerick has quit [Ping timeout: 250 seconds]
joshbuddy has joined #ipfs
anticore has quit [Quit: bye]
jhulten has joined #ipfs
jaboja has joined #ipfs
cemerick has joined #ipfs
joshbuddy has quit [Quit: joshbuddy]
<Guest55618> you could amke your own physical mesh network to repalace the internet. what hardwar/software would you use?
Soft- is now known as Soft
<The_8472> physical as in wired?
<The_8472> or physical as in not-internet-overlay?
<Guest55618> physically wired
Guest55618 is now known as VegemiteToast
VegemiteToast is now known as VegemiteToast_
<VegemiteToast_> not running through the internet.
<The_8472> i'd get an excavator and then dig up all the gold needed to pay for the wiring
<VegemiteToast_> I heard it was bad to daisy chain switches.
<VegemiteToast_> and then?
<VegemiteToast_> It's hypothetical money is not an obstical
<VegemiteToast_> if every only pays for half the price to connect to their neighbor/peer it probably wouldn't cost that much
<VegemiteToast_> long distance it would
<ScoreUnder> VegemiteToast_: and I suppose human nature is not an obstacle either? I doubt it would ever happen because it would require lots of people locally to buy into it, and that requires everyone being a techie. Others will be happy with their client-server subscriber model internet access and I'm sure ISPs would do all they can to keep the mesh separate
<The_8472> digging fiber trenches to each of your neighbors + some box to route potentially gigabits of traffic is not that cheap
<Kubuxu> VegemiteToast_: dark fibers and FPGAs.
<VegemiteToast_> u really thing every peer wouldd need fiber?
<The_8472> and even if money is not an issue, you still need human labor. you could hire a whole nation to do the digging and wiring...
<VegemiteToast_> think*
<The_8472> well, who wants slow internet?
<Kubuxu> Not every but many, WiFI links have high latency.
<VegemiteToast_> 1 gigabit ethernet. with each peer server atleast to others.
<VegemiteToast_> serving*
<The_8472> well, he said "physically wired", which rules out wifi anyway
<VegemiteToast_> two*
<The_8472> everyone also need to pipe other people's data through their nodes, not just their own
<Kubuxu> And almost everywhere where new infrastructure fiber links are laid.
<VegemiteToast_> ye.
<The_8472> since it's a mesh it'll flow through many many hops
<VegemiteToast_> yip
<The_8472> and you would need some serious optical switching otherwise the optical-electronic conversion on hundreds and hundreds of hops will kill latency
<VegemiteToast_> but caching data on nodes would reduce hopage.
<The_8472> then you also need storage
<ScoreUnder> VegemiteToast_: you're thinking of a web replacement rather than an internet replacement then
<VegemiteToast_> would kill voip and live shit though
<VegemiteToast_> ye. I suppose
<The_8472> just think how a path from china to south america would look like
<VegemiteToast_> so u would have to have an exchange in every suburb just like ISPs?
<VegemiteToast_> to reduce hops
<ScoreUnder> then you could connect all the nearby exchanges together into one super-exchange... then again, and you've reinvented the ISP :^)
<Kubuxu> If you were to create global mesh, separate from internet, you still need long range high capacity links.
<The_8472> or the non-profit IXP at least
<The_8472> we have a bunch of those here in europe
<The_8472> but of course they can only accept large customers, i.e. ISPs, individuals would overwhelm them
<The_8472> i guess you could kinda have IXPs-in-small in some way... but that still requires digging lots of trenches
<Kubuxu> ScoreUnder: no, if you choose different networking design, cjdns for example.
<VegemiteToast_> are cjdns hops any faster?
<The_8472> Kubuxu, a mesh is nice locally. but how to you connect countries and continents?
<The_8472> or even cities for that matter
<VegemiteToast_> international and national fiber backbone is a must
<The_8472> meshes pretty much end at the city boundary
<VegemiteToast_> but locat mesh is probably do-able
<VegemiteToast_> local*
<VegemiteToast_> question is can u have a mix of the two. and how would u do it?
<The_8472> but given unlimited resources we could just move all humans into a single city!
<Kubuxu> The_8472: you use long range fibers and open market.
<The_8472> Kubuxu, you'd still need someone to build and run them. at best you can replace for-profit ISPs with non-profit infrastructure providers
<VegemiteToast_> If one city made a meshnet they could sell access to their mesh to the next city over.
<Kubuxu> Partially the same way current internet is laid out as on global scale internet looks like mesh.
<VegemiteToast_> it could be profitable
<The_8472> VegemiteToast_, nobody wants to do the accounting
<The_8472> accountants waste human life-hours and money
<VegemiteToast_> hehe
<Kubuxu> The_8472: noone says about non profit, cjdns will be including pay per packet in undefined future (when we figure it out).
<VegemiteToast_> or make installation of a nodes hardware, quid pro quo, If you server files for me. I'lll install it for free.
<Kubuxu> VegemiteToast_: Berlin has really nice local mesh if you are interested.
<The_8472> that's close to how peering and "I put my CDN cache into your server rack" deals work
libman has quit [Ping timeout: 260 seconds]
<VegemiteToast_> is it viable?
<The_8472> Kubuxu, in which case would paying for packets have some utility?
<VegemiteToast_> after the initial investment of wardware. the customer would serve files for the rest of his/her life.
<VegemiteToast_> in excahnge for access to the mesh.
<Kubuxu> you need money even if not for profit then for upgrades and maintenance.
<Kubuxu> also: non-profit != free
<The_8472> i know
joshbuddy has joined #ipfs
<VegemiteToast_> make people pay you to become a certified tech dude
<The_8472> but you can do that by paying for the hardware, not for the routing
<VegemiteToast_> then u have workers
mildred has joined #ipfs
<VegemiteToast_> hi i'm john and I'm meshnet certified worker
<VegemiteToast_> badge no. 1337
<VegemiteToast_> make them get recertified every year.
<VegemiteToast_> lol
<VegemiteToast_> sound like a pyramid scheme but its not
<Kubuxu> What if I move around the world and I just want to use network? Also PPP has nice benefit of limiting attack and stimulating infrastructure development.
<VegemiteToast_> ppp?
<Kubuxu> Pay per Packet
<ScoreUnder> (I was wondering what on earth point to point protocol had to do with it)
<Kubuxu> sorry for that
<VegemiteToast_> if u moved around the world u could be certified to set up your own node. or get some one to come out and set one up. or get payed to convince others to be a node.
<The_8472> Kubuxu, well, idk, paying for capacity sounds nicer psychologically. you just pay a flat fee per day/month/year
<The_8472> if you pay for traffic instead of capacity you worry about your usage
<VegemiteToast_> just make it.. if u serve more traffic ur price goes down.
<Kubuxu> The_8472: it is hard to pay per capacity as you don't know whom to pay
<The_8472> but solving the easy problem causes psychological anguish
<Kubuxu> in pay per packet you just pay ones you route through.
<Kubuxu> It isn't easy problem, it is impossible problem, now in the internet you have backbone to which the capacity is calculated, in mesh you don't have that.
<Kubuxu> also everything depends what is the price.
<VegemiteToast_> what I really want is a way to server files to the guy next door. but still be able to connect accross the world with reasonalbe ping
<ScoreUnder> VegemiteToast_: "Oy bill hop on my wifi a sec and I'll send it over"
<VegemiteToast_> without having to interact with people
jaboja has quit [Ping timeout: 250 seconds]
<VegemiteToast_> and changing networks all the time
<VegemiteToast_> I want a dual network situation
<Kubuxu> And that is what global mesh is about, you have planty of local links then you have hubbing and few intercity, -coutry, -continental.
<VegemiteToast_> is cjdns really up to the job atm?
<Kubuxu> VegemiteToast_: cjdns already offers that. You have global network but you can easily setup local mesh that will be part of that global network.
<VegemiteToast_> and u need a cjdns open-wrt router?
<VegemiteToast_> routers*
<Kubuxu> no you can use your PC, notebook anything that runs anything
<Kubuxu> for a mesh local you probably need a dedicated router (possibly RouterBoard) with antenna and so on.
<VegemiteToast_> how do u pysically wire to another cjdns computer and to the internet?
<The_8472> why do you want to physically wire something?
<The_8472> wlan is easier on short distances
<VegemiteToast_> speed
Encrypt has quit [Quit: Quitte]
<The_8472> directed antenna?
<VegemiteToast_> are they significantly faster?
<The_8472> so you can use <newest standard> between two houses
<Kubuxu> You need someone that already has connection to Hyperboria to connect to the rest Hyperboria.
<Kubuxu> only problem of WiFi is about 2-10ms of latency.
<VegemiteToast_> what like the best sorta range u can get out of the latest wireless routers?
<VegemiteToast_> is 50 meters doable?
<Kubuxu> With dedicated antenna?
<VegemiteToast_> dedicated.. as in directional?
<VegemiteToast_> i suppose ye
<Kubuxu> as big external.
zorglub27 has joined #ipfs
<VegemiteToast_> oh. ye. what are they like?
<VegemiteToast_> is it viable?
<Kubuxu> With clear line of sight up to 2-5km
<VegemiteToast_> huh. and the other end needs a big ass antenna too?
<The_8472> VegemiteToast_> oh. ye. what are they like? <- whatever tradeoff between size and distance you want to make
<VegemiteToast_> huh. and the other end needs a big ass antenna too?
<The_8472> if you want to connect two houses next to each other you obviously don't need a dish
libman has joined #ipfs
<VegemiteToast_> isn't it illeage to transmit that far without permission
<Kubuxu> They are about 70cm 20cm 20cm
<Kubuxu> no
<The_8472> depends on your jurisdiction and the power involved
<Kubuxu> range and size depends on type of antenna
hellertime has joined #ipfs
cemerick has quit [Ping timeout: 264 seconds]
chriscool has joined #ipfs
rombou has joined #ipfs
patcon has joined #ipfs
rombou has quit [Ping timeout: 264 seconds]
pfraze has joined #ipfs
chriscool has quit [Quit: Leaving.]
patcon has quit [Ping timeout: 256 seconds]
chriscool has joined #ipfs
simonv3 has joined #ipfs
NightRa has quit [Quit: Connection closed for inactivity]
joshbuddy has quit [Quit: joshbuddy]
zorglub27 has quit [Read error: Connection reset by peer]
kahiru has joined #ipfs
zorglub27 has joined #ipfs
anticore has joined #ipfs
chriscool has quit [Ping timeout: 265 seconds]
pfraze has quit [Remote host closed the connection]
joshbuddy has joined #ipfs
<whyrusleeping> dignifiedquire: sup
<dignifiedquire> whyrusleeping: still waiting for a fix to build migrations and update
<whyrusleeping> bork bork bork
<whyrusleeping> dignifiedquire: uhm... not sure why it failed
<whyrusleeping> what does 'make ipfs-update' yield?
<whyrusleeping> dignifiedquire: does dists/ipfs-update/tag_versions have anything in it/
<whyrusleeping> ?
libman has quit [Remote host closed the connection]
<dignifiedquire> v1.0.0
<dignifiedquire> in tag_versions
chriscool has joined #ipfs
<whyrusleeping> rm -rf releases/ipfs-update
<whyrusleeping> make ipfs-update
anticore has quit [Ping timeout: 276 seconds]
<whyrusleeping> it looks to me like the 'git versions' command thing failed to find anything
<whyrusleeping> for some reason
<dignifiedquire> same
<whyrusleeping> same as which?
<whyrusleeping> the first gist or the second?
<ipfsbot> [js-ipfs] diasdavid pushed 1 new commit to feat/bootstrap: http://git.io/vudhT
<ipfsbot> js-ipfs/feat/bootstrap ad95949 David Dias: add bootstrap list and bootstrap list core test
<dignifiedquire> same as the second
<dignifiedquire> it's still saying dirty
<whyrusleeping> as the second? then you didnt actually remove that directory...
<whyrusleeping> hrm...
pfraze has joined #ipfs
<whyrusleeping> alright
<whyrusleeping> so your version of grep isnt the same as mine
<dignifiedquire> whyrusleeping: :D who said bash scripts were cross platform
<whyrusleeping> dignifiedquire: alrgiht, so we need a way of doing this:
<whyrusleeping> given two files A and B
<whyrusleeping> remove all lines from A that are also in B
<dignifiedquire> I know how to do this in JavaScript ;)
<ScoreUnder> whyrusleeping: sorted files? comm does that
<lgierth> for l in $(cat A); do [ ! (grep -e $l B) ] && echo $l; done
<lgierth> something like that
<ScoreUnder> lgierth: no
<lgierth> yeah it's inefficient of course ;)
<ScoreUnder> (bad ifs-splitting, unwanted path expansion, iterating a file with "for", bad test command, unquoted variables, echo to echo a variable)
pskosinski has joined #ipfs
<ScoreUnder> s/inefficient/dangerous/ IMO
<lgierth> or array intersection
<lgierth> yeah i agree :)
<whyrusleeping> current impl is 'cat A | grep -f B -vE > out'
<lgierth> but god to know that you're someone i can ping for bash code review :P i'm working on something
<ScoreUnder> whyrusleeping: Why grep -vE rather than something like -vFx
pskosinski_ has quit [Read error: Connection reset by peer]
<ScoreUnder> -Fx meaning it will match exact lines in both files, rather than regexes and partial lines
<whyrusleeping> but apparently it doesnt work in Mac OSXland
<whyrusleeping> because its the set of flags that mean 'filter by the given regex'
<whyrusleeping> mm, probably better. i dont memorize grep flags
<ScoreUnder> as for comm, it takes sorted inputs, so: comm -23 <(sort -u -- A) <(sort -u -- B)
<ScoreUnder> unless you can output the files sorted in which case you can just do "comm -23 -- A B"
<ScoreUnder> not sure if the sorting is something you need or something you need to avoid
<whyrusleeping> ScoreUnder: that first bashism looking one should work fine in a makefile?
<ipfsbot> [js-ipfs] diasdavid pushed 1 new commit to feat/bootstrap: http://git.io/vuFf8
<ipfsbot> js-ipfs/feat/bootstrap 52b89b0 David Dias: add bootstrap add and rm and bootstrap add and rm tests
<whyrusleeping> dignifiedquire: do you have 'comm' on osx?
zorglub27 has quit [Remote host closed the connection]
<ScoreUnder> whyrusleeping: I don't have it on my FreeBSD machine but I do have -F, -f and -x on grep
<dignifiedquire> whyrusleeping:
<dignifiedquire> comm version
<dignifiedquire> usage: comm [-123i] file1 file2
<whyrusleeping> cool
<whyrusleeping> ScoreUnder: hrm...
<whyrusleeping> dignifiedquire: does your grep have -v and -E ?
<ScoreUnder> whyrusleeping: as for the bashisms working in a makefile, depends on the shell I guess. You can always call bash explicitly
<dignifiedquire> ❯ grep -v
<dignifiedquire> usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
<dignifiedquire> [-e pattern] [-f file] [--binary-files=value] [--color=when]
<dignifiedquire> [--context[=num]] [--directories=action] [--label] [--line-buffered]
<dignifiedquire> [--null] [pattern] [file ...]
<ScoreUnder> whyrusleeping: i.e. bash -c 'comm -23 <(sort -u -- "$1") <(sort -u -- "$2")' bash A B
<whyrusleeping> ScoreUnder: true
zorglub27 has joined #ipfs
<ScoreUnder> then again it's entirely possible to do this (slowly) without bashisms or external programs
<tperson> What about good old diff?
<whyrusleeping> dignifiedquire: can you replace the second line of the 'versions' rule in dists/ipfs-update/Makefile with:
<whyrusleeping> comm -23 <(sort -u -- tag_versions) <(sort -u -- filtered_versions) > versions
<ScoreUnder> tperson: would come out as: diff <(sort -u -- A) <(sort -u -- B) | grep "^<" | cut -c3-
* whyrusleeping sighs
<whyrusleeping> okay i'll boot into osx and try things
<tperson> lol
<dignifiedquire> whyrusleeping: checking
<dignifiedquire> whyrusleeping: pretty much the same
<whyrusleeping> alright, osx time... bleech
<whyrusleeping> after i finish listening to this song
disgusting_wall has joined #ipfs
<ScoreUnder> Since I'm sure you'd be thrilled to see how nasty it looks, pure sh version: remove_lines() { file1=$1 file2=$2; set --; while IFS= read -r line; do set -- "$@" "$line"; done <"$file2"; while IFS= read -r line; do found=false; for line_b do [ "$line_b" = "$line" ] && found=true && break; done; $found || printf %s\\n "$line"; done <"$file1"; }; remove_lines A B
<whyrusleeping> dignifiedquire: I found a workaround for you: https://wiki.archlinux.org/index.php/Installation_guide
<ScoreUnder> :^)
<dignifiedquire> whyrusleeping: :D:D
<dignifiedquire> you could rewrite it in go, you need go to be installed to build it anyway
<whyrusleeping> i'm so lazy
<dignifiedquire> or I convert it to 2000 lines of JavaScript :D
<ScoreUnder> lol
<whyrusleeping> you would need jquery for that for sure
<ansuz> ^
<The_8472> don't forget to transpile it so it runs on older engines!
<tperson> Is this a shell script you are trying to fix?
<tperson> asm.js
<ScoreUnder> tperson: looks like a makefile
jgraef has joined #ipfs
<tperson> dignifiedquire: Is this the make file for the webui?
<whyrusleeping> lol, ansuz always showing up for a js joke
<whyrusleeping> tperson: its a bashfile Makescript
<dignifiedquire> tperson: no distributions
<whyrusleeping> with some javascraps mingled around
<ansuz> turned off highlights for 'ansuz'
<ansuz> turned them on for 'js'
<ansuz> probably not actually
<ansuz> I can never tell when I'm serious
<tperson> This the app-release script
<tperson> ?
pfraze has quit [Remote host closed the connection]
<dignifiedquire> tperson: no ipfs-update
rendar has quit [Ping timeout: 276 seconds]
rendar has joined #ipfs
hellertime has quit [Quit: Leaving.]
<whyrusleeping> dignifiedquire: i pushed new code
<whyrusleeping> appears to work for me
<whyrusleeping> but my internet is spotty and cant quickly clone the full repo to test out the builds
<dignifiedquire> whyrusleeping: I'm pulling code
<dignifiedquire> whyrusleeping: it's building things :)
<tperson> What is wrong with grep -f B -vE ?
<whyrusleeping> doubtful
<whyrusleeping> :D
<ScoreUnder> my marmalade rn https://0x0.st/oCQ.jpg
<whyrusleeping> tperson: doesnt work on OSX for some reason
<tperson> Works on my laptop :D
<whyrusleeping> ScoreUnder: that looks like tasty marmalade
<ipfsbot> [js-ipfs] diasdavid pushed 3 new commits to feat/bootstrap: http://git.io/vuFm7
<ipfsbot> js-ipfs/feat/bootstrap 292e75f David Dias: refresh repo for each battery of tests
<ipfsbot> js-ipfs/feat/bootstrap f404a8f David Dias: mend
<ipfsbot> js-ipfs/feat/bootstrap af42da7 David Dias: mend
<ScoreUnder> I hope it is. Bought the value and non-value for comparison. Because my life is that interesting :D
<whyrusleeping> tperson: odd...
<whyrusleeping> even if B is empty?
<tperson> Nope, not that lol
<whyrusleeping> haha
<whyrusleeping> i've purchased more licorice than i'd care to admit to
<whyrusleeping> to compare
<whyrusleeping> i hate how much latency i've got right now...
<dignifiedquire> whyrusleeping: alright I got a folder full of stuff for ipfs-update, need the same fixes for fs-repo-migrations I guess
<whyrusleeping> they should already be made
* whyrusleeping is going to sleep
<tperson> OSX should fix there shit, that is really annoying that grep fails if the file supplied by -f is empty
<jgraef> I'm creating a lot of DAG nodes and my network is clogged. So I guess that outbound traffic comes from inserting keys into the DHT?
<The_8472> is the bandwidth saturated or are you seeing dropped packets?
<The_8472> packet drops could indicate conntrack table saturation. turning off conntrack or reducing timeouts in the nat device could help. assuming you have control over it
pfraze has joined #ipfs
<jgraef> As far as I can tell the bandwidth is not saturated. I'm not behind a NAT (I'm in the university's VPN).
<The_8472> well, still could be a stateful firewall
robcat has joined #ipfs
robcat has quit [Client Quit]
<dignifiedquire> whyrusleeping: okay cool :)
janosch2 has joined #ipfs
janosch2 is now known as jgraef2
<jgraef2> I disconnected the VPN and now everything is fine again :)
jgraef has quit [Ping timeout: 256 seconds]
zorglub27 has quit [Quit: zorglub27]
jgraef2 is now known as jgraef
<dignifiedquire> whyrusleeping: got another bug :P dist.json is empty for fs-repo-migrations
mildred has quit [Ping timeout: 256 seconds]
hartor1 has joined #ipfs
hartor has quit [Ping timeout: 264 seconds]
hartor1 is now known as hartor
jgraef has quit [Ping timeout: 245 seconds]
<achin> if a gateway returns "context deadline exceeded" does it keep the hash in its wantlist?
jgraef has joined #ipfs