stebalien changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.4.18 and js-ipfs 0.33 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS, the InterPlanetary FileSystem: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of Con
pkx3 has quit [Read error: Connection reset by peer]
BeerHall has joined #ipfs
pkx3 has joined #ipfs
Papa_Alpaka has quit [Remote host closed the connection]
pkx3 has quit [Read error: Connection reset by peer]
pkx3 has joined #ipfs
dimitarvp has joined #ipfs
ygrek has joined #ipfs
pkx3 has quit [Read error: Connection reset by peer]
pvh has quit [Quit: Connection closed for inactivity]
pkx3 has joined #ipfs
pkx3 has quit [Read error: Connection reset by peer]
sammacbeth has joined #ipfs
pkx3 has joined #ipfs
thomasanderson has joined #ipfs
sammacbeth has quit [Ping timeout: 250 seconds]
pkx3 has quit [Read error: Connection reset by peer]
thomasanderson has quit [Ping timeout: 252 seconds]
pkx3 has joined #ipfs
pkx3 has quit [Read error: Connection reset by peer]
sammacbeth has joined #ipfs
pkx3 has joined #ipfs
pkx3 has quit [Read error: Connection reset by peer]
pkx3 has joined #ipfs
cratliff has quit [Ping timeout: 244 seconds]
pkx3 has quit [Read error: Connection reset by peer]
vmx has quit [Quit: Leaving]
pkx3 has joined #ipfs
pkx3 has quit [Read error: Connection reset by peer]
sammacbeth has quit [Ping timeout: 250 seconds]
thomasanderson has joined #ipfs
thomasanderson has quit [Ping timeout: 264 seconds]
pkx3 has joined #ipfs
pkx3 has quit [Read error: Connection reset by peer]
aarshkshah1992 has joined #ipfs
cratliff has joined #ipfs
pkx3 has joined #ipfs
pkx3 has quit [Read error: Connection reset by peer]
pkx3 has joined #ipfs
aarshkshah1992 has quit [Remote host closed the connection]
aarshkshah1992 has joined #ipfs
pkx3 has quit [Read error: Connection reset by peer]
pkx3 has joined #ipfs
pkx3 has quit [Read error: Connection reset by peer]
aarshkshah1992 has quit [Remote host closed the connection]
aarshkshah1992 has joined #ipfs
aarshkshah1992 has quit [Remote host closed the connection]
aarshkshah1992 has joined #ipfs
aarshkshah1992 has quit [Remote host closed the connection]
kapil____ has joined #ipfs
cratliff has quit [Ping timeout: 250 seconds]
Chaos_Zero has quit [Ping timeout: 272 seconds]
<jamiew>
shardratic[m]: this channel is bridged to all 3 of Matrix, IRC, and Discord, so faux-bots pass messages along
thomasanderson has joined #ipfs
clemo has quit [Ping timeout: 250 seconds]
aarshkshah1992 has joined #ipfs
thomasanderson has quit [Ping timeout: 252 seconds]
ygrek has quit [Ping timeout: 240 seconds]
reit has joined #ipfs
cratliff has joined #ipfs
thomasanderson has joined #ipfs
pep7_ has quit [Ping timeout: 250 seconds]
thomasanderson has quit [Ping timeout: 252 seconds]
thomasanderson has quit [Ping timeout: 264 seconds]
thomasanderson has joined #ipfs
_whitelogger has joined #ipfs
thomasanderson has quit [Remote host closed the connection]
aarshkshah1992 has quit [Remote host closed the connection]
aarshkshah1992 has joined #ipfs
BeerHall has quit [Quit: BeerHall]
VylGryph has joined #ipfs
Vyl has quit [Ping timeout: 245 seconds]
aarshkshah1992 has quit [Ping timeout: 245 seconds]
graffen has quit [Ping timeout: 264 seconds]
thomasanderson has joined #ipfs
graffen has joined #ipfs
zeden has quit [Quit: WeeChat 2.2]
achitora has joined #ipfs
thomasanderson has quit [Remote host closed the connection]
spinza has quit [Quit: Coyote finally caught up with me...]
KG5KEY has quit [Ping timeout: 240 seconds]
spinza has joined #ipfs
<Shardratic4328[m>
Regarding blockchain data
<Shardratic4328[m>
IPFS itself is a data store
<Shardratic4328[m>
We index all decentralized data, including blockchain data and can include decentralized databases like Bluzelle and IPFS
<Shardratic4328[m>
*edit:* ~~We index all decentralized data, including blockchain data and can include decentralized databases like Bluzelle and IPFS~~ -> We index all decentralized data, blockchain data and can include decentralized databases like Bluzelle and IPFS
lassulus_ has joined #ipfs
lassulus has quit [Ping timeout: 268 seconds]
lassulus_ is now known as lassulus
sammacbeth has joined #ipfs
kapil____ has quit [Quit: Connection closed for inactivity]
aarshkshah1992 has joined #ipfs
sammacbeth has quit [Quit: Ping timeout (120 seconds)]
lidel` has joined #ipfs
lidel has quit [Ping timeout: 240 seconds]
lidel` is now known as lidel
thomasanderson has joined #ipfs
achitora has quit [Quit: Залишаю]
Thominus has joined #ipfs
rhalff has quit [Remote host closed the connection]
rhalff has joined #ipfs
KG5KEY has joined #ipfs
arthuredelstein has quit [Ping timeout: 250 seconds]
arthuredelstein has joined #ipfs
KG5KEY has quit [Remote host closed the connection]
stoopkid has quit [Quit: Connection closed for inactivity]
thomasanderson has quit [Ping timeout: 264 seconds]
graphene has quit [Remote host closed the connection]
graphene has joined #ipfs
thomasanderson has joined #ipfs
thomasanderson has quit [Remote host closed the connection]
cratliff has quit [Ping timeout: 240 seconds]
rhalff has quit [Ping timeout: 268 seconds]
thomasanderson has joined #ipfs
sammacbeth has joined #ipfs
thomasanderson has quit [Ping timeout: 250 seconds]
Pulse2496 has joined #ipfs
anacrolix has joined #ipfs
anacrolix has quit [Remote host closed the connection]
thomasanderson has joined #ipfs
kesenai has joined #ipfs
anacrolix has joined #ipfs
thomasanderson has quit [Ping timeout: 252 seconds]
anacrolix has quit []
kesenai_ has joined #ipfs
kesenai has quit [Ping timeout: 250 seconds]
inetic has joined #ipfs
sourkraut has joined #ipfs
<sourkraut>
hello! how active is this chat, generally? wheres a good place to keep up with people working on ipfs-related projects or exchange ideas about my own?
sourkraut has left #ipfs [#ipfs]
graphene has quit [Remote host closed the connection]
graphene has joined #ipfs
thomasanderson has joined #ipfs
thomasanderson has quit [Ping timeout: 260 seconds]
kesenai_ has quit [Ping timeout: 250 seconds]
inetic has quit [Ping timeout: 252 seconds]
ylp has joined #ipfs
kesenai has joined #ipfs
kesenai has quit [Remote host closed the connection]
petroav has joined #ipfs
kesenai has joined #ipfs
sourkraut has joined #ipfs
sourkraut has left #ipfs [#ipfs]
sourkraut has joined #ipfs
kesenai_ has joined #ipfs
thomasanderson has joined #ipfs
kesenai has quit [Ping timeout: 250 seconds]
thomasanderson has quit [Ping timeout: 250 seconds]
sourkraut has quit [Quit: leaving]
florianH has joined #ipfs
roidelapluie29 has joined #ipfs
roidelapluie29 has quit [Remote host closed the connection]
fireglow has quit [Remote host closed the connection]
jamesaxl has joined #ipfs
aarshkshah1992 has quit [Quit: Leaving...]
fireglow has joined #ipfs
vyzo has quit [Quit: Leaving.]
vyzo has joined #ipfs
petroav has quit [Ping timeout: 272 seconds]
rendar has joined #ipfs
thomasanderson has joined #ipfs
tguillem13 has joined #ipfs
tguillem13 has quit [Ping timeout: 246 seconds]
thomasanderson has quit [Ping timeout: 250 seconds]
aLeSD has quit [Ping timeout: 250 seconds]
fireglow has quit [Quit: Gnothi seauton; Veritas vos liberabit]
aLeSD has joined #ipfs
fireglow has joined #ipfs
Pulse2496 has quit [Quit: ⠟⠥⠊⠞]
kapil____ has joined #ipfs
<shoku[m]1>
yup
thomasanderson has joined #ipfs
thomasanderson has quit [Ping timeout: 264 seconds]
graphene has quit [Remote host closed the connection]
graphene has joined #ipfs
kesenai_ has quit [Remote host closed the connection]
Caterpillar2 has quit [Quit: You were not made to live as brutes, but to follow virtue and knowledge.]
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
Caterpillar has joined #ipfs
thomasanderson has joined #ipfs
saki has joined #ipfs
saki has quit [Remote host closed the connection]
thomasanderson has quit [Ping timeout: 250 seconds]
<maxwebfree[m]>
Can anyone suggest a solution for the IPFS/Filecoin demand side problem? OK, there are will be thousands of nodes with shared storage provided. But who will be the ones who will be paying for that?
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
woss_io has joined #ipfs
spinza has quit [Quit: Coyote finally caught up with me...]
unawaking has quit [Read error: Connection reset by peer]
thomasanderson has joined #ipfs
clemo has joined #ipfs
thomasanderson has quit [Ping timeout: 252 seconds]
<kidseekingthrill>
But i am struggeling now to set the correct dns txt record for my normal domain blog.we-make-your.website
<nofwayy[m]1>
Shardratic#4328:
<kidseekingthrill>
i am adding a txtrecord: hostname: blog value: dnslink=/ipns/QmcLLhttvq61XdBoQ9HtFSBarhbZmdH6wzdB32KQJnFriZ
<kidseekingthrill>
when i go to blog.we-make-your.website i am redirected to my normal server and not the ipfs node
<kidseekingthrill>
do i have to change the server block in ngnix? the dns txtrecord should not redirect to my ip :thinkpepe:
spencer has quit [Ping timeout: 260 seconds]
thomasanderson has joined #ipfs
<jamiedubs[m]1>
maxwebfree: I do think that is the larger of the problems (demand for the network). For comparison, Sia and Storj are flush with storage but barely any demand for network storage and traffic
spencer has joined #ipfs
sammacbeth has quit [Ping timeout: 250 seconds]
thomasanderson has quit [Ping timeout: 250 seconds]
<ylp>
kidseekingthrill: links to other pages doesn't work. I think they are resolved relative to the hash of the page and not the hash of the root folder
aLeSD has quit [Read error: Connection reset by peer]
aLeSD has joined #ipfs
aLeSD has quit [Max SendQ exceeded]
aLeSD has joined #ipfs
aLeSD has quit [Max SendQ exceeded]
aLeSD has joined #ipfs
aLeSD has quit [Max SendQ exceeded]
aLeSD has joined #ipfs
aLeSD has quit [Remote host closed the connection]
aLeSD has joined #ipfs
aLeSD has quit [Max SendQ exceeded]
spencer has quit [Remote host closed the connection]
thomasanderson has quit [Ping timeout: 250 seconds]
Encrypt has joined #ipfs
Minimons has joined #ipfs
<Obo[m]1>
Does anybody know if there's a way to keep track of how many times each hash is requested from a particular node?
spencer has quit [Remote host closed the connection]
spencer has joined #ipfs
jamesaxl has quit [Quit: WeeChat 2.2]
chrysn has joined #ipfs
fleeky has quit [Quit: Ex-Chat]
fleeky has joined #ipfs
<postables[m]>
Obo depends in what context. One way would be to continuously watch your dht logs to see if you're node is asked if they have blocks that is needed
<postables[m]>
however if you're talking about whether or not you're node is being called via API to download a file, I'm not sure if that kind of stuff is logged
ShalokShalom has joined #ipfs
<ShalokShalom>
is there any kind of peer to peer technique build in the current browsers?
spencer has quit [Remote host closed the connection]
dscdvvd has joined #ipfs
franck has joined #ipfs
thomasanderson has joined #ipfs
dfdsv has quit [Ping timeout: 240 seconds]
spencer has joined #ipfs
thomasanderson has quit [Ping timeout: 250 seconds]
graphene has quit [Remote host closed the connection]
graphene has joined #ipfs
dethos has joined #ipfs
dethos has quit [Client Quit]
spencer has quit [Remote host closed the connection]
Thominus has quit [Remote host closed the connection]
Jesin has quit [Quit: Leaving]
Jesin has joined #ipfs
franck has quit [Ping timeout: 256 seconds]
ygrek has quit [Ping timeout: 250 seconds]
<bitspill>
Hey, I've been doing some small tests with ipfs and transfers seem to be taking excessively long and consuming more bw than necessary to pin ~1.5GB with times varying from 25 minutes best case to worst case 1h40m for example QmPpfVepSxfkgSHnKSNV2NbYCkqcc9493K6d4ybdgSfEqd took 55m to pin but there are 5 different providers of it
<bitspill>
I've seena few github issues referencing issues in bitswap but they seem fairly old
<bitspill>
is this still a problem and if so is there work being done on that front or an ETA?
<bitspill>
The goal is to make available ~10,000 datasets totaling ~32TB via ipfs but so far performance seems lackluster
graphene has quit [Remote host closed the connection]
xcm has quit [Remote host closed the connection]
graphene has joined #ipfs
xcm has joined #ipfs
plexigras has quit [Ping timeout: 250 seconds]
thomasanderson has joined #ipfs
cratliff has quit [Ping timeout: 250 seconds]
thomasanderson has quit [Ping timeout: 264 seconds]
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
spencer has joined #ipfs
graphene has quit [Remote host closed the connection]
graphene has joined #ipfs
spencer has quit [Ping timeout: 240 seconds]
Encrypt has quit [Quit: Quit]
<postables[m]>
bitspill: what kind of network connectivity (fibre, dsl, etc..) and bandwidth do your nodes have?
<bitspill>
postables[m]: 2 datacenter 1gb/s-1gb/s, a residential 100mb/s-10mb/s and 2 commercial 80mb/s-60mb/s
tacyarg[m] has joined #ipfs
<postables[m]>
are your residential hosts the providers?
<postables[m]>
*edit:* ~~are your residential hosts the providers?~~ -> are your residential+commercial hosts the providers?
<bitspill>
the source #1 node is a server at caltech, we've accessed it via gateway.ipfs.io so that should also be a provider and from there timing how it downloads to various commercial/residential addresses
<bitspill>
the timings I've mentioned are coming from another datacenter attempting to pin
<bitspill>
the commercial/residential are in the same ballpark
clemo has quit [Ping timeout: 246 seconds]
thomasanderson has joined #ipfs
<postables[m]>
ah okay, you might be reaching some limitations with bitswap, how many nodes does QmPpfVepSxfkgSHnKSNV2NbYCkqcc9493K6d4ybdgSfEqd have? I'm attempting to pin it from my datacenter node as well right now, currently at:
<postables[m]>
`fetched/Processed 6293 nodes`
<postables[m]>
```
<postables[m]>
real 0m8.417s
<postables[m]>
```
<postables[m]>
is how long it took my node
<bitspill>
it just finish downloading in 8s?
vyzo has quit [Quit: Leaving.]
<bitspill>
what version you running, and what region is that node?
<bitspill>
interesting
<postables[m]>
`ipfs version 0.4.18`
<postables[m]>
it's located in british columbia
vyzo has joined #ipfs
<bitspill>
how can I find how many nodes it comprises? I saw it somewhere over 15,000
<bitspill>
right now it should be pinned in CAlifornia and Germany both datacenter nodes at least plus a commercial in sandiego
<bitspill>
how does `QmXPs8GH5ctLHEJTLzeN1RJVgbyCdLp7uSVKespnTtBim8` go for you?
thomasanderson has quit [Ping timeout: 250 seconds]
<bitspill>
I think that's only provided at our origin in Caltech and the ipfs gateway node
<bitspill>
then I'll try it on our 2nd dc node
<postables[m]>
ok well until i wipe this environment (sometime end of this month) `QmPpfVepSxfkgSHnKSNV2NbYCkqcc9493K6d4ybdgSfEqd` will be provided by my nodes
<bitspill>
QmXP is a little over 4GB
<postables[m]>
ok let me try `QmXPs8GH5ctLHEJTLzeN1RJVgbyCdLp7uSVKespnTtBim8`
<bitspill>
also, do you have any other configuration settings tuned I'm just using `ipfs init -profile=server`
<postables[m]>
stuck at `Fetched/Processed 17 nodes` for `QmXP`
<postables[m]>
im using server profile, along with badger datastore
<postables[m]>
my nodes have been online for quite sometime, each has about ~1200 peers so idk if that has something to do with it
<postables[m]>
yea no dice on `QmXP` hanging at 17 nodes
<bitspill>
interesting
<bitspill>
lemme get a peer count quick
<bitspill>
`ipfs swarm peers | wc -l` -> 465
<bitspill>
Is there a command I can run to determine the total node count of `QmXP`?
<postables[m]>
actually no i also have bloom filter, and hash on read configured
<postables[m]>
no idea, I guess `ipfs object stat` but it doesn't have anything specificalyl related to node count, just `blocks` or something
<postables[m]>
but yea you'll want to be careful with hash on read, this was the first thing i've pinned with hash on read that produced significant load, looks like im gonna have to give these nodes some more cpu power
<bitspill>
oh wait I'm getting spammed with `ERROR engine: open /services/tomography/.ipfs/datastore/3428412.ldb: too many open files engine.go:258` in the background
<postables[m]>
you're just using flatfs datastore right?
<bitspill>
yes, whatever the default is
<bitspill>
the data has all been pinned with `--no-copy` though
<bitspill>
so it uses the original files rather than bringing them internally
<bitspill>
ipfs repo stat
<bitspill>
NumObjects: 16985830
<bitspill>
RepoSize: 52783921668
<bitspill>
RepoPath: /services/tomography/.ipfs
<bitspill>
StorageMax: 10000000000
<bitspill>
Version: fs-repo@7
mazeto has joined #ipfs
<postables[m]>
i've never used the --no-copy` stuff before so couldn't say anything about that. but I might try using badger
<postables[m]>
if you've got a lot of CPU power at hand i might also try enabling hash on read in the config
<bitspill>
ok, I'll look into it. /services/tomography/data contains 32TB which we're trying to pin so thus the use of --no-copy
<postables[m]>
trying to pin `QmXPs8GH5ctLHEJTLzeN1RJVgbyCdLp7uSVKespnTtBim8` now, starting to be processed
<postables[m]>
you should still be able to use `--no-copy` with badger, but IIRC badger is better designed for handling large files the way you are
chjk6x has quit [Ping timeout: 240 seconds]
<bitspill>
`/services/tomography/.ipfs$ ls datastore | wc -l` -> 9037
<bitspill>
seems the flat file store makes a lot of ldb files, I'll look at badger now
woss_io has quit [Ping timeout: 246 seconds]
thomasanderson has joined #ipfs
<postables[m]>
node appears to have stalled out at `Fetched/Processed 11157 nodes` for `QmXP`
<postables[m]>
*edit:* ~~node appears to have stalled out at `Fetched/Processed 11157 nodes` for `QmXP`~~ -> ipfs appears to have stalled out at `Fetched/Processed 11157 nodes` for `QmXP`
xcm has quit [Remote host closed the connection]
<bitspill>
so far still just going crazy with unable to accept tcp4 connection or read datastore/x.ldb too many open files
<bitspill>
I feel like badger is the right track for me to pursue now
xcm has joined #ipfs
<bitspill>
is there a page that says how to enable it somewhere? github.com/ipfs/go-ds-badger seems to be the source but not seeing what config I need to change to activate it
thomasanderson has quit [Ping timeout: 252 seconds]
mazeto has quit [Read error: Connection reset by peer]
<bitspill>
ok, think I found it `ipfs config profile apply badgerds` and to use ipfs-ds-convert
<postables[m]>
yup!
<postables[m]>
you can also do `ipfs init --profile=server,badger`
<postables[m]>
*edit:* ~~you can also do `ipfs init --profile=server,badger`~~ -> you can also do `ipfs init --profile=server,badger` if you're starting your node for the first time
sander has quit [Ping timeout: 250 seconds]
sander has joined #ipfs
sammacbeth has quit [Ping timeout: 244 seconds]
<bitspill>
wonder how long converting a flatfs repo `NumObjects: 16985830 RepoSize: 52783921668` to badger will take
<bitspill>
gonna try on a test node first before poking the beast though
sammacbeth has joined #ipfs
ygrek has joined #ipfs
graphene has quit [Remote host closed the connection]
sammacbeth has quit [Quit: Ping timeout (120 seconds)]
graphene has joined #ipfs
sammacbeth has joined #ipfs
slaweq has joined #ipfs
slaweq has quit [Remote host closed the connection]