stebalien changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.4.18 and js-ipfs 0.33 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS, the InterPlanetary FileSystem: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of Con
b3rsf0_ has quit [Read error: Connection reset by peer]
mischat has joined #ipfs
ctOS has joined #ipfs
thomasanderson has joined #ipfs
plexigras has quit [Ping timeout: 240 seconds]
MCFX2_ has joined #ipfs
aarshkshah1992 has joined #ipfs
<ctOS>
Quick question on GC in ipfs-go. It removes everything every time it runs? Other than pinned content, that is. So as soon as the storage cache reaches 90% of 10Gb it clears everything?
MCFX2 has quit [Ping timeout: 246 seconds]
thomasanderson has quit [Remote host closed the connection]
<postables[m]>
Your GC process I believe should also run periodically (not 100% sure) but in theory yes once your node has reached X% of whatever the configured upper limit is 10GB by default that's what should happen
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
<ctOS>
It seems to run periodically (every hour by default) put only triggers once it exceeds 90% of max storage; at which point it removes everything except what is pinned. This seems like a weird design for a cache.
thomasanderson has joined #ipfs
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
thomasanderson has quit [Remote host closed the connection]
iczero has quit [Quit: rip]
iczero has joined #ipfs
<postables[m]>
i mean it makes sense to me. If you've got a max limit and you're caching stuff but haven't reached that limit, it seems logical to to strat running as well once you hit a max storage
<postables[m]>
*edit:* ~~i mean it makes sense to me. If you've got a max limit and you're caching stuff but haven't reached that limit, it seems logical to to strat running as well once you hit a max storage~~ -> i mean it makes sense to me. If you've got a max limit and you're caching stuff but haven't reached that limit, it seems logical to to strat running as well once you hit a max storage limit or within the max storage limit
sammacbeth has joined #ipfs
<LHommedesCitrons>
whats GC
<postables[m]>
garbage collection
<LHommedesCitrons>
oh
<ToxicFrog>
postables[m]: to me it would make more sense to clean out, say, the 25% least used/oldest accessed stuff
<LHommedesCitrons>
oh so go-ipfs does caching?
<ToxicFrog>
Otherwise if there's, say, 1GB of really commonly accessed but not pinned stuff, it'll get blown away and have to be re-fetched every time the cache fills up with cruft.
<postables[m]>
@ToxicFrog: That is definitely possible, but that requires a lot of overhead to do. I would imagine in the future IPFS would do that
<postables[m]>
L'Homme des Cirtons: pretty much
<LHommedesCitrons>
ah
<ToxicFrog>
postables[m]: as in, right now it doesn't keep any track of access patterns/times that would allow it to invalidate parts of the cache more intelligently?
<postables[m]>
@ToxicFrog: right now i believe that is how it works
sammacbeth has quit [Ping timeout: 246 seconds]
<postables[m]>
pretty sure its literally just "is this CID pinned? No? blow it up" lol
* ToxicFrog
nods
<ToxicFrog>
There's lots of prior art in the Linux block cache and the ZFS ARC to crib from once the feature does get implemented
sammacbeth has joined #ipfs
<postables[m]>
i mean IPFS is frigging modular as hell, you could probably even introduce that as a go-ipfs plugin
<postables[m]>
although i think the biggest blocker, is i don't believe the datastore keeps track of access times
<postables[m]>
superficially i guess you could use like `atime` or w.e.
clemo has quit [Ping timeout: 245 seconds]
daMaestro has quit [Quit: Leaving]
appa_ has joined #ipfs
_whitelogger has joined #ipfs
kapil____ has quit [Quit: Connection closed for inactivity]
mischat has quit [Ping timeout: 250 seconds]
smc62[m] has joined #ipfs
cwahlers has quit [Ping timeout: 252 seconds]
cwahlers has joined #ipfs
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
Imaginatrix has joined #ipfs
user_51 has quit [Ping timeout: 240 seconds]
JadziaDax has joined #ipfs
JadziaDax has left #ipfs [#ipfs]
user_51 has joined #ipfs
mischat has joined #ipfs
kapil____ has joined #ipfs
spinza has quit [Quit: Coyote finally caught up with me...]
thomasanderson has joined #ipfs
spinza has joined #ipfs
mischat has quit [Ping timeout: 252 seconds]
zzach has quit [Ping timeout: 268 seconds]
zzach has joined #ipfs
ctOS has quit [Quit: Connection closed for inactivity]
_whitelogger has joined #ipfs
lassulus_ has joined #ipfs
lassulus has quit [Ping timeout: 246 seconds]
lassulus_ is now known as lassulus
Imaginatrix has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
aarshkshah1992 has quit [Quit: Leaving...]
thomasan_ has joined #ipfs
thomasanderson has quit [Ping timeout: 246 seconds]
<postables[m]>
out of curiosity i keep seeing `sharness` tests repetead in the go-ipfs codebase, what does this mean?
Imaginatrix has joined #ipfs
ncb^ has quit []
mischat has joined #ipfs
Imaginatrix has quit [Client Quit]
nphb^ has joined #ipfs
Imaginatrix has joined #ipfs
cwahlers_ has joined #ipfs
cwahlers has quit [Ping timeout: 272 seconds]
Steverman has quit [Ping timeout: 240 seconds]
mischat has quit [Ping timeout: 250 seconds]
gluck666[m] has joined #ipfs
thomasan_ has quit [Remote host closed the connection]
_whitelogger has joined #ipfs
dimitarvp has quit [Quit: Bye]
refpga has quit [Read error: Connection reset by peer]
refpga has joined #ipfs
thomasanderson has joined #ipfs
Imaginatrix has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
thomasanderson has quit [Ping timeout: 244 seconds]
Imaginatrix has joined #ipfs
mischat has joined #ipfs
reit has quit [Quit: Leaving]
_whitelogger_ has joined #ipfs
_whitelogger has quit [Ping timeout: 250 seconds]
xcm is now known as Guest26461
Guest26461 has quit [Read error: Connection reset by peer]
xcm has joined #ipfs
Imaginatrix has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
mischat has quit [Ping timeout: 252 seconds]
ONI_Ghost has quit [Ping timeout: 245 seconds]
D14na[m] has joined #ipfs
xcm has quit [Read error: Connection reset by peer]
D14na[m] has left #ipfs ["User left"]
xcm has joined #ipfs
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
cwahlers has joined #ipfs
cwahlers_ has quit [Ping timeout: 250 seconds]
cris has quit [Ping timeout: 240 seconds]
_whitelogger has joined #ipfs
mischat has joined #ipfs
dasj19 has joined #ipfs
rno_ has joined #ipfs
rno has quit [Ping timeout: 272 seconds]
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
}ls{ has joined #ipfs
mischat has quit [Ping timeout: 250 seconds]
vyzo has quit [Quit: Leaving.]
vyzo has joined #ipfs
MDude has quit [Quit: Going offline, see ya! (www.adiirc.com)]
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
jamesaxl has joined #ipfs
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
yf33 has joined #ipfs
dasj19 has quit [Ping timeout: 246 seconds]
reit has joined #ipfs
Mateon3 has joined #ipfs
Mateon1 has quit [Ping timeout: 246 seconds]
Mateon3 is now known as Mateon1
fleeky has quit [Ping timeout: 240 seconds]
cwahlers_ has joined #ipfs
cwahlers has quit [Ping timeout: 246 seconds]
fleeky has joined #ipfs
reit has quit [Quit: Leaving]
nonono has joined #ipfs
reit has joined #ipfs
reit has quit [Client Quit]
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
chiui_ has joined #ipfs
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
thomasanderson has joined #ipfs
thomasanderson has quit [Ping timeout: 246 seconds]
BeerHall has joined #ipfs
kapil____ has quit [Quit: Connection closed for inactivity]
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
spinza has quit [Quit: Coyote finally caught up with me...]
<pmatteo1998>
i have a file on host A i'm trying to download from host B , now , i could resolve the hash ,so i have peerid of A , but both cat and get are terribly slow , how can i add A directly to my peerlist? so i can relunch cat an *hopefully* A sends me directly the file?
<postables[m]>
standard Depending on what gateway you're using, it may or may not be set to validate the mime-type. Usually you can easily detect mime-type by analyzing content. Are you familiar with golang?
<standard[m]>
I am in the process of learning golang - I also have my own gateway setup through AWS
<postables[m]>
@pmatteo1998: Yes that can definitely be the case if the content isn't very popular. You can easily connect your peers directly using:
<standard[m]>
I am writing my stack in node, can I still use this?
<standard[m]>
I am just parsing the url of the gateway + hash into an html5 audio element.
chiui_ has quit [Ping timeout: 246 seconds]
<postables[m]>
hmm if your stack is mostly node, might not be beneficial to add some extra golang dependencies if they aren't neccessary. I imagine nodejs has similar functionality. however you can also read the mimesniff specification here https://mimesniff.spec.whatwg.org/
<postables[m]>
do you have an exmaple hash that you can share?
<standard[m]>
This one should not work when accessing through a gateway url : QmQ7wt8rmoKKZ9JxH3r1kQuPacYjA8DHhhW2x9ysBtWPrP
<standard[m]>
This one should work: QmXt4S4JpdJrUc2AjsYbM6xKTHdTxynbfDiPU1knVZ295F
<postables[m]>
ok im trying those from my gateway now, will let you know when they resolves
<standard[m]>
if it takes too long I will send you mine
<postables[m]>
might as well, wouldn't hurt 😄
<pmatteo1998>
@postables resolve gives me only the peer id not the multiaddr
<postables[m]>
@pmatteo1998: `ipfs id` should give you the full connection
<postables[m]>
*edit:* ~~@pmatteo1998: `ipfs id` should give you the full connection~~ -> @pmatteo1998: `ipfs id` should give you the full multiaddr that you can feed into `ipfs swarm connect`
BeerHall has quit [Quit: BeerHall]
cwahlers has joined #ipfs
cwahlers_ has quit [Ping timeout: 268 seconds]
rain1 has joined #ipfs
rain2 has quit [Ping timeout: 245 seconds]
clemo has joined #ipfs
Steverman has quit [Ping timeout: 244 seconds]
i1nfusion has quit [Remote host closed the connection]
refpga has quit [Remote host closed the connection]
Lochnair has quit [Ping timeout: 245 seconds]
malaclyps has quit [Read error: Connection reset by peer]
malaclyps has joined #ipfs
nonono has quit [Ping timeout: 244 seconds]
<postables[m]>
@pmatteo1998: sounds like you might have a connectivity issue (most likely blocked ports or something). What's the network setup like? are these nodes on the same lan, etc..
Steverman has joined #ipfs
<pmatteo1998>
@postables A isa 4g network connection (15mbps) B 1gbps home connection
mentos1386 has joined #ipfs
<pmatteo1998>
@postables i have no problem on tcp port on the 4g , i tried a udp hole puching (with a stun) and it worked
mentos1386 has quit [Client Quit]
MDude has joined #ipfs
mischat has joined #ipfs
kapil____ has joined #ipfs
<postables[m]>
ah yea home connections can be very troublesome sometimes
<pmatteo1998>
@postables i have done any possible test
Mottengrotte has joined #ipfs
<pmatteo1998>
and it worked every single time
<pmatteo1998>
the house connection for me is not a problem
<pmatteo1998>
more probably is issue with libp2p
mischat has quit [Ping timeout: 252 seconds]
ctOS has joined #ipfs
spamfox has joined #ipfs
spamfox has quit [Quit: Leaving]
pmatteo1998 has quit [Ping timeout: 240 seconds]
Lochnair has joined #ipfs
clemo has quit [Ping timeout: 244 seconds]
<Swedneck>
does adding stuff to `ipfs files` pin them?
clemo has joined #ipfs
cwahlers has quit [Ping timeout: 250 seconds]
cwahlers has joined #ipfs
rendar has joined #ipfs
stoopkid has quit [Quit: Connection closed for inactivity]
<ToxicFrog>
Swedneck: yes. The root of the `ipfs files` tree is always considered to be pinned.
<voker57>
network is usually a bottleneck if you're not adding without a daemon
lidel` has joined #ipfs
lidel has quit [Ping timeout: 245 seconds]
lidel` is now known as lidel
<Swedneck>
it's on the same network as my desktop though
nast has joined #ipfs
<Swedneck>
also: does adding hashes to mfs fetch the data?
nast has quit [Remote host closed the connection]
Steverman has joined #ipfs
ctOS has joined #ipfs
<ctOS>
Has anyone made a monitoring/watchlist for IPFS CIDs? A simple webapp where I can dd IPFS hshes and be notified when/if that hash becomes available again. Or maybe something like an uptime monitor?
purisame has joined #ipfs
dasj19 has quit [Quit: dasj19]
<Magik6k>
@swedneck: Do both use standard datastore config (flatfs)? If yes, then it might be related to filesystem differences (check what consumes most cpu in htop (kernel vs userspace bars))
<Magik6k>
Adding to mfs shouldn't fetch anything afaik
<Swedneck>
so if i want to help host ipfs.io i'd have to pin it?
<Swedneck>
and no, the server uses badgerdb
<Magik6k>
Yes, basically, but the hash can change, so you'd have to watch `ipfs dns -r ipfs.io` and pin those hashes
<Magik6k>
(there is wip feature which will watch for changing ipns names and pin them - 'ipfs name follow')