stebalien changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.5.1 and js-ipfs 0.43.1 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of Conduct: https://git.io/vVBS0
<swedneck1>
nixfreak: well there's "decentralized" which simply means that the platform isn't bound to a single point, which includes everything from mastodon to matrix to ipfs, and there's "distributed" which is what IPFS is, and means almost everyone who uses the platform is part of the network and recieves and sends data to others
<swedneck1>
things that are decentralized but not distributed are usually "federated", meaning that there are multiple servers that talk to each other, this is what matrix and mastodon do
<swedneck1>
probably the easiest way to make a decentralized app which is actually usable would be to have the client written in javascript and served via IPFS, and use matrix as the backend
hsn has joined #ipfs
<swedneck1>
but of course you could definitely make a fully distributed app by using things like IPFS, libp2p, orbitdb, textile, etc..
<codepy69>
what are the system requirements for ipfs? would it run ok on a 512 MB rpi?
<postDiscord[m]>
if you just want to run a dht client with like absolutely no peers probably
<postDiscord[m]>
if you want to be a dht server and have like 1k connections definitely no
levi501dtablet has joined #ipfs
<swedneck1>
i think you could run ipfs on such a low-spec device, but you probably won't have much fun
<swedneck1>
ipfs is for sure gonna take up like half the memory
<swedneck1>
and you'd have to limit the connections to like 200
}ls{ has quit [Ping timeout: 264 seconds]
hsn has quit [Ping timeout: 264 seconds]
<nergal[m]1>
what about low-power profile?
}ls{ has joined #ipfs
smelly_ has quit [Ping timeout: 265 seconds]
levi501dtablet has quit [Ping timeout: 246 seconds]
stripedpajamas has quit [Quit: sleeping...]
hsn has joined #ipfs
smellyonionman has joined #ipfs
<smellyonionman>
Good evening server
hsn has quit [Ping timeout: 272 seconds]
teej has quit [Ping timeout: 240 seconds]
hhes has quit [Ping timeout: 240 seconds]
xinbenlv has quit [Ping timeout: 240 seconds]
zootella has quit [Ping timeout: 246 seconds]
hhes has joined #ipfs
chicoxyzzy has quit [Ping timeout: 258 seconds]
PyHedgehog has quit [Ping timeout: 260 seconds]
redlegion has quit [Ping timeout: 244 seconds]
smellyonionman has quit [Ping timeout: 264 seconds]
zootella has joined #ipfs
xinbenlv has joined #ipfs
mikolalysenko has quit [Ping timeout: 258 seconds]
jimpick has quit [Ping timeout: 240 seconds]
georgyo has quit [Ping timeout: 240 seconds]
physpi has quit [Ping timeout: 240 seconds]
smellyonionman has joined #ipfs
Laurian__ has quit [Read error: Connection reset by peer]
mowcat has quit [Remote host closed the connection]
carrot^ has quit [Ping timeout: 240 seconds]
zootella has quit [Ping timeout: 246 seconds]
redlegion has joined #ipfs
bret has quit [Ping timeout: 272 seconds]
PyHedgehog has joined #ipfs
mikolalysenko has joined #ipfs
zootella has joined #ipfs
jimpick has joined #ipfs
bret has joined #ipfs
georgyo has joined #ipfs
teej has joined #ipfs
carrot^ has joined #ipfs
Laurian__ has joined #ipfs
physpi has joined #ipfs
chicoxyzzy has joined #ipfs
hsn has joined #ipfs
hsn has quit [Ping timeout: 256 seconds]
Jeanne-Kamikaze has joined #ipfs
Sasara is now known as Maralis
Maralis has quit [Quit: Leaving]
Sasara has joined #ipfs
Sasara has quit [Client Quit]
Relis has quit [Quit: This computer has gone to sleep]
Guest79601 has quit [Read error: Connection reset by peer]
Guest79601 has joined #ipfs
fazo96 has quit [Ping timeout: 240 seconds]
lva has joined #ipfs
mauz555 has joined #ipfs
lva has quit [Client Quit]
ipfs-stackbot has quit [Remote host closed the connection]
xcm has quit [Read error: Connection reset by peer]
ipfs-stackbot has joined #ipfs
xcm has joined #ipfs
intoo_ has joined #ipfs
dqx_ has joined #ipfs
muslimm has joined #ipfs
dannycarrera has joined #ipfs
namenom[m] has joined #ipfs
gavlee has quit []
gavlee has joined #ipfs
muslimm has quit [Ping timeout: 265 seconds]
}ls{ has joined #ipfs
jadedctrl has joined #ipfs
xcm has quit [Ping timeout: 246 seconds]
jadedctrl has quit [Ping timeout: 256 seconds]
xcm has joined #ipfs
Nact has quit [Ping timeout: 240 seconds]
ylp has quit [Quit: Leaving.]
Pie-jacker875 has quit [Quit: No Ping reply in 180 seconds.]
Pie-jacker875 has joined #ipfs
xcm has quit [Ping timeout: 246 seconds]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
xcm has joined #ipfs
dannycarrera has quit [Remote host closed the connection]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
intoo_ has quit [Remote host closed the connection]
jadedctrl has joined #ipfs
arch-nemesis has joined #ipfs
jadedctrl has quit [Read error: Connection reset by peer]
Mateon1 has quit [Ping timeout: 264 seconds]
Mateon1 has joined #ipfs
mowcat has joined #ipfs
jadedctrl has joined #ipfs
xcm has quit [Ping timeout: 246 seconds]
Newami has joined #ipfs
Newami has quit [Remote host closed the connection]
mauz555 has quit []
xcm has joined #ipfs
Guest79601 has quit [Ping timeout: 264 seconds]
hqdruxn08 has quit [Ping timeout: 256 seconds]
tooetooDiscord[4 has joined #ipfs
ZaZ has joined #ipfs
__jrjsmrtn__ has quit [Quit: Bye !]
jet_ has joined #ipfs
<RubenKelevra[m]>
<hsn "with 50 connections nobody will "> This isn't correct. It just takes longer to push stuff to the DHT, but since this is just done once in every 12 hours, there's not an issue.
<RubenKelevra[m]>
ˈt͡sɛːzaɐ̯: which version do you use? Do you use QUIC? Do you use badgerds?
<RubenKelevra[m]>
Also since 0.5.0 the connection limit isn't "strict" as it was before. If connections are deemed in use, the won't be terminated. So IPFS can sometimes use significantly more connections than the upper limit, if you ask the daemon to do a lot of stuff.
<RubenKelevra[m]>
* Also since 0.5.0 the connection limit isn't "strict" as it was before. If connections are deemed in use, they won't be terminated. So IPFS can sometimes use significantly more connections than the upper limit, if you ask the daemon to do a lot of stuff.
<postDiscord[m]>
🤔 why even have a connection limit then lol
inuvato[m] has joined #ipfs
levi501dtablet has joined #ipfs
hqdruxn08 has joined #ipfs
jet_ has left #ipfs ["Leaving"]
levi501dtablet has quit [Ping timeout: 265 seconds]
<RubenKelevra[m]>
to kill useful idle connections.
<RubenKelevra[m]>
* to kill idle connections.
<RubenKelevra[m]>
post (Discord): feel free to work out a better concept. :)
<JCaesar>
0.5.1, badgerds. No memory of enabling quic, so I guess I am not using it.
cheet has quit [Quit: ZNC 1.8.x-nightly-20190530-13450c7f - https://znc.in]
<JCaesar>
And, I happened to observe the event yesterday. It's something I've never seen under linux. IPFSs virtual memory usage goes to 4GB, the resident size is at around 700 MiB. Yet, 3GB of RAM+SWAP are used.
<JCaesar>
Connection count was above the limit, but nothing crazy. Like 600 instead of the configured 450.
jadedctrl has quit [Quit: Vision[0.10.3]: i've been blurred!]
<JCaesar>
Maybe that's badgerds memory mapping lots of files simultaneously?
<RubenKelevra[m]>
ˈt͡sɛːzaɐ̯: The connections are expected to rise above the limit if there's network traffic requiring it. 0.5 shifted from the 0.4 behavior to kill connections regardless of the current status of them, to killing only connections which doesn't deemed to be useful and currently idle.
<RubenKelevra[m]>
That's what I read, but I might be not entirely correct about that.
<RubenKelevra[m]>
ˈt͡sɛːzaɐ̯: badgerds has issues with extremly high peaks of memory usage during (badger) garbage collection
<JCaesar>
Either way 600 is fine.
<RubenKelevra[m]>
this is fixed upstream and will be included in the next badger-release
<RubenKelevra[m]>
ˈt͡sɛːzaɐ̯: keep in mind, that taking part in the DHT uses some memory. If you're looking into how you're memory footprint can be lowered, changing your node to dhtclient do make sense.
<RubenKelevra[m]>
ˈt͡sɛːzaɐ̯: and virtual memory use doesn't matter. Linux doesn't really clean up behind the Go garbage collector, that's why the virtual memory rises
<JCaesar>
It probably won't make enough of a difference.
KempfCreative has quit [Ping timeout: 256 seconds]
<JCaesar>
That's what I thought, too. But it doesn't fit the symptom that virtual memory rises with overall physical memory usage.
<RubenKelevra[m]>
Well, taking part on the DHT makes a large difference in terms of incoming connections. Since there are constantly requests from nodes. More connections means more memory usage
<RubenKelevra[m]>
Linux does also not clear up unused memory from the Go runtime physically, unless it's under memory pressure
<RubenKelevra[m]>
if you're curious, one of my nodes is currently at 5.5g res and 14g virtual :)
<RubenKelevra[m]>
while the cluster service is at 1.8g res and 10.9 virtual
<RubenKelevra[m]>
total memory is 16g, while "memory available" is 13.8g
<RubenKelevra[m]>
so "res" doesn't show "used" memory properly for ipfs
<RubenKelevra[m]>
systemd shows for ipfs.service: Memory: 12.6G
<RubenKelevra[m]>
ˈt͡sɛːzaɐ̯: have you looked at `/proc/pressure/memory`? This gives the best indication on how the system is under "stress" finding memory for it's operations
<JCaesar>
Not at the time. That doesn't happen to be exposed by prometheus-node-exporter?
<RubenKelevra[m]>
ˈt͡sɛːzaɐ̯: I have no idea what prometheus exports. I use netdata to monitor my nodes
<RubenKelevra[m]>
ˈt͡sɛːzaɐ̯: maybe just add a small cronjob which sends you a mail if avg300 is going above 2.0? :)
<RubenKelevra[m]>
Also, some older kernels won't show this values at all. They are a pretty recent addition.
ZaZ has quit [Quit: Leaving]
<JCaesar>
No need. It's exposed as node_pressure_memory_waiting_seconds_total. And during those "events", it rises by about 0.8 seconds per second.
<RubenKelevra[m]>
Cool!
rationalize has joined #ipfs
<RubenKelevra[m]>
ˈt͡sɛːzaɐ̯: everything below 2.0 shouldn't be a huge issue IMHO.
<RubenKelevra[m]>
it just shows that the system isn't idleing but currently trying to catch up with the requests it got
<RubenKelevra[m]>
To make this clear, I'm talking about the avg300 values
<JCaesar>
I'm not sure how the avg300 is calculated from the total seconds…
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
muslimm has joined #ipfs
muslimm has quit [Max SendQ exceeded]
<JCaesar>
(btw, I'm also observing the what systemd reports as memory usage, and that was also lower than 1GB the entire time. So I wouldn't trust that either.)
sabry97[m] has joined #ipfs
dqx_ has quit [Ping timeout: 260 seconds]
RobiX has joined #ipfs
RobiX has quit [Client Quit]
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
hsn has quit [Ping timeout: 272 seconds]
Adbray has quit [Quit: Ah! By Brain!]
arch-nemesis has quit [Ping timeout: 260 seconds]
DevDanDiscord[m] has joined #ipfs
Adbray has joined #ipfs
tsrt^ has quit []
ffl^ has joined #ipfs
levi501dtablet has joined #ipfs
rationalize has quit [Ping timeout: 260 seconds]
<RubenKelevra[m]>
ˈt͡sɛːzaɐ̯: well, avg* are percent values, so 2% of the time in the last 300 seconds a stall occurred
<RubenKelevra[m]>
ˈt͡sɛːzaɐ̯: but node_pressure_memory_waiting_seconds_total sounds more like it's observing the absolute value in us called total.