aschmahmann changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.7.0 and js-ipfs 0.52.3 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of Conduct: https://git.io/vVBS0
hsn has quit [Ping timeout: 268 seconds]
arcatech_ has quit [Quit: Be back later.]
Nact has quit [Quit: Konversation terminated!]
XORed has quit [Remote host closed the connection]
XORed has joined #ipfs
}ls{ has quit [Ping timeout: 252 seconds]
}ls{ has joined #ipfs
misuto has quit [Ping timeout: 240 seconds]
ipfs-stackbot has quit [Remote host closed the connection]
ipfs-stackbot has joined #ipfs
ctOS has quit [Quit: Connection closed for inactivity]
cymatics has joined #ipfs
misuto has joined #ipfs
koo55 has quit [Ping timeout: 260 seconds]
tech_exorcist has quit [Quit: tech_exorcist]
tsunamionioncere has joined #ipfs
TeDChange has quit [Ping timeout: 252 seconds]
cymatics has quit [Quit: Konversation terminated!]
XORed has quit [Read error: Connection reset by peer]
XORed has joined #ipfs
TeDChange has joined #ipfs
XORed has quit [Ping timeout: 265 seconds]
XORed_ has joined #ipfs
XORed_ has quit [Read error: Connection reset by peer]
XORed has joined #ipfs
ctOS has joined #ipfs
royal_screwup21 has joined #ipfs
royal_screwup21 has quit [Ping timeout: 260 seconds]
<CSDUMMI[m]>
I want to develop a little something with IPFS and Python 3. What is the proper Library for that. I know, there is a py-ipfs implementation in development, but by no means ready for production last I checked. So what else should I use?
TheMeltdown has joined #ipfs
TheMeltdown is now known as Guest93189
Guest84395 has quit [Ping timeout: 260 seconds]
opal has joined #ipfs
koo55 has joined #ipfs
opal has quit [Remote host closed the connection]
opal has joined #ipfs
<McSinyx[m]1>
CSDUMMI it's ready for production, albeit not using state of the art concurrency that would be provided by httpx
<McSinyx[m]1>
depending on your use case you can use that lib or roll out your own IPFS HTTP API client
hsn has quit [Ping timeout: 240 seconds]
royal_screwup21 has quit [Quit: Connection closed]
royal_screwup21 has joined #ipfs
fiws has quit [Quit: i quit]
fiws has joined #ipfs
niyawe has quit [Remote host closed the connection]
niyawe has joined #ipfs
baojg has quit [Remote host closed the connection]
baojg has joined #ipfs
<CSDUMMI[m]>
<McSinyx[m]1 "CSDUMMI it's ready for productio"> You mean this py-IPFS, that has not section on install and usage is ready for production? https://github.com/ipfs-shipyard/py-ipfs#install
fiws has quit [Quit: i quit]
fiws has joined #ipfs
tech_exorcist has joined #ipfs
baojg has quit [Remote host closed the connection]
baojg has joined #ipfs
royal_screwup21 has quit [Quit: Connection closed]
royal_screwup21 has joined #ipfs
royal_screwup21 has quit [Ping timeout: 240 seconds]
plntyk has quit [Remote host closed the connection]
royal_screwup21 has joined #ipfs
clemo has joined #ipfs
royal_screwup21 has quit [Ping timeout: 265 seconds]
nrh^ has quit []
dsrt^ has joined #ipfs
royal_screwup21 has joined #ipfs
<McSinyx[m]1>
no the http api client one
plntyk has joined #ipfs
clemo has quit [Ping timeout: 260 seconds]
CGretski has quit [Ping timeout: 248 seconds]
Sascha1337[m] has joined #ipfs
baojg has quit [Remote host closed the connection]
ipfs-stackbot has quit [Remote host closed the connection]
baojg has joined #ipfs
ipfs-stackbot has joined #ipfs
}ls{ has joined #ipfs
jocque[m] has joined #ipfs
royal_screwup215 has joined #ipfs
zeden has joined #ipfs
tech_exorcist has quit [Quit: tech_exorcist]
tech_exorcist has joined #ipfs
royal_screwup21 has quit [Quit: Connection closed]
royal_screwup215 has quit [Quit: Connection closed]
royal_screwup21 has joined #ipfs
royal_screwup21 has quit [Ping timeout: 240 seconds]
royal_screwup21 has joined #ipfs
royal_screwup217 has joined #ipfs
<eleitl[m]>
My 6 TB data set pinning test completed (had to split the rest of it in a few batches). ~/.ipfs (on SSD) is less than 12 GB, so pretty good.
<eleitl[m]>
I should probably reboot the system to test that everything will come up correctly, and look at memory use over time.
<Sascha1337[m]>
6TB data on IPFS? holy moly
<Piddle>
i got 6GB if thats impressive 😎
<eleitl[m]>
6 TB on a 4 GB RAM Atom D510 toaster, more impressively.
<RomeSilvanus[m]>
My 40TB add is still going for 6 days :’c
<eleitl[m]>
Good luck with it, Rome Silvanus
<eleitl[m]>
I would be interested to know what the resource usage (RAM, mostly) for a TB stored with filestore is. I thought the 256 kB chunks were tracked in RAM.
<McSinyx[m]1>
eleitl, i don't get it, does that mean ipfs compress 6 TB down to only 12 GB?
<Discordian[m]>
He's using filestore, it doesn't make a copy of the data, so it uses 12GB for just tracking the blocks
<eleitl[m]>
Exactly.
<eleitl[m]>
In principle you could put ~/.ipfs on the SATA spindle as well, but it would hurt performance a lot.
<eleitl[m]>
I would test the same setup on a RPi4 with 4 or 8 GB RAM and an USB SSD plus HDD, but I don't have one. So the Atom is a reasonably good approximation.
<eleitl[m]>
Largest USB drives are at 18 TB currently, so this would mean you could pin a lot of data with a small embedded. Or so at least in theory.
<RomeSilvanus[m]>
You could buy one of these 100TB server ssds
royal_screwup21 has quit [Quit: Connection closed]
<Discordian[m]>
(I swear I'll be diving into ipfs-sync ASAP, just doing on-boarding rn 😅)
<eleitl[m]>
Howeve, this is about putting content CDN as close to the network edge as possible, not just CPE, but your living-room-equipment.
<eleitl[m]>
* However, this is about putting content CDN as close to the network edge as possible, not just CPE, but your living-room-equipment.
<Discordian[m]>
Those aren't SSDs tho
<Discordian[m]>
Oh the bigger ones have NVMEs, still not 100TB though 😛
<eleitl[m]>
zfs with NVMe for cache/slog
<Discordian[m]>
Yeah they'd probably work great. I'm glad I don't have that much data I want on IPFS rn
<Discordian[m]>
Closer to 4GB, lives happily on my SSD
<eleitl[m]>
My model is /r/selfhosted /r/Datahoarders which have a RPi 4 or better and a single large external drive -- no need for noise/AC and large power bills.
<Discordian[m]>
Passive cooling is so nice sometimes
<eleitl[m]>
Indeed. In an ideal world this all should be handled by an embedded router.
plntyk has quit [Remote host closed the connection]
plntyk has joined #ipfs
theseb has joined #ipfs
royal_screwup217 has quit [Quit: Connection closed]
royal_screwup21 has quit [Quit: Connection closed]
royal_screwup21 has joined #ipfs
matt-h has quit [Ping timeout: 250 seconds]
royal_screwup21 has quit [Ping timeout: 240 seconds]
matt-h has joined #ipfs
arcatech has joined #ipfs
royal_screwup21 has joined #ipfs
Nukien2 has joined #ipfs
Mikaela has quit [Ping timeout: 240 seconds]
royal_screwup21 has quit [Quit: Connection closed]
royal_screwup21 has joined #ipfs
Mikaela has joined #ipfs
royal_screwup21 has quit [Ping timeout: 268 seconds]
<Discordian[m]>
Am a bit curious though, does it re-post old things actually? I get all excited about new things, but the docs post there is from last year, but reposted today.
<RomeSilvanus[m]>
Discordian: If the folder contents change while ipfs-sync is doing the indexing will it detect these changes and do them after, or only at the next indexing run?
maggotbrain has quit [Remote host closed the connection]
royal_screwup21 has quit [Ping timeout: 240 seconds]
<stebalien>
Sorry, I've cleaned up the bot spam. We've recently deployed changes to the blog which likely triggered new posts to discord which triggered new posts here...
horsefeathers has joined #ipfs
royal_screwup21 has joined #ipfs
horsefeathers has quit [Ping timeout: 240 seconds]