stebalien changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.4.18 and js-ipfs 0.33 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS, the InterPlanetary FileSystem: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of Con
<Swedneck>
pretty certain it only takes ipfs resources
<Swedneck>
`Error: cp: cannot get node from path /home/tim/2TB_HDD/Pictures/memes_on_IPFS/rms.png: file does not exist`
<Swedneck>
plus the fact that it doesn't accept relative paths, which you'd expect if it could use normal files
uncle_ben has quit [Remote host closed the connection]
<deltab>
ah okay
<Swedneck>
`ipfs files add` would also just be more intuitive
<Swedneck>
at least to people who are used to `ipfs add`
<lanzafame1>
The API and the commands are the 'same' thing. See go-ipfs-cmds
jesse22 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
mischat has quit [Remote host closed the connection]
<Swedneck>
so should i open an issue there?
ctOS has joined #ipfs
appa has quit [Quit: WeeChat 1.9.1]
henriquev has joined #ipfs
<lanzafame1>
Hmm just reading back through the discussion... If its the unintuitivness of the commands, open it in go-ipfs. Am I understanding the issue correctly?
thomasan_ has joined #ipfs
<Swedneck>
the issue is that you have to run 2 commands to add something to mfs
<Swedneck>
which is both unintuitive and really annoying
thomasan_ has quit [Remote host closed the connection]
appa has joined #ipfs
<lanzafame1>
Yeah open an issue in go-ipfs for that
quetzalc0atl has quit [Quit: WeeChat 1.6]
xelra has quit [Ping timeout: 240 seconds]
xelra has joined #ipfs
akkad is now known as ober
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
user_51 has quit [Ping timeout: 245 seconds]
Belkaar has quit [Read error: Connection reset by peer]
user_51 has joined #ipfs
Belkaar has joined #ipfs
Belkaar has joined #ipfs
zeden has quit [Quit: WeeChat 2.3]
}ls{ has quit [Ping timeout: 246 seconds]
}ls{ has joined #ipfs
lassulus_ has joined #ipfs
lassulus has quit [Ping timeout: 246 seconds]
lassulus_ is now known as lassulus
henriquev has quit [Quit: Connection closed for inactivity]
reit has joined #ipfs
lord| has quit [Quit: WeeChat 2.3]
lord| has joined #ipfs
BeerHall has quit [Quit: BeerHall]
reit has quit [Quit: Leaving]
_whitelogger has joined #ipfs
reit has joined #ipfs
obensource has quit [Ping timeout: 240 seconds]
ctOS has quit [Quit: Connection closed for inactivity]
ctOS has joined #ipfs
mischat has joined #ipfs
mischat has quit [Ping timeout: 245 seconds]
obensource has joined #ipfs
_whitelogger has joined #ipfs
BeerHall has joined #ipfs
zopsi_ has quit [Quit: Oops]
zopsi has joined #ipfs
thomasan_ has joined #ipfs
thomasan_ has quit [Remote host closed the connection]
ctOS has quit [Quit: Connection closed for inactivity]
clemo has joined #ipfs
Ai9zO5AP has quit [Ping timeout: 246 seconds]
cheet has quit [Quit: ZNC 1.8.x-nightly-20190111-e415d9f5 - https://znc.in]
cheet has joined #ipfs
woss_io has joined #ipfs
Ai9zO5AP has joined #ipfs
BeerHall has quit [Quit: BeerHall]
uncle_ben has joined #ipfs
<uncle_ben>
How do I add a file to generate a CIDv1 hash?
<uncle_ben>
ok found it: ipfs add --cid-version 1 [file]
<seba-->
uncle_ben write a post on discuss.ipfs or maybe we could start a wiki somewhere
<uncle_ben>
seba--, Well I just posted the command, so I'm not sure why I'd need to post on discuss.ipfs, but a wiki for users is a good idea. The default action on the add command is to use CIDv0 and CIDv1 is described as "experimental" in the documentation. Since I'd rather not create double the disk storage on my local repo, I'm wondering if I should just always use CIDv1, but it seems the majority of the content being shared is on the older format
<seba-->
well whole IPFS is quite experimental
<seba-->
:)
<uncle_ben>
haha true, and it seems they've been working on it for a couple of years already, so it's likely to become the new standard soon...probably safe to use exclusively for content, at least if it's not critically important
<seba-->
well it doesn't work that well yet for NAT
<uncle_ben>
In that case, I guess I'll just encode it in both formats for now, though it's a bit of a nuisance
woss_io has quit [Read error: Connection reset by peer]
trqx has quit [Remote host closed the connection]
<uncle_ben>
IF IPFS becomes the standard for Web 3.0 isn't that going to lead to very high CPU usage since all content will need to be decrypted and piece together from the merkle dag?
trqx has joined #ipfs
maxzor has joined #ipfs
fazo has joined #ipfs
spinza has quit [Quit: Coyote finally caught up with me...]
woss_io has joined #ipfs
<seba-->
uncle_ben, most of the sites are on HTTPS now
<seba-->
so it's already all encrypted
dimitarvp has joined #ipfs
spinza has joined #ipfs
pecastro has joined #ipfs
<uncle_ben>
seba--, True, but isn't there more CPU overhead because of Merkle Dag? I guess I'm wondering if there's going to be both a performance penalty and more electricity usage from a full web migration to IPFS
<seba-->
what's about merkle dag?
<seba-->
merkle dag is approx. like torrents work
<seba-->
to me, the biggest issue is how searching of content works
<seba-->
but for now i haven't came to that yet
<uncle_ben>
Right, but torrents are have more lag and overhead because of dht and needing to stay connected with lots of peers
<seba-->
but from user experience it works suboptimal
<seba-->
as you can see, even for just pinging it's a lot of overhead ^.^
<uncle_ben>
Sure, I'm not a programmer, so I won't be able to contribute much, I just think it's a cool technology. I'm not trying to be negative about it, I'm just trying to understand all the advantages and disadvantages compared to what we have now as far as web content distribution
<uncle_ben>
Yeah, I'm trying to understand it too as much as a non-programmer can. I'm guessing IPFS will be used selectively for certain use cases, versus all web content being hosted on the network
<seba-->
i don't know
<seba-->
also low power devices are a problem
<seba-->
it consumes a lot of CPU and RAM
<seba-->
way more than torrent
<voker57>
true ipfs has some performance-atrocious code
<uncle_ben>
why can't they just use the bittorrent protocol instead?
<seba-->
i don't think it's a problem of protocol, but i might be wrong
<seba-->
uncle_ben, what is your interesting IPFS? do you want to make anything with it?
<uncle_ben>
well it's a problem if the current protocol has so many performance issues. again, i'm not very knowledgable about it, so maybe we're overlooking something or that the performance kinks are already known and being worked out.
<seba-->
uncle_ben, i don't know what it's being done tbh :>
<seba-->
i just know that it consumes too much resources :D
<uncle_ben>
seba--, I just like the idea of content producers being able to host their own content as opposed to being controlled by the YouTube, Facebook, etc. Being demonitized and deplatformed is a real threat these days. It would be nice if we could move to something more platform neutral and censorship resistant.
Mateon3 has joined #ipfs
<seba-->
depltformed yes, i don't see how you would solve "demonitized"
<seba-->
:>
Mateon1 has quit [Ping timeout: 244 seconds]
Mateon3 is now known as Mateon1
<seba-->
i think IPFS can be censored just as well
<seba-->
you just ban a hash
<seba-->
big deal lol
<uncle_ben>
seba--, well, I'm thinking far ahead, like in 10 or 15 years when people can bypass the financial system to get paid for their services in crypto
<seba-->
you can already bypass it
<seba-->
but on youtube people live out of advertisment
<uncle_ben>
My understanding is that the hash bans are at the gateway level, so if IPFS really took off, there would be plenty of gateways that wouldn't ban
<seba-->
not out of direct payments
<seba-->
that's rarer
<seba-->
well you could ban it in DHT
<seba-->
or bootstrap nodes would be required to ban users etc.
<seba-->
it's possible
BeerHall has joined #ipfs
<seba-->
i mean for a knowledgable user is not a problem
<seba-->
but i'm talking about average joe
<seba-->
:)
<seba-->
but it is harder sure
<uncle_ben>
Well, I'm not knowledgeable enough to know how to override those kinds of blocks, but I'd imagine if IPFS became too ban happy people would just migrate to something similar, but freer, just like people ditched CDs for napster and bittorrent
BeerHall has quit [Client Quit]
<uncle_ben>
I really don't think the people working on IPFS are thinking that way, though
<uncle_ben>
probably would only take action in extreme cases
<seba-->
well you also aren't anonymous
<seba-->
hm
<uncle_ben>
I bet there will be an anonymity layer added eventually
<seba-->
like anyone could for example search who downloaded a file which is illegal
<seba-->
find peers
<seba-->
like their IDs and their IPs from that
<uncle_ben>
right, I'm not talking about criminal activity so much as just having an unpopular political view that gets you banned or deplatformed, or arrested in you live in a particularly shitty country
<seba-->
you can already do that anyway
<seba-->
youcan host your own web server
<seba-->
nobody cares
<seba-->
lol
woss_io has quit [Ping timeout: 244 seconds]
<uncle_ben>
right
woss_io has joined #ipfs
<seba-->
the problem is that if you don't use let's say twitter/facebook/youtube
<seba-->
the majority of people won't follow you at all
<seba-->
you would be almost "dark web"
<seba-->
like internet now is twitter/facebook/google ... they own all, incl. whatsapp, instagram, youtube, snapchat, etc.
<seba-->
if those 3 coorps ban you
<seba-->
you don't exist for like 99% users on the internet
<seba-->
they click or maybe google it, not directly enter
<seba-->
so IPFS won't really change much for the average user ... but startups might use IPFS tech to start webpages/webapps/etc.
<uncle_ben>
But standards can change fairly quickly. I think most content producers have niche followings anyway, so they don't really need to reach "the masses." If I really enjoyed someone's work and they got deplatformed, I'd visit a particular site to get their content. And producers could still use the normal media channels, but post IPFS hashes on their content pages so people had an alternative source as a backup.
woss_io has quit [Read error: Connection reset by peer]
<uncle_ben>
Just like sometimes producers post their bitcoin address on their YouTube page, in addition to their patreon, paypal, etc. You might be banned by patreon and paypal, but bitcoin is outside of the normal channels of control, so why not use both?
<uncle_ben>
If someone posted their IPFS hash on their YouTube page, I'd rather watch the video that way instead of being datamined and profiled by Google and the NSA
<seba-->
uncle_ben, sure, but how do you get popular?
<seba-->
thing is that you need those
<seba-->
to become popular
<seba-->
once you're popular it doesn't matter
<uncle_ben>
Right, I'm just saying people need to rely on multiple tools to protect what they've built instead of having it taken from them on a whim
<uncle_ben>
It *does* matter when you become popular because that's when your politics come under a microscope from those in power. They don't give a shit what people's views are if they only have 100 followers
<uncle_ben>
Unless they are so radical that it breaks the law in some way
<seba-->
uncle_ben, yes, IPFS does make easy some stuff
<seba-->
i'm not saying that
<seba-->
but if we look at for example pewdipay or whatever he is spelled, the richest youtuber
<seba-->
he can't afford to go away
<seba-->
he would lose most of the money
<seba-->
which he gets mainly from youtube's advertising
<seba-->
he could use IPFS to distribute the content
<seba-->
but he would have to make individual deals with advertisers
<seba-->
and not just use a platform
<uncle_ben>
I understand, but I'm arguing those are edge cases because most of the content producers are not at the absolute apex like that guy. Most content producers are making modest income from their followers and as more people eventually migrate over to crypto, the reliance on advertisers will become less and less. Especially if things like micropayments and lightning network become a thing
<seba-->
if people switch to a whole different model
<seba-->
now whole web is about advertisment
<seba-->
now you're talking about donation/subscription model
<seba-->
i mean the tech is here already
<seba-->
you can use it
<seba-->
but the problem is how to get users TO your platform in general :)
<seba-->
that's why those 3 have a monopoly
<seba-->
because if you want to be heard, you have to be on those 3
<uncle_ben>
I know, but you're talking about the status quo and I'm talking about where things are headed, or at least might be. We already see the beginnings of it with people making donations on YouTube during live broadcasts and other similar services.
<uncle_ben>
I'd rather support someone directly in a simple way than have to deal with commericials or them shilling for someone for half of their show
<seba-->
well bitcoin is shitty because it's slow and has high transaction costs ^.^
zeden has joined #ipfs
<uncle_ben>
Yeah, but that's going to be a non-issue with lightning network and even with segwit, it's really not expensive anymore. Whether it's bitcoin or some other crypto, people will eventually be able to make micropayments for content
<uncle_ben>
And you could have a decentralized kind of index to content producers that set out a price list for certain kinds of content. e.g. content made-to-order
<seba-->
sure, but not yet :)
<seba-->
have you ever tried to use crypto for something useful?
<seba-->
:D
<seba-->
it's hard as fuck
<seba-->
and expensive
<uncle_ben>
Right, but you originally asked me why I like IPFS and it's basically because it's about building something better than what we currently have. Something that will mature after a certain period of time. I know the disadvantages, but think of how shitty web 1.0 tech was compared with today. Back then even paypal was edgy
<uncle_ben>
Well, maybe the winning crypto will be DAG-based like IOTA? That answers some of your criticisms about bitcoin, even though some people claim IOTA is too centralized
<seba-->
:)
<seba-->
well you can try making something with IPFS
<seba-->
why not
<seba-->
:D
<seba-->
IPFS + crypto
<seba-->
like a webpage
<seba-->
:>
<uncle_ben>
Well IPFS has filecoin and zeronet uses the bitcoin protocol and there's a bunch of players trying to build stuff on ethereum, so idk
<seba-->
sure
<seba-->
try making an usable webpage first :D
<uncle_ben>
It all becomes usable once there's mass adoption. Most players will lose, but someone will develop something that catches on. People are getting sick of being controlled 24/7 by Facebook/Google/Twitter and all the political correctness. The time is ripe for a decentralized revolution of some kind. Maybe not bitcoin, maybe not IPFS, but something
<seba-->
i know, but i'm saying, make it :>
<seba-->
why not
<uncle_ben>
I don't have any programming skill, I'm just an armchair quarterback
<uncle_ben>
Programmers need a highly mathematical mind and inclination. I'm too right brained, i.e. scatter-brained and undisciplined for coding, but I highly respect those who can
<seba-->
hm, alright :)
<uncle_ben>
Anyway, nice chatting with you
<seba-->
uncle_ben, are you going?
<uncle_ben>
For a while anyway
malaclyps has quit [Read error: Connection reset by peer]
malaclyps has joined #ipfs
clemo has quit [Ping timeout: 246 seconds]
clemo has joined #ipfs
<Seb[m]1>
Hey Guys. I am trying to setup a private network through IPFS. Do this require all nodes to be on one network or can the nodes be on separate networks?
<Starism[m]>
80.199.141.90
lidel` has joined #ipfs
lidel has quit [Ping timeout: 272 seconds]
lidel` is now known as lidel
Steverman has joined #ipfs
dsiypl4_ has joined #ipfs
mischat has joined #ipfs
xelra has quit [Remote host closed the connection]
<seba-->
Seb[m]1, what do you mean
<seba-->
as in VPN?
ikari-pl has joined #ipfs
<Seb[m]1>
So I want to setup a Private network with two peers. One peer on my internetconnection and one on another internet connection. Is that possible?
<Seb[m]1>
Cause we are having trouble connecting to one another
<Seb[m]1>
It could be a port-fowarding issue however
<seba-->
Seb[m]1, well try telnet [ip] [port]
<seba-->
if you get multistream written
<seba-->
it works
spinza has quit [Quit: Coyote finally caught up with me...]
Encrypt has quit [Quit: Quit]
inkernel has joined #ipfs
BowStar has joined #ipfs
Encrypt has joined #ipfs
Encrypt has quit [Client Quit]
Encrypt has joined #ipfs
biplavo has joined #ipfs
BowStar has quit [Quit: Ciao 42 42 42]
spinza has joined #ipfs
xelra has joined #ipfs
<Seb[m]1>
It doesnt ;P
<Seb[m]1>
So that means that my port forwarding is incorrect right?
mischat has quit [Remote host closed the connection]
<seba-->
yeah
<seba-->
Seb[m]1 what did you do
uncle_ben has quit [Remote host closed the connection]
xelra has quit [Remote host closed the connection]
BeerHall has joined #ipfs
BeerHall has quit [Client Quit]
biplavo has quit [Quit: Page closed]
<DarkDrgn2k[m]>
i been having problems dealing wiht large datasets latly
biplavo has joined #ipfs
<DarkDrgn2k[m]>
for example hashing (ipfs add) a a 40 gigs dataset keeps casuing ` Error: read tcp 127.0.0.1:45482->127.0.0.1:5001: use of closed network connection`
<Ablibu[m]>
I want something which is already available not still in development
<Swedneck>
only option i can think of is setting up your own pinning service
<DarkDrgn2k[m]>
tehre are pinning services around..
<Swedneck>
but then you have to guarantee a bunch of stuff
<Ablibu[m]>
Mm
<Ablibu[m]>
When filecoin would stop being in development and actually start to be usable
<Ablibu[m]>
Pinning services would become obsolete, right?
<Ablibu[m]>
*edit:* ~~When filecoin would stop being in development and actually start to be usable~~ -> When filecoin will stop being in development and actually start to be usable
<Ablibu[m]>
*edit:* ~~Pinning services would become obsolete, right?~~ -> Pinning services will become obsolete, right?
<Swedneck>
i wouldn't say obsolete
<Swedneck>
they'll just become a part of one big pinning service
<Swedneck>
and they can still offer additional services, like https://temporal.cloud does
lunokhod[m] has joined #ipfs
engali has quit [Ping timeout: 260 seconds]
<Ablibu[m]>
I'm gonna see 👍
abaiste^ has quit []
hc^ has joined #ipfs
hacman has joined #ipfs
hc^ has quit [Client Quit]
hcs^ has joined #ipfs
refpga has quit [Read error: Connection reset by peer]
refpga has joined #ipfs
zeden has quit [Quit: WeeChat 2.3]
Steverman has joined #ipfs
Ai9zO5AP has quit [Ping timeout: 268 seconds]
Ai9zO5AP has joined #ipfs
<Starism[m]>
I really need a port forwarding guide, this shit never work
<Swedneck>
you need a better router :P
lunokhod[m] has left #ipfs ["User left"]
silotis has quit [Ping timeout: 250 seconds]
silotis has joined #ipfs
pcre_ has quit [Ping timeout: 246 seconds]
pcre has joined #ipfs
pcre has quit [Ping timeout: 240 seconds]
<postables[m]>
Siacoin is complete utter fucking trash they absolutely dismissed all the ASIC pre-purchasers. Ablibubu as swedneck said I'll be integrating filecoin and storj into Temporal when they are ready
<postables[m]>
I'm actually starting prototyping storj integration the next week.
pcre has joined #ipfs
<Swedneck>
what i love (i.e. find absolutely laughably stupid) about sia is how they require you to invest in them before you can earn anything
<postables[m]>
Yup I mean it used to be pretty cool and before this ASIC thing I had a lot of faith in them. But that faith was lost after my company and I put in about 25k to th ASICs and still haven't seen a single one
<Swedneck>
you ordered stuff and didn't get it? That sounds rather illegal
<postables[m]>
Yep
mbanaga[m] has joined #ipfs
<postables[m]>
In all honesty though pursuing legal action will just cost so much more money than it's worth. I figured the better option was to remove planned support for sia from temporal, and not have ant nodes in our datacenter
<Swedneck>
ouch
mbanaga[m] has left #ipfs [#ipfs]
skywavesurfer has quit [Ping timeout: 252 seconds]
<Ablibu[m]>
postables For what are you going to integrate Storj?
<Ablibu[m]>
From what I've found they give you 10% of the revenue, and take 90%..
<Ablibu[m]>
And why specifically Temporal?
<Ablibu[m]>
Do you have any experience with them?
<Ablibu[m]>
I don't get how you would integrate a service with them, the process, you know..
<postables[m]>
Ablibu : I think you might be confusing the open-source kickback program they had. It was basically a 10% revenue share for a few open-source projects that are integrating with Storj. I believe that was only given to a couple big players, filezilla, mongodb, etc....
<postables[m]>
As for why, because I'm the cto for the company developing temporal so it shouldnt be too hard 😛
skywavesurfer_ has joined #ipfs
skywavesurfer has quit [Ping timeout: 252 seconds]
<postables[m]>
the pricing model for storj is pretty interesting, you're basically paid for i believe bandwidth usage, as well as storing data? might just be storing data though
<Ablibu[m]>
Ah, so they stopped that model? Yeah I was referring to that
<Ablibu[m]>
Now I get it 😂
<postables[m]>
im not sure if they stopped it but im fairly positive they aren't accepting new peple
<Ablibu[m]>
Would it be possible to earn storj by making visitors become nodes?
<Ablibu[m]>
So someone who would install my app and after consenting to it, would become an active node..
<postables[m]>
that's probably better fitted for IPFS tbh
<Ablibu[m]>
Or it is just storage related like Dropbox and such?
<Ablibu[m]>
Mm
hphs^ has joined #ipfs
<Ablibu[m]>
So it is more oriented to classic storage usage..
<postables[m]>
storj? it's basically decentralized S3, but the way storage nodes work in STORJ it wouldn't really jive with what you say you want your app to do. IPFS nodes are something I think are useful even if the lifetime of the node is fairly short lived, which would probbaly be what would happen with an app based system (node would be offline less than it was online, and taking your node offline wouldn't seriously impact the rest
<postables[m]>
of the network). Whereas with STORJ if you take your node offline, a lot of storage redistribution calculation has to happen
<Ablibu[m]>
Mm I see 👍
<Ablibu[m]>
But because we would talk about ipfs again as the only possible solution -> I need to make a pinning service
<Ablibu[m]>
*edit:* ~~But because we would talk about ipfs again as the only possible solution -> I need to make a pinning service~~ -> But because we are talking about ipfs again as the only possible solution -> I need to make a pinning service
<Ablibu[m]>
To actually make a revenue
<Ablibu[m]>
*edit:* ~~To actually make a revenue~~ -> To actually get a revenue
<postables[m]>
What is your app trying to do? you can incorporate pinning into the app
<Ablibu[m]>
I haven't still developed anything meaningful, but I was thinking about a transport app
jahhein has joined #ipfs
<Ablibu[m]>
Or a notification service.. I'm still in the infancy
<Ablibu[m]>
*edit:* ~~Or a notification service.. I'm still in the infancy~~ -> Or a notification service.. I'm still in the infancy anyway
<Ablibu[m]>
But I was also looking to which ways could help you know earn money
<Ablibu[m]>
To see how feasible it would be to implement in conjunction with any possible idea
<postables[m]>
i mean i would probably brainstorm a few ideas and prototype them, its hard to make anything of substance without a concrete idea to build off of
<postables[m]>
there's a lot of ways to build apps that are benficial to IPFS and the ecossytem as a whole so IMO i would try to target those things
jahhein has quit [Quit: Hope i didnt fall asleep with a beer in my hand... Zzz..]
<Ablibu[m]>
You are right
<Ablibu[m]>
I will see, thanks 👍
daMaestro has joined #ipfs
<dirkmc>
Hello, I’m in san francisco this weekend, are there any ipfs related events going on?
thomasan_ has joined #ipfs
Taoki has joined #ipfs
thomasan_ has quit [Remote host closed the connection]
<shoku[m]>
But you could see if people are down for something!
Nukien has joined #ipfs
Nukien has quit [Client Quit]
Nukien has joined #ipfs
belc has joined #ipfs
<belc>
I added a file to IPFS and lost the hash
<belc>
now what?
<belc>
do I have to re-add it?
<belc>
I also pinned it to my node
<seba-->
yeah readd it
<seba-->
you won't really make a duplicate
<seba-->
as that's impossible
<seba-->
if you have pinned it
<seba-->
you can check what you pinned
<seba-->
:)
<Swedneck>
there's also the --only-hash option
<belc>
I think ipfs pin ls is broken or something because it hangs for a lot of minutes, and I only have like 1300 files
<belc>
I think I'll use --only-hash
<belc>
it's a 8gb file but I can wait
<belc>
thanks guys
xcm has quit [Ping timeout: 245 seconds]
<postables[m]>
that sounds like `ipfs pin ls` behavior
hphs^ has quit [Ping timeout: 240 seconds]
hphs^ has joined #ipfs
<seba-->
"only 1300 files"
<seba-->
:> :>
xcm has joined #ipfs
<belc>
ah
<belc>
umm
<postables[m]>
I'm at 1.7mil lol 😂 `NumObjects: 1744407`
<belc>
so there is no local index of the files I have pinned
<belc>
I guess
<postables[m]>
if you want a speedy way to manage your pins, store them in a database
<postables[m]>
`ipfs pin ls` is the local index
<belc>
then why is it so slow?
<belc>
well whatever
<postables[m]>
Because you're initially reading from disk and then I believe walking the dog for everything to pull up even linked objects. `ipfs pin ls --type=recursive` should be a lot faster
<DarkDrgn2k[m]>
also ipfs pin add <- ugh was a HORRIBLE way to download stuff
<DarkDrgn2k[m]>
i did a wget of ipfs.io/ipfs/HASH and ipfs pin add HASH
<DarkDrgn2k[m]>
wget was done and ipfs pin add was at 15%
<Swedneck>
sad
<DarkDrgn2k[m]>
veryr
hphs^ has joined #ipfs
<DarkDrgn2k[m]>
not to mention that it used about 25% of the total bandwith it downloaded with uploads lol
<seba-->
yes
<seba-->
i think the implementation has issues
<seba-->
but i don't know why
zane has joined #ipfs
Guest36724 has joined #ipfs
Guest36724 has left #ipfs [#ipfs]
spinza has joined #ipfs
belc has quit []
Steverman has quit [Ping timeout: 245 seconds]
hphs^ has quit [Ping timeout: 250 seconds]
<postables[m]>
if the fs goes read only, it means you can't write to the disk so it matters a lot
<postables[m]>
how are you guys having issues with 50GB? I added a 1TB file
<postables[m]>
*edit:* ~~how are you guys having issues with 50GB? I added a 1TB file~~ -> how are you guys having issues with 50GB? I added a 1TB file and it worked, took like 12 hours to do the entire thing
<postables[m]>
if you're having issues with experimental features, open bug reports
<postables[m]>
DarkDrgn2K: keep in mind when you do wget, you're not going out to the network to download files, you're downloading from one source. In the case of using a gateway, you're most likely hitting the gateway's cache
<Swedneck>
well i'm using badger and that rabin chunker
<Swedneck>
but even without both it was painfully slow
<postables[m]>
what does `iostat` say when you're adding stuff?
<postables[m]>
thats pretty slow, i dont think IPFS is the culprit tbh
<postables[m]>
at that point its probably your CPU and maybe disk
stoopkid has quit [Quit: Connection closed for inactivity]
<postables[m]>
when I'm writting to a RAID6 iSCSI backed storage a file thats on an SSD, I've pushed it up to 75MB/s write speed and thats over a network, and writting 2 parity bits
<Swedneck>
hmm
<postables[m]>
*edit:* ~~when I'm writting to a RAID6 iSCSI backed storage a file thats on an SSD, I've pushed it up to 75MB/s write speed and thats over a network, and writting 2 parity bits~~ -> when I'm writting to a RAID6 iSCSI backed storage from a file thats on an SSD, I've pushed it up to 75MB/s write speed and thats over a network, and writting 2 parity bits
<Swedneck>
cpu is high usage but not that high
<postables[m]>
*edit:* ~~when I'm writting to a RAID6 iSCSI backed storage from a file thats on an SSD, I've pushed it up to 75MB/s write speed and thats over a network, and writting 2 parity bits~~ -> when I'm writting to a RAID6 iSCSI backed storage from a file thats on an SSD, I've pushed it up to 75MB/s write speed and thats over a network, and writting 2 parity bits without any optimizations
<postables[m]>
what kind of disks are you writting to? are they direct attached, are they networked? SSDs, HDDS?
<Swedneck>
2TB old HDD direct attach
<postables[m]>
RPM?
<Swedneck>
no clue
<postables[m]>
what CPU do you have?
<Swedneck>
CPU: AMD A8-6500 APU (4) @ 3.5GHz
<postables[m]>
is there a link where I can download the file to try adding it?
<postables[m]>
also the disk your folder is stored on, is it on the same HDD as the datastore?