#ipfs on 2019-01-02 — irc logs at freenode.irclog.whitequark.org

00:39 <postables[m]> swedneck: turn on debugging for your daemon, run the ipfs add command with debugging. What issues are you experiencing? I've had isuses in the past with adding and pinning a file at the same time, but that was fixed in release 0.4.18 i believe

00:39 <postables[m]> *edit:* ~~swedneck: turn on debugging for your daemon, run the ipfs add command with debugging. What issues are you experiencing? I've had isuses in the past with adding and pinning a file at the same time, but that was fixed in release 0.4.18 i believe~~ -> swedneck:swedneck.xyz : turn on debugging for your daemon, run the ipfs add command with debugging. What issues are you experiencing? I've had isuses in the past with

00:39 <postables[m]> adding and pinning a file at the same time, but that was fixed in release 0.4.18 i believe

00:40 <Swedneck> it's just really really really painfully slow

00:40 <Swedneck> over 10h to add 60 gigs of data

00:41 <Swedneck> (which takes like, 10 min on my desktop)

00:42 <Swedneck> how do i turn on debugging?

00:52 <postables[m]> `ipfs -D <your-command>` so `ipfs -D daemon` or `ipfs -D add`

00:52 <postables[m]> I tkae it you're not doing this test on a desktop? where are you testing, what are the sepcs of th emachine you're testing this on, what are the sepcs of your desktop

00:53 <Swedneck> it's on a desktop i use as a server, it's an AMD A8-6500 APU (4) @ 3.5GHz, 20+ GB of ram, and a 2TB hard drive

00:56 <Swedneck> do i need to let it run `ìpfs add` to completion?

01:30 <postables[m]> I remember we talked about hash on read and bloom filters awhile ago. Do you have hash on read enabled on the server? IF so, the CPU is probs your bottleneck

01:30 <Swedneck> i do not

01:31 <postables[m]> hmm 🤔 what does `htop` show your core utilization is like when you're running the add

01:32 <Swedneck> fairly high, but not above 90% on any core

01:32 <postables[m]> hmm

01:33 <postables[m]> can you try running a pin on your server for the hash if it can reach your node running on your desktop?

01:33 <postables[m]> *edit:* ~~can you try running a pin on your server for the hash if it can reach your node running on your desktop?~~ -> can you try running a pin on your server for the hash if it can reach your node running on your desktop? could help isolate whether its a disk level issue perhaps

01:33 <Swedneck> sure, they're on the same LAN btw

01:34 <postables[m]> cool that should make for some easy debugging then. What're the specs like on your desktop which you aren't having the issue on?

01:35 <Swedneck> ryzen 5 1600, 16GB ram, OS is on an ssd but the repo is on HDD (and i'm using --nocopy)

01:36 <Swedneck> oh boy right, another issue

01:36 <Swedneck> `Error: pin: open /home/ipfs/fdroid-mirror/repo/a2dp.Vol_121.apk.asc: no such file or directory`

01:37 <Swedneck> i had tried to add the directory before using --nocopy on the server, and now i can't seem to unpin that file

01:41 <postables[m]> hmm could possibly be a CPU issue. Although if this is happening repeatedly with your usage of `--nocopy` sounds like you're running into a bug with an experimental feature. Might be worth opening a bug report on the ipfs repo

01:41 <postables[m]> *edit:* ~~hmm could possibly be a CPU issue. Although if this is happening repeatedly with your usage of `--nocopy` sounds like you're running into a bug with an experimental feature. Might be worth opening a bug report on the ipfs repo~~ -> hmm could possibly be a CPU issue but experimental feature bug sounds more likely. Although if this is happening repeatedly with your usage of `--nocopy` sounds like you're running

01:41 <postables[m]> into a bug with an experimental feature. Might be worth opening a bug report on the ipfs repo

01:41 <postables[m]> Do you have this issue not using `--nocopy`

01:42 <Swedneck> well the issue was caused by using it, i think

01:42 <Swedneck> i'm not using --nocopy anymore

01:47 <postables[m]> strange, might want to open a bug report for the `no such file directory` issue. If possible, I would try using a fresh repo that hasn't been used with experimental features. May also want to try `ipfs repo verify`. I would also double check that on the new node, you're repo max GB limit isn't set so that adding the 60GB file would reach the limit.

01:47 <postables[m]> Are you using flatfs?

01:47 <postables[m]> *edit:* ~~strange, might want to open a bug report for the `no such file directory` issue. If possible, I would try using a fresh repo that hasn't been used with experimental features. May also want to try `ipfs repo verify`. I would also double check that on the new node, you're repo max GB limit isn't set so that adding the 60GB file would reach the limit.

01:47 <postables[m]> Are you using flatfs?~~ -> strange, might want to open a bug report for the `no such file directory` issue. If possible, I would try using a fresh repo that hasn't been used with experimental features. May also want to try `ipfs repo verify`. I would also double check that on the new node, you're repo max GB limit isn't set so that adding the 60GB file would reach the limit.

01:47 <postables[m]> Are you using flatfs for your repo?

01:48 <Swedneck> the server is using badger

01:48 <Swedneck> i'm gonna try just wiping the directory i'm adding from and re-downloading the contents first

01:48 <Swedneck> i could just have fucked up the files somehow

01:53 <Swedneck> there were definitely too many files at least, it should've been 48 gigs lol

01:57 <Swedneck> hmm, my server is really slow at downloading stuff as well..

01:57 <postables[m]> re-downloading from IPFS?

01:59 <Swedneck> no, rsync

01:59 <Swedneck> cpu usage is around 50% for all cores still

02:00 <Swedneck> 5200 rpm drives aren't that much slower than 7200 rpm ones, right?

02:32 <postables[m]> for this kind of stuff, I believe they are

02:33 <Swedneck> hmm

02:36 <Swedneck> think a hybrid drive would be worth it for this?

02:37 <postables[m]> somewhat, you should be fine with just a 7.2K RPM one

02:37 <postables[m]> you can use 10KRPM if you need a little more speed without breaking the bank

02:39 <Swedneck> well 1TB hybrid is 60 bucks

02:40 <Swedneck> and no 10k available :(

02:40 <postables[m]> if 1TB is 7.2K rpm u should be fine

02:48 <Swedneck> i'm actually not sure it's a 5200 rpm drive, but i guess there's no other explanation?

03:25 <DarkDrgn2k[m]> HDD (not ssd) out-preform on sequencial writes

03:25 <DarkDrgn2k[m]> so if you are writing one large file a 7.2k rpm drive will be quite a bit faster then the 5.2 one

03:25 <DarkDrgn2k[m]> (they will even outpreform ssd usually)

03:26 <DarkDrgn2k[m]> if its random read/write i donno how much differnt it would be...ive only used those for long term storage type setups

03:39 <Swedneck> Well it's mostly a bunch of apks

08:08 <xialvjun[m]> Is there any document about IPFS gateway api ?

09:31 <aarshkshah1992> xialvjun https://docs.ipfs.io/reference/api/http/

09:33 <xialvjun[m]> I mean the gateway api, not the http api. http://localhost:8080/ipfs/<path> and it can get many type of things, I want to know it

09:37 <xialvjun[m]> aarshkshah1992:

10:51 <grawity> hey, where can I find a table of multihash prefixes? like, what does '12D...' mean, as opposed to the usual 'QmN...'

10:57 <olizilla> grawity: the raw table lives here https://github.com/multiformats/multicodec/blob/master/table.csv

10:57 <olizilla> but you might find this helpful too https://github.com/ipld/cid#how-does-it-work

14:18 <ikari`> o hai

15:09 <abbiya> hello

15:09 <abbiya> https://github.com/libp2p/js-libp2p-webrtc-star

15:10 <abbiya> I did not understand anything that is written in the above project

15:10 <abbiya> i am looking for ways to discover webrtc peers based on topics

15:11 <abbiya> can anyone please help understand my problem and lead to answers>?

17:13 <seba-> hm

17:14 <seba-> are there any charts

17:14 <seba-> on the growth of IPFS?

17:30 <ikari`> from other stupid questions: should my peer count be always 0?:P

20:53 <postables[m]> ikari`: if your peercount is always 0 then it likely means you have a firewall issue

23:29 <Kolonka[m]> https://trends.google.com/trends/explore?date=all&q=ipfs

23:29 <Kolonka[m]> very interesting to see china on there

23:32 <Swedneck> wtf hong kong