aschmahmann changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.7.0 and js-ipfs 0.52.3 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of Conduct: https://git.io/vVBS0
hsn has quit [Ping timeout: 268 seconds]
mindCrime_ has joined #ipfs
LiftLeft has joined #ipfs
SirPHOENiX1 has joined #ipfs
teal has joined #ipfs
}ls{ has quit [Ping timeout: 240 seconds]
}ls{ has joined #ipfs
Mateon2 has joined #ipfs
Mateon1 has quit [Ping timeout: 252 seconds]
Mateon2 is now known as Mateon1
ipfs-stackbot has quit [Remote host closed the connection]
ipfs-stackbot has joined #ipfs
chiui has joined #ipfs
asimons04[m] has left #ipfs ["User left"]
chiuii has quit [Ping timeout: 276 seconds]
russ152 has joined #ipfs
chiui has quit [Ping timeout: 276 seconds]
devdevgoat has quit [Ping timeout: 240 seconds]
jcea has quit [Ping timeout: 276 seconds]
teal has quit [Quit: food]
jrt is now known as Guest65265
Guest65265 has quit [Killed (tepper.freenode.net (Nickname regained by services))]
jrt has joined #ipfs
dpl has quit [Ping timeout: 240 seconds]
xentec_ has quit [Quit: memento mori]
xentec has joined #ipfs
xentec has quit [Remote host closed the connection]
xentec has joined #ipfs
LiftLeft has quit [Ping timeout: 246 seconds]
<Discordian[m]>
<owocean[m] "i have an alarmingly low number "> How many peers do you have? O:
KempfCreative has joined #ipfs
gimzmoe has joined #ipfs
LiftLeft has joined #ipfs
<owocean[m]>
at this moment
<owocean[m]>
251
<owocean[m]>
earlier it was like only 100
reit has quit [Quit: Leaving]
<Discordian[m]>
I have 597 rn, do you have port 4001 forwarded? I find it helps a lot to have it forwarded.
chachasmooth has quit [Ping timeout: 250 seconds]
chachasmooth has joined #ipfs
Guest15978 has quit [Changing host]
Guest15978 has joined #ipfs
Guest15978 has joined #ipfs
<owocean[m]>
i do not, i can try it
jaiya has joined #ipfs
gimzmoe has quit [Ping timeout: 265 seconds]
otakuboi has joined #ipfs
Guest15978 has quit [Disconnected by services]
Guest15978 has joined #ipfs
Guest15978 has quit [Changing host]
Guest15978 has joined #ipfs
Guest15978 has joined #ipfs
Guest15978 has quit [Disconnected by services]
Guest15978 has joined #ipfs
Guest15978 has quit [Changing host]
Guest15978 has joined #ipfs
Guest15978 has joined #ipfs
Guest15978 is now known as AlwaysLivid[m]
chachasmooth has quit [Ping timeout: 250 seconds]
chachasmooth has joined #ipfs
otakuboi has quit [Quit: Leaving]
opa has joined #ipfs
opa7331 has quit [Ping timeout: 276 seconds]
KempfCreative has quit [Ping timeout: 240 seconds]
Arwalk has quit [Read error: Connection reset by peer]
p3n has joined #ipfs
Arwalk has joined #ipfs
mindCrime_ has joined #ipfs
mindCrime_ has quit [Ping timeout: 252 seconds]
Nact has quit [Quit: Konversation terminated!]
russ152 has joined #ipfs
reit has joined #ipfs
devdevgoat has quit [Ping timeout: 246 seconds]
<Piddle>
Anyone using .eth domains with IPFS use an /ipns/ content address? I set my site up with it and it seems to load fast, but once its no longer cached I wonder if the same will be true.
<Piddle>
I typically setup a DNSLink on my domains, however I didn't see the option with the Ethereum domain- does Ethereum's DNS do the DNSLink automatically if anyone knows? Thank you
<aschmahmann>
As I mentioned in the issue I think what's going on there is that there was some corruption in your blockstore.
<aschmahmann>
`vole bitswap check <cid> <multiaddress>` is the command for checking if a peer has a given CID
dpl has joined #ipfs
hsn has joined #ipfs
Piejacker875 has quit [Remote host closed the connection]
mindCrime_ has joined #ipfs
Pie-jacker875 has joined #ipfs
Arwalk has quit [Read error: Connection reset by peer]
Arwalk has joined #ipfs
mindCrime_ has quit [Ping timeout: 252 seconds]
einyx has quit [Ping timeout: 260 seconds]
niv has quit [Ping timeout: 250 seconds]
einyx has joined #ipfs
SirPHOENiX1 has quit [Ping timeout: 240 seconds]
russ152 has joined #ipfs
devdevgoat has quit [Ping timeout: 260 seconds]
grinja has quit [Remote host closed the connection]
SirPHOENiX1 has joined #ipfs
grinja has joined #ipfs
niv has joined #ipfs
opal has quit [Ping timeout: 240 seconds]
mamu has quit [Ping timeout: 252 seconds]
opal has joined #ipfs
zeden has joined #ipfs
moony has quit [Quit: Bye!]
iczero has quit [Quit: rip]
coniptor has quit [Ping timeout: 260 seconds]
moony has joined #ipfs
iczero has joined #ipfs
coniptor has joined #ipfs
coniptor has quit [Ping timeout: 252 seconds]
justanotherdude has quit [Quit: RAGEQUIT]
justanotherdude has joined #ipfs
coniptor has joined #ipfs
justanotherdude has quit [Quit: RAGEQUIT]
supercoven has quit [Ping timeout: 268 seconds]
Meli has quit [Quit: After 8w 3d 19h 1m 8s of wasteful lurking, Meli's brain 63gf4u1ted! X_x]
Meli has joined #ipfs
<kallisti5[m]>
ah. ok. never heard if vole tool.
<kallisti5[m]>
Yeah, repo has 3 CID's corrupted. How does that even happen? This machine really hasn't had any issues. I created it fresh 2-3 days ago.
<kallisti5[m]>
Is there a way to figure out what files point at that CID so I can re-add just the impacted CID?
<kallisti5[m]>
* ah. ok. never heard of the vole tool.
<RomeSilvanus[m]>
Shouldn't gc take carte of that?
<RomeSilvanus[m]>
* Shouldn't gc take care of that?
KempfCreative has joined #ipfs
<aschmahmann>
If the data is pinned as opposed to only being protected via MFS then `ipfs pin verify` should tell you which pins are busted which might help you out. From there tools like `ipfs ls` could be of use
<aschmahmann>
Not sure why the corruption happened, it could be a problem anywhere from go-ipfs, to badger (the datastore I noticed you were using in the issue), to your OS or your HDD. If you have any ideas as to when/why it occurred (e.g. the machine or go-ipfs crashed while you were adding some files) that would be helpful info to add onto the issue you filed.
<aschmahmann>
One place go-ipfs can save you here is if you have a copy of the data elsewhere then after the bad blocks are cleared out it'll automatically try and fetch the blocks from anyone it can find who has them.
<aschmahmann>
If the data is only protected via MFS and you want to use the CLI tools (instead of writing Go code that accesses the IPFS repo) then it's doable too, but might need a bit of elbow grease. Lmk and I can try and give some pointers.
<aschmahmann>
The commands that'll likely be of use are `ipfs refs -r` and `ipfs files ls -l` along with the `--enc=json` flags if you're like me and would prefer to work with JSON then text output for anything potentially complicated.
Nact has quit [Quit: Konversation terminated!]
tech_exorcist has quit [Remote host closed the connection]
<kallisti5[m]>
hm, getting "merkledag: not found" after 185 CID's with ```ipfs refs -r (CID from MFS)```
<kallisti5[m]>
* hm, getting "merkledag: not found" after 185 CID's with `ipfs refs -r (A CID from MFS)`
<kallisti5[m]>
the former 185 CID's don't match the corrupt CID's from the repo check
<kallisti5[m]>
Troubleshooting stuff stuff is painful ^_^
arcatech has joined #ipfs
<kallisti5[m]>
I would just regenerate the whole 22 GiB repo, but then I lose the whole MFS since I can't set the root MFS cid.. which means i'm starting over. We have ~400-500GiB to get into IPFS... I feel like we're going to have a bad time with the tooling today âšī¸
<aschmahmann>
I suspect you could do more poking around there (e.g. walking upwards from the bottom and doing `ipfs refs -r` on them to see which ones are closest to the problem).
<aschmahmann>
using `ipfs files ls` before piping into `ipfs refs -r` might be more tedious but will likely give you a better idea as to what's wrong
<aschmahmann>
Although last I checked badger imports data pretty quickly so doing a reimport doesn't seem like a bad idea.
pecastro has quit [Ping timeout: 252 seconds]
<kallisti5[m]>
still waiting on the ```ipfs block rm (BAD CID)``` lol
musalbas has quit [Remote host closed the connection]
<aschmahmann>
Does `ipfs repo gc` take a long time on your machine? `ipfs block rm` has to do something similar at the moment in order to make sure your block isn't actually needed somewhere else.
<aschmahmann>
I'm not sure about the shape of your data (how many objects, how many pins, the amount of overlap in data, etc.), but with 22GB of data I wouldn't suspect it to be that slow. I'm a little concerned that something else is going on with your repo/machine then just a few missing blocks (are they all missing, or do some just have garbage data).
<kallisti5[m]>
ok. Before I re-add 22GiB again.. is there a better repo format to use?
<kallisti5[m]>
* ok. Before I re-add 22GiB again.. is there a better repo format to use for large amounts of data?
<aschmahmann>
badger is the fastest of the bulitin datastores and flatfs (the default) is slower but due to the lack of doing anything too fancy tends to be the most reliable.
<kallisti5[m]>
speed's going to be important. the 22GiB is made up of a lot of files
<aschmahmann>
I guess my 2c is that generally backing up data is good and needs to happen anyway because nothing can protect you from a random HDD failure, so I'd choose Badger but that's just me đ
hsn has quit [Ping timeout: 240 seconds]
<kallisti5[m]>
I haven't even gotten to trying to put our ports repos into IPFS. That's 151GiB across ~13k files
<aschmahmann>
i.e. having multiple IPFS nodes (and perhaps using ipfs-cluster to sync them) or backing up the repos periodically
<aschmahmann>
For context I've added the 1TiB Ubuntu repo to IPFS using badger and it was about as fast as copy-pasting the folder
<aschmahmann>
* For context I've added the 1TiB Ubuntu repo to go-ipfs using badger and it was about as fast as copy-pasting the folder