aschmahmann changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.7.0 and js-ipfs 0.52.3 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of Conduct: https://git.io/vVBS0
<RomeSilvanus[m]>
I’ll try it out when I’m back home in a few hours
<RomeSilvanus[m]>
I’ll try it out when I’m back home in a few hours
<RomeSilvanus[m]>
I’ll try it out when I’m back home in a few hours
Mateon2 has joined #ipfs
Mateon2 has joined #ipfs
Mateon2 has joined #ipfs
<eleitl[m]>
Discordian: Looks great. What are massive quantities in terms of numbers?
<eleitl[m]>
Discordian: Looks great. What are massive quantities in terms of numbers?
<eleitl[m]>
Discordian: Looks great. What are massive quantities in terms of numbers?
<Discordian[m]>
I tested with about 18k files in a directory
<Discordian[m]>
I tested with about 18k files in a directory
<Discordian[m]>
I tested with about 18k files in a directory
jcea has joined #ipfs
jcea has joined #ipfs
jcea has joined #ipfs
<Discordian[m]>
The example I linked there is the osrs wiki
<Discordian[m]>
The example I linked there is the osrs wiki
<Discordian[m]>
The example I linked there is the osrs wiki
<eleitl[m]>
Would a million scale? Not all in the same directory, of course.
<eleitl[m]>
Would a million scale? Not all in the same directory, of course.
<eleitl[m]>
Would a million scale? Not all in the same directory, of course.
Mateon1 has quit [Ping timeout: 265 seconds]
Mateon1 has quit [Ping timeout: 265 seconds]
Mateon1 has quit [Ping timeout: 265 seconds]
Mateon2 is now known as Mateon1
Mateon2 is now known as Mateon1
Mateon2 is now known as Mateon1
<Discordian[m]>
It should, as long as it works on IPFS, it should work with ipfs-sync
<Discordian[m]>
It should, as long as it works on IPFS, it should work with ipfs-sync
<Discordian[m]>
It should, as long as it works on IPFS, it should work with ipfs-sync
<Discordian[m]>
It just uses `add` to add the files, then does `files cp` to put it on your MFS. Then it recursively updates the pins using the root directory, and produces an IPNS key associated with it.
<Discordian[m]>
It just uses `add` to add the files, then does `files cp` to put it on your MFS. Then it recursively updates the pins using the root directory, and produces an IPNS key associated with it.
<Discordian[m]>
It just uses `add` to add the files, then does `files cp` to put it on your MFS. Then it recursively updates the pins using the root directory, and produces an IPNS key associated with it.
<eleitl[m]>
It's too bad filestore is not yet official, since it seems to work quite well in practice.
<eleitl[m]>
It's too bad filestore is not yet official, since it seems to work quite well in practice.
<eleitl[m]>
It's too bad filestore is not yet official, since it seems to work quite well in practice.
<Discordian[m]>
Yeah this newest update will actually have filestore/nocopy support
<Discordian[m]>
Yeah this newest update will actually have filestore/nocopy support
<Discordian[m]>
Yeah this newest update will actually have filestore/nocopy support
<Discordian[m]>
It's committed, I just need to do the release
<Discordian[m]>
It's committed, I just need to do the release
<Discordian[m]>
It's committed, I just need to do the release
<RomeSilvanus[m]>
Eventually all my folders I’d like to add should be around 30 million files
<RomeSilvanus[m]>
Eventually all my folders I’d like to add should be around 30 million files
<RomeSilvanus[m]>
Eventually all my folders I’d like to add should be around 30 million files
<Discordian[m]>
Certainly be the biggest test yet, it'll take a long time to add, but at least it'll do it in the background
<Discordian[m]>
Certainly be the biggest test yet, it'll take a long time to add, but at least it'll do it in the background
<Discordian[m]>
Certainly be the biggest test yet, it'll take a long time to add, but at least it'll do it in the background
<Discordian[m]>
I wonder how the leveldb database will handle 30mil files
<Discordian[m]>
I wonder how the leveldb database will handle 30mil files
<Discordian[m]>
I wonder how the leveldb database will handle 30mil files
<Discordian[m]>
It should be fine, but we'll see
<Discordian[m]>
It should be fine, but we'll see
<Discordian[m]>
It should be fine, but we'll see
<RomeSilvanus[m]>
Inb4 another data enter fire
<RomeSilvanus[m]>
Inb4 another data enter fire
<RomeSilvanus[m]>
Inb4 another data enter fire
<RomeSilvanus[m]>
* Inb4 another datacenter fire
<RomeSilvanus[m]>
* Inb4 another datacenter fire
<RomeSilvanus[m]>
* Inb4 another datacenter fire
<eleitl[m]>
Thanks, Discordian. I hope 0.3.0 is soon out.
<eleitl[m]>
Thanks, Discordian. I hope 0.3.0 is soon out.
<eleitl[m]>
Thanks, Discordian. I hope 0.3.0 is soon out.
<Discordian[m]>
No problem! I'll likely release today, I just woke up though, so I don't want to release if I'm forgetting something I wanted in it
<Discordian[m]>
No problem! I'll likely release today, I just woke up though, so I don't want to release if I'm forgetting something I wanted in it
<Discordian[m]>
No problem! I'll likely release today, I just woke up though, so I don't want to release if I'm forgetting something I wanted in it
dqx_ has quit [Remote host closed the connection]
dqx_ has quit [Remote host closed the connection]
dqx_ has quit [Remote host closed the connection]
koo6 has joined #ipfs
koo6 has joined #ipfs
koo6 has joined #ipfs
Icebear2 has joined #ipfs
Icebear2 has joined #ipfs
Icebear2 has joined #ipfs
bonz060_ has joined #ipfs
bonz060_ has joined #ipfs
bonz060_ has joined #ipfs
RoseBus has joined #ipfs
RoseBus has joined #ipfs
RoseBus has joined #ipfs
koo6 has quit [Ping timeout: 256 seconds]
koo6 has quit [Ping timeout: 256 seconds]
koo6 has quit [Ping timeout: 256 seconds]
LinusCDE2 has quit [Quit: I hate my DLan!]
LinusCDE2 has quit [Quit: I hate my DLan!]
LinusCDE2 has quit [Quit: I hate my DLan!]
mowcat has joined #ipfs
mowcat has joined #ipfs
mowcat has joined #ipfs
<Discordian[m]>
Rome Silvanus: Do you think this issue would effect you heavily? IE: Do you think you'll be removing directories, then re-adding them with the same content? https://github.com/TheDiscordian/ipfs-sync/issues/18
<Discordian[m]>
If not, I'll release now, and fix for v0.3.1. If so, I'll fix that bug first.
<Discordian[m]>
If not, I'll release now, and fix for v0.3.1. If so, I'll fix that bug first.
<Discordian[m]>
Rome Silvanus: Do you think this issue would effect you heavily? IE: Do you think you'll be removing directories, then re-adding them with the same content? https://github.com/TheDiscordian/ipfs-sync/issues/18
<Discordian[m]>
If not, I'll release now, and fix for v0.3.1. If so, I'll fix that bug first.
<Discordian[m]>
Rome Silvanus: Do you think this issue would effect you heavily? IE: Do you think you'll be removing directories, then re-adding them with the same content? https://github.com/TheDiscordian/ipfs-sync/issues/18
Icebear2 has quit [Remote host closed the connection]
Icebear2 has quit [Remote host closed the connection]
Icebear2 has quit [Remote host closed the connection]
fling has quit [Ping timeout: 260 seconds]
fling has quit [Ping timeout: 260 seconds]
fling has quit [Ping timeout: 260 seconds]
koo6 has joined #ipfs
koo6 has joined #ipfs
koo6 has joined #ipfs
<RomeSilvanus[m]>
Sometimes when reorganize my archive
<RomeSilvanus[m]>
Sometimes when reorganize my archive
<RomeSilvanus[m]>
Sometimes when reorganize my archive
<RomeSilvanus[m]>
But it will mostly be moving them to a different folder or renaming their parent folder
<RomeSilvanus[m]>
But it will mostly be moving them to a different folder or renaming their parent folder
<RomeSilvanus[m]>
But it will mostly be moving them to a different folder or renaming their parent folder
<RomeSilvanus[m]>
It’s not that often though
<RomeSilvanus[m]>
It’s not that often though
<RomeSilvanus[m]>
It’s not that often though
<Discordian[m]>
Hmm renames aren't handled very gracefully right now, you might want to wait for that to get more stable.
<Discordian[m]>
Hmm renames aren't handled very gracefully right now, you might want to wait for that to get more stable.
<Discordian[m]>
Hmm renames aren't handled very gracefully right now, you might want to wait for that to get more stable.
<Discordian[m]>
Mostly because renames will trigger rehashes because fsnotify will tell me something got renamed, but not what it was renamed to.
<Discordian[m]>
Mostly because renames will trigger rehashes because fsnotify will tell me something got renamed, but not what it was renamed to.
<Discordian[m]>
Mostly because renames will trigger rehashes because fsnotify will tell me something got renamed, but not what it was renamed to.
Nukien2 has joined #ipfs
Nukien2 has joined #ipfs
Nukien2 has joined #ipfs
<RomeSilvanus[m]>
I can live with that for now
<RomeSilvanus[m]>
I can live with that for now
<RomeSilvanus[m]>
I can live with that for now
<Discordian[m]>
Okay cool 🙂. I'll get v0.3.0 prepped in 5mins or so then
<Discordian[m]>
Okay cool 🙂. I'll get v0.3.0 prepped in 5mins or so then
<Discordian[m]>
Okay cool 🙂. I'll get v0.3.0 prepped in 5mins or so then
<RomeSilvanus[m]>
Btw "2. Copy the sample config file `config.json.sample`" this file should be in the binary archive if you include it in the readme, it's only in the git tree right now
<RomeSilvanus[m]>
Btw "2. Copy the sample config file `config.json.sample`" this file should be in the binary archive if you include it in the readme, it's only in the git tree right now
<RomeSilvanus[m]>
Btw "2. Copy the sample config file `config.json.sample`" this file should be in the binary archive if you include it in the readme, it's only in the git tree right now
<Discordian[m]>
Is it not bundled? Thanks, I'll fix that
<Discordian[m]>
Is it not bundled? Thanks, I'll fix that
<Discordian[m]>
Is it not bundled? Thanks, I'll fix that
<Discordian[m]>
I'll bundle it in the package. The binary will create a sample too if you point it somewhere to generate it
<Discordian[m]>
I'll bundle it in the package. The binary will create a sample too if you point it somewhere to generate it
<Discordian[m]>
I'll bundle it in the package. The binary will create a sample too if you point it somewhere to generate it
<Discordian[m]>
Fixed, will be bundled in next release (v0.3.1), thanks for the report!
<Discordian[m]>
Fixed, will be bundled in next release (v0.3.1), thanks for the report!
<Discordian[m]>
Fixed, will be bundled in next release (v0.3.1), thanks for the report!
<RomeSilvanus[m]>
Also is "~/.config/systemd/user/" actually correct with user or should I replace it with my username? I don't really into systemd
<RomeSilvanus[m]>
Also is "~/.config/systemd/user/" actually correct with user or should I replace it with my username? I don't really into systemd
<RomeSilvanus[m]>
Also is "~/.config/systemd/user/" actually correct with user or should I replace it with my username? I don't really into systemd
<Discordian[m]>
Yeah it's correct because it's a user service
<Discordian[m]>
Yeah it's correct because it's a user service
<Discordian[m]>
Yeah it's correct because it's a user service
<Discordian[m]>
systemd you have user and system services
<Discordian[m]>
systemd you have user and system services
<Discordian[m]>
systemd you have user and system services
<RomeSilvanus[m]>
Okay then !
<RomeSilvanus[m]>
Okay then !
<RomeSilvanus[m]>
Okay then !
<RomeSilvanus[m]>
Could've been that you use 'user' as a generic term
<RomeSilvanus[m]>
Could've been that you use 'user' as a generic term
<RomeSilvanus[m]>
Could've been that you use 'user' as a generic term
<Discordian[m]>
No, I'll check the ordering, possible it's doing an operation (like crawling dirs to make a list)
<Discordian[m]>
No, I'll check the ordering, possible it's doing an operation (like crawling dirs to make a list)
<Discordian[m]>
No, I'll check the ordering, possible it's doing an operation (like crawling dirs to make a list)
<kaepora>
OK now it resolved very quickly. ???
<kaepora>
OK now it resolved very quickly. ???
<kaepora>
OK now it resolved very quickly. ???
<RomeSilvanus[m]>
Crawling thorugh everything probably will take a few hours
<RomeSilvanus[m]>
Crawling thorugh everything probably will take a few hours
<RomeSilvanus[m]>
Crawling thorugh everything probably will take a few hours
<Discordian[m]>
Yeah, I just looked at the logic, it'll build the internal DB, no output (I'll change that), then it'll move it all over to IPFS
<Discordian[m]>
Yeah, I just looked at the logic, it'll build the internal DB, no output (I'll change that), then it'll move it all over to IPFS
<Discordian[m]>
Yeah, I just looked at the logic, it'll build the internal DB, no output (I'll change that), then it'll move it all over to IPFS
<RomeSilvanus[m]>
A bit more verbose would be nice. Processes that don't update their output are never nice.
<RomeSilvanus[m]>
A bit more verbose would be nice. Processes that don't update their output are never nice.
<RomeSilvanus[m]>
A bit more verbose would be nice. Processes that don't update their output are never nice.
<Discordian[m]>
<kaepora "OK now it resolved very quickly."> Are your 2 local nodes connected to eachother correctly? Pubsub should be quite speedy
<Discordian[m]>
<kaepora "OK now it resolved very quickly."> Are your 2 local nodes connected to eachother correctly? Pubsub should be quite speedy
<Discordian[m]>
<kaepora "OK now it resolved very quickly."> Are your 2 local nodes connected to eachother correctly? Pubsub should be quite speedy
<Discordian[m]>
<RomeSilvanus[m] "A bit more verbose would be nice"> Haha yeah, it wasn't an issue until now (millions of files is certainly a nice benchmark)
<Discordian[m]>
<RomeSilvanus[m] "A bit more verbose would be nice"> Haha yeah, it wasn't an issue until now (millions of files is certainly a nice benchmark)
<Discordian[m]>
<RomeSilvanus[m] "A bit more verbose would be nice"> Haha yeah, it wasn't an issue until now (millions of files is certainly a nice benchmark)
<Discordian[m]>
FWIW the internal DB is much faster than the IPFS adding process, and that process is extremely verbose
<Discordian[m]>
FWIW the internal DB is much faster than the IPFS adding process, and that process is extremely verbose
<Discordian[m]>
FWIW the internal DB is much faster than the IPFS adding process, and that process is extremely verbose
<RomeSilvanus[m]>
I think it's just important for any software project really to have the option of displaying what's actually going on. Sometimes processes tend to hang and you'll never find out why.
<RomeSilvanus[m]>
I think it's just important for any software project really to have the option of displaying what's actually going on. Sometimes processes tend to hang and you'll never find out why.
<RomeSilvanus[m]>
I think it's just important for any software project really to have the option of displaying what's actually going on. Sometimes processes tend to hang and you'll never find out why.
<RomeSilvanus[m]>
Maybe also consider adding a flag such as --status to the daemon so one can randomly check up on it.
<RomeSilvanus[m]>
Maybe also consider adding a flag such as --status to the daemon so one can randomly check up on it.
<RomeSilvanus[m]>
Maybe also consider adding a flag such as --status to the daemon so one can randomly check up on it.
<RomeSilvanus[m]>
or maybe --lig
<RomeSilvanus[m]>
or maybe --lig
<RomeSilvanus[m]>
or maybe --lig
<RomeSilvanus[m]>
* or maybe --log
<RomeSilvanus[m]>
* or maybe --log
<RomeSilvanus[m]>
* or maybe --log
<aschmahmann>
kaepora: I would double check that your nodes are actually reachable over the public internet (i.e. not stuck behind a NAT). You can use any online port checker tool to see if the public IP address+port combinations spit out by `ipfs id` are reachable
<aschmahmann>
kaepora: I would double check that your nodes are actually reachable over the public internet (i.e. not stuck behind a NAT). You can use any online port checker tool to see if the public IP address+port combinations spit out by `ipfs id` are reachable
<aschmahmann>
kaepora: I would double check that your nodes are actually reachable over the public internet (i.e. not stuck behind a NAT). You can use any online port checker tool to see if the public IP address+port combinations spit out by `ipfs id` are reachable
gmoro has joined #ipfs
gmoro has joined #ipfs
gmoro has joined #ipfs
<Discordian[m]>
> <@khaosfox:matrix.org> I think it's just important for any software project really to have the option of displaying what's actually going on. Sometimes processes tend to hang and you'll never find out why.
<Discordian[m]>
> <@khaosfox:matrix.org> I think it's just important for any software project really to have the option of displaying what's actually going on. Sometimes processes tend to hang and you'll never find out why.
<Discordian[m]>
> <@khaosfox:matrix.org> I think it's just important for any software project really to have the option of displaying what's actually going on. Sometimes processes tend to hang and you'll never find out why.
<Discordian[m]>
Yeah it'll have an HTTP API to get nicer info on demand, and I'll make it explain it's building the internal db too while it does that. It's currently in the alpha stages (not on v1 yet)
<Discordian[m]>
Yeah it'll have an HTTP API to get nicer info on demand, and I'll make it explain it's building the internal db too while it does that. It's currently in the alpha stages (not on v1 yet)
<Discordian[m]>
> Maybe also consider adding a flag such as --status to the daemon so one can randomly check up on it.
<Discordian[m]>
> Maybe also consider adding a flag such as --status to the daemon so one can randomly check up on it.
<Discordian[m]>
Yeah it'll have an HTTP API to get nicer info on demand, and I'll make it explain it's building the internal db too while it does that. It's currently in the alpha stages (not on v1 yet)
<Discordian[m]>
> Maybe also consider adding a flag such as --status to the daemon so one can randomly check up on it.
<Discordian[m]>
Definitely appreciate the feedback
<Discordian[m]>
Definitely appreciate the feedback
<Discordian[m]>
Definitely appreciate the feedback
<RomeSilvanus[m]>
Neat-o
<RomeSilvanus[m]>
Neat-o
<RomeSilvanus[m]>
Neat-o
<Discordian[m]>
Honestly it does that step so fast usually, I forgot how it worked
<Discordian[m]>
Honestly it does that step so fast usually, I forgot how it worked
<Discordian[m]>
Honestly it does that step so fast usually, I forgot how it worked
<kaepora>
aschmahmann: Thanks, I'm checking
<kaepora>
aschmahmann: Thanks, I'm checking
<kaepora>
aschmahmann: Thanks, I'm checking
<kaepora>
I may have asked this yesterday and I'm sorry if it's a stupid question, but how can I convert a hash that starts with "Qm" to one that starts with "1"?
<kaepora>
I may have asked this yesterday and I'm sorry if it's a stupid question, but how can I convert a hash that starts with "Qm" to one that starts with "1"?
<kaepora>
I may have asked this yesterday and I'm sorry if it's a stupid question, but how can I convert a hash that starts with "Qm" to one that starts with "1"?
<Discordian[m]>
I'm wondering if maybe the way I handle memory for sending files to the daemon will break something too if you have massive files and low memory, we'll see.
<Discordian[m]>
I'm wondering if maybe the way I handle memory for sending files to the daemon will break something too if you have massive files and low memory, we'll see.
<Discordian[m]>
I'm wondering if maybe the way I handle memory for sending files to the daemon will break something too if you have massive files and low memory, we'll see.
<RomeSilvanus[m]>
What actually happens when you change a lot of files and folder while ipfs-sync isn't running, does it need to rescan everything or does inotify tell it everything that happened?
<RomeSilvanus[m]>
What actually happens when you change a lot of files and folder while ipfs-sync isn't running, does it need to rescan everything or does inotify tell it everything that happened?
<RomeSilvanus[m]>
What actually happens when you change a lot of files and folder while ipfs-sync isn't running, does it need to rescan everything or does inotify tell it everything that happened?
jesse22 has joined #ipfs
jesse22 has joined #ipfs
jesse22 has joined #ipfs
<Discordian[m]>
I don't think you can, Qm is a multihash of an RSA key, 1 is a multihash of a EdDSA key
<Discordian[m]>
I don't think you can, Qm is a multihash of an RSA key, 1 is a multihash of a EdDSA key
<Discordian[m]>
I don't think you can, Qm is a multihash of an RSA key, 1 is a multihash of a EdDSA key
<aschmahmann>
kaepora: you don't `12...` is the base58 representation of the identity multihash (suitable for small keys like ed25519) and `Qm...` is the base58 representation of SHA2-256 (used for larger keys like RSA).
<aschmahmann>
kaepora: you don't `12...` is the base58 representation of the identity multihash (suitable for small keys like ed25519) and `Qm...` is the base58 representation of SHA2-256 (used for larger keys like RSA).
<aschmahmann>
kaepora: you don't `12...` is the base58 representation of the identity multihash (suitable for small keys like ed25519) and `Qm...` is the base58 representation of SHA2-256 (used for larger keys like RSA).
<Discordian[m]>
<RomeSilvanus[m] "What actually happens when you c"> It rescans on startup using the internal db, and it'll post changes to MFS after. However currently, if you delete things, it won't detect that.
<Discordian[m]>
<RomeSilvanus[m] "What actually happens when you c"> It rescans on startup using the internal db, and it'll post changes to MFS after. However currently, if you delete things, it won't detect that.
<Discordian[m]>
<RomeSilvanus[m] "What actually happens when you c"> It rescans on startup using the internal db, and it'll post changes to MFS after. However currently, if you delete things, it won't detect that.
<aschmahmann>
* kaepora: you don't `1...` is the base58 representation of the identity multihash (suitable for small keys like ed25519) and `Qm...` is the base58 representation of SHA2-256 (used for larger keys like RSA).
<aschmahmann>
* kaepora: you don't `1...` is the base58 representation of the identity multihash (suitable for small keys like ed25519) and `Qm...` is the base58 representation of SHA2-256 (used for larger keys like RSA).
<aschmahmann>
* kaepora: you don't `1...` is the base58 representation of the identity multihash (suitable for small keys like ed25519) and `Qm...` is the base58 representation of SHA2-256 (used for larger keys like RSA).
<RomeSilvanus[m]>
Oh okay !
<RomeSilvanus[m]>
Oh okay !
<RomeSilvanus[m]>
Oh okay !
<Discordian[m]>
Renames will result in duplicate entries, but because of filestore, it won't be duplicated data. (only while offline)
<Discordian[m]>
Renames will result in duplicate entries, but because of filestore, it won't be duplicated data. (only while offline)
<Discordian[m]>
Renames will result in duplicate entries, but because of filestore, it won't be duplicated data. (only while offline)
<RomeSilvanus[m]>
Oh nice, it actually progressed
<RomeSilvanus[m]>
Oh nice, it actually progressed
<RomeSilvanus[m]>
Oh nice, it actually progressed
<RomeSilvanus[m]>
It seems to scan each folder I put into the config after another instead of all folders at once. Nice. I feared it would want to scan _all_ folders first and then start the adding process.
<RomeSilvanus[m]>
Not optimal, but still faster.
<RomeSilvanus[m]>
Not optimal, but still faster.
<RomeSilvanus[m]>
It seems to scan each folder I put into the config after another instead of all folders at once. Nice. I feared it would want to scan _all_ folders first and then start the adding process.
<RomeSilvanus[m]>
Not optimal, but still faster.
<RomeSilvanus[m]>
It seems to scan each folder I put into the config after another instead of all folders at once. Nice. I feared it would want to scan _all_ folders first and then start the adding process.
<Discordian[m]>
<aschmahmann "kaepora: you don't `12...` is th"> (that's b58mh actually I believe)
<Discordian[m]>
<aschmahmann "kaepora: you don't `12...` is th"> (that's b58mh actually I believe)
<Discordian[m]>
<aschmahmann "kaepora: you don't `12...` is th"> (that's b58mh actually I believe)
<kaepora>
aschmahmann: But this doesn't make sense. IPFS represents Ed25519 keys as having Qm IDs, but when it gives you their IPNS names, it gives it as a `1...` hash
<kaepora>
aschmahmann: But this doesn't make sense. IPFS represents Ed25519 keys as having Qm IDs, but when it gives you their IPNS names, it gives it as a `1...` hash
<kaepora>
aschmahmann: But this doesn't make sense. IPFS represents Ed25519 keys as having Qm IDs, but when it gives you their IPNS names, it gives it as a `1...` hash
<Discordian[m]>
I think you're referring to a bug in js-ipfs
<Discordian[m]>
I think you're referring to a bug in js-ipfs
<Discordian[m]>
I think you're referring to a bug in js-ipfs
<kaepora>
I've been trying all week to get the `1...` IPNS name of an Ed25519 key without publishing something via IPNS and I still can't. I can only get the Qm hash and that was after fixing the bug we found yesterday, Discordian[m]
<kaepora>
I've been trying all week to get the `1...` IPNS name of an Ed25519 key without publishing something via IPNS and I still can't. I can only get the Qm hash and that was after fixing the bug we found yesterday, Discordian[m]
<kaepora>
I've been trying all week to get the `1...` IPNS name of an Ed25519 key without publishing something via IPNS and I still can't. I can only get the Qm hash and that was after fixing the bug we found yesterday, Discordian[m]
<kaepora>
Sigh. Let me see
<kaepora>
Sigh. Let me see
<kaepora>
Sigh. Let me see
<Discordian[m]>
Right, you submitted the patch, is there maybe a bug in the patch too? The issue was it's forming the multihash correctly, then representing it incorrectly
<Discordian[m]>
Right, you submitted the patch, is there maybe a bug in the patch too? The issue was it's forming the multihash correctly, then representing it incorrectly
<Discordian[m]>
Right, you submitted the patch, is there maybe a bug in the patch too? The issue was it's forming the multihash correctly, then representing it incorrectly
<kaepora>
Yes...
<kaepora>
Yes...
<kaepora>
Yes...
<Discordian[m]>
* Right, you submitted the patch, is there maybe a bug in the patch too? The issue was it's forming the multihash incorrectly, then representing it incorrectly
<Discordian[m]>
* Right, you submitted the patch, is there maybe a bug in the patch too? The issue was it's forming the multihash incorrectly, then representing it incorrectly
<Discordian[m]>
* Right, you submitted the patch, is there maybe a bug in the patch too? The issue was it's forming the multihash incorrectly, then representing it incorrectly
<Discordian[m]>
Ah, that's unfortunate
<Discordian[m]>
Ah, that's unfortunate
<Discordian[m]>
Ah, that's unfortunate
<aschmahmann>
both 1... and Qm... are base58btc multihashes.
<aschmahmann>
both 1... and Qm... are base58btc multihashes.
<aschmahmann>
both 1... and Qm... are base58btc multihashes.
<kaepora>
wat
<kaepora>
wat
<kaepora>
wat
<Discordian[m]>
Because the multihash was formed incorrectly, all that's been fixed is the representation. It still needs to be formed correctly.
<Discordian[m]>
I'll look into adding a verbose option, just make it dumb as much as I can think of, and keep adding more
<Discordian[m]>
I'll look into adding a verbose option, just make it dumb as much as I can think of, and keep adding more
<Discordian[m]>
I'll look into adding a verbose option, just make it dumb as much as I can think of, and keep adding more
<Discordian[m]>
Would help with debugging for devs and users.
<Discordian[m]>
Would help with debugging for devs and users.
<Discordian[m]>
Would help with debugging for devs and users.
<Discordian[m]>
* I'll look into adding a verbose option, just make it dump as much as I can think of, and keep adding more
<Discordian[m]>
* I'll look into adding a verbose option, just make it dump as much as I can think of, and keep adding more
<Discordian[m]>
* I'll look into adding a verbose option, just make it dump as much as I can think of, and keep adding more
<Discordian[m]>
Maybe using `fscache` would speed it up?
<Discordian[m]>
Maybe using `fscache` would speed it up?
<Discordian[m]>
Maybe using `fscache` would speed it up?
<RomeSilvanus[m]>
Ill let it run again but it will probably be the same outcome
<RomeSilvanus[m]>
Ill let it run again but it will probably be the same outcome
<RomeSilvanus[m]>
Ill let it run again but it will probably be the same outcome
<Discordian[m]>
Currently ipfs-sync doesn't use `fscache`
<Discordian[m]>
Currently ipfs-sync doesn't use `fscache`
<Discordian[m]>
Currently ipfs-sync doesn't use `fscache`
<Discordian[m]>
I'm wondering if your add is so quick because of `fscache` 🤔
<Discordian[m]>
I'm wondering if your add is so quick because of `fscache` 🤔
<Discordian[m]>
I'm wondering if your add is so quick because of `fscache` 🤔
Nukien2 has joined #ipfs
Nukien2 has joined #ipfs
Nukien2 has joined #ipfs
<RomeSilvanus[m]>
I didnt use the fscache option in the beginning and it was basically the same speed
<RomeSilvanus[m]>
I didnt use the fscache option in the beginning and it was basically the same speed
<RomeSilvanus[m]>
I didnt use the fscache option in the beginning and it was basically the same speed
<Discordian[m]>
Hmm, yeah I'll get those loggers rolling I suppose. I wonder why it's being like this. I like the idea of adding loads of output
<Discordian[m]>
Hmm, yeah I'll get those loggers rolling I suppose. I wonder why it's being like this. I like the idea of adding loads of output
<Discordian[m]>
Hmm, yeah I'll get those loggers rolling I suppose. I wonder why it's being like this. I like the idea of adding loads of output
<RomeSilvanus[m]>
Hmmm
<RomeSilvanus[m]>
Hmmm
<RomeSilvanus[m]>
Hmmm
<RomeSilvanus[m]>
2021-03-16T16:48:40.130+0100 ERROR engine blockstore.Get(bafkreid3cifamxzz64fiha3vp27r66623nyftra2xylzvoggoxshblavwm) error: open /home/rome/{long path}.png: no such file or directory
<RomeSilvanus[m]>
2021-03-16T16:48:40.130+0100 ERROR engine blockstore.Get(bafkreid3cifamxzz64fiha3vp27r66623nyftra2xylzvoggoxshblavwm) error: open /home/rome/{long path}.png: no such file or directory
<RomeSilvanus[m]>
2021-03-16T16:48:40.130+0100 ERROR engine blockstore.Get(bafkreid3cifamxzz64fiha3vp27r66623nyftra2xylzvoggoxshblavwm) error: open /home/rome/{long path}.png: no such file or directory
<RomeSilvanus[m]>
Its throwing errors for files that dont exist anymore
<RomeSilvanus[m]>
Its throwing errors for files that dont exist anymore
<RomeSilvanus[m]>
Its throwing errors for files that dont exist anymore
<RomeSilvanus[m]>
I guess they are still in ipfs
<RomeSilvanus[m]>
I guess they are still in ipfs
<RomeSilvanus[m]>
I guess they are still in ipfs
<Discordian[m]>
Ah yeah if the dir updates during initial sync, it'll get a lot of errors
<Discordian[m]>
Ah yeah if the dir updates during initial sync, it'll get a lot of errors
<Discordian[m]>
Ah yeah if the dir updates during initial sync, it'll get a lot of errors
<Discordian[m]>
Oh that's a weird error
<Discordian[m]>
Oh that's a weird error
<Discordian[m]>
Oh that's a weird error
<Discordian[m]>
That probably comes from the IPFS daemon by the looks of it
<Discordian[m]>
That probably comes from the IPFS daemon by the looks of it
<Discordian[m]>
That probably comes from the IPFS daemon by the looks of it
<RomeSilvanus[m]>
Oh, I just saw an error
<RomeSilvanus[m]>
Oh, I just saw an error
<RomeSilvanus[m]>
Oh, I just saw an error
<RomeSilvanus[m]>
{"ID":"Shared by other people", "Dir":"/home/rome/google-drive/Shared by other people", "Nocopy": true},
<RomeSilvanus[m]>
{"ID":"Shared by other people", "Dir":"/home/rome/google-drive/Shared by other people", "Nocopy": true},
<RomeSilvanus[m]>
{"ID":"Shared by other people", "Dir":"/home/rome/google-drive/Shared by other people", "Nocopy": true},
<RomeSilvanus[m]>
The ID doesnt like spaces
<RomeSilvanus[m]>
The ID doesnt like spaces
<RomeSilvanus[m]>
The ID doesnt like spaces
<RomeSilvanus[m]>
So "/home/rome/google-drive/Shared by other people/SOMEFILE" becomes "/home/rome/google-drive//SOMEFILE"
<RomeSilvanus[m]>
So "/home/rome/google-drive/Shared by other people/SOMEFILE" becomes "/home/rome/google-drive//SOMEFILE"
<RomeSilvanus[m]>
So "/home/rome/google-drive/Shared by other people/SOMEFILE" becomes "/home/rome/google-drive//SOMEFILE"
<RomeSilvanus[m]>
That may be the problem with the hanging
<RomeSilvanus[m]>
That may be the problem with the hanging
<RomeSilvanus[m]>
That may be the problem with the hanging
<Discordian[m]>
Oh yeah ID can't have spaces, that may have caused something wonky yeah
<Discordian[m]>
Oh yeah ID can't have spaces, that may have caused something wonky yeah
<Discordian[m]>
Oh yeah ID can't have spaces, that may have caused something wonky yeah
<Discordian[m]>
Dir should be able to have spaces though, odd
<Discordian[m]>
Dir should be able to have spaces though, odd
<Discordian[m]>
Dir should be able to have spaces though, odd
<RomeSilvanus[m]>
You should probably clarify that somewhere/throw an error when parsing the config
<RomeSilvanus[m]>
You should probably clarify that somewhere/throw an error when parsing the config
<RomeSilvanus[m]>
You should probably clarify that somewhere/throw an error when parsing the config
<Discordian[m]>
I'll look into encoding it correctly, I can support spaces in the `Dir` field, just apparently doesn't work rn.
<Discordian[m]>
I'll look into encoding it correctly, I can support spaces in the `Dir` field, just apparently doesn't work rn.
<Discordian[m]>
I'll look into encoding it correctly, I can support spaces in the `Dir` field, just apparently doesn't work rn.
<Discordian[m]>
ID though, likely can't have spaces
<Discordian[m]>
ID though, likely can't have spaces
<Discordian[m]>
ID though, likely can't have spaces
<RomeSilvanus[m]>
Yeah, I just remembered they can't. But I forgot about it so it's probably best to check because other user will probably do the same.
<RomeSilvanus[m]>
Yeah, I just remembered they can't. But I forgot about it so it's probably best to check because other user will probably do the same.
<RomeSilvanus[m]>
Yeah, I just remembered they can't. But I forgot about it so it's probably best to check because other user will probably do the same.
bluto[m] has quit [Quit: Idle for 30+ days]
bluto[m] has quit [Quit: Idle for 30+ days]
bluto[m] has quit [Quit: Idle for 30+ days]
<RomeSilvanus[m]>
Trying it again with _ instead of space
<RomeSilvanus[m]>
Trying it again with _ instead of space
<RomeSilvanus[m]>
Trying it again with _ instead of space
<Discordian[m]>
Next update has better escaping, and verbose flag (which doesn't do everything, but will tell a lot more info)
<Discordian[m]>
Next update has better escaping, and verbose flag (which doesn't do everything, but will tell a lot more info)
<Discordian[m]>
Next update has better escaping, and verbose flag (which doesn't do everything, but will tell a lot more info)
Donnakufus[m] has joined #ipfs
Donnakufus[m] has joined #ipfs
Donnakufus[m] has joined #ipfs
<Discordian[m]>
gonna release v0.3.1 to give you a little bit more assistance
<Discordian[m]>
gonna release v0.3.1 to give you a little bit more assistance
<Discordian[m]>
gonna release v0.3.1 to give you a little bit more assistance
<Discordian[m]>
Where dk.Dir is what you specified. So it's prefixing `/ipfs-sync/google-drive/` for the namespace, then trimming out rest of the path, so it can go into MFS correctly. So I think it's working correctly.
<Discordian[m]>
Where dk.Dir is what you specified. So it's prefixing `/ipfs-sync/google-drive/` for the namespace, then trimming out rest of the path, so it can go into MFS correctly. So I think it's working correctly.
<Discordian[m]>
Where dk.Dir is what you specified. So it's prefixing `/ipfs-sync/google-drive/` for the namespace, then trimming out rest of the path, so it can go into MFS correctly. So I think it's working correctly.
<Discordian[m]>
The double `//` should work fine I believe
<Discordian[m]>
The double `//` should work fine I believe
<Discordian[m]>
The double `//` should work fine I believe
<RomeSilvanus[m]>
Its "google-drive/some folder/"
<RomeSilvanus[m]>
Its "google-drive/some folder/"
<RomeSilvanus[m]>
Its "google-drive/some folder/"
<RomeSilvanus[m]>
Thats the dir I specify. In the folder google-drive is a dir with a space in its name.
<RomeSilvanus[m]>
Thats the dir I specify. In the folder google-drive is a dir with a space in its name.
<RomeSilvanus[m]>
Thats the dir I specify. In the folder google-drive is a dir with a space in its name.
<RomeSilvanus[m]>
The // is because it should be /some folder/
<RomeSilvanus[m]>
The // is because it should be /some folder/
<RomeSilvanus[m]>
The // is because it should be /some folder/
<RomeSilvanus[m]>
it just leaves it out completely
<RomeSilvanus[m]>
it just leaves it out completely
<RomeSilvanus[m]>
it just leaves it out completely
<Discordian[m]>
Yeah if you have `some folder` in the `Dir` field, it'll be trimmed out, as the root folder isn't really added, it's just crawled
<Discordian[m]>
Yeah if you have `some folder` in the `Dir` field, it'll be trimmed out, as the root folder isn't really added, it's just crawled
<Discordian[m]>
Yeah if you have `some folder` in the `Dir` field, it'll be trimmed out, as the root folder isn't really added, it's just crawled
<Discordian[m]>
If that makes sense
<Discordian[m]>
If that makes sense
<Discordian[m]>
If that makes sense
Newami has quit [Remote host closed the connection]
Newami has quit [Remote host closed the connection]
Newami has quit [Remote host closed the connection]
<Discordian[m]>
So if you did `/home/user/my folder/` with ID `MyFolder` files will be found in `/ipfs-sync/MyFolder/` in the MFS, not `/ipfs-sync/MyFolder/my folder/`
<Discordian[m]>
So if you did `/home/user/my folder/` with ID `MyFolder` files will be found in `/ipfs-sync/MyFolder/` in the MFS, not `/ipfs-sync/MyFolder/my folder/`
<Discordian[m]>
So if you did `/home/user/my folder/` with ID `MyFolder` files will be found in `/ipfs-sync/MyFolder/` in the MFS, not `/ipfs-sync/MyFolder/my folder/`
joey has quit [Remote host closed the connection]
joey has quit [Remote host closed the connection]
joey has quit [Remote host closed the connection]
<RomeSilvanus[m]>
But I want to add "/ipfs-sync/MyFolder/my folder/" and not "/ipfs-sync/MyFolder/"
<RomeSilvanus[m]>
But I want to add "/ipfs-sync/MyFolder/my folder/" and not "/ipfs-sync/MyFolder/"
<RomeSilvanus[m]>
But I want to add "/ipfs-sync/MyFolder/my folder/" and not "/ipfs-sync/MyFolder/"
<RomeSilvanus[m]>
Im confused
<RomeSilvanus[m]>
Im confused
<RomeSilvanus[m]>
Im confused
<Discordian[m]>
You can think of MFS as a link to your files, like a symlink, you don't really need to worry about it
<Discordian[m]>
You can think of MFS as a link to your files, like a symlink, you don't really need to worry about it
<Discordian[m]>
You can think of MFS as a link to your files, like a symlink, you don't really need to worry about it
<Discordian[m]>
Like if you're not using MFS or planning to use MFS, it's really just a technical detail to make pinning easier
<Discordian[m]>
Like if you're not using MFS or planning to use MFS, it's really just a technical detail to make pinning easier
<Discordian[m]>
Like if you're not using MFS or planning to use MFS, it's really just a technical detail to make pinning easier
<RomeSilvanus[m]>
Hm, okay. But the problem still exists.
<RomeSilvanus[m]>
Hm, okay. But the problem still exists.
<RomeSilvanus[m]>
Hm, okay. But the problem still exists.
<Discordian[m]>
The hanging problem?
<Discordian[m]>
The hanging problem?
<Discordian[m]>
The hanging problem?
Nukien2 has quit [Quit: ZNC 1.7.5+deb4 - https://znc.in]
Nukien2 has quit [Quit: ZNC 1.7.5+deb4 - https://znc.in]
Nukien2 has quit [Quit: ZNC 1.7.5+deb4 - https://znc.in]
<Discordian[m]>
Is it hanging on generating headers?
<Discordian[m]>
Is it hanging on generating headers?
<Discordian[m]>
Is it hanging on generating headers?
<RomeSilvanus[m]>
No, that it doesnt put in the folder name
<RomeSilvanus[m]>
No, that it doesnt put in the folder name
<RomeSilvanus[m]>
No, that it doesnt put in the folder name
<Discordian[m]>
You could sync the whole `google-drive` folder to achieve that effect 🤔
<Discordian[m]>
You could sync the whole `google-drive` folder to achieve that effect 🤔
<Discordian[m]>
You could sync the whole `google-drive` folder to achieve that effect 🤔
<Discordian[m]>
But at the end of the day, it's a CID, so I'm not sure why the dir name is important
<Discordian[m]>
But at the end of the day, it's a CID, so I'm not sure why the dir name is important
<Discordian[m]>
But at the end of the day, it's a CID, so I'm not sure why the dir name is important
<RomeSilvanus[m]>
Because its hanging and for me it looks like its trying to add a nonexisting dir
<RomeSilvanus[m]>
Because its hanging and for me it looks like its trying to add a nonexisting dir
<RomeSilvanus[m]>
Because its hanging and for me it looks like its trying to add a nonexisting dir
mowcat has quit [Remote host closed the connection]
mowcat has quit [Remote host closed the connection]
mowcat has quit [Remote host closed the connection]
<Discordian[m]>
Is the line that shows what the path on your system is, that it's reading from
<Discordian[m]>
`2021/03/16 17:32:35 File updated: /home/rome/google-drive/{folder with space}/{file}.mpg`
<Discordian[m]>
Is the line that shows what the path on your system is, that it's reading from
<Discordian[m]>
`2021/03/16 17:32:35 File updated: /home/rome/google-drive/{folder with space}/{file}.mpg`
<Discordian[m]>
`2021/03/16 17:32:35 File updated: /home/rome/google-drive/{folder with space}/{file}.mpg`
<Discordian[m]>
Is the line that shows what the path on your system is, that it's reading from
<Discordian[m]>
It'll generate the dirs it needs to on MFS, if it's hanging on headers, that means the read is either slow, or broken somehow causing it to stall infinitely
<Discordian[m]>
It'll generate the dirs it needs to on MFS, if it's hanging on headers, that means the read is either slow, or broken somehow causing it to stall infinitely
<Discordian[m]>
It'll generate the dirs it needs to on MFS, if it's hanging on headers, that means the read is either slow, or broken somehow causing it to stall infinitely
<pmow[m]>
coughgdrive cough
<pmow[m]>
coughgdrive cough
<pmow[m]>
coughgdrive cough
<Discordian[m]>
Actually the new verbose flag will tell you that process as it does it too, assuming you get passed that step
<Discordian[m]>
Actually the new verbose flag will tell you that process as it does it too, assuming you get passed that step
<Discordian[m]>
Actually the new verbose flag will tell you that process as it does it too, assuming you get passed that step
<RomeSilvanus[m]>
This one worked, but the add doesnt work. In the add command {folder with space} is missing.
<RomeSilvanus[m]>
This one worked, but the add doesnt work. In the add command {folder with space} is missing.
<RomeSilvanus[m]>
This one worked, but the add doesnt work. In the add command {folder with space} is missing.
<Discordian[m]>
`/ipfs-sync/` refers to an MFS path it'll copy to at a later step, at the headers step, it hasn't even *looked* at MFS yet. You can look at that path with `ipfs files ls /ipfs-sync/`
<Discordian[m]>
`/ipfs-sync/` refers to an MFS path it'll copy to at a later step, at the headers step, it hasn't even *looked* at MFS yet. You can look at that path with `ipfs files ls /ipfs-sync/`
<Discordian[m]>
`/ipfs-sync/` refers to an MFS path it'll copy to at a later step, at the headers step, it hasn't even *looked* at MFS yet. You can look at that path with `ipfs files ls /ipfs-sync/`
<RomeSilvanus[m]>
This just tells me that it doesnt exist
<RomeSilvanus[m]>
This just tells me that it doesnt exist
<RomeSilvanus[m]>
This just tells me that it doesnt exist
<Discordian[m]>
Yeah, it likely hasn't made the dir yet, because it's stuck on the headers step, going by your output
<Discordian[m]>
Yeah, it likely hasn't made the dir yet, because it's stuck on the headers step, going by your output
<Discordian[m]>
Yeah, it likely hasn't made the dir yet, because it's stuck on the headers step, going by your output
<RomeSilvanus[m]>
But shouldnt it be adding the file after that step then?
<RomeSilvanus[m]>
But shouldnt it be adding the file after that step then?
<RomeSilvanus[m]>
But shouldnt it be adding the file after that step then?
<Discordian[m]>
When you see `Creating parent directory...`, that's when the path is created
<Discordian[m]>
When you see `Creating parent directory...`, that's when the path is created
<Discordian[m]>
When you see `Creating parent directory...`, that's when the path is created
<Discordian[m]>
No because it adds to IPFS normally, which doesn't use MFS, then it generates a path for the file in MFS, and adds the CID to that path. The CID is obtained from the `add` step
<Discordian[m]>
No because it adds to IPFS normally, which doesn't use MFS, then it generates a path for the file in MFS, and adds the CID to that path. The CID is obtained from the `add` step
<Discordian[m]>
No because it adds to IPFS normally, which doesn't use MFS, then it generates a path for the file in MFS, and adds the CID to that path. The CID is obtained from the `add` step
<RomeSilvanus[m]>
So then IPFS is given a incomplete path
<RomeSilvanus[m]>
So then IPFS is given a incomplete path
<RomeSilvanus[m]>
So then IPFS is given a incomplete path
<Discordian[m]>
It used to write to MFS directly, but there's a bug currently in block creation, where using that API results in broken files.
<Discordian[m]>
It used to write to MFS directly, but there's a bug currently in block creation, where using that API results in broken files.
<Discordian[m]>
It used to write to MFS directly, but there's a bug currently in block creation, where using that API results in broken files.
<Discordian[m]>
It's not at that step yet though, if you're stuck on headers
<Discordian[m]>
It's not at that step yet though, if you're stuck on headers
<Discordian[m]>
It's not at that step yet though, if you're stuck on headers
<RomeSilvanus[m]>
This is very confusing because it prints generating headers after trying to add a file
<RomeSilvanus[m]>
This is very confusing because it prints generating headers after trying to add a file
<RomeSilvanus[m]>
This is very confusing because it prints generating headers after trying to add a file
<RomeSilvanus[m]>
It still puts // in without the folder name though
<RomeSilvanus[m]>
It still puts // in without the folder name though
<RomeSilvanus[m]>
It still puts // in without the folder name though
arcatech has quit [Quit: Be back later.]
arcatech has quit [Quit: Be back later.]
arcatech has quit [Quit: Be back later.]
<Discordian[m]>
Yeah, it's really just an implementation detail tbh
<Discordian[m]>
Yeah, it's really just an implementation detail tbh
<Discordian[m]>
Yeah, it's really just an implementation detail tbh
<Discordian[m]>
Oh that header looks fine actually
<Discordian[m]>
Oh that header looks fine actually
<Discordian[m]>
Oh that header looks fine actually
<Discordian[m]>
I can't for the life of me see why spaces are so different lol
<Discordian[m]>
I can't for the life of me see why spaces are so different lol
<Discordian[m]>
I can't for the life of me see why spaces are so different lol
<Discordian[m]>
Not for that step specifically, it literally just does `io.Copy(part, f)`
<Discordian[m]>
Not for that step specifically, it literally just does `io.Copy(part, f)`
<Discordian[m]>
Not for that step specifically, it literally just does `io.Copy(part, f)`
LiftLeft3 has joined #ipfs
LiftLeft3 has joined #ipfs
LiftLeft3 has joined #ipfs
<RomeSilvanus[m]>
Its weird because it just seem to hang for a long time when a direct ipfs add is very fast
<RomeSilvanus[m]>
Its weird because it just seem to hang for a long time when a direct ipfs add is very fast
<RomeSilvanus[m]>
Its weird because it just seem to hang for a long time when a direct ipfs add is very fast
<RomeSilvanus[m]>
Maybe it really takes a very long time to add these
<RomeSilvanus[m]>
Maybe it really takes a very long time to add these
<RomeSilvanus[m]>
Maybe it really takes a very long time to add these
<RomeSilvanus[m]>
I'll try running everything again for like an hour or so
<RomeSilvanus[m]>
I'll try running everything again for like an hour or so
<RomeSilvanus[m]>
I'll try running everything again for like an hour or so
LiftLeft has quit [Ping timeout: 246 seconds]
LiftLeft has quit [Ping timeout: 246 seconds]
LiftLeft has quit [Ping timeout: 246 seconds]
<Discordian[m]>
Alright, assuming on first add nothing changes, it should add everything eventually. But you have a very massive dataset, and that's really the best way to have something go wrong heh. I've only tested with ~18k files, and that works great.
<Discordian[m]>
Alright, assuming on first add nothing changes, it should add everything eventually. But you have a very massive dataset, and that's really the best way to have something go wrong heh. I've only tested with ~18k files, and that works great.
<Discordian[m]>
Alright, assuming on first add nothing changes, it should add everything eventually. But you have a very massive dataset, and that's really the best way to have something go wrong heh. I've only tested with ~18k files, and that works great.
dqx_ has joined #ipfs
dqx_ has joined #ipfs
dqx_ has joined #ipfs
<RomeSilvanus[m]>
That one folder is only a few 100 gbs
<RomeSilvanus[m]>
That one folder is only a few 100 gbs
<RomeSilvanus[m]>
That one folder is only a few 100 gbs
<Discordian[m]>
"only" my 18k dataset is like 2GB, and it took 2hrs to add lol. I might look into speeding it up by calling `add` less often, and streaming more data to it at once.
<Discordian[m]>
"only" my 18k dataset is like 2GB, and it took 2hrs to add lol. I might look into speeding it up by calling `add` less often, and streaming more data to it at once.
<Discordian[m]>
"only" my 18k dataset is like 2GB, and it took 2hrs to add lol. I might look into speeding it up by calling `add` less often, and streaming more data to it at once.
<RomeSilvanus[m]>
Hm okay
<RomeSilvanus[m]>
Hm okay
<RomeSilvanus[m]>
Hm okay
<RomeSilvanus[m]>
As I said, maybe it really is just that slow compared to pure ipfs
<RomeSilvanus[m]>
As I said, maybe it really is just that slow compared to pure ipfs
<RomeSilvanus[m]>
As I said, maybe it really is just that slow compared to pure ipfs
<Discordian[m]>
The only real overhead ipfs-sync introduces is during the initial scan, and leveldb writes for hashes. Your header taking 40+mins to gen for a 700mb file is insane to me though, can't for the life of me understand why that'd happen.
<Discordian[m]>
The only real overhead ipfs-sync introduces is during the initial scan, and leveldb writes for hashes. Your header taking 40+mins to gen for a 700mb file is insane to me though, can't for the life of me understand why that'd happen.
<Discordian[m]>
The only real overhead ipfs-sync introduces is during the initial scan, and leveldb writes for hashes. Your header taking 40+mins to gen for a 700mb file is insane to me though, can't for the life of me understand why that'd happen.
<RomeSilvanus[m]>
I will report results later !
<RomeSilvanus[m]>
I will report results later !
<RomeSilvanus[m]>
I will report results later !
<Discordian[m]>
Thank you, much appreciated 🙂
<Discordian[m]>
Thank you, much appreciated 🙂
<Discordian[m]>
Thank you, much appreciated 🙂
<Discordian[m]>
I just did a 1.1GiB file in seconds, very strange
<Discordian[m]>
I just did a 1.1GiB file in seconds, very strange
<Discordian[m]>
I just did a 1.1GiB file in seconds, very strange
LiftLeft3 has quit [Ping timeout: 245 seconds]
LiftLeft3 has quit [Ping timeout: 245 seconds]
LiftLeft3 has quit [Ping timeout: 245 seconds]
LiftLeft has joined #ipfs
LiftLeft has joined #ipfs
LiftLeft has joined #ipfs
royal_screwup21 has quit [Quit: Connection closed]
royal_screwup21 has quit [Quit: Connection closed]
royal_screwup21 has quit [Quit: Connection closed]
john2gb has quit [Read error: Connection reset by peer]
john2gb has quit [Read error: Connection reset by peer]
john2gb has quit [Read error: Connection reset by peer]
royal_screwup21 has joined #ipfs
royal_screwup21 has joined #ipfs
royal_screwup21 has joined #ipfs
john2gb has joined #ipfs
john2gb has joined #ipfs
john2gb has joined #ipfs
rsheftel31521 is now known as rsheftel
rsheftel31521 is now known as rsheftel
rsheftel31521 is now known as rsheftel
royal_screwup21 has quit [Ping timeout: 265 seconds]
royal_screwup21 has quit [Ping timeout: 265 seconds]
royal_screwup21 has quit [Ping timeout: 265 seconds]
<Discordian[m]>
Mind you, off an NVME SSD
<Discordian[m]>
Mind you, off an NVME SSD
<Discordian[m]>
Mind you, off an NVME SSD
bengates has quit [Remote host closed the connection]
bengates has quit [Remote host closed the connection]
bengates has quit [Remote host closed the connection]
bengates has joined #ipfs
bengates has joined #ipfs
bengates has joined #ipfs
ylp has quit [Quit: Leaving.]
ylp has quit [Quit: Leaving.]
ylp has quit [Quit: Leaving.]
bengates has quit [Ping timeout: 260 seconds]
bengates has quit [Ping timeout: 260 seconds]
bengates has quit [Ping timeout: 260 seconds]
<pmow[m]>
Would a baseline with a regular 70-90iops spinning rust be a good idea?
<pmow[m]>
Would a baseline with a regular 70-90iops spinning rust be a good idea?
<pmow[m]>
Would a baseline with a regular 70-90iops spinning rust be a good idea?
<RomeSilvanus[m]>
I just have 4x10TB in Lx software raid as one partition
<RomeSilvanus[m]>
I just have 4x10TB in Lx software raid as one partition
<RomeSilvanus[m]>
I just have 4x10TB in Lx software raid as one partition
<pmow[m]>
oh, my misunderstanding - I thought I saw google-drive in your path and you were using that as a storage backend
<pmow[m]>
oh, my misunderstanding - I thought I saw google-drive in your path and you were using that as a storage backend
<pmow[m]>
oh, my misunderstanding - I thought I saw google-drive in your path and you were using that as a storage backend
<RomeSilvanus[m]>
Thats just name becasue I mirror my drive to that folder
<RomeSilvanus[m]>
Thats just name becasue I mirror my drive to that folder
<RomeSilvanus[m]>
Thats just name becasue I mirror my drive to that folder
royal_screwup21 has joined #ipfs
royal_screwup21 has joined #ipfs
royal_screwup21 has joined #ipfs
gimzmoe has quit [Quit: WeeChat 1.9.1]
gimzmoe has quit [Quit: WeeChat 1.9.1]
gimzmoe has quit [Quit: WeeChat 1.9.1]
Guest67 has joined #ipfs
Guest67 has joined #ipfs
Guest67 has joined #ipfs
<RomeSilvanus[m]>
Okaay it now progresses
<RomeSilvanus[m]>
Okaay it now progresses
<RomeSilvanus[m]>
Okaay it now progresses
gimzmoe has joined #ipfs
gimzmoe has joined #ipfs
gimzmoe has joined #ipfs
<RomeSilvanus[m]>
The verbose helps
<RomeSilvanus[m]>
The verbose helps
<RomeSilvanus[m]>
The verbose helps
<RomeSilvanus[m]>
Its throwing lots of file not found errors because I deletet that folder once an takes a long time between adds but it works
<RomeSilvanus[m]>
Its throwing lots of file not found errors because I deletet that folder once an takes a long time between adds but it works
<RomeSilvanus[m]>
Its throwing lots of file not found errors because I deletet that folder once an takes a long time between adds but it works
<Discordian[m]>
Yeah the file not found errors should hopefully be harmless. The one you showed earlier seemed to be referring to missing blocks, so I hope your ipfs daemon is figuring all that out.
<Discordian[m]>
Yeah the file not found errors should hopefully be harmless. The one you showed earlier seemed to be referring to missing blocks, so I hope your ipfs daemon is figuring all that out.
<Discordian[m]>
Yeah the file not found errors should hopefully be harmless. The one you showed earlier seemed to be referring to missing blocks, so I hope your ipfs daemon is figuring all that out.
<Discordian[m]>
If it's finished the initial sync, when you do adds/updates, it'll take time to update IPNS too, that in my experience is the longest wait.
<Discordian[m]>
If it's finished the initial sync, when you do adds/updates, it'll take time to update IPNS too, that in my experience is the longest wait.
<Discordian[m]>
If it's finished the initial sync, when you do adds/updates, it'll take time to update IPNS too, that in my experience is the longest wait.
<Discordian[m]>
You can tune it to sync more/less often too. Rn IPNS syncs every 10s by default
<Discordian[m]>
You can tune it to sync more/less often too. Rn IPNS syncs every 10s by default
<Discordian[m]>
You can tune it to sync more/less often too. Rn IPNS syncs every 10s by default
Guest67 has left #ipfs [#ipfs]
Guest67 has left #ipfs [#ipfs]
Guest67 has left #ipfs [#ipfs]
<RomeSilvanus[m]>
What is generating file headers actually doing?
<RomeSilvanus[m]>
What is generating file headers actually doing?
<RomeSilvanus[m]>
What is generating file headers actually doing?
<vaultec81[m]1>
I am surprised there isn't a fully featured mutability layer as part of IPFS
<vaultec81[m]1>
I am surprised there isn't a fully featured mutability layer as part of IPFS
<vaultec81[m]1>
I am surprised there isn't a fully featured mutability layer as part of IPFS
nikil511 has joined #ipfs
nikil511 has joined #ipfs
nikil511 has joined #ipfs
<vaultec81[m]1>
* I am surprised there isn't a fully featured mutability layer as part of IPFS yet
<vaultec81[m]1>
* I am surprised there isn't a fully featured mutability layer as part of IPFS yet
<vaultec81[m]1>
* I am surprised there isn't a fully featured mutability layer as part of IPFS yet
<nikil511>
am trying to build a weather station that stores data directly to IPFS.
<nikil511>
am trying to build a weather station that stores data directly to IPFS.
<nikil511>
am trying to build a weather station that stores data directly to IPFS.
<nikil511>
Any advice on how should I store temporal/spatial (weather data) on IPFS , in order to assist a smart contract (chainlink adapter) discover them ?
<nikil511>
Any advice on how should I store temporal/spatial (weather data) on IPFS , in order to assist a smart contract (chainlink adapter) discover them ?
<nikil511>
Any advice on how should I store temporal/spatial (weather data) on IPFS , in order to assist a smart contract (chainlink adapter) discover them ?
<nikil511>
is this possible or a good idea? -> /ipns/mydomain.com/lat/lon/year/month/day
<nikil511>
is this possible or a good idea? -> /ipns/mydomain.com/lat/lon/year/month/day
<nikil511>
is this possible or a good idea? -> /ipns/mydomain.com/lat/lon/year/month/day
<nikil511>
How do I link new CID with deep paths? (e.g. append every day a new dataset)
<nikil511>
How do I link new CID with deep paths? (e.g. append every day a new dataset)
<nikil511>
How do I link new CID with deep paths? (e.g. append every day a new dataset)
arcatech has joined #ipfs
arcatech has joined #ipfs
arcatech has joined #ipfs
<Discordian[m]>
It looks fine, you should be able to add those directories in and have them function as expected. Just update IPNS with the CID of the root (mydomain.com) whenever you do a change, and you should be golden. ipfs-sync does this: https://github.com/TheDiscordian/ipfs-sync
<Discordian[m]>
It looks fine, you should be able to add those directories in and have them function as expected. Just update IPNS with the CID of the root (mydomain.com) whenever you do a change, and you should be golden. ipfs-sync does this: https://github.com/TheDiscordian/ipfs-sync
<Discordian[m]>
It looks fine, you should be able to add those directories in and have them function as expected. Just update IPNS with the CID of the root (mydomain.com) whenever you do a change, and you should be golden. ipfs-sync does this: https://github.com/TheDiscordian/ipfs-sync
<Discordian[m]>
But it's not quite production ready.
<Discordian[m]>
But it's not quite production ready.
<Discordian[m]>
But it's not quite production ready.
<nikil511>
thanks for the reply Discordian :-)
<nikil511>
thanks for the reply Discordian :-)
<nikil511>
thanks for the reply Discordian :-)
<Discordian[m]>
No problem! 🙂
<Discordian[m]>
No problem! 🙂
<Discordian[m]>
No problem! 🙂
<nikil511>
ipfs-sync looks cool ... but is looking at the filesystem, and I am planning to add/publish the files from a microcontroler hardware directly. Can I have the ipfs-sync looking at something else? at my node files ? and will it update the DNS TXT too ?
<nikil511>
ipfs-sync looks cool ... but is looking at the filesystem, and I am planning to add/publish the files from a microcontroler hardware directly. Can I have the ipfs-sync looking at something else? at my node files ? and will it update the DNS TXT too ?
<nikil511>
ipfs-sync looks cool ... but is looking at the filesystem, and I am planning to add/publish the files from a microcontroler hardware directly. Can I have the ipfs-sync looking at something else? at my node files ? and will it update the DNS TXT too ?
arcatech has quit [Quit: Be back later.]
arcatech has quit [Quit: Be back later.]
arcatech has quit [Quit: Be back later.]
<Discordian[m]>
It doesn't update DNS txt, but you can point your DNS to an IPNS address. ipfs-sync currently only adds files from the filesystem unfortunately. You could have another software build the tree on disk using data from the microcontroller, and ipfs-sync could sync that though.
<Discordian[m]>
It doesn't update DNS txt, but you can point your DNS to an IPNS address. ipfs-sync currently only adds files from the filesystem unfortunately. You could have another software build the tree on disk using data from the microcontroller, and ipfs-sync could sync that though.
<Discordian[m]>
It doesn't update DNS txt, but you can point your DNS to an IPNS address. ipfs-sync currently only adds files from the filesystem unfortunately. You could have another software build the tree on disk using data from the microcontroller, and ipfs-sync could sync that though.
<Discordian[m]>
* It doesn't update DNS txt, but you can point your DNS to an IPNS address, and it'll update the IPNS. ipfs-sync currently only adds files from the filesystem unfortunately. You could have another software build the tree on disk using data from the microcontroller, and ipfs-sync could sync that though.
<Discordian[m]>
* It doesn't update DNS txt, but you can point your DNS to an IPNS address, and it'll update the IPNS. ipfs-sync currently only adds files from the filesystem unfortunately. You could have another software build the tree on disk using data from the microcontroller, and ipfs-sync could sync that though.
<Discordian[m]>
* It doesn't update DNS txt, but you can point your DNS to an IPNS address, and it'll update the IPNS. ipfs-sync currently only adds files from the filesystem unfortunately. You could have another software build the tree on disk using data from the microcontroller, and ipfs-sync could sync that though.
<eleitl[m]>
Does your microcontroller have enough resources to run ipfs?
<eleitl[m]>
Does your microcontroller have enough resources to run ipfs?
<eleitl[m]>
Does your microcontroller have enough resources to run ipfs?
<nikil511>
no, I will host my own node, and have the mcu do HTTP posts to my node
<nikil511>
no, I will host my own node, and have the mcu do HTTP posts to my node
<nikil511>
no, I will host my own node, and have the mcu do HTTP posts to my node
<Discordian[m]>
You could build all the same functions of ipfs-sync into the microcontroller. What ipfs-sync does it add all the files, then do `files cp` to copy all the files to MFS in the desired tree, then it simply updates IPNS with the CID of the root directory.
<Discordian[m]>
You could build all the same functions of ipfs-sync into the microcontroller. What ipfs-sync does it add all the files, then do `files cp` to copy all the files to MFS in the desired tree, then it simply updates IPNS with the CID of the root directory.
<Discordian[m]>
You could build all the same functions of ipfs-sync into the microcontroller. What ipfs-sync does it add all the files, then do `files cp` to copy all the files to MFS in the desired tree, then it simply updates IPNS with the CID of the root directory.
<swedneck>
the MFS is probably the easiest way to append data, yeah
<swedneck>
the MFS is probably the easiest way to append data, yeah
<swedneck>
the MFS is probably the easiest way to append data, yeah
<Discordian[m]>
I'm not sure how it works, yeah maybe so. Their website seems more eth related, so I don't really follow how IPFS fits into their service. It looks like it's just a public gateway for that aspect, but not sure.
<Discordian[m]>
I'm not sure how it works, yeah maybe so. Their website seems more eth related, so I don't really follow how IPFS fits into their service. It looks like it's just a public gateway for that aspect, but not sure.
<Discordian[m]>
I'm not sure how it works, yeah maybe so. Their website seems more eth related, so I don't really follow how IPFS fits into their service. It looks like it's just a public gateway for that aspect, but not sure.
<Discordian[m]>
Oh yup, does something with writing
<Discordian[m]>
Oh yup, does something with writing
<Discordian[m]>
Oh yup, does something with writing
<swedneck>
i have to say though, why are people not just asking the company they're trying to work with lmao
<swedneck>
i have to say though, why are people not just asking the company they're trying to work with lmao
<swedneck>
i have to say though, why are people not just asking the company they're trying to work with lmao
<swedneck>
"hi, how do i use your product?"
<swedneck>
"hi, how do i use your product?"
<swedneck>
"hi, how do i use your product?"
<Discordian[m]>
Haha that's a really good question, my first idea is to always ask the company
<Discordian[m]>
Haha that's a really good question, my first idea is to always ask the company
<Discordian[m]>
Haha that's a really good question, my first idea is to always ask the company
<Discordian[m]>
Their IPFS docs are just a 1:1 copy of the go-ipfs docs
<Discordian[m]>
Their IPFS docs are just a 1:1 copy of the go-ipfs docs
<Discordian[m]>
Their IPFS docs are just a 1:1 copy of the go-ipfs docs
<nikil511>
it seems to me that ipfs-sync makes sense when it is running on the node machine, but on an MCU (weather station) I will not have the full file system, so I want to send the daily values (JSON) to IPFS via RPC API
<nikil511>
it seems to me that ipfs-sync makes sense when it is running on the node machine, but on an MCU (weather station) I will not have the full file system, so I want to send the daily values (JSON) to IPFS via RPC API
<nikil511>
it seems to me that ipfs-sync makes sense when it is running on the node machine, but on an MCU (weather station) I will not have the full file system, so I want to send the daily values (JSON) to IPFS via RPC API
<nikil511>
on the MCU I can only have HTTP calls, and I want to avoid calling an additional middleware in my node
<nikil511>
on the MCU I can only have HTTP calls, and I want to avoid calling an additional middleware in my node
<nikil511>
on the MCU I can only have HTTP calls, and I want to avoid calling an additional middleware in my node
<Discordian[m]>
<swedneck "> <@discordian:matrix.thedisco.z"> Yeah maybe I can see if I can poke at it, and learn something
<Discordian[m]>
<swedneck "> <@discordian:matrix.thedisco.z"> Yeah maybe I can see if I can poke at it, and learn something
<Discordian[m]>
<swedneck "> <@discordian:matrix.thedisco.z"> Yeah maybe I can see if I can poke at it, and learn something
<Discordian[m]>
<nikil511 "on the MCU I can only have HTTP "> Yeah that code is all HTTP calls.
<Discordian[m]>
<nikil511 "on the MCU I can only have HTTP "> Yeah that code is all HTTP calls.
<Discordian[m]>
<nikil511 "on the MCU I can only have HTTP "> Yeah that code is all HTTP calls.
<nikil511>
ok I see it now, thanks!
<nikil511>
ok I see it now, thanks!
<nikil511>
ok I see it now, thanks!
<Discordian[m]>
Specifically, for adding files, the calls you'll be interested in are: `add`, `files/mkdir`, `files/rm` (if overwriting), then `files/cp`
<Discordian[m]>
Specifically, for adding files, the calls you'll be interested in are: `add`, `files/mkdir`, `files/rm` (if overwriting), then `files/cp`
<Discordian[m]>
Specifically, for adding files, the calls you'll be interested in are: `add`, `files/mkdir`, `files/rm` (if overwriting), then `files/cp`
<swedneck>
oh `files write` doesn't support adding directories either :/
<swedneck>
oh `files write` doesn't support adding directories either :/
<swedneck>
oh `files write` doesn't support adding directories either :/
<Discordian[m]>
Yeah but it'll create parents for you
<Discordian[m]>
Yeah but it'll create parents for you
<Discordian[m]>
Yeah but it'll create parents for you
<Discordian[m]>
Honestly `files write` is way smoother than what I do now, but it's just not reliable yet
<Discordian[m]>
Honestly `files write` is way smoother than what I do now, but it's just not reliable yet
<Discordian[m]>
Honestly `files write` is way smoother than what I do now, but it's just not reliable yet
arcatech has joined #ipfs
arcatech has joined #ipfs
arcatech has joined #ipfs
<Discordian[m]>
Dietrich suggested I make a video on that, so I'll probably do that in the near future. That way people can just build things like ipfs-sync easily, get the core features in whatever they want if they need them.
<Discordian[m]>
Dietrich suggested I make a video on that, so I'll probably do that in the near future. That way people can just build things like ipfs-sync easily, get the core features in whatever they want if they need them.
<Discordian[m]>
Dietrich suggested I make a video on that, so I'll probably do that in the near future. That way people can just build things like ipfs-sync easily, get the core features in whatever they want if they need them.
arcatech has quit [Client Quit]
arcatech has quit [Client Quit]
arcatech has quit [Client Quit]
royal_screwup21 has quit [Quit: Connection closed]
royal_screwup21 has quit [Quit: Connection closed]
royal_screwup21 has quit [Quit: Connection closed]
royal_screwup21 has joined #ipfs
royal_screwup21 has joined #ipfs
royal_screwup21 has joined #ipfs
ninekeysdown has joined #ipfs
ninekeysdown has joined #ipfs
ninekeysdown has joined #ipfs
LinusCDE has joined #ipfs
LinusCDE has joined #ipfs
LinusCDE has joined #ipfs
chiui has quit [Ping timeout: 245 seconds]
chiui has quit [Ping timeout: 245 seconds]
chiui has quit [Ping timeout: 245 seconds]
ipfslog has quit [Remote host closed the connection]
ipfslog has quit [Remote host closed the connection]
ipfslog has quit [Remote host closed the connection]
ipfslog2 has joined #ipfs
ipfslog2 has joined #ipfs
ipfslog2 has joined #ipfs
jadedctrl has quit [Ping timeout: 245 seconds]
jadedctrl has quit [Ping timeout: 245 seconds]
jadedctrl has quit [Ping timeout: 245 seconds]
Nukien2 has joined #ipfs
Nukien2 has joined #ipfs
Nukien2 has joined #ipfs
ipfs-stackbot has quit [Remote host closed the connection]
ipfs-stackbot has quit [Remote host closed the connection]
ipfs-stackbot has quit [Remote host closed the connection]
jadedctrl has joined #ipfs
jadedctrl has joined #ipfs
jadedctrl has joined #ipfs
ipfs-stackbot has joined #ipfs
ipfs-stackbot has joined #ipfs
ipfs-stackbot has joined #ipfs
jadedctrl has quit [Ping timeout: 260 seconds]
jadedctrl has quit [Ping timeout: 260 seconds]
jadedctrl has quit [Ping timeout: 260 seconds]
mowcat has joined #ipfs
mowcat has joined #ipfs
mowcat has joined #ipfs
distributedGuru has joined #ipfs
distributedGuru has joined #ipfs
distributedGuru has joined #ipfs
PendulumSwinger has quit [Quit: Ping timeout (120 seconds)]
PendulumSwinger has quit [Quit: Ping timeout (120 seconds)]
PendulumSwinger has quit [Quit: Ping timeout (120 seconds)]