aschmahmann changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.7.0 and js-ipfs 0.52.3 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of Conduct: https://git.io/vVBS0
john2gb0 has quit [Remote host closed the connection]
john2gb0 has joined #ipfs
john2gb0 has quit [Read error: Connection reset by peer]
john2gb0 has joined #ipfs
john2gb0 has quit [Read error: Connection reset by peer]
john2gb0 has joined #ipfs
john2gb0 has quit [Read error: Connection reset by peer]
john2gb0 has joined #ipfs
arcatech has joined #ipfs
john2gb0 has quit [Read error: Connection reset by peer]
john2gb0 has joined #ipfs
arcatech has quit [Ping timeout: 240 seconds]
john2gb0 has quit [Ping timeout: 260 seconds]
chiui has joined #ipfs
royal_screwup21 has quit [Quit: Connection closed]
royal_screwup21 has joined #ipfs
haivron has quit [Read error: Connection reset by peer]
royal_screwup21 has quit [Ping timeout: 240 seconds]
Ringtailed-Fox has joined #ipfs
RingtailedFox has quit [Read error: Connection reset by peer]
Ringtailed-Fox has quit [Read error: Connection reset by peer]
Ringtailed-Fox has joined #ipfs
mowotter has quit [Remote host closed the connection]
_jrjsmrtn has joined #ipfs
__jrjsmrtn__ has quit [Ping timeout: 246 seconds]
royal_screwup21 has joined #ipfs
arcatech has joined #ipfs
royal_screwup21 has quit [Ping timeout: 246 seconds]
arcatech has quit [Ping timeout: 240 seconds]
drathir_tor has quit [Remote host closed the connection]
}ls{ has quit [Ping timeout: 256 seconds]
ctOS has quit [Quit: Connection closed for inactivity]
Ringtailed_Fox has joined #ipfs
}ls{ has joined #ipfs
Ringtailed-Fox has quit [Read error: Connection reset by peer]
<anomie[m]>
Can dag-json documents be validated? I can use dag-cbor just fine but I think dag-json would be more user friendly for my use case.
elusive has joined #ipfs
dvim has quit [Quit: Connection closed for inactivity]
<vaultec81[m]1>
<anomie[m] "Can dag-json documents be valida"> What do you mean by validated?
<anomie[m]>
<vaultec81[m]1 "What do you mean by validated?"> To test if it's formed correctly, according to spec.
<vaultec81[m]1>
It absolutely can
<Donnakufus[m]>
Hello everyone I'm alone who care for a chat
arcatech has joined #ipfs
<anomie[m]>
<vaultec81[m]1 "It absolutely can "> How?
<vaultec81[m]1>
<anomie[m] "How? "> Through anyone of the many object schema libraries available out there
<Donnakufus[m]>
<vaultec81[m]1 "Through anyone of the many objec"> LoL 🤣
kuno1 has joined #ipfs
ysodope[m] is now known as yrsodope[m]
arcatech has quit [Ping timeout: 240 seconds]
royal_screwup21 has joined #ipfs
royal_screwup21 has quit [Ping timeout: 264 seconds]
elusive has quit [Quit: Leaving]
mindCrime_ has quit [Ping timeout: 260 seconds]
chachasmooth has quit [Ping timeout: 256 seconds]
jesse22 has joined #ipfs
}ls{ has quit [Quit: real life interrupt]
chachasmooth has joined #ipfs
jrt is now known as Guest38669
jrt has joined #ipfs
Newami has joined #ipfs
Guest38669 has quit [Ping timeout: 246 seconds]
maggotbrain has quit [Quit: Gone to Croatan...]
Newami has quit [Quit: Leaving]
arcatech has joined #ipfs
arcatech has quit [Client Quit]
p2p2p has quit [Ping timeout: 256 seconds]
paddymahoney has quit [Ping timeout: 246 seconds]
opa7331 has joined #ipfs
opa has quit [Ping timeout: 264 seconds]
dsrt^ has quit []
tfl^ has joined #ipfs
royal_screwup21 has joined #ipfs
royal_screwup21 has quit [Ping timeout: 264 seconds]
MrSprinkles has joined #ipfs
MrSparkle has quit [Ping timeout: 264 seconds]
kn0rki has joined #ipfs
names_j has joined #ipfs
endolith has joined #ipfs
MrSprinkles has quit [Quit: MrSprinkles]
MrSparkle has joined #ipfs
matt-h has quit [Read error: Connection reset by peer]
matt-h has joined #ipfs
Forkk has quit [Ping timeout: 240 seconds]
names_j has quit [Quit: Connection closed]
Forkk has joined #ipfs
jesse22 has quit [Ping timeout: 256 seconds]
Forkk has quit [Ping timeout: 244 seconds]
Forkk has joined #ipfs
opal has quit [Ping timeout: 268 seconds]
royal_screwup21 has joined #ipfs
royal_screwup21 has quit [Ping timeout: 240 seconds]
ipfs-stackbot has quit [Remote host closed the connection]
<frandavid100>
Hi all
ipfs-stackbot has joined #ipfs
veegee has quit [Ping timeout: 245 seconds]
veegee has joined #ipfs
opal has joined #ipfs
LiftLeft has quit [Ping timeout: 256 seconds]
bengates has joined #ipfs
tfl^ has quit []
ylp has joined #ipfs
clemo has joined #ipfs
bengates has quit [Remote host closed the connection]
bengates has joined #ipfs
clemo has quit [Ping timeout: 256 seconds]
clemo has joined #ipfs
mrx86[m] has joined #ipfs
bengates has quit [Remote host closed the connection]
bengates has joined #ipfs
drathir_tor has quit [Remote host closed the connection]
drathir_tor has joined #ipfs
Guest67 has joined #ipfs
Guest67 has left #ipfs [#ipfs]
konubinix has quit [Quit: Coyote finally caught me]
bronzylobster40[ has joined #ipfs
bronzylobster40[ has left #ipfs [#ipfs]
konubinix has joined #ipfs
nikil511 has joined #ipfs
<nikil511>
Hello, I am trying to add files to IPFS from an arduino MCU using the HTTP API. Can someone point me / give me an example in cURL for adding a file inside a specific directory?
<frandavid100>
Thanks a lot, I'll try that when I get home
LinusCDE has joined #ipfs
royal_screwup21 has joined #ipfs
luna_is_here has quit [Ping timeout: 256 seconds]
KeiraT- has joined #ipfs
KeiraT has quit [Remote host closed the connection]
KeiraT- is now known as KeiraT
luna_is_here has joined #ipfs
luna_is_here has quit [Read error: Connection reset by peer]
luna_is_here has joined #ipfs
<i_am_hari_seldon>
<frandavid100 "Also, IPFS seems to keep killing"> I have the same problem. I solved the issue by running IPFS in a container connected to VPN
Ringtailed_Fox has quit [Read error: Connection reset by peer]
dsrt^ has joined #ipfs
jcea has joined #ipfs
<Discordian[m]>
<frandavid100 "Hi all"> Heya, fancy seeing you here!
drathir_tor has quit [Remote host closed the connection]
drathir_tor has joined #ipfs
<frandavid100>
Hey Discordian!! 😀
<Discordian[m]>
Once you enable experimental filestore BTW the flag you'll want while adding is `--nocopy`. I just recently added `filestore` support to [ipfs-sync](https://github.com/TheDiscordian/ipfs-sync) if you're interested.
ZaZ has quit [Quit: Leaving]
}ls{ has joined #ipfs
drathir_tor has quit [Ping timeout: 268 seconds]
Ringtailed-Fox has quit [Read error: Connection reset by peer]
Ringtailed-Fox has joined #ipfs
tech_exorcist has joined #ipfs
drathir_tor has joined #ipfs
wallacer has joined #ipfs
endolith has left #ipfs [#ipfs]
luna_is_here has quit [Ping timeout: 256 seconds]
luna_is_here has joined #ipfs
p2p2p has joined #ipfs
royal_screwup21 has quit [Quit: Connection closed]
royal_screwup21 has joined #ipfs
Ringtailed-Fox has quit [Read error: Connection reset by peer]
Ringtailed-Fox has joined #ipfs
opal has joined #ipfs
royal_screwup21 has quit [Ping timeout: 260 seconds]
plntyk has quit [Quit: Leaving]
xtro has left #ipfs ["WeeChat 3.1"]
Joris[m]1 has joined #ipfs
jcea has quit [Ping timeout: 240 seconds]
<Joris[m]1>
What is ipfs-sync? And why are there at least two different projects under that name?
<Discordian[m]>
I didn't realise there were existing softwares with the same name when I released it. However ipfs-sync (my version) takes a directory on your system, and will sync it to IPFS. It'll ensure there's effectively a copy of the tree on MFS, and then produce an IPNS address (that it'll keep updated) to access your data.
<Discordian[m]>
I use it for my website
Mateon2 has joined #ipfs
sim590_ has joined #ipfs
XORed has joined #ipfs
supercoven has joined #ipfs
lo|rd has joined #ipfs
<RomeSilvanus[m]>
There are around 3 or four different project called ipfs-sync
<RomeSilvanus[m]>
<del>god</del> creativity is dead and we killed it
luna_is_here_ has joined #ipfs
<Discordian[m]>
tbh I've never been creative with my naming. I used to number all my functions like "Function1", "Function2", etc. And also have "Var1", "Var2" lol.
<Discordian[m]>
It is interesting looking at those projects though, they have similar goals
<Joris[m]1>
I wonder whether those projects for ipfs-sync could be united under one project / specification?
<Discordian[m]>
Honestly I've assimilated almost every feature and more at this point
<RomeSilvanus[m]>
Next goal: Make your fully self written ipfs implementation including an ipfs OS
<Joris[m]1>
IPFS OS
<Discordian[m]>
Hahaha no need to reinvent go-ipfs though! That daemon is awesome
<RomeSilvanus[m]>
You sure? Maybe implement one in Brainfuck?
<Discordian[m]>
I might look at the other golang implementation, they say they're atomic, makes me wonder if maybe they don't use the HTTP API (otherwise I don't see how it's atomic)
<Discordian[m]>
Hah, I'd write it in `*><>` if I did a meme implementation of IPFS iteself
<RomeSilvanus[m]>
Well, it seems to work fine when running both the gc and ipfs-sync
<RomeSilvanus[m]>
TAkes 4 Gb of ram
<RomeSilvanus[m]>
Nice
<RomeSilvanus[m]>
* Takes 4 Gb of ram
<Discordian[m]>
Haha that's good to hear
Centauro[m] has joined #ipfs
Centauro[m] has left #ipfs [#ipfs]
<Discordian[m]>
Looked into the other Go implementation. It's basically a young version of what I made, but doesn't use the HTTP API.
Nukien2 has quit [Quit: ZNC 1.7.5+deb4 - https://znc.in]
<Discordian[m]>
* Looked into the other Go implementation. It's basically a younger version of what I made, but doesn't use the HTTP API. I wonder if that dev would consider contributing to mine 🤔
<Joris[m]1>
I wonder what is such an essential service, that multiple developers came up with it independently, but cannot be part of the official go-ipfs implementation.
<Joris[m]1>
* I wonder what is such an essential service, that multiple developers came up with it independently, but that cannot be part of the official go-ipfs implementation.
<Discordian[m]>
Oh they want something like this as an official tool. Possibly officially, I'll be making video guides on how to make software like ipfs-sync, to help people out with the concepts
<RomeSilvanus[m]>
A lot of open source projects have the problem that the people responsible for it rather talk about something for 5 years instead of just trying it out for once and then looking how it will go.
<RomeSilvanus[m]>
* A lot of open source projects have the problem that the people responsible for it rather discuss about something for 5 years instead of just trying it out for once and then looking how it will go.
plntyk has joined #ipfs
<Discordian[m]>
The devs are just super busy, and expanding, but I find progress is pretty quick in this space
<Discordian[m]>
Sometimes to scale, a lot of planning is needed
<Discordian[m]>
However you'll always find spaces of people who'd rather talk theory, rather than apply it
aLeSD has quit [Remote host closed the connection]
gde33 has quit [Read error: Connection reset by peer]
gde33 has joined #ipfs
henriquev has quit [Ping timeout: 245 seconds]
henriquev has joined #ipfs
Ringtailed-Fox has quit [Read error: Connection reset by peer]
<RomeSilvanus[m]>
Its leaving out the folder directly above the two folders I adeed which it ddint do before
<Discordian[m]>
Yeah, I know what you meant
<Discordian[m]>
Now with or without the trailing `/`, it'll do that, which is correct
<RomeSilvanus[m]>
Why is it better than just adding it like it is on the filesystem though?
<Discordian[m]>
It was a parsing issue, really. Without the trailing `/` it just looked 1 higher than it was supposed to
<Discordian[m]>
It is effectively the same as the filesystem, it just doesn't replicate your entire FS tree, just root dir and down
Guest_2 has quit [Remote host closed the connection]
Guest_2 has joined #ipfs
<RomeSilvanus[m]>
Hmm
<RomeSilvanus[m]>
However that means if I add multiple dirs with subfolders I will not be able to share one of the dirs by copying the ID since all the other folders with their subfolders will be also under that ID
<Discordian[m]>
Yeah but you can still access subdirs with something like IPNSkey/subdir
<RomeSilvanus[m]>
True, but I cannot share the whole directory that contains the subdirs since its mixed with files and folders that aren't originaly in there
<Discordian[m]>
It shouldn't include anything new, just the original tree 🤔. You'll get an IPNS key for each root dir you sync, and they're isolated from each-other
<RomeSilvanus[m]>
Well, in the web interface all folders I add will be diretcly under /ipfs-sync/ and not separated under the folder they're actually in so ther is no way to do that from the web interface unless you copy every single subfolder ID manually
<RomeSilvanus[m]>
And if I copy the ID of /ipfs-sync/ then it will inlcude the content from all other folders too
<Discordian[m]>
You might be looking at fragments from the issue before
<Discordian[m]>
`/ipfs-sync/` will contain random files/dirs likely from the bug earlier
<Discordian[m]>
But otherwise yeah, each dir you add will be in `/ipfs-sync/`, and inside of those dirs is their regular tree you'd find on the fs
<RomeSilvanus[m]>
I did delete it in the web interface and removed the database dir before the new try
<Discordian[m]>
Oh okay, yeah so if you wanted to share all the dirs, you'd have to sync their root dir. Or make like, a symlink directory of everything you want to be shared, then share that
<Discordian[m]>
Because you're right, manually syncing `/ipfs-sync/` is manual work
<Discordian[m]>
That way you can curate what directories are added to ipfs-sync, while not reorganizing them on your filesystem.
<Discordian[m]>
* That way you can curate what directories are added to ipfs-sync within that directory, while not reorganizing them on your filesystem.
Newami has quit [Client Quit]
<RomeSilvanus[m]>
This would however be a lot of manual work to upkeep
<RomeSilvanus[m]>
It snot necessarily a big problem for me now
<RomeSilvanus[m]>
I just find it to be a weir design decision and can see other people having a problme with it
<RomeSilvanus[m]>
* I just find it to be a weir design decision and can see other people having a problem with it
<Discordian[m]>
Yeah to do what you want, I think I'd need some sort of mapping feature that people could customise. It'd be hard/impossible for me to guess every use-case
<RomeSilvanus[m]>
* I just find it to be a weird design decision and can see other people having a problem with it
<RomeSilvanus[m]>
Wouldn't it just be creating the parent dir in the mfs? You read the full paht on the fs at the beginning anway so it should be readily available
<Discordian[m]>
I figure this type of mapping is almost literally just symlinks, but on MFS, so I figured people would just use symlinks tbh
defnofdumb_ has quit [Ping timeout: 264 seconds]
<Discordian[m]>
It doesn't read the full parent actually, just the added root dir
<RomeSilvanus[m]>
* Wouldn't it just be creating the parent dir in the mfs? You read the full path on the fs at the beginning anway so it should be readily available
<Discordian[m]>
I think filestore does something with parents, I get a lot of data dumped after I switched to filestore
<Discordian[m]>
But ipfs-sync itself doesn't read them
<RomeSilvanus[m]>
But you have to parse the path inside the dir field in the json
<RomeSilvanus[m]>
* But you have to parse the path inside the dir field in the json in any case so you already would have access to it?
arcatech has joined #ipfs
<Discordian[m]>
Yeah, I *can* find the parent, but I don't want to crawl anything not explicitly assigned by the user. Maybe I could add a "add to parent" type feature, if it's something people usually want. I'll open an issue, see if anyone has interest.
<Discordian[m]>
BTW thanks to every for the stars today, the number has increased today 🙂
<Discordian[m]>
* BTW thanks to everyone for the stars today, the number has increased today 🙂
defnofdumb has joined #ipfs
<RomeSilvanus[m]>
The folders I add are mostly mirrored from google drive where I do the organizing and are just supposed to be read only for everything accessing them. So playing around with symlinks would required a lot of manual work to keep in mind what has changed where.
<RomeSilvanus[m]>
An option like this would actually be nice (and honestly should be the default since a user would expect the folder structure be kept the way they put it in the config)
<Discordian[m]>
<RomeSilvanus[m] "Sorry for complaining so much"> It's valuable feedback, no worries!
<Discordian[m]>
I think a symlink would be just as much work TBH, as how often are root directories renamed/moved/deleted? Rn you'd still have to edit the config to signify the change anyways.
<RomeSilvanus[m]>
Yeah, I don't wanna be one of those people who complain to oss projects to implement all their niche ideas
<RomeSilvanus[m]>
It's more the case on when I added different folders with subdirs in different locations on the FS as to not have all the subfolders get mixed together in the mfs. Plus it provides an easy way to share the whole folder with it's content if you so desire. Which can be practical for when you want to share different content with different people without having everyone have access to everything.
<Discordian[m]>
I think you'd convinced me a but further with that block tbh, it does seem easier for someone to quickly enable a feature like that, without worrying about setting up symlinks.
<Discordian[m]>
* I think you'd convinced me a bit further with that block tbh, it does seem easier for someone to quickly enable a feature like that, without worrying about setting up symlinks.
<Discordian[m]>
It would be nice for you to be happy with the MFS tree, as I feel like MFS is often ignored, and making it more useful would be a benefit.
<Discordian[m]>
So I'll try to add that feature for you, when I think of a good logical way to implement it with a simple flag.
<Discordian[m]>
Probably just gonna be `AddToParent`
<RomeSilvanus[m]>
It's just what I would expect. Since you as a user intent to find your files and folders organized the same way as their are on the FS. And more inexperienced useerr will probably be wondering why it looks different in the mfs and where their folder structure has gone also.
<Discordian[m]>
It's funny because me personally, I'd be annoyed if that was the default haha, but as an option, I see the benefit now.
<RomeSilvanus[m]>
Nice. But don't make yourself work too hard just because of my suggestions, it's not that terrible for me now.
<Discordian[m]>
Eh, I'm not working too hard, don't worry.
<Discordian[m]>
I'd rather get this thing as useful as possible, so I really appreciate your feedback
<Discordian[m]>
I certainly appreciate your patience too 😉
<Discordian[m]>
Did v0.3.3 work out BTW?
<RomeSilvanus[m]>
👌
<RomeSilvanus[m]>
... I forgot the set is as executable !
<Discordian[m]>
I stripped out part of the error handling for that and added in `FIXME`s, so it's only a temp fix, but glad it works out.
<RomeSilvanus[m]>
Running it now (again)
<RomeSilvanus[m]>
Nothing is as permanent as a temporary solution.
arcatech has joined #ipfs
<Discordian[m]>
Yeah I basically just need to add a generic error handler for error the daemon sends over, see if the field is empty, if so, move on. I was checking if any output happens in certain spots, and handling that as an error lol.
<Discordian[m]>
* Yeah I basically just need to add a generic error handler for errors the daemon sends over, see if the field is empty, if so, move on. I was checking if any output happens in certain spots, and handling that as an error lol.
drathir_tor has quit [Remote host closed the connection]
<RomeSilvanus[m]>
if [ -n "$error" ]; then
<RomeSilvanus[m]>
rm -rf /
<RomeSilvanus[m]>
fi
<Discordian[m]>
Yeah! Just nuke everything, less machines to have issues that way.
<Discordian[m]>
Also rn I make a Win binary, but I bet the Win binary doesn't work, as it definitely doesn't expect `\` directory paths.
<Discordian[m]>
Need to VM that at some point and make it properly handle Windows madness
<RomeSilvanus[m]>
Doesn't the windows cmd work fine with /?
<RomeSilvanus[m]>
Rclone is written in go and can take them just fine, unless they do some internal converting
<RomeSilvanus[m]>
also >windows
<Discordian[m]>
Maybe, if I wrote it properly xplatform, it'd definitely work on Windows. However rn if a uses specifies a path like `C:\mydir\`, I'm betting it won't even make it through the new sanity checks.
<Discordian[m]>
* Maybe, if I wrote it properly xplatform, it'd definitely work on Windows. However rn if a user specifies a path like `C:\mydir\`, I'm betting it won't even make it through the new sanity checks.
<RomeSilvanus[m]>
Just make the user install cygwin as punishment for using windows
<Discordian[m]>
Haha WSL works great actually, I never use cygwin these days
<Discordian[m]>
It'd definitely work under WSL
<RomeSilvanus[m]>
So if you would add that option in and I#d enable it would I have to reindex everything or is there a way to move the folders into the right directory?
<Discordian[m]>
Uhm, honestly, it'd probably need a reindex currently. Unless I add some sort of migration feature. Good point though, don't want people to be screwed over by enabling or disabling the feature at some point.
likolo[m] has quit [Quit: Idle for 30+ days]
M0xdz[m] has quit [Quit: Idle for 30+ days]
mistakia[m] has quit [Quit: Idle for 30+ days]
FledgeShiu[m] has quit [Quit: Idle for 30+ days]
musabal[m] has quit [Quit: Idle for 30+ days]
milano92[m] has quit [Quit: Idle for 30+ days]
ifrz[m] has quit [Quit: Idle for 30+ days]
Brian[m]7 has quit [Quit: Idle for 30+ days]
micahzoltu[m] has quit [Quit: Idle for 30+ days]
nehyaaaa[m] has quit [Quit: Idle for 30+ days]
quinncuatro[m] has quit [Quit: Idle for 30+ days]
jeffkhull[m] has quit [Quit: Idle for 30+ days]
tyjak[m] has quit [Quit: Idle for 30+ days]
rueking[m] has quit [Quit: Idle for 30+ days]
tundeglobal[m] has quit [Quit: Idle for 30+ days]
gwa[m] has quit [Quit: Idle for 30+ days]
leolemon[m] has quit [Quit: Idle for 30+ days]
andreas2021[m] has quit [Quit: Idle for 30+ days]
jan[m]5 has quit [Quit: Idle for 30+ days]
Jona[m]1 has quit [Quit: Idle for 30+ days]
chewyhamster[m] has quit [Quit: Idle for 30+ days]
sam[m]11 has quit [Quit: Idle for 30+ days]
someonejv[m]1 has quit [Quit: Idle for 30+ days]
Donieck666Kamili has quit [Quit: Idle for 30+ days]
lowry_s[m] has quit [Quit: Idle for 30+ days]
wassili[m] has quit [Quit: Idle for 30+ days]
masterberg[m] has quit [Quit: Idle for 30+ days]
walmart[m] has quit [Quit: Idle for 30+ days]
ultra_nick[m] has quit [Quit: Idle for 30+ days]
marcgroos[m] has quit [Quit: Idle for 30+ days]
arcticdeal[m] has quit [Quit: Idle for 30+ days]
Daafuku[m] has quit [Quit: Idle for 30+ days]
angryagain[m] has quit [Quit: Idle for 30+ days]
urzone[m] has quit [Quit: Idle for 30+ days]
slowswingsam1[m] has quit [Quit: Idle for 30+ days]
leo1616[m] has quit [Quit: Idle for 30+ days]
yovghg[m] has quit [Quit: Idle for 30+ days]
<RomeSilvanus[m]>
Maybe there's some ipfs commands to modify the mfs, I didn't read through any docs for it yet.
<Discordian[m]>
Okay updated the issue, you won't need to reindex.
<Discordian[m]>
There are, I can unlink a directory, and link it somewhere else
<jonathan444[m]>
Is it true that IPFS is initially not a distributed storage by default until some other node finds a hash and decides to pin/cache the file?
<jonathan444[m]>
For example, if I run a node along with 8000 other peers in swarm, and then I add a file using ipfs add, that file is still only on my node right? So if my ipfs node is down, file cannot be retrieved from other node's gateway right?
<Joris[m]1>
Theoretically yes. At least that is my interpretation.
<Joris[m]1>
But IPFS gives you no way of knowing/checking, that a file has been fetched from you.
<swedneck>
but yes, data will only be on your node unless another node fetches it from you, you can't push data onto others
Newami has joined #ipfs
<Joris[m]1>
That raises a question for me, that I have been bothered by for a while now: What is the relation between the GDPR (DSGVO) and IPFS?
<eleitl[m]>
If a node offers a web gateway you can fetch a particular CID which will remain in that ipfs node until garbage-collected.
<jonathan444[m]>
<swedneck "> <@jonathan444:matrix.org> Is i"> I read somewhere there IPFS by-default pins a file on the primary node where the file was added. If that's true, the file is guaranteed to be accessible from any IPFS gateway in swarm as long as my node is up and connected with the swarm, right?
Joris[m]1 is now known as CSDUMMI[m]
<swedneck>
<jonathan444[m] "I read somewhere there IPFS by-d"> sure, unless you decide to immediately remove the data just because you feel like it:D
<CSDUMMI[m]>
<jonathan444[m] "I read somewhere there IPFS by-d"> I think so.
<swedneck>
> <@jonathan444:matrix.org> I read somewhere there IPFS by-default pins a file on the primary node where the file was added. If that's true, the file is guaranteed to be accessible from any IPFS gateway in swarm as long as my node is up and connected with the swarm, right?
<swedneck>
* sure, unless you decide to immediately remove the data just because you feel like it :D
<swedneck>
point being, the network is made up of independent nodes that you can't control in any way, so you can't rely on others to store your data unless you have some out-of-band method of enforcing it, such as Filecoin or simply asking a friend to pin something and relying on them being nice to you
<jonathan444[m]>
Cool. So to be on the safer side, it's better to run multiple nodes and pin files on each so in case a node is down, other is still available to get the file from. Agree?
<jonathan444[m]>
<swedneck "point being, the network is made"> Gotcha! Thanks a lot!
<swedneck>
<jonathan444[m] "Cool. So to be on the safer side"> yes. one of the great things about ipfs is that it makes adding redundancy absolutely trivial compared to HTTP
<swedneck>
no centralized load balancers needed, just spin up as many nodes as you want and users of the network will find them automatically
<swedneck>
and you'd use something like ipfs-cluster to sync data between your nodes
Pikipeanut has joined #ipfs
<jonathan444[m]>
Wow. I didn't know about ipfs-cluster. Is it helpful to spawn multiple nodes and will it ensure that file added to any node will also be pinned on others in the cluster?
<swedneck>
it's only really helpful if it's on separate physical devices
<swedneck>
if you run two nodes on one computer there's no real benefit since any failure will bring down both
<jonathan444[m]>
Agreed. Thanks! 👍
<swedneck>
it's the same logic as any backup/load balance/CDN system, since IPFS can do the work of all 3
jcea has joined #ipfs
edperry has joined #ipfs
rudolph has quit [Quit: Leaving]
chiuii has joined #ipfs
edperry has quit [Quit: Connection closed]
edperry has joined #ipfs
<edperry>
I am looking at a IPFS implmentation within our corp network. I see the version is still <1.0 is it stable enough for a corporate enviroment. Also in the road map, I see there is an issue with +1gb files can anyone tell me the scope of that?
<Discordian[m]>
I find it pretty stable, but depending on what you do, you could ofc run into bugs.
<Discordian[m]>
I haven't had any issues with >1GB files.
<Discordian[m]>
Only bug I'd ran into was after I hammered the `files/write` API with >18k files. As a work-around, now I just use `add`, then put it into MFS manually.
<edperry>
like what kind of bugs, I guess I can read though the issues on the git site to see. which is a good idea.
<edperry>
Ah ok don't think I will be initially doing that much r/w
<RomeSilvanus[m]>
I sure hope not since I have files that are up to 280GB
<Discordian[m]>
<edperry "like what kind of bugs, I guess "> Yeah I'd check the Github tracker, but really for any project before v1 release, there may be unexpected issues we can't really forsee.
<Discordian[m]>
Personally though, having a great time with it.
<edperry>
yah ok, good to hear there is someone with +200gb don't think I will be dealing with that much data initially
<edperry>
well there is always a unexpected issues, we are IT where the impossible happens ever day
<Discordian[m]>
Ofc, Rome shows me fun bugs in my software all the time.
<RomeSilvanus[m]>
Anyway I'm trying to add around 40TB with 30.000.000+ files so we'll see how that goes
<Discordian[m]>
Lmfao, using ipfs-sync?
<edperry>
we just do so many FTP's sftp's ..... and so on with multi cloud and vendors thoguth it might be a good fit
<RomeSilvanus[m]>
I would add the full 144TB I have stashed on my google drive but I don't have that kind of space on my server
<Discordian[m]>
This sure will be a big test
<Discordian[m]>
It should add fine ... after an eternity.
<RomeSilvanus[m]>
yes, using ipfs-sync
<edperry>
ok so there does not seem to be an upper limit on FS space, other then disk capacity
<edperry>
hmm, have to look at ipfs-sync
<edperry>
not there yet though, just standing up a basic cluster and learning
<Discordian[m]>
<edperry "ok so there does not seem to be "> You can set a limit for the gc to kick in. It won't clear out pinned/MFS files, but it'll clear out the cache if it gets near your set limit.
<RomeSilvanus[m]>
What exactly does the hashing files step actually do, and could't it start adding files whil it's still running to speed everything up a bit?
<edperry>
Hmm, will read the site over the weekend and put it on my list of TODO to check out
<CSDUMMI[m]>
I want to make it possible to send data from one machine to another, both running IPFS. What can I do with IPFS in this scenario?
<Discordian[m]>
<RomeSilvanus[m] "What exactly does the hashing fi"> Hashing files builds a local DB of hashes so it can track if files are modified while the daemon is offline. Sometimes a comparison is done to determine if a change happened.
<eleitl[m]>
Rome Silvanus: if you want to publish a lot of files on a single node you should enable the filestore experiment and use --nocopy
<RomeSilvanus[m]>
* What exactly does the hashing directory step actually do, and could't it start adding files whil it's still running to speed everything up a bit?
<Discordian[m]>
He is using nocopy, but I'm assuming he also has another server pinning the data or something
<edperry>
Discordian[m] mmm GC limits well that could be a tunning to look at
<RomeSilvanus[m]>
Nah, just this one
<RomeSilvanus[m]>
I'm a cheapskate
<edperry>
@csdu
<RomeSilvanus[m]>
Hold together by ductape and a ruined ubuntu installation
<RomeSilvanus[m]>
* Held together by ductape and a ruined ubuntu installation
<Discordian[m]>
<CSDUMMI[m] "I want to make it possible to se"> You could use libp2p, IPFS has some libp2p features you can use built-in
<edperry>
also been playing with IPFS in docker but most of the docs talk about VM's other then the Networking in docker is there any know issues ? I'd rather not spawn up multi vm in multi clouds to run it
<Discordian[m]>
You could add the file into IPFS, get the CID, send the CID over via libp2p, receiver pins the CID, there, data shared.
<Discordian[m]>
<edperry "also been playing with IPFS in d"> Not sure, I assume docker works fine.
<Discordian[m]>
I don't personally use docker if I can avoid it
<RomeSilvanus[m]>
I never had problems with ipfs in docker, except that time when it was spamming the private ip range in the datacenter and Hetzner sent me angry emails
<edperry>
LOL RomeSilvanus[m]
LiftLeft has joined #ipfs
<edperry>
I should avoid that since I am running in a provate network
<RomeSilvanus[m]>
And I put basically everything into docker since it makes it much easier to manage r/w access, permissions, network, port, ips, etc. And it keeps the local system clean.
LiftLeft has quit [Max SendQ exceeded]
LiftLeft has joined #ipfs
<RomeSilvanus[m]>
This wasn't a docker specific problem, just that ipfs started scanning everything. I ended up blocking it in iptables.
<edperry>
yah that is sort of what I was thinking and with Azure you can run docker containers without any servers so even more benifit
<edperry>
I was interested in reading the Kubernetes deployment but that is way to complex for what I am thinking right now
<edperry>
Thanks everyone, I have to run, but be warned I will be back :D
<RomeSilvanus[m]>
Don't forget to share all you CID so we can enjoy your company secrets
<RomeSilvanus[m]>
* Don't forget to share all you CIDs so we can enjoy your company secrets
<Discordian[m]>
Haha welcome to the community and cya!
<RomeSilvanus[m]>
* Don't forget to share all you CIDs so we all can enjoy your company secrets
<edperry>
LOL sure, though access to my proviate network wont be :D
<Discordian[m]>
Yeah there's a private swarm feature for IPFS if you wanted your own private network
<Discordian[m]>
It'll encrypt everything using the swarm key
<RomeSilvanus[m]>
How secure is this actually?
<Discordian[m]>
Looks decently secure, unless the swarm key gets compromised
<RomeSilvanus[m]>
Nice
<RomeSilvanus[m]>
Nice <del>Now I can finally share my illegal pictures of hand holding with the world</del>
<RomeSilvanus[m]>
* <del>Now I can finally share my illegal pictures of hand holding with the world</del>
<Discordian[m]>
Lol, also eventually ipfs-sync will be able to handle encryption, and allow you to share dirs with others, and the encryption key will be securely transmitted.
<RomeSilvanus[m]>
Nice²
Meli has quit [Ping timeout: 276 seconds]
Pikipeanut has quit [Quit: A bientôt!]
Meli has joined #ipfs
xti has joined #ipfs
xti has left #ipfs [#ipfs]
zeden has joined #ipfs
theseb has joined #ipfs
chiuii has quit [Ping timeout: 244 seconds]
Newami has quit [Quit: Leaving]
Newami has joined #ipfs
Newami has quit [Client Quit]
<jonathan444[m]>
Is there any way to implement Geo-fencing on public IPFS making sure that only the nodes of a particular country eg. US can only connect with my node? This is because a law in the US requires companies to store their customers data only in US (?)
<Discordian[m]>
<frandavid100 "I've got no idea what that would"> Like for example if you had no peers, you could add some nodes, and maybe find some through them.
<Discordian[m]>
<frandavid100 "Also, what's the use of finding "> To be able to share data of course!
<Discordian[m]>
If you had a node on a LAN restricted from the web, connected to a node that's connected to the web, you'd still be able to make full use of IPFS
<Discordian[m]>
Also if you ever didn't have access to the net, you'd still be able to share between the nodes, and have them discover each-other without issue.
<Discordian[m]>
If a LAN node is sharing something, it'd be frustrating if you relied entirely on bootstrapping from the net, and they didn't happen to find eachother. If you were trying to share a file for example.