stebalien changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.5.1 and js-ipfs 0.43.1 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of Conduct: https://git.io/vVBS0
daMaestro has joined #ipfs
Adbray has joined #ipfs
catonano_ has joined #ipfs
catonano has quit [Ping timeout: 240 seconds]
catonano_ is now known as catonano
daMaestro has quit [Quit: Leaving]
lord| has joined #ipfs
qnix[m] has joined #ipfs
vyzo has joined #ipfs
Belkaar has quit [Ping timeout: 258 seconds]
Belkaar_ has joined #ipfs
Jesin has quit [Quit: Leaving]
Jesin has joined #ipfs
katian_ has joined #ipfs
KempfCreative has joined #ipfs
user_51_ has joined #ipfs
matt-h has quit [Quit: Leaving]
user_51 has quit [Ping timeout: 240 seconds]
chachasmooth has quit [Ping timeout: 244 seconds]
chachasmooth has joined #ipfs
katian_ has quit [Quit: Leaving]
jrt has quit [Killed (wolfe.freenode.net (Nickname regained by services))]
jrt has joined #ipfs
jcea has quit [Quit: jcea]
ipfs-stackbot has quit [Remote host closed the connection]
ipfs-stackbot has joined #ipfs
vyzo has quit [Ping timeout: 272 seconds]
Jeanne-Kamikaze has joined #ipfs
Nact has quit [Quit: Konversation terminated!]
<JaCL[m]>
Hey, i have read that redirection of ipfs subdomains to localhost is supposed to work ie foo.tld/ipfs/cid > cid.ipfs.foo.tld but I have not been able to have it work. I have also tried through other gateways that support subdomains but it also does not work there. But i can access it via foo.tld:localhost:8080. Just wont redirect.
matthewcroughan has joined #ipfs
<matthewcroughan>
Hi!
<matthewcroughan>
Wow it's popular in here
<matthewcroughan>
If I have two systems nas1 and nas2, they are in separate buildings with a separate network. My laptop roams. Sometimes it is on the internet. Sometimes it is on the nas1 network, sometimes on the nas2 network, will IPFS know when I am on the LAN and avoid routing over the internet?
<JaCL[m]>
separate lans?
<matthewcroughan>
yes
<matthewcroughan>
nas1 is at home, nas2 is in a co-working space I am a member of, sitting under a desk.
<matthewcroughan>
I'm sorry, but you have the curse of knowledge lol.
Caterpillar has quit [Ping timeout: 256 seconds]
<matthewcroughan>
Again, I have not touched IPFS.
<JaCL[m]>
The way i use it is i have vps 1 and vps 2 vps 1 runs wireguard and i run my node locally with that external address. It is peered with vps 2 which is running a gateway
<JaCL[m]>
The documentation is very very good
<matthewcroughan>
I have read some of it, but I came here to ask something that is not written.
<matthewcroughan>
And you have just given me config docs. I am not running a node yet
<matthewcroughan>
I've given you an example, what about my example scenario is wrong or misconceptualised?
<matthewcroughan>
Also saying "1 runs wireguard" does not explain anything :D
<JaCL[m]>
Once you set your nodes up use ipfs id to show your ip and cid of that node. Set the "peering.peer" section on your second node with that info. Repeat the process on the other
<matthewcroughan>
I think you are speaking about something I did not ask.
<matthewcroughan>
I am not asking for the implementation, config, etc. I am asking for the high level overview.
<JaCL[m]>
You have 2 nas seperate lans that you want connected yes?
<matthewcroughan>
I cannot possibly understand what you just told me, unless you tell me what it is that I am getting wrong, on the high level.
<matthewcroughan>
Yes, but the fact that I want them connected is *not important*
<matthewcroughan>
I am asking about my *laptop*.
<matthewcroughan>
My laptop is roaming. Sometimes it is **ON THESE LANS**
<matthewcroughan>
I am asking how my ***client*** interacts. And whether it superfluously goes over the internet, or not.
<JaCL[m]>
if you are not on the lan using that gateway it will be route over the internet
<matthewcroughan>
"using that gateway"
<matthewcroughan>
I'm sorry, I am having a hard time understanding you.
<JaCL[m]>
localhost:8080
<matthewcroughan>
Okay? What's running there? You have not explained that?
<matthewcroughan>
What is "that gateway" ?
<matthewcroughan>
also, use commas?
<matthewcroughan>
I appreciate you trying to help. I really do. But you are talking past me. I am asking you whether my use case is possible or not. I do not need to know about the details of the nodes, unless you are going to not assume knowledge and tell me exactly the details, beyond "do x, do y."
<matthewcroughan>
I do not need the details. I just want the high level overview, please.
Caterpillar has joined #ipfs
matthewcroughan[ has joined #ipfs
<matthewcroughan>
Joining the matrix crashed my synapse :(
<JaCL[m]>
Short answer is yes, it will route over the internet. If you leave the lan and request a file you have on your nas it will be route over the internet.
<matthewcroughan>
Okay.
<JaCL[m]>
You can "pin" the files while you are on the lan so as to access them later offline. In that case it will be served to you locally
<matthewcroughan>
Now what if my laptop is on the same LAN as the nodes?
<matthewcroughan>
do IPFS clients automagically recognise this and prevent routing over the internet accidentally?
<matthewcroughan>
I thought this was the IPFS concept.
<matthewcroughan>
Also, wow.. I cannot join the Matrix, at all
<matthewcroughan>
My homeserver doesn't have enough ram for it.. That sucks
<matthewcroughan>
Synapse just keeps crashing.
<JaCL[m]>
Ipfs is content based. So you basically just ask the network "hey who has this file?" if your node is the only one that has the file "pinned" it will say "hey ye i have the file, here you go"
<matthewcroughan>
Yes, that does not answer my question at all though.
<matthewcroughan>
That has nothing to do with what I asked.
<matthewcroughan[>
Oh no..
<matthewcroughan>
test
<matthewcroughan[>
Ok, great. I'm here now. But typing in IRC actually mentions me on IPFS :D
<matthewcroughan[>
Mentions me on Matrix*
<JaCL[m]>
You should read up on DHT (Distributed hash table) it may give you a little more insight on what i mean
<matthewcroughan[>
I do not think it will.
<matthewcroughan[>
My question had nothing to do with how content is addressed on IPFS.
<matthewcroughan[>
Only to do with routing, but you have for some reason talked about how content is addressed.
<matthewcroughan[>
I appreciate your attempt to help, but for some reason you are not interpreting what I am asking correctly.
<JaCL[m]>
<matthewcroughan "If I have two systems nas1 and n"> Yes until you leave the lan
<matthewcroughan[>
didnt1able: How are conflicts managed?
<JaCL[m]>
pardon?
<matthewcroughan[>
Is everything in IPFS atomic?
<matthewcroughan[>
If I make a file, is it possible to end up in a situation where 1 node disagrees with another about which file was made first.
<matthewcroughan[>
Let's say I have two machines, attempting to make the same file.
<matthewcroughan[>
Because of the latency between `nas1` and `nas2`, if I tried to make the same file on both LANs, what would happen?
<JaCL[m]>
yes absolutly. The moment you change a file on nas1 it will change the CID and it will be different than on nas2
<matthewcroughan[>
So IPFS handles conflict resolution, by creating two versions of the file?
<JaCL[m]>
The second the file is altered, the CID is also. Its part of the built in security. The way around this is with ipns i believe. Im not 100% up to snuff on how that really works. But from my understanding is you can add a folder with ipfs add -r foo/bar/ and ipfs name publish <cid>
<matthewcroughan[>
That still does not answer the simple question I have.
<JaCL[m]>
The same piece of content added to two different IPFS nodes using the same settings will produce exactly the same CID.
<matthewcroughan[>
Okay, now let's take it one step further.
<matthewcroughan[>
let's say I have a folder on both IPFS nodes called `notes/`
<matthewcroughan[>
`notes/mydiary.txt` exists, has some content. It is the same CID on all nodes, at this time.
<matthewcroughan[>
`laptop1` edits the note on the same LAN as `nas1`
<matthewcroughan[>
`laptop2` waits a bit, but the note has not been fully distributed. `laptop2` prematurely edits `notes/mydiary.txt`, makes some changes, and writes the file.
<matthewcroughan[>
What happens when `notes/mydiary`, as written by `laptop1` finally propogates to `nas2`?
<matthewcroughan[>
* `laptop1` edits the note on the same LAN as `nas1`
<matthewcroughan[>
`laptop2` waits a bit, but the note has not been fully distributed. `laptop2` prematurely edits `notes/mydiary.txt`, on the same LAN as `nas2` makes some changes, and writes the file.
<matthewcroughan[>
my laptop only wants to see one file.
<matthewcroughan[>
This is a simple and basic merge conflict.
<matthewcroughan[>
Is it not possible to set one node as the master?
<matthewcroughan[>
yes, I guess it will produce two CIDs. But I only want 1.
<matthewcroughan[>
I want 1 CID, and I want `nas2` to decide which CID is the correct one.
<JaCL[m]>
You could used ipfs cluster for that but thats above my pay grade
Jeanne-Kamikaze has quit [Quit: Leaving]
<matthewcroughan[>
So merge conflicts result in 2 CIDs
<matthewcroughan[>
And there is no easy way to set a node to be master?
<JaCL[m]>
with cluser nas 2 will be aware of the changes on nas 1
<JaCL[m]>
and replicate
<matthewcroughan[>
Oh
<matthewcroughan[>
I did not realise this was required. I thought this was simply the way it already is.
<matthewcroughan[>
I thought cluster, was default.
<matthewcroughan[>
Okay, so back to the drawing board..
<JaCL[m]>
No no its seperate
<matthewcroughan[>
I want `nas1` and `nas2` to be permanently, atomically synchronized. Like atomic clocks in orbiting satellites.
<JaCL[m]>
ipfs cluster will do that
<JaCL[m]>
cluster.ipfs.io
<matthewcroughan[>
And then can I mount this and use it for my notes on my laptop? :D
<JaCL[m]>
then you can cluster the 2 nodes and they will hold the same data same CIDs
<matthewcroughan[>
And when I am on the same LAN as `nas1`, I will get a full 1Gb/s right?
<matthewcroughan[>
Since it will not be going over the internet
<matthewcroughan[>
I will not need to pin the file
<matthewcroughan[>
* I will not need to pin the file on my laptop
<JaCL[m]>
you will need to pin the file
<JaCL[m]>
pin -r
<JaCL[m]>
pin add -r
<matthewcroughan[>
I do not understand?
<matthewcroughan[>
I thought I could just access IPFS files like NFS?
<JaCL[m]>
on your nas. once clusterd you will need to ipfs add -r notes/ and it will give you a hash. that pins it with the -r
<matthewcroughan[>
pinning downloads the file to my laptop?
<matthewcroughan[>
and takes up space on the disk?
<JaCL[m]>
pinning holds it on the nas
<JaCL[m]>
pinning, yes
<matthewcroughan[>
No, this is backwards
<matthewcroughan[>
I want to put all my media on the NAS, then access it without downloading files to my laptop
<matthewcroughan[>
100% transient and happening over ethernet, like NFS
<JaCL[m]>
throw it all in one directory and add it
<JaCL[m]>
you dont pull all the files in when you grab the cid on your laptop
<JaCL[m]>
you must select individually or by folder
<matthewcroughan[>
I do not want the file to occupy any space on the laptop other than RAM
<matthewcroughan[>
do you know what NFS and SMB are?
<JaCL[m]>
it wont until you pull it in
<matthewcroughan[>
I do not know why you have to make this so hard :D
<JaCL[m]>
yes its not like nfs or smb
<matthewcroughan[>
Can you just tell me whether what I want is possible or not
<matthewcroughan[>
Can I or can I not use IPFS like NFS
<JaCL[m]>
not in the simple way that i assume you want
<matthewcroughan[>
I want to mount `/ipfs/` on all of my devices, and `ls` and see **all of the contents** of `nas1` and `nas2`
<matthewcroughan[>
A fuse mount would do this, would it not?
<JaCL[m]>
if you want to access it with the file manager
<matthewcroughan[>
so when I `ls` my mount location, will I see the entire contents of `nas1` and `nas2
<matthewcroughan[>
* so when I `ls` my mount location, will I see the entire contents of `nas1` and `nas2`
KempfCreative has quit [Ping timeout: 265 seconds]
<JaCL[m]>
Okay i dont think what you want to do is possible
<JaCL[m]>
or atleast wont be a great experience at all
<matthewcroughan[>
I want to use IPFS instead of SMB, NFS, etc.
<matthewcroughan[>
nas1 is at home. nas2 is at work.
<matthewcroughan[>
Same CIDs, atomically synced.
<matthewcroughan[>
When I'm away, use internet.
<JaCL[m]>
If you are using it as a working directory your going to have a bad time
<matthewcroughan[>
No local files stored on laptop.
<matthewcroughan[>
:(
<matthewcroughan[>
I kinda thought this was the point of IPFS though
deltab has joined #ipfs
<matthewcroughan[>
My literal description is the entire premise of IPFS
<matthewcroughan[>
Get what you can, from anywhere, in the shortest route, with the highest bandwidth
<JaCL[m]>
if you want to dump a b unch of files in a folder and have access to it regardless of location it will work well. if you want to use it as a working directory you will have a bad time
<matthewcroughan[>
so why not use something like syncthing then and avoid this complication?
<matthewcroughan[>
syncthing to sync the nas, serve files via NFS/SMB
<matthewcroughan[>
Or can I maybe use IPFS to sync the NAS, and continue to serve the files via SMB/NFS?
<JaCL[m]>
syncthing is probably what you want or seafile. ipfs is great because you can just access the files anywhere anytime with the hash
<matthewcroughan[>
seafile is out of scope
<matthewcroughan[>
seafile/nextcloud are not at all what I want, I do not know where you got that idea
<matthewcroughan[>
I want a single mountpoint, on all my roaming devices, to use nodes as working directories.
<JaCL[m]>
ew nextcloud
<JaCL[m]>
not ipfs not for a working directory
<JaCL[m]>
static sure
<matthewcroughan[>
I'm genuinely confused what it's for then
<matthewcroughan[>
I thought the entire premise of IPFS was that any content could be served from anywhere, for any purpose, in the shortest route
<matthewcroughan[>
yknow, hence INTERPLANETARY?
<matthewcroughan[>
I do not understand why being a working directory is not good?
<matthewcroughan[>
If it cannot accomplish working directories, then what is it good for?
<JaCL[m]>
lmao go read the docs dude
<matthewcroughan[>
I have read parts, do you expect me to read the whole thing in one night?
<matthewcroughan[>
I am here, asking you the question genuinely.
<matthewcroughan[>
The docs won't answer this for me tonight, but you can.
<matthewcroughan[>
What is it that makes IPFS unable to do working directories? Is it that it's not done yet?
<matthewcroughan[>
Or is this always going to be something that works poorly?
<matthewcroughan[>
Latest code anyone has written it seems
fling_ is now known as fling
Kubuxu has quit [Ping timeout: 240 seconds]
StephenWeb3Capit has quit [Ping timeout: 240 seconds]
<matthewcroughan[>
didnt1able: You have linked me to implementation details that need to be abstracted away from the user.
thattommyhall has joined #ipfs
<matthewcroughan[>
I do not care what order of things needs to happen in order to get the functionality of editing content.
Belkaar has joined #ipfs
Belkaar has joined #ipfs
Belkaar has quit [Changing host]
<matthewcroughan[>
If IPFS does "not support editing content", yet someone has written a paragraph that allows for the same functionality, then that is what the person asking the question means.
<JaCL[m]>
its not whatt its built for
<matthewcroughan[>
This is not the story told by marketing :D
Kubuxu has joined #ipfs
<matthewcroughan[>
The story told by marketing is that it is a filesystem you can mount the world with.
<matthewcroughan[>
And this is exactly why I came here to ask, because otherwise I would have had to really use IPFS to figure this out.
<matthewcroughan[>
That is in the context of a public network.
FennecCode has joined #ipfs
<matthewcroughan[>
That is not in the context of a private network. In a private network, none of the points made in the section you linked are true, they are out of scope.
sneakweb has joined #ipfs
<matthewcroughan[>
> If you can retrieve a file from someone nearby instead of hundreds or thousands of miles away, you can often get it faster. This is especially valuable if your community is networked locally
<matthewcroughan[>
Great. So I can get it fast, but I can't edit video on it. I cannot use it like I use SMB/NFS.
<matthewcroughan[>
Is it not easy to understand how I can get misled, and why I thought I was able to do this sort of stuff with it?
<JaCL[m]>
different tools for different jobs
<matthewcroughan[>
My idea was to get rid of SMB/NFS, and just have an IPFS FUSE mount at the root of all of my devices, and have a happy life.
wlp1s1 has joined #ipfs
hexa- has quit [Ping timeout: 240 seconds]
StephenWeb3Capit has joined #ipfs
<matthewcroughan[>
Additionally, the IPFS website claims it could lower bandwidth for Videos by 60%
<matthewcroughan[>
If what you've said so far is true, then that claim can never be realised.
<JaCL[m]>
serving video
<JaCL[m]>
as in CDN
<JaCL[m]>
more people watch the video more peers serving the content
<matthewcroughan[>
I do not understand how downstream and upstream are any different.
<matthewcroughan[>
reading vs writing
wlp1s1 is now known as iczero
<matthewcroughan[>
It is over my head as to why this has an impact, especially on a private network
hexa- has joined #ipfs
<matthewcroughan[>
Why does my use case suddenly change when I am writing instead of reading? This doesn't make sense.
<JaCL[m]>
you need to understand the tools and what you need for the job
<matthewcroughan[>
Private Network + MFS should absolutely be a replacement for SMB/NFS. I do not see so far why this is untrue.
<JaCL[m]>
Ipfs is not for editing video
<matthewcroughan[>
I understand what you are saying about the immutable FS only being useful as a CDN and verification/security.
<matthewcroughan[>
But a private cluster + mfs, should be fine for editing video. Why not?
<matthewcroughan[>
It may not be true today, due to lack of software to mount and interact with MFS. But what prevents this from being possible?
<JaCL[m]>
Technically there is nothing stopping you, i would be interested on your results
<matthewcroughan[>
These people do not know about MFS :D
<matthewcroughan[>
> I recently tried to use IPFS to share 20 gigabytes of data on a local network, using ipfs-desktop.
<matthewcroughan[>
Apparently it hashes the entire file, meaning it will read it entirely, so I killed it because it took way too long. Apparently there are other methods for fingerprinting large files, like reading a limited set of chunks. I asked on IRC but I did not get a thorough answer.
<matthewcroughan[>
* > I recently tried to use IPFS to share 20 gigabytes of data on a local network, using ipfs-desktop.
<matthewcroughan[>
> Apparently it hashes the entire file, meaning it will read it entirely, so I killed it because it took way too long. Apparently there are other methods for fingerprinting large files, like reading a limited set of chunks. I asked on IRC but I did not get a thorough answer.
<matthewcroughan[>
Okay, so this is not an issue if we use MFS right?
<matthewcroughan[>
If this was an issue for him, I don't know how people are going to distribute video on IPFS
<JaCL[m]>
distribute is VERY different from edit
<matthewcroughan[>
yes
<matthewcroughan[>
this guy was simply trying to distribute it
<JaCL[m]>
look up dtube its a great example of ipfs distributing video
<matthewcroughan[>
Oh, I used that a while ago
<matthewcroughan[>
I remember it sucking incredibly bad, and it still does to this day.
<matthewcroughan[>
No offense is meant by that, it's clear to anyone who uses it.
<matthewcroughan[>
There are limitations, and everything is not as great as proposed.
<matthewcroughan[>
I knew there was a 'blockchain' component to it, though I did not know it was actually IPFS.
<matthewcroughan[>
open bazaar is the best project I've seen use IPFS. It is absolutely perfect
RzR has joined #ipfs
<matthewcroughan[>
It is slow. And I understand why it is slow after this reading. But that doesn't change the fact that it is slow, and is going to remain slow until something happens. And I don't know what 'something' is.
<matthewcroughan[>
I thought though, that a private cluster would have no bottlenecks in this regard.
<matthewcroughan[>
Actually, it looks as if DTUBE has improved a lot..
<matthewcroughan[>
I'm going to guess it's using that fancy js-ipfs stuff.. This is kind of amazing
<JaCL[m]>
the more people who use it the stronger and faster
<matthewcroughan[>
Yes, like tor.
<JaCL[m]>
Like a torrent
<matthewcroughan[>
So what is the solution I want?
<matthewcroughan[>
I swear, it has to be MFS
<matthewcroughan[>
Do you know anything about filecoin? Do you know if they planned to use MFS or something?
<matthewcroughan[>
I always thought filecoin was vaporware.
<JaCL[m]>
filecoin? no idea
<JaCL[m]>
im not into crypto currencies
<matthewcroughan[>
Filecoin was some ICO that claimed it would monetize IPFS
<matthewcroughan[>
1 Filecoin == 1GB on IPFS, for example, for you to use
<matthewcroughan[>
And knowing what I know now, this would never work. Unless there was something about MFS.
<matthewcroughan[>
Additionally, this would only work if the recipient also hosted an IPFS node and was keeping this content alive for you.
<matthewcroughan[>
I do not see how you could convince people to keep your 1GB file around, on the public nodes
<matthewcroughan[>
It doesn't really make sense, I don't think it will work
<JaCL[m]>
Il do it lol depending what it is
<apotheon>
I think the idea was that people would pay filecoins for others to keep things live.
<matthewcroughan[>
Which requires more software to manage in the middle
<apotheon>
. . . or maybe "mining" would work by proof-of-hosting, now that I think about it.
<apotheon>
It was a while ago. I don't recall what I read.
<matthewcroughan[>
the Sia network is this, but in a way that makes complete sense, because it is a separate network, with the full intent of monetizing the content inside
<matthewcroughan[>
Content on Sia, requires monetization. 1GB == $x
<matthewcroughan[>
On IPFS, there is no concept of monetization. Adding it on top does not work.
<matthewcroughan[>
> MFS unreliable: I'll "ipfs files cp" something then Ctrl-C the daemon, start the daemon, and that change is missing. I believe it's this bug: https://github.com/ipfs/go-ipfs/issues/5868
<matthewcroughan[>
OK. So the bottom line is that IPFS is just not ready yet.
<matthewcroughan[>
The issue has been fixed. However, I bet there are still issues lurking.