stebalien changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.4.19 and js-ipfs 0.34 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS, the InterPlanetary FileSystem: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of
spinza has quit [Quit: Coyote finally caught up with me...]
vandemar has quit [Read error: Connection reset by peer]
vandemar has joined #ipfs
clemo has quit [Ping timeout: 250 seconds]
renich has quit [Quit: renich]
placer14 has quit [Quit: placer14]
placer14 has joined #ipfs
mdrights has joined #ipfs
placer14 has quit [Quit: placer14]
ygrek has joined #ipfs
daopu[m] has joined #ipfs
The_8472 has quit [Ping timeout: 252 seconds]
The_8472 has joined #ipfs
shizy has joined #ipfs
daMaestro has joined #ipfs
infinitesum has joined #ipfs
infinitesum has quit [Client Quit]
infinitesum has joined #ipfs
infinitesum has quit [Client Quit]
infinitesum has joined #ipfs
infinitesum has quit [Client Quit]
infinitesum has joined #ipfs
noresult has joined #ipfs
noresult has quit [Client Quit]
noresult has joined #ipfs
shizy has quit [Quit: WeeChat 2.4]
xlued has quit [Read error: Connection reset by peer]
xlued has joined #ipfs
Belkaar has quit [Read error: Connection reset by peer]
Belkaar has joined #ipfs
Belkaar has joined #ipfs
mowcat has quit [Remote host closed the connection]
merethan has quit [Remote host closed the connection]
xlued has quit [Ping timeout: 245 seconds]
cp- has quit [Quit: Disappeared in a puff of smoke]
verin0x has quit [Ping timeout: 255 seconds]
verin0x has joined #ipfs
<TUSF>
Copenhagen_Bram: Yeah, there's no need to worry on that end. By default, the only thing on your node is whatever you ask for, much like torrents. The main difference is that IPFS will cache things you request and "seed" them in the background until you run garbage collection.
<TUSF>
The only time someone else can put something onto your node, is if you have a public-facing gateway
<TUSF>
Generally, it's good practice to not expose your node's API to the internet.
cp- has quit [Quit: Disappeared in a puff of smoke]
cp- has joined #ipfs
_whitelogger has joined #ipfs
Steverman has quit [Ping timeout: 245 seconds]
m_tadeu_ has joined #ipfs
The_8472 has quit [Ping timeout: 240 seconds]
m_tadeu has quit [Read error: Connection reset by peer]
tangent has joined #ipfs
tangent128 has quit [Read error: Connection reset by peer]
tangent is now known as Guest16809
infinitesum has quit [Quit: infinitesum]
infinitesum has joined #ipfs
The_8472 has joined #ipfs
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
daMaestro has quit [Quit: Leaving]
obensource has joined #ipfs
zane has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
arpit has joined #ipfs
Pulse2496 has joined #ipfs
^andrea^71608945 has joined #ipfs
^andrea^7160894 has quit [Ping timeout: 246 seconds]
Adbray has quit [Read error: Connection reset by peer]
kapil____ has joined #ipfs
nimaje has quit [Ping timeout: 250 seconds]
brabo has quit [Ping timeout: 252 seconds]
ylp has joined #ipfs
arpit_ has joined #ipfs
arpit has quit [Quit: Leaving]
lnostdal has quit [Ping timeout: 246 seconds]
Adbray has joined #ipfs
lnostdal has joined #ipfs
BeerHall has joined #ipfs
BeerHall has quit [Ping timeout: 240 seconds]
BeerHall has joined #ipfs
test[m6 has joined #ipfs
arpit_ has quit [Ping timeout: 250 seconds]
Fessus has quit [Quit: Leaving]
mowcat has joined #ipfs
alexgr has joined #ipfs
clemo has joined #ipfs
clemo has quit [Ping timeout: 245 seconds]
pie__ has joined #ipfs
clemo has joined #ipfs
mdrights has quit [Ping timeout: 245 seconds]
mdrights has joined #ipfs
BeerHall has quit [Quit: BeerHall]
<carrot^>
Testing our IPFS for the first time. I have two systems running IPFS. I have published a file and received a hash for that file. I use the web GUI to try to "explore" for it on the IPLD, and it's still searching (10 minutes later). I then instead try to access it directly via ipfs.io/ipfs/myhash and the page is still connecting 10 minutes later. I am now wondering if I'm doing something wrong or if this is normal
<carrot^>
To clarify, I'm trying to publish on one machine and access it on the other as a test. Ideally I want to find it, and then share it as well to test that functionality too.
<geoah>
carrot^ are your ipfs nodes accessible from the outside properly? ie port forwarding correctly setup or upnp enabled and used
brabo has joined #ipfs
<carrot^>
I imagine so, what's the best way to test that? When I use `ipfs swarm peers` it shows numerous connections
<geoah>
that can be connections initialized from your end -- something like canyouseeme.org will probably help assuming figure out what port you should be looking at
<geoah>
the peerinfo should contain all your addresses
<carrot^>
hmm. the peerinfo showed numerous IPs and ports: I tried all of them just for overkill and all of them showed as closed. I think you're right about it not being internet-open. I'm on a macbook, so I'm still not familiar with the firewall settings.
<carrot^>
Going into firewall, it actually says "Firewall is off: all income connections are allowed"
nimaje has joined #ipfs
ekardnam has joined #ipfs
<carrot^>
I do have 8 errors in the console though like this: `17:29:30.813 ERROR bitswap: Ended request for cid (QmPKPhRf2FaxEPeYPa1PebmoLLENgHyq7Sd8UJSSVpnti4) not in progress providerquerymanager.go:358`
<carrot^>
I tried using `ipfs name publish hash` to see if that would give me a different result for accessing it, and it said it was a success in publishing
<carrot^>
Published to `hashhere`: /ipfs/`otherhash`
<carrot^>
I'm assuming that won't help solve my problem of initial accessibility though.
yeehi has joined #ipfs
chiui has joined #ipfs
Steverman has joined #ipfs
spinza has quit [Quit: Coyote finally caught up with me...]
mdrights has quit [Ping timeout: 268 seconds]
spinza has joined #ipfs
rendar has joined #ipfs
<geoah>
carrot^ how are you port forwarding?
<carrot^>
good question. i have not done any manual port forwarding, I assumed it was set up automatically
<carrot^>
for what it's worth, the web GUI shows I have incoming data (18kb/s)
<carrot^>
although I don't know if that's just measuring the data coming in from connections I initiated or not (probably)
MuffinPimp has quit [Ping timeout: 246 seconds]
<geoah>
carrot^ start by checking if your router supports upnp and check the upnp mappings on it to see if the ipfs daemon has registered it self
mowcat has quit [Remote host closed the connection]
<carrot^>
logged into the router: shows as this:
<carrot^>
`libp2p55278TCP4001 MYIP`
<geoah>
cool, check if canyouseeme.org works for 55278
<carrot^>
Error: I could not see your service on MYIP on port (55278)
<geoah>
if not, you might have a firewall on the box you are running ipfs that is blocking the connection
<carrot^>
im running it from my macbook pro at the moment. the firewall is disabled, but i'm learning about it to see what disabled actually means.
}ls{ has joined #ipfs
vmx has joined #ipfs
manray has quit [Ping timeout: 255 seconds]
clemo has quit [Ping timeout: 250 seconds]
natewalck has quit [Quit: Connection closed for inactivity]
merethan has joined #ipfs
manray has joined #ipfs
yeehi_ has joined #ipfs
yeehi has quit [Ping timeout: 246 seconds]
malaclyps has quit [Read error: Connection reset by peer]
malaclyps has joined #ipfs
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
barnacs has quit [Remote host closed the connection]
placer14 has joined #ipfs
placer14 has quit [Quit: placer14]
placer14 has joined #ipfs
clemo has joined #ipfs
Steverman has quit [Ping timeout: 245 seconds]
spinza has quit [Quit: Coyote finally caught up with me...]
Steverman has joined #ipfs
mdrights has joined #ipfs
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
spinza has joined #ipfs
zeden has joined #ipfs
Steverman has quit [Ping timeout: 255 seconds]
Zerataxutf8[m] is now known as ZerataX[m]
mdrights has quit [Quit: WeeChat 2.3]
barnacs has joined #ipfs
Soo_Slow has joined #ipfs
Ai9zO5AP has joined #ipfs
Soo_Slow has quit [Remote host closed the connection]
bpol80 has joined #ipfs
Steverman has joined #ipfs
merethan has quit [Remote host closed the connection]
brabo has quit [Ping timeout: 245 seconds]
Pulse2496 has quit [Ping timeout: 246 seconds]
xcm has quit [Remote host closed the connection]
ygrek has joined #ipfs
xcm has joined #ipfs
brabo has joined #ipfs
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
reit has quit [Ping timeout: 245 seconds]
bmwiedemann has joined #ipfs
<bmwiedemann>
is that normal, that go-ipfs daemon uses 23GB RAM after a day (on my 32GB system this is eating most of the RAM already) or could there be memory leaks?
tombusby has quit [Remote host closed the connection]
<carrot^>
Should display a bunch of random public keys with random IPs
<carrot^>
(I mean, check and see if the hash works inside IPFS, rather than on the ipfs.io website)
chiui has quit [Ping timeout: 245 seconds]
Fessus has joined #ipfs
ekardnam has quit [Ping timeout: 255 seconds]
infinitesum has quit [Quit: infinitesum]
ZaZ has quit [Ping timeout: 255 seconds]
infinitesum has joined #ipfs
renich has quit [Ping timeout: 245 seconds]
MuffinPimp has quit [Quit: Goodbye.]
MuffinPimp has joined #ipfs
Fessus has quit [Quit: Leaving]
infinitesum has quit [Quit: infinitesum]
infinitesum has joined #ipfs
mowcat has quit [Remote host closed the connection]
infinitesum has quit [Client Quit]
infinitesum has joined #ipfs
zane has joined #ipfs
eknadon has joined #ipfs
zane has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<abueide[m]>
carrot^: its not working for me but i only have 4 peers and idk if other content will even work for me
zane has joined #ipfs
<abueide[m]>
carrot^: nevermind, it ended up working it just took a while
<carrot^>
great! how about with the IPNS hash too? QmUrwJWBoGFz5nh5UiFsZ6PbakoxZ2Bjh4WPuSSF2vQZMe
<carrot^>
(I presume those are loaded the same way?)
rendar has quit []
<abueide[m]>
ipfs resolve -r /ipns/QmUrwJWBoGFz5nh5UiFsZ6PbakoxZ2Bjh4WPuSSF2vQZMe: routing: not found
<abueide[m]>
is what i got
<abueide[m]>
usually with ipns you can get a file with /ipns/hash/path/to/file
<abueide[m]>
i'm not sure if that has to do with the issue but probably not
<carrot^>
`Published to QmUrwJWBoGFz5nh5UiFsZ6PbakoxZ2Bjh4WPuSSF2vQZMe: /ipfs/QmZrZsVgEubGYy71HqxGWhRME7F2eWdZWzb6yufj2CV3Sx`
<Mikaela>
it took a long time to load, but I can now see http://[::1]:8082/ipfs/QmZrZsVgEubGYy71HqxGWhRME7F2eWdZWzb6yufj2CV3Sx
<carrot^>
Mikaela: thank you!
<carrot^>
I am now convinced that the normal hash routing works, just need to figure out what I'm doing wrong for the IPNS one
<Mikaela>
I guess it's time for me to swap Syncthing and IPFS ports so it would be closer to what everyone and everything assumes
<Mikaela>
I haven't ever published IPNS, but I think it should be similar. Do you have EnableAutoRelay enabled if you are behind NAT?
dasj19 has joined #ipfs
Soo_Slow has quit [Ping timeout: 268 seconds]
<carrot^>
good question, i'll check
<carrot^>
okay dumb of me, but I didn't see anything referencing it so I didn't know but, turns out IPNS addresses are accessed by /ipns/ rather than /ipfs/
<carrot^>
Also I'm dumb, the error was because I was trying to use IPNS on the non-IPNS hash
<Mikaela>
At least you are finding out the problems to probably not repeat them in the future and I like how you tell the solutions without being poked about it separately :D
infinitesum has quit [Quit: infinitesum]
Taoki has joined #ipfs
thelounge65 has joined #ipfs
boredmanicrobot has joined #ipfs
<carrot^>
next on the to-do list for me is figuring out how to download the file and rebroadcast it in the most easiest/streamlined way
* carrot^
checks the --help
boredmanicrobot has quit [Client Quit]
thelounge65 has quit [Client Quit]
reit has joined #ipfs
eknadon has quit [Quit: Leaving.]
infinitesum has joined #ipfs
infinitesum has quit [Quit: infinitesum]
<carrot^>
hmm. not getting much of anywhere, but thought if I could at least see the current files I'm sharing, that would get me somewhere,
<carrot^>
and lo' and behold, the web UI doesn't even show the file I'm sharing anymore
<carrot^>
how do i list the files im sharing currently in the CLI?
<carrot^>
i'm confused -- so it does work, but it just hangs while operating?
<carrot^>
more importantly -- is there a way to get the normal IPFS hash from the IPNS address?
<carrot^>
if I have that, i don't need to use the IPNS address for pinning anyway
<carrot^>
I was just worried that people who wanted to share my file too wouldn't be able to as they'd likely be accessing it by the IPNS hash solely
<carrot^>
if they can derive the IPFS CID from it though, then they can pin that instead of course
eknadon has joined #ipfs
pecastro has joined #ipfs
eknadon has quit [Max SendQ exceeded]
eknadon has joined #ipfs
i1nfusion has quit [Remote host closed the connection]
eknadon has quit [Max SendQ exceeded]
i1nfusion has joined #ipfs
eknadon has joined #ipfs
eknadon has quit [Read error: Connection reset by peer]
<carrot^>
okay so I apparently successfully "pinned" it on both machines, how would I go about testing to make sure it's shared across both machines?
<carrot^>
is there a way to see the current nodes sharing it?
kapil____ has joined #ipfs
<Mikaela>
I don't understand it either, but seeing which nodes are sharing it should probably be "ipfs dht findprovs <hash>"
<carrot^>
is there a way to list current pins/files being shared? one machine shows the proper file, the other just says "hosting 5MB of files" but doesn't show any files when i click on it
<carrot^>
ah so apparently lots of other nodes have also pinned my file
<carrot^>
that command returns my IPNS hash plus 4 others
<carrot^>
so at least 3 others have also shared my file (was that people in here?)
<carrot^>
one day I should hope to be proficient enough to help you as well :D
<Mikaela>
I think it doesn't know about pin status and finds it from caches of the people who clicked it here, maybe gateways, and unless we pin it, it will disappear from us when the gc runs
<Mikaela>
:)
monokrome has joined #ipfs
<monokrome>
hiii
<monokrome>
Does anyone know if IPFS is suitable for large, frequently changing files?
<monokrome>
EG, does it apply diffs or upload the files over and over again?
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
Fessus has joined #ipfs
<carrot^>
monokrome: as I understand it, each file has it's own unique hash (like a git repo would) and when you upload a new version of the file/folder, the has changes completely -- but you can use IPNS to create a "namespace" hash like a domain name works to reference it for the "latest" updated version
i1nfusion has quit [Remote host closed the connection]
<carrot^>
I'm about to test the whole "changing files" thing right now for the first time though
<monokrome>
carrot^: Sure, but my use-case would be one large file and I'm wondering if it diffs the changes
<carrot^>
each changed version of the file would have its own unique hash, but I don't think it automatically tells you (like github for example) what's different about the two versions, it just provides them
<monokrome>
EG, if I flip 3 bits in the file, do I upload 3 bits, a small chunk, or the entire file?
<carrot^>
I don't know for sure
i1nfusion has joined #ipfs
<carrot^>
I'd think the entire file
i1nfusion has quit [Read error: Connection reset by peer]
<monokrome>
Hopefully not
<Mikaela>
I think in the repo only the small chunk changes as there is deduplication, but it may hash it entirely, I am not sure either.
i1nfusion has joined #ipfs
<monokrome>
If only a small chunk changes and there's a tree to rebuild the entire file then it seems safe to use large files
<Mikaela>
I have seen some javascript library being referred to often in context that it takes a lot less space in ipfs repo than outside of it, but I cannot find the source (maybe if I was a coder and cared about the library)
<monokrome>
ah, that'd just be because two of the same files will have the same hash, I think
<carrot^>
makes sense
<monokrome>
I think another way to think about this is whether a 1TB file is taking up 1TB in someone's disk or if it is sharded
<monokrome>
If it's sharded then files are collections of smaller ones which means it would only need to update whatever size those chunks are
<carrot^>
not necessarily, but not sure
<Mikaela>
ttps://github.com/ipfs/archives/issues/35#issuecomment-164002475 - here, 22 GB shrunk into 13 GB
<carrot^>
you might also consider sharing a folder instead and having the file be split into smaller files
<monokrome>
I can just create a 1TB file and see how it goes lol
<carrot^>
then each file would have its own unique hash and you'd only need to reupload/redownload the ones that changed
<monokrome>
yeah but then the folder hash has metadata about all the files and stuff, right?
<Mikaela>
I think the 1TB file size depends on if there is anything to deduplicate and what file format it is, I think someone said that Linux images may not deduplicate well due to isos having bytes in different positions or something
clemo has quit [Quit: clemo]
<monokrome>
Sure, but the question is about diffing more than deduping
<monokrome>
I'm wondering if throwing a LUKS disk on IPFS would be abusing the system or not
<monokrome>
sounds like it would
Fessus has quit [Remote host closed the connection]
Fessus has joined #ipfs
<Mikaela>
I think no one else would be hosting it unless they especially requested it. I have read about some trying to run distributions or binaries on top of IPFS, but hitting IPFS not storing executable bit, but that probably wouldn't affect LUKS
<carrot^>
^
<monokrome>
I'm going to run a node that can host it
<carrot^>
it doens't automatically propagate everything you host
<monokrome>
but if nobody else hosts it then I guess that means it doesn't shard and IPFS isn't the right thing to use
<carrot^>
it only seems to propagate the hash for it
<carrot^>
if you get others (like a torrent) to share it too (i learned from Mikaela that that is "pinning"), then they too will download it and serve it as well I think
<monokrome>
Yeah, I think the default config only accepts up to 10GB files anyway
<monokrome>
I'm just trying to use it as a dropbox alternative but w/ local encrypt/decrypt
<monokrome>
Yeah, I'm not sure that one shards the data either
<monokrome>
or diffs files
<Mikaela>
and that is my understanding too, sometimes I add files that I don't know if anyone else cares about, but at least they will be available if someone cares. But currently my main use for IPFS is loading images so I don't have to put them to git(hub pages)
<carrot^>
did you try Tahoe-LAFS?
<monokrome>
haven't heard of that
<Mikaela>
oh, sorry, I got hung up on dropbox alternative and I am using syncthing to sync my PCSX2 savestates & memcards
<monokrome>
Mikaela: I need a dropbox alternative that is good at hosting a single 1TB file
<monokrome>
the single file can't be easily deduplicated since it's a LUKS disk
* Mikaela
has no ideas to that, she wouldn't even have 1 TB of space anywhere
dasj19 has quit [Quit: dasj19]
* monokrome
has 50TB on her NAS, but finding a way to sync a folder on it around to everything >.<
* monokrome
explodes
alexgr has quit [Read error: Connection reset by peer]
<Mikaela>
https://tahoe-lafs.org/trac/tahoe-lafs is what carrot^ suggested, I haven't used it personally, but my partner uses it if I have understood correctly. It's supposedly similar to IPFS, but also handles private files or networks between just your servers and I have no idea on deduplication or sharding
<monokrome>
ya I can't find anything on sharding either
<carrot^>
ask in #tahoe-lafs !
<monokrome>
I think I'll probably code a sharding layer on top of IPFS maybe, dunno
<monokrome>
oh there's a channel? O_o nice
Fessus has quit [Remote host closed the connection]
<monokrome>
thanks y'all <3
Fessus has joined #ipfs
<Mikaela>
You're welcome, good luck :)
infinitesum has quit [Quit: infinitesum]
infinitesum has joined #ipfs
Fessus has quit [Remote host closed the connection]
Fessus has joined #ipfs
infinitesum has quit [Client Quit]
infinitesum has joined #ipfs
zane has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
sbani has quit [Ping timeout: 246 seconds]
Jesin has quit [Quit: Leaving]
pecastro has quit [Ping timeout: 246 seconds]
Jesin has joined #ipfs
seba-- has quit [Changing host]
seba-- has joined #ipfs
Ai9zO5AP has quit [Quit: WeeChat 2.4]
clemo has joined #ipfs
i1nfusion has quit [Remote host closed the connection]
i1nfusion has joined #ipfs
ygrek has quit [Ping timeout: 258 seconds]
zeden has joined #ipfs
Soo_Slow has quit [Remote host closed the connection]
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
screensaver25 has joined #ipfs
zane has joined #ipfs
spinza has quit [Quit: Coyote finally caught up with me...]
screensaver has quit [Ping timeout: 250 seconds]
alexgr has joined #ipfs
xcm has quit [Remote host closed the connection]
xcm has joined #ipfs
spinza has joined #ipfs
hlon[m] has joined #ipfs
xelra has quit [Remote host closed the connection]
xelra has joined #ipfs
zeden has quit [Quit: WeeChat 2.4]
kapil____ has quit [Quit: Connection closed for inactivity]
lordcirth_ has joined #ipfs
<lordcirth_>
I can confirm RAM usage improvements in the latest git.
<lordcirth_>
Previously, while pinning Wikipedia, it would hit it's 6GB RAM cap in 3 hours and be killed by systemd. It's been running at 5GB for ~16 hours now.