aschmahmann changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.7.0 and js-ipfs 0.52.3 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of Conduct: https://git.io/vVBS0
bengates has quit [Remote host closed the connection]
__jrjsmrtn__ has joined #ipfs
bengates has joined #ipfs
rogc[m] has joined #ipfs
ecloud_ has joined #ipfs
ecloud has quit [Ping timeout: 240 seconds]
Arwalk has quit [Read error: Connection reset by peer]
pecastro has joined #ipfs
Arwalk has joined #ipfs
<MarkusKuhlmann[m>
Hi everyone,
<MarkusKuhlmann[m>
Thanks in advance!
<MarkusKuhlmann[m>
Im trying to setup a gitlab CI/CD pipeline, where I want to `ipfs add -r .` an entire directory to my IPFS node. In a second step, I want to `ipfs name publish HASH`. I cant seem to find a way to get hold of the root hash for the directory in a programmatic way. Can someone help me out here?
<McSinyx[m]>
you can upload to the gateway but those nodes don't have the capacity to store for long
arcatech has quit [Quit: Be back later.]
LHLaurini2 has quit [Client Quit]
<owocean[m]>
why?
<McSinyx[m]>
or even deny the call
<McSinyx[m]>
because they have the sole purpose of being a gateway, not pinning
<Discordian[m]>
Yeah most gateways aren't writeable
<owocean[m]>
but if i request a file from them, they'll still eventually get me the file even if they dont host it right?
<Discordian[m]>
Yeah
<McSinyx[m]>
yep if it's pinned by someone
<Discordian[m]>
They'll also cache what they serve temporarily
<owocean[m]>
<McSinyx[m] "yep if it's pinned by someone"> can that someone be a node running on MY machine?
<Discordian[m]>
Yes
<owocean[m]>
i see
<owocean[m]>
i dont need any node to host the file longer than a day
<owocean[m]>
so upload to my node, pin it, then call it from a gateway?
<Discordian[m]>
It doesn't have to be pinned, it just has to be available. So for example if you add a file, and then try to retrieve it on a gateway, as long as your node advertises it has it (probably via DHT), the gateway will retrieve and serve it
<owocean[m]>
pretty sick
<owocean[m]>
thank you for the help
<Discordian[m]>
No problem! Glad you're enjoying IPFS 🙂
bengates_ has quit [Read error: Connection reset by peer]
bengates has joined #ipfs
mamu has joined #ipfs
reit has joined #ipfs
Arwalk has quit [Read error: Connection reset by peer]
Arwalk has joined #ipfs
devdevgoat has joined #ipfs
arcatech has joined #ipfs
tech_exorcist has quit [Quit: tech_exorcist]
Arwalk has quit [Read error: Connection reset by peer]
ram19890 has quit [Quit: Konversation terminated!]
Arwalk has joined #ipfs
justanotherdude has joined #ipfs
arcatech has quit [Quit: Be back later.]
Caterpillar has quit [Quit: You were not made to live as brutes, but to follow virtue and knowledge.]
GvP has joined #ipfs
covid-1984[m] has joined #ipfs
<covid-1984[m]>
Is there such thing as a hybrid IPFS model? For example: you have a video self-hosted with peer-tube, but you want to boost it / reduce your bandwidth using IPFS. Possible? Done already? Thoughts?
<covid-1984[m]>
* Is there such thing as a hybrid IPFS model? For example: you have a video self-hosted with PeerTube, but you want to boost it / reduce your bandwidth using IPFS. Possible? Done already? Thoughts?
<Discordian[m]>
Yeah you could put the videos you want on a node, and have that node also be a gateway
<covid-1984[m]>
And then how could end users / views toggle whether they "re-stream" (what is the technical term for it)?
<Discordian[m]>
Well users using the gateway wouldn't be able to re-share, but users using IPFS could pin whatever content they want, and it'd also automatically re-share from their cache.
<covid-1984[m]>
I think they should be able to set how much bandwidth they want to use... Though I don't know if that is an IPFS feature.
<covid-1984[m]>
But at least viewers on mobile networks should not, I'd say.
<Discordian[m]>
I don't believe IPFS has that feature (yet). It would be nice to have though
<covid-1984[m]>
Yes, filter out low bandwidth / low latency "nodes"
<covid-1984[m]>
<Discordian[m] "Yeah you could put the videos yo"> Is gateway an IPFS-specific term?
<covid-1984[m]>
I'm new to IPFS
<Discordian[m]>
Yes, a gateway refers to a node that's acting as effectively an HTTP portal into IPFS.
<covid-1984[m]>
<Discordian[m] "Yeah you could put the videos yo"> And then you'd write an IPFS plugin or something that allows you to determine which nodes get direct streams from the server vs IPFS streams from another user?
<Discordian[m]>
Well if a user connects over IPFS you know they're using IPFS because they're not using HTTP. If they're using HTTP, they're not using IPFS
<Discordian[m]>
Don't need to change anything
<Discordian[m]>
Client-side you might need to setup some sort of client
<Discordian[m]>
But the node itself will be fine
<covid-1984[m]>
Well all users visiting the website will visit via HTTP... But then I wanna find the nicest way to reduce load using IPFS
LiftLeft has joined #ipfs
<Discordian[m]>
For example if someone is using IPFS Companion, with their own local node, if you have an IPFS website, it can automatically redirect them to the IPFS version, connecting them over IPFS. You can detect how they're connected to your website likely by checking the hostname (but maybe not always, I need to play with that technique more).
<Discordian[m]>
As for using the gateway, there will be no local IPFS node, so you could just use the gateway
<Discordian[m]>
Honestly IMO it'd be easier to just do everything over IPFS, it'll automatically load balance for you, and you could just set your own node as a bootstrap node
<Discordian[m]>
If you connect via gateway, could just use js-ipfs to retrieve the content
<covid-1984[m]>
Could you do tiers?
<covid-1984[m]>
For example, say you have a private video streaming site. And those you pay $10/mo should get top quality, and those who are in there free would have lower quality.
<Discordian[m]>
Uhmmm yes
<Discordian[m]>
Maybe
<Discordian[m]>
Actually not sure sec
<covid-1984[m]>
"tiered video quality" or something...
<covid-1984[m]>
It should be part of load balancing imo
<covid-1984[m]>
Because instead of no one being able to stream 1080p - let everyone stream 320p
<covid-1984[m]>
depending upon how much capacity your IPFS network has
<Discordian[m]>
The idea is you spread load balancing out among your users. You could restrict higher quality data via HTTP auth using reverse proxy and simply not share hq data on your own other nodes
Arwalk has quit [Read error: Connection reset by peer]
<covid-1984[m]>
Ya but think of the scaling quality of this feature in the IPFS network... If you're a node receiving 1080p, then you could maybe only stream to 1 other. But if you're receiving 320p, then you could send to 3 others for the same amount of bandwidth as 1080p.
<Discordian[m]>
If people want 320p, IPFS doesn't really treat media any differently than any other data.
Arwalk has joined #ipfs
<covid-1984[m]>
hmmm I guess I should try to use it like everyone else and then push this feature request later.
<Discordian[m]>
Maybe it'd be cool to see some sort of media categorization feature for alike data
<covid-1984[m]>
But it seems like the primary issue with distributed systems... If you don't have enough nodes then performance sucks.
<Discordian[m]>
* Maybe it'd be cool to see some sort of media categorization feature for alike data (though I'm not sure how it'd be done, it sounds cool)
<Discordian[m]>
Yeah, like you can leverage the wider network of course, but it's all still maturing
<covid-1984[m]>
Ya I'm more thinking private sites where you need an account. Then not many nodes likely to be online.
<Discordian[m]>
Oh yeah absolutely not, especially if they're using PrivateNetworks
<covid-1984[m]>
Also, I like the idea of being able to ban malicious nodes - possible in IPFS?
<Discordian[m]>
Currently only via reverse proxy AFAIK
<Discordian[m]>
I'd also like to see that feature
<Discordian[m]>
* Currently only via reverse proxy / firewall AFAIK
<covid-1984[m]>
which reverse proxy are ipfs users using?
<covid-1984[m]>
like traefik?
* jwh
uses traefik
<Discordian[m]>
I use Nginx personally, not sure what's most popular
<covid-1984[m]>
nice, already using treafik, so that's solved :)
<jwh>
plenty of choices though, pick whichever suits you
<covid-1984[m]>
The main case I want the hybrid model for is live streaming
<covid-1984[m]>
The hybrid model being shifting to more IPFS nodes rather than server bandwidth - during the live stream
<covid-1984[m]>
Because self hosting w/ live streaming gets expensive.
<Discordian[m]>
I'd like to see a good live streaming model on IPFS, not sure if one exists yet (I'd love to be corrected)
<Discordian[m]>
Oh actually libp2p might be able to handle that just fine
<covid-1984[m]>
<Discordian[m] "This definitely looks like a fun"> A blog post by you? I'm definitely trying to network with people who want to work on this :)
<Discordian[m]>
> It is possible. I guess one way to do that is to use pubsub 3 with one common topic for your platform with the available streams, etc., and then a topic per stream, that users can join. You can use the peerID of the streamer as a topic name, to save a time for your users (not having to ask around who knows which peers to contact to watch the stream). The app itself or the website can be decentralized via IPFS.
<Discordian[m]>
This is what I want to see!!
<Discordian[m]>
<covid-1984[m] "A blog post by you? I'm definite"> Yup! I'll try to get it published to the IPFS blog if we decide to do it
<jwh>
I looked at doing hls over ipfs, it's just not fast enough (yet)
<Discordian[m]>
I cannot collaborate on that project unfortunately, but I can share the journey!
<jwh>
DHT lookups take way to long
<Discordian[m]>
I keep DHT disabled in browser, I haven't been able to make it work correctly
<jwh>
I don't expect it will be that performant in a browser tbf, but even on well connected servers with absurdly high active peer limits
<jwh>
still a lot of unreachable dht nodes
justanotherdude has quit [Quit: RAGEQUIT]
<Discordian[m]>
You're making me want to experiment sooner than later haha
arcatech has joined #ipfs
<jwh>
do iiitttt
<Discordian[m]>
Gotta get the IPFS Chat blogpost up first maybe livestreaming will be the second or third project if people think it'd be useful
<Discordian[m]>
Second project will likely be a game tbh, I want to do a game example that has UGC
<covid-1984[m]>
Self hosted live streaming would be golden
<covid-1984[m]>
Where you have some granularity of control during the start of the stream so you don't DDoS yourself
<Discordian[m]>
Haha we'll see. I think these posts are mostly aiming to give people what they'd need to build their own toys, but from a working base.
<Discordian[m]>
Dw, for my tutorials I believe we're aiming for a more "turn-key" approach, so they'll likely all optionally use docker if there's a complicated setup.
<Discordian[m]>
Word of warning though, it's my first time using docker ;p. Seems simple enough once you're rolling though
<covid-1984[m]>
All good... Happy to comment / help with your docker files
<covid-1984[m]>
I'd recommend sticking to alpine-based dockerfiles and using docker-compose for everything.
<Discordian[m]>
I've used a debian image, and haven't used docker compose unfortunately.
<Discordian[m]>
But it's 2 lines to setup and run
<covid-1984[m]>
you just make one docker-compose.yml and define your services.
<covid-1984[m]>
instead of making your own nginx, use the most popular nginx letsencrypt helper compose project
ylp has quit [Quit: Leaving.]
zeden has joined #ipfs
<Discordian[m]>
Hmm, might have saved time (that part's already complete)
<Discordian[m]>
I should have DMd you when I was setting this up lmao
<covid-1984[m]>
great :) I'll stay in here
<Discordian[m]>
Haha cheers, I'll send the guide your way too once we get more rolling with it (or maybe before). Definitely want the docker setup to at least be easy. I don't personally use docker ever though, but we wanted some way for people to get their node stuff quickly rolling.
mindCrime_ has joined #ipfs
<covid-1984[m]>
public service announcement: iptables are totally ignored by docker, and sysadmins might not realize it
arcatech has joined #ipfs
p8m_ has joined #ipfs
bengates has quit [Remote host closed the connection]
p8m has quit [Ping timeout: 240 seconds]
LinusCDE has quit [Remote host closed the connection]
mamu has quit [Ping timeout: 240 seconds]
NikolaPerovi[m] has joined #ipfs
<Discordian[m]>
Lmao I had no idea
<jwh>
technically not ignored, you just need to adjust ruleset to use the docker chain
<jwh>
(also docker sucks at iptables)
HenriCarnot[m] has joined #ipfs
mindCrime_ has quit [Ping timeout: 265 seconds]
mamu has joined #ipfs
LinusCDE has joined #ipfs
mamu has quit [Ping timeout: 240 seconds]
haijuno has joined #ipfs
bengates has joined #ipfs
jesse22 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
bengates has quit [Ping timeout: 240 seconds]
konghans[m] has joined #ipfs
reit has quit [Ping timeout: 260 seconds]
russ152 has joined #ipfs
chiui has joined #ipfs
devdevgoat has quit [Ping timeout: 240 seconds]
jesse22 has joined #ipfs
kex has quit [Read error: Connection reset by peer]
kex has joined #ipfs
chiui has quit [Remote host closed the connection]
chiuii has joined #ipfs
kex has quit [Ping timeout: 240 seconds]
theseb has quit [Quit: Leaving]
kex has joined #ipfs
mamu has joined #ipfs
bengates has joined #ipfs
kex has quit [Ping timeout: 265 seconds]
kex has joined #ipfs
bencevans6 has joined #ipfs
chiui has joined #ipfs
chiuii has quit [Quit: Leaving]
bengates has quit [*.net *.split]
LiftLeft has quit [*.net *.split]
g2anj has quit [*.net *.split]
lord| has quit [*.net *.split]
is_null has quit [*.net *.split]
tedious has quit [*.net *.split]
arthuredelstein has quit [*.net *.split]
cxl000 has quit [*.net *.split]
cypher has quit [*.net *.split]
wking has quit [*.net *.split]
graffen has quit [*.net *.split]
dexter0 has quit [*.net *.split]
search_social has quit [*.net *.split]
paddymahoney has quit [*.net *.split]
matthewcroughan has quit [*.net *.split]
bencevans6 is now known as bencevans
bencevans has quit [*.net *.split]
is_null has joined #ipfs
g2anj has joined #ipfs
tedious has joined #ipfs
LiftLeft has joined #ipfs
bengates has joined #ipfs
lord| has joined #ipfs
arthuredelstein has joined #ipfs
cypher has joined #ipfs
paddymahoney has joined #ipfs
cxl000 has joined #ipfs
matthewcroughan has joined #ipfs
dexter0 has joined #ipfs
wking has joined #ipfs
search_social has joined #ipfs
graffen has joined #ipfs
search_social has quit [Max SendQ exceeded]
matthewcroughan has quit [Max SendQ exceeded]
is_null has quit [Max SendQ exceeded]
lord| has quit [Max SendQ exceeded]
matthewcroughan has joined #ipfs
search_social has joined #ipfs
is_null has joined #ipfs
lord| has joined #ipfs
kex has quit [Ping timeout: 240 seconds]
Renich[m] has joined #ipfs
bengates has quit [Ping timeout: 252 seconds]
Moonspell has joined #ipfs
<lidel>
Hi friends, if anyone wants to help with testing new IPFS Desktop v0.15.0-rc1 (macOS, Windows or Linux), and can spend 5-15 minutes, details are in https://github.com/ipfs/ipfs-desktop/issues/1812
<lidel>
<Discordian[m] "lidel: should I see "Pins" here?"> No, we removed it as everything in MFS is protected from gc:
mamu has quit [Ping timeout: 240 seconds]
<Discordian[m]>
Oh okay makes sense
<FuseTim[m]>
Yeah it has been removed from webui.ipfs.io as well
<Discordian[m]>
> That is why we v2.12 makes Files (MFS) the preferred way to keep things around by regular users, and will move more of our GUIs in that direction.
<Discordian[m]>
Mmm I love this direction
<Discordian[m]>
Anythingg specific you want tested? AppImage and tarball seem to work
mamu has joined #ipfs
<lidel>
Nothing specific, just a smoke-test (in the past Electron broke things like Tray on linux etc).
<Discordian[m]>
Ah I 100% understand that Electron is ... fun :D
kn0rki has joined #ipfs
kex has quit [Quit: bye]
kex has joined #ipfs
<Discordian[m]>
I wonder if I should be telling people to save something into MFS instead of pinning it if they're just an end user. Probably reduce confusion a lot. I feel like this direction will go far
Arwalk has quit [Read error: Connection reset by peer]
mindCrime_ has joined #ipfs
Arwalk has joined #ipfs
mindCrime_ has quit [Ping timeout: 240 seconds]
f4r598 has joined #ipfs
f4r59 has quit [Read error: Connection reset by peer]
Nact has quit [Quit: Konversation terminated!]
bengates has joined #ipfs
arcatech has quit [Quit: Be back later.]
haijuno has quit [Ping timeout: 276 seconds]
bengates has quit [Ping timeout: 240 seconds]
zeden has quit [Quit: WeeChat 3.0.1]
Arwalk has quit [Read error: Connection reset by peer]
SkyLounge[m] has left #ipfs ["User left"]
zeden has joined #ipfs
Arwalk has joined #ipfs
mindCrime_ has joined #ipfs
zeden has quit [Quit: WeeChat 3.0.1]
zeden has joined #ipfs
zeden has quit [Client Quit]
rsheftel1 has quit [Ping timeout: 276 seconds]
rsheftel1 has joined #ipfs
<kallisti5[m]>
lidel: *technically* you can change "files" to "pins" in the url bar and it still worksish
zeden has joined #ipfs
<kallisti5[m]>
I feel like y'all missed an array of valid url's or something :-)
zeden has quit [Client Quit]
zeden has joined #ipfs
f4r598 has quit [Read error: Connection reset by peer]
f4r5983 has joined #ipfs
zeden has quit [Quit: WeeChat 3.0.1]
zeden has joined #ipfs
AlexanderGoncalv has joined #ipfs
mindCrime_ has quit [Ping timeout: 260 seconds]
LHLaurini has joined #ipfs
haad_ has quit [Ping timeout: 245 seconds]
<owocean[m]>
can someone explain how the ipfs uri scheme works
<owocean[m]>
and how i can actually like, handle it
haad has joined #ipfs
<owocean[m]>
with chrome maybe?
Bat`O_ has quit [Ping timeout: 245 seconds]
Bat`O has joined #ipfs
zeden has quit [Quit: WeeChat 3.0.1]
figyfaldaa[m] has joined #ipfs
KempfCreative has joined #ipfs
<lidel>
<kallisti5[m] "lidel: *technically* you can cha"> I think that was left to facilitate transition, in case someone really needs it. But that screen will go away at some point.
jesse22 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<kallisti5[m]>
Makes sense
<lidel>
Discordian: yes, for regular users MFS is way easier to wrap head around and to manage "things i want to keep around", and the concept of "pinning" is easier to explain in context of remote pins and their value (ensuring this thing is kept around by some other node, so others can reach even when you "close the laptop").
jcea has quit [Quit: jcea]
jcea1 has joined #ipfs
<kallisti5[m]>
I just realized ipfs files chcid doesn't do what I thought it did... so there's no way to "update" the CID your MFS is attached to?
<kallisti5[m]>
* I just realized ipfs files chcid doesn't do what I thought it did... so there's no way to "update" the (root) CID your MFS is attached to?
<kallisti5[m]>
(sorry, trying to not be a constant source of trouble :-) Just trying to figure out a workflow)
<lidel>
kallisti5: hm.. good question. I don't think you can override MFS root. `ipfs files cp /ipfs/{dir-cid} /` will create an item named `/{dir-cid}`, so can't override this way
<kallisti5[m]>
That seems problematic. I mean.. what if I have 100 people mirroring that MFS root?
<kallisti5[m]>
Then my node goes down... I'd have to rebuild it knowing the state of it
<kallisti5[m]>
* Then my node goes down... I'd have to rebuild it knowing the state of it to update it.
<kallisti5[m]>
* Then my node goes down/explodes/disk fails... I'd have to rebuild it knowing the state of it to update it.
<lidel>
Why you need to mirror the root specifically? Can't you use a subdir in MFS?
<kallisti5[m]>
sure, but i'm still rebuilding. ```ipfs files setroot /ipns/hpkg.haiku-os.org``` would be pretty handy
<kallisti5[m]>
* sure, but i'm still rebuilding. `ipfs files setrootcid /ipns/hpkg.haiku-os.org` would be pretty handy
<lidel>
yeah, I guess for now you need to do `ipfs files cp $(ipfs resolve -r /ipns/hpkg.haiku-os.org) /hpkg.haiku-os.org`
<lidel>
it will create subdir `/hpkg.haiku-os.org`
<lidel>
no, it will create `/{cid-of-that-dnslink}`
<kallisti5[m]>
eeh. so unless I put everything into a subdirectory in MFS, you really can't easily recover the old file structure
Arwalk has quit [Read error: Connection reset by peer]
<kallisti5[m]>
"pointy bits everywhere" -- IPFS 😆
<lidel>
hm.. I seem to miss the point why subdirectory is a problem. It makes it easier to reason about data, you can even name them to indicate time of the snapshot + have `latest` to indicate the current one.
<kallisti5[m]>
I like IPFS btw, and want to see it used more. Just a lot of the issues I run into are paper cuts around stuff like this
<kallisti5[m]>
I make automation assuming /thing /thingb, etc.
<kallisti5[m]>
now I need to update that automation if I lose data.. /things/thing /things/thingb
<kallisti5[m]>
also ```ipfs files cp $(ipfs resolve -r /ipns/hpkg.haiku-os.org) /hpkg.haiku-os.org``` isn't super clear
<kallisti5[m]>
(take this as user feedback.. i'm honestly not complaining :laugh
<kallisti5[m]>
* (take this as user feedback.. i'm honestly not complaining :amused:
<kallisti5[m]>
* (take this as user feedback.. i'm honestly not complaining
Arwalk has joined #ipfs
<lidel>
ack, for what its worth, it is also problematic the other way, `ipfs files cp /ipns/hpkg.haiku-os.org /hpkg.haiku-os.org` would make users think they copied a living directory that will get updated when dnslink gets changed, which is not the case (what is copied is static snapshot)
kn0rki has quit [Quit: Leaving]
<lidel>
by requiring an immutable path, at least there is no ambiguity
<lidel>
Would be cool to have a way to "pin" `/ipns/` addrs in a way that follows updates, but its tricky to make a distinction betwen mutable and immutable, when you start mixing them in MFS.
KempfCreative has quit [Ping timeout: 252 seconds]
<lidel>
Curiously, this topic got brought up recently in the context of remote pinning services. As we have more services that can be used with `pin remote` commands, we may look into having ability to "pin" IPNS addr remotely, and have service automatically repin new CID every time IPNS pointer changes. Details TBD, but this sounds like useful pattern for website and dataset hosting.
<lidel>
Ok friends, its EOD for me, have a nice weekend! 💤
haijuno has joined #ipfs
pecastro has quit [Ping timeout: 260 seconds]
Mateon1 has quit [Remote host closed the connection]
Mateon1 has joined #ipfs
mindCrime_ has joined #ipfs
LinusCDE has quit [Read error: Connection reset by peer]