wavis has quit [Quit: Connection closed for inactivity]
TheWhisper has joined #ipfs
A124 has joined #ipfs
ligi has joined #ipfs
ligi has quit [Changing host]
ligi has joined #ipfs
chrisg_ has quit [Ping timeout: 240 seconds]
<elimisteve>
Thank you lgierth and achin for your guidance the other day re: pointing me to https://ipfs.io/api/v0/ls?arg=/ipfs/... to view folder contents. Super useful! Now I would like to create a folder that multiple people can contribute to. Is that possible right now? Thanks.
ilmu has joined #ipfs
shizy has joined #ipfs
<zignig>
elimisteve: not at the moment , merging merkeldags in a sane way has been talked about but no working tools exist.
<zignig>
elimisteve: that said , once the publish/subcribe stuff lands it will be possible.
<zignig>
i'm also working on a share-o-tron for mfs , but it still buggy a as termite mound.
<zignig>
whyrusleeping: wassup ?
ygrek has quit [Ping timeout: 265 seconds]
Oatmeal has joined #ipfs
<elimisteve>
zignig: where can I learn how pub/sub will work? I saw references to it on GitHub but not a specific design
<zignig>
I don't think the docs exist yet, can you read golang source ??? ;P
<elimisteve>
or is pub/sub far enough out that the design isn't there?
<elimisteve>
ah so some pub/sub code already exists? Sweet
<pinbot>
now pinning /ipfs/QmS2vtQU3tZqdx7LY8dJrWeeRec288bGgDH6jEMCkcbnWX
<whyrusleeping>
elimisteve: uhm... i'm not sure what youre talking about?
<whyrusleeping>
zignig: the go-ipfs branch is feat/floodsub
<whyrusleeping>
we're gonna ship it to master pretty soon though
<elimisteve>
whyrusleeping: zignig brought up pubsub in response to my original question:
<elimisteve>
> Now I would like to create a folder that multiple people can contribute to. Is that possible right now? Thanks.
<zignig>
elimisteve: my bad , pub sub will allow this kind of infrastructure to be built, it does not do it at the moment.
ppham has quit [Remote host closed the connection]
<pinbot>
[host 4] failed to grab refs for /ipfs/QmS2vtQU3tZqdx7LY8dJrWeeRec288bGgDH6jEMCkcbnWX: Post http://[fc3d:9a4e:3c96:2fd2:1afa:18fe:8dd2:b602]:5001/api/v0/refs?arg=/ipfs/QmS2vtQU3tZqdx7LY8dJrWeeRec288bGgDH6jEMCkcbnWX&stream-channels=true&r=true&encoding=json&: dial tcp [fc3d:9a4e:3c96:2fd2:1afa:18fe:8dd2:b602]:5001: getsockopt: connection timed out
<zignig>
elimisteve: what is your application for ?
matoro has joined #ipfs
JesseW has joined #ipfs
<elimisteve>
zignig: Functionality-wise I just need multiple people to be able to add files to a directory
<elimisteve>
but I'm building an end-to-end encrypted wiki. I can store these (encrypted wiki page) files in Dropbox, the CrypTag app running on Sandstorm, and any local folder
<elimisteve>
and I have started implementing an IPFS backend
<zignig>
and then have that directory available to all participands.
<elimisteve>
zignig: yes exactly
Kane` has joined #ipfs
<elimisteve>
so if only one person can add content to an IPFS folder, then you could still securely share any information -- wiki pages, chat messages, files, etc -- but only one person could create those files, ha
<elimisteve>
I have some graphical app demos and several command line utilities for encrypting and storing files in such a way that they can be tagged and then searched by tag without encrypting anything
<zignig>
elimisteve: I have been thinking about how to do that too , will read up.
<elimisteve>
so if I can get multiple users sharing a folder, I can basically use CrypTag to turn each IPFS folder into a shared, encrypted, uncensorable database and backend as a service for apps to talk to
<elimisteve>
which I'm excited about :-) and hope people like jbenet and whyrusleeping are, too!
<whyrusleeping>
the whole reason i started hacking on ipfs was to have a shared filesystem across all my devices
<zignig>
whyrusleeping: shared file system for free**
<zignig>
:)
<whyrusleeping>
so yeah, i'm pretty interested in that elimisteve :D
<elimisteve>
whyrusleeping: sweet :-D. When might it be possible for multiple IPFS users to add files to a directory? I suppose a hack would be to grab some ipfs-generated key and share it with someone?
shizy has quit [Ping timeout: 250 seconds]
<whyrusleeping>
elimisteve: i'm not going to be able to have time to directly work on that stuff for quite a while
<whyrusleeping>
i can however prioritize spending time on the pieces of ipfs that make that easier to build
<whyrusleeping>
now that we have pubsub, it becomes much easier to build such an app
<elimisteve>
okay, what needs to exist, but doesn't, to make it possible for multiple people to edit a folder?
<whyrusleeping>
really, for just that description of it, pubsub
<whyrusleeping>
and we have that now
<whyrusleeping>
its experimental
<whyrusleeping>
and we're going to merge it into master in its experimental state
<whyrusleeping>
but it should allow you to build such an application
<whyrusleeping>
once you start talking about encryption and write permissions and stuff you will need multikey stuff
<whyrusleeping>
which we're hoping to have implemented later this year
JesseW has quit [Quit: Leaving.]
JesseW has joined #ipfs
<lgierth>
whyrusleeping: i was wondering, is floodsub reliable?
<elimisteve>
Awesome. So I have my own crypto layer already
<lgierth>
i.e. are there ways for subscriber to make sure they eventually get all messsages?
<elimisteve>
whyrusleeping: I can have multiple people add files to a directory without multikey stuff there yet?
<whyrusleeping>
lgierth: no
Aeon has quit [Ping timeout: 248 seconds]
<whyrusleeping>
lgierth: we're going to build that layer on top of the current layer
<whyrusleeping>
elimisteve: you could write an app that enables it, yes
<whyrusleeping>
using pubsub for coordination of writes
Aeon has joined #ipfs
<elimisteve>
whyrusleeping: interesting, okay... What will be an obvious sign that pubsub is merged into master?
<elimisteve>
I want to know exactly when that happens. And is the source of https://github.com/libp2p/go-floodsub the best place I should turn to learn about how pubsub works/is about to work?
ppham has quit [Remote host closed the connection]
wolfden has quit [Quit: whoops]
Guest78451 has quit [Remote host closed the connection]
computerfreak has quit [Remote host closed the connection]
espadrine_ has joined #ipfs
reit has quit [Ping timeout: 244 seconds]
pfrazee has joined #ipfs
pfrazee has quit [Ping timeout: 265 seconds]
<haad>
victorbjelkholm: hey hey! actually, I just pushed something that gets rid of the whole thing. we decided that Orbit will always spawn its own daemon and not try to use a running one.
<pinbot>
now pinning /ipfs/QmXpZ7yvSYP1XjyczUTgycnehUe1qntwsmtbTS9vbqGerv
<victorbjelkholm>
haad, awesome! is in the pubsub branch of orbit?
<haad>
let's see if pinbot can reach me behinf the great firewall
<haad>
victorbjelkholm: yup!
<victorbjelkholm>
haad, great, I'll test it out in a bit
<haad>
victorbjelkholm: would be awesome if you did a fresh clone and build it (not just 'run electron' but 'npm run build' and then try to run the app in bin/)
<victorbjelkholm>
haad, how is china?
<haad>
victorbjelkholm: thank you!
<victorbjelkholm>
haad, sure, I'll try with a fresh clone
<haad>
victorbjelkholm: haha, hot, rainy and asian :) <3
<errl>
Hmm, if I have two clients and one client is offline, how could I ensure that a hash destined for the offline client is received once the client's status is online again?\
<victorbjelkholm>
haad, just like pinbot, I cannot access that hash... Tried locally and two different servers...
<nuun>
victorbjelkholm: same here
<nuun>
I only tried the hash though
<victorbjelkholm>
errl, guess you have two ways, either is to poll from client A to client B until client B is online, other is for client B to ask for content once he is online
<victorbjelkholm>
probably other ways as well, just want comes to my mind
Bheru27 has quit [Remote host closed the connection]
Bheru27 has joined #ipfs
robogoat has quit [Ping timeout: 244 seconds]
<errl>
victorbjelkholm: I believe the latter would be better in my use case, but how would client B know which content to grab?
<errl>
That is where I'm stuck.
<victorbjelkholm>
errl, you can have a hash publish under IPNS that contains the HEAD of your content, that clients would check
<victorbjelkholm>
could be a list of IPNS to check as well, to add redundancy
<errl>
But the IPNS would have to be online correct? I would like to avoid depending on the need for a service to be online to cache the message.
<errl>
I would like for the communication to strictly happen within the protocol itself, and be contained.
<victorbjelkholm>
errl, well, you have to publish to IPNS at least once per 24h I think, for the peers to keep the record, but there won't be a single point of failure, as long as the IPNS is updated daily by someone
<errl>
victorbjelkholm: Would the content pushed to the IPNS be mutable? Or would the address my content is stored at within IPNS be unique?
Paul[m] has joined #ipfs
ylp2 has quit [Ping timeout: 264 seconds]
<victorbjelkholm>
errl, IPNS works by mapping your peerid -> ipfs hash, so you don't push content to IPNS, you just put a pointer to a hash
Guest4055 has quit [Read error: Connection reset by peer]
<achin>
NAT STRONK
<haad>
Kubuxu: yeah :/
<Kubuxu>
do you want access to one of storage nodes and do the same I do?
<dignifiedquire>
Kubuxu: you still need me for access?
<haad>
Kubuxu: no, that's fine
<Kubuxu>
dignifiedquire: yeah, could you add go-libp2p-transport to go group with write accesss
Aelius has quit [Quit: quit]
<dignifiedquire>
ipfs or libp2p org?
<dignifiedquire>
nvm found it
<dignifiedquire>
done
<Kubuxu>
ty
<Kubuxu>
My go-reuseport epoll code works.
rendar has quit [Ping timeout: 248 seconds]
<Kubuxu>
ipfs used to run 100+ system threads after rapid dial (ipfs add early in the life of a damon).
<Kubuxu>
now it is at 11 system threads
<dignifiedquire>
nice
<Kubuxu>
I am so so fracking happy you won't believe it.
<dignifiedquire>
I believe you :)
Boomerang has joined #ipfs
<dignifiedquire>
so, I was told that non compiled languages are quicker to work with..but npm installs these days take longer than downloading compiling gcc :(
<Kubuxu>
Hahah
<Kubuxu>
and there goes the "short dev time"
<Kubuxu>
"short build time"
<dignifiedquire>
I was working with rust yesterday, and downloading + compiling deps would be quicker than an npm install
<Kubuxu>
funny thing is: with well written C projects with good makefiles/build systems compile times aren't bad
<Kubuxu>
C++ times are bad but it is C++
<Kubuxu>
Go is just awesome
<Kubuxu>
It is possible that incremental compile and run in Go might be faster than loading big project into node process.
ppham has joined #ipfs
quotidien_afk has quit [Remote host closed the connection]
gozala_ has quit [Remote host closed the connection]
kitcambridge has quit [Remote host closed the connection]
<Kubuxu>
`cdtemp` is my new favorite alias
quotidien_afk has joined #ipfs
anonymuse has joined #ipfs
<achin>
does that make a new temp directory, and cd you into it?
<victorbjelkholm>
dignifiedquire, so, when are we gonna start using something like gx-js so we can slowly get away from using npm?
<dignifiedquire>
victorbjelkholm: I'd rather use npm on ipfs tbh
gozala_ has joined #ipfs
<dignifiedquire>
so that I can keep the exact same ui, and don't have to around and republish things
<victorbjelkholm>
when we move to (npm on ipfs || gx-js) ?
<victorbjelkholm>
yeah, for me that's fine as well
<dignifiedquire>
npm on ipfs is blocked by unixfs sharding in go-ipfs, not sure what the current state of that is
<dignifiedquire>
Kubuxu: do you know?
<Kubuxu>
I think why has something that is working but it needs standardization and deployment.
<victorbjelkholm>
dignifiedquire, why is that blocking npm on ipfs?
<dignifiedquire>
because the npm registry is so big adding it as it is currenlty to ipfs fails
<victorbjelkholm>
dignifiedquire, ah, but what if you don't add the entire registry at once? :p
<victorbjelkholm>
like, it's not a very efficient solution
<dignifiedquire>
daviddias: experiemented a lot with that earlier this year, and this was where he got stuck as far as I know
<Kubuxu>
the adding is only about 48-72h, and it is run and forget
<Kubuxu>
he did manual sharding IIRC
<Kubuxu>
manual aka custom
<Kubuxu>
on the application level
kitcambridge has joined #ipfs
<victorbjelkholm>
dignifiedquire, yeah, if you go about the registry-mirror way of adding the entire registry at once, true. I'm thinking there are better ways to start using it like today
<dignifiedquire>
not sure, the last time I tried to run it failed pretty hard for me, like made ipfs instance panic on gc and such things
<achin>
for the wikipedia stuff i'm working on, i had to do manual sharding as well. this is a hard issue
<dignifiedquire>
victorbjelkholm: how?
<dignifiedquire>
achin: well the code is there, so maybe we can get it in soonish
<Kubuxu>
achin: solution in comming
<achin>
i'm happy to test any ipfs code branches
<victorbjelkholm>
dignifiedquire, many ways, one is to have some cli that just downloads packages from versions field in package.json, kind of what I did with everythingstays.com, another is to mount /ipfs and use that. Both are easy but mounting hangs sometimes
<victorbjelkholm>
and then in the preinstall, run shell script that checks for ipfs and if so, downloads dependencies and npm will ignore them
<achin>
Kubuxu: thanks. i'll try to fire it up in a new development ipfs node this weekend
<Kubuxu>
but it is just a test
<Kubuxu>
the object format won't be readable by non-sharding enabled nodes
<Kubuxu>
and the format might be still a subject to change.
<achin>
i think that's ok for an initial test
ilmu has joined #ipfs
<Kubuxu>
victorbjelkholm, dignifiedquire: other way could be registry cache proxy
<victorbjelkholm>
Kubuxu, yeah, feels like you won't have enough control then, I would like it to be closer to how you do it with gx, adding packages and then referencing them
<victorbjelkholm>
instead of automagic adding all the packages
<Kubuxu>
this is the second stage
<Kubuxu>
the nmp-on-ipfs stage is about fixing the current issue, where on arcticjs someone has localy every package you could want but accessing it is impossible
rendar has joined #ipfs
pixelated has joined #ipfs
PrinceOfPeeves has joined #ipfs
pixelated is now known as Guest46360
ppham has quit [Remote host closed the connection]
<victorbjelkholm>
Kubuxu, yeah, I'm still not confident that's the right way, I think a lot of packages on npm are not needed, day-to-day
<victorbjelkholm>
as long as you get the top 5%, the community can help out with the rest
<Kubuxu>
yes but they can sit on one of our storage nodes, there is no hard to that
cryptix has joined #ipfs
cryptix has quit [Client Quit]
<Kubuxu>
the main problem is that due to have npm registry works you have to create global index of packages
cryptix has joined #ipfs
<Kubuxu>
for that you need all packages
rugu has joined #ipfs
<Kubuxu>
or have authors start publishing ipfs hashes of their packages and store that metadata somewhere
<Kubuxu>
which is just a index we are creating
<Kubuxu>
hmm, but right we could have X% in the index
<Kubuxu>
👍
<rugu>
hey any idea why https://github.com/ipfs/js-libp2p-ipfs now runs only on pull-streams and not the conventional push-streams. In the example process.stdin.pipe(conn).pipe(process.stdout) no longer works as it did previously.
<Kubuxu>
iirc you have to convert between node streams and pull streams
Encrypt has quit [Quit: Quitte]
<rugu>
previously this was not the case. The example as of a month ago used process.stdin.pipe(conn) but now uses pull( pull.values(['hey']), conn, pull.through(console.log), pull.collect((err, data) => { if (err) { throw err } console.log('received echo:', data.toString()) }) )
<rugu>
damn, sorry for the bad editing.
<rugu>
s
<rugu>
s
<Kubuxu>
yeah that is the case
<Kubuxu>
dignifiedquire: ^^
<rugu>
cause i have never worked with pull streams before, seems more complex imo
<Kubuxu>
it is really quite simple (saying as non)
<dignifiedquire>
rugu: you can find the history here: https://github.com/ipfs/js-ipfs/issues/362 , the short version is node streams have many issues that we had problems with
<Kubuxu>
dignifiedquire: it might be good to have short "this project uses pull-streams" in the readmes
<rugu>
dignifiedquire : Thanks a lot. Will go through the entire thing and make changes appropriately. Appreciate the help and dev work you guys put in.
ppham has quit [Remote host closed the connection]
<wak-work>
the vendor deps have that version listed but should have QmSF8fPo3jgVBAy8fpdjjYqgG87dkJgUprRBHRd2tmfgpP to be consistent with all of the .go files
popadomalos has quit [Client Quit]
dyce has quit [Quit: WeeChat 1.0.1]
Encrypt has quit [Quit: Sleeping time!]
<lgierth>
mh you're right, good catch
<lgierth>
i wonder why this wasn't caught in ci
<lgierth>
i'm trying make build with a clean $GOPATH/src/gx
<geoah>
it needs to change in go-notifier, go-libp2p, and go-ipfs
<lgierth>
aah the "new" goprocess is probably a dependency of some package, so that's why it gets fetched
<lgierth>
geoah: just update package.json and it's fine
<lgierth>
thanks for all the extraction work btw!
<geoah>
go-notifier and go-libp2p still use the "old" 0.0.0, so these need to change as well I think
<geoah>
at least this is what gx deps --tree --highlight=goprocess says
<geoah>
I'll check it out first thing in the morning and make the relevant PRs :) sorry about that
<lgierth>
great thanks so much!
<lgierth>
and no worries at all - gx is pretty good but we need it to get even better, e.g. it could check whether all gx/ipfs/ imports have an their entry in package.json
<noffle>
it'd be great if we could collect the biggest gx pain points and put together a milestone roadmap with them, to direct interested folks toward the biggest issues
zat0 has quit [Remote host closed the connection]
<wak-work>
the biggest problem i have is that not everything seems to be vendored by gx path
<lgierth>
biggest pain: gx not everywhere yet, lol
<wak-work>
yeah i guess
<lgierth>
but word
<wak-work>
the way i build go packages i actually rewrite imports automatically using the gx package.json information
<wak-work>
but unfortunately i didn't account for things like having two goprocess's
<lgierth>
and some kind of "here's how i maintain a go project using gx" story
captain_morgan has quit [Ping timeout: 250 seconds]
<lgierth>
wak-work: yeah package.json and imports definitely should be consistent, so it's good to have tooling for it
<wak-work>
my problem right now is this "go/src/github.com/ipfs/go-ipfs/namesys/republisher/repub.go:82: cannot use p (type "gx/ipfs/QmSF8fPo3jgVBAy8fpdjjYqgG87dkJgUprRBHRd2tmfgpP/goprocess".Process) as type "gx/ipfs/QmQopLATEYMNg7dVqZRNDfeE2S1yKy8zrRh5xnYiuqeZBn/goprocess".Process in argument to goprocessctx.OnClosingContext"
<wak-work>
basically
keks has quit [Ping timeout: 276 seconds]
<lgierth>
oh the old one is actually still used, not just in package.json?
wuch has quit [Quit: WeeChat 1.5]
<wak-work>
it's actually my own fault i suppose
<wak-work>
im actually trying to figure out what GX does
<lgierth>
whyrusleeping: (unrelated to the above) -- can you imagine anything in IPNS that would yield a bump in resolution time exactly every minute?
rendar has quit [Quit: std::lower_bound + std::less_equal *works* with a vector without duplicates!]
ola has quit [Quit: QUIT]
xelra has quit [Ping timeout: 265 seconds]
galois_dmz has quit []
captain_morgan has joined #ipfs
<lgierth>
whyrusleeping: check the new pages dashboard when you get a chance -- the difference between ipld.io and multiformats.io resolution time is adorable, and my gut instinct is that it's because of the difference in XOR distance from the individual gateways
<lgierth>
and the graph for ipfs.io shows the ideal RTT (it's a non-ipns dnslink)
thufir has joined #ipfs
<lgierth>
wak-work: it just fetches stuff, and rewrites imports
<wak-work>
lgierth: yeah i just realized im doing something wrong with my import rewriter
<wak-work>
:q
<lgierth>
gx can rewrite imports for you
<lgierth>
eeh gx-go can
<wak-work>
is there a command that just rewrites imports?
<lgierth>
:)
<lgierth>
in both ways even
<lgierth>
check gx-go rewrite
<wak-work>
ill look at that
<lgierth>
then i guess rewriting needs to be more prominent
<wak-work>
playing with gx-go now
zat0 has joined #ipfs
structuralist has quit [Remote host closed the connection]
structuralist has joined #ipfs
structuralist has quit [Ping timeout: 265 seconds]
appa-nerd has joined #ipfs
matoro has quit [Ping timeout: 248 seconds]
<wak-work>
lgierth: how come gx-go --rewrite doesn't rewrite references to the package itself?
<wak-work>
am i missing something
<lgierth>
doesn't know its hash
<lgierth>
you can't have circular dependencies
<wak-work>
oh.....
<wak-work>
that's fair
<lgierth>
:)
<wak-work>
how am i supposed to handle that/
<wak-work>
by having links to the import paths?
<lgierth>
what are you trying to achieve_
<lgierth>
?
<lgierth>
(i gotta run in a few as i'm running out of battery)
<wak-work>
i just want to make sure these references are fixed
<wak-work>
basically i just want to self reference for the build
<wak-work>
i know i could rewrite the files with sed
<lgierth>
i see -- gx-go should be rewriting these when it fetches leveldb
<wak-work>
yeah it doesn't
<wak-work>
and gx-go rewrite doesn't fix it either
<wak-work>
hmm
<lgierth>
mh that is weird
bsm117532 has quit [Quit: Leaving.]
bsm117532 has joined #ipfs
<lgierth>
have you tried removing the packages from src/gx/ipfs/? gx-go sometimes had trouble recovering packages that hadn't been properly dealt with in a previous run
<lgierth>
but that should have been fixed
<wak-work>
it's a clean checkout every time
xelra has joined #ipfs
<wak-work>
it's actually untarring a clean set of files from gx install
<lgierth>
ok then i'm out of ideas for the moment :(
<wak-work>
yeah ill figure it out
jbenet-on-plane has joined #ipfs
<lgierth>
feel free to open an issue in gx-go
<lgierth>
whyrusleeping will be traveling with limited internet access for the next 5 or so days so he might miss everything we're talking here
<lgierth>
jbenet-on-plane: i have dns comments -- typing them up
<jbenet-on-plane>
lgierth: hello o/ -- yeah i think i have 6 more hrs to go
<jbenet-on-plane>
well, morning somewhere :)
<lgierth>
*the* morning
<jbenet-on-plane>
it will be 2p in China i think
<lgierth>
morning is an eternal thing i guess
<jbenet-on-plane>
OH yes-- well, the sun is to my left
<lgierth>
:)
<jbenet-on-plane>
it's pretty cool-- huge shining orb in the sky in the same spot for the last 5 hours.
structuralist has joined #ipfs
<jbenet-on-plane>
i'm glad we have one.
<lgierth>
yeah! although sometimes i'm disappointed we only have one
<lgierth>
same for the moon
<lgierth>
rings would be kinda nice too
<jbenet-on-plane>
yeahhhh some of those double sun sunsets/rises would look amazing
<jbenet-on-plane>
ringssss sigh yeah, i want to look up from saturn
<lgierth>
who knows, we might get a webcam up there
<jbenet-on-plane>
i think getting a probe into saturn just for pics would be well worth it
<lgierth>
haha yeah
<jbenet-on-plane>
a few years ago i had an idea that could fund itself: kickstart a project to ship a bunch of RC cars with telemetry to the moon. rent play time to pay for sending those and more
<lgierth>
:>
<jbenet-on-plane>
it would have like a 2s lag
<jbenet-on-plane>
at best
<lgierth>
the UX could be predictive based on surface data though
<jbenet-on-plane>
totally doable. would be so much fun to explore the moon
<lgierth>
like mosh for ssh
<jbenet-on-plane>
ah yeah!
<lgierth>
twitch plays rc-cars-on-the-moon, anyone?
<jbenet-on-plane>
ahahahhahaha
wavis has quit [Quit: Connection closed for inactivity]