<whyrusleeping>
AphelionZ: probably don't want to use pubsub for just user to user comms
_whitelogger has joined #ipfs
Adbray has quit [Read error: Connection reset by peer]
}ls{ has joined #ipfs
elkalamar has quit [Ping timeout: 252 seconds]
rjharmon has quit [Quit: Connection closed for inactivity]
elkalamar has joined #ipfs
koalalorenzo[m] has quit [Ping timeout: 264 seconds]
codebam__ has quit [Ping timeout: 252 seconds]
Kolonka[m] has quit [Ping timeout: 240 seconds]
turt2live has quit [Ping timeout: 252 seconds]
burdakovd[m] has quit [Ping timeout: 260 seconds]
althaser[m] has quit [Ping timeout: 260 seconds]
swedneck[m]2 has quit [Ping timeout: 252 seconds]
jcgruenhage has quit [Ping timeout: 260 seconds]
peddie[m]1 has quit [Ping timeout: 240 seconds]
lordpipe has quit [Ping timeout: 255 seconds]
mindhog[m] has quit [Ping timeout: 255 seconds]
dryajov[m]1 has quit [Ping timeout: 260 seconds]
swedneck[m] has quit [Ping timeout: 260 seconds]
k0mpjut0r has quit [Ping timeout: 276 seconds]
kallisteiros has joined #ipfs
mindhog[m] has joined #ipfs
lordpipe has joined #ipfs
not-a-dog has joined #ipfs
k0mpjut0r has joined #ipfs
peddie[m]1 has joined #ipfs
mkg20001 has joined #ipfs
jcgruenhage has joined #ipfs
dcallagh has joined #ipfs
not-a-dog has quit [Ping timeout: 276 seconds]
turt2live has joined #ipfs
kallisteiros has quit [Ping timeout: 272 seconds]
colatkinson has joined #ipfs
kallisteiros has joined #ipfs
fridim has quit [Ping timeout: 264 seconds]
colatkinson has quit [Quit: colatkinson]
gozala has quit [Ping timeout: 264 seconds]
fridim has joined #ipfs
kallisteiros has quit [Ping timeout: 272 seconds]
kallisteiros has joined #ipfs
gozala has joined #ipfs
konubinix has joined #ipfs
kallisteiros has quit [Ping timeout: 252 seconds]
kallisteiros has joined #ipfs
Alpha64 has quit [Read error: Connection reset by peer]
Rboreal_Frippery has quit [Ping timeout: 252 seconds]
kallisteiros has quit [Ping timeout: 244 seconds]
Novitsh has joined #ipfs
bomb-on has quit [Quit: SO LONG, SUCKERS!]
ylp has joined #ipfs
crcastle[m] has joined #ipfs
ZaZ has joined #ipfs
rendar has joined #ipfs
fazo has joined #ipfs
the-erm has joined #ipfs
not-a-dog has joined #ipfs
not-a-dog has quit [Ping timeout: 276 seconds]
Mottengrotte has joined #ipfs
ilyaigpetrov has joined #ipfs
reit has quit [Read error: Connection reset by peer]
ZaZ1 has joined #ipfs
ZaZ has quit [Ping timeout: 264 seconds]
ZaZ1 has quit [Quit: Leaving]
trqx has quit [Remote host closed the connection]
trqx has joined #ipfs
tombusby has quit [Remote host closed the connection]
tombusby has joined #ipfs
not-a-dog has joined #ipfs
not-a-dog has quit [Ping timeout: 276 seconds]
not-a-dog has joined #ipfs
not-a-dog has quit [Ping timeout: 252 seconds]
trqx has quit [Remote host closed the connection]
trqx has joined #ipfs
fazo has quit [Quit: fazo]
fazo has joined #ipfs
Mateon3 has joined #ipfs
Mateon1 has quit [Ping timeout: 264 seconds]
Mateon3 is now known as Mateon1
plexigras has joined #ipfs
bomb-on has joined #ipfs
goiko has joined #ipfs
reit has joined #ipfs
paul_uk has joined #ipfs
<paul_uk>
hey all. I'm about to develop my own job queue, something like sidekiq. The biggest problem that I have. Is that I want a multi-master setup for the controller nodes. Could ipfs help me setup some sort of auto-discovery and dns setup where one director comes up, then it can broadcast to workers that it can co-ordinate jobs and then other directors can come online to provide ha/resilience and load-balancing for the
<paul_uk>
workers? thanks
<paul_uk>
*not director, controller.
mauz555 has joined #ipfs
xelra has joined #ipfs
fazo has quit [Quit: fazo]
fazo has joined #ipfs
mauz555 has quit [Ping timeout: 252 seconds]
xelra has quit [Remote host closed the connection]
xelra has joined #ipfs
kallisteiros has joined #ipfs
aispark has joined #ipfs
aispark has left #ipfs [#ipfs]
malaclyps has quit [Read error: Connection reset by peer]
malaclyps has joined #ipfs
rtjure has quit [Ping timeout: 276 seconds]
reit has quit [Quit: Leaving]
tombusby has quit [Ping timeout: 250 seconds]
mauz555 has joined #ipfs
rtjure has joined #ipfs
<fiatjaf>
paul_uk, no
<fiatjaf>
paul_uk, perhaps using libp2p directly, but I can't figure if that's a good idea or not
tombusby has joined #ipfs
tombusby_ has joined #ipfs
tombusby has quit [Remote host closed the connection]
<fiatjaf>
I don't actually understand your use case completely and I don't know what is sidekiq, but I would guess yes.
Adbray has joined #ipfs
<fiatjaf>
maybe you could wait someone better to answer you, or ask on #libp2p
dimitarvp has joined #ipfs
<paul_uk>
hehe. im not overly sure myself. many job queues have a pain point where the database is not shared. whereas in my circumstance im using memsql which can scale indefinitely. so that part is covered. what im really missing, is having a distributed set of controllers which can keep track of jobs. know when a worker is alive or not, etc.
<paul_uk>
im going to have a play around in a bit and try and work out the best plan of action.
<paul_uk>
fiatjaf: thanks :)
mauz555 has quit [Ping timeout: 252 seconds]
Mottengrotte has quit [Ping timeout: 276 seconds]
mauz555 has joined #ipfs
Mottengrotte has joined #ipfs
Steverman has joined #ipfs
<fiatjaf>
paul_uk, why not sharing a redis instance?
<paul_uk>
fiatjaf: memsql is an in-memory database like redis and is cluster-aware by default.
<paul_uk>
my current architecture can do a multiple billion rows in the database and billions of impressions a month. with something like redis, it's not as simple.
<fiatjaf>
oh, didn't know that, just imagined redis would be somewhat capable
<fiatjaf>
if you're sharing a database, why can't you use it as a single source of value for what nodes are online and what nodes are doing what?
tg has quit [Ping timeout: 265 seconds]
<paul_uk>
fiatjaf: yes thats exactly what im going to do. but i need some sort of controller to tell the database whether workers go offline or not. lets say a worker gets 10 jobs, starts pocessing them and then dies. in the db, it would be set as running and never completed. so i need a controller to heartbeat the worker. but i just dont want one director. i want quite a few to make it resilient.
<paul_uk>
sorry director == controller. i read the segment blog on building a job queue and they use director as the term for a worker.
<paul_uk>
but regardless it's all good. in my spare time im building out a VOD platform and IPFS is something i've wanted to start working on. so there is some synergy here anyway.
mauz555 has quit [Remote host closed the connection]
mauz555 has joined #ipfs
tg has joined #ipfs
mindCrime has joined #ipfs
rtjure has quit [Ping timeout: 272 seconds]
<AphelionZ>
whyrusleeping: ok. is there any recommended way? libp2p? or is that still just mainly for dht/distributed/broadcast?
mauz555 has quit [Ping timeout: 252 seconds]
rtjure has joined #ipfs
<fiatjaf>
libp2p is for anything
<fiatjaf>
you can use it to dial to specific nodes and ping them
<AphelionZ>
I'm trying to do peer to peer comms
<fiatjaf>
comms?
<fiatjaf>
commits?
<fiatjaf>
communities?
<fiatjaf>
communists?
<fiatjaf>
commputers?
<r0kk3rz>
communications :P
<paul_uk>
hehe
Mottengrotte_ has joined #ipfs
Mottengrotte has quit [Read error: Connection reset by peer]
<r0kk3rz>
paul_uk: is this something aimed at being public? or mostly private?
Steverman has quit [Quit: WeeChat 2.2]
<AphelionZ>
communications
the-erm has quit [Ping timeout: 252 seconds]
<AphelionZ>
communists communication
Steverman has joined #ipfs
<paul_uk>
r0kk3rz: initially private. in the future who knows. there is sandglass, faktory and machinery right now. but for my purposes, don't fulfill my needs.
<paul_uk>
it's private due to the memsql requirement. i want to focus on my needs now, rather than doing X amount of extra work.
<r0kk3rz>
in terms of message passing to keep everyone in sync, there are probably better solutions available for that
<r0kk3rz>
but if you want to shovel around a lot of data, and that data doesnt change too much, then ipfs is good for that
Mottengrotte_ has quit [Quit: Mottengrotte_]
m0lok has joined #ipfs
kallisteiros has quit [Ping timeout: 268 seconds]
swedneck[m] has joined #ipfs
mindCrime has quit [Ping timeout: 244 seconds]
tombusby_ has quit [Ping timeout: 250 seconds]
not-a-dog has joined #ipfs
mindCrime has joined #ipfs
not-a-dog has quit [Ping timeout: 252 seconds]
tombusby has joined #ipfs
yuitimothy has quit [Ping timeout: 256 seconds]
<ilyaigpetrov>
Hi. I want to understand how pubsub works, how peer discovers new messages given he knows the topic?
mindCrime has quit [Ping timeout: 272 seconds]
kallisteiros has joined #ipfs
Xesxen has quit [Remote host closed the connection]
Xesxen has joined #ipfs
HeavyMetal has quit [Ping timeout: 264 seconds]
<ilyaigpetrov>
DHT, I guess, is the answer.
HeavyMetal has joined #ipfs
mindCrime has joined #ipfs
Fessus has quit [Ping timeout: 244 seconds]
Xesxen has quit [Remote host closed the connection]
kants has joined #ipfs
cris has joined #ipfs
<AphelionZ>
what are some use cases for pubsub, if not peer to peer communication?
Xesxen has joined #ipfs
reit has joined #ipfs
Alpha64 has joined #ipfs
<ilyaigpetrov>
AphelionZ: multicasts? E.g. casting event stream to different regions, several controlled writers, many subscribers.
<AphelionZ>
mm gotcha
<swedneck[m]>
yo someone test gateway.swedneck.xyz again
<swedneck[m]>
i figured out that i shouldn't set the listen ip to 0.0.0.0, instead leave it as 127.0.0.1
<swedneck[m]>
now to figure out how to make the nginx reverse proxy behave the same for riot.swedneck.xyz as it does for api.swedneck.xyz and gateway.swedneck.xyz
<swedneck[m]>
actually it seems i don't need the api reverse proxy?
ylp has quit [Quit: Leaving.]
roygbiv has joined #ipfs
NeoTeo has joined #ipfs
<swedneck[m]>
hmm, i can't seem to make riot.swedneck.xyz point to gateway.swedneck.xyz/ipns/riot.swedneck.xyz
graffen has quit [Ping timeout: 265 seconds]
graffen has joined #ipfs
tombusby has quit [Ping timeout: 250 seconds]
tombusby has joined #ipfs
kaminishi has joined #ipfs
Fessus has joined #ipfs
kn0rki has joined #ipfs
not-a-dog has joined #ipfs
not-a-do_ has joined #ipfs
not-a-dog has quit [Ping timeout: 252 seconds]
roygbiv has quit [Quit: ™]
plexigras has quit [Quit: WeeChat 2.2]
plexigras has joined #ipfs
bomb-on has quit [Quit: SO LONG, SUCKERS!]
tenchooo has quit [Remote host closed the connection]
tenchooo has joined #ipfs
Jesin has quit [Quit: Leaving]
<swedneck[m]>
i have a cname on the riot subdomain that points to the gateway subdomain, and a _dnslink.riot record with the ipfs path
Jesin has joined #ipfs
kallisteiros has quit [Ping timeout: 272 seconds]
kallisteiros has joined #ipfs
<lidel>
swedneck[m], you have redindant CNAME and A for riot.swedneck.xyz
<jamiew>
lidel: loving all the firefox dweb work, really exciting seeing your demos at all-hands over the last few months
not-a-dog has quit [Remote host closed the connection]
not-a-dog has joined #ipfs
* lidel
\m/
not-a-dog has quit [Remote host closed the connection]
not-a-dog has joined #ipfs
not-a-dog has quit [Remote host closed the connection]
mauz555 has quit [Ping timeout: 252 seconds]
not-a-dog has joined #ipfs
<lidel>
slowly getting there: if we have DNSLink enabled by default and if protocol handler from https://github.com/ipfs-shipyard/ipfs-companion/pull/533 eventually lands, we could do seamless update from http:// to ipfs:// :-^-)
<swedneck[m]>
born just in time to see IPFS made seamless (-ish)
<fiatjaf>
lidel, I've read somewhere that you want to create an ecosystem of window.ipfs apps
<fiatjaf>
why don't you make it possible for these apps to generate and use ipfs.key
<fiatjaf>
so these keys may serve as an authentication method
<fiatjaf>
you visit a window.ipfs-enabled app and start using it immediately, their server may store your preferences and whatever for when you're coming back, and you don't need to create accounts
kallisteiros has quit [Ping timeout: 268 seconds]
<lidel>
fiatjaf, ipfs.key is blocked in window.ipfs as a quick fix (we dont want wensites to publish using your self key), the end goal is to have it sandboxed per app like we do with ipfs.files
<lidel>
*websites
<fiatjaf>
good to hear
<fiatjaf>
but js-ipfs has a problem with ipfs.key too. they require you to initialize js-ipfs with a keyring and I don't know what else