infinity0 has quit [Remote host closed the connection]
infinity0 has joined #ipfs
infinity0 has quit [Remote host closed the connection]
infinity0 has joined #ipfs
infinity0 has quit [Remote host closed the connection]
infinity0 has joined #ipfs
leeola has quit [Quit: Connection closed for inactivity]
infinity0 has quit [Remote host closed the connection]
kerem has quit [Ping timeout: 260 seconds]
azdle has joined #ipfs
dansup is now known as stabb
stabb is now known as dansup
screensaver has quit [Ping timeout: 260 seconds]
Guest45287 has joined #ipfs
ShalokShalom has joined #ipfs
ShalokShalom__ has quit [Ping timeout: 255 seconds]
gmoro has quit [Remote host closed the connection]
gmoro has joined #ipfs
shizy has joined #ipfs
shoragan has quit [Ping timeout: 240 seconds]
shoragan has joined #ipfs
shoragan has joined #ipfs
shoragan has quit [Changing host]
Aranjedeath has quit [Ping timeout: 260 seconds]
sprint-helper has quit [Remote host closed the connection]
sprint-helper1 has joined #ipfs
sirdancealot has quit [Ping timeout: 264 seconds]
gmoro has quit [Remote host closed the connection]
gmoro_ has joined #ipfs
ZarkBit_ has joined #ipfs
<kevina>
whyrusleeping: around, are you happy with #3743, if so I will rebase to fix the conflicts
ZarkBit has quit [Read error: Connection reset by peer]
discopatrick has quit [Ping timeout: 260 seconds]
misuto has quit [Quit: No Ping reply in 180 seconds.]
ShalokShalom has quit [Quit: No Ping reply in 180 seconds.]
xnbya has quit [Quit: No Ping reply in 180 seconds.]
PorcoRosso70 has quit [Ping timeout: 260 seconds]
ajvpot has quit [Ping timeout: 260 seconds]
ZarkBit_ is now known as ZarkBit
silotis has quit [Quit: No Ping reply in 180 seconds.]
misuto has joined #ipfs
luizirber has quit [Ping timeout: 260 seconds]
xnbya has joined #ipfs
ShalokShalom has joined #ipfs
Ekho has quit [Quit: An alternate universe was just created where I didn't leave. But here, I left you. I'm sorry.]
discopatrick has joined #ipfs
silotis has joined #ipfs
e337 has quit [Ping timeout: 260 seconds]
ajvpot has joined #ipfs
luizirber has joined #ipfs
e337 has joined #ipfs
shizy has quit [Ping timeout: 240 seconds]
aeternity has quit [Ping timeout: 260 seconds]
dimitarvp has quit [Quit: Bye]
aeternity has joined #ipfs
Ekho has joined #ipfs
neurrowcat has quit [Quit: Deebidappidoodah!]
shizy has joined #ipfs
Ekho- has joined #ipfs
Ekho has quit [Disconnected by services]
Ekho- is now known as Ekho
m_anish has joined #ipfs
luizirber_ has joined #ipfs
luizirber has quit [Ping timeout: 260 seconds]
m_anish__ has quit [Ping timeout: 260 seconds]
Confiks has quit [Ping timeout: 260 seconds]
maxlath has quit [Quit: maxlath]
Confiks has joined #ipfs
bertschn_ has joined #ipfs
bertschneider has quit [Ping timeout: 240 seconds]
arkimedes has joined #ipfs
fjl__ has quit [Read error: Connection reset by peer]
fjl__ has joined #ipfs
special has quit [Ping timeout: 240 seconds]
ednilmoob has quit [Ping timeout: 240 seconds]
caveat has quit [Ping timeout: 246 seconds]
traverseda has quit [Ping timeout: 252 seconds]
caveat has joined #ipfs
ckwaldon has quit [Ping timeout: 260 seconds]
ednilmoob has joined #ipfs
realisation has joined #ipfs
ckwaldon has joined #ipfs
special has joined #ipfs
special is now known as Guest74517
traverseda has joined #ipfs
aeternity has quit [Ping timeout: 260 seconds]
aeternity has joined #ipfs
shizy has quit [Ping timeout: 260 seconds]
<whyrusleeping>
kevina: i'll take a look now
Aranjedeath has joined #ipfs
caiogondim has joined #ipfs
mirizliv has joined #ipfs
<mirizliv>
hey, guys, anyone around?
<mirizliv>
i'm building a small cluster from pi zero w units.. i started looking into ways for them to share a file system.. i haven't looked at ipfs in a while but was wondering if its possible to do?
Good_Sir has joined #ipfs
<mirizliv>
i know glusterfs and ceph can do this but figured i'd check for something a little more lightweight first
Akaibu has quit [Quit: Connection closed for inactivity]
<whyrusleeping>
mirizliv: hrm... ipfs doesnt really provide something gluster-like
<whyrusleeping>
You could run ipfs-cluster on them
<whyrusleeping>
but ipfs cluster is still very early, i wouldnt yet recommend it for any sort of 'real' use
<whyrusleeping>
but if youre just messing around with things it might be fun to give it a try
<mirizliv>
ah, i understand jeromy. figured it was worth asking anyway
<mirizliv>
yea, i might do that - i'll give it a look! thanks
<whyrusleeping>
cool cool, let us know any issues you run into.
<mirizliv>
will do, thanks again
<whyrusleeping>
feel free to open as many issues as you want over at github.com/ipfs/ipfs-cluster
MrControll has quit [Ping timeout: 260 seconds]
<whyrusleeping>
no problem :)
asyncsec has joined #ipfs
<achin>
i just built ipfs-cluster from source. should i join someones existing cluster? or should i just play around with it a bit?
ckwaldon has quit [Ping timeout: 260 seconds]
<whyrusleeping>
achin: i'd play around with it a bit for now, we don't have any existing clusters set up
nannal has quit [Ping timeout: 268 seconds]
<achin>
l
<achin>
k
nannal has joined #ipfs
mirizliv has quit [Quit: Page closed]
ilackarms has joined #ipfs
robattila256 has quit [Quit: WeeChat 1.7.1]
<achin>
neat use of Raft
robattila256 has joined #ipfs
<whyrusleeping>
:)
<whyrusleeping>
Its swappable too
<achin>
for other consensus algorithms?
<whyrusleeping>
yeap
<achin>
cool
<whyrusleeping>
if you want to write a paxos module, you could use that too
<whyrusleeping>
could even back it with an ethereum smart contract
<achin>
sadly, i don't have 1 ipfs node right now. going to need to find some other machines to hijack
<Quiark_>
mmm.. which parts require consensus?
<whyrusleeping>
Quiark_: coordinating the set of things to pin between peers
tilgovi has joined #ipfs
<Quiark_>
so the project's goal is to guarantee that each CID will be pinned on 'replication_factor' nodes, eventually, even in the face of failure
<whyrusleeping>
yeap
<achin>
which is really important for, well, important datasets. but also datasets that are too large for just 1 node to pin
<whyrusleeping>
yeah
<Quiark_>
but it doesn't do splitting, only replicating, no?
<whyrusleeping>
currently only full replication across the cluster
<Quiark_>
or maybe it's for load-balancing the reads, right?
<whyrusleeping>
Once we get the code a little more mature we will be adding in different RAID-like replication strategies
<crankylinuxuser>
technically, if you can reconstruct the subhashes of files, you can put them manually in a larger hash - that would allow replication where no node carries a full set
<crankylinuxuser>
I was able to do that with .3.1 a year ago.
caiogondim has quit [Quit: caiogondim]
Guest217328[m] has joined #ipfs
Good_Sir has quit [Quit: Leaving]
gmcabrita has quit [Quit: Connection closed for inactivity]
<achin>
there's no need for that kind of reconstruction
<crankylinuxuser>
AH. I was looking at moving a 1tb database back and forth at work. I wanted to experiment in using IPFS in seeing if multiple clients, with each some files pinned, could be linked to a ipfs hash that combined them all. They can.
<crankylinuxuser>
This can be used in things like OpenStreet maps, where things are logically laid out in zones. Each zone can be a hash. Each country is made up of lots of zones. Then continents and oceans, then world.
Aranjedeath has quit [Quit: Three sheets to the wind]
stevenaleach has joined #ipfs
ilackarms has quit [Ping timeout: 260 seconds]
realisation has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<lgierth>
the largest dataset i've transferred is 4 TB, with 0.4.7 or so
<traverseda>
Here's a question. Could we do auto-mirroring of ssl-enabled content? Basically using an ssl-certificate as the ipns domain? Of course the servers themselves wouldn't be providing an ipfs service, but someone else could, and we could verify it via ssl. Basically an auto-mirror plugin that uses ssl to verify that the content is up to date and the like
stevenaleach has joined #ipfs
stevenaleach has quit [Ping timeout: 268 seconds]
<kythyria[m]>
SSL doesn't require publishing hashes you could use to verify.
traverseda has quit [Ping timeout: 240 seconds]
stevenaleach has joined #ipfs
JayCarpenter has quit [Quit: Page closed]
stevenaleach has quit [Ping timeout: 240 seconds]
<Scio[m]>
Can anyone tell me if there's better way to reconnect a js-ipfs node after a network interruption other than `stop()`ing and `start()`ing it?
traverseda has joined #ipfs
ShalokShalom has quit [Read error: Connection reset by peer]
ShalokShalom has joined #ipfs
rendar has joined #ipfs
<walle303[m]>
scio: I just tell it to connect to a few peers again if the peer count drops too low
<Scio[m]>
walle303: How do you do that?
<walle303[m]>
scio: on mobile at the moment so I can't look at the code, but it should be something along the lines of node.connect, node.peer.add, or node.peer.connect
<walle303[m]>
scio: Should be in the documentation
<Scio[m]>
walle303: Okay, I'll go look at the doc. Thanks for the help!
Atrix has joined #ipfs
<walle303[m]>
scio: it's node.swarm.connect, and using node.swarm.peers you can get the list of peers (node being whatever you named your node)
<walle303[m]>
I forgot I could just ssh into my machine to check
<Scio[m]>
walle303: Getting the peer list also makes the swarm reconnect? Please don't bother sshing just for me, i was just about to test connecting to the signalling server myself!
<walle303[m]>
scio: peer list doesn't reconnect, just lets you know what you are connected to
<Scio[m]>
walle303: Could you tell me what node you try to connect to? I just tried connecting to the webrtc-star server and one of the bootstrap nodes it didn't do anything; the `swarm.peers()` is still empty.
<walle303[m]>
scio: I sent you some info, check your matrix invites
<Scio[m]>
walle303: Received, thanks. Sorry for all the trouble. I'm looking at it right now
jafow has joined #ipfs
warner has quit [Quit: ERC (IRC client for Emacs 25.1.2)]
ylp has joined #ipfs
corvinux has joined #ipfs
mildred1 has quit [Read error: Connection reset by peer]
realisation has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
s_kunk has quit [Ping timeout: 264 seconds]
espadrine has joined #ipfs
robattila256 has quit [Ping timeout: 240 seconds]
s_kunk has joined #ipfs
cxl000 has joined #ipfs
ztx is now known as trqx
nannal_ has joined #ipfs
nannal has quit [Ping timeout: 240 seconds]
Atrix has quit [Quit: Leaving]
<reit>
lgierth: do you happen to have any info on how many files there were, how large they were on average, and how long it took vs. connection bandwidth?
asyncsec has joined #ipfs
asyncsec has quit [Client Quit]
maxlath has joined #ipfs
ckwaldon has joined #ipfs
nikor has joined #ipfs
atbentley has joined #ipfs
neuthral_ has quit [Quit: leaving]
Guest41259 is now known as jayaura
neuthral has joined #ipfs
nunofmn has joined #ipfs
corvinux has quit [Ping timeout: 268 seconds]
nunofmn has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
cranky-sleeping is now known as crankylinuxuser
nunofmn has joined #ipfs
Guest217616[m] has joined #ipfs
tilgovi has quit [Ping timeout: 245 seconds]
kenshyx has joined #ipfs
corvinux has joined #ipfs
kenshyx has quit [Quit: Leaving]
fabrixxm is now known as Fab-Mat-rix
nunofmn has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
nunofmn has joined #ipfs
maxlath has quit [Quit: maxlath]
atbentley has quit [Ping timeout: 240 seconds]
jkilpatr has quit [Ping timeout: 260 seconds]
tom1 has joined #ipfs
nunofmn has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
nunofmn has joined #ipfs
rcat has joined #ipfs
Boomerang has joined #ipfs
jkilpatr has joined #ipfs
nunofmn has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
bertschneider has quit [Quit: Bye bye]
citizenErased has quit [Ping timeout: 240 seconds]
arpu has quit [Remote host closed the connection]
nunofmn has joined #ipfs
nunofmn has quit [Client Quit]
citizenErased has joined #ipfs
arpu has joined #ipfs
Akaibu has joined #ipfs
konubinix has quit [Remote host closed the connection]
konubinix has joined #ipfs
nunofmn has joined #ipfs
Guest57820 has joined #ipfs
Guest33263 has quit [Ping timeout: 258 seconds]
dimitarvp has joined #ipfs
caiogondim has joined #ipfs
asyncsec has joined #ipfs
wmoh has joined #ipfs
leeola has joined #ipfs
maxlath has joined #ipfs
gmoro has quit [Read error: Connection reset by peer]
<lgierth>
both boxes have 1GbE ports and are in datacenters close to each other
<lgierth>
i think i got something like 25 MBps data throughput
ZarkBit has joined #ipfs
<lgierth>
and 30 or 35 MBps on the wire
<lgierth>
make sure to start the receiving end with --routing=none so it doesn't get bound on sending out provider records for the new stuff it receives
<reit>
i see
<reit>
likely you were using the normal ~/.ipfs/ storage method, i wonder if using filestore (files on disk) would grant better results
<reit>
i guess at some point if it has to go away to the dht each time you'll have an unavoidable bottleneck
<lgierth>
i doubt it. kubuxu has been working on ipfs/go-sbs which had promising synthetic benchmarks so far
<lgierth>
the disk io bottleneck is more about the properties of the flatfs datastore
<lgierth>
it doesn't scale well up to these repo sizes. writes get slower the bigger the repo
<lgierth>
you can try it with some large dataset on a fresh repo and see the add performance decline over time
<reit>
is that limited to flatfs (what i am understanding to mean storing files directly in ~/.ipfs/), or is the result the same when using the new filestore code (storing the files directly on disk)?
<lgierth>
mmh i'm doing something bad with that pin earlier
<lgierth>
ooh i know
pinbot has quit [Remote host closed the connection]
<lgierth>
reit: not sure -- try it i guess. but i'm not sure you can retrieve stuff into nocopy/filestore
<lgierth>
you can definitely add stuff into it, just not sure about retrieval
<reit>
hm, it might be worth a try
<reit>
one last thing before i stop bugging you, do you happen to know which machine was io-bound (sender/receiver)?
<lgierth>
receiver
<reit>
oh good, i was worried it'd be the sender (i am thinking of the case where you have a central repo supplying a large dataset to many receivers)
<victorbjelkholm>
lgierth: "make sure to start the receiving end with --routing=none so it doesn't get bound on sending out provider records for the new stuff it receives" < should add this to the issue where there is bunch of workarounds for faster transfers
<lgierth>
victorbjelkholm: where's that issue :)
<victorbjelkholm>
hah, searched quickly and didn't find it so thought you would know...
mildred1 has quit [Read error: Connection reset by peer]
mildred1 has joined #ipfs
wmoh has joined #ipfs
chungy has joined #ipfs
nyseion[m] has joined #ipfs
mildred2 has quit [Ping timeout: 260 seconds]
Guest13105 has quit [Ping timeout: 260 seconds]
Guest13105 has joined #ipfs
ShalokShalom has joined #ipfs
wmoh has quit [Ping timeout: 260 seconds]
tom1 has joined #ipfs
robattila256 has quit [Quit: WeeChat 1.7.1]
reit has quit [Ping timeout: 240 seconds]
ilackarms has joined #ipfs
wmoh has joined #ipfs
wmoh has quit [Ping timeout: 240 seconds]
wmoh has joined #ipfs
neurrowcat has joined #ipfs
wmoh has left #ipfs [#ipfs]
MDude has quit [Remote host closed the connection]
tom1 has quit [Quit: WeeChat 1.7.1]
nannal_ has joined #ipfs
nannal has quit [Ping timeout: 260 seconds]
tom1 has joined #ipfs
ianopolous has joined #ipfs
Encrypt has quit [Quit: Quit]
robattila256 has joined #ipfs
rendar has quit [Quit: std::lower_bound + std::less_equal *works* with a vector without duplicates!]
tom1 has quit [Quit: WeeChat 1.7.1]
nunofmn has joined #ipfs
robattila256 has quit [Quit: WeeChat 1.7.1]
tom1 has joined #ipfs
robattila256 has joined #ipfs
sirdancealot has quit [Ping timeout: 260 seconds]
Caterpillar has quit [Quit: You were not made to live as brutes, but to follow virtue and knowledge.]
strykerkkd has joined #ipfs
kerem has joined #ipfs
jsgrant__ has quit [Quit: Peace Peeps. o/ If you need me asap, message me at msg[at]jsgrant.io & I'll try to get back to you within 24 hours.]
jsgrant has quit [Remote host closed the connection]
asyncsec has quit [Remote host closed the connection]
asyncsec has joined #ipfs
fragMental has quit [Ping timeout: 245 seconds]
ashark has quit [Ping timeout: 240 seconds]
jkilpatr has quit [Ping timeout: 260 seconds]
nunofmn has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
nunofmn has joined #ipfs
<kerem>
hey Kubuxu, mind sparing a moment for me to discuss further on https://github.com/ipfs/go-ipfs/issues/3807? I'm new to the repo but determined to get something done in there:)
<Kubuxu>
kerem: see my comment and: core/core.go#564 and keystore package
<kerem>
I just saw your reply, thanks!
<Kubuxu>
kerem: updated my response too
ilackarms has quit [Ping timeout: 240 seconds]
<kpcyrd>
I have a question about gx: it's seems the files that got pulled by gx still reference repos I don't have
<kpcyrd>
eg: vendor/gx/ipfs/QmaFNtBAXX4nVMQWbUqNysXyhevUj1k4B1y5uS45LC7Vw9/fuse/fs/serve.go:23:2: cannot find package "bazil.org/fuse" in any of:
<kpcyrd>
is this a bug or am I missing something?
neurrowcat has quit [Quit: Deebidappidoodah!]
achin has quit [Quit: WeeChat 1.4]
<Kubuxu>
90% you are missing gx-go
urtukyxat has joined #ipfs
<Kubuxu>
it shouldn't go to vendor
<Kubuxu>
rm -rf vendor
<Kubuxu>
and check which gx-go
<urtukyxat>
hello, I have a pull-streams question and I figured you all might know where I could go ask
tom1 has quit [Ping timeout: 260 seconds]
urtukyxat has quit [Client Quit]
<Kubuxu>
and he is gone
achin has joined #ipfs
robattila256 has quit [Quit: WeeChat 1.7.1]
<flyingzumwalt>
she/he/they. not clear. either way, gone. 😞
<kpcyrd>
Kubuxu: forgot to symlink go-gx in my bin, I've removed vendor/ and tried again, same error :/
<kpcyrd>
`gx install --global` ran successfully in both runs
<Kubuxu>
flyingzumwalt: it is frequent on IRC, I just like to point it out
<Kubuxu>
hmm, it should work with gx-go
<flyingzumwalt>
yeah. I know. It's part of why I pushed for discourse -- so people don't have to understand IRC in order to catch the answers to their questions
<flyingzumwalt>
I was just pointing out that they might not be a "he"
<kpcyrd>
Kubuxu: how is gx-go used during this process? it looks like gx install didn't notice it's not there
<Kubuxu>
it is helper tool, I think there if feature for package to "force" it
<Kubuxu>
it might be wise to force it by default for go
<kpcyrd>
any hints on how to do it? I'm currently trying `gmake install_unsupported` on openbsd, but the errors don't look openbsd specific yet
Oatmeal has quit [Quit: Suzie says, "TTFNs!"]
<Kubuxu>
make sure that $GOPATH/bin is in $PATH
<Kubuxu>
you might also want to set $GOBIN to $GOPATH/bin
ianopolous has quit [Ping timeout: 268 seconds]
jkilpatr has joined #ipfs
<kpcyrd>
Kubuxu: ok, I almost got it, I just can't get nofuse to work