<cblgh>
hsanjuan: what timeframe are you looking for feedback within?
<cblgh>
i've been eyeing ipfs cluster for my own application but i won't have time to play with until around april probably
<hsanjuan>
cblgh: not really imposing a timeframe, will be happy to receive feedback when you're ready
<cblgh>
hsanjuan: good to know!
<cblgh>
also congrats on getting it this far :3
<cblgh>
always feels nice when things start to get out there for people to use
<hsanjuan>
Thx <3
maxlath has quit [Ping timeout: 260 seconds]
rcat has joined #ipfs
koalalorenzo has quit [Ping timeout: 240 seconds]
ShalokShalom has quit [Read error: Connection reset by peer]
<Mateon1>
Well, for now I don't have multiple machines to try ipfs-cluster with, I've been looking at the repo for a while
<Mateon1>
Well, I do, but my raspberry pi is dying, and my VPS doesn't handle IPFS very well. I should probably move it to a machine with some more RAM
<Mateon1>
And my laptop is... Well, it's a laptop, so it's mobile, not always on, etc.
<cblgh>
hm, is there any way to list all of pinbot's hashes?
<cblgh>
would be a neat way to discover new things
<Mateon1>
cblgh: I found an old listing on ipfs-search.com a while ago
<cblgh>
that's one way to do it!
espadrine_ has joined #ipfs
<Mateon1>
I can't find it now
maxlath has joined #ipfs
maxlath has quit [Ping timeout: 240 seconds]
Guest125071[m] has joined #ipfs
maxlath has joined #ipfs
galois_d_ has quit [Remote host closed the connection]
maxlath has quit [Ping timeout: 260 seconds]
cemerick has joined #ipfs
<AphelionZ>
daviddias: might you be able to coach me in getting ipfs.refs.local implemented in js-ipfs?
<daviddias>
AphelionZ: for sure
<AphelionZ>
maybe just a couple links as to what I need to look at? It would be great to learn. I'm really good with JS; I think I just need a primer on how things are structured
<AphelionZ>
sweet
<daviddias>
we store all our objects in a IPFS Repo
<clownpriest>
is there a way to add files to a folder without rehashing all the parents in the merkle tree? i haven't tried to see if this is what ipfs actually does, but i'm assuming the entire folder just gets rehashed
<lgierth>
yeah it just rehashes the entire thing. you can use the `ipfs object patch` commands to mutate and existing DAG
<lgierth>
s/DAG/merkledag/
<clownpriest>
wouldn't the parent nodes in the merkle dag then need to have their own hashes updated then?
<lgierth>
yeah you need to bubble up the changed hashes
<lgierth>
i have a need for a tool for that too -- e.g. something that parses rsync output and only updates
<lgierth>
i have this 4 TB dataset which gets updated using rsync
<clownpriest>
wouldn't "bubbling" up imply a complete rehash? or would you just hash the existing hashes for the unchanged nodes?
<lgierth>
you take the hash of the new file a/b/c.txt and add a link called c.txt to a/b
<lgierth>
then take the new a/b hash and update the link called b in a
<lgierth>
it's really annoying without a tool doing it for you :)
<lgierth>
but the low-level tool is there (ipfs object patch), so it's open to scripting
<clownpriest>
i thought the hashes for any given object include the links of all its children too? so a change in 'b' would mean the hash for 'a' would need rehashing
<lgierth>
yeah that's what i said
<clownpriest>
oh
<lgierth>
:)
<clownpriest>
it sounds like whatever you do, the root always needs a rehash though, regardless of where in the chain the diff is made, but you can avoid rehashing things below (or horizontal to) the chain where you made the diff. or I'm not actually following...
<clownpriest>
if that's the case, there's no way to avoid a root rehash?
<clownpriest>
i'm sorry if being dumb here...
<clownpriest>
i'm being*
<AphelionZ>
clownpriest: I think the concern you're looking at is solved by ipns
ygrek_ has joined #ipfs
<AphelionZ>
I think :) I'm still learning too
<clownpriest>
AphelionZ: thanks, i'll take a closer look at that
<AphelionZ>
but content-addressable means "if the content changes, the hash changes"
<AphelionZ>
but you can keep a consistent ipns hash and move the content hash around underneath it and have something consistent to rely on
<clownpriest>
that would be pretty great. though I would also like to avoid having to rehash a directory that's 100's of TB's. but maybe there are certain tradeoffs that are inherent here
<clownpriest>
assuming that the underlying data than an ipns address refers to needs to get rehashed at some point
<AphelionZ>
yeah that would definitely take several, like double digit, minutes to rehash i guess, wouldnt it
<clownpriest>
i was assuming way more. I've never "ipfs add"ed anything that huge
<AphelionZ>
high double digits, I guess
<AphelionZ>
it also seems to scale with the number of files too, not just the size
<clownpriest>
:(
<AphelionZ>
well im not saying dont do it, im saying just be aware of potential pitfalls
<AphelionZ>
keep in mind it's all alpha :)
<clownpriest>
yeah. if a 100TB folder rehash takes less than an hour it's probably worth it to me
<clownpriest>
yeah for sure lol
ygrek_ has quit [Ping timeout: 240 seconds]
<AphelionZ>
whyrusleeping knows a lot about tonnes of hashes and huge files
<whyrusleeping>
i like hashes
<clownpriest>
hashes ftw
<whyrusleeping>
biggest thing i've added so far is 4TB
<clownpriest>
do you remember how long that took?
<whyrusleeping>
i can get things to bottleneck on sha256 hashing speed right now
john2 has joined #ipfs
<whyrusleeping>
and disk read speeds
<whyrusleeping>
I think 4TB took around 2.5-3 days
<clownpriest>
holy shit....
<whyrusleeping>
Yeah, the issue i was seeing was getting bottlenecked by metadata writing stuff
<jbenet>
only hashing should be much faster
<clownpriest>
any chance an upgrade to blake2 is in the cards? or are we sha256 furever?
<whyrusleeping>
clownpriest: soon
<jbenet>
yeah we're talking about it :)
<clownpriest>
i see
<clownpriest>
can you hold the metadata in memory? in a streaming/garbage collected way? or is it explosively huge?
<whyrusleeping>
clownpriest: this is talking about disk metadata
<clownpriest>
ahh
<whyrusleeping>
its an issue with out flatfs storage engine
<whyrusleeping>
we're working on a replacement that will make things much faster
<victorbjelkholm>
dignifiedquire: you have a link for the stream?
m3lt has joined #ipfs
<dignifiedquire>
victorbjelkholm: no stream today
m3lt has left #ipfs [#ipfs]
<victorbjelkholm>
Kubuxu: if you're streaming, could you put the link here?
<Kubuxu>
victorbjelkholm: not streaming, just got home, didn't have time to set up
<victorbjelkholm>
Kubuxu: ah, I see. Thanks to both of you
ebcz has quit [Ping timeout: 240 seconds]
maxlath has quit [Ping timeout: 260 seconds]
matoro has quit [Ping timeout: 240 seconds]
ygrek_ has joined #ipfs
akkad has quit [Excess Flood]
zabirauf has joined #ipfs
akkad has joined #ipfs
zabirauf has quit [Ping timeout: 258 seconds]
wallacoloo_____ has joined #ipfs
<A124>
"Announced soon."
wallacoloo_____ has quit [Client Quit]
<lgierth>
yep no stream today, sorry :( recording though
<lgierth>
join the zoom call if you want, no camera obligation :)
ygrek_ has quit [Ping timeout: 264 seconds]
<captain_morgan>
I'm trying to get started with ipfs again but everytime I try to do anything I get "WARN: [...] Running query with no peers!" from my daemon
<whyrusleeping>
captain_morgan: what version are you trying?
<whyrusleeping>
and are you behind a very restrictive firewall?
<lgierth>
or ran ipfs bootstrap rm --all
<captain_morgan>
0.3.8-dev not super restrictive
<lgierth>
yeeeah that's very very old
<lgierth>
please update to v0.4.6
<captain_morgan>
oh, hmm, I thought I just downloaded it
<lgierth>
0.3.x is not supported anymore since like 9 months :)
<captain_morgan>
maybe I had an old version sitting around still
<whyrusleeping>
captain_morgan: Yeah, you must have just had it laying around
<whyrusleeping>
0.3.8 was released october 2015 :P
<captain_morgan>
ok upgrade and seeing nat warnings now, so that does appear to be the issue
chris613 has joined #ipfs
matoro has joined #ipfs
Jesin has quit [Quit: Leaving]
Jesin has joined #ipfs
palkeo has joined #ipfs
palkeo has joined #ipfs
palkeo has quit [Changing host]
<captain_morgan>
whoa, I may have had a long, long standing bug in my router that was breaking UPNP
ygrek_ has joined #ipfs
<captain_morgan>
could explain mass trouble I had with syncthing also
<victorbjelkholm>
ged_: should be working, at least last time I tried it, it was working. Could you specifically what's not working in the PR so we can fix it if something is wrong?
<ged_>
I might be doing something wrong, but here it goes: Project runs fine, I can add peers by opening 2 tabs, copying 1st one's hash, constructing such peer address: /ip4/127.0.0.1/tcp/9999/ws/ipfs/<hash> and pasting it to the second tab's peer input field
<ged_>
it shows up in the list
<ged_>
I add a file in the 1st tab and copy it's hash to 2nd tab's input field for fetching
<ged_>
and it seems that stream.on() isn't called
mildred has quit [Ping timeout: 240 seconds]
jonnycrunch has quit [Quit: leaving]
<ged_>
The tutorial kinda lacks the end - steps to see if the solution is working properly
jonnycrunch has joined #ipfs
Akaibu has quit [Quit: Connection closed for inactivity]
Encrypt has quit [Quit: Quit]
<ged_>
also the added file via web client does not appear to be reachable using ipfs.io gateway. I'm running go-ipfs in background. I might be missing the point of this tutorial :)
jonnycrunch has quit [Client Quit]
jonathan2 has joined #ipfs
jonathan2 has quit [Client Quit]
<whyrusleeping>
captain_morgan: copy paste those warnings and errors here?
<captain_morgan>
and these are with a pin running in another terminal
<Guest30493>
captain_morgan: can you ping 127.0.0.2?
Guest30493 is now known as Ronsor
Ronsor has quit [Changing host]
Ronsor has joined #ipfs
jonnycrunch1 has joined #ipfs
<captain_morgan>
Yes
<Ronsor>
hrm odd
<ged_>
victorbjelkholm: any clue what I'm doing wrong?
jkilpatr has quit [Quit: Leaving]
s_kunk has joined #ipfs
jonnycrunch has quit [Quit: leaving]
jonnycrunch1 is now known as jonnycrunch
<whyrusleeping>
captain_morgan: do you have extra debug logs turned on?
<captain_morgan>
IPFS_LOGGING=warning
<whyrusleeping>
Ah, okay
jkilpatr has joined #ipfs
<whyrusleeping>
Most of those are completely normal
<captain_morgan>
ok, I figured some where
<whyrusleeping>
Yeah, with so many random connections to random peers that may be going offline or behaving weirdly, youre definitely gonna see some 'failed to do thing' messages
jonnycrunch has quit [Quit: jonnycrunch]
Akaibu has joined #ipfs
<captain_morgan>
sure, I'd expect that
<captain_morgan>
trouble is, coming in fresh, hard to tell which are normal, and which are actual problems
dgrisham has quit [Quit: WeeChat 1.7]
<captain_morgan>
trying to focus on WARN and above
jonnycrunch has joined #ipfs
seharder_ has left #ipfs [#ipfs]
<Kubuxu>
we have to clean up the WARN channel
<captain_morgan>
still too verbose? seems like a lot of these could be downgraded to INFO
<Kubuxu>
yeah
john4 has joined #ipfs
john2 has quit [Ping timeout: 256 seconds]
mildred has joined #ipfs
tmg has joined #ipfs
<captain_morgan>
whyrusleeping, should this suggestion still be valid? I get a new wanring, no transport for address
<whyrusleeping>
install with 'go get github.com/whyrusleeping/natest'
<captain_morgan>
I think I've resolved the nat problem, I'm seeing the entry in my UPNP table but I'll give it a try
<whyrusleeping>
run with just 'natest'
<whyrusleeping>
flyingzumwalt: pinning
<whyrusleeping>
flyingzumwalt: is your upload moderately good?
<flyingzumwalt>
not sure. I'm at embassy SF
<whyrusleeping>
flyingzumwalt: mind running 'ipfs swarm connect /ip4/5.9.59.34/tcp/4001/ipfs/QmRv1GNseNP1krEwHDjaQMeQVJy41879QcDwpJVhY8SWve' ?
<whyrusleeping>
I think your nat traversal might be wonky (or our infra node isnt trying very hard)
<flyingzumwalt>
> success
<whyrusleeping>
grrrreat, thanks
<flyingzumwalt>
wierdly, I tried loading it from the gateway before posting the link in irc. it seemed to work, but I only loaded the directory listing
<whyrusleeping>
flyingzumwalt: yeah, your node probably made an outbound connection to the gateway
john4 has quit [Ping timeout: 264 seconds]
Guest34297 has joined #ipfs
espadrine has quit [Ping timeout: 240 seconds]
gavinmcdermott has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
jonathan1 has joined #ipfs
<jonathan1>
help
jonathan1 has quit [Quit: leaving]
<Ronsor>
///help
<Ronsor>
/help
<whyrusleeping>
hrm?
<Ronsor>
i was trying to type '/help'
<Ronsor>
and in the wrong channel too
<Ronsor>
whyrusleeping: i finished ipfsircd v2.01
<Ronsor>
adds some bugfixes
structuralist has joined #ipfs
jonnycrunch2 has joined #ipfs
zabirauf has joined #ipfs
<victorbjelkholm>
ged_: yeah, right now go-ipfs <> js-ipfs doesn't work out-of-the-box, we're working on it. We don't have DHT in js-ipfs either, which makes content discovery harder. Right now you have to manually dial to go-ipfs nodes you know have websockets + multiplex enabled
zabirauf has quit [Ping timeout: 240 seconds]
<ged_>
victorbjelkholm: yeah, figured that out as well. so I should connect to go-ifps from web js-ifps node, right?
<victorbjelkholm>
ged_: you'll need to add listening to the websocket connection in go-ipfs config then you should take that address and connect to it from the input field of the example
jkilpatr has quit [Ping timeout: 240 seconds]
kulelu88 has joined #ipfs
jkilpatr has joined #ipfs
<ged_>
victorbjelkholm: I've modified the config. Now I have this address: /ip4/127.0.0.1/tcp/9999/ws. How do I build it to the multiaddr format? /ip4/127.0.0.1/tcp/9999/ws/<hashOfJSNode>?
<frood>
does ipfs-js exclusively use websockets right now?
<frood>
/s/ipfs-js/js-ipfs
wallacoloo_____ has joined #ipfs
wallacoloo_____ has quit [Client Quit]
espadrine has joined #ipfs
kulelu88 has quit [Ping timeout: 264 seconds]
tilgovi has joined #ipfs
kulelu88 has joined #ipfs
martinBrown has quit [Remote host closed the connection]
martinBrown has joined #ipfs
Guest34297 has quit [Ping timeout: 260 seconds]
cemerick has quit [Ping timeout: 246 seconds]
Ohelig has quit [Quit: And Harry, I remembered. It was a police officer.]
kulelu88 has quit [Ping timeout: 260 seconds]
ged_ has quit [Ping timeout: 260 seconds]
ZaZ has quit [Read error: Connection reset by peer]
Ohelig has joined #ipfs
maxlath has joined #ipfs
kulelu88 has joined #ipfs
shakalaka has quit [Quit: bye.]
shakalaka has joined #ipfs
clownpriest has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
jonnycrunch2 has quit [Ping timeout: 240 seconds]
anewuser_ has joined #ipfs
maxlath has quit [Quit: maxlath]
Encrypt has joined #ipfs
anewuser has quit [Ping timeout: 268 seconds]
jonnycrunch has quit [Quit: Lost terminal]
structuralist has quit [Remote host closed the connection]
rcat has quit [Remote host closed the connection]
structuralist has joined #ipfs
cxl000 has quit [Quit: Leaving]
monkhood_ has joined #ipfs
structuralist has quit [Ping timeout: 246 seconds]
monkhood has quit [Ping timeout: 240 seconds]
Boomerang has quit [Quit: Lost terminal]
tilgovi has quit [Ping timeout: 246 seconds]
rcat has joined #ipfs
rcat has quit [Client Quit]
zabirauf has joined #ipfs
rcat has joined #ipfs
luisbg[m] has joined #ipfs
luisbg has joined #ipfs
luisbg has left #ipfs ["WeeChat 1.6"]
jonnycrunch has joined #ipfs
rendar has quit [Ping timeout: 240 seconds]
zabirauf has quit [Ping timeout: 264 seconds]
phorse` has quit [Remote host closed the connection]
ladron`` has quit [Read error: Connection reset by peer]
luisbg has joined #ipfs
jonnycrunch has quit [Ping timeout: 268 seconds]
jonnycrunch has joined #ipfs
stoopkid has quit [Quit: Connection closed for inactivity]
luisbg has left #ipfs ["WeeChat 1.6"]
Eru has joined #ipfs
jonnycrunch has quit [Ping timeout: 268 seconds]
dimitarvp has quit [Quit: Bye]
phorse has joined #ipfs
bwerthmann has quit [Quit: leaving]
bwerthmann has joined #ipfs
bwerthmann has quit [Client Quit]
bwerthmann has joined #ipfs
rcat has quit [Quit: leaving]
clownpriest has joined #ipfs
rcat has joined #ipfs
Encrypt has quit [Quit: Quit]
ashark_ has quit [Ping timeout: 240 seconds]
rendar has joined #ipfs
jkilpatr has quit [Ping timeout: 260 seconds]
structuralist has joined #ipfs
Akaibu has quit [Quit: Connection closed for inactivity]
palkeo has quit [Ping timeout: 240 seconds]
Jesin has quit [Quit: Leaving]
Jesin has joined #ipfs
bwn has quit [Ping timeout: 256 seconds]
rcat has quit [Quit: leaving]
mildred has quit [Ping timeout: 246 seconds]
horrified has quit [Quit: gg weechat 800MiB mem usage]
horrified has joined #ipfs
bwn has joined #ipfs
jkilpatr has joined #ipfs
matoro has quit [Ping timeout: 264 seconds]
Jesin has quit [Quit: Leaving]
espadrine has quit [Ping timeout: 260 seconds]
kulelu88 has quit [Ping timeout: 260 seconds]
matoro has joined #ipfs
ygrek_ has quit [Ping timeout: 268 seconds]
shizy has quit [Ping timeout: 240 seconds]
bwn has quit [Ping timeout: 260 seconds]
Jesin has joined #ipfs
mildred has joined #ipfs
monkhood_ has quit [Ping timeout: 240 seconds]
kulelu88 has joined #ipfs
tilgovi has joined #ipfs
bwn has joined #ipfs
cemerick has joined #ipfs
ligi has quit [Ping timeout: 240 seconds]
cemerick has quit [Ping timeout: 246 seconds]
matoro has quit [Ping timeout: 260 seconds]
wallacoloo_____ has joined #ipfs
Caterpillar has quit [Quit: You were not made to live as brutes, but to follow virtue and knowledge.]
kulelu88 has quit [Ping timeout: 240 seconds]
rendar has quit [Quit: std::lower_bound + std::less_equal *works* with a vector without duplicates!]