pfraze has quit [Remote host closed the connection]
arpu has joined #ipfs
simonv3[m] has joined #ipfs
devbug has joined #ipfs
matoro has joined #ipfs
lawliet has quit [Quit: Leaving]
<voxelot>
alu: doing a hackathon at my place this weekend to implement the new merkle verification we have in blockchains now on https://github.com/nginnever/fileswarm
<JesseW>
voxelot: hm, say more?
<voxelot>
so soon i might have client running that you could upload a file too, download from, and view the seeders
<voxelot>
you'll be paying for it in ether atm though
giodamelio has quit [Quit: ZNC 1.6.1 - http://znc.in]
giodamelio has joined #ipfs
<voxelot>
it's keeping fileswarm from running in electron, browser works since you can require the prebuilt without the webpack error, any thoughts on that or where i can start digging for a fix?
giodamelio has left #ipfs [#ipfs]
giodamelio has joined #ipfs
<voxelot>
for some reason electron is crashing when rendering the prebuilt for me
cemerick has joined #ipfs
BadgerSong has joined #ipfs
BadgerSong has quit [Ping timeout: 250 seconds]
devbug has joined #ipfs
PrinceOfPeeves has quit [Ping timeout: 244 seconds]
devbug has quit [Ping timeout: 276 seconds]
mvollrath has joined #ipfs
mvollrath has quit [Ping timeout: 265 seconds]
taaem has quit [Remote host closed the connection]
taaem has joined #ipfs
taaem has quit [Remote host closed the connection]
taaem has joined #ipfs
anewuser has quit [Ping timeout: 258 seconds]
taaem has quit [Remote host closed the connection]
mvollrath has joined #ipfs
BadgerSong has joined #ipfs
BadgerSong has quit [Ping timeout: 244 seconds]
<Remram[m]>
Alright I want to build a file-syncing app on top of IPFS
<Remram[m]>
how do I use any of your stuff?
<Remram[m]>
I don't even know what's libp2p and what's IPFS
<Remram[m]>
I don't even know for sure that IPFS uses libp2p yet
<Remram[m]>
I have read every readme I could find, but
<Remram[m]>
It seems that you are trying to develop a protocol, a standard for people to build apps on top of, yet for anything more advanced than "ipfs add" "ipfs pin" there seems to be no way to determine what the system can even do short of reading the sources
apiarian has quit [Ping timeout: 276 seconds]
<Remram[m]>
it's frustrating 😭
<Remram[m]>
you have too many components to not have docs
apiarian has joined #ipfs
herzmeister has quit [Quit: Leaving]
herzmeister has joined #ipfs
<JesseW>
Remram[m]: I certainly can't argue that the docs can and should be better -- but have you seen https://github.com/ipfs/specs/
<JesseW>
there's quite a bit there
<Remram[m]>
About the implementation architecture?
r04r is now known as zz_r04r
<JesseW>
yeah (although quite likely not *enough*)
<Remram[m]>
that's unusual
mgue has quit [Ping timeout: 244 seconds]
<geoah>
whyrusleeping :D thanks for the gx tips
<whyrusleeping>
:)
<whyrusleeping>
thanks for using gx, i'm making notes of all the things that could be easier and about stuff to document better
<whyrusleeping>
Remram[m]: what is it you want to know?
<geoah>
I had the same idea about the github bot -- just reporting dupes would be a lifesaver :P
<JesseW>
geoah: thanks for explaining about the package splitting -- that makes plenty of sense; it just wasn't clear to me from the diff
<whyrusleeping>
geoah: yeah :)
<whyrusleeping>
really cool thing that will happen once this is finalized: ipns-pub can finally stop being broken all the time
<JesseW>
I want to do more, but I'll make a new PR when I do
<whyrusleeping>
JesseW: if you ever need my attention on anything please just ping me on irc, its pretty hard to keep up with everything on github
<whyrusleeping>
especially on repos other than go-ipfs
<JesseW>
certainly, will do
<JesseW>
Heh, I'll do one better -- I'll make PRs to *mention* that fact in the READMEs of the various repos you have as part of ipfs. :-)
<geoah>
rofl JesseW I can help on that :P
<JesseW>
geoah: please do!
BadgerSong has joined #ipfs
<geoah>
awesome... all checks on ipfs/go-ipfs#3185 actually passed.
<geoah>
I think I can now happily call it a day and get some sleep :P
<JesseW>
geoah: congrads!
mgue has quit [Quit: WeeChat 1.5]
<geoah>
^_^ thanks JesseW
BadgerSong has quit [Ping timeout: 264 seconds]
<geoah>
whyrusleeping thank you for all your help today (and these past days) -- hopefully there aren't many things to fix and bug you with on the remaining PRs :D
silotis has quit [Remote host closed the connection]
silotis has joined #ipfs
cemerick has quit [Ping timeout: 265 seconds]
cubemonkey has quit [Read error: Connection reset by peer]
silotis has quit [Remote host closed the connection]
silotis has joined #ipfs
pfraze has joined #ipfs
ipfs has joined #ipfs
ipfs has quit [Client Quit]
_whitelogger has joined #ipfs
BadgerSong has joined #ipfs
BadgerSong has quit [Ping timeout: 276 seconds]
pfraze has quit [Remote host closed the connection]
dansup has quit [Ping timeout: 252 seconds]
ylp has quit [Ping timeout: 260 seconds]
herzmeister has quit [Ping timeout: 250 seconds]
dansup has joined #ipfs
ylp has joined #ipfs
BadgerSong has joined #ipfs
herzmeister has joined #ipfs
cyberwolf has joined #ipfs
BadgerSong has quit [Ping timeout: 240 seconds]
wavis has quit [Quit: Connection closed for inactivity]
dansup has quit [Ping timeout: 258 seconds]
rendar has joined #ipfs
cyberwolf has quit [Quit: Konversation terminated!]
ylp has quit [Ping timeout: 260 seconds]
ylp has joined #ipfs
dansup has joined #ipfs
wubalubadubdub has joined #ipfs
BadgerSong has joined #ipfs
BadgerSong has quit [Ping timeout: 265 seconds]
<wubalubadubdub>
Sorry, might be a bit of a noob question, but I wasn't able to find information about it or piece it together myself: (How) does ipfs work behind a dynamic IP adress? Can I publish a site using a machine at home or do I have to use a server with a static ip adress?
corvinux has joined #ipfs
corvinux has quit [Ping timeout: 240 seconds]
JesseW has quit [Ping timeout: 244 seconds]
<Kubuxu>
wubalubadubdub: you can publish using DynIP, but if you machine changes address or goes offline you need someone to rehost it.
cketti has quit [Quit: Leaving]
<wubalubadubdub>
Kubuxu: Hm, ok, thank you. I get disconnected by my ISP once every 24 hours. Need to find a workaround then.
<Kubuxu>
if ipfs reconnects then it should be fine
<Kubuxu>
it will publish new peer records, which will point to new IP
<Kubuxu>
at least it should work like that
<wubalubadubdub>
Ah ok.
<wubalubadubdub>
Well, I have to try it I guess. Thank you.
amiller_ has joined #ipfs
amiller_ has quit [Changing host]
amiller has joined #ipfs
espadrine has joined #ipfs
kode54 has quit [Ping timeout: 252 seconds]
BadgerSong has joined #ipfs
corvinux has joined #ipfs
BadgerSong has quit [Ping timeout: 265 seconds]
corvinux has quit [Ping timeout: 244 seconds]
vapid has joined #ipfs
bashrc has quit [Ping timeout: 265 seconds]
devbug has joined #ipfs
vapid has quit [Ping timeout: 265 seconds]
wubalubadubdub has quit [Remote host closed the connection]
dignifiedquire has joined #ipfs
kode54 has joined #ipfs
vapid has joined #ipfs
BadgerSong has joined #ipfs
BadgerSong has quit [Ping timeout: 250 seconds]
devbug has quit [Quit: ZZZzzz…]
PseudoNoob has joined #ipfs
computerfreak has joined #ipfs
ygrek has quit [Ping timeout: 244 seconds]
HastaJun_ has quit [Ping timeout: 264 seconds]
HastaJun_ has joined #ipfs
computerfreak has quit [Remote host closed the connection]
computerfreak has joined #ipfs
cyberwolf has joined #ipfs
m0ns00n has quit [Quit: quit]
gamemanj has joined #ipfs
cyberwolf has quit [Quit: Konversation terminated!]
herzmeister has quit [Quit: Leaving]
herzmeister has joined #ipfs
ppham has quit [Remote host closed the connection]
ppham has joined #ipfs
ppham has quit [Ping timeout: 252 seconds]
Encrypt has joined #ipfs
TheRec has joined #ipfs
vapid has quit [Remote host closed the connection]
vapid has joined #ipfs
m0ns00n_ has joined #ipfs
computerfreak has quit [Remote host closed the connection]
taaem has joined #ipfs
apiarian has quit [Quit: zoom]
apiarian has joined #ipfs
Encrypt has quit [Quit: Quitte]
apiarian has quit [Ping timeout: 250 seconds]
vapid has quit [Remote host closed the connection]
Foxcool has quit [Ping timeout: 265 seconds]
vapid has joined #ipfs
ppham has joined #ipfs
vapid has quit [Remote host closed the connection]
vapid has joined #ipfs
Encrypt has joined #ipfs
reit has quit [Quit: Leaving]
derpies has joined #ipfs
dansup has quit [Ping timeout: 252 seconds]
vapid has quit [Remote host closed the connection]
Foxcool has joined #ipfs
<geoah>
morning all :)
vapid has joined #ipfs
XKS has joined #ipfs
<geoah>
Kubuxu: why would wuba* need a static ip or dyndns to host a site on IPFS? Isn't that kinda missing the point a bit? Or am I missunderstanding the question?
<Kubuxu>
geoah: no. There is just quite a lot of things that have to go right when his IP changes for everything to work correctly.
<Kubuxu>
1. Refresh of unpn lease has to go through a router, he gets new lease, our system processes it and publishes new peerstore records, and those records reach peers that are interested in his content.
<Kubuxu>
it should work
<Kubuxu>
dyndns wouldn't help at all as ipfs doesn't know about dns
M-Purple has joined #ipfs
<geoah>
hm yeah that makes at least for while there aren't many pins for your content
<gamemanj>
Kubuxu: Given that the IP generally only changes when the router reboots, and people tend to reboot routers and computers connected to them at around about the same time (going offline every night), things should be fine...?
<Kubuxu>
IP changes also when your ISP changes your lease
<Kubuxu>
and this is the case with wubalu.. question
<geoah>
ipv6 should make things a bit better, at least in europe
<gamemanj>
geoah: define "europe"...
<geoah>
I think most providers here will give out an almost static subnet to users
<geoah>
gamemanj: uk, de, gr are the ones I know :P
<gamemanj>
if you mean "uk" as in "united kingdom", the real question is "do we even have IPv6"
<Kubuxu>
they will have quite soon
<gamemanj>
Like, for example, if I was to turn on IPv6 on my computer and see if the network actually let me use it...
<Kubuxu>
as they have to do their internet infra from ground 0
<Kubuxu>
as it is really old
<gamemanj>
scope link, no IPv6 routine.
<Kubuxu>
countries that are screwed are those that did internet infra 5 years ago instead of 10
<gamemanj>
*routing
<gamemanj>
Kubuxu: What's wrong with 5 years ago?
<geoah>
gamemanj: you can contact your provider and ask them to enable it
<Kubuxu>
it was just done, and doesn't need to be modernized
<gamemanj>
Kubuxu: ... so of course nobody allocates the budget for the IPv6 upgrade.
<Kubuxu>
meaning routers with no or not full v6
<Kubuxu>
yup
<Kubuxu>
you might get v6 when your provider has to do an upgrade of infra
<Kubuxu>
but they won't do an infra update just for v6
<gamemanj>
geoah: I'm pretty sure that's a "maybe", and not a definite
<geoah>
gamemanj: I assume you got bt ? :P
<geoah>
sky and virgin should already have ipv6 in most places
<gamemanj>
There's a bit of complicated explaination about that. See, here, as in, where I am typing, the home router gets turned off every night due to "fire risk". It's on Virgin, BTW, no IPv6.
BadgerSong has joined #ipfs
<geoah>
wait what?
<geoah>
> the home router gets turned off every night due to "fire risk"
<geoah>
so... you live in the 90s? :P
<gamemanj>
Where my Pi, aka "the thing that runs daemons" is, it's on BT. It also does not have IPv6.
<geoah>
if your BT router is newer than 4 or 5 try contacting them, if your local DSLAM has ipv6 enabled they will hapilly enable it
<geoah>
you can see if your router supports it under "advanced settings > broadband > ipv6 status"
<gamemanj>
Meh. The only real benefit to IPv6 is that, if they're doing things properly , there's no need to worry about the "one port two users" problem.
taaem has quit [Remote host closed the connection]
<geoah>
imo the real benefit in ipv6 is anything p2p
<geoah>
by that I mean from things like ipfs to webrtc and anything that people will think of once they don't have to punch through nat and shit
<gamemanj>
For punching through NAT, I'm sure the makers of the Dolphin emulator won't mind you "borrowing" their STUN server (Me? Wanting revenge on people for removing support for older GPUs, and trying to get people to help me? I would never do such a thing!)
<ansuz>
I found out the other day that edge isn't implementing webrtc
<ansuz>
:|
<gamemanj>
"isn't" meaning they haven't yet, or won't ever?
<ansuz>
won't
<ansuz>
they're implementing "objectRTC"
<gamemanj>
It's the Curse Of Internet Explorer in full swing.
<gamemanj>
They try doing something from the classical Microsoft playbook
<gamemanj>
and either everybody jumps to it, then the other browsers implement it so nobody cares,
<gamemanj>
or nobody jumps to it, and that's another reason nobody uses IE
<gamemanj>
(because IE refuse to implement the real standard)
BadgerSong has quit [Ping timeout: 276 seconds]
<ansuz>
I only really test in firefox
reit has joined #ipfs
corvinux has joined #ipfs
<ianopolous>
does anyone know if the videos from the lisbon workshop have been posted?
BadgerSong has joined #ipfs
BadgerSo1g has joined #ipfs
BadgerSong has quit [Ping timeout: 255 seconds]
computerfreak has joined #ipfs
pfraze has joined #ipfs
herzmeister has quit [Quit: Leaving]
ebel has quit [Ping timeout: 260 seconds]
herzmeister has joined #ipfs
ebel has joined #ipfs
corvinux has quit [Ping timeout: 258 seconds]
BadgerSo1g has quit [Quit: leaving]
dmr has quit [Ping timeout: 250 seconds]
zz_r04r is now known as r04r
<whyrusleeping>
TIL: 'expr 0' has a non-zero exit code
<gamemanj>
but expr 2 has a zero error code
ppham has quit [Remote host closed the connection]
<gamemanj>
I think the error code is meant to emulate C booleans
amiller has quit [Ping timeout: 250 seconds]
ppham has joined #ipfs
<gamemanj>
"echo What is the answer to the ultimate question ; read number ; expr 42 == $number ; echo $?" seems to pretty much describe what they were trying to achieve with that
mvollrath has quit [Ping timeout: 258 seconds]
ppham has quit [Ping timeout: 276 seconds]
<gamemanj>
"if expr" seems to be where this becomes really useful
<whyrusleeping>
ah, i see
<gamemanj>
This is mostly guesswork and me trying it out to see what happens :)
<geoah>
hey whyrusleeping
<whyrusleeping>
geoah: heyo
<geoah>
I expected this channel a bit more lively with 800+ users :P
<whyrusleeping>
geoah: yeah, i think that might be because we try to stay more on topic
marksei has joined #ipfs
marksei has quit [Max SendQ exceeded]
<whyrusleeping>
when the channel gets off onto some tangent i'll see random people i've never talked to before talking up a storm
<geoah>
rofl nice :P we need an #ipfs-random or something :P
<whyrusleeping>
i'm totally fine having an #ipfs-random type channel
<whyrusleeping>
or, alternatively, an #ipfs-dev
computerfreak has quit [Remote host closed the connection]
Guest60584 has joined #ipfs
PseudoNoob has quit [Remote host closed the connection]
Guest60584 has quit [Changing host]
amiller has joined #ipfs
<geoah>
cause you are right, if people keep spamming with random stuff it makes devs' lives hard :P
amiller has quit [Ping timeout: 240 seconds]
<whyrusleeping>
yeah, its mainly ansuz that spams random stuff
<geoah>
I would probably add my name to that list as well :P
Guest10486 has joined #ipfs
<cjd>
oh yeah, he's constantly spamming random shit
<cjd>
BTW what do you guys think of the fact that Ethereum is the same price as Monero /
<cjd>
?
<whyrusleeping>
cjd: idk, i don't play magic
<cjd>
it's not magic, it's beaniebabies !
<whyrusleeping>
ooooooh, okay
<whyrusleeping>
cjd: does ansuz do anything other than play babyfoot at work?
mvollrath has quit [Ping timeout: 244 seconds]
Guest10486 has quit [Ping timeout: 244 seconds]
<cjd>
y, u trying to hire him ?
<cjd>
if so the answer is absolutely not
<gamemanj>
I'm pretty sure the game is tiddlywinks.
<gamemanj>
As in, the game where you flick little counters about the room.
<cjd>
maybe its a good idea to diversify between ethereum and monero so that when your ethereum tanks you can buy drugs to help you forget
<gamemanj>
If I had a time machine, I would have shorted against Ethereum-related enterprises long ago, and if anyone asked why, I would have said "It looked like a bubble." I don't see why it'll tank now, or at least not for a while. Maybe when people realize that Ethereum applies a tax on everything due to gas costs.
herzmeister has quit [Quit: Leaving]
herzmeister has joined #ipfs
<cjd>
and doesn't offer anything that ETC doesn't also offer at a discount...
fleeky_ has quit [Quit: Ex-Chat]
espadrine has quit [Ping timeout: 260 seconds]
cketti has joined #ipfs
<whyrusleeping>
cjd: lol, us americans buy drugs with cold hard freedom dollars
<gamemanj>
"freedom dollars"
<gamemanj>
I can play the patriotic line! Us British people, we buy our caffine-related substances with cold hard British pounds, ay mate!
<geoah>
rofl @ freedom dollars
<cjd>
pretty sure the stuff on the street is shit compared to darkmarket drugs
<gamemanj>
...is "ay mate" how we're supposed to sound?
<gamemanj>
Can someone who knows British stereotypes please inform me if "ay mate" sounds correct?
<geoah>
gamemanj: doesn't sound right :P
<gamemanj>
That figures.
<geoah>
maybe oi mate?
<geoah>
but ay is very pirate-y
<gamemanj>
rig't mate?
<gamemanj>
Or something? IDK. I think we may be going on a rather large tangent.
<gamemanj>
(That's probably my fault.)
<whyrusleeping>
cjd: can you make cjdns work betterer?
<Kubuxu>
whyrusleeping: he is dropping the project :p
<whyrusleeping>
cjd: cmon, wikis are cool and all, but packet switched overlay networking is where its at
<cjd>
you need to make a shitcoin so I can pump'n'dump it and then I don't need to work
mvollrath has joined #ipfs
amiller_ has joined #ipfs
<gamemanj>
IPFS-over-Hyperboria would be fun
Blame_ has quit [Quit: Connection closed for inactivity]
palkeo has joined #ipfs
apiarian has joined #ipfs
apiarian has quit [Ping timeout: 264 seconds]
Oatmeal has joined #ipfs
Oatmeal has quit [Max SendQ exceeded]
computerfreak has joined #ipfs
<whyrusleeping>
geoah: on go-libp2p-loggables wanna do a rebase and bump CI to go1.7?
<whyrusleeping>
you've inadvertently made us set up CI on all the subrepos
jemc has joined #ipfs
<geoah>
whyrusleeping: hehe I'll consider that a good thing :P -- on it
<whyrusleeping>
yeah, it was actually one of our bigger goals
<whyrusleeping>
so thats pretty helpful of ya
<geoah>
whyrusleeping: travis is failing with go: release
<geoah>
The command "GIMME_OUTPUT=$(gimme release) && eval "$GIMME_OUTPUT"" failed and exited with 1 during .
<whyrusleeping>
yeah, don't use release
<geoah>
tip and 1.7 or just 1.7?
<whyrusleeping>
just 1.7 and tip if you want
zorglub27 has joined #ipfs
<ianopolous>
does anyone know if the videos from the lisbon workshop have been posted?
<geoah>
whyrusleeping: loggable should be fine now
<dignifiedquire>
hmm I think the node creation is skewing this drastically
apiarian has joined #ipfs
<dignifiedquire>
but I think the first part where js-ipfs needs optimizing will be all crypto related things, as they do seem a bottleneck
<dignifiedquire>
e.g. hmac in node-forge is quite optimized but 100% javascript with no fallback to node.js/webcrypto
cubemonkey has joined #ipfs
apiarian has quit [Ping timeout: 250 seconds]
neurrowcat has joined #ipfs
<geoah>
whyrusleeping: omfg kad-dht is done <3 :P
cubemonkey has quit [Ping timeout: 258 seconds]
Oatmeal has joined #ipfs
gamemanj has joined #ipfs
clownpriest has joined #ipfs
<whyrusleeping>
:D
<whyrusleeping>
geoah: now we've got to quickly get it into go-ipfs
<whyrusleeping>
and then fixup ipns-pub
<clownpriest>
hey, is it possible to crawl the entire merkedag though an ipfs gateway?
<dignifiedquire>
geoah: 👏 👏
<whyrusleeping>
clownpriest: depends on what you mean by 'the entire merkledag'
<clownpriest>
i'm probably not understanding the real mechanics of the network yet, but is there some global dag refering to all the data being pinned on the network?
gorhgorh has quit [Quit: No Ping reply in 180 seconds.]
silotis has quit [Remote host closed the connection]
<whyrusleeping>
clownpriest: not exactly, think of it as a forest of merkletrees (dags)
cubemonkey has joined #ipfs
gorhgorh has joined #ipfs
<clownpriest>
whyrusleeping: i see, and those merkletrees are distinct objects on the network? with no interconnection between them?
silotis has joined #ipfs
<whyrusleeping>
They *can* have connections between them
<clownpriest>
right, by referencing other object/block keys?
<whyrusleeping>
yeah
<whyrusleeping>
if i add a directory on my computer into ipfs, its not going to reference any other random stuff around the world
<whyrusleeping>
and its unlikely that at that moment, anything else is referencing my new content
Oatmeal has quit [Ping timeout: 250 seconds]
<clownpriest>
and if someone on the network wanted to grab that data they would need to know the hash of that specific folder that was added?
<whyrusleeping>
yes
<clownpriest>
cool
<clownpriest>
i'm trying to build something to index the data on the network to make it searchable
<clownpriest>
so it looks like crawling is limited by how many hashes you're aware of a priori and other references to hashes made within those initial keys
Encrypt has quit [Ping timeout: 240 seconds]
<whyrusleeping>
clownpriest: if you want a view of all hashes a given node discovers you can do: ipfs log tail | jq '.event = "handleAddProvider" | .key'
<clownpriest>
very cool, that's helpful
<clownpriest>
and is there a way to scan through a whole bunch of peers? ideally all of them?
fleeky has joined #ipfs
gwillen has quit [Ping timeout: 240 seconds]
wavis has joined #ipfs
paulharris has quit [Ping timeout: 250 seconds]
<whyrusleeping>
nope, you cant access anything from other nodes like that
m3s has quit [Ping timeout: 276 seconds]
<clownpriest>
darn
<clownpriest>
i guess that's a good thing though
<clownpriest>
so when you say "given node", who are these given nodes?
<geoah>
clownpriest: your local node/s
<clownpriest>
oh
<clownpriest>
so there's really no way of finding content on the network that you don't already know exists? (either publicized by the node that's pinning it or your own node)
<gamemanj>
clownpriest: there is one way
<gamemanj>
but it's hit & miss
<gamemanj>
you do IPNS lookups on every node you can find
<clownpriest>
interesting, how does this end up missing? because nodes go offline?
<gamemanj>
well, yeah, because you have to find the nodes
<gamemanj>
there's no massive ledger of all the IPNS records in circulation
<gamemanj>
you have to find peer IDs, then do as many lookups as possible, record results, and then (optionally) pin it all so that you can't lose it
Encrypt has joined #ipfs
<geoah>
you could also probably sniff the dht, but due to the way distance is calculated you might need a significant number of dht nodes :P
<geoah>
also that would only show you the hashes that people are looking up
<gamemanj>
which means it's basically useless.
<geoah>
I'm still unclear on the use case so I'm justing thinking weird stuff :P
<gamemanj>
usecase: "How do I find data on IPFS...
paulharris has joined #ipfs
<clownpriest>
gamemanj: basically useless because of needing to pin everything? or because of the peer lookup hits and misses?
<gamemanj>
clownpriest: You don't need to pin everything, that's optional.
<gamemanj>
I was talking about geoah's "sniff the DHT"
<clownpriest>
geoah: i want to build an ipfs search engine
apiarian has joined #ipfs
<clownpriest>
gamemanj: i see
<gamemanj>
As for IPFS search, I do keep wondering if there's some way to log the IPFS blocks that go through your system
m3s has joined #ipfs
m3s has joined #ipfs
m3s has quit [Changing host]
<gamemanj>
that would allow getting object roots, and from there you could try to recover data
<clownpriest>
gamemanj: the objects that go through your own node are just those that you explicitly requested through, right?
<gamemanj>
clownpriest: I'm not quite sure? If it is, then that method's doomed
gwillen has joined #ipfs
<gamemanj>
But I'm pretty BitSwap just wouldn't work if there wasn't some bartering material to work with,
<clownpriest>
gamemanj: well, i might definitely be wrong, but I remember hearing that your own node doesn't just shuffle around data or store anything you dont explicitly ask for, for security (and other) reasons
<gamemanj>
and that bartering material has to come from somewhere...
<gamemanj>
Unless, like, the trading algorithm's setup such that a debt only has to be repaid if someone was asking for it in the first place, but it's still...
<geoah>
if i remember correctly from the kademlia dht paper, nodes actually advertise their hashes on the network
<geoah>
I don't remember 100% and I'm not sure if IPFS does the same
<geoah>
but it's worth a look imo
apiarian has quit [Ping timeout: 244 seconds]
<gamemanj>
geoah: I'm pretty sure there's a supposedly deprecated and impractical (aka everything I ever wanted from a program) tool that can make a list
<geoah>
still the XOR based distance is a nightmare, but again it depends on the table of nearest hashes each node keeps
<gamemanj>
"ipfs diag net"
<gamemanj>
it's the most amazing command ever
<clownpriest>
geoah: yeah definitely, going to look into that. though if those node ID's aren't available through the ipfs interface, then it seems like that's not info that's deemed suitable as being user facing
<geoah>
should I run it or will I bring down the network? :P
<gamemanj>
geoah: I don't think it will.
<gamemanj>
But who knows?
<geoah>
damn it tool a while but it replied
<JesseW>
whyrusleeping: btw, your jq command was wrong; here's a corrected one: ipfs log tail | jq 'select(.event == "handleAddProvider") | .key'
<clownpriest>
thank you JesseW
Oatmeal has joined #ipfs
<gamemanj>
geoah: oh, great, it talks about a "default timeout" with no override
<gamemanj>
that's annoying
<whyrusleeping>
JesseW: ooooh, is that why?
<gamemanj>
I'd be happy to wait if it meant I got a better report
<gamemanj>
also, the format's actually a graph
<gamemanj>
you did read that right
<clownpriest>
regardless of what's currently possible, search is a pretty fundamental ability that needs to be considered somehow. maybe even making changes to the ipfs spec to allow that ability?
<JesseW>
clownpriest: I think the idea is that search is layered on top, just like it is with the HTTP/HTML web
<Kubuxu>
JesseW: thanks docs PRs :p
<JesseW>
Kubuxu: glad to do it!
<gamemanj>
clownpriest: IIRC, the way things work is that if you ipfs add something, it doesn't get magically uploaded to everybody in comms range. hence, no search.
<clownpriest>
JesseW: yeah but it seems like the http/html graph is much more dense than the ipfs graph
<clownpriest>
hence more amenable to crawling
<gamemanj>
clownpriest: ipfs net diag > ipnsTrawlList
<JesseW>
clownpriest: oh? the http server graph is "all possible unicode characters" of close to arbitrary length -- that doesn't seem dense at all
<gamemanj>
JesseW: I think clownpriest is talking about the lack of hyperlinking
<JesseW>
and the IPv4 address space is, while feasible to enumerate (or at least it was) it's certainly not close to dense
<clownpriest>
gamemanj: yep
<gamemanj>
JesseW: like, how many IPFSsites do you know that link to other IPFSsites
<JesseW>
ah, I see
<gamemanj>
JesseW: thankfully, IPNS + "Let's Lookup Every Peer On The Network" makes up for it
<JesseW>
I think we're just early days yet
apiarian has joined #ipfs
<gamemanj>
I bet a better algorithm for peer listing could be worked out, even with shell scripts
<gamemanj>
like, pick a random peer and choose them as a bootstrap
<gamemanj>
so that way you get a connection graph local to them, thus potentially revealing further-out nodes
<JesseW>
one issue with additional links between arbitrary IPFS objects is that such links can only be one-directional.
<JesseW>
unlike DNS/HTML links
<clownpriest>
gamemanj: i tried the ipfs diag net command, looks like valuable info
<gamemanj>
as far as I can tell, the best regex is "ID Qm............................................"
<gamemanj>
make sure to use grep -
<gamemanj>
*-o
<gamemanj>
that ought to extract just the IDs
<clownpriest>
this is a graph of peer id's?
<gamemanj>
clownpriest: AFAIK, yep!
Encrypt has quit [Quit: Quitte]
<gamemanj>
At least, I'm on there.
<gamemanj>
For me, anyway.
<clownpriest>
lol cool
apiarian has quit [Ping timeout: 244 seconds]
<gamemanj>
Also, "QmarsUDwQ5ak34KU6xJbr2iMuatB7e69WfuSjEBG3v7zvu" shows up on there, so either it's definitely showing a LOT of the network, or I got lucky.
<geoah>
`ipfs diag net | grep -oE '[A-Za-z0-9]{46}'
<gamemanj>
geoah: Great! Though I'd suggest 44 and prefixing Qm.
<geoah>
awh true
<gamemanj>
That pretty much guarantees it.
<clownpriest>
so once I have a list of peer ID's, i can query their pinned content? or at least whatever is public (remember there being a way to add permissions so things are private)
<gamemanj>
whyrusleeping: uh, I'm not sure who's talking to who here
clownpriest has joined #ipfs
<whyrusleeping>
the one i sent earlier set the event field of every log message to "handleAddProvider"
<gamemanj>
did we talk about events?
<whyrusleeping>
oh, i think i misinterpreted the 'null address' reference
<gamemanj>
whyrusleeping: You were talking to clownpriest
<gamemanj>
about a jq command
<gamemanj>
so presumably...
<whyrusleeping>
ah
<whyrusleeping>
yeah
<whyrusleeping>
my bad
<whyrusleeping>
bleh
<gamemanj>
Anyway, clownpriest, if you want the scripts, I can give you them after I'm done cleaning them up
<gamemanj>
and adding a script to launch 8 instances at once like I had to do manually
<clownpriest>
that would be cool, definitely appreciate that. probably going to write everything from scratch eventually anyway, but good to have a guide
<gamemanj>
Well, you will have to rewrite it for some sort of progressive search or something,
<JesseW>
gamemanj: If you are referring to peerIDs that don't map to any /ipfs/ address -- that's just because the owner of that peer hasn't published anything. If you are just using the peer to *receive* data, or to publish specific objects, there's no need to publish anything attached to your peer ID...
<gamemanj>
the IPFS address to the right shows up a LOT
<whyrusleeping>
gamemanj: thats an empty directory
<whyrusleeping>
its the default value for ipns entries
<whyrusleeping>
the 'null space' address
<gamemanj>
whyrusleeping: except there's a distinction between "no entry" and "does not exist"
<gamemanj>
there's Error: Could not resolve name.
<gamemanj>
and then there's UNLLsPACC
<whyrusleeping>
Could not resolve name generally means the node is offline
<JesseW>
hm, intereesting
<whyrusleeping>
or somethings wrong in its republisher method
<gamemanj>
odd, they show up on the nodegraph
<whyrusleeping>
hrm... that is weird then
<gamemanj>
else they wouldn't have shown up on the scanner
<gamemanj>
Maybe I need to rescan
<whyrusleeping>
right
<whyrusleeping>
no, the scan is fine
<whyrusleeping>
ipns entries should be resolveable 24 hours after a node goes offline
<whyrusleeping>
unless there is severe network churn
<gamemanj>
does "severe network churn" mean "lots of joins/leaves" or "the result of a lot of people running ipfs diag net"?
<gamemanj>
Just wondering :)
<whyrusleeping>
joins/leaves
<JesseW>
Hm, it might be nice to mention the "magic" multi-hash QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn in the FAQ.
<gamemanj>
It's not really magic, but it is very widespread. And is rather recognizable, too...
<JesseW>
Something like: "You may see the multihash QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn often -- it represents an empty directory, so it will show up anywhere such a directory is referred to."
<clownpriest>
any chance irc uses port 5001? because i'm getting "Error: serveHTTPApi: manet.Listen(/ip4/127.0.0.1/tcp/5001) failed: listen tcp4 127.0.0.1:5001: bind: address already in use" when running "ipfs daemon"
<whyrusleeping>
yeah, its completely luck that the empty directory hashes out to 'null space'
<JesseW>
amusing, though
<whyrusleeping>
JesseW: that would be a nice thing to have in the docs somewhere
<gamemanj>
clownpriest: not a clue. Maybe try finding stuff listening on that port...
<clownpriest>
ipfs daemon was working on the raspberry pi on my local network, not my main computer now tho
<clownpriest>
netstat doesn't show anything....ugh
<gamemanj>
Apparently the ports are changable
<gamemanj>
you have to set some config options
<clownpriest>
nah, about to try that
<gamemanj>
also, I found a catpic
<gamemanj>
eh? scan files don't contain...
<gamemanj>
I'll probably have to rescan.
rendar has quit [Ping timeout: 276 seconds]
<gamemanj>
Well, at least it's not a several-hours operation.
<gamemanj>
apparently the pipe decided not to work today.
cubemonkey has quit [Read error: Connection reset by peer]
<gamemanj>
Beginning second attempt at scanning and this time results should be kept.
ygrek has joined #ipfs
<gamemanj>
Huh. Apparently this rescan was different to the previous.
<gamemanj>
Found out about "Qmarz". I need a guidebook on how to deal with evil dopplegangers...
<gamemanj>
On the other hand, what if they were there first? That would make me (or to be more precise, my Pi) the evil doppleganger!
dmr has joined #ipfs
sametsisartenep has quit [Quit: leaving]
<geoah>
whyrusleeping: I'm sure that I had removed the packages -- but to be fair it was kinda late
<geoah>
I actually removed them now and waiting for travis -- I had already replaced the deps with the external ones so if I didn't miss anything it should be ok.
<whyrusleeping>
lol
<whyrusleeping>
i'm a bit scared to look at the new dep tree
<whyrusleeping>
its gonna be quite a bit larger
<geoah>
travis says ok on the first build
<geoah>
yeah that's why I didn't paste it :P
<whyrusleeping>
once libp2p is more broken apart it will get better
<JesseW>
hm, what is the difference between localhost:5001 and localhost:8080?
<gamemanj>
JesseW: 5001 is the special "API" node, which you can send commands to.
<JesseW>
hm
<gamemanj>
It can also be used as a gateway, but only for "authorized" (by hash) content, unless you explicitly disable that (since the content gets access to, uh, your entire node)
<JesseW>
Ahhhhh, that makes sense then
<JesseW>
where is the list of authorized hashes?
<gamemanj>
Probably embedded somewhere in go-ipfs source.
<gamemanj>
It's definitely everything involved in the WebUI though.
<JesseW>
hm, interesting
warner` is now known as warner
<dignifiedquire>
daviddias: is the varint thing the only issue left for interop?
<gamemanj>
well, what could possibly go wrong *pins XKCD archive*
<gamemanj>
(I'm kidding, it's a bit big)
<JesseW>
hm, how big?
<gamemanj>
I don't have a precise measurement
<gamemanj>
but given that the "go back to beginning of XKCD" button exists
<JesseW>
hm, doesn't `ipfs object stat` give you that?
<gamemanj>
Oh, didn't check
<gamemanj>
but then again it would take a while to stat it
<JesseW>
I dunno if it actually does
<g215[m]>
it shouldn't
<g215[m]>
you can just get the size out of the first object
<clownpriest>
CumulativeSize: 173960019
<g215[m]>
if I calculated correctly, this should be 165MB
<JesseW>
hm, it'd be nice to add a --human option to the cli
<gamemanj>
wait, what ?
<gamemanj>
165MB?
<JesseW>
seems likely
<clownpriest>
yep
<gamemanj>
that's... pretty big
gchristensen has joined #ipfs
* gamemanj
goes to a console and spams Ctrl-C
<JesseW>
that seems small to me
<clownpriest>
downloading now lol, taking a while
<gchristensen>
Hi, how is the hash of a file generated?
<JesseW>
that's less than a 20th of a gigabyte
<geoah>
the whole xkcd archive? 165? o_O
<kpcyrd>
Indeed. I have objects that would take up 60GB if you pin them recursively
* geoah
is amazed
<gamemanj>
60GB?
<gamemanj>
how
<geoah>
is there a way to calculate the total size of your pinned stuff?
<kpcyrd>
gamemanj: full archlinux mirror
<gamemanj>
kpcyrd: AUR package to switch pacman to download-from-IPFS?
<kpcyrd>
gamemanj: currently defunct, the disks are very close to death and I couldn't keep it uptodate
<kpcyrd>
gamemanj: a friend of mine offered help to get patches into pacman for this, but you just have to `pacman -S go-ipfs`, configured it to autostart and add `http://localhost:8080/ipns/mirror.rxv.cc/...` into your /etc/pacman.d/mirrorlist
<kpcyrd>
^ don't do this, the mirror is currently broken
<gamemanj>
Well, I was mostly joking anyway :)
gchristensen has left #ipfs ["WeeChat 0.4.2"]
<gamemanj>
In other news, QmUA3NNaKVMRDBLZuwLSD4ADDrEcekkcgRG1xUMwXEi87Z contains my stuff on IPNS scanning.
wavis has quit [Quit: Connection closed for inactivity]
<kpcyrd>
gamemanj: point is, there are no patches needed, it's a config line away and you're ready to go. I suggest downloading from your favorite gateway tho, had performance issues on my laptop and pacman discarded it as too slow
<kpcyrd>
I'm expecting it to get better after more people use it and the objects are properly distributed
<gamemanj>
kpcyrd: Of course, keeping the mirror up to date with all these upda---*Systemd-4957-beta-gamma-indigo has been released!*tes could be a bit of a pai*gnome-shell v732 has been released!*n to manage.
<gamemanj>
Especially with package renami**Replace libdbus with core/dbus?**
<gamemanj>
ng and things like that, which mess up an**Replace letsencrypt with community/certbot?**y chance at trying to use diffs.
<JesseW>
gamemanj: btw, doesn't seem to be available yet localhost:8080/ipfs/QmUA3NNaKVMRDBLZuwLSD4ADDrEcekkcgRG1xUMwXEi87Z
<gamemanj>
JesseW: The script only checks for the IPNS records, not the content in them.
<gamemanj>
Wait... oh
<gamemanj>
that
<gamemanj>
Odd, it should be available
<gamemanj>
It's Available Locally After All (tm)
<JesseW>
:-P
<gamemanj>
ok, the official ipfs.io gateway sees it
<JesseW>
and it just came up
<gamemanj>
Also, note that since ipfs name resolve is a bit slow, but quite parallelizable, it makes 8 processes.
<dignifiedquire>
whyrusleeping: is secio using 4byte lp only in the etm stream, or also in the header?
<clownpriest>
gamemanj: thanks for the scripts
JesseW has quit [Ping timeout: 260 seconds]
pfraze has quit [Remote host closed the connection]
espadrine_ has joined #ipfs
lawliet has joined #ipfs
ppham has joined #ipfs
ygrek has quit [Ping timeout: 250 seconds]
espadrine_ has quit [Ping timeout: 260 seconds]
opal has quit [Excess Flood]
opal has joined #ipfs
taaem has quit [Ping timeout: 276 seconds]
computerfreak has quit [Quit: Leaving.]
<kpcyrd>
gamemanj: it's not really that hard. I think I rsync my mirror every hour and use a script so `ipfs add -r` finishes in less than an hour. Republish to ipns, done. mirror-wise, it's only files and I don't have to care about libdbus getting moved to core/dbus :)
mildred has joined #ipfs
<kpcyrd>
archlinux was actually easier than debian, which I never finished
<kpcyrd>
also because that would mean using most of my servers disk for the mirror project
Foxcool has quit [Ping timeout: 252 seconds]
gamemanj has quit [Ping timeout: 260 seconds]
<kpcyrd>
and the "this file may change, this file doesn't change" rules are way harder to write for debian
<kpcyrd>
(^ the reason I can `ipfs add` 60 GB on slow spinning disks in less than an hour)
<clownpriest>
whyrusleeping: hey, it doesn't look like there's a function in go-ipfs-api for the diag command, would you accept a pull request for it? just wrote one for diag net
lawliet has quit [Ping timeout: 244 seconds]
Oatmeal has quit [Ping timeout: 244 seconds]
computerfreak has joined #ipfs
Encrypt has quit [Quit: Sleeping time!]
mildred has quit [Ping timeout: 265 seconds]
ppham has quit [Remote host closed the connection]
ppham has joined #ipfs
Aeon has quit [Remote host closed the connection]
ppham has quit [Ping timeout: 255 seconds]
Aeon has joined #ipfs
Aeon has joined #ipfs
Aeon has quit [Changing host]
r04r is now known as zz_r04r
Oatmeal has joined #ipfs
ckwaldon has quit [Ping timeout: 250 seconds]
PseudoNoob[m] has joined #ipfs
ckwaldon has joined #ipfs
jemc has quit [Ping timeout: 260 seconds]
ckwaldon1 has joined #ipfs
ckwaldon has quit [Ping timeout: 244 seconds]
ckwaldon1 is now known as ckwaldon
espadrine has quit [Ping timeout: 260 seconds]
ckwaldon1 has joined #ipfs
ckwaldon has quit [Ping timeout: 240 seconds]
ckwaldon1 has quit [Ping timeout: 276 seconds]
wuch has quit [Quit: = ""]
ecloud has quit [Ping timeout: 240 seconds]
ckwaldon has joined #ipfs
Oatmeal has quit [Ping timeout: 240 seconds]
jemc has joined #ipfs
PseudoNoob has quit [Remote host closed the connection]
wallacoloo has joined #ipfs
ckwaldon has quit [Ping timeout: 244 seconds]
ecloud has joined #ipfs
ckwaldon has joined #ipfs
wallacoloo has quit [Remote host closed the connection]
wallacoloo has joined #ipfs
matoro has quit [Ping timeout: 252 seconds]
JesseW has joined #ipfs
rendar has quit [Quit: std::lower_bound + std::less_equal *works* with a vector without duplicates!]
apiarian has joined #ipfs
ckwaldon has quit [Ping timeout: 240 seconds]
Guest65883 has quit [Ping timeout: 260 seconds]
ianopolous has quit [Remote host closed the connection]