<achin>
(i feel like "cryptocurrency" is distracting me from figuring out what bitcoin *really is*)
domanic has quit [Ping timeout: 250 seconds]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
<whyrusleeping>
jbenet: i pushed a better makefile for udt
<whyrusleeping>
it runs on both of my operating systems
<jbenet>
whyrusleeping just reviews
<jbenet>
d*
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
pfraze_ has quit [Ping timeout: 246 seconds]
akhavr has quit [Read error: Connection reset by peer]
pfraze has joined #ipfs
akhavr has joined #ipfs
ikreymer has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ikreymer has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
nicolagreco has joined #ipfs
akhavr has joined #ipfs
nicolagreco has quit [Client Quit]
amstocker has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
<ikreymer>
hello, i am very new to ipfs, but am very interested in its applications, particularly in the field of web archiving, where i specialize. first, is this the right place for questions or would somewhere else be better? email/github/etc...
<jbenet>
second, you should talk with davidar, "The Archivist" of IPFS he's in australia so TZ overlap may be off, but he's very active both here and on github
wopi has quit [Read error: Connection reset by peer]
wopi has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
<jbenet>
ikreymer: definitely open an issue in /archives or /notes about whatever you want to help archive
fazo has joined #ipfs
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ikreymer>
great, thank you.. i am interested in generic web archiving, seems like /archives may be most appropriate. i see there's already an issue regarding memento protocol there
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
<jbenet>
ikreymer would be fantastic to get some help implementing all this stuff.
<jbenet>
particularly with people familiar with the protocols
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
amstocker has quit [Ping timeout: 246 seconds]
akhavr has quit [Read error: Connection reset by peer]
<ikreymer>
well a bit more background, i am working on a project called, https://webrecorder.io (there's also a newer version that allows sign-up), which allows users to record and then replay any web site. it produces a WARC file (standard in web archiving, same as what IA uses).
<ikreymer>
currently users, can just download this as a single file, which contains all HTTP request and response traffic.. and i am contemplating what it would mean if instead users could put the data into ipfs
<whyrusleeping>
TIL: chromium doesnt use branches
akhavr has joined #ipfs
<ikreymer>
the idea is to allow anyone to create a web archive of any web site, but also allow users to keep the private/share, as needed.. and want to be very careful regarding security and privacy
<ikreymer>
i suppose i should make a github issue under /archives and take the conversation there :)
<jbenet>
ikreymer yeah. all sounds great.
<jbenet>
and we'd love to help
samiswellcool has quit [Quit: Connection closed for inactivity]
notduncansmith has joined #ipfs
HastaJun has quit [Ping timeout: 264 seconds]
notduncansmith has quit [Read error: Connection reset by peer]
<davidar>
Archivist checking in! :)
akhavr has quit [Remote host closed the connection]
akhavr has joined #ipfs
<davidar>
ikreymer (IRC): yes, very interested in archiving web pages/sites
<davidar>
webrecorder looks really cool
<spikebike>
is that intended for things like "watch my complex workflow to get XYZ done"?
<spikebike>
or watch how the site changes over a long period of time?
<ikreymer>
davidar: hi davidar, i am writing up an github issue now :) i see that this sort of relates to archives#7 also
<davidar>
ikreymer (IRC): cool. yeah, definitely, I'd really like people to be able to do "DYI internet archive" stuff
<ikreymer>
spikebike: it's more for being able to archive highly dynamic content, javascript, user interaction, etc.. and also create personal archives. the operation is similar to the wayback machine, but allows for on-demand capture and users can download/keep what they've recorded
<spikebike>
interesting
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
<davidar>
whyrusleeping (IRC): pong!
<davidar>
1 packet transmitted, average latency 5h
<ikreymer>
my current thinking is that storing the data, one WARC record at a time, would be the way to go.. now what i don't know is how best to resolve a URL + timestamp to the hash of the stored file. this is essentially what the memento protocol also provides (eg. answering this question is part of implementing Memento)
<whyrusleeping>
davidar: i dont really remember what i was going to ask you to be honest
akhavr has quit [Read error: Connection reset by peer]
* davidar
googles memento
akhavr has joined #ipfs
<davidar>
whyrusleeping, something about this being a dream? ;)
<whyrusleeping>
davidar: lol, it still might be
<whyrusleeping>
one thing i could tell you:
<whyrusleeping>
i updated my repobench program
<whyrusleeping>
which will be helpful moving towards improving add performance
<whyrusleeping>
take a look at the numbers it gives and let me know if there are any other metrics i should test
<whyrusleeping>
(since youre the one being bitten the hardest by add sucking)
<ikreymer>
davidar: memento protocol is basically just a standardized way to represent what the wayback machine does, given a timestamp and URL, return the closest known copy of that URL that this archive has.
akhavr has quit [Read error: Connection reset by peer]
<jbenet>
davidar ikreymer: whatever is done, it would be nice to get it in either unixfs or some other native datastructure that ipfs understands. that way the archives can be stored in an ipfs-seekable way
akhavr has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<davidar>
whyrusleeping (IRC): perhaps also add "flat directory with lots of small files created in a random order" to check if inode lookups are causing any slowdown
<davidar>
ikreymer (IRC): jbenet (IRC): cool, I'll have a read and try to work out how we could do something similar in ipfs
<fwippy>
my understanding is that the ipfs user inside the container has some UID; and permissions on the host use that same UID; whether or not that corresponds to a real user
akhavr has quit [Remote host closed the connection]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
simonv3 has quit [Quit: Connection closed for inactivity]
akhavr has joined #ipfs
joshbuddy has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
joshbuddy has quit [Client Quit]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
captain_morgan has quit [Ping timeout: 246 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
danslo has quit [Remote host closed the connection]
akhavr has quit [Read error: Connection reset by peer]
<jbenet>
fwippy: yeah. frankly, user perms inside a container dont make much sense to me. wish they got rid of users entirely
akhavr has joined #ipfs
<jbenet>
capabilities in the filesystem are much better (filepath is a capability, i.e. if you know the filepath you can mutate the thing. i.e. paths include encoded keys)
<jbenet>
warner: o/ have you tried doing this? i'm sure you have-- more like, does it work as nicely as i think it would?
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
fazo has quit [Quit: WeeChat 1.3]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
danslo has joined #ipfs
captain_morgan has joined #ipfs
joshbuddy has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
captain_morgan has quit [Ping timeout: 264 seconds]
dlight has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has joined #ipfs
pfraze_ has quit [Ping timeout: 256 seconds]
pfraze has joined #ipfs
pfraze has quit [Ping timeout: 246 seconds]
pfraze has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has quit [Ping timeout: 256 seconds]
pfraze has joined #ipfs
wopi has quit [Read error: Connection reset by peer]
wopi has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has quit [Ping timeout: 250 seconds]
voxelot has joined #ipfs
voxelot has joined #ipfs
groxx has quit [Quit: Connection closed for inactivity]
sseagull has quit [Quit: leaving]
pfraze_ has joined #ipfs
voxelot has quit []
pfraze has quit [Ping timeout: 256 seconds]
<jbenet>
whyrusleeping: have you pushed the changes to iptb? are they vendored? can start fixing the old tests if so
domanic has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze_ has quit [Ping timeout: 240 seconds]
<ipfsbot>
[go-ipfs] rht opened pull request #1756: (WIP) Move parts of `ipfs add` into lib core/coreunix (master...fix/add) http://git.io/vnx5Q
<achin>
what's the difference between partners in "ipfs bitswap stat" and "ipfs swarm peers" ?
<jbenet>
achin parters are peole you're getting (or have gotten recently) blocks from
<jbenet>
achin swarm peers are all peers you're directly connected to, (i.e. forming the dht with).
<achin>
makes sense, thanks
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has quit [Ping timeout: 240 seconds]
mburns_ has joined #ipfs
mburns_ has quit [Client Quit]
akhavr has quit [Remote host closed the connection]
<ipfsbot>
[ipfs] mburns opened pull request #98: Add Freenode Web IRC badge to README (master...freenode-irc-badge) http://git.io/vnxNy
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
ygrek has quit [Ping timeout: 265 seconds]
* zignig
has had too many beers.
HostFat has quit [Ping timeout: 240 seconds]
akhavr has quit [Remote host closed the connection]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
Tv` has quit [Quit: Connection closed for inactivity]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
reit has quit [Quit: Leaving]
ryepdx has quit [Ping timeout: 264 seconds]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
ryepdx has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
Qwertie has joined #ipfs
<Qwertie>
Hey everyone o/
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
Not_ has quit [Remote host closed the connection]
akhavr has quit [Read error: Connection reset by peer]
<jbenet>
ion: this is awesome, please keep these coming. we have stuff to work against!
akhavr has joined #ipfs
<Qwertie>
Does anyone know how to connect to neocities sites with ipfs?
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
<ion>
jbenet: Incidentally, this download grinded my ADSL connection to a halt (for other applications) due to bufferbloat but i’m hoping the switch to UTP will help. My IRC connection had a ping timeout solely because of it and my Android phone deemed the connection to be unusable and switched to the mobile network. :-P
akhavr has quit [Read error: Connection reset by peer]
<whyrusleeping>
ion, are you interested in working on that code?
akhavr has quit [Remote host closed the connection]
akhavr has joined #ipfs
<zignig>
whyrusleeping: justify your existence.
<ion>
whyrusleeping: I’ll need to learn go first and I also have hard time getting things done. But that would be interesting. I’ll ping you if I seem to have the energy to look at it some time.
joshbuddy has quit [Quit: joshbuddy]
od1n1 has quit [Ping timeout: 250 seconds]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
joshbuddy has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
joshbuddy has quit [Client Quit]
akhavr has joined #ipfs
ygrek has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
<whyrusleeping>
zignig, I am in no state to reason about existential crises
<pinbot>
now pinning /ipfs/QmfEuq2GyZkjGTr8ohLRjLG1QR39fHzaBR2fUTgKrXVf1m
od1n1 has joined #ipfs
<jbenet>
ion: wow that's rough. what's your typical BW ?
<jbenet>
(i downloaded that archive in <20m iirc)
akhavr has joined #ipfs
<pinbot>
[host 0] failed to pin /ipfs/QmfEuq2GyZkjGTr8ohLRjLG1QR39fHzaBR2fUTgKrXVf1m: Post http://[fc98:424c:b433:d7e2:7ee3:9541:73ff:2cdb]:5001/api/v0/pin/add?arg=/ipfs/QmfEuq2GyZkjGTr8ohLRjLG1QR39fHzaBR2fUTgKrXVf1m&encoding=json&stream-channels=true&r=true&: dial tcp [fc98:424c:b433:d7e2:7ee3:9541:73ff:2cdb]:5001: connection timed out
<pinbot>
[host 1] failed to pin /ipfs/QmfEuq2GyZkjGTr8ohLRjLG1QR39fHzaBR2fUTgKrXVf1m: Post http://[fce3:c53b:c3c5:2f54:8bb0:b6d9:898e:f140]:5001/api/v0/pin/add?arg=/ipfs/QmfEuq2GyZkjGTr8ohLRjLG1QR39fHzaBR2fUTgKrXVf1m&r=true&encoding=json&stream-channels=true&: dial tcp [fce3:c53b:c3c5:2f54:8bb0:b6d9:898e:f140]:5001: connection timed out
<pinbot>
now pinning /ipfs/QmfEuq2GyZkjGTr8ohLRjLG1QR39fHzaBR2fUTgKrXVf1m
<jbenet>
hmmm that's interesting-- they should work now o/
<pinbot>
[host 1] failed to grab refs for /ipfs/QmfEuq2GyZkjGTr8ohLRjLG1QR39fHzaBR2fUTgKrXVf1m: Post http://[fce3:c53b:c3c5:2f54:8bb0:b6d9:898e:f140]:5001/api/v0/refs?arg=/ipfs/QmfEuq2GyZkjGTr8ohLRjLG1QR39fHzaBR2fUTgKrXVf1m&encoding=json&stream-channels=true&r=true&: dial tcp [fce3:c53b:c3c5:2f54:8bb0:b6d9:898e:f140]:5001: connection timed out
akhavr has quit [Read error: Connection reset by peer]
akhavr has quit [Remote host closed the connection]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
qqueue has quit [Ping timeout: 240 seconds]
Guest73396 has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
<cryptix>
gmorning
akhavr has joined #ipfs
<whyrusleeping>
cryptix: g'marnin
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
joshbuddy has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
wopi has quit [Read error: Connection reset by peer]
wopi has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
ygrek has quit [Ping timeout: 272 seconds]
akhavr has joined #ipfs
qqueue has joined #ipfs
captain_morgan has joined #ipfs
<Guest73396>
good morning
akhavr has quit [Remote host closed the connection]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
bedeho has quit [Ping timeout: 240 seconds]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
<davidar>
mornin' guvna!
M-mistake has quit [Quit: node-irc says goodbye]
akhavr has quit [Read error: Connection reset by peer]
captain_morgan has quit [Ping timeout: 246 seconds]
akhavr has joined #ipfs
M-mistake has joined #ipfs
reit has quit [Quit: Bye]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
* whyrusleeping
is confused and decides he might need to sleep
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
<davidar>
whyrusleeping (IRC): which level of inception will that be now?
qqueue has quit [Ping timeout: 255 seconds]
wopi has quit [Read error: Connection reset by peer]
akhavr has quit [Read error: Connection reset by peer]
wopi has joined #ipfs
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
<dawuud>
jbenet: i'm looking at go-multiaddr-net now and trying to think carefully about how to give it a pluggable transport system...
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
<dawuud>
in particular it's clear that the Dialer in net.go must be more modular...
akhavr has quit [Quit: akhavr]
Hybla has quit [Ping timeout: 268 seconds]
Encrypt has joined #ipfs
qqueue has joined #ipfs
<ion>
Re: https://github.com/ipfs/go-ipfs/issues/1754 “0.3.8 changelog WIP”, wouldn’t it more convenient for each commit to include a changelog update if appropriate (under an “unreleased” heading) instead of scanning the commits manually to generate a changelog later?
reit has joined #ipfs
qqueue has quit [Ping timeout: 240 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ikreymer has quit [Remote host closed the connection]
Guest73396 has quit [Ping timeout: 268 seconds]
joshbuddy has quit [Quit: joshbuddy]
akhavr has joined #ipfs
qqueue has joined #ipfs
captain_morgan has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
qqueue has quit [Ping timeout: 250 seconds]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
qqueue has joined #ipfs
samiswellcool has joined #ipfs
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
rachitome has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
<drathir>
mornin...
akhavr has quit [Read error: Connection reset by peer]
Guest73396 has joined #ipfs
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
rachitome has quit [Remote host closed the connection]
akhavr has joined #ipfs
jhulten has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
jhulten has quit [Ping timeout: 240 seconds]
Guest73396 has quit [Ping timeout: 246 seconds]
Guest73396 has joined #ipfs
wopi has quit [Read error: Connection reset by peer]
wopi has joined #ipfs
xp3ll3d has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
<xp3ll3d>
just finsihed watching the 4 hours of IPFS usergroup presentations by Juan. I'm excited by it *BUT* I am a windows/.Net developer. Is there any word word on a .Net assembly that can hook into IPFS?
akhavr has joined #ipfs
<xp3ll3d>
I checked on nuget.org but there aren't any .net assemblies there. A google search was bring up too many other results
sericiculturist has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
null_radix has quit [Read error: Connection reset by peer]
null_radix has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
screensaver has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
screensaver has quit [Remote host closed the connection]
akhavr has joined #ipfs
screensaver has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
qqueue has quit [Ping timeout: 252 seconds]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
wopi has quit [Read error: Connection reset by peer]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
wopi has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
<mappum>
xp3ll3d: i don't believe there has been any IPFS .net development yet. but the go-ipfs daemon has a HTTP API and it would be possible to make a client library. for an example, see the JS client implementation: https://github.com/ipfs/node-ipfs-api
<xp3ll3d>
Thanks for the response. Was hoping to embed it in some .Net apps (Win10 IoT, Windows Phone, etc). Could have made for some cool tech demos (I've got a file on my desktop that I want to share with my WinPhone)
<xp3ll3d>
But the browserfied version of nodejs version may be embeddable
<mappum>
yep, it will be awesome to have browserified node-ipfs on mobile :D
infinity0 has quit [Remote host closed the connection]
qqueue has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<xp3ll3d>
ahh figured it out. I searched through the source code. Discovered that the environment variable has been deprecated. CORS is configured in the config file now
Guest18452 has quit [Ping timeout: 240 seconds]
Guest73396 has quit [Read error: Connection reset by peer]
pinbot has quit [Ping timeout: 244 seconds]
Guest73396 has joined #ipfs
pinbot has joined #ipfs
Guest73396 has quit [Read error: Connection reset by peer]
Guest73396 has joined #ipfs
Guest73396 has quit [Read error: Connection reset by peer]
akhavr has quit [Remote host closed the connection]
akhavr has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Guest73396 has joined #ipfs
<blame>
so, if we include a DNS proxy/man-in-middle server with ipfs, that users could set as a dns provider. What tricks could we do with it?
<blame>
the immediately apparent ones are that we could re-direct websites with the ipfs txt records to the local gateway
akhavr has quit [Remote host closed the connection]
akhavr has joined #ipfs
<fazo>
I updated my little ipns mirroring daemon, now it works and it's published on npm
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
notduncansmith has joined #ipfs
<fazo>
it's called ipfs-sync for anyone that wants to try it out
notduncansmith has quit [Read error: Connection reset by peer]
<ion>
fazo: cool
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
pfraze has joined #ipfs
<blame>
davidar: I'm missing a step here. That is interesting but I am not sure why you sent it to me
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
pau_ramon has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
bedeho has joined #ipfs
<davidar>
Blame (IRC): cool stuff you could do with a proxy, transparently mirror pages to ipfs
<blame>
You could only mitm the DNS requests, so ti would only tell you what websites the user went to. Unless you mitm and replace it with a proxy that does all of that.
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
Guest18452 has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
Guest18452 has quit [Read error: Connection reset by peer]
pfraze has quit [Ping timeout: 246 seconds]
pfraze has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
silotis has quit [Remote host closed the connection]
atrapado has joined #ipfs
pfraze has quit [Ping timeout: 240 seconds]
pfraze has joined #ipfs
<ion>
Blame: https would have problems with that.
akhavr has quit [Read error: Connection reset by peer]
pfraze has quit [Ping timeout: 256 seconds]
<ion>
Also every protocol which doesn’t include the hostname in the query.
silotis has joined #ipfs
<ion>
and/or handshake
akhavr has joined #ipfs
<blame>
why would it? The proxy could deal with https->http downgrade (not a good idea but it could)
<blame>
dns is effectively unsecured and we can wrap any attempt at securing it behind a proxy. It is why DNS poisoning is so effective.
pfraze has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
pfraze has quit [Ping timeout: 256 seconds]
ikreymer has joined #ipfs
pfraze has joined #ipfs
<blame>
Essentially, without adding on a proxy I don't think we want. The only use case is a domain-name -> ipns static http content mapping. We would also need to ensure that non-http related DNS did not get that bounce
akhavr has quit [Read error: Connection reset by peer]
<blame>
which is effectively impossible
akhavr has joined #ipfs
jhulten has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
ygrek has joined #ipfs
akhavr has joined #ipfs
jhulten has quit [Ping timeout: 272 seconds]
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
* blame
files under bad ideas
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
wopi has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
wopi has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
pfraze has quit [Ping timeout: 264 seconds]
pfraze has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
pfraze has quit [Ping timeout: 244 seconds]
akhavr has quit [Read error: Connection reset by peer]
pfraze has joined #ipfs
domanic has joined #ipfs
akhavr has joined #ipfs
<ion>
Whether we’re talking about IPFS or not, an actual HTTP proxy is always better than modifying DNS and MitMing HTTP. For instance, there are bound to be HTTP requests the proxy is unable to handle and when the client *knows* it’s talking to a proxy, it can simply retry with a direct connection.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ion>
And again, modifying DNS breaks just about every other protocol.
Algolkm_ has joined #ipfs
Algolkm_ has quit [Client Quit]
Quiark has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ebarch has quit [Quit: Gone]
pfraze has quit [Remote host closed the connection]
Guest18452 has joined #ipfs
<ion>
An HTTP proxy that caches things within IPFS is a great idea, just make it the right way to reap the benefits without breaking things. :-)
ygrek_ has joined #ipfs
ebarch has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Spinnaker has joined #ipfs
ygrek has quit [Ping timeout: 250 seconds]
jhulten has joined #ipfs
ygrek_ has quit [Remote host closed the connection]
ygrek_ has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pau_ramon has quit [Ping timeout: 246 seconds]
pfraze has joined #ipfs
Encrypt has joined #ipfs
rlindsgaard has joined #ipfs
xelra has quit [Ping timeout: 256 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic has quit [Ping timeout: 250 seconds]
<rlindsgaard>
hey, I just read the paper and have a couple of questions regarding the network design
crossdiver has joined #ipfs
<rlindsgaard>
isn't there a theoretical possibility that a certain object simply vanishes
<ion>
It’s a very real possibility.
<achin>
sure. that happens all the time when the only node that has the object goes offline
<clever>
or if that node chooses to delete his final copy
<ion>
I have personally destroyed objects in IPFS!
<rlindsgaard>
ah, so when I add a file to ipfs it does not immediately begin copying it to another place!?
<clever>
nope, nodes only ever store files that the user has manualy asked for (or seperate program within the system)
<ion>
rlindsgaard: That’s correct. Anyone loading the content will serve it until it’s garbage collected from their cache. They can choose to pin it which prevents it from being GC’d.
<clever>
so if i upload some home movie to my server and dont tell anybody the hash, it never gets shared with anyone
<rlindsgaard>
not much perma in that i recon :P
<ion>
Anything people want to keep around is permanent independent of whether the initial publisher keeps providing it.
<achin>
well, if the community recognizes that some important content exists on IPFS, maybe people can pin it. this protects against the original uploading from deleting their copy, effectively making it permanent
<rlindsgaard>
sure, it's not a bad idea at all
<clever>
for some content like pictures and video, there will always be idiots taking a screenshot of a picture, causing dups
<ion>
If IPFS gains traction, there are bound to be entities like archive.org, archive.is, nyud.net and Google’s cached pages that keep web content around.
<clever>
and some sites like facebook re-compress pictures on upload, so even if you save it right, it wont align in IPFS
<achin>
with HTTP, if i take a mirror of somesite.com, and somesite.com does down, you really have no way to find my mirror, unless i make some announcement that you'll see. this isn't the case with IPFS -- you'll automatically be able to find my mirror
<clever>
ion: one problem i can see with things like archive.org, how do you map a url back to an objectid?
<achin>
(the term "mirror" isn't exactly the right term to use for IPFS, but it's close enough for this example)
<clever>
there has to be a central database that knows the hash of every file
<ion>
clever: In IPFS the URL is the object ID.
<rlindsgaard>
I think that I got the impression from the presentation that nodes interactively communicate torrent like in order to make things permanent
<clever>
ion: i thought the objectid was just the hash of the file contents?
<ion>
clever: To be more precise, the canonical address is /ipfs/<hash>
<clever>
rlindsgaard: there is come communication like that, but no way to push data out and make it survive forever by force
<clever>
ion: so if i want to get the copy of example.com on july 2nd of 2014, how will i know what hash to query?
<achin>
clever: probably a database stored in IPFS itself. you'd find the database via IPNS
<rlindsgaard>
clever: yeah, that is actually quite nice if you want things to be ephemeral
<ion>
clever: Oh, historical content indexed by time is another issue. Yes, someone will need to generate and maintain such an index.
<clever>
ah, but only one group can maintain that database, and they would still have the authority to delete the pointers
<achin>
(or maybe you'd find the page via IPNS. unsure if it will be capable of something like that)
<clever>
the data itself cant be deleted, but finding its hash after they say 'we dont like this' could be difficult
warner has quit [Quit: ERC (IRC client for Emacs 24.5.1)]
<ion>
Except when people mirror the index itself.
od1n1 has quit [Ping timeout: 268 seconds]
<clever>
they would need to save every version of the ipns pointer
<clever>
which doesnt seem that hard
<clever>
and pin them
<achin>
ipfs doesn't really have a way to do distributed content generation
<rlindsgaard>
thanks for the clarification, it was really helpful :)
<clever>
ive also been wondering about how to do linux package distro, and you would basicaly need a single person managing an IPNS to map packages->hashes
<clever>
so its not that different from normal (an https server as the authority)
<clever>
just that the users would be able to cross-share, the parts once the package is in the wild
<clever>
which leads to the next issue, you have the entire package on your FS, but ipfs can only use the compressed form in its database, so a GC would still delete it
<achin>
what if the distro maintainer published the scripts they used to produce the packages, allowing community members to also publish stuff
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<clever>
achin: thats exactly how several distros work (nixos, gentoo), but the script wont always produce a binary duplicate
<ion>
clever: There’s ongoing work on that.
<clever>
and you wont know the hash of the compilers output until youve ran it
<clever>
ion: have you heard much about how nixos works?
<achin>
if the original maintainer goes away, there are already community members who can produce the necessary packages and provide them over IPFS (assuming the build process produces identical results, which seems like a solvable problem)
<ion>
clever: I have read about it but I haven’t used it yet.
warner has joined #ipfs
<clever>
achin: in nixos, the packages are still signed by the source, because it would be easy to slip something extra in at compile time and nobody would know until they also compiled a copy
<clever>
ion: that hash covers every other package it depends on (libs, and the compile scripts)
<clever>
so if you alter the dependancies in any way, it creates a new version
<ion>
clever: Right
<clever>
and the entire store is read-only
<achin>
if you had enough community members building the packages, maybe you wouldn't need signatures? just verify that community members produce the same hash. this assumes that an attack is not likely to be made on every builder
<clever>
ion: nixos uses .nar files to package things, its basicaly just tar without the timestamps, non-reproducable fields
<clever>
in theory, you could create a custom filesystem, that keeps everything in .nar format, and unpacks the packages on the fly
<clever>
you could then tie that into ipfs, and either share the bare .nar files with ipfs (better performance)
<clever>
or query the local ipfs for a .nar on demand
<clever>
then you are always seeding stuff you have, you cant GC something and still have it installed
<clever>
without that custom FS, i would have 2 copies of every package, in the ipfs database, and unpacked in /nix/store/
<clever>
and id be more likely to GC the copy that helps the community
<ion>
There has been talk about giving the daemon the ability to serve contents from the filesystem without having to have an extra copy in ~/.ipfs/blocks.
<clever>
that would help massively, then the ipfs node can be offline and the os still work
<clever>
so you would just need a fuse layer that can map the .nar files into /nix/store/
<Xe>
oh man
<Xe>
are you guys looking at putting nix into ipfs?
fleeky has joined #ipfs
<clever>
Xe: just thinking about how it could be done
<Xe>
that would be killer
<achin>
yeah, there are a lot of neat things you might be able to do with software distribution
<clever>
it would need a few parts, a fuse layer to allow processing unpacked .nar files as if they existed in /nix/store, some modifications to nix to skip unpacking entirely, and some changes to make it fetch the .nar's via a binary cache on ipns
<clever>
and for install time, it would need to allow uncompressed writing to /nix/store
apophis has joined #ipfs
<whyrusleeping>
gmorning everyone
<M-hash>
are .nix scripts static config or a turing complete language?
<whyrusleeping>
M-hash: its a turning complete functional language
<ion>
good ****ing
<clever>
M-hash: and the package manager will also hash the death out of nearly every input
<clever>
so if you modify the script in any way that can impact the package, it changes the output path
<clever>
so it wont be used by people who want an un-modified version
<achin>
i know there is lots of talk about web apps on IPFS. but you could even distribute/run regular apps over IPFS. for example, to run ion's bitswap stat script, simply: sh /ipfs/QmduKdXx6E1WTGmf867wEDf7anr49ysC4GeFbYYsiuB3gk
<whyrusleeping>
achin: yeap!
<whyrusleeping>
we're looking at the possiblity of adding an executable flag to files too
<achin>
neato
<clever>
if/when ipfs no longer needs files in ~/.ipfs/blocks, i'm guessing that there would need to be an agreement between ipfs and the other apps, to never modify a file that has been added
wiretapped-cb has quit [Read error: Connection reset by peer]
<ion>
achin: IIRC jbenet said in a talk he’d like to have a universal programming language in IPFS where an object can be defined as the code that generates it. And code (in a pure, total language) can generate binaries performing arbitrary IO so running the code can be 100 % safe until you actually run the generated binary.
<clever>
though it could just be as simple as ~/.ipfs/shared_files/ and any app choosing to use that folder must never modify a file
<clever>
including for file creating, create the file elsewhere, then use rename() to atomicly move it in
<achin>
once that is done, i could do something like export PATH=/ipns/${ME}/bin:${PATH} to have my custom utilites available on all my machines
<achin>
(so +1 for adding an execbit, whyrusleeping :D )
<clever>
achin: i'm sure you could modify the fuse layer to just claim +x on every file for now
<ion>
clever: Modifying files isn’t that much of a problem if you accept that any content added that way can disappear at any moment. The daemon just needs to keep track of the tree using inotify or equivalent and check the hash of what it read from the filesystem before sending it to others.
<clever>
achin: thats how fat32 does things, heh
<clever>
ion: ah yeah forgot about inotify, but the re-hashing on read will be a bit expensive
<achin>
clever: but i don't want everyone else to have to hack their version of ipfs
<clever>
achin: yeah, thats still an issue
<clever>
ion: about the only change i can see ipfs needing, is an api to tell it to save certain files to ~/.ipfs/shared_files under certain names, and then nix can just do everything else on its own
<clever>
and when nix generates files localy, it can just save them there
<ion>
clever: After hashing a file once, it could trust its contents until inotify tells someone’s writing to it.
<clever>
the ipns stuff could be done entirely with the existing api
<clever>
yeah, but it may have to rehash things on each reboot, the first time somebody asks for it
<clever>
and keep a db of hashes -> filenames
<ion>
clever: It could cache the file list along with hashes and mtimes and assume that anyone doesn’t change a file without changing the mtime while it’s not running.
<ion>
s/list/tree/
<multivac>
ion meant to say: clever: It could cache the file tree along with hashes and mtimes and assume that anyone doesn’t change a file without changing the mtime while it’s not running.
<ion>
multivac: THANK YOU
<clever>
the main value of ~/.ipfs/shared_files/ is that you dont need a full ipfs node in your initrd, just to boot the system
<clever>
it can run offline, purely from the files in the dir
<clever>
though that now brings up a new crazy idea, a small usb stick, that contains kernel+initrd+ipfs node
<ion>
s/(((((((.|.+)+)+)+)+)+)+)+/:-)/
<clever>
and downloads the entire os on bootup
<clever>
just have to pin the init script, the dependencies in nixos will cover the whole os from that 1 point
<achin>
ion is in ur bot, haxing ur regexs
xelra has joined #ipfs
<clever>
the ~/.ipfs/shared_files would also be exempt from ipfs gc
<clever>
that would have to be managed entirely by nix
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
joshbuddy has joined #ipfs
<clever>
ion: hmm, which hashing algo is ipfs using?, any prefixing like git?
<clever>
its just a tad longer then sha256
<clever>
oh, but it does contain the 256 hash
<clever>
bingo, an extra 4 characters at the start of the filename, 1220 + sha256(contents)
pfraze has quit [Remote host closed the connection]
<clever>
ion: any idea what that prefix is?
<M-hash>
multihash
<clever>
heh, right from the name itself!
jatb has joined #ipfs
<clever>
M-hash: but what is the second hashing algo?
<achin>
note that the hash produced by "ipfs add" isn't exactly the hash of the contents of the file on disk. there are some additional unixfs framing bytes
joshbuddy has quit [Quit: joshbuddy]
<clever>
that would allow room for things like +x
<clever>
is there a link with some docs explaining that?
<achin>
i'm not sure what data they encode, but you can view them by pipping `ipfs object data QmZULkCELmmk5XNfCgTnCyFgAVxBRBXyDHGGMVoLFLiXEN` into your favorite hexeditor
<clever>
nixos is also a bit cryptic about how it encodes everything into its hashes
<clever>
yesterday i changed the initrd build script from "ip addr add 192.168.2.30
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
akhavr has quit [Read error: Connection reset by peer]
<achin>
for fun, here is a block that uses a different hash: 8VuZG4KeAu6jCSwQDXbZBfze9KzKS7UB1daTSmgXiw13QLMuFz8YQN3MrpWu9TmQs4bSqDu7wTSMFfWd2JCz2VhaEM
akhavr has joined #ipfs
<achin>
i have no idea if go-ipfs will do the right thing with that
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
<clever>
achin: Error: context deadline exceeded
<achin>
bummer
akhavr has quit [Read error: Connection reset by peer]
<ion>
It’s perpetually in my wantlist.
akhavr has joined #ipfs
<achin>
i can "ipfs get" that thing, but maybe my daemon doesn't know i have it?
<achin>
i assumed that it just looked in ~/.ipfs/blocks but maybe not
akhavr has quit [Read error: Connection reset by peer]
<rschulman>
if you have the item in your local store, it just returns it from there, yes
<rschulman>
But i also get a context deadline exceeded on it
<clever>
64 bytes from 8.8.8.8: icmp_seq=3 ttl=53 time=3512 ms
<achin>
hm, i ahve the block in my own wantlist. weird
<clever>
i think ipfs was to blame, i killed the daemon and it recovered
akhavr has joined #ipfs
<achin>
s/block/node/
<multivac>
achin meant to say: hm, i ahve the node in my own wantlist. weird
<ion>
clever: It’s because of bufferbloat.
<whyrusleeping>
this is quite the bizarre experiment
<clever>
ion: it was even causing irc to get packet loss on a different machine, and causing dns to fail
<ion>
clever: The switch from TCP to UDT will probably help, IIRC UDT tries not to fill the router buffers.
* blame
has his daily existential crisis about choosing a handle that is not a commonly used English word...
<whyrusleeping>
bufferbloat?
<clever>
ion: what was it even sending that it managed to clog up the pipes so much?
<achin>
i'm not sure how to tell bitswap that i actually own this thing
<whyrusleeping>
mmmm, if you guys are seeing bufferbloat, i'm supposed to email someone about it
<clever>
let me restart the daemon and see if it returns
<ion>
clever: I had the same thing when doing this benchmark <https://github.com/ipfs/go-ipfs/issues/1750#issuecomment-143402049>. My IRC connection timeouted once and my Android phone decided the connection through my WLAN is unusable and switched to the mobile network.
<clever>
chrome kept telling me the dns server was down, and i have a direct gigabit link to it, lol
<clever>
and when did did work, the reddit style sheet vanished
Tristitia has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
<rschulman>
clever: When are you talking about? trying to request that hash that achin posted?
notduncansmith has quit [Read error: Connection reset by peer]
<rschulman>
Because when I did that I saw my own network access slow way down too.
<ion>
whyrusleeping: Email whom? Every router manufacturer in the world?
<clever>
its even returned after restarting the ipfs daemon
<clever>
rschulman: i only tried to request it once, and the problem persisted long after the error msg
<whyrusleeping>
nope, this guy we talked to who apparently wrote the patch to fix bufferbloat in openwrt
<whyrusleeping>
or something like that
<rschulman>
clever: Yes, same here. I'm still getting lag.
<clever>
rschulman: let me check wireshark and see what the heck ipfs is doing
<clever>
i have 180mbit download, so its pretty hard to clog up this link
<ion>
whyrusleeping: Alas, you can’t fix bufferbloat by fixing only one of the devices on the route.
<clever>
dang, the problem has gone away now that i got wireshark open!
<ion>
clever: Try running ipfs repo gc and requesting whatever started it again.
<clever>
yep, theres the lag
<clever>
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
<clever>
tcp 0 1148 192.168.2.22:4001 174.103.185.41:17485 ESTABLISHED 16491/ipfs
<clever>
and a crap-ton of 608's
<clever>
it looks like it fired the same request out of 20-30 different tcp connections at once
<clever>
tcp 0 608 192.168.2.22:4001 192.99.81.39:4001 ESTABLISHED 16491/ipfs
<ion>
Trying to download a large flat directory full of small files seems to make it especially bad since the daemon adds all the blocks to the wantlist at once.
rendar has quit [Ping timeout: 240 seconds]
<ion>
Such as in the issue comment i linked.
<ion>
OTOH the case of a single large file is also not optimal because it lets the wantlist become empty periodically.
<ion>
(as seen earlier in the same issue)
Algolkm_ has joined #ipfs
<clever>
what i cant figure out, is why its spamming so many nodes with packets like this, thats not how a dht should work
<clever>
it should check between the peers it knows of,to find out which is nearest to the object hash
<clever>
then ask them
<clever>
and the dht should be spread out, in tiers, with each tier having more nodes, of a farther distance
<clever>
so keep the average number of hops equal
rendar has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
clever have you read the coral paper?
<whyrusleeping>
tldr, the system of tiered dhts you describe is what coral does
fazo has quit [Remote host closed the connection]
<clever>
the tiers i was mentioning are usualy called buckets
<clever>
slowly coming back to me
<clever>
reading the first pdf, and its looking like the basic problem bit torrent and the merkledag solves
<clever>
hashing at the block level, rather then the entire file
<whyrusleeping>
we are going to implement a similar system, but for now, we're just using kademlia
<whyrusleeping>
and kademlia is fairly spammy
<achin>
it seems like kademllia requires nodes and keys to exist in the same namespace
<whyrusleeping>
the other reason that we are spammy is that we arent disconnecting from peers as they are kicked from the kbucket routing table
<clever>
the system i'm remembering, just uses the same hash on the nodes, to generate a nodeid
<clever>
so all nodeid's and fileid's are hashes of the same length
<clever>
another key difference that ive noticed between ipfs and other systems, is that the nodeid in ipfs is constant
<whyrusleeping>
ipfs nodes and keys are in the same namespace
<achin>
they can be different lengths, though
<whyrusleeping>
yeah, but the dht rehashes everything
<achin>
ah
<clever>
so from what i understand so far, the DHT is storing hash -> protobufobject mappings, and that protobuf can encode a file, or directory
<clever>
and acording to this .proto file, it also has symlinks
<clever>
but i dont see how the merkledag breaks one file up into 2 objects yet
<achin>
the file/directory/symlink stuff is anothe rlayer on top of the core merkledag, and isn't required
Pierette has quit [Ping timeout: 240 seconds]
<clever>
ah
<clever>
so the unixfs.proto stuff is the whole file, before splitting it up?
<clever>
and merkledag is then how that one giant object gets stored as seperate objects?
<achin>
there are nodes, which contain zero or more links to other nodes, and zero or more bytes of opaque data
<clever>
and then it will just concat the opaque data in every node in the tree, in some set order
<clever>
to create the unixfs object?
<clever>
or whatever else happened to be stored in it?
intracoelomic has joined #ipfs
<achin>
i think it's something close to straight contatenation, but i'm not 100%
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<achin>
ah, so the top-level merkelnode is holds unixfs data, as well does everything it links to
<clever>
so if i had the hash of an ubuntu install cd, that hash would be the top level merklenode ?
<achin>
but the top-level unixfs object doesn't have any data. this might be what signals to ipfs to concat the unixfs data from the links
<achin>
you mean if you hash the .iso or something?
ygrek has joined #ipfs
<clever>
the ipfs id you obtain after doing 'ipfs add foo.iso'
<ion>
clever: Here’s a single file. Run ipfs ls /ipfs/QmTrRng9EWBWCLmpHVgKx3NDzHG8J8Y76Vuo21eYqKbyid and you’ll see it’s split into ~45-megabyte pieces. Run ipfs ls QmVQMuvDtoGh78KA8HeVCyjLTfhgicfquthKRUBviFLuqT (the first top-level piece) and you’ll see it’s split into ~250-kilobyte chunks. ipfs ls QmT5d5PVm7nzQ5wZvfEZ8yw2JufE5X3t5vaVwDhxwJXZ7D will list no links, that one is the first object with actual
<achin>
so both are unixfs objects that don't have any data
<achin>
but only have links to other nodes which contain unixfs data
<ion>
What happens if you have a unixfs object with both data and links?
<achin>
i don't know, let me create one
<clever>
yeah, similar to bitcoin transactions, the tree of transactions is either a pair of hashes, or a pair of transactions, or a single transaction
<ion>
It would be very reasonable if it concatenated the data followed by the links.
<clever>
heh, i managed to crash ipfs
<clever>
eip 0xb779c9e8
<ion>
Say, have the header of a file along with links to the rest.
<clever>
the wall of backtraces doesnt even fit in my scrollback buffer
<clever>
it may have been a segfault, but its not a normal error format
hannes3 has joined #ipfs
<clever>
i can see how you could implement seeking by being lazy, if you know the offset of each chunk that the top level names, you can skip directly to the part of the tree you want
<clever>
and not fetch the begining
<hannes3>
what about privacy in ipfs? who gets to see what i am accessing/looking for?
<achin>
your peers can see what blocks you want
<clever>
currently, your nodeid is constant, so everybody you download from has a life-long id to pair that to
nessence has quit [Remote host closed the connection]
<clever>
and they can obtain the ip for that nodeid,at any time
<clever>
so if you download something nasty from a honeypot, they can find your current ip every time you come online
<ion>
hannes3: If you need privacy, IPFS will be able to work on top of Tor for instance.
<clever>
what about packet sniffing, could a 3rd party monitor a link and figure out who is downloading what hashe?
<ion>
The communications are encrypted AFAIK.
<clever>
ah, so only the node i download from can pair file parts to my life-long id
<hannes3>
do my dht requests only go to the nearest peers or do they get relayed further? (no much of an idea how a dht works to be honest)
<clever>
i'm wondering the same thing
nessence has joined #ipfs
Encrypt has quit [Ping timeout: 264 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<achin>
ion: try to get QmUoVT1yVUzYeEEfyHnDBq9qg6aR8F1bMaNpg8uJjgaLFH
<hannes3>
i also wonder about latency, if i have a website with eg 10 files (html, js, images, css) how will latency stack up in a typical case (not cached close to me)?
nessence has quit [Remote host closed the connection]
<clever>
thats what http2 is meant to solve, pushing the js+images+css before you ask for them, but yeah, ipfs just makes that problem 10x worse
<ion>
achin: How big is it? It seems to be taking a while.
<achin>
a patched version of QmTrRng9EWBWCLmpHVgKx3NDzHG8J8Y76Vuo21eYqKbyid
<clever>
it seems like it would be usefull to allow getting the size of an object quickly
<achin>
i thought it would have been quick for you, since you had all of the data
<clever>
without having to traverse the entire merkledag
Encrypt has joined #ipfs
<achin>
anyway, you're right, ion. the data in the root unixfs is concatenated with the data from all the links
<ion>
achin: It’s just sitting in my wantlist perpetually.
<ion>
achin: Huh, it loaded immediately when i restarted the daemon.
<achin>
odd
<achin>
i wonder why ipfs decided on 45MB chunks
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
compleatang has quit [Quit: Leaving.]
akhavr has quit [Remote host closed the connection]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
joshbuddy has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
akhavr has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
jfis has quit [Quit: q]
AdamThePhantump has joined #ipfs
<AdamThePhantump>
hello inhabitants of planet IPFS. I am a ghost. And I have a problem with my node.
<achin>
ohi
<ion>
I’m afraid the current implementation only supports corporeal users, but pull requests are welcome.
<ion>
All the developers are alive and real, so the needs of dead or fictional users haven’t been given that much attention.
<AdamThePhantump>
So, when I turn off my node and and start it up again this is what I get: Error: Post http://127.0.0.1:5001/api/v0/version?enc=json&stream-channels=true: dial tcp 127.0.0.1:5001: connectex: No connection could be made because the targ et machine actively refused it.
<AdamThePhantump>
and I have to regen my keypair and everything, losing the files I've got uploaded.
<AdamThePhantump>
can anyone tell me how to have it save my key so it looks for that everytime I boot up my server?
<ion>
Your key is stored in ~/.ipfs/config
<ion>
What prints that error?
<AdamThePhantump>
when I quit and try to reinit the daemon.
<AdamThePhantump>
that gives the error.
<ion>
Reinit? Do you mean restart?
<AdamThePhantump>
and then I am forced to do ipfs init -f.
<AdamThePhantump>
otherwise it throws an error if I don't force it.
<ion>
You only need to run init once. You can start the daemon by running ipfs daemon.
<AdamThePhantump>
but my machine is one I need to shut off each day. it's my personal machine not a big fancy server.
<ion>
That doesn’t affect anything.
<AdamThePhantump>
so that means i have to init each time i start up this dinky old dell.
<ion>
No, just start the daemon.
legobanana has joined #ipfs
<AdamThePhantump>
but if i quit it first it complains that this machine refused it's requests to link.
<AdamThePhantump>
when i try to start the daemon
dignifiedquire has quit [Quit: dignifiedquire]
<ion>
What is the exact command you ran and the exact output?
<AdamThePhantump>
ipfs daemon and the output is what you see above. that's after i keyboard interrupted it just to tinker and experiment
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<AdamThePhantump>
ion this is the interrupt output: Received interrupt signal, shutting down... Received another interrupt before graceful shutdown, terminating...
<ion>
If “ipfs daemon” fails to start or work, please file an issue at <https://github.com/ipfs/go-ipfs/issues>, pasting the full input and output from your terminal to the report.
ygrek has quit [Ping timeout: 246 seconds]
<AdamThePhantump>
uh, yeah.... and wait a week or more to get a response that may or may not help. I'm good.
voxelot has joined #ipfs
<achin>
but you might get a helpful reply right away!
<ion>
Either you want to get the problem fixed or you don’t, providing the developers the knowledge about it will increase the likelihood of it getting fixed.
<AdamThePhantump>
it's a lottery!
<AdamThePhantump>
achin: :/ silly.
ygrek has joined #ipfs
<jbenet>
AdamThePhantump: we respond to issues usually within 24hr
<jbenet>
And, I believe this was fixed. Try updating your nose
<jbenet>
Node*
* ion
tries updating his nose
<AdamThePhantump>
updating my nose XD
fwippy has quit [Remote host closed the connection]
<AdamThePhantump>
jbenet: i tried using the update command and it says "ipfs update is disabled until we can deploy the binaries to you over ipfs itself"
<jbenet>
Yeah we can now, we just haven't built it
<jbenet>
whyrusleeping something's off gobuilder's not building master still?
<AdamThePhantump>
says: Last built 1 week ago using go version go1.5 linux/amd64 and 2771 seconds of time
<jbenet>
AdamThePhantump: I see you have no nose. This is beyond our testing parameters. YMMV
<jbenet>
AdamThePhantump: right it should be built a few hours ago
<jbenet>
From last merge
<jbenet>
I tried triggering a build, we'll see if it builds
<AdamThePhantump>
YMMV?
<jbenet>
Your mileage may vary
<AdamThePhantump>
in this case, your spirit may vary. cause i'm dead.
<AdamThePhantump>
should i leave my machine on and in sleep mode or similar until a new build is up? cause with this week old build it'll use the men in black memory erase pen on my rsa key making me overwrite it.
fazo has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<AdamThePhantump>
jbenet: are you the one that did the demo video?
<AdamThePhantump>
jbenet: gimme the hash for the cat on the web as you called it. :P
voxelot has quit [Ping timeout: 246 seconds]
<clever>
achin, ion: due to that website showing a bit too much info, i wrote a small script to parse the unixfs structure, http://pastebin.com/JPARCMPv
<clever>
i can now see that the first unixfs node contains just block sizes and the total size, no data, no merkle links
<jbenet>
Maybe later, am on mobile atm.
<achin>
there's a version of protobuf for javascript?
<AdamThePhantump>
ok jbenet
<clever>
achin: exactly what i used
<achin>
clever: neat. i assume it's pure JS?
<AdamThePhantump>
and jbenet, it worked! you fixed it!
<clever>
achin: nodejs, the chrome JS engine with some wrappers around posix functions
<clever>
achin: and c++ extensions
<clever>
achin: the script: http://pastebin.com/2kgNdzQx and to get the protobuf extension, 'npm install node-protobuf'
<achin>
clever: btw, you might as well just dump the data to ipfs instead of pastebin :)
<clever>
the daemon is off right now
<clever>
its DoS'ing the router, lol
<AdamThePhantump>
achin: what's your peer ID?
<AdamThePhantump>
clever: merkling merkles
<AdamThePhantump>
jbenet: can i access the webui of my node from my other machines or only on the machine running my node?
pfraze has quit [Remote host closed the connection]
pfraze has joined #ipfs
<ion>
By default it only listens on 127.0.0.1
<AdamThePhantump>
ion: but can i set it to listen on my public IP or at least the public IP of this machine?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ion>
AdamThePhantump: There's probably a setting for that. You probably don't want to do that, though.
voxelot has joined #ipfs
voxelot has quit [Changing host]
voxelot has joined #ipfs
captain_morgan has quit [Ping timeout: 246 seconds]
<AdamThePhantump>
ion: but I don't wanna have to remote or come back to this machine every time I wanna visit planet ipfs.
fwippy has joined #ipfs
<clever>
from the command line, how do i manipulate ipns?
rendar has quit []
<achin>
with the "ipfs name" command
<clever>
achin: and is there anything in place to allow managing multiple ipns nodes from one client?
<achin>
at teh moment you can only have 1 ipns name (your peer ID)
<achin>
there is work underway to design a new IPNS system, but i'm afraid i've not seen any details about how it will work
<clever>
so id need different ~/.ipfs and daemons
<AdamThePhantump>
clever: what's your ID? I wanna connect to you.
<achin>
right. you'll probably have to jump through some hoops, though, to run more than 1 daemon on a single machine
<clever>
dont know it right now, and my daemon is offline
<clever>
achin: ive got 12+ machines right now, heh
Encrypt has quit [Quit: Sleeping time!]
<clever>
only 2 have ipfs installed so far
<AdamThePhantump>
you mean 2 nodes.
atrapado has quit [Quit: Leaving]
apophis has quit [Quit: This computer has gone to sleep]
reit has quit [Quit: Leaving]
screensaver has quit [Remote host closed the connection]
apophis has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
reit has joined #ipfs
voxelot has quit [Ping timeout: 246 seconds]
<ion>
AdamThePhantump: Exposing the gateway isn't as bad as exposing the web UI but anyone could still use your bandwidth. When you don't have convenient access to a local gateway, you could use the ipfs.io public gateway.
fwippy has quit [Ping timeout: 256 seconds]
<AdamThePhantump>
yeah but that only works when I have a hash to plugin.
akhavr has quit [Remote host closed the connection]
akhavr has joined #ipfs
akhavr has quit [Read error: Connection reset by peer]
<daviddias>
jbenet: do you happen to be still around?
<jbenet>
daviddias yeah but writing something. write and ill respond in a bit
<daviddias>
whyrusleeping: gave me a run through of bitswap yesterday and know I was looking to understand if there is any secret sauce on the chunking into blocks and block store that I should be aware of to go after than for node-ipfs
akhavr has quit [Read error: Connection reset by peer]
jhulten has joined #ipfs
<daviddias>
when you finish, if you can do a brain dump of everything you know about that, would be great :)