<jbenet>
excited that it's getting all over and is super popular :)
<jbenet>
re HN, it got vote-ring killed. they have really, really good voting ring detection. it will kill posts with fire if anyone upvotes without visiting the link first, or if they're only there for that link (i.e. go to the site and just view + vote for that one), and certainly will if anyone upvotes from a direct link.
<jbenet>
and rightly so. this link may be dead, but we could submit it again with a query string (it rightly counts that as a different link). i'd submit it tomorrow at ~8/9am PST. best time to do so.
<whyrusleeping>
jbenet: tomorrow is an apple event i think
<kyledrake>
The direct link on the blog hasn't been submitted yet, so that could be an option
<kyledrake>
Tomorrow is Apple Day
<jbenet>
so? there's more than one slot. or sure, wait another day.
steve__ has joined #ipfs
<kyledrake>
It's best to do while there's momentum, but I can try again on Thursday.
rschulman_ has quit [Quit: rschulman_]
Eudaimonstro has quit [Ping timeout: 240 seconds]
<jbenet>
kyledrake: agreed. will have much less momentum at night, so tomorrow or thu works i think. **shrug**
<kyledrake>
I was thinking the momentum would be gone on thurs, but yeah could try a morning post later.
voxelot has quit [Ping timeout: 260 seconds]
steve__ has quit [Ping timeout: 260 seconds]
thomasreggi has quit [Remote host closed the connection]
thomasreggi has joined #ipfs
<jbenet>
kyledrake: we posted our alpha video once and got like 2 upvotes. (we didnt upvote it to avoid ring detections). i posted it a few days later and frontpaged with >100. so **shrug**
<spikebike>
I heard that 0.3.5 will fix the too many file handle issue, is 0.3.5 out yet?
<jbenet>
spikebike we're at 0.3.8
<spikebike>
cool, thanks, I'll upgrade.
<spikebike>
I looked around for the current version and didn't see it anywhere obvious
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<multivac>
"And Nobody Delivers" would be a nice name for a rock band.
steve__ has joined #ipfs
<jbenet>
whyrusleeping: postmates.
<whyrusleeping>
jbenet: i dont think theyre where i am
<whyrusleeping>
unfortunately
hellertime has joined #ipfs
<rschulman_>
huh… I’ve just noticed that the firefox gateway redirect addon isn’t working anymore
thomasreggi has quit [Remote host closed the connection]
amstocker has quit [Ping timeout: 246 seconds]
thomasreggi has joined #ipfs
azm_ has quit [Ping timeout: 246 seconds]
step21 has quit [Ping timeout: 246 seconds]
mondkalbantrieb has quit [Quit: No Ping reply in 180 seconds.]
silotis_ has quit [Quit: No Ping reply in 180 seconds.]
machrider has quit [Ping timeout: 246 seconds]
rjeli has quit [Ping timeout: 246 seconds]
mg- has quit [Ping timeout: 246 seconds]
mondkalbantrieb has joined #ipfs
silotis has joined #ipfs
boxxa has joined #ipfs
edrex has joined #ipfs
<rschulman_>
oh, it only redirects gateway.ipfs.io, not just ipfs.io
<jbenet>
rschulman_ may want to PR a fix.
<rschulman_>
jbenet: Yeah, just looking at the code now.
mg- has joined #ipfs
azm has joined #ipfs
rjeli has joined #ipfs
step21_ has joined #ipfs
machrider has joined #ipfs
step21_ is now known as step21
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Eudaimonstro has joined #ipfs
<whyrusleeping>
jbenet: fun thing i just learned
<whyrusleeping>
adding a bunch of files has nothing at all to do with the NAT issue and too many file descriptors
steve__ has quit [Ping timeout: 252 seconds]
<whyrusleeping>
i put a watch on it, and it goes spiking up to 1024 all on its own
<whyrusleeping>
and DiscoverNAT doesnt appear to return...
<yoshuawuyts>
I was wondering: what would be the recommended way of storing git repos in ipfs? I'm not afraid I will, but it'd be scary if for some reason I'd be kicked out of GitHub.
Eudaimonstro has quit [Ping timeout: 250 seconds]
<spikebike>
yoshuawuyts: github has competition you know and git makes it really easy to switch providers.
<yoshuawuyts>
spikebike: yeah, either you host your own or you trade one centralized provider for another. Think storing git repos to ipfs would make for a good first project on ipfs ^^
<spikebike>
yoshuawuyts: I 100% agree
<spikebike>
yoshuawuyts: just make sure you keep a copy of your github repo so you have options if it disappears.
patcon has quit [Ping timeout: 255 seconds]
Eudaimonstro has joined #ipfs
<rschulman_>
why on earth would GitHub care if you hosted your git repository on IPFS?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
simonv3 has quit [Quit: Connection closed for inactivity]
akhavr1 has joined #ipfs
steve__ has joined #ipfs
akhavr has quit [Ping timeout: 250 seconds]
akhavr1 is now known as akhavr
Eudaimonstro has quit [Ping timeout: 240 seconds]
pfraze has joined #ipfs
<jbenet>
lgierth: apparently people are sometimes seeing 500s on the gateways trying to hit kyledrake's post. this is bad, particularly given the post
<whyrusleeping>
yeah.... i was trying to point that out earlier today
<whyrusleeping>
people were seeing 500's on the ipfs.io
<spikebike>
ya, 30-50% of the time when I looked, umm, around 3pm
<spikebike>
3pm PST
thomasreggi has quit [Remote host closed the connection]
<spikebike>
went on for 10s of minutes
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<spikebike>
most page loads were missing 1 or more of css, image, background, or gateway
<rschulman_>
yeah, the benefits outlined in kyledrake’s post only apply if you’re running the ipfs program yourself. :)
<rschulman_>
unfortunately
<spikebike>
ya, reminds me of the problem the previous cache fixed, umm... corel cache or somesuch?
<spikebike>
saw it pop up on hacker news and not get much attention, then slashdot which did
JasonWoof has quit [Ping timeout: 244 seconds]
<jbenet>
guys, please DO NOT trigger the HN voting ring detector. it happened to kyle's post, probably because some overenthusiastic people upvoted before reading, or only upvoted that post, etc.
<whyrusleeping>
if its that easy to kill a post, censoring hacker news would be easy
<jbenet>
whyrusleeping: i got confirmation. the post triggered the voting ring detector, hard. yet i dont think anybody was trying to. these algos are actually very tolerant of false positives and avoid false negatives. "good content can just be reposted later".
<spikebike>
to be honest I've never figured out how to upvote on hn
<spikebike>
I read it in feedly, I'm not sure it's even possible.
<spikebike>
either that or I got shadowbanned or soemthing
rschulman_ has quit [Quit: rschulman_]
JasonWoof has joined #ipfs
voxelot has joined #ipfs
<Leer10>
spikebike that post was amazing
Eudaimonstro has joined #ipfs
thomasreggi has joined #ipfs
inconshreveable has quit [Remote host closed the connection]
hellertime has quit [Quit: Leaving.]
steve__ has quit [Ping timeout: 252 seconds]
inconshreveable has joined #ipfs
<jbenet>
Leer10 thank the amazing kyledrake -- he's phenomenal
rschulman_ has joined #ipfs
<spikebike>
Leer10: heh, ya, thus the extra traffic
Pharyngeal has joined #ipfs
<Leer10>
yeah if only they could have accessed that page via IPFS
<kyledrake>
Ahh thanks! Yeah I caught a few 502s, but you reload and it comes right back up. Don't worry about it though, This Is A Good Thing because you're collecting debug info.
<whyrusleeping>
atgnag: run 'ipfs daemon' in another shell
<atgnag>
whyrusleeping: Is it gonna have to syncronize? I haven't ran it in weeks, and my internet is slow, so I guess it might take a while to catch up.
<spikebike>
the thunderbird folks made a blog post about considering e2e encryption for thunderbird.
<atgnag>
Oh, nevermind. It's working now.
<spikebike>
I was pondering using IPFS to publish GPG keys and a thunderbird module to publish/search for keys
<atgnag>
So, how do I view QmNhFJjGcMPqpuYfxL62VVB9528NXqDNMFXiqN5bgFYiZ1/e-for-the-permanent-web.html in a browser without using the ipfs.io gateway?
<whyrusleeping>
atgnag: with a daemon running: localhost:8080/ipfs/QmNhFJjGcMPqpuYfxL62VVB9528NXqDNMFXiqN5bgFYiZ1/e-for-the-permanent-web.html
<atgnag>
whyrusleeping: Oh, that's nice.
<atgnag>
Sorry for not RTFMing as much as I probably should. Thanks for the help.
<whyrusleeping>
no worries, youre asking easy to answer questions
<spikebike>
atgnag: and you could keep a copy by using ipfs to pin it
SoreGums has joined #ipfs
<atgnag>
So, is there a link-directory somewhere?
goodells has joined #ipfs
<SoreGums>
Cool idea, MPAA/RIA is going to love this, haha, oh well...
<atgnag>
It's not the book, it's a manga based on the book. ;)
<voxelot>
hmm i pulld up atgnag's node quick
<whyrusleeping>
my internet is slower than a dead turtle right now
<voxelot>
< through the gateway
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
oh yeah, looks like the gateways found it
<atgnag>
I think the reason it didn't work is that my internet is easily congestible, and I was trying to download something at the time.
<whyrusleeping>
atgnag: thats very likely
<rschulman_>
there we go, came right up now for me
<atgnag>
Actually, not to be a pain in the ass, but I decided to move it into my books folder and share that. You can unpin the ealier thing if you want. The one with my books is QmUGe3p5HKct7DrYVM5K2D6Uxggvxoc8KjiHCiFdJkg5yG
<jbenet>
wow slashdot is driving more traffic than HN
<jbenet>
wish we had gotten to push dist.ipfs.io before this-- or even just cut 0.3.8
<atgnag>
To where?
<atgnag>
Is the network getting hugged?
<atgnag>
I actually discovered this a couple months ago, but didn't try using it unitl recently.
<atgnag>
Well, I failed then, and I got it to work now.
reit has joined #ipfs
<SoreGums>
spikebike: sure it is - looks like ipfs handles plain old text pretty well... that's all that is required
boxxa has quit [Quit: Connection closed for inactivity]
NfNitLoop has joined #ipfs
<SoreGums>
looking to setup a node maybe two, just curious about disk space... anything specific about that or just watch the webui and see what happens?
<whyrusleeping>
SoreGums: it only uses disk space as you tell it to
<whyrusleeping>
no content that you do not request will be put on your computer
<SoreGums>
oh, so its in the "as content is accessed it will become distributed" - that's not how i read it - let me read again...
<whyrusleeping>
SoreGums: ah, yeah. when you access content, it it put on your computer
<whyrusleeping>
temporarily, until a GC is run
jmaguire has joined #ipfs
<whyrusleeping>
unless you decide to pin that content, in which case it stays local
<SoreGums>
yeah i saw the pin, that's cool
<NfNitLoop>
Hmm, I saw mention of the DNS TXT record in the video, but `ipfs --help dns` seems to recommend a different format?
<NfNitLoop>
Not quite sure how to set up the DNS -> signed hash -> object hash lookup chain.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<atgnag>
How do you pin? With ipfs get?
<whyrusleeping>
NfNitLoop: take a look at the TXT records for ipfs.io
<whyrusleeping>
atgnag: 'ipfs pin add -r <hash>'
<atgnag>
whyrusleeping: Oops, somehow I missed that in the --help output. Thanks.
rschulman_ has quit [Quit: rschulman_]
<NfNitLoop>
whyrusleeping: It seems to point directly to ipfs/QmeYYwD4y4DgVVdAzhT7wW5vrvmbKPQj8wcV2pAzjbj886, which is an object, so you'd have to update the TXT record every time you modified it?
<whyrusleeping>
NfNitLoop: until ipns stabilizes, yeah.
<whyrusleeping>
ipns works right now, but has a few bugs that will be addressed moderately soon
<whyrusleeping>
so we dont use it in production
<atgnag>
Here is a site that will be great for trolling people. Have fun.
<davidar>
I kind of feel like the ipfs.io landing page isn't the best introduction for new users
<SoreGums>
an index of links to content - it is just tex, there is no content - however kat could be taken offline, howver ipfs can't cause there isn't anything to takedown once it is up to speed
jhulten has joined #ipfs
<whyrusleeping>
NfNitLoop: ipns bugs: rebroadcast doesnt work yet, so if you dont run publish once a day or so, your record will drop out of the network
<whyrusleeping>
and also we dont have a solid notion of validity in the current implementation, so you just have to trust whatever record you find as the 'most accurate' one
<whyrusleeping>
we have the fixes for both of these all spec'd out, just need some polishing on certain parts and implementation
<NfNitLoop>
aah.
<spikebike>
SoreGums: sites like pirate bay are a search engine, reputation system, tagging, and categorizatoin. Not "just a text file"
<SoreGums>
spikebike: agreed, at the same time they are an index, which is just text at its most base form
<spikebike>
SoreGums: sure, but they reason they are popular is because they are easy to use and hard for people to replace popular torrents with crap ones.
<spikebike>
they change 100s of times a day, constantly updated, comments, feedback, etc.
<SoreGums>
yup, ipfs could be used with clever js clients in browser too - everything is there from what i read in the paper - difficult to layout in a sentence though...
<jbenet>
lgierth: ok i freed up some disk space + rebooted some containers. should no longer 502.
<jmaguire>
Do we have that clever JS for the browser yet to work with?
<whyrusleeping>
jmaguire: not yet, but we're making steady progress
<ipfsbot>
[go-ipfs] whyrusleeping created use-new-goprocess (+1 new commit): http://git.io/vZsdK
<ipfsbot>
go-ipfs/use-new-goprocess 6c2967a Jeromy: use new methods from goprocess/context, remove thirdparty/waitable...
<voxelot>
ipfs.io loading normally for me now
<jmaguire>
whyrusleeping: Is there a name for it that I can look it up by?
<jmaguire>
I'd be interested in making some small applications on IPFS as PoCs once that's going, or contributing if I'm able
<whyrusleeping>
jmaguire: this is the 'official' repo for the in browser js stuff: https://github.com/ipfs/ipfs.js but nothing is there yet
<SoreGums>
ipfs at its core is a protocol right? so if whatever exists follows the rules then clients can be written in anything
<whyrusleeping>
SoreGums: yeap!
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jmaguire>
Is there a reason you guys went with /ipfs/ at the root as a denotation for an IPFS address as opposed to the perhaps more standard protocol:// format?
<jmaguire>
And is ipfs.js blocked by ipfs-node? ("If you want this faster, [help us out!](/ipfs/ipfs-node)")
<jbenet>
jmaguire because "protocol://" doesn't work with unix. the ":/" breaks unix mountability.
<jbenet>
jmaguire we'll be using a placeholder thing like "unixweb://" or "x://" soon as a transitionary workaround
<whyrusleeping>
jmaguire: yes, ipfs.js is blocked on ipfs-node
<jmaguire>
Ah right, I forgot it's a FUSE mount. protocol:// would attempt to launch it with an application
<jbenet>
jamguire it's meant to work in _both_ the web browser and the OS fs.
<Xe>
excited to see the neocites collab!
<ipfsbot>
[go-ipfs] whyrusleeping opened pull request #1667: use new methods from goprocess/context, remove thirdparty/waitable (master...use-new-goprocess) http://git.io/vZsFT
<jbenet>
(jmaguire merging the unix/web rift, bringing balance to the force, or something like that.)
<jmaguire>
haha :)
<blame>
neocities + ipfs is #2 on hackernews right now
<atgnag>
I get the error "routing: not found" when I do ipfs dht findprovs
<jmaguire>
whyrusleeping: Is there any info on the expected architecture or means through which ipfs.js will work? I'm assuming it will reach out to an API (running on your own machine?) in order to do writes for example?
<atgnag>
The command still seems to work though.
<whyrusleeping>
atgnag: that just means it couldnt find K peers
<atgnag>
whyrusleeping: What is K?
<whyrusleeping>
the dht tries to find 20 peers for a given query and 'fails' if it cant
<atgnag>
Ah, okay.
<jmaguire>
I don't mean to be a burden, just quite interested in that aspect as I really want to see IPFS get dynamic. Seems like a big step to me.
<whyrusleeping>
jmaguire: you'll want to talk to daviddias when he wakes up
<jmaguire>
Thanks :)
<whyrusleeping>
hes on GMT time i think, so it will be a few hours
<spikebike>
jmaguire: I've pondered that, in the use cases I've considered so far it seemed like a local app would be better then ipfs hosted dynamic content.
<jmaguire>
And I'm -5, so perhaps during my work day tomorrow :)
<jmaguire>
spikebike: Could you elaborate on what you mean by that?
<spikebike>
jmaguire: for something multi-user, dynamic, and interactive an app seems way better than trying to replace a server farm running a php front end and a key/value or sql backend
<SoreGums>
so i've got no content and don't want anything in particular, however the whole system needs nodes to be connectors and horde content to provide resilience so i'm failing to see how "it only uses disk space as you tell it to" is supposed to work...
<spikebike>
jmaguire: like say facebook, ebay, whatever chat, etc.
<voxelot>
jmaguire, i'm helping with the ipfs.js
<voxelot>
some neet things happening there :)
<atgnag>
voxelot: ipfs.js? Tell me moer!
<atgnag>
*more
<voxelot>
full browser implementation
<jmaguire>
spikebike: So are you saying that you think that something like Snapchat would be a better PoC than something like Facebook, or are you saying that you think it would be easier to write a dynamic, multi-user app (such as a phone app, or application on a computer) than it would be to write a website backed w/ IPFS?
<voxelot>
i'd check out webtorrent if you guys haven't
<spikebike>
SoreGums: the DHT is the most redundant/resiliant part. There's not much other stored locally, unless it's caching something you asked for. You don't cache for trandom 3rd parties.
<spikebike>
PoC
<spikebike>
?
<jmaguire>
spikebike: proof of concept
<voxelot>
<--- working on social media ipfa :)
<spikebike>
just saying that using IPFS building blocks to write a local client seems like the right approach
<spikebike>
instead of trying to save facebook cycles on their current approach
<jmaguire>
Okay got you. Logic on the client, data on IPFS
<jmaguire>
Not logic embedded in IPFS data
<voxelot>
^
<atgnag>
voxelot: is that theoretically possible?
<voxelot>
yup!
<spikebike>
You could upload 100 pictures to IPFS and notify your friends about them and have ipfs handle the downloading/caching so it works well even if you have 1M friends.
<spikebike>
bbiab
<atgnag>
voxelot: Do you think webstorage can be used to pin stuff, or is that not what it's for?
<jmaguire>
In the theoretical app we're discussing, would you notify your friends over a separate protocol, or using some trickery based on where you've stored data in IPFS?
<spikebike>
and the IPFS dht can help find the nodes your friends are using
<jmaguire>
So latter then, ya?
<spikebike>
ya
<jmaguire>
voxelot: Can you tell me about the plans for ipfs.js? What its goals are? :)
<voxelot>
finishing node first i think is the plan
<voxelot>
browserify can get us real close
<atgnag>
jmaguire: 04:31 <voxelot> full browser implementation
<voxelot>
then we need hybrid nodes for the tcp connections
<voxelot>
webrtc
<mappum>
webrtc can be one of the supported transports, you wouldn't necessarily need TCP
<voxelot>
^
<jmaguire>
With a full browser implementation, has anyone given any thought to any of the other web "problems" (e.g. JS being the only client-side language any browser supports?)
<atgnag>
Is webrtc UDP?
<mappum>
it could be added to go-ipfs, then browser nodes would be fully connected
<atgnag>
jmaguire: webassembly might solve the javascript thing
<jmaguire>
atgnag: If you trust binaries
<mappum>
and emscripten
<ipfsbot>
[go-ipfs] whyrusleeping force-pushed use-new-goprocess from 6c2967a to c3280ce: http://git.io/vZsbp
<ipfsbot>
go-ipfs/use-new-goprocess c3280ce Jeromy: use new methods from goprocess/context, remove thirdparty/waitable...
<jmaguire>
though i guess most people don't read through their JS code either :P
<atgnag>
jmaguire: You should read what it is. webassembly works withing the javascript sandbox
<voxelot>
think the only problem is traversing some NATs and safari having no plans to implement webrtc
notduncansmith has joined #ipfs
<atgnag>
*within
<voxelot>
but meh, so one browser needs a pluggin
<mappum>
atgnag: that doesn't mean you don't have to trust it at all
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
certain browser vendors may be considering a more tight knit integration. :)
<atgnag>
mappum: Yes, but you don't need to trust it anymore than you trust javascript.
<atgnag>
(theoretically at least_
<mappum>
right
<jmaguire>
atgnag: The difference is I can read JS and I can't read binary
<SoreGums>
spikebike: right so if all sources of the data disappear then only the links/metadata is left and you get orphan links which the status quo with the http web as it is.
<jmaguire>
And binary would irreversibly remove info (though again, you could say minifiers do the same)
<atgnag>
jmaguire: Can you read minimized javascript?
<jmaguire>
C'mon man, I already said that :P
<SoreGums>
basically how does point 7. work - "As a database: applications can write directly to the Merkle DAG data model and get all the versioning, caching, and distribution IPFS provides" - this must take up disk space
<voxelot>
my IDE can read min.js lol
<voxelot>
just format
<atgnag>
Honestly, I don't see people reverse-engineering how a website works by reading the javascript very often.
<jmaguire>
I'm really not against WebAssembly, but I've seen that criticism brought up a lot.
<jmaguire>
atgnag: No, I do agree with you.
<atgnag>
Okay.
<mappum>
SoreGums: you might lose the links/metadata, too. basically you will just need to ensure somehow that it persists somewhere (maybe pay a few servers some cryptocurrency, and periodically get them to prove they still have the data)
<jmaguire>
atgnag: I just don't know that the majority does/will.
<atgnag>
That being said, webassembly might gave exploit developers new angles to work from.
thomasreggi has quit [Remote host closed the connection]
<atgnag>
It opens up whole new methodologies to be taken advantage of, so who knows if it will work well.
<SoreGums>
mappum: ok, i see what i'm thinking - there is a manual part of the system to ensure data persistence, i thought this was automagically taken care of :P
thomasreggi has joined #ipfs
thomasreggi has quit [Remote host closed the connection]
<jmaguire>
atgnag: In any case, WebAssembly doesn't really "fix" the "problem" of JS being the only client-side lang supported by browsers
<mappum>
SoreGums: if the data is popular and lots of nodes download it, then it sort of is automatically taken care of (that's how bittorrent works)
thomasreggi has joined #ipfs
<jmaguire>
It is more reminiscent of writing language bindings imo
<atgnag>
jmaguire: Yeah, I guess.
<mappum>
but there are no guarantees
<jmaguire>
The problem with that is of course compatibility between incompatible languages
<atgnag>
Though honestly, I think it's the want to use nothing but browsers to do things that's part of the problem.
<SoreGums>
example "ipfs node notices that this is being requested alot, i've got 50gb available i've some free space i'll grab some of this"
<atgnag>
But that's a whole new bag of worms.
<jmaguire>
atgnag: I don't have a want to use nothing but browsers. Not sure why you think that
<whyrusleeping>
SoreGums: that behaviour will be implemented as a bitswap agent
<atgnag>
jmaguire: I wasn't talking about you, I was talking about a large portion of the population.
<jmaguire>
I'm looking more at what status quo is. Adoption is the hardest part of anything new. The easiest way to fix adoption problems is to keep things the same for end users.
<whyrusleeping>
i wrote up a proof of concept patch that did that a little while back, but i'm not sure where it got to
<jmaguire>
Then you can differ more
<SoreGums>
whyrusleeping: right so i did read the paper correctly, to be implemented then, cool #notcrazy ;)
<jmaguire>
That was poorly written, but... hand my mom (a Windows user all her life) a Mac, and she won't like it, no matter how "more usable" or "user friendly" it is.
<whyrusleeping>
yeap! thats a feature thats very important to me
<jmaguire>
ignoring the clear biases in my statement ^
<SoreGums>
and this is why i want to run nodes - for the good of my will, haha
<whyrusleeping>
exactly! for the same reason i run tor nodes
<mappum>
SoreGums: and a tangential project we plan on working on next, to incentivize what you are suggesting: http://filecoin.io/
<atgnag>
Is there a channel where I can request content to be pinned? I feel like having people asking to have their stuff pinned all the time could get annoying.
<voxelot>
yup, i'll be running a full hybrid node on my OVH server at least
<ipfsbot>
[go-ipfs] jbenet pushed 2 new commits to master: http://git.io/vZsNV
<ipfsbot>
go-ipfs/master c3280ce Jeromy: use new methods from goprocess/context, remove thirdparty/waitable...
<ipfsbot>
go-ipfs/master 2107930 Juan Benet: Merge pull request #1667 from ipfs/use-new-goprocess...
<mappum>
atgnag: that's a good idea
<atgnag>
For now though, I'd really like this pinned. QmWZQaqiooxrKP6EbSXVaurKwBKZWJZCgvGkmgqrbrrrjg
<jmaguire>
atgnag: I will say however, mobile adoption is removing the bias towards browsers
domanic has joined #ipfs
<whyrusleeping>
atgnag: your link is so slow
<atgnag>
jmaguire: I guess that's true.
<atgnag>
whyrusleeping: It's my internet that's slow.
<whyrusleeping>
atgnag: thats what i meant
deltab has quit [Ping timeout: 250 seconds]
<SoreGums>
out of what i have read so far, i'm interested purely from the "i could contribute to this for minimal cost" - like you say about tor nodes, same thing, must be a lot of us like this. cause i'm sure i'll be using ipfs as a consumer, this will balance it out a bit
<atgnag>
whyrusleeping: It is. It really is. I get 16Kbps down minus overhead. I have no idea what the upload speed is.
<atgnag>
Oh wow, 5 people pinned it already. Thanks guys! <3
<atgnag>
I can only imagine how slow it is uploading to 5 different people…
<whyrusleeping>
atgnag: things that show up in findprovs dont necessarily mean its pinned, just that they have it on their computer
<atgnag>
whyrusleeping: Ah, okay.
<whyrusleeping>
(i've pinned it for you, but cant claim that anyone else has)
<atgnag>
I suppose some people probably just viewed it then.
<jmaguire>
I will pin it once it finishes :P
<voxelot>
that site looks so crazy i have to pin it :)
Tv` has quit [Quit: Connection closed for inactivity]
<atgnag>
voxelot: That's kind of what I was going for.
<atgnag>
So, anyone else counting down until the first IPFS-involved copyright controversy?
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
atgnag: its not really any different than the existing web in that regard
<jmaguire>
I imagine that would essentially end up like torrents
<atgnag>
whyrusleeping: That's true.
<atgnag>
That doesn't keep jimmies from being rustled though.
<atgnag>
Before there were torrents, DCC was all the rage.
<atgnag>
Anyways, may I suggest a room #ipfs-requests or the like be created, for people to request their stuff gets pinned, and maybe to request content on ipfs?
sharky has quit [Ping timeout: 272 seconds]
<Leer10>
atnag: what the shit is that
<Leer10>
it's like the 5th world vomited over freenet
<atgnag>
Is there any way to tell if someone has finished pinning my site?
<whyrusleeping>
atgnag: no easy way, but you could query providers for every single sub-hash of your root
<whyrusleeping>
'ipfs refs -r <hash>' gives you all hashes below a given hash
<whyrusleeping>
(note: that would be incredibly tedious)
<alu>
uh
<atgnag>
whyrusleeping: Yeah, I got that.
zabirauf has joined #ipfs
<atgnag>
Why is everyone quiet all the sudden?
<atgnag>
So, do you think people in China can access IPFS right now?
<atgnag>
Though, I doubt many if any of them are at the moment, but I'm serious how censorship-resistant ipfs is out of the box.
mildred has quit [Read error: Connection reset by peer]
mildred has joined #ipfs
voxelot has quit [Ping timeout: 244 seconds]
ygrek has joined #ipfs
<spikebike>
the people I talk to in china basically say for any general purpose apps to work relaibly they end up buyign a VPN
<atgnag>
Well…
<atgnag>
I guess censorship won't work very well if it interferes with stuff that's supposed to work.
<spikebike>
well serving over ssl can be ok if you are careful whose certs you protect and don't use common DNS/IPs that are on the block list
<SoreGums>
oh yeah the proper darkside of the coin, cp - that kind of sucks...
<atgnag>
So, IPNS isn't implemented in the release yet?
pfraze has quit [Remote host closed the connection]
<daviddias>
Morning whyrusleeping atgnag ! :)
jhulten has quit [Remote host closed the connection]
jhulten has joined #ipfs
<daviddias>
you can use ipfs from a browser through the node-ipfs-api (https://github.com/ipfs/node-ipfs-api)now, but what we are really shooting for is to have a compatible IPFS node running on the browser
domanic has quit [Ping timeout: 260 seconds]
pfraze has joined #ipfs
jhulten has quit [Ping timeout: 265 seconds]
ygrek has quit [Remote host closed the connection]
<davidar>
still not exactly clear on what ipld is aiming for
<daviddias>
for sure :)
<daviddias>
By parts, because IPLD is more than just a move to LD
<atgnag>
What about IPNS though?
<daviddias>
1) By going into JSON, it means people can just dropping their JSON blobs into IPFS, without having to structure them in our current protobuf scheme (and I mean dropping is not ipfs add a file with JSON blobs, but actually having a data structure that can use MerkleDAG links to its own advantage)
<cryptix>
gmorning
<daviddias>
and just that is an awesome benefit
<daviddias>
and then we could say, or we reserve a bunch of keys (part of the structure) inside your JSON blobs so that our IPFS things could understand which were MerkleDAG links and retrieve them
<daviddias>
or we could say, you have to be the one unwrapping your own datastructure and just asking us for the blobs you are linking to
<daviddias>
both can be done, but the first would mean that data structures would have to be more strict, while the later means that developers would need to do an extraordinary amount of extra work
<daviddias>
to make this simpler and more attractive, what we found was LD, where devs could still add their current objs, and we can add the information we need for the data to be linked inside IPFS
<daviddias>
giving the benefits of the first approach, where we have 'the keys we want' to unwrap the structure without the hassle of the second
<daviddias>
having LD just for MerkleDAGLinks is the first level
<daviddias>
because in essence and due to LD nature, we can expand the range of possibilities way more
<daviddias>
Also developers can leverage that, we can start having MerkleDAGLinks that not only are identified that are MerkleDAGLinks and should be resolved inside IPFS, but we can also infer which type of data they are pointing to
marianoguerra has joined #ipfs
<atgnag>
Is there a nice comparison between ipfs and zeronet?
pfraze has quit [Remote host closed the connection]
* spikebike
googles zeronet
<spikebike>
whenever I see "bitcoin crypto" I think slow as shit.
akhavr has quit [Read error: Connection reset by peer]
atomotic has joined #ipfs
akhavr has joined #ipfs
<ehd>
morning :)
<atgnag>
Morning? In what timezone?
<ehd>
CEST
<spikebike>
I do wonder if it's better to start with a p2p filesystem and try to make a p2p platform, or make a p2p platform and then add a fs
<spikebike>
heh, it's morning in all timezones in the usa
<spikebike>
cept alaska and hawaii
ygrek_ has joined #ipfs
ygrek has quit [Ping timeout: 240 seconds]
<davidar>
daviddias: yeah, I'm pretty sure I understand that part of it, and am really looking forward to it :)
<davidar>
the bit that's confusing me is what the jsonld @ directives have to do with it
<davidar>
they seem like orthogonal problems
<davidar>
If everything's going to be internally encoded by cbor, couldn't you just extend cbor with a pointer (ie hash) primitive?
<davidar>
Or are the @ directives solving a different problem?
<daviddias>
cbor is just a serialization format to pass on the wire
<davidar>
To put it another way, why not put the ipfs-specific only in the wire format (cbor) only, and erase it before it gets to userland (json)?
<daviddias>
the @directives let us have any type of key become any type of data we want
<davidar>
ipfs specific metadata*
<daviddias>
davidar: that option would be parallel to protobufs, but it would require us to have a specific wire format
<davidar>
Sorry, I'm probably just being dense, but I'm looking at the examples in https://github.com/ipfs/go-ipld/issues/4 and I'm not getting what @context is actually achieving
<daviddias>
@context is just a 'header' (per se) on a JSON blob that describes it
<daviddias>
image that you have a dumb JSON blob like { name: 'David' }
<davidar>
Isn't that what multicodec is supposed to do?
<daviddias>
you would say that 'name' is clearly a person name, because as a Human is really easy to understand that. but then if I give you { name: 'Tesla' }, now we have a conflict, because name can be both person and company name
<daviddias>
@context helps the 'machine' infer which type (not type as in language type) the data is
<davidar>
Why does that need to be built into ipld though?
<daviddias>
the always big question is "should we really create another standard", and typically the answer is 'let's try to leverage the research and development already done and take the best we can use'
jhulten has joined #ipfs
jhulten has quit [Ping timeout: 264 seconds]
<davidar>
Why does context/schema need to be transmitted inband (within the json object) rather than just attached to the object (like mime types)?
<davidar>
ie the pair (schema, object) instead of {@context:...}
<daviddias>
By transmitting it 'inband' we create self described data structures
<davidar>
but why does ipld mandate self described data?
<daviddias>
it doesn't 'mandate' davidar, it is a very good approach though :)
<davidar>
either it's an inherent part of ipld or it isn't
<davidar>
Sorry, I'm not trying to be argumentative, just unclear on the scope of ipld
<davidar>
Ie how much metadata is absolutely required for ipld to function, and which bits are just optional conventions
<lgierth>
morning
<daviddias>
I believe what I'm failing here, is communicating clearly the context and how we reached this point, it is way easier to understand a decision if we know the history behind it
zabirauf has quit [Ping timeout: 246 seconds]
jamescarlyle has joined #ipfs
<davidar>
I'm not disagreeing with self describing data being a good idea
<davidar>
Just whether it's optional or required?
<daviddias>
the exercise I would challenge you to do, is come up with a solution for representing data inside IPFS, that tries not to reinvent standards completely (if possible) and focuses on good UX for developers
<davidar>
Obviously ipld will need some information to be able to traverse merkle links
<davidar>
But @ context seems to be trying to do a lot more than that
<davidar>
OK
<jamescarlyle>
just catching up from the log
bedeho has quit [Ping timeout: 250 seconds]
<jamescarlyle>
but what would really help me (maybe others) is some worked examples, comparing an RDF graph (triples) with different approaches to the structure within the merkledag
<daviddias>
davidar: yes, @context offers a lot of extensibility
<jamescarlyle>
json-ld is just a serialisation - it's the RDF graph to Merkle graph comparison that I'd like to understand
<davidar>
It seems odd to be restricted to the json data model, when json isn't even actually being used
<davidar>
You have a non-json wire format being deserialised into native data structures (which also contain more information than representable by json)
<davidar>
daviddias: anyway, leaving arguments about implementation details until another time :), can I say "ipld = json + types" ?
mildred has quit [Quit: Leaving.]
<SoreGums>
is there anything worth copying from app.net in terms of describing data? (just throwing that out there)
mildred has joined #ipfs
<davidar>
SoreGums: I'm not seeing anything about describing data on that page?
<cryptix>
SoreGums: -v? you mean that adn 'tweets' also could have more metadata for specific client use
<cryptix>
also: kyles ipfs@neocities post is still on fire on HN.. i have 5 comments queued up 'posting to fast' :<
<davidar>
cryptix: thanks for the reminder, tried to answer a few of the open questions :)
jhulten has joined #ipfs
higuita has joined #ipfs
<higuita>
hi, ipfs right now can only be used for static content, right?
<cryptix>
higuita: nothing stops you from hosting a next gen append-only log on ipfs :)
<higuita>
no dynamic pages, like php, ruby and similars
ei-slackbot-ipfs has quit [Remote host closed the connection]
inconshreveable has joined #ipfs
nicknikolov has quit [Read error: Connection reset by peer]
ei-slackbot-ipfs has joined #ipfs
<davidar>
higuita: all of the dynamic stuff has to be done client-side
<higuita>
ok
nicknikolov has joined #ipfs
<davidar>
which isn't all that unusual these days anyway
<higuita>
so right now is a distributed "S3" bucket :)
<cryptix>
yup - thats the spirit. just dont think it has to be all static files. you could also load sqlite files from it on the client side or what not
<cryptix>
its not limited to html+css/image assets is what i mean
jhulten has quit [Ping timeout: 264 seconds]
<higuita>
is there any way to know how much nodes there are and their "location" (europe, asia, north america, south america, africa, etc)
<cryptix>
there is a connections view there with a fancy globe :)
<higuita>
yes, i understand, i was just checking if there was some kind of php binding for external languages or some kind of proxy for that
<higuita>
i 'm asking for the nodes, not the clients
<cryptix>
the go-ipfs impl is a daemon (node) and can act as a client
<davidar>
higuita: nodes don't execute arbitrary code, if that's what you're asking
atgnag has quit [Read error: Connection reset by peer]
<cryptix>
+ there are various 'api bindings' starting to spring up, giving you the add/cat primitives - there is an meta/tracking issue on github about those
<higuita>
right now i have a site that is used in africa and middle east, but there are always performance problems connecting to usa and europe, so i was checking if we could use this to improve a little the performance :)
<higuita>
even just static content could help :)
<cryptix>
higuita: totally! just run the go-impl in daemon mode and expose that (by default is localhost:8080)
<davidar>
higuita: the rule of thumb is, if you can host it on github pages, then you can host it on ipfs :)
<higuita>
ok, we will do some tests
<cryptix>
higuita: if you can instruct people to run ipfs locally and give them the browser addons, the content will also replicate
<lgierth>
jbenet: whyrusleeping: it's mostly the docker logs... ansible 2.0 still isn't released which has support for --log-driver=none. i think docker supports that on the daemon level nowadays, so that might be an option. otherwise, use infrastructure.git/solarnet/cleanup.sh, but edit it to not run on all gateway hosts at once, it restarts ipfs because docker can't deal with the log file being deleted (who knows why)
<hannes5>
ah, sounds like i havent really understood it yet then
<hannes5>
i thought each file/url would get its own hash
<lgierth>
hannes5: the 1970 timestamp is so that it can always be cached
<lgierth>
no matter how bad the client time is set
<davidar>
hannes5: it does, but you also have directory objects that link to those hashes with nice names
<cryptix>
hannes5: they each have their own hash. a directory is just an object with multiple <name, hash> entries. so you can use $roothash/path/to/file or just $hashOfFile
<lgierth>
we are now #1 on hn
<cryptix>
lgierth: yup :)
<cryptix>
still have posts queued.. hit some kind of throttle :[
<cryptix>
davidar: non-issue imho. batteries will improve, BTLE/wifi will reduce power load
<cryptix>
mashing 24/7 on LTE.. well.. yea...
<hannes5>
ah nice
<davidar>
cryptix: yeah, that was my initial thought, but how often is the person next to you going to have the files you want?
<hannes5>
is there some standard way to attach metadata to hashes (files in this case)? eg original timestamp, arbitrary text, etc?
<ehd>
davidar: relevant for teams working together on the same resources
RX14 has joined #ipfs
<cryptix>
depends on where you are :) its increasingly the case for uni networks or freifunk communities
<ehd>
yea
benjaminjackman has joined #ipfs
<cryptix>
its also increasingly true for 'my brother just send me this youtube link'
<davidar>
of course, definitely useful, but does it answer the battery usage question in general?
<RX14>
so, i'm running go-ipfs installed with go get, and it keeps crashing, but as soon as I try to capture the crash log, it stops crashing
<cryptix>
RX14: that sounds like fun.. what platform are you on?
<RX14>
32bit linux
<davidar>
as in, people aren't going to want to run full nodes on their phones if it's always using their data connection
<davidar>
*ipfs nodes
<reit>
on mobile: i've been thinking that as IPFS gains traction it will become a natural cost-saving measure for ISPs to provide their own nodes their clients can 'proxy' through
<cryptix>
RX14: try 'IPFS_LOGGING=info ipfs daemon 2>&1 | tee ~/ipfs.log'
<cryptix>
this will reroute the output to a log file while still displaying it
<RX14>
yeah, but I can't get it to crash now
<RX14>
because I'm observing the bug
<reit>
so, your mobile company would be the gateway you request data from
<RX14>
:P
<lgierth>
reit: for that it'd be good get big sites on board
<davidar>
reit: yeah, that would definitely be cool
<lgierth>
i'm looking at you wikipedia
<reit>
and they'd only need to retrieve it once for all of their customers
<reit>
yeah
<davidar>
but need to get over the adoption hump first :)
<lgierth>
ipfs.js <3<3
<reit>
^^
<ehd>
oprah moment
<cryptix>
davidar: yea.. i think we want some kind of 'client only' nodes for mobile that can step up to be a daemon for general use but i dont know.. i want to see numbers first on ho wmuch the battery is drained actually
jamescarlyle has quit [Remote host closed the connection]
<cryptix>
davidar: or cap the number of connections on mobile
<RX14>
so, does IPFS deduplicate?
<RX14>
for example if you store a file with the same hash, but it's already there, it's not stored twice
<cryptix>
RX14: jup
<davidar>
cryptix: but the GP's question is how to deal with lots of client-only nodes
<RX14>
because if you store the same file twice it returns a different hash, but I assume that's just a different pointer to the same blocks
<cryptix>
RX14: larger files are also chunked and hashed by block
<RX14>
yes, ok
<cryptix>
RX14: uhm... ipfs add of the same file should always produce the same hash
<davidar>
well, it's kind of in the definition of a hash function...
<cryptix>
davidar: re to many.. to theoretical for my schedule right now
<RX14>
huh, now it's returning the same hashes
<ehd>
i feel like we're not going to have too few full IPFS server nodes
<RX14>
must be my imagination :P
<davidar>
ehd: that's the point of making browsers into ipfs nodes :)
<ehd>
God, I want that so bad :)
<cryptix>
ehd davidar: there could also be some kind of arbiter service that wakes up consenting mobile nodes to elevate from client-only to full node
<ehd>
It's the difference between me giving someone a self-sufficient, self-contained URL (ipfs node + web app) and a URL which does API requests to my specialized limited IPFS api gateway
<RX14>
with ipfs.js, how would it work with multiple tabs
<ehd>
you can send all kinds of message and data between tabs nowadays
<RX14>
hmmn, so one tab would be the node... and when you close that tab?
<RX14>
what about multiple browsers on one PC
<reit>
for mobile: a neat system might be, when the reciprocation system gets more advanced, to allow for nodes to build up a reputation while they're at home sitting on charge/wifi
<reit>
that they can 'cash in' when they're out and about, to download stuff without reuploading anything at the time
<cryptix>
sounds iffy to me, too - i thought of it more like an browser wide extension
<davidar>
browser wide extension would be better though
<RX14>
it would, but then the extension needs to be installed
<ehd>
problem on iOS might be that the IPFS node app wouldn't be allowed to wake up in the background for too long, even when on wifi and charging/charged. would have to investigate
<davidar>
what did whyrusleeping mean by his comment earlier that certain browser vendors looking at tighter ipfs integration?
<davidar>
cryptix: but that's exactly what I'm trying to do :p
<cryptix>
i needed to build a rooted image with gapps recently for an unmaintained devboard... what a ride
Encrypt has joined #ipfs
<davidar>
don't think we need to worry about the current #1, it's got weird js scrolling so the HN community is definitely going to flame it :p
<davidar>
cryptix: yeah, the whole rooting situation is a bit stupid
<davidar>
might have been able to avoid it if linux had switched to gplv3
<davidar>
but could also have just resulted in a fork I guess
<davidar>
cryptix: but rooting is easier than jailbreaking, no?
<flounders>
Google pretty much is already running their own fork of the kernel anyway.
rschulman_ has quit [Quit: rschulman_]
jamescarlyle has joined #ipfs
<davidar>
(other than the "benefit" of the ios platform being homogeneous)
<davidar>
flounders: true, that's also stupid
<davidar>
lots of things about android are stupid
<davidar>
just less bad that ios :)
<davidar>
*than
<cryptix>
davidar: well - if you have need to find the 0days for the jailbreak... sure
jamescarlyle has quit [Remote host closed the connection]
<cryptix>
davidar: needed to start of a custom image for this one board nobody cares about which dev didnt make it easier beause its some weird release inbetween 4.3 and 4.4
<davidar>
cryptix: what i mean is, jailbreaking ios is easier only because there's only a handful of devices
<drathir>
android was god idea in theory, but in practice become like always... piece of trash...
<cryptix>
davidar: exactly - and one source of os
<davidar>
oh crap, i guess you all own macs?
* drathir
preffer n900 still, and no better device exclude neo900...
f[x] has quit [Ping timeout: 246 seconds]
<davidar>
multivac: i need backup!
<multivac>
Not a bloody clue, sir.
* flounders
owns primarily Lenovo devcies with the one exception being a Raspberry Pi.
<davidar>
dammit multivac
<cryptix>
davidar: lol - i bought a macbook back in '08 served me pretty well
<davidar>
flounders: which os
<cryptix>
still does
<flounders>
I'm running Arch on all 3 of my current machines.
<flounders>
Fedora is on the old dinosoar that stays powered down.
* drathir
arch+bsd
* davidar
high fives flounders
<davidar>
and drathir
<cryptix>
drathir: <3 :)
* drathir
shamed *hidding*
<cryptix>
i also run linux and bsd on a couple of machines
<davidar>
wait, what were we arguing about again?
<davidar>
oh yeah
<flounders>
I have had to deal with OS X at work, and it's more of a pain to work with at the system level.
saulo_ has joined #ipfs
<drathir>
davidar cryptix thanks a lot...
<davidar>
android fragmentation vs desktop linux fragmentation
<drathir>
ofc vps at debian...
<saulo_>
hey. am I the only one getting "forbidden" for many files in the webUI?
<davidar>
android fragmentation is only because manufacturers and telecoms suck
<drathir>
but about arch have a install workin till 6y w/o problems... and in my opinion arch is stable as rock...
<davidar>
now lets fight about which linux distro is the best :p
<drathir>
but still i guessing osx better than m$...
<flounders>
It sounds like most of us are running Arch.
<drathir>
davidar: easy no fight at all only opinions ^^
<RX14>
i tried the go-ipfs AUR package and it didn't build :(
<davidar>
nah, i'm ubuntu, just because I'm lazy :)
<flounders>
Oddly I prefer dealing with Windows in a server environment over OS X.
<davidar>
debian on server
<drathir>
RX14: reported?
<davidar>
i don't get osx servers
<cryptix>
saulo_: sounds strange. need to see the acutall message and what was requested
<cryptix>
saulo_: also browser version and platform maybe
<davidar>
drathir: yeah, and you can count on me, to meet you in the parking lot :p
<davidar>
drathir: assuming you are within walking distance of my local area
<drathir>
davidar: sounds a little like "fight club" invitation ^^
jamescarlyle has joined #ipfs
<drathir>
davidar: jokin ofc...
jamescarlyle has quit [Remote host closed the connection]
<davidar>
hehe
<davidar>
now i've completely forgotten what we were actually talking about
<davidar>
something to do with ipfs...
<flounders>
Android and iOS development or procrastinating it.
<drathir>
yea i need to check aur ipfs package...
<saulo_>
windows 7. chrome 41. ipfs on docker in another machine with ports forward as described in the read.me . webui loads. in the JS console I get 403 - Forbidden for most of the requests created by bundle.js to the :5001/api endpoint (config, version, update and id). If I open the said URLs individually in another tab, the files are downloaded and shown without a problem
<saulo_>
@cryptix
<davidar>
saulo_: probably a cors issue
<cryptix>
saulo_: yea +1 davidar
<davidar>
by deafault ipfs only allows requests from the local machine
<davidar>
for security reasons
<saulo_>
davidar: sounds like it
<cryptix>
dunno how it shows up through docker
<saulo_>
but I don't get the usual CORS warning
<davidar>
drathir: yeah, I'm always jealous of AUR
<saulo_>
the deamon shows no error
<davidar>
PPAs kind of suck in comparison
<davidar>
saulo_: the referrer header
<drathir>
davidar: aur and pacman is the major things keep me at arch...
<drathir>
exclude ideology ofc and light as possible and free of customize structure...
<davidar>
drathir: i though pacman itself was kind of fragile?
<davidar>
saulo_: it just seems to quietly drop things if the referrer isn't localhost
<drathir>
no better package manager than pacman i using... fast no problems in dependencies like its in debian...
jhulten has quit [Remote host closed the connection]
<RX14>
1.4.2
<RX14>
i need to get this laptop home so I can update it's packages without worrying about the bandwidth limit
<RX14>
celeron with 1.5gb of ram, it's really testing to use
<davidar>
RX14: does `go get -u github.com/ipfs/go-ipfs/cmd/ipfs` work?
<RX14>
yes, it does
<RX14>
that's how i'm running IPFS now
<drathir>
RX14: normal aur version or git one?
<RX14>
normal, not git
<RX14>
git worked
<drathir>
k
<davidar>
sounds like an issue with the aur package then
<RX14>
-git is basically the same as go get -u github.com/ipfs/go-ipfs/cmd/ipfs, except packages and global
<RX14>
i assume
<saulo_>
davidar: with API_ORIGIN=http://URL:5001 it partially work. although im not getting 304 anymore. I'm still getting 400 for the update/check?stream-channels (bad request) , config?arg=Gateway (500 internal server error) and /ipfs/HASH/static/fonts/fontawesome (404)
hannes5 has quit [Quit: Leaving]
<davidar>
saulo_: fontawesome one is normal
<davidar>
why hasn't that been fixed yet?
<davidar>
saulo_: actually, yeah, I'm getting the same errors
<davidar>
jbenet: whyrusleeping: when did the webui break?
<RX14>
I get those FA errors but the UI is plenty usable
<davidar>
mine isn't usable at all
<davidar>
lots of 40x errors
<RX14>
the 404 on the font-awesome woff2 is just a fallback error
<davidar>
saulo_: are you on the most recent master?
<RX14>
it uses the woof because the woff2 isn't there
<RX14>
but if it's totally unusable, that seems a deperate issue
<saulo_>
davidar: yes. I've just downloaded the docker image
<mmuller>
well, if it's hosted off IPFS, _that_ would be something.
<RX14>
but until ipfs.js happens, that's not going to happen
<mmuller>
I suppose they could run gateways.
<RX14>
i suppose, but then meh
<vanila>
typo in the paper "IPFS nodes communicate regualarly"
<RX14>
is there a typo?
<cryptix>
btw: /ipfs/$hash link detection browser addons which redirect to local nodes are already a thing :)
<RX14>
i think that's correct
<RX14>
mmuller, looking through the code they might eb running their own IPFS gateway
<cryptix>
regu_a_larly
fleeky has quit [Quit: Leaving]
<RX14>
oh, i'm blind
<cryptix>
thought so too :))
<RX14>
guess i'm used to english typos
<cryptix>
vanila: can you open an issue at github?
<vanila>
sorry
fiatjaf has joined #ipfs
<cryptix>
i think the tex src is also available there if you like :)
<fiatjaf>
how is neocities hosting the content?
<fiatjaf>
the ipfs content, I mean
<cryptix>
fiatjaf: i think, before they just did the usual generate>expose webroot with clasical apache/nginx. now they also generate>ipfsAddWebRoot and publish the hash
<cryptix>
fiatjaf: you might also want to ask kyledrake :))
<cryptix>
kyledrake: kudos on that post btw - really nice. the mentioned 'will publish the hash of your site on the profile page' is future tense, right?
<RX14>
so if i did it with IPFS, I would need to run a gateway
<cryptix>
yea, 'ipfs daemon' which opens localhost:5001 for the json api
<RX14>
and i bet NGINX + CDN will be faster than a public gateway
<cryptix>
it's how the 'cli' binary works too when there is a daemon running, all its interaction goes over that json api
<RX14>
cryptix, but if we can't run the client on windows, there's no point is there?
<RX14>
can you run the daemon on windows?
<cryptix>
yup
<RX14>
ohhh
<RX14>
nice
<cryptix>
i fixed some month ago. but i dont think we run CI tests on windows yet so... :)
<cryptix>
its a bit more alpha than the osx/linux versions :)
<RX14>
i would implement HTTP fallback I think
<RX14>
because of the alpha status
<RX14>
but using IPFS to distribute objects would be great
voxelot has quit [Ping timeout: 256 seconds]
<cryptix>
i have a couple of small websites where I generate html with src="/ipfs/..." on the server it does a redirect to the public gateways for these but if you have the browser plugin it hits the local node. so i dont actually expose the gateway myself, just a node with the data
<RX14>
it's not a website
<cryptix>
well.. then you are much more free in your choices anyway
<RX14>
yes, i am
<RX14>
i don't suppose there is a way to get iobjects by sha1 hash?
<RX14>
because it's a hash ob the object + metadata isn;t it?
<cryptix>
also thought about that couple of times, the problem is the chunking
<RX14>
yes, it is
<RX14>
it would be great if that was possible
<cryptix>
we could compute the 'unchunked' hash during the add process and also add it to some index. would be really handy for all kinds of tarballs, isos etc
<RX14>
yep
<RX14>
if you implemented that, it would mean the client can start downloading from IPFS without an API call to get the IPFS location
<RX14>
for my application
Leer10 has quit [Remote host closed the connection]
jamescarlyle has quit [Remote host closed the connection]
<higuita>
how do ipfs manage disk space?
jhulten has joined #ipfs
<higuita>
i assume it creates local copies of the blocks, but where/when does it stop or clean up?
<cryptix>
higuita: currently you need to run 'ipfs repo gc' which will dropp unpinned objects
<higuita>
ok
<cryptix>
higuita: automatic/threshold based gc is planned but not in yet
<higuita>
so for now is manual
<cryptix>
yup
kanzure has joined #ipfs
<higuita>
ok
<higuita>
thanks
<cryptix>
very welcome :)
<wasabiiii>
FUSE filesystem going to be writable eventually?
<cryptix>
wasabiiii: /ipfs/ can't really be but the /ipns/ one is (for your key at least)
<cryptix>
RX14: sorry - i dont see why you cant work with the chunked hashes
<cryptix>
RX14: i only saw it as handy for like, this AUR wants tar with sha1==xyz so lets see if ipfs has it
<RX14>
i can work with chunked hashes...
<RX14>
i just need to get the root node
<RX14>
which you cannot get by sha sum
<RX14>
it's not a blocker to using IPFS
<RX14>
i just need to host my own lookup service from sha1 -> ipfs address
<RX14>
and it would be nice if IPFS had that inbuilt
jhulten has quit [Ping timeout: 244 seconds]
<cryptix>
okay :) just wanted to make clear that its just a different hash for the same content just chunked. The funny hashes come from multihash: https://github.com/jbenet/multihash
<RX14>
so, what exactly does ipfs checksum, what you are uploading, or the whole object
<mmuller>
RX14: it's a Merkle DAG, it hashes (not checksums) blocks and "containers"
<mmuller>
containers can be, for example, directories or large files represented as lists of blocks
pfraze has joined #ipfs
<mmuller>
blocks are chunks of file data
<mmuller>
so smaller files are represented as single blocks and referenced by the hash of the entire content
<mmuller>
larger files are referenced by hash of the block list
<mmuller>
and I think all of that is correct :-) somebody hit my with a cluestick if not.
<RX14>
so, how would you go about calculating the IPFS hash of some data?
<rschulman_>
Yeah, RX14 I think you want to read up on Merkle Trees
<RX14>
yeah, i do
<RX14>
i havent looked at them
<mmuller>
not sure which hash is used by default, but the possible hashing algos and the multihash method of encoding them are documented here: https://github.com/jbenet/multihash
<rschulman_>
the end result hash is a hash of a document that describes the hashes of all of the chunks in the file
<rschulman_>
unless I’m mistaken.
<RX14>
so I would need to chunk the file, hash the chunks, then hash the node. but I would need to get total binary compatibility with how IPFS stores nodes
<RX14>
otherwise the checksum would be wrong
<mmuller>
well, mostly files should be represented as a single chunk - IIRC the block size is quite large, 64M? I think?
<mmuller>
so for smaller files it should be adequate to hash the content.
Encrypt has quit [Quit: Quitte]
jamescarlyle has joined #ipfs
mildred has quit [Ping timeout: 272 seconds]
<RX14>
but I think each node has a header right
<vanila>
how many people created ipfs?
<fiatjaf>
cryptix: is it safe/recommended that we use the public gateways to serve our content? (like pointing to gateway.ipfs.io?)
<RX14>
fiatjaf, well it requires trust in whoever maintains the gateway
<cryptix>
fiatjaf: depends on the content :P
<fiatjaf>
cryptix: how do you use this browser plugin? is there a browser plugin? how do you write a url in the html to make the content fetchable from the browser plugin?
<lgierth>
fiatjaf: you can even point to ipfs.io/
<fiatjaf>
my question is actually if the gateway will support everybody linking to it.
<lgierth>
will take about 30 mins for the dns update to propagate
inconshreveable has joined #ipfs
<cryptix>
rschulman_: yea :) i got lost in it too
<rschulman_>
hehe
<rschulman_>
I can see where the change needs to happen…
<benjaminjackman>
@cryptix do you know if people set an entry in /etc/hosts to redirect to 127.0.0.1 for ipfs.io rather than use the extension you listed
<RX14>
how ready is ipns?
<lgierth>
RX14: it doesn't have ordering yet, i.e. you can and will hit old objects. it's being worked on
<cryptix>
benjaminjackman: then you get into running the daemon on port 80.. :/ (not to speak of :443 and generating a cert for it..)
<cryptix>
lgierth: basically has to be (if it is avail in online and offline mode)
<lgierth>
right
<RX14>
umm, i don't trust whoever made the java library
<lgierth>
:)
<RX14>
they rolled their own json parser
<lgierth>
heh
<RX14>
when there are like 6
<RX14>
for java
<benjaminjackman>
RX14: no dependencies that way :) he's not the first person to do that
<RX14>
maven exists
<RX14>
for a reason
<RX14>
this is what's wrong with the java community
<RX14>
reinventing the wheel
<benjaminjackman>
what version of what json library should he use such that he will have 0 version conflicts and someone else won't complain about his terrible choice in a json library had he only used X instead
<RX14>
shading exists for a reason
<RX14>
now i'm off
<benjaminjackman>
answer that question then file an issue on his tracker
tjgillies__ has quit [Quit: Connection closed for inactivity]
chiui has quit [Ping timeout: 250 seconds]
Tv` has joined #ipfs
<vanila>
How would I create a list tht anyone can add to with ipfs?
<reit>
to make it work you'd need both your users and your server to run a custom ipfs build
<reit>
i've actually implemented such a system, and it works but it's not really ideal
<reit>
imo this is the critical hole in IPFS's current implementation, i don't see it properly deprecating http until there's a standard, supported way to send custom messages through the network to specific nodes
<benjaminjackman>
lgierth: from what you linked: "In the future, browsers will support IPFS content addressing directly, and not require HTTP at all, making this spec (and probably HTTP) obsolete."
<vanila>
couldn't it be a service built with ipfs rather than a custom?
<whyrusleeping>
reit: if you want to pin a bunch of stuff from someone else, just have it all in a dir, and pin that dir hash
<benjaminjackman>
lgierth: pretty optimistic :) but i agree
<lgierth>
benjaminjackman: well we're prepared for the case they don't ;) gateways + addons
<lgierth>
hrm i'm debugging the cache-control and etag headers
thomasreggi has quit [Remote host closed the connection]
thomasreggi has joined #ipfs
taneli has joined #ipfs
<taneli>
hi what does MaxSizeMB in Log section of config control?
<whyrusleeping>
taneli: we removed that recently, actually
<whyrusleeping>
so right now, nothing
<taneli>
it's still in 0.3.8-dev
<taneli>
but ok
<whyrusleeping>
yeah, i mean, we removed the stuff that it controlled
<whyrusleeping>
and forgot to remove the config key
<taneli>
so ... are there any limits on how much space ipfs uses to store data?
<taneli>
locally
<whyrusleeping>
currently no, but thats in the works
<whyrusleeping>
ipfs only stores data that you request
roerick has joined #ipfs
roerick is now known as gendale_
<whyrusleeping>
so if you request a 5GB file, its gonna store 5GB on your disk
patrickod has joined #ipfs
<taneli>
ok, great
<taneli>
is there any sort of gc for throwing out old files?
<taneli>
content i mena
<taneli>
i think i saw somewhere it uses leveldb
<whyrusleeping>
'ipfs repo gc'
<whyrusleeping>
will run a GC
<whyrusleeping>
we use leveldb for a few things, but all the data is stored in a flat filesystem similar to how git does it
<gendale_>
@all congrats on the HN feature
<whyrusleeping>
gendale_: thanks :)
<gendale_>
the write up is great as well.
<gendale_>
though I didn't quite understand exactly what neocities implemented
<gendale_>
are they publishing the hash of user websites? on a profile or something similar?
jamescarlyle has joined #ipfs
<gendale_>
ah, sorry, found it >> How we're using IPFS now
chiui has joined #ipfs
devbug has joined #ipfs
hellertime has joined #ipfs
adrusi has joined #ipfs
<wasabiiii>
Never used go before, having some issues figuring out how to get ipfs installed on my Ubuntu box. Have go installed. Have set GOPATH. Ran go get.... no ipfs in path. Can't find a binary. Not sure what I should be adding to PATH.
<vanila>
you should find ipfs in $GOPATH/bin/ I think
<cryptix>
wasabiiii: yup $GOPATH/bin
<wasabiiii>
$GOPATH has no bin. Just a pkg and src
<whyrusleeping>
wasabiiii: 'go get -u github.com/ipfs/go-ipfs/cmd/ipfs'
<wasabiiii>
That's doing something. What's the -u.
<whyrusleeping>
update
<whyrusleeping>
you can also try 'go install github.com/ipfs/go-ipfs/cmd/ipfs'
<wasabiiii>
First one got it.
<whyrusleeping>
sweet
<wasabiiii>
Guess I need to figure out what the hell this Go thing is.
chiui has quit [Ping timeout: 252 seconds]
chiui has joined #ipfs
devbug has quit [Read error: Connection reset by peer]
Eudaimonstro has joined #ipfs
Encrypt has quit [Quit: Eating time:*]
<williamcotton>
has anyone made a demo web application where a file is dragged and dropped on to the page and the IPFS content-addressable keys are generated and the file is submitted to a server that'll make the file available on IPFS?
<whyrusleeping>
williamcotton: take a look at the webui
captainbland has joined #ipfs
<williamcotton>
whyrusleeping: ok! where is that?
<whyrusleeping>
once youre running an ipfs daemon, navigate to localhost:5001/webui in your web browser
<popoffka>
hi guys. just learned about IPFS, looks awesome. is it OK if I ask a few basic questions?
Eudaimonstro has quit [Read error: Connection reset by peer]
<williamcotton>
QmRJzsvyCQyizr73Gmms8ZRtvNxmgqumxc2KUp71dfEmoj # sha256 in base58
<williamcotton>
dunno how I missed that the first time I read it
<whyrusleeping>
popoffka: of course, fire away
<popoffka>
so, if somebody requests a file I don't have, the request might be routed through my machine, and then it will cache the requested file, right?
<williamcotton>
ok, so roughly if I take a sha1 in 0x format, then append 0x12, then convert to base58, that's the "Q"
<williamcotton>
er, prepend
<williamcotton>
er, sha256
<williamcotton>
:P
<whyrusleeping>
popoffka: only content you request is placed on your machine
<popoffka>
whyrusleeping: oh, really? I see. so, files that are not accessed often are likely to be unavailable if their original hoster is down?
gkelly has joined #ipfs
jamescarlyle has quit [Remote host closed the connection]
<whyrusleeping>
popoffka: yeah, similar to torrents in that regard
<popoffka>
OK, that makes sense. still, can I configure the maximum size of the cache?
<whyrusleeping>
currently, no. but we have limits on the datastore size in progress
<popoffka>
gotcha. that answers all of the immediate questions I have, I'll go play with the client then, thanks for the great work :)
thomasreggi has quit [Remote host closed the connection]
<whyrusleeping>
popoffka: glad you like it!
thomasreggi has joined #ipfs
<wasabiiii>
So how do you guys envision this stuff working for file shares? Create a peer id for each logical file system one might want to use? Running as a different user? How should /ipfs,/ipns be made available to the entire system?
<wasabiiii>
How about writable file systems, integrating into POSIX user ids?
<drathir>
there was planned some new documents speaks if good remember?
<williamcotton>
mappum: yeah, I saw that! very nice :)
<williamcotton>
this is going to be very quick and painless to incorporate
<williamcotton>
hey, so does IPFS compute like SHA-1 and SHA-256 and support looking up content by either? like, do files have multiple content-addressable keys?
<williamcotton>
or does it default to the multihash sha256 in base58 for pretty much everything?
jamescarlyle has joined #ipfs
<mappum>
the latter, basically everything is sha256 in base58
<williamcotton>
k
<ipfsbot>
[go-ipfs] whyrusleeping created feat/unwant (+1 new commit): http://git.io/vZcrT
<ipfsbot>
go-ipfs/feat/unwant c38ca00 Jeromy: implement unwant command to remove blocks from wantlist...
<ipfsbot>
[go-ipfs] whyrusleeping opened pull request #1672: implement unwant command to remove blocks from wantlist (master...feat/unwant) http://git.io/vZcrR
deltab has joined #ipfs
<drathir>
i wonder if the hash of the same file at any of person node is the same?
<williamcotton>
yeah, the same bytes result in the same hash
<williamcotton>
drathir: I don't know much about the details of IPFS, I'm just talking about how these kind of cryptographic hash functions work :)
<vanila>
the same hash strongly implies the same file too
<whyrusleeping>
the only thing we have to worry about is the possibility that sha256 is broken
<drathir>
williamcotton: bc wonder if its possible when requested a file fetch it from the closest possible node and how much that behaviour will be secure...
<williamcotton>
collisions are one thing, having the collision mean anything is another
<williamcotton>
like, imagine you were serving up some javascript with sha-256 keys
atrapado has joined #ipfs
<williamcotton>
ok, you found a colision... but is it executable javascript?
<williamcotton>
and can you make a collision in to any kind meaningful javascript that could be used to do Bad Things (tm)?
<vanila>
a collision might produce a cycle in the merkle DAG? that could cause crashes/disk exaustion maybe
amstocker has joined #ipfs
<williamcotton>
dunno, I don't really know about that stuff :)
<williamcotton>
just talking about content-addressable systems in general
<mmuller>
williamcotton: it all depends on the security of the hashing algorithm.
<mmuller>
accidental collisions are astronomically unlikely for sha1 and sha2
<whyrusleeping>
a massive volcanic eruption could end the world next week too, crashing all the computers in the world :P
<mmuller>
it is theoretically possible to engineer a sha1 hash collision in less than brute force time
<mmuller>
but no such attack has ever been demonstrated
<williamcotton>
and I dare someone to find me a colliding md5 on a javascript file that would load up something that could steal a session token in a cookie :)
<vanila>
I think you can do that for js
<vanila>
sorry for md5*
<whyrusleeping>
williamcotton: yeah, md5 is pretty horribly broken
<williamcotton>
there are limits to that of course, right?
<williamcotton>
I mean, it could take a very long time to generate a hash to do something very specific
<williamcotton>
or is it just totally fucked? haha
<williamcotton>
I dunno, I'm definitely not advocating using md5 :)
<mmuller>
williamcotton: yes, where "long time" depends on the security of the hashing algo
<popoffka>
in MD5, you can easily generate two meaningful files that include some blocks of meaningless data, then "shuffle the meaningless data around" until the hashes match
<popoffka>
and because of MD5's weaknesses (incl. the length extension attack), this is easy
<popoffka>
SHA2, AFAIK, isn't susceptible to the length extension attack, so getting two meaningful files with the same hash should be _significantly_ harder
<rschulman_>
well, its been an interesting day in IPFS land.
<williamcotton>
popoffka: thanks for the clarification :)
<mmuller>
popoffka: yeah, sha1 neither - both include message length in the hash.
<vanila>
I think sha2 has a few modes like sha256, sha512 and sha512/224
akhavr has quit [Ping timeout: 250 seconds]
<vanila>
it's the / ones that you can't length extend (the ones thaht don't truncate include the whole state of the hash function, which is how you can extend)
* mmuller
just finished implementing sha2-256 and uncovering a bug in his sha1
<spikebike>
sha2-256 is very reasonable today, but I'd definitely want to include whatever possible to minimize the pain of moving to a different algorithm if a weakness is discovered
<mmuller>
spikebike: I expect that's part of the goal of multihash
<williamcotton>
what about two files that collide in both md5, sha-1, and sha-256?
<vanila>
but with multihash: If two people uploaded the same file under different algorithms - it wouldn't be deduplicated would it?
<popoffka>
hm, yeah, it seems like vanila is right: pure untruncated SHA2 is susceptible to length extension
akhavr has joined #ipfs
<williamcotton>
with bitstore we've been computing sha-1, sha-256 and btih... and soon base58 multi-hash sha256 ;)
<popoffka>
though any kinds of SHA2 collisions, meaningful or not, are still unfeasible, fortunately :)
border_ has quit [Ping timeout: 246 seconds]
border has joined #ipfs
<williamcotton>
I like btih and ipfs hashes because they're content-delivery related
<ogd>
pshh i already use sha3
<ogd>
jk im just trying to sound smart
<williamcotton>
:P
amstocker has quit [Ping timeout: 246 seconds]
<ogd>
though i will rock the FIPS202 one day
<williamcotton>
I'm a hash algorithm poseur, I'm just gonna follow the hot trends that someone else leads with ;)
<ogd>
well sha3 is standardized by nist now, which makes it pretty mainstream. i suggest you look for sha4 candidates and become an early adopter
<williamcotton>
I was at the Internet Archive a couple of weeks ago and asking them what they used but I couldn't find the right person to talk to
<vanila>
blake2b is realy good and doesn't permit length extention
<vanila>
skein is excellent as well
<whyrusleeping>
vanila: skein is soooo expensive though
<vanila>
I read it was very efficient, but that was in their own paper
<whyrusleeping>
our data input is limited by hashing speed
<williamcotton>
computing hashes in browser javascript is something I appreciate for UX purposes
jamescarlyle has joined #ipfs
<drathir>
whyrusleeping: the reasons using sha256 vs sha512 is speed of generating?
<williamcotton>
and there are size contraints to worry about as well, especially when you start embedding hashes in to things like the Bitcoin blockchain
<williamcotton>
which is definitely the world's most expensive database
<williamcotton>
speed, size, security
<taneli>
how do i make ipfs store stuff somewhere else than ~/.ipfs/ ?
<mmuller>
popoffka: yes, I think you're right, I was confused about LE attacks.
revolve has joined #ipfs
<spikebike>
williamcotton: ya, the global blockshain performance is worse than yo'd get on a single raspberry pi keeping a log
jamescarlyle has quit [Remote host closed the connection]
<williamcotton>
it was definitely not designed for performance :)
<williamcotton>
the best practice for interop would be to take as many different kinds of hashes as possible, and stay very future forward thinking
<williamcotton>
but that's really expensive
<popoffka>
taneli: idk if this is configurable, but you can always make a symlink
<williamcotton>
and you know what? if things break, and these systems fall about or die off... it'll be ok, just go pour yourself a lemonade and put on the latest Jimmy Buffet album
<taneli>
popffka: there is datastore.path but changing it makes no difference ... i guess symlink is the only way
<popoffka>
taneli: oh, right, that's weird. did you restart the daemon (if it's running) after changing it?
voxelot has quit [Ping timeout: 244 seconds]
<williamcotton>
has anyon deployed IPFS to a CoreOS cluster?
voxelot has joined #ipfs
Encrypt has joined #ipfs
Eudaimonstro has joined #ipfs
<drathir>
that mean if someone get QmdLirDCjVkLxcBRYeY6WiM5Ge98dP7DJL59KoTK1wMCd9 is able to download file?
<drathir>
or need have full path with ip?
<williamcotton>
drathir: as far as I know, that's the point! similar to using a BitTorrent InfoHash in a magnetic link
chiui has quit [Ping timeout: 244 seconds]
<drathir>
watchin videos i also think thats would be really great in connection with cjdns the delivery always the short possible path...
<drathir>
clearnet is slow...
<mikolalysenko>
what is the cheapest physical storage device on the market right now?
Bat`O has quit [Ping timeout: 246 seconds]
DrPete_ has joined #ipfs
<revolve>
drathir: have you tried implementing a DHT before?
Bat`O has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<drathir>
revolve: honestly im really fresh at ipfs, all the time discover and trying to better understanding how all workin together...
<drathir>
mikolalysenko: i guess 8tb frim segate...
<drathir>
the new ones...
chiui has joined #ipfs
<revolve>
ok basically the tradeoff I can see being made (not an ipfs dev but doing the same thing) is that you want your peers to know of the peers that /you/ know
<revolve>
and one of the best ways to do that is to make contact with a decent amount of peers generally
<taneli>
popoffka: no daemon, so something else was amiss
<revolve>
what you're trading is shortest-path-first for overall network availability for everyone
<taneli>
it works ok with a symlink
<drathir>
6-8tb if good remember, but warning are the cheapest, but also the higher possibility of failure on the hdd...
<drathir>
of the*
<drathir>
revolve: but if You connect with more ppl the traffic not incerase?
<popoffka>
btw, is there some convention for HTML links to IPFS content? they are easy to do with relative links from other IPFS content, but what about clearnet?
<drathir>
dramatically incerase i mean... bc of big amount of peers connected and data to keep that connections running sended?
<popoffka>
because we'd probably want to encourage people to run their own gateways
<popoffka>
which would be pretty useful if everybody linked to "gateway.ipfs.io"
<revolve>
drathir: the traffic only incerase when the polymerase transcription goes faulty
<williamcotton>
sort of like connecting to irc.freenode.net
<revolve>
drathir: s'just udp packets though. basic tenet of kademlia is to send the addresses of know peers in every message so the recipient can add any they don't recognise, then those peers also gain a new peer.
<drathir>
atm only have the default config connections...
<revolve>
so for every extra node you contact, the sturdier the routing table can be
<drathir>
thats interesting...
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<revolve>
should let multiple users collaborate on a hyperdocument and save revisions
<revolve>
should also be able to ask peers if they have data for the hash of a url if it's not available from its original source
<revolve>
brb
border__ has quit [Ping timeout: 268 seconds]
Eudaimonstro has quit [Ping timeout: 246 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<vijayee_>
does "ipfs object put" store merkledag objects? Does it store any kind of object? I assume not the unixfs objects
chiui has quit [Ping timeout: 246 seconds]
<revolve>
whyrusleeping: what's the scaling problem with tcp?
mildred has joined #ipfs
edrex has quit [Remote host closed the connection]
edrex has joined #ipfs
<benjaminjackman>
"should also be able to ask peers if they have data for the hash of a url if it's not available from its original source" has anyone tried changing this into a couple smaller problems, 1 having the role of a referrer, finding out who can satisfy a particular hash by providing the data (or some subset of it) in exchange for some 'credits', 2 actually providing the data, or some subset of it a (partial-)provider again charging 'c
<benjaminjackman>
i just say the bit torrent is an auction paper, maybe that covers some similar concepts?
<whyrusleeping>
benjaminjackman: have you read the whitepaper?
<revolve>
benjaminjackman: your quote relates to a different project, named uroko
<revolve>
and I'm the principal developer
pfraze has joined #ipfs
<benjaminjackman>
ok, a lot of this stuff has been bouncing around in my head for a while, i figured this was the place to find some people thinking down similar lines
<benjaminjackman>
whyrusleeping: the whitepaper on ipfs?
<revolve>
essentially no: because the resources themselves are fuel enough.
<revolve>
nodes have an altruism rating you assign them based on whether they either route you to good traffic or serve you good traffic. your altrusim will generally be positive because you've probably not modified your client code to perform bad routing, even if you're not sharing any revisions of anything
<benjaminjackman>
ok i didn't see that whitepaper yet i will read it and try to catch up some more
<revolve>
but if you want revisions from the dht, you kinda have to be on the dht, and you'll probably browse some stuff and store that, and you might even make some of it available to peers. totally opt-in and private by default
<benjaminjackman>
what about people that can only leech but have lots of cold hard cash?
<revolve>
IE you don't tell peers you have the data for things by default; someone can just do a rainbow table of urls and match the hashes you say you have content for
<revolve>
they make huge investments and can't have real friends
<benjaminjackman>
revolve: 'you don't tell them what you have by default' ok so that is to protect the nodes from advertising they have things they don't want other to know they have
<revolve>
yeah
rschulman_ has quit [Ping timeout: 250 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<benjaminjackman>
revolve: haha ok
<benjaminjackman>
revolve: so the idea would be that they maybe get a computer buddy to set something up for them?
ELFrederich has joined #ipfs
<ipfsbot>
[go-ipfs] DavidHowlett closed pull request #1619: ipfs get needed the -o flag to function as intended (master...master) http://git.io/vGYj7
<revolve>
oh also, yknow, the fact you got something from the dht means you're probably ok with relaying that to anyone else who asks you for that revision, too
<ELFrederich>
does ipfs have a concept of versions? Is it possible to link to the latest version of something?
Eudaimonstro has joined #ipfs
<benjaminjackman>
revolve: do you have a link for uroko
<revolve>
I'm actually working on the DHT at the moment
<revolve>
it's just a multiplayer caching proxy at the moment
<revolve>
yeah
pfraze has quit [Remote host closed the connection]
jamescarlyle has joined #ipfs
<benjaminjackman>
revolve: do you have more detail than the readme, or should i just check out the source?
<revolve>
for dht notes there may be a dht branch
<revolve>
contains the most recent code
<revolve>
I'm coming up against a bug in Gevent where things run twice if it's a green thread within a subprocess, and it's within the same address space
<revolve>
methods at one memory address running twice
kevmoo has joined #ipfs
mildred has quit [Ping timeout: 250 seconds]
<benjaminjackman>
revolve: yuck, that sounds like the class of bug using python shouldn't ever have
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<gendale_>
ELFrederich: check out IPNS, good to think about in similar fashion to git.
jamescarlyle has quit [Remote host closed the connection]
kevmoo has quit []
ygrek has quit [Ping timeout: 244 seconds]
<cojy>
is ipns working right now? i couldn't figure it out or find examples of it
jamescarlyle has joined #ipfs
<mmuller>
cojy: yes, at least to some extent. try 'ipfs name resolve QmRfvckFHNSTHFrVwRCd8svQ6mRm7dFXLYBwbUJVQQXHbY'
<cojy>
i get: Error: routing: not found
jamescarlyle has quit [Client Quit]
<whyrusleeping>
finally! working internet again!
<whyrusleeping>
cojy: ipns is not entirely 'working'
<whyrusleeping>
especially today, where i'm seeing a lot of network churn, records will drop out of the network fairly quickly
qgnox has joined #ipfs
arrdem has joined #ipfs
<arrdem>
o/
inconshreveable has joined #ipfs
dignifiedquire has joined #ipfs
<arrdem>
is there a good way to debug what nodes I'm connected to and the sate of the DHT? I'm trying to tap a desktop from a laptop on LAN and seeing kinda absurd resolve times
<spikebike>
ipfs swarm peers
<spikebike>
btw, kudos on the default IPv6 enabled stuff, I'm seeing more ipv6 peers
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has joined #ipfs
marianoguerra has joined #ipfs
<drathir>
spikebike: connect to hype also...
pfraze has quit [Remote host closed the connection]
<spikebike>
er, waht?
<voxelot>
ipv6 enabled will allow us to dial cjdns
<drathir>
btw pinning get deleted affter daemon reboot?
<voxelot>
shouldn't drathir
devbug has joined #ipfs
<drathir>
k need to test again, thanks voxelot for confirm...
<whyrusleeping>
resolve times are being pretty wonky today, our network prior to kyles article was around 100 nodes, and today we are seeing around 80 nodes of churn on top of that
<whyrusleeping>
so nodes are coming online, having dht info put to them, and then going offline
<whyrusleeping>
which makes resolves take longer
<revolve>
surely there's more than 100 ipfs nodes
vanila has quit [Quit: Leaving]
<spikebike>
My single node has 80 ipv4 and 25 ipv6
<drathir>
also wonder if real filenames in future will be possible to keep?
<spikebike>
(peers)
<whyrusleeping>
revolve: we havent been trying to grow the network very huge, so its stayed between 90 and 140
<whyrusleeping>
drathir: use 'ipfs add -w <thing>'
<revolve>
sweet
<whyrusleeping>
i see 202 peers online right now
<whyrusleeping>
212
<voxelot>
bout as many people as in chat, how do you see that number?
<whyrusleeping>
either 'ipfs diag net | grep connected | wc -l'
<voxelot>
ohh peer discovery happens fast, had only 4 when i booted my node up and just went up to like 30
<drathir>
whyrusleeping: oh used w/o -w thanks...
<whyrusleeping>
or you can run 'ipfs swarm peers | wc -l' on a highly connected node
<spikebike>
or on 2 and merge the results
<drathir>
what is a command to check peers ?
<drathir>
k
<drathir>
laag
<voxelot>
ipfs swarm peers
<voxelot>
hah
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<spikebike>
ipfs isn't ready to replace this irc channel... yet ;-)
<voxelot>
i'm spending a lot of time on peer discovery atm, trying to get bootstrap converted into node
<drathir>
ipfs swarm peers | wc -l
<drathir>
207
<drathir>
not so bad i guess...
<spikebike>
swarm peers will only show nodes in your routing Table, right?
<whyrusleeping>
spikebike: yes
<spikebike>
i.e. if there's 1M nodes it will be something like 8*logN(1M) or somesuch
<whyrusleeping>
yeeep
<spikebike>
cool, just checking
<whyrusleeping>
thats one of the issues we're seeing with tcp, lol
<spikebike>
and the command should be ipfs diag net | grep connections
<spikebike>
oops no
<spikebike>
nevermind
bjp has joined #ipfs
devbug has quit [Ping timeout: 264 seconds]
<spikebike>
estimating the population of large DHTs is a bit trickier of course.
<whyrusleeping>
diag net does a recursive traversal
<revolve>
so you're telling me it's possible to do a tcp dht over the internet?
<whyrusleeping>
well, i mean... yeah
<revolve>
without people having to explicitly forward their ports?
<whyrusleeping>
yeap
<voxelot>
isn't our k-bucket set to 20? i'm seeing 24-27 in my swarm, they are different?
marianoguerra has quit [Ping timeout: 246 seconds]
<whyrusleeping>
k-bucket is 20
<whyrusleeping>
doesnt mean you can only have 20 nodes in your table
<voxelot>
hmm i thought it would kick peers out if oyu went over, interesting
<whyrusleeping>
it might kick them out of their respective buckets
<voxelot>
ahh gotcha
<whyrusleeping>
but we dont immediately disconnect from a peer thats dropped from a bucket
<whyrusleeping>
they might have ongoing bitswap transfers or something
hyperbot has quit [Remote host closed the connection]
hyperbot has joined #ipfs
<voxelot>
so the swarm list is a combined k-buckets based, incase i need something from germany
<voxelot>
seeing a lot of german connections, woot
<whyrusleeping>
this is really fun, i get to see all the code we need to fix :P
<whyrusleeping>
too many open file descriptors up the butt
<drathir>
btw the dag from pinned items always throw 404...
<whyrusleeping>
drathir: yeah, i'm seeing that
notduncansmith has joined #ipfs
<rdbr>
could ipfs replace bittorrent?
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
rdbr: yeap!
<drathir>
rdbr: if able to check file as torrent do yea..
<drathir>
i mean partial download...
border has quit [Ping timeout: 240 seconds]
border_ has joined #ipfs
<rdbr>
hmm, then this could get really interesting..
<drathir>
eg have damaged iso file downloading them by torrent faster bc of few mega damaged than downloading the whole file once again...
* drathir
saved a lot of time on slow connection in that method... ;p
<rdbr>
I'm interested in my university's reaction
<rdbr>
they throttle bittorrent to a global limit of 10Mb/s
hellertime1 has joined #ipfs
hellertime has quit [Read error: Connection reset by peer]
<drathir>
rdbr: burnbit
<vijayee_>
whyrusleeping: can communicate or write to the dht from an ipfs service written in the style of your example?
<vijayee_>
is it a service you can connect a client that is not the node itself?
<whyrusleeping>
vijayee_: yeap! you can do anything an ipfs node does with that
<vijayee_>
I may be architecturally confuzzled but I might have used words good
<whyrusleeping>
but doing things that way you dont get the api and CLI interface
<vijayee_>
ok
<vijayee_>
I may have to experiment with this
<whyrusleeping>
yeah, let me know how that goes. I want to be able to support people using ipfs all sorts of different ways
<vijayee_>
I'm trying to create password authenticated accounts on top of ipfs
<vijayee_>
I was trying to see if I could write the account names and hashes to the dht
<rdbr>
drathir, I don't see how burnbit would help
<drathir>
rdbr: good for filess available in direct links on the networks, and with no server support of resume... You have sure file is fine, bc torrent verify the file...
cypher has joined #ipfs
dexus_ has joined #ipfs
dexus_ is now known as Andexus
<drathir>
btw that is interesting the hash of file + name is different than hash only file... but i guess it adding the extra information that why them are different...
<Andexus>
Hey, could anyone help me find out what's wrong when trying to run the ipfs webui? It says "Tell the author that this fails on your sysem: bygg serve"
<Andexus>
Actually, npm install also fails. instead of bygg serve it's "node intall.js"
<vijayee_>
whyrusleeping: Do you know what you dial to reach the dht service ? Perhaps... con, err := corenet.Dial(nd, target, "/ipfs/dht")
notduncansmith has joined #ipfs
inconshreveable has quit [Remote host closed the connection]
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has quit [Remote host closed the connection]
Andexus has quit [Quit: Lost terminal]
inconshreveable has joined #ipfs
edrex has quit [Read error: Connection reset by peer]
rschulman_ has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
pfraze has quit [Ping timeout: 268 seconds]
pfraze has joined #ipfs
atrapado has quit [Quit: Leaving]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
inconshreveable has quit [Read error: No route to host]
inconshreveable has joined #ipfs
Encrypt has quit [Quit: Sleeping time!]
domanic has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Eudaimonstro has quit [Remote host closed the connection]
rsynnest has joined #ipfs
<rsynnest>
hi all, I'm wondering how ipfs plans to handle server side scripting?
<rsynnest>
im a big fan of the concept, but had never thought of server side scripting until just now
<whyrusleeping>
rsynnest: ipfs is a content distribution protocol, it doesnt handle any sort of server side scripting in the current way thats done
<whyrusleeping>
there are a few different ways you could acheive similar things, one of which would be to use something like ethereum
<rsynnest>
right, but some of the reasoning for its use is to have a "permanent web", which doesn't really apply if the server side content isn't included
<drathir>
rsynnest: use cjdns in connection with ipfs...
<drathir>
You will get secure ipv6 access + boost of ipfs delivery data...
<drathir>
all in secure way...
<rsynnest>
regardless of secure filesharing, if I have a site that is serving content from a database, and I don't want that database content to be public, then my clients are still reliant on my specific server, and none of the distributed servers will be able to serve my server's db content
<rsynnest>
im unfamiliar with ethereum, what does it do that solves the server side content issue?
domanic has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
voldial has joined #ipfs
<pjz>
rsynnest: ethereum is blockchain computation
<pjz>
rsynnest: that is, distributed decentralized unblockable computation
pfraze has quit [Remote host closed the connection]
<drathir>
rsynnest: that mean database or webserver must be mirrored...
<rsynnest>
so essentially, the webserver/database is not distributed
<drathir>
not sure if there could help fuse support for database folder, bc that can broke things too...
<drathir>
ipfs rsync of database folder...
<drathir>
but that all black magic ideas...
<rsynnest>
right but that is a huge security hole. You wouldn't want clients hosting your passwords database for example.
<rsynnest>
you would need to own your own mirror
<rsynnest>
which is already how HTTP handles distribution, with load balancing and mirrored servers
<fiatjaf>
rsynnest: I think the use case you're thinking of is not one that ipfs is trying to fulfill at the moment.
<drathir>
with cjdns You can securly connect 2 locations w/o problem and sending pass in cleartext too...
<drathir>
fiatjaf: yea there are tools for that...
<blame>
if your passwords database is hashed and salted, and you hash the password server-side (to avoid replay attack) than what is the issue
<blame>
the only leaked data is the emails.
<rsynnest>
since the database is not distributed, you are still relying on a single server to use the website
<drathir>
loadbalancers database replications and otherthings for clustering purposes...
<rsynnest>
single point of failure
<fiatjaf>
rsynnest: to me, what you're saying is much like: "what? self-driving cars? that doesn't help me in any way, how are you going to handle roads? cars need roads."
<fiatjaf>
drathir: what do you mean?
<voxelot>
i'm using ipfs as a distributed database, clients hold their own data and serve it to who they choose
dignifiedquire has quit [Quit: dignifiedquire]
<voxelot>
ipfs could in theory replace the need for servers
<rsynnest>
voxelot: that's interesting, and it works with local storage systems and sites that are purely client-side
<voxelot>
sure, you can host client side javascript on ipfs nodes
<spikebike>
voxelot: sure, that's true of p2p in general
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<drathir>
fiatjaf: i mean that needed egsist solutions for that purposes clustering ones... w/o app support for cjdns is need to make new software which will be supporting power of ipfs...
<rsynnest>
right, that's what im saying, it's basically like vanilla bittorrent
<voxelot>
right, all spikebike, all p2p is a way of replacing servers, ipfs just gives us some nifty tools create a distributed web
<rsynnest>
so what are you saying that apps will be developed in accordance with the new architectures that wont rely on databases to serve user-specific content?
<spikebike>
voxelot: ya, and a distributed filesystem could be used by a p2p client to be more than a filesystem.
<spikebike>
access to connections to other peers, read/write to DHT, and a blob transfer system (forget the IPFS name for it), allows for easier to write p2p clients
<voxelot>
rsynnest: yeah, you can look at neocities, they are starting to host their sites on ipfs in the hopes that if their servers go down one day, the sites never will
<spikebike>
ah, access to bitswap is what I was trying to describe
<RJ2>
thanks
simonv3 has joined #ipfs
captainbland has quit [Ping timeout: 256 seconds]
<rsynnest>
voxelot: so if you have secure content, ie: passwords. how do you share that and allow other peers to host it without introducing a security hole? the peers would then be a middleman between your users and their passwords.
<rsynnest>
servers don't just serve static content. every webserver in the world uses server side code, that is what node.js, PHP, Tomcat, etc
<rsynnest>
i might just be stuck thinking in the current web architecture and cant see what you're saying
inconshreveable has quit [Remote host closed the connection]
<voxelot>
rsynnest: had a lot of the same questions when i first starting looking into this and some of the awnsers are still being built