<groxx>
I'm reading up on some of the fundamentals of ipfs, that's all going well. but I'm not finding much about ipns - is there a primary source for info on whatever the current state of ipns is?
<atgnag>
Is there a way to share a file and have it keep its extension?
<atgnag>
For example, when I shared a tarball, firefox just saw it as a BIN file (which basically means it has no idea what it is).
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<lgierth>
atgnag: try ipfs add -w, it wraps a file with a directory
<lgierth>
that's /ipfs/<hash>/file.ext then
<atgnag>
lgierth: Thanks.
<atgnag>
The directories don't cost anything, do they?
<lgierth>
it lacks validation of order, and permanence (i.e. the publishing node needs to be online)
<groxx>
lgierth: yeah, I found that one at least. it doesn't really describe _how_ it works though
<groxx>
too early for that maybe?
<groxx>
(though in that case it's a bit odd that it's part of the default install)
rschulman_ has quit [Quit: rschulman_]
rschulman_ has joined #ipfs
captainbland has quit [Ping timeout: 246 seconds]
rschulman_ has quit [Quit: rschulman_]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
joshbuddy has quit [Quit: joshbuddy]
joshbuddy has joined #ipfs
cSmith has quit [Changing host]
cSmith has joined #ipfs
sseagull has joined #ipfs
hellertime has joined #ipfs
Score_Under has joined #ipfs
<Score_Under>
The IPFS docs have broken links to all headings, because a <base> attribute causes them all to link to the homepage. https://ipfs.io/docs/commands/
anon35345 has joined #ipfs
<davidar>
pjz: sitemap.xml
<davidar>
atgnag: yeah, webui broken for me too recently
notduncansmith has quit [Read error: Connection reset by peer]
<davidar>
Blame: voxelot: trying to collect ideas here ^
slothbag has quit [Read error: Connection reset by peer]
tsenior has quit [Ping timeout: 244 seconds]
amstocker has joined #ipfs
<blame>
davidar: I got your im before I drove home and spent the drive thingking about ideas. Then I read the thread and found you had already written all of my ideas.
<blame>
I think the "static index" + js client search is a good idea. Do gossip based database merging. Make it a prifix based directory of json blobs, so you can just not bother changing unaltered blobs and keep them around via ipfs content addressing.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<blame>
the problem becomes how to do you get the "head node" to the client? we could make it a "special port 5001 thing" like the webui, where it gets provided the ipfs node's current index root.
<jbenet>
i think right now people will get stuck and close the window on registration. have to say why, and guide them or whatever
<jbenet>
<jbenet>
davidar: wanna enable issues/
<jbenet>
?
<jbenet>
lgierth: night!! miss you!
<davidar>
Matthew: any thoughts re registration issues?
<davidar>
jbenet, done
<davidar>
jbenet: you're in London right?
<achin>
hi all! does `ipfs add` have a way to just get the hash of a file, but not actually add it to ipfs? (like git-hash-object without the -w flag). this could be a way to see if a local piece of content has already been added to ipfs
<davidar>
achin: ipfs add --help
<trock>
why does webui not show connections at times... if I stop and restart daemon it usually starts working
<trock>
swarm peers shows many connections
<achin>
oh! --only-hash sorry, i missed that :( thanks davidar
<pinbot>
now pinning /ipfs/QmeC4bkqK4Bwc4RVwYPRDnYx4emCJ77Ke3ZgYqqSH9uH4j
<davidar>
jbenet: I think mapbox is somewhere near NYC? Feel like bugging them about ipfs integration? :)
<substack>
davidar: they're in DC
<jbenet>
davidar: email probably easier. i'm happy to take a trip down there if they want me to. Could also talk to libs + museums there
<jbenet>
hey substack o/ :) -- will review your RFCed spec in next day or so
<substack>
cool!
<pinbot>
[host 3] failed to grab refs for /ipfs/QmeC4bkqK4Bwc4RVwYPRDnYx4emCJ77Ke3ZgYqqSH9uH4j: Post http://[fc4e:5427:3cd0:cc4c:4770:25bb:a682:d06c]:5001/api/v0/refs?arg=/ipfs/QmeC4bkqK4Bwc4RVwYPRDnYx4emCJ77Ke3ZgYqqSH9uH4j&encoding=json&stream-channels=true&r=true&: dial tcp [fc4e:5427:3cd0:cc4c:4770:25bb:a682:d06c]:5001: connection timed out
<substack>
jbenet: already getting some good feedback from ssb folk
<substack>
jbenet: it would be good to see how this fits in with ipfs objects, since it treads over some of the same ground
<jbenet>
sounds good, will check it out
<davidar>
substack, I feel like the rest of the world has a very restricted definition of "near" ☺
<jbenet>
substack: btw, not sure if we've pushed the update to specs repo yet, but we're making the data model be "just json" (really cbor in the wire + disk, but json as a data model)
<davidar>
jbenet, hehe, OK. But how am I supposed to live vicariously when I can do email myself?
<pjz>
jbenet: have you looked at all the NDN literature?
<davidar>
jbenet: I think I've seen a few nice looking PDF.js based viewers floating around
<davidar>
jbenet: I'm also trying to get the IA book reader running on ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
groxx has quit [Quit: Connection closed for inactivity]
<jbenet>
davidar: haha fair enough. i dont mind visiting DC. it's not far, and it's full of nice buildings and museums. very nice ... if you can ignore the madness happening there.
nessence has quit [Remote host closed the connection]
<jbenet>
pjz: i've seen some. not all, i think Van Jacobson is right and i want to help him. it hinkt he way to do it is generate demand for NDN-style things by doing IPFS. replacing HTTP is way easier than IP. (chicken-egg problem in IP)
hellertime has quit [Quit: Leaving.]
Quiark has quit [Ping timeout: 255 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
davidar: that's great!
<davidar>
jbenet, my current problem is that browsers can't read jpeg2k
<davidar>
Oh wait, maybe they do, hmm...
siraj has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
siraj is now known as dudeman
dudeman is now known as dudeman21
dudeman21 is now known as amandashow
<davidar>
nope
<davidar>
dammit mozilla
amandashow has quit []
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
warptangent has quit [Quit: bah bah quit quit]
<ipfsbot>
[node-ipfs] RichardLitt force-pushed feature/improve-readme from 1fc1771 to 062ce63: http://git.io/vZKzg
<ipfsbot>
node-ipfs/feature/improve-readme 062ce63 Richard Littauer: Added a Contribute section, with small rewrites...
<ipfsbot>
[node-ipfs] RichardLitt pushed 1 new commit to master: http://git.io/vZKz2
<ipfsbot>
node-ipfs/master 52d2342 Richard Littauer: Merge pull request #16 from ipfs/feature/improve-readme...
<ipfsbot>
[node-ipfs] RichardLitt deleted feature/improve-readme at 062ce63: http://git.io/vZKzw
<jbenet>
Bat`O i cant recall a reason not to have it. probably just hasn't been added
<Bat`O>
jbenet: can you point me where it should be added ?
<jbenet>
go-ipfs/core/commands probably
<jbenet>
there's a listing of the commands in the API. may want to also ask whyrusleeping
tsenior has quit [Ping timeout: 250 seconds]
rschulman_ has quit [Quit: rschulman_]
rschulman_ has joined #ipfs
<jbenet>
cryptix lgierth: you're right, this should happen. it's been bugging me for a while.
<jbenet>
will get it for free with new keystore, because keys will be ipfs objects.
rschulman_ has quit [Client Quit]
<ion>
I added a directory using ipfs add -r, then added one of the files within the directory using the web UI. The latter action added the hash of the file to the list in the Files/Files tab and it persists over a restart. The x button doesn’t work because “indirect pins cannot be removed directly”.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Bat`O>
jbenet: repo gc via the api is working, i'm just stupid, nevermind
<Bat`O>
i had an api call already in my browser so i thought that the daemon was running
<Bat`O>
/o\
<ion>
Is anyone collecting IPFS statistics? It would be fun to see an estimate of the number of users over time, especially when its popularity grows.
<whyrusleeping>
g'mornin everyone
<noffle>
hey hey
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ansuz_ has joined #ipfs
ansuz has left #ipfs [#ipfs]
captainbland has quit [Ping timeout: 265 seconds]
taneli has quit [Quit: Lost terminal]
ansuz_ is now known as ansuz
Eudaimonstro has joined #ipfs
captain_morgan has quit [Ping timeout: 244 seconds]
harlan_ has joined #ipfs
<substack>
html version 3.0.0 QmRQgTikDCJk8PfuoCRrH25dHxkSr9jnfop9ZqJbVkFf29
<achin>
"ipfs diag net" shows there are about 215 nodes in the network. this is smaller than i expected. is this actually the full size of the network?
<whyrusleeping>
achin: yep. that sounds about right
<achin>
cool
dignifiedquire has joined #ipfs
<whyrusleeping>
yeah, we havent been focusing too much on growing the number of nodes in the network yet
<achin>
understandable
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<achin>
but when i first saw ipfw i was "omg cool i want", so i'm surprised others didn't think the same :)
<whyrusleeping>
achin: glad to hear it :) and others will come, we just have to make things even more awesome
hjoest3 has quit [Read error: Connection reset by peer]
hjoest has joined #ipfs
<ianopolous>
Hey guys, I'm trying to understand the resulting size of data in the ipfs datastore. After clearing it, I add a 100MiB file and the disk usage in the datastore spikes to 400MiB, then after a minute it presumably does a gc and drops to 105MiB. Any idea what all those temporary files are?
<whyrusleeping>
ianopolous: i'm actually not sure... thats strange. can you tell which directories the files are in?
<whyrusleeping>
ianopolous: ah, yeah. youre going to want to update
<whyrusleeping>
thats leveldb being fat, we've fixed things since
<ianopolous>
ah righto thanks
deltab has joined #ipfs
notduncansmith has joined #ipfs
hjoest has quit [Read error: Connection reset by peer]
notduncansmith has quit [Read error: Connection reset by peer]
<ianopolous>
whyrusleeping, that's the version that this gets me: go get -u github.com/ipfs/go-ipfs/cmd/ipfs
ghutzriop has joined #ipfs
<achin>
maybe it's being installed into an unexpected GOPATH?
<ion>
Is it possible to get progress output from an ipfs pin add -r <hash>?
<ianopolous>
yep got an older version installed somewhere else.
hjoest has joined #ipfs
<whyrusleeping>
ion: that would be nice. we should add that
<achin>
another idea: add the ability to "tag" hashes. a hash->string map that would, for example, allow me to assign a human-readable description to things that i've pinned
<whyrusleeping>
achin: yep, would love that too
edrex has quit [Ping timeout: 260 seconds]
edrex has joined #ipfs
<ghutzriop>
hi, I ignored ipfs for a while on my server and "ipfs swarm peers" is now running for a while. I don't even know if there are still open connections. the webui shows 0 and respons quickly
Spinnaker has quit [Ping timeout: 252 seconds]
<whyrusleeping>
ghutzriop: i've noticed that too on occasion
<whyrusleeping>
ipfs swarm peers will sometimes take a long time, and then return a bunch of peers, some of them without IDs
<ghutzriop>
whyrusleeping, looks like a bug that needs to be adressed in the near future
<nomoremoney>
hey! I managed to build a small network using every peer as a server/client with their own storage that other users can "connect" to. I'm wondering about some ways for peers to find each other "randomly" or another way. Anyone have any ideas how to make this happen?
<ghutzriop>
whyrusleeping, probably. thanks for making the bug report, so I don't have to do it :)
<Vyl>
'Small network?'
<nomoremoney>
Vyl, you'll have to know the ID of the peer and manually enter it. The network being what's interfaced via the GUI
<ion>
While i’m pinning a directory tree seeded by achin, i simultaneously have a lot of traffic from and to {mercury,neptune,uranus}.i.ipfs.io. I’m curious as to what’s being transferred.
<whyrusleeping>
nomoremoney: if youre conncted to other nodes on the network, you can just try and retrieve ipfs objects that dont exist
<whyrusleeping>
it will have the side effect of connecting to a bunch of different peers trying to find the thing
<nomoremoney>
@whyrusleeping so basically checking the connected peers across ipfs?
<nomoremoney>
yeah, I don't want to create any negative effect
<jbenet>
nomoremoney: they should connect to each other automatically
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<nomoremoney>
@jbenet, sure, but I'm not sure that peer actually has the "profile" file or not, so I'll need to make sure it exists
<Vyl>
Just finding my place in the conversation.
<ianopolous>
if I add a file, then pin rm -r it, then repo gc, should all the blocks for that file be gone? (and the resulting size of the blockstore)
pfraze has quit [Remote host closed the connection]
ab14 has joined #ipfs
captain_morgan has joined #ipfs
pfraze has joined #ipfs
mildred has joined #ipfs
hjoest2 has joined #ipfs
hjoest has quit [Read error: Connection reset by peer]
captain_morgan has quit [Ping timeout: 272 seconds]
<dignifiedquire>
jbenet: I should have some time in the coming week to continue on the design for electron-app
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
captain_morgan has joined #ipfs
hjoest2 has quit [Read error: Connection reset by peer]
hjoest has joined #ipfs
chriscool has quit [Ping timeout: 240 seconds]
rendar has quit [Ping timeout: 264 seconds]
jfis has joined #ipfs
mondkalbantrieb has quit [Quit: No Ping reply in 180 seconds.]
mondkalbantrieb has joined #ipfs
captainbland has quit [Ping timeout: 264 seconds]
rendar has joined #ipfs
captain_morgan has quit [Ping timeout: 252 seconds]
<whyrusleeping>
ab14: there isnt an index, but there is a rather large amount of content floating around
edrex has quit [Ping timeout: 250 seconds]
notduncansmith has joined #ipfs
<ab14>
whyrusleeping: ok, is there a way to stumble upon it? So far all I can do is putting something into ipfs and view it on a public gateway - I want more to play with :)
notduncansmith has quit [Read error: Connection reset by peer]
edrex has joined #ipfs
pfraze has quit [Remote host closed the connection]
tsenior has quit [Ping timeout: 265 seconds]
<whyrusleeping>
ab14: yeah, its not super easy to just stumble upon content yet, we should really put effort towards making an index or something
pfraze has joined #ipfs
<whyrusleeping>
(sooooo many things to do!)
<achin>
that could reasonabley be delagated to the community. maybe start with a github wiki page or something?
Tv` has joined #ipfs
<whyrusleeping>
achin: good idea
<Vyl>
Do we have much content to index?
<Vyl>
Chicken-and-egg issue here.
sharky has quit [Quit: Leaving]
hjoest2 has joined #ipfs
<Gaboose>
<whyrusleeping> ab14: there isnt an index, but there is a rather large amount of content floating around
<Gaboose>
Vyl ^
<Vyl>
What sort of content?
hjoest has quit [Ping timeout: 264 seconds]
amstocker has quit [Ping timeout: 246 seconds]
<Vyl>
I've seen freenet. Lots of content there too, most of it rather disreputable. One cannot build a network on porn and piracy alone. Well, one actually could, but it's not going to go mainstream easily.
mildred has joined #ipfs
<ansuz>
actually, you totally can
<ansuz>
it's called "the internet"
<ansuz>
welcome
<voxelot>
the more we use the gateways the better idea we have of content out there
<voxelot>
is there an easy way to browse the refs that are caching on my server?
<voxelot>
could start indexing that stuff?
rschulman_ has joined #ipfs
<whyrusleeping>
voxelot: 'ipfs refs local' shows all hashes local to a machine
<voxelot>
yeah that's what i look at, but to see the content i have to look at each individually
<voxelot>
most just end up downlaoding straigh to my OS lol, that can be scary
<whyrusleeping>
voxelot: yeah, its a little weird that way
<ab14>
i think i'll set up a python script, basically "ipfs cat > thisfile" for each in "ipfs refs local", then "file thisfile" and go from the determined filetype (i just got some random png image in a quick test)
<voxelot>
is there a way to see the files in the datastore not just the refs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
ab14: that would be super cool! and i would love to see the script when youre done so others can use it
<ion>
I’m pin add -r’ing a 262-megabyte directory tree shared by achin and not doing anything else with ipfs. The size of (initially more or less empty) ~/.ipfs/blocks has grown to 83 MB while the ipfs daemon has downloaded 689 MB according to ipfs stats bw, and the pin command is still running. Is this kind of behavior to be expected?
<whyrusleeping>
voxelot: $IPFS_PATH/blocks/***
<ion>
I have been seeing a lot of traffic to and from *.i.ipfs.io meanwhile.
pfraze has quit [Remote host closed the connection]
<whyrusleeping>
ion: yeeep, those are the gateways
patcon has joined #ipfs
rschulman_ has quit [Quit: rschulman_]
Encrypt has joined #ipfs
<ion>
I mean, what else is my node doing if only 12 % of the bytes downloaded are part of the thing i asked it to pin?
<lgierth>
dht maintenance, queries from other nodes
rschulman_ has joined #ipfs
amstocker has joined #ipfs
<Vyl>
That's too much.
<voxelot>
if we recursively cat over our refs and log the output could we parse that for anything searchable?
<voxelot>
would love to see your script too ab14
<voxelot>
maybe host a site on ipfs that outputs file types and gateways to the content
amstocker has quit [Ping timeout: 244 seconds]
<ipfsbot>
[go-ipfs] whyrusleeping created fix/ipns-fuse-tests (+1 new commit): http://git.io/vZPW7
<ipfsbot>
go-ipfs/fix/ipns-fuse-tests 192ee64 Jeromy: dont use coremock for fuse tests...
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
rschulman_ has quit [Quit: rschulman_]
hjoest has joined #ipfs
hjoest2 has quit [Ping timeout: 264 seconds]
captainbland has joined #ipfs
<ipfsbot>
[go-ipfs] whyrusleeping opened pull request #1695: dont use coremock for fuse tests (master...fix/ipns-fuse-tests) http://git.io/vZP4E
_Vi has joined #ipfs
rschulman_ has joined #ipfs
atrapado has joined #ipfs
<whyrusleeping>
Blame: i love the background of your cachewarmer
<lgierth>
well it eats 70% cpu on my little laptop
<lgierth>
= 1.5 cores
<whyrusleeping>
lol
<larsks>
Is it possible to create a "directory" in ipfs by passing in an explicit list of names/hashes (rather than submitting a multipart archive containing file content)?
rschulman_ has quit [Client Quit]
<achin>
you can also create a direcdtory with "ipfs add -r"
<ion>
My ipfs daemon seems to be consistently uploading at 4× the download rate and the rate graph looks exactly the same for both apart from the scale. It’s almost as if my download rate is artificially limited, while my upload rate saturates my upstream bandwidth.
pfraze has joined #ipfs
tsenior has joined #ipfs
rschulman_ has joined #ipfs
notduncansmith has joined #ipfs
hjoest has quit [Read error: Connection reset by peer]
notduncansmith has quit [Read error: Connection reset by peer]
<ion>
I have been downloading a 262-megabyte thing from achin for a bit over 2 hours and it’s still not done. Meanwhile the daemon has downloaded over 800 megabytes according to ipfs stats bw.
rschulman_ has quit [Client Quit]
captainbland has quit [Ping timeout: 264 seconds]
rschulman_ has joined #ipfs
_Vi has quit [Ping timeout: 268 seconds]
_Vi has joined #ipfs
harlan_ has quit [Quit: Connection closed for inactivity]
tsenior` has joined #ipfs
<Vyl>
Clearly this is going to need addressing.
tsenior has quit [Ping timeout: 272 seconds]
CounterPillow has joined #ipfs
<CounterPillow>
Let's be honest here guys, porn is THE killer app for ipfs.
warner has quit [Quit: ERC (IRC client for Emacs 24.5.1)]
hjoest has joined #ipfs
chriscool has joined #ipfs
edrex has quit [Remote host closed the connection]
edrex has joined #ipfs
_Vi has quit [Ping timeout: 255 seconds]
<ab14>
whyrusleeping, voxelot: here's some quick hackery: it creates a folder and puts the local refs in it sorted by filetype (and tries to put the proper extension on it). no indexing so far, but it's very easy to browse the files https://gist.github.com/schuhumi/fce924289aea23d88144
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<voxelot>
ab14: nice! i'm going to make a quik site that indexes by files type and hosts back the content
<ab14>
i just noticed a problem: big files get split up, they need to be put toghether again to view them (i have a image and a video that's only partly there). i think the script needs to follow the links recursively as well..
<blame>
whyrusleeping: blame voxelot . I was going to use my GoL background, but I think it is too noisy.
mildred has quit [Ping timeout: 246 seconds]
<lgierth>
70 % cpu *whistles innocently*
<blame>
lgierth: try blamestross.com I've not tested the CPU usage of that
<lgierth>
that one's fine actually
<lgierth>
maybe 5%
<lgierth>
less redraws
<lgierth>
*fewer
<blame>
I might see how it looks this eve.
<blame>
I have the scale and speed set a lot bigger and slower than I know low CPU machines can handle
notduncansmith has joined #ipfs
therealplato has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
_Vi has joined #ipfs
hjoest has quit [Ping timeout: 264 seconds]
voxelot has quit [Ping timeout: 250 seconds]
_Vi has quit [Ping timeout: 255 seconds]
hjoest has joined #ipfs
rschulman_ has quit [Quit: rschulman_]
<Gaboose>
ab14: ipfs cat must already follow the links recursively if you give it the root ref, right?
<Gaboose>
even if you don't have all the blocks locally, but they're still somewhere in ipfs network
nessence has joined #ipfs
bedeho has joined #ipfs
_Vi has joined #ipfs
chriscool has quit [Quit: Leaving.]
chriscool has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
tsenior` has quit [Ping timeout: 252 seconds]
_Vi has quit [Ping timeout: 255 seconds]
rschulman_ has joined #ipfs
<ab14>
Gaboose: hmm, take this as example: "ipfs cat QmWQc2ttwSQcu98ZkZMWCQpVDo1d1KecQRfqp3fjuyySHV > file.jpg" how to get the rest of the image?
tsenior` has joined #ipfs
chriscool has quit [Ping timeout: 246 seconds]
<achin>
if that is just 1 block of the image, you need to know its parent
domanic has joined #ipfs
<ab14>
achin: but that's not possible if i recall correctly, right?
<achin>
right
<ab14>
danged, that makes indexing a lot harder..
<Gaboose>
if one of the refs you're iterating over *is* its parent, then you can ignore this first block
<Gaboose>
ofc knowing when to ignore sounds hard to me
ghutzriop has quit [Ping timeout: 246 seconds]
<ab14>
because to be certain that it can be ignored you'd have to find it's parent.. but nodes that have that block of the image should be likely to have the parent as well, because how would that image block end up on the node without someone requesting the whole image ..?
<Gaboose>
bitswap strategies, sometimes they help cache rare blocks, at least in theory according to the white paper
<Gaboose>
or you can ignore the block if there's a way to find that the file is corrupt
<Gaboose>
i.e. not full
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Gaboose>
maybe 'ipfs object stat' command can help?
<Gaboose>
if NumLinks == 0 and BlockSize == 262158, then it's a full size leaf block
<Gaboose>
and probably part of a bigger split file
pfraze has quit [Remote host closed the connection]
magneto1 has quit [Ping timeout: 268 seconds]
tsenior` has quit [Ping timeout: 264 seconds]
<Gaboose>
but then, NumLinks == 0 and BlockSize < 262158 can be either a full small file or the last block of a big file
<Gaboose>
so i dunno
<ab14>
that's pretty neat! determining whether a file is complete or not would probably be hopeless, maybe when restricting the test to very few popular filetypes (but for e.g. txt it would be completely impossible)
pfraze has joined #ipfs
nessence has quit [Remote host closed the connection]
voxelot has joined #ipfs
<ab14>
the size trick could really work, QmfEAJwzneXomMdkVYRRj1AE4DgL7mQt7GG1qMYnEox9AG is the first block of a webm file and it has the exact same properties
rschulman_ has quit [Quit: rschulman_]
nessence has joined #ipfs
<ab14>
and to differentiate between full small file and last block of a big file one could use the file command in that when it is the last block of a big file the header is missing and the filetype can't be determined (except for things like txt)
<Gaboose>
with enough filters like that you could sort of, almost be left with the more interesting files :)
atrapado has quit [Quit: Leaving]
<ab14>
yup, it could really work out. one question though: what does the parent of a e.g. big image look like? i learned about ipfs yesterday so not all things are quite clear yet :P
<ab14>
it has to have a pointer to the first block, but that can't be about it, right?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Gaboose>
it often has pointers to all blocks
<Gaboose>
in theory it could have a list of pointers to lists of pointers
carstn has joined #ipfs
<Gaboose>
this one's a full large image: ipfs object stat QmS8Zs9PiK1iFBUtW4P9FowrfMFWaux85L1jSJE3KGJwNf
<Gaboose>
you seem to be learning quickly :) it's two months for me
<ab14>
thanks :) it just occured to me that i could simply try it out ... - and it looks exactly like you described
<ab14>
[simon@hptablet tmp]$ ipfs ls QmcaJWfXsxnBbT6yxNJfsp7Uu1DfvY7bCMANRYhQWMZ5MG
<ab14>
wallpaper of the swiss alps if you want one xD
Quiark has joined #ipfs
<Gaboose>
pretty
<ab14>
for indexing that's pretty neat actually, because when the parent sums up a very big file (like a movie or big image), the indexing server only has to pull the first block and read the header from that
<Vyl>
Nifty.
<Vyl>
The design looks sound, but there seems to be a lot of overhead judging by those talking earlier.
Guest63066 was kicked from #ipfs by jbenet [Guest63066]
Guest63066 has joined #ipfs
<Guest63066>
too long :/
<karissa>
thats a new one
<Rylee>
me too, thanks
<Guest63066>
too many people :/
<ogd>
Guest63066: please dont do that
<pfraze>
whats going on
Guest63066 was banned on #ipfs by jbenet [Guest63066!*@*]
<Luzifer>
O_o
<__uguu__>
huh ?_?
<gwillen>
thanks jbenet
<jbenet>
sorry everyone.
<cleichner>
jbenet: <3
<mappum>
those attacks happen a lot in some channels, i think it's a milestone that now the #ipfs channel is big enough to be a target :)
<CounterPillow>
Congratulations on just pissing off everyone in here Guest63066
<rschulman_>
mappum: Was just thinking the same thing. :)
<gwillen>
just ignore it
<jbenet>
hi everyone!
<gwillen>
it's a standard trolling technique
<yoshuawuyts>
\o/
<gwillen>
ban and move on
<jbenet>
it's like a big party now
<rschulman_>
ooh, a nick spammer, we must be important
<mappum>
now that you're all here...
<pfraze>
one of these channel members is the murderer
<Luzifer>
jbenet: kick is missing after the ban ;)
<jbenet>
"i've gathered you here tonight for ..."
<CounterPillow>
>mIRC
<CounterPillow>
heh
Guest63066 was kicked from #ipfs by jbenet [ah, right!]
<eternaleye>
I think they were actually trying to get attention, hoping someone they hilighted would respond.
<Luzifer>
compleatang has quit [Quit: Leaving.]
<CounterPillow>
Someone from UTC+1 using mIRC (thus, Windows)
<eternaleye>
The "too many people" thing looked like they were actually stupid enough to think that would _work out positively_
<ogd>
jbenet: hey re twitter: whats your current favorite file archive format that supports random access
<Rylee>
CounterPillow: could be mirc in wine
<eternaleye>
Which I have seen before...
<Rylee>
it's something that people do
<jbenet>
ogd: oh, car.
magneto1 has joined #ipfs
<jbenet>
our own, we're making it now
<eternaleye>
jbenet: Any docs to link?
<pfraze>
ogd: maybe that car thing will be something
<eternaleye>
jbenet: Since I've actually got some bits and pieces of such a design laying around too.
<ogd>
jbenet: is the design done?
<jbenet>
ogd: not finalized, working on the index atm.
<jbenet>
i want random access
<jbenet>
and verification (+ easy sigs in the front)
<ogd>
jbenet: yea those are the 2 features i want too
<mappum>
jbenet: are we calling it multifile?
<jbenet>
ogd: we're making it be json-compatible too, so store abitrary merkle-json (( ipld ))
<ogd>
jbenet: but i was thinking it could be done if you use the first entry in a tar and use it as the index, then just update that whenever you update any other entries
<ogd>
jbenet: then its still a tar
<jbenet>
ogd: some thoughts here https://github.com/ipfs/archive-format/issues but that's only a small part, rest is scattered all over github, and the logs of this channel. i'll gather it together soon we've just been busy
<jbenet>
ogd: nah, i'm ok with being 1:1 compatible with tar
<jbenet>
ogd: so: cat tarx <file.car >file.tar
<jbenet>
car*
<jbenet>
ogd: so: car tarx <file.car >file.tar
<ogd>
jbenet: i'd have to see the reason why a new format is necessary before i agree :)
magneto1 has quit [Ping timeout: 264 seconds]
compleatang has joined #ipfs
<jbenet>
meh, i have them. you can read about them when i write the doc
<ogd>
ok lol
<jbenet>
i just dont have time to convince people anymore :)
<Vyl>
Archive formats. Fun. I've written one before... it was pretty ugly. Not ideal for this though.
<jbenet>
but anyway, when do you need this?
GUEST666 has joined #ipfs
<GUEST666>
?
<ogd>
jbenet: no deadline, its for the container stuff we're doing (to exchange filesystem data via files)
<jbenet>
GUEST666 if you spam once more, i'll ban your whole "*!*@dynamic.rabatp1-76-57-137-41.wanamaroc.com"
notduncansmith has joined #ipfs
<GUEST666>
yes u can.
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
ogd: yeah, same here. btw, if you want just a straight merkle-tar, we put one into ipfs recently to merkelize docker images
<jbenet>
(dedup within docker tars)
<GUEST666>
interplanetary ??
<GUEST666>
hummm...
<jbenet>
ogd: ok if you give me a few days you'll have a doc, i owe it to someone else anyway
<GUEST666>
!run url
<M-hash>
"merkelize docker images"?
<jbenet>
M-hash gave a talk about it, it'll be out soon
<ogd>
jbenet: i think we'll start with tar and keep our options open for the future, we aren't trying to solve as many problems as you yet
<ogd>
jbenet: just need something simple to ship now
<ogd>
jbenet: but i'll watch that archive repo so i can get updates
hjoest has quit [Ping timeout: 264 seconds]
<jbenet>
ogd: ok, i'll ping also when it's good to review
GUEST666 has left #ipfs [#ipfs]
ghutzriop has joined #ipfs
hjoest has joined #ipfs
dignifiedquire has quit [Quit: dignifiedquire]
rjeli has quit [Excess Flood]
<Gaboose>
jbenet: hi, do you spend any time working on filecoin these days?
rjeli has joined #ipfs
<Gaboose>
there are some things in the design that were bugging me
<karissa>
damn after that guy did that I had to restart my browser
<jbenet>
btw, there's a way to get protobuf-level efficiency with a cbor stream. it's not standard, but im going to propose it to them later this year, so hence we dont care about storing stuff as cbor instead of protobuf.
erickl has joined #ipfs
<jbenet>
(i.e. cbor can be just as compact. we just need to write one tool, and we're ok paying the small cost before we do that (and probably not high with gzip)