<whyrusleeping>
rschulman: no, its a different protobuf, just decode it
<rschulman>
ohhhh
<rschulman>
which .proto file?
<whyrusleeping>
rabble: i want mobile stuff to happen, but i think ipfs might be a little too resource hungry right now
<whyrusleeping>
by all means try it and post your results somewhere!
<whyrusleeping>
(also, link to the android one?)
<rschulman>
whyrusleeping: why does a hash that I created by doing echo “blah” | ipfs add have unixfs data in it?
<whyrusleeping>
because 'add' is for add files
<pjz>
rschulman: what unixfs data?
<whyrusleeping>
adding*
<rschulman>
whyrusleeping: Well, that would make sense then, wouldn’t it?
<whyrusleeping>
rschulman: only sometimes
<whyrusleeping>
we need to put some data in there to make it possible for us to do directories vs files and file chunking layours
<rschulman>
whyrusleeping: it would be nice
<whyrusleeping>
it *is* nice ;)
voxelot has quit [Ping timeout: 244 seconds]
Leer10 has quit [Ping timeout: 272 seconds]
Leer10 has joined #ipfs
<whyrusleeping>
jbenet: can you give me an example use case for code that would call the Next() (io.Reader, error) method?
<whyrusleeping>
it doesnt work well anywhere in our codebase it seems
<whyrusleeping>
because everywhere i would use it, we just end up call ioutil.ReadAll() on the reader (which overallocates memory :( ) and putting the bytes into a dag.Node to be written to disk
<jbenet>
daviddias: havent looked at it after whyrusleeping's vinyl fix
<jbenet>
daviddias i can look now
<daviddias>
that would be nice, thank you :)
<jbenet>
daviddias: i think i found another bug in the dht test -- will update the pr momentarily
<M-davidar>
Is this the right place to ask (possibly stupid :) questions about ipfs?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
M-davidar yeah
<daviddias>
M-davidar: every question is welcome :)
<jbenet>
hey daviddias if users do not supply a cb, is it always set to noop? or should it throw (programmer error)?
<jbenet>
M-davidar: yeah, what daviddias said :) -- and, no question is stupid. we have a conceptual space of epic proportions
<jbenet>
so it's hard to understand
<jbenet>
(or rather, not as easy as i would like it to be, up front)
<daviddias>
well, you decide how your function works, if there is some async call inside your function, throwing an error won't work (or do any good)
<daviddias>
I typically just `noop
<daviddias>
noop' it
<daviddias>
and that is what I've seen the most, following the callback pattern
<jbenet>
if there's no cb, then it will eventually throw, when it returns. which is at least _some_ feedback
<jbenet>
(want to avoid the case of "why isnt this working???" when an err is silenced by a noop we insert
<M-davidar>
I've been thinking about trying to mirror citeseerx.ist.psu.edu to ipfs, and was interested in whether anyone has any thoughts as to how feasible that would be?
<jbenet>
M-davidar: very feasible! are you part of citeseerx?
<daviddias>
jbenet definitely agree, but encouraging the use of the callback pattern is not a bad thing
<jbenet>
daviddias: oh i entirely agree, -- i just mean whether we should throw or not if the user supplied cb or not.
<jbenet>
(not = silent error, yes = a bit annoying if you have to pass a noop. though i guess it puts you fully in control, so that's more understandable)
<M-davidar>
jbenet: No, just a user afraid of it disappearing at some point in the future :)
<jbenet>
M-davidar: i totally agree with you. i think this should be done with their permission though. remember what happened to aaron swartz.
voxelot has quit [Ping timeout: 252 seconds]
<daviddias>
ah, got it, the throw if the user doesn't supply a `cb`. If the `cb` has a `resuls` arg, so definitely throw, otherwise the user will never even get the results of calling that func, if the cb has a `err` field, I guess is debatable if it should throw, but do agree that it is good to remember the user to handle error, if the `cb` has no arguments and only
<daviddias>
tells the user when something is 'ready', then I guess there is no need to throw.
<jbenet>
M-davidar: i'm working on a "dead-org's switch" post about how publicly funded orgs who -- for whatever reason cannot liberate content during their lifetime -- to setup external enxrypted caches and a dead-org switch which releases if the org dies (i.e. an M:N sig requirement to reconstruct a key, etc).
<jbenet>
M-davidar: hopefully, orgs can see that that allows them to at the very least setup distributed replicas/backups even if their present-day copyright-laden legal framework does not let them make the raw content be generally available to the general public (funders)
<jbenet>
M-davidar: Awesome!!! \o/
<jbenet>
M-davidar: we should make an "Open Data Compliance" index of all the research artifact repositories.
kbala has quit [Quit: Connection closed for inactivity]
<Luzifer>
jbenet: works. tagged commits are now buildable even if they are not master/HEAD
<jbenet>
M-davidar: one big listing that scores the repository (maybe gives it a color {green, yellow, red}) -- like http://iswebrtcreadyyet.com/
notduncansmith has joined #ipfs
<jbenet>
M-davidar: ok since that's good, proceed with your Qs :)
notduncansmith has quit [Read error: Connection reset by peer]
<M-davidar>
jbenet: like opendoar.org ?
<jbenet>
M-davidar dont see the sort of thing im looking for there, but that's an org that might be able to provide a lot of the required data
<M-davidar>
jbenet: Anyway, my main question is, given that I don't have spare terabytes of disk space lying around, does ipfs have any kind of infrastructure set up that people could donate disk space towards mirroring public resources like this?
<daviddias>
oh, that one even fails sooner
<daviddias>
let me take a look
<jbenet>
daviddias: looks like the res array is empty then.
<jbenet>
(which may be a dht query finishing and returning no resulsts? not sure on the semantics there)
<jbenet>
M-davidar: this is a much requested feature-- we're specing out how "clustering" ipfs nodes will work, so that they gather together their available disk space and distribute pieces.
<jbenet>
M-davidar: this is a protocol _on top of_ ipfs core, that's non trivial in itself, so will take some time to figure out.
<jbenet>
M-davidar: but kyledrake is also interested in this so maybe can find stopgaps before.
jedahan has joined #ipfs
<daviddias>
jbenet: yep, you are right.
<jbenet>
daviddias: know of a script to setup irc reporting on github?
<M-davidar>
jbenet: how about on-demand mirroring i.e. get a mapping of hashes to external resources, and have a server that pulls them in when requested
<jbenet>
also, wait, is that a test to the real dht? lol
<jbenet>
i think it's intermittent because it's using the real dht.
<daviddias>
ahah \o/ living on the edge
<daviddias>
jbenet not one that I vet no. But https://hubot.github.com/, should be one of the best ones to do github reporting to IRC, (besides a million other things)
<daviddias>
jbenet: good to merge? :)
<daviddias>
push the green button :P
<jbenet>
daviddias: oh i meant just a script that posts to github api and sets it up, alraedy have the config options in mind.
notduncansmith has quit [Read error: Connection reset by peer]
jedahan has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
domanic_ has quit [Ping timeout: 246 seconds]
<whyrusleeping>
gooood morning!
notduncansmith has joined #ipfs
<rschulman>
howdy whyrusleeping
notduncansmith has quit [Read error: Connection reset by peer]
besenwesen is now known as besenghost
<pguth2>
Gooooood morning :D
<whyrusleeping>
hows everyone doing?
<cryptix>
hey :) whyrusleeping
* cryptix
is nerding at random
<cryptix>
had a delightful day yesterday and now my plate for today was mostly empty
<cryptix>
still have dishes and shopping on the stack but otherwise totally nice day
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
wooo!
<cryptix>
yes :))
<whyrusleeping>
thats always nice
<cryptix>
even got my teeth a little into ipfs
<cryptix>
maybe i can fix the putHandler on writable gateway later
<whyrusleeping>
:D
<cryptix>
whyrusleeping: i was going to use the add-link,rm-link stuff from object patch (which can handle bubbling up changes from a node to its root i was told)
<whyrusleeping>
it can indeed!
<cryptix>
perfect.. most of the code right now tries to deal with that and its a total mess
<whyrusleeping>
yeah... just use merkledag/util
<whyrusleeping>
its got everything you should need
<jbenet>
cryptix: have you worked on that git protocol hack?
<jbenet>
what was missing on it?
<cryptix>
jbenet: i got derailed on the '2nd method to fetch objects'
<jbenet>
(as a community, we should endeavor to finish the many hacks we have in-flight instead of context switching so much)
<jbenet>
what was the 2nd method to fetch objects?
<cryptix>
i need to get back into it but you basically need to handle the dependency graph of unpacked objects.. which is quite a mouthfull...
<cryptix>
jbenet: packed vs unpacked objects
<cryptix>
i think i'll scratch that stuff until somebod needs it
<cryptix>
and focus on pushing
<cryptix>
otherwise i need a libgit to handle parsing the objects trees
<cryptix>
jbenet: yea.. i hope not. those are cgo libgit2 bindings
<cryptix>
jbenet: i think i can pull of push without full fledged git bindings
<jbenet>
reming me why cgo matters?
<cryptix>
well.. i fear it does all the os.Open stuff in c land
<cryptix>
thus making it hard to port to vfs/ipfs
<cryptix>
if needed we can go through >clone to /tmp and work there> back but id like to make it sexy :))
<cryptix>
and again. i dont think this 2nd method for clone is actually needed if with do the ipfs-git-rehost dance with packed objects for now
<jbenet>
yeah though non-existent is worse
<cryptix>
if it makes more sense (dedup wise) to store them unpacked id like to discuss that with somebody more familiar with the process
<whyrusleeping>
implementing the git object format stuff is pretty simple
<cryptix>
or hand it over all together
<whyrusleeping>
linus wrote some pretty basic formats
<cryptix>
yea the pack format sure
atrapado has joined #ipfs
<cryptix>
but then it goes into 'client ask s for hash x' you can just give it the object for that hash but to reconstruct the commit you need all the objects that it points to, too
<cryptix>
thus you need an index of how they all fit together
<cryptix>
basically you end up doing all the work that a packed repo already did for you
<cryptix>
and i dont want to duplicate that right now
<cryptix>
and again: not necessary for push at all
<whyrusleeping>
ah
<whyrusleeping>
okay then
<jbenet>
hey mappum are you around?
<mappum>
jbenet: yep, what's up?
<cryptix>
if im wrong there is an issue and fork button on github.com/cryptix/git-remote-ipfs :)
notduncansmith has joined #ipfs
<jbenet>
i'm looking into the file to multpart stuff in node-ipfs-api
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
is it one big multipart stream where everything is nested in a tree underneath?
<jbenet>
or it it a sequence?
<jbenet>
better put: is it a tree or a list
Tv` has joined #ipfs
<jbenet>
i want to do the vinyl->multipart async, so returning a readable stream and have it start writing out before all the files are read
<mappum>
i don't think i wrote that code, but it certainly can be either (in go-ipfs it is a tree)
<whyrusleeping>
jbenet: working towards 100% test coverage on mfs pointed out a bug in the unixfs/format.go
<whyrusleeping>
there, 84.5% isnt bad
<ipfsbot>
[go-ipfs] whyrusleeping pushed 1 new commit to feat/mfs: http://git.io/vOQL8
<ipfsbot>
go-ipfs/feat/mfs 0844025 Jeromy: test coverage!!...
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
daviddias: jbenet could one of you publish the node-ipfs-api to npm?
<daviddias>
whyrusleeping: I can
<daviddias>
Published to npm.
<daviddias>
Released new version 2.1.1 successfully.
<daviddias>
whyrusleeping: ^ :)
<whyrusleeping>
wooo! :)
domanic_ has joined #ipfs
<daviddias>
with tag and everything -> https://github.com/ipfs/node-ipfs-api/tree/v2.1.1 . whyrusleeping if you ever find yourself constantly publishing modules and want to avoid repeating all the commands, `npm-release` is a sweet tool
<whyrusleeping>
isnt it just 'npm version patch' 'npm publish' ?
<daviddias>
it also creates a tag on the git repo
<daviddias>
and well, I believe that `npm version patch` wasn't there went it was first released
<mappum>
as long as your commit messages follow the AngularJS conventions
<daviddias>
mappum semantic-release is pretty cool :) met the dev at Data Terra Nemo (when I first learned about it)
<daviddias>
still haven't adopted it to my workflow yet though
<mappum>
i think it makes a lot of sense, the only big difference is changing the commit message conventions. i made a git template to make that easier to follow
<daviddias>
wanna post the link herE?
besenghost is now known as besenwesen
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has quit [Ping timeout: 264 seconds]
simonv3 has joined #ipfs
bren2010 has quit []
bren2010 has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
jedahan has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
domanic_ has quit [Ping timeout: 240 seconds]
<jbenet>
hey mafintosh or ogd: do you guys know of a "vinyl-tree" or "vinyl-flatten" type module? i basically need to take a stream of vinyl files, flatten into a hierarchy and walk it recursively. (to turn it into a multipart stream)
inconshreveable has quit [Remote host closed the connection]
Encrypt has joined #ipfs
inconshreveable has joined #ipfs
a1uz10nn has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<ipfsbot>
[go-ipfs] whyrusleeping pushed 1 new commit to feat/mfs: http://git.io/vOQdQ
<ipfsbot>
go-ipfs/feat/mfs 839f2da Jeromy: test for previously found bug in dagmod...
inconshreveable has quit [Ping timeout: 246 seconds]
Encrypt has quit [Quit: Quitte]
<whyrusleeping>
krl: ping
voxelot has quit [Ping timeout: 255 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
nvm, figured it out
martinklepsch has quit [Ping timeout: 240 seconds]
martinklepsch has joined #ipfs
<edrex>
is pinning an IPNS node something that's supported?
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
a1uz10nn has joined #ipfs
<rschulman>
edrex: Everything you insert into IPFS is automatically pinned by your own node.
<rschulman>
I believe
<rschulman>
if your ipns node points to a hash that you didn’t insert, though, I don’t know what happens.
<a1uz10nn>
can livestrean defcon?
voxelot has joined #ipfs
a1uz10nn has quit [Read error: Connection reset by peer]
patcon has joined #ipfs
bren2010_ has joined #ipfs
patcon has quit [Ping timeout: 264 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
dignifiedquire has quit [Quit: dignifiedquire]
a1uz10nn has joined #ipfs
bren2010 has quit [Remote host closed the connection]
bren2010 has joined #ipfs
bren2010 has quit [Remote host closed the connection]
bren2010 has joined #ipfs
bren2010 has quit [Client Quit]
bren2010 has joined #ipfs
bren2010 has quit [Client Quit]
bren2010 has joined #ipfs
domanic_ has quit [Ping timeout: 265 seconds]
bren2010 has quit [Remote host closed the connection]
bren2010 has joined #ipfs
bren2010 has quit [Remote host closed the connection]
<cryptix>
edrex rschulman: i dont think you can pin /ipns/... directly (as in it will stay up to date)
<cryptix>
you would have to remove the old pin at some point and i havn't seen any listening/event api (which is planned as pub/sub iirc) cc whyrusleeping
<cryptix>
resolving /ipns/.. to /ipfs/ and pinning that is possible ofc
<cryptix>
to drop just the hold pin's you would need some manual book-keeping
Encrypt has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
mildred has joined #ipfs
kbala has joined #ipfs
jedahan has joined #ipfs
jedahan has quit [Client Quit]
patcon has joined #ipfs
domanic_ has joined #ipfs
jedahan has joined #ipfs
domanic_ has quit [Remote host closed the connection]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
voxelot has quit [Ping timeout: 260 seconds]
dignifiedquire has joined #ipfs
mildred has quit [Ping timeout: 250 seconds]
mildred has joined #ipfs
patcon has quit [Ping timeout: 252 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<edrex>
who all was at the PDX meetup last month? trying to map faces to nicks. jbenet, kyledrake, ogd are the only ones I have so far
<edrex>
seems like the pubsub thing is orthogonal to whether the peer implementation lets users pin IPNS nodes
<edrex>
i mean, the peer could just poll until an effecient mechanism is available
<edrex>
an important use case would automatic mirroring
<cryptix>
edrex: agreed, but its basically all there to do it on top with scripts right now
domanic has joined #ipfs
inconshreveable has joined #ipfs
<edrex>
oh, and caseorganic and bret
<bret>
heyoo
<edrex>
hi
<jbenet>
edrex: who were you?
<jbenet>
and hello o/
<bret>
edrex: also mathpunk
<bret>
new job taking all my time, apologies for my absence
<edrex>
hi jbenet :) i think i was somewhat responsible for your talk going way over https://twitter.com/edrex
<edrex>
bret: yeah, he doesn't irc
<edrex>
is there much spec'd out around private sharing? i guess there are two possible approaches: cleartext objects where the source peer authenticates the request, or encrypted objects
<edrex>
full disclosure: I'm authoring a detailed comparison of camlistore and ipfs
<edrex>
or maybe private sharing is out of scope.
<edrex>
camlistore takes the first approach: the requester provides the hash of a certificate signed by the owner of the object declaring that the requester is allowed to access it
<edrex>
there's also support for transitive sharing. in that case the requester has to provide the path back to the directly shared object
<edrex>
it's a bit awkward though
<edrex>
btsync's approach seems pretty straightforward to implement on IPFS tho
<edrex>
you basically just specify a root that you want to share, and a list of peer pubkeys to share with. with btsync it's mutable, and each peer share can either be RO or RW
a1uz10nn has quit [Read error: Connection reset by peer]
MrKeon has joined #ipfs
<MrKeon>
Hello
jedahan has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<voxelot>
think we are looking at cjdns here to accomplish data privacy
<voxelot>
but thanks for that info edrex, cool stuff that i'm looking into now :)
<edrex>
i found https://github.com/syncthing/syncthing via that last doc, thinking to use that. Nice to use a tool purpose-built rather than to try to shoe horn a zillion use cases into one tool.
Encrypt has quit [Quit: Quitte]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Eudaimonstro has joined #ipfs
Eudaimonstro has quit [Remote host closed the connection]
Eudaimonstro has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
atrapado has quit [Quit: Leaving]
<cryptix>
edrex: syncthing is nice for a single-user dropbox replacement
<cryptix>
using it with multiple people gets messy... :-/
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<edrex>
cryptix: setting it up now for single user. does it support RO/RW authorizations like btsync?
<edrex>
guess i'll find out soon enough :)
Eudaimonstro has quit [Ping timeout: 240 seconds]
inconshreveable has quit [Read error: Connection reset by peer]
inconshreveable has joined #ipfs
freedaemon has quit [Remote host closed the connection]
<whyrusleeping>
MrKeon: heyo!
<MrKeon>
What up pinball masta!!
<whyrusleeping>
MrKeon: hows it goin?
Eudaimonstro has joined #ipfs
okket_ has quit [Excess Flood]
okket has joined #ipfs
<jbenet>
edrex oh hi! o/ welcome
<ipfsbot>
[go-ipfs] whyrusleeping pushed 1 new commit to feat/mfs: http://git.io/vOdMI
<ipfsbot>
go-ipfs/feat/mfs 169c669 Jeromy: add in write and mkdir commands...
<MrKeon>
The life of WebUI is interesting at the moment