<jbenet>
timgws: oh it's probably just slow-- we need to merge #1218 ?
<jbenet>
timgws: do you see something going wrong? (or is it just hanging?)
<timgws>
seems like something in fuse hangs whatever application (in my case either Terminal or iTerm) is performing the action
<jbenet>
oh... that's not so hot.
ehmry has quit [Ping timeout: 265 seconds]
<timgws>
jbenet, it seems like the current fuse code is not threaded...
inconshreveable has quit [Read error: Connection reset by peer]
<whyrusleeping>
timgws: are you on OSX?
<jbenet>
timgws bug whyrusleeping about it -- he's the maintainer of that part
<timgws>
whyrusleeping, yeah
inconshreveable has joined #ipfs
<whyrusleeping>
yeah.... i very much dislike OSX fuse
<whyrusleeping>
im pretty sure we disabled the async tests for fuse in OSX
<timgws>
so it's a darwin bug, then?
<whyrusleeping>
i think its oxsfuse
<whyrusleeping>
its a different program than the linux one
<timgws>
yeah, I understand that
<jbenet>
well-- osx fuse works for some systems. not that we like it, but we're probably using it "not like it wants to be"
<jbenet>
meaning, it is possible to get osx fuse to work and not be awful.
<jbenet>
all the time, at least.
<whyrusleeping>
Tv`: thoughts?
<timgws>
most of the apis should be the same on OS X fuse & Linux fuse, though. So if it works on Linux it "should" work fine on OS X fuse with a few modifications
<timgws>
I use the word "should" very lightly, though.
<whyrusleeping>
timgws: do you have a repro for what youre seeing?
<timgws>
not yet. I was going to spend the weekend looking at it.
<whyrusleeping>
okay, cool
<timgws>
getting into Go & OS X kernel internals... what could possibly go wrong XD
<whyrusleeping>
timgws: you could die
<whyrusleeping>
but thats unlikely
<jbenet>
timgws: OpenNIC is a good idea :)
<timgws>
agreed.
<timgws>
I wrote some registrar software for tlds... whois, epp code etc.:)
<whyrusleeping>
at a javascript meetup, theyre talking about how the future of webapps is with entirely clientside javascript
ehmry has joined #ipfs
<jbenet>
timgws: why do all registrars suck?
<jbenet>
i've yet to see one whose UI isn't a nightmare.
<doublec>
headbite: I used similar for mirroring to freenet: wget --mirror --convert-links --adjust-extension --page-requisites -np -nH -N
<headbite>
well I feel a little better.... pretty close options. :)
<doublec>
it's a great tool!
<headbite>
yeah... I started with just screenshooting a few sites..... thinking.... how can I get some easy content onto ipfs....
<headbite>
then my girlfriend was like... that's not even clickable
<doublec>
I have a simple image heavy site I should try putting on ipfs and see what it goes like
<whyrusleeping>
jbenet: any more discussion on 1218?
<headbite>
doublec: yeah, after the wget I just "ipfs add -r ." and it seems to be mostly working... the youtube links won't work in offline mode but the other videos that are actually on the site made it.
<whyrusleeping>
i would love to have the electron shell app going
<whyrusleeping>
would make it a lot easier to point people at ipfs for installing
ehmry has quit [Quit: Leaving]
<whyrusleeping>
why does bootstrapping take so long...
<whyrusleeping>
im bootstrapping local peers to other local peers
<whyrusleeping>
and it takes like, five seconds
EricJ2190 has quit [Ping timeout: 240 seconds]
<doublec>
headbite: I uploaded my image one QmWom8ULsNCbtMgNMk82aJT7G6gU55mSGxVaC2i1fuyX3s
<whyrusleeping>
doublec: is your daemon online?
<doublec>
whyrusleeping: yes
<doublec>
whyrusleeping: I'm loading it from another node to test
<headbite>
doublec: it's loading pretty quick
<whyrusleeping>
ah, got it
<whyrusleeping>
my daemon may have been the offline one >.>
<doublec>
For freenet I have a modified version of this site that has a hidden div to load the full size images in the background with no-display so thumbnail viewers help keep the full images in freenet
<doublec>
ipfs works very well for this site. I'm impressed.
<whyrusleeping>
:D
<Tv`>
whyrusleeping: sorry haven't kept an eye on the channel today, have been fighting UEFI etc
<Tv`>
whyrusleeping: osxfuse should survive a typical ls -lR, for sure
<Tv`>
i needed to hammer it with writes to make it get stuck
<Tv`>
oh actually i got that busy loop with a loop on open+read+close
<Tv`>
osxfuse? it'll kick out a fs that's too slow
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<timgws>
right! maybe it would be a viable option to pass a setting along to osxfuse that gives it a hint that it's a slow fs?
<timgws>
but seeing the issues I am having with programs like Firefox & Finder, it might be done by design... slow directory traversal is killing these apps.
<doublec>
timgws: are you loading it in firefox using a file:// URL to the fuse mounted directory?
<timgws>
yeah
<timgws>
I know that there is the gateway/http server
<doublec>
timgws: I get timeouts even with the http server - maybe the same is happening with fuse.
<timgws>
it just saddens me that fuse doesn't "just work"
<doublec>
"context deadline exceeded" for example.
<doublec>
timgws: do you see errors shown in the 'ipfs daemon' output?
<timgws>
nothing
<jbenet>
timgws: yeah-- it saddens me too-- we're working on it and improving, but fuse is complicated. also-- do note this is waaay non-optimized code atm
<jbenet>
so there's a lot of room to go there-- as ipfs gets faster, fuse will behave better
<timgws>
<jbenet>
<timgws>
I think an intermediate solution might be just to count how long since the last request was made, and then return back an incomplete list if the call has been open for too long maybe?
<timgws>
reading that sentence back in my head, I realise it makes not mich sense.
<jbenet>
timgws: no-- that violates correctness.
<jbenet>
what if `ls` randomly returned subsets of the real thing?
<jbenet>
would not be fun.
<doublec>
yeah, that would cause issues for apps that want to know when everything is there. Imagine running 'tar' or 'zip' to archive a directory but it has missing stuff due to the timeout.
<timgws>
OK, what about returning EBUSY or ENETRESET ?
<jbenet>
not sure -- whyrusleeping Tv` o/
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Tv`>
timgws: ain't no such thing as slow filesystem on osx. 1 minute max.
<Tv`>
ETIMEDOUT or such
pfraze has quit [Remote host closed the connection]
inconshr_ has quit [Ping timeout: 265 seconds]
bengo has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Tv`>
what weirds me out is that dust has an interrupt mechanism but osxfuse doesn't use it
<Tv`>
*fuse
flugsio has quit [Quit: new kernels]
flugsio has joined #ipfs
sharky has quit [Ping timeout: 255 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
sharky has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
concurrency is hard
<jbenet>
Yeahhhh
<whyrusleeping>
i think i just found another bitswap bug that was a result of incorrect channel buffering
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
anshukla has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
Zombywuf has quit [Remote host closed the connection]
<jbenet>
mmmmm
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
notduncansmith has quit [Read error: Connection reset by peer]
iyrsx has quit [Ping timeout: 256 seconds]
<krl>
so the root is the 'spine' going down the middle
therealplato has quit [Read error: Connection reset by peer]
<whyrusleeping>
gotcha
<whyrusleeping>
looks pretty sweet, what are you planning on using it for?
hellertime1 has joined #ipfs
hellertime has quit [Read error: Connection reset by peer]
yewxcr76y has quit [Read error: Connection reset by peer]
lgierth has quit [Quit: Ex-Chat]
<krl>
whyrusleeping: for ssb-like feeds
<krl>
but with constant time access to the beginning, even if you have not synced the feed before
<krl>
has some nice properties, you can for example prove/check that two messages are in the same feed, if you only got the root hash
<whyrusleeping>
ooooooh, okay
<whyrusleeping>
thats pretty sweet
<krl>
and yes, this is also kind of a direction i want to push ipns in
<krl>
either directly, or as something tightly integrated with it
<whyrusleeping>
as far as a chain of entries?
<krl>
yeah, so instead of having the hash of the last message in your update, you have the hash of _all prior messages_
<wking>
krl: I don't think that's specific to the ipns entries in the DHT. Folks can chose to point those at a commit object if they want (once we get commit objects)
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<krl>
wking: yes, i just hope it will not just be totally ephemeral
<krl>
like, i don't like the idea of ipns history getting lost
<krl>
say, you have a long history of publishing a ipns entry pointing at a commit log, then you change history to hide an exploit or whatever, and just point your ipns at the newly redacted commit object
<krl>
at this point i'd want ipns to cry bloody murder
<wking>
I don't think the DHT should operate on that level
<krl>
no, the dht can contain pointers just fine
<krl>
but when you update your local name, a name you knew before, ipns should verify that it all checks out
<wking>
?
<krl>
sorry if i'm unclear
therealplato has joined #ipfs
<wking>
maybe we're talking about different things when we say "IPNS" ;). I just mean the DHT references stored with the ipns/<pubkey-hash> entries.
<krl>
the local ipns resolver could keep track of the pointers to the commit objects, and warn you if it has been redacted
hsvcr has joined #ipfs
<wking>
but maybe the reference doesn't point at a commit object
<krl>
then you should get a notification of this as well, imo
<wking>
or maybe it points at a commit object based on your finger tree instead of a commit object that's more like Git's linked-graph
<krl>
and reduce your trust accordingly
<wking>
I think trust is more about trusting the signer who's pushing links to the DHT, and less about properties of the links they're pushing
<krl>
yes, but you also want to reduce the amount of damage someone you trusted mistakenly can do
compleatang has joined #ipfs
<wking>
what sort of damage are you worried about?
mildred has joined #ipfs
<wking>
if you don't trust the publisher, just stick too immutable IPFS references and audit all of those before linking it into your tree.
<whyrusleeping>
anyone here familiar-ish with the npm codebase?
<wking>
but I doubt I'll ever have time to do that^ ;)
<wking>
krl: I don't see how "this is a valid commit based on a previously known commit" help protect against a malicious publisher
hsvcr has quit [Read error: Connection reset by peer]
<krl>
wking: it helps against showing you a fake history
<krl>
this could of course be manually checked, but when you can automate it i think you should
<wking>
why not just put your exploit into a valid history?
<wking>
when you pull the new reference and see that the history has been advanced, you'll look through the diff to check for exploits (or however you convince yourself that the update is safe)
<wking>
but you can do that with any object you have a reference to. It doesn't have to be a commit
<krl>
wking: i'll have to think more about how to express exactly what i'm worried about.
<wking>
anyhow, I think the not-a-fast-forward check is useful in the context of an IPFS-based version control framework, just like Git checks for this sort of thing when you do a non-ff pull. But I don't see how it's a good fit at the generic DHT-reference level or a way to protect against malicious publishers.
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<krl>
maybe a special type / software setup for append-only logs would suffice
xeb_ has joined #ipfs
hellertime1 has quit [Read error: Connection reset by peer]
xeb_ has quit [Read error: Connection reset by peer]
hellertime has joined #ipfs
<krl>
anyway, keeping a local log of prior names you've seen should be desirable
hsvcr has joined #ipfs
pfraze has quit [Remote host closed the connection]
pfraze has joined #ipfs
<jbenet>
krl: re "append-only log structures" -- that's really good. (also like the diagrams, what tool did you use?)
<krl>
jbenet: node-cairo
<krl>
and lots of manual twiddling.
<jbenet>
krl, whyrusleeping: this is my fault for not pushing this out yet, but i made DHT records be ipfs objects (though handled specially), so that (like the pins that Tv just wrote) they have nice tree properties, etc. it addresses the concern you had about making sure IPNS record dags stick around safely. sorry i need to finish this and push it out
* whyrusleeping
taps his foot impatiently
neoteo has quit [Quit: Be back later ...]
<whyrusleeping>
youre taking so long to write that that i decided to give up go and learn javascript
notduncansmith has joined #ipfs
<krl>
jbenet: looking forward to seeing it
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
krl -- correct me if i'm wrong -- but what you're worried about is that ipns records need to be handled in a particular way, at the ipns level. that is, updating a name carries important (security and UX) implications that need to be addressed, that most generic constructions of dht records + ipfs object pinning don't capture very well explicitly. (though it
<jbenet>
can, as wking mentions, all be done with just object creation + distribution + pinning operations)
<krl>
jbenet: kind of yes. like this, you will want to keep an (append-only) log of names you're interested in, to see how they change. The question is if everyone should keep their own logs, or if the log should be part of the update
<krl>
if everyone keeps their own log, and miss different updates, they will have inconsistent state
<krl>
as if the log is first-class, you will be sure that you did not miss updates
<jbenet>
and, speaking concretely, "don't just point to a commit that ipns hopes is accessible, have ipns make sure it's there". as wking mentions this can be solved by making the records themselves into ipfs objects, and have ipns "keep them around" (which can be done with standard pinning, or a "Special Pin" that the user can't get rid of by just doing `ipfs pin
<jbenet>
ls --type=all | ipfs pin rm` or whatever.
<jbenet>
yeah, i think we're all in agreement, and gravitating towards the same thing.
<jbenet>
btw, krl, one important concern people keep bringing up is generating ipns records in multiple nodes at once. which i addressed too.
<krl>
ah, yes that is an interesting problem
<krl>
jbenet: something like forkdb?
<jbenet>
so it's a bit more than just a single log, because it may have multiple heads at one moment. (the gist of how i addressed this is: multiple heads are ok, applications who do this (by explicitly giving the same key to multiple nodes) take on the responsibility of deciding between multiple records. they have to agree on a "validity" anyway (time validity or
<krl>
jbenet: i think this is kind of related to 'uncle blocks' in cryptocurrency
<jbenet>
oh that's cool-- i think i remember now
tilgovi has quit [Ping timeout: 264 seconds]
<jbenet>
krl: yeah-- though IIRC uncles aren't used (for more than adding "weight"). apps in ipfs may be able to use multiple ipns heads as they want to, with the understanding that the rest of the network is not required to store them (i.e. records are stored: { Key: Value }, not { Key: []Value } (--- though perhaps this is something that maybe should be relaxed to
<jbenet>
[]Value ---)
<krl>
jbenet: hmm. that might be true (about not being used)
<jbenet>
i mean, the existence of the uncle blocks allows you to look at transactions and infer things, like "so and so tried to double spend" -- which the more clever approaches will use to punish bad actors.
anshukla has joined #ipfs
<jbenet>
(am sure ethereum does, for example)
<jbenet>
whyrusleeping, krl, wking: need anything CR-ed?
<jbenet>
how are we doing on sprint?
<krl>
jbenet: i have a thing in /objects/ but i think i'll just push my current work over it
<krl>
some things have been refactored pretty throughly
<krl>
i'll probably be able to finish it up my next sitting, which should be tomorrow
<wking>
jbenet: I need more time to get my rerolled #1208 pushed, so nothing that needs review at the moment
<jbenet>
sgtm
<whyrusleeping>
jbenet: i drew you a picture
<jbenet>
whyrusleeping yeah-- the construction is currently confusing and that picture didn't help.
compleatang has quit [Quit: Leaving.]
<jbenet>
we can merge it, but bitswap's implementation is pushed further into the realm of mystery to me-- which is not something that's good. there should be multiple people capable of debugging it and fixing things quickly (code coverage)
<jbenet>
i'll try reading it again.
mildred has quit [Quit: Leaving.]
<jbenet>
whyrusleeping: work on something else then, i won't get to it until later today or tomorrow.
tilgovi has joined #ipfs
mildred has joined #ipfs
<whyrusleeping>
I'll try drawing a better picture
<whyrusleeping>
and I'm still working on other bitswap refractors, trying to simplify a lot of what is currently there
hsvcr has quit [Ping timeout: 272 seconds]
hsvcr has joined #ipfs
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Blame>
jbenet, whyrusleeping, + anybody else who is interested: I'm starting UrDHT delopment and push (we just had out first meeting)
<Blame>
And I'd love to have a chat to talk about what we are doing, how we can help each other, and maybe some advice
<jbenet>
Blame: sgtm. happy to chat. next week ideally.
<Blame>
Monday would be best for me
<Blame>
after that my week gets interesting
<Blame>
the 30 second rundown for lurkers: UrDHT is a FREE OSS project I am doing in conjunction with some other grad stuents (and anybody who wants to help)
<Blame>
We are making high preformance reference implementations of many different DHT protocols, and system to devlop new protcol and build on top of them easily
<Blame>
Immediate targets are Go and python3. Im hoping to be able to replace IPFS's DHT implimentation if you want to.
<jbenet>
sound great-- though to make that work we'll have to agree on the interfaces.
<jbenet>
i strongly suggest that you also make it possible for all of these to use a swappable network layer -- and be transport agnostic.
<Blame>
Yep. WhyRuSleeping showed me the interface you are already using and it looks pretty compatible.
<whyrusleeping>
jbenet: going to the store in a bit, gonna buy some paper so i can make a better drawing. computer drawing is not as easy as they would like you to think
<whyrusleeping>
in other news: i dont own any paper
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
hellertime has quit [Quit: Leaving.]
anshukla has quit [Remote host closed the connection]
anshukla has joined #ipfs
anshukla has quit [Remote host closed the connection]
<whyrusleeping>
jbenet: heres an interesting thing i just discovered in bitswap
<whyrusleeping>
we do a 'send wantlist to providers(key)'
<whyrusleeping>
which sends a full wantlist to each of the providers for that key
<whyrusleeping>
but, as soon as we get connected to a given peer, we send them our wantlist
<whyrusleeping>
so all we really need to do is ensure we are connected to providers for that key
<whyrusleeping>
and rely on the network notifications to trigger wantlist sending
ehmry has joined #ipfs
atrapado has joined #ipfs
hsvcr has quit [Read error: Connection reset by peer]
tilgovi has quit [Ping timeout: 256 seconds]
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
RzR has quit [Excess Flood]
RzR has joined #ipfs
ehmry has quit [Ping timeout: 256 seconds]
<jbenet>
whyrusleeping: ah. maybe the per-peer bitswap connection can figure out that it just sent a full wantlist and suppress the second.
<whyrusleeping>
well, im looking at changing the code for sendWantlistToProviders to simply call 'ConnectTo(provider)'
<jbenet>
whyrusleeping: "A just happened, B doesn't need to" is way more robust than "assuming A works out, B shouldn't need to..."
<jbenet>
yeah i think that will break quick.
neoteo has joined #ipfs
<jbenet>
now there's a latent assumption on the "new connection / notification" part.
<whyrusleeping>
so youre saying we cant rely on the notifications?
<whyrusleeping>
the problem i have with "A just happened, B doesn't need to" is how do you know that B doesnt need to? are you sure nothing happened in the time between A and B?
<jbenet>
no-- i'm saying we shouldn't add latent assumptions to subparts of the program. code changes and someone is going to come by in a few weeks and say "ohhh we probably don't _need_ to send a full wantlist on every new connection..." if you want to change the model so that "on finding out a new peer we _MUST_ send a full wantlist always or things will not work"
<jbenet>
that's a much stronger change.
<whyrusleeping>
ah, okay
<whyrusleeping>
thats the change i'm making
<whyrusleeping>
we have in our Connected handler to send a full wantlist to new peers
<jbenet>
i dont think you're understanding, that's a protocol change, it has to bubble up all the way to making strong claims about connections and what happens when they start. it couples things in a concerning way.
<jbenet>
try to model it abstractly, without code, as just mathematical statements
<jbenet>
or a state machine.
neoteo has quit [Ping timeout: 272 seconds]
<jbenet>
the simpler the statemachine the easier to understand, model, implement, evolve the protocol.
<whyrusleeping>
yeap
<jbenet>
on the "are you sure nothing happened in the time between A and B?" that's a great point--
<whyrusleeping>
i'm realizing i need to rewrite a LOT of bitswap now
<whyrusleeping>
we were relying really heavily on the rebroadcast timer
<whyrusleeping>
and that "things happening between A and B" was biting us a lot, but the rebroadcast came around and made things okay
<whyrusleeping>
which was why things were slow
<jbenet>
i think that question depends on what the "Bitswap Peer Connection" means, if it _owns_ the connection (sees all messages passing in both directions) then you can be sure.
<jbenet>
and supression is safe.
<jbenet>
yeah i don't understand the current implementation. i think about things visually-- i can draw diagrams for every part of the codebase except bitswap right now.
<jbenet>
(that's probably worth doing, too)
<jbenet>
whyrusleeping: all this stuff would be easier if all these things were separate modules
<jbenet>
with their own spec, and robust test suite
<whyrusleeping>
mind if i trash bitswap and respec/rewrite it from scratch?
<whyrusleeping>
i think it will be much easier for us to start fresh
<whyrusleeping>
than try to fix whats here
hsvcr has joined #ipfs
<whyrusleeping>
and will make it easier to simplify things by starting with a blank slate
<jbenet>
Hmmm, i dont know that's a good use of time. that will take at least a week.
<whyrusleeping>
it will solve all our bitswap related perf issues
notduncansmith has joined #ipfs
<whyrusleeping>
simplify the codebase
<whyrusleeping>
make it more modular and easy to test independantly
<jbenet>
hahah i think i've heard this once or twice before ;P
notduncansmith has quit [Read error: Connection reset by peer]
<whyrusleeping>
lol
<jbenet>
i think having bitswap as an independent library is a good idea. maybe we can start there-- move it out and move the tests there.
<whyrusleeping>
mmkay
<jbenet>
we can look at it in isolation and try to glean what the problem is
<jbenet>
it will also look a lot smaller.
<jbenet>
"ipfs/go-bitswap" ?
<jbenet>
actually-- before we do this
<jbenet>
wouldn't it be a good idea to have that script to vendor with ipfs?
<whyrusleeping>
the package manager?
<whyrusleeping>
we were waiting on s3 pinning
<jbenet>
not the full, full thing.
<whyrusleeping>
maybe i missed this conversation
<jbenet>
just something that vendors using git-ipfs-rehost.
<jbenet>
yeah i guess i hoped pinbot was up to the challenge
<jbenet>
whyrusleeping: we need to write a migration for the pinning stuff Tv wrote
<whyrusleeping>
mmkay, i can work on that
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
neoteo has joined #ipfs
ei-slackbot-ipfs has quit [Remote host closed the connection]
ei-slackbot-ipfs has joined #ipfs
hsvcr has quit [Quit: Leaving]
inconshreveable has quit [Remote host closed the connection]
<jbenet>
hey krl -- are you around?
<krl>
yep
<jbenet>
wanted to sync up on two things (a) webui and (b) datastructures.
<jbenet>
have 5 min now? or later?
<krl>
now is fine
<krl>
also, to ditch the old pinning view logic is probably a good idea, especially when Tvs stuff gets merged
<jbenet>
ok, on datastructures, -- since ipfs is a datastructure transport, we need to make it super easy to make arbitrary data structures. since you've made some i want to discuss ways of making it easier, particularly tuned to web use cases. -- AND i want to make a whole suite of tools that make it super easy to visualize instances of them, or automatically
<jbenet>
generate examples (like what you did, but have a script that generates a set
<krl>
ah yes, i was thinking about this also
<jbenet>
so it's easy to both make and play with various datastructures -- which i think will be useful to get people to think about their object models in webapps as just data structures (instead of "things in a database")
<krl>
i think having monoids over the trees could maybe be made more general
<krl>
monoids as in commuting values, like 'element count'
<krl>
and yes, cumulativesize is a monoid for example
<jbenet>
yep
<krl>
but it's not enforced
<jbenet>
what might make it more general?
<krl>
have a set of monoids for each object
<krl>
like: links, data, monoids
<jbenet>
when i was first making the choice, i wanted element count and other things,i pared it down to the essentials for transfer, but could certainly play with more.
<krl>
and then automate/enforce them when building the tree
Guest96_ has joined #ipfs
<jbenet>
interesting
<jbenet>
that's a very interesting proposition.
<wking>
will the IPFS core understand each of the monoid types? Otherwise who's enforcing?
notduncansmith has joined #ipfs
<jbenet>
i think this goes hand in hand with the idea for the "language" for data
EricJ2190 has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<jbenet>
(the idea was to come up with a simple language that outputs the data segement. i.e. "data" is really "code". ( kolmogorov complexity, etc)
<krl>
hmm, i don't see directly how they are related
<jbenet>
so a unixfs file would either have "print <binary>" or or "for object in links: output object.data"
<krl>
oh, you mean include the code that computes the operation?
<jbenet>
related because the monoid is just a computation
<jbenet>
yes
<jbenet>
or rather, the rules of the monoid can be enforced with computation
pfraze has quit [Remote host closed the connection]
<jbenet>
i think this is one of those things that could be hyper revolutionary if done right, but it has to _strictly_ improve the performance of the simplest case: moving files around
<jbenet>
(i.e. we shouldnt do anything that makes the "move files around" use case worse)
<jbenet>
krl: the worry with monoids is storing even more data per object
<jbenet>
the overhead hurts, quick.
<jbenet>
if this was tunable / selectable, that could be ok.
<krl>
you could just have a few required ones, and apply others to other trees
<jbenet>
thoughts wking ?
<wking>
I think this is all at a higher level than I'm interested in at the moment ;)
<krl>
it'll just be a tradeoff you can do differently for different trees
<jbenet>
haha fair.
<jbenet>
krl: right.
<wking>
I certainly wouldn't want to spec out the language
<jbenet>
wking: i suspect we don't need to.
<jbenet>
i'm sure someone has already.
<krl>
:3
<krl>
urbit?
<jbenet>
haha
<Blame>
You could appraoch it like an assembly and make the "outptu the following raw binary" as a 1 byte command
<Blame>
literally minimizes overhead
<jbenet>
krl: i'm sure there have been many publications on this topic.
<krl>
well, i guess ethereum is kinda doing it?
<jbenet>
CS research is roughly 5-20years ahead of implementations.
<jbenet>
Blame: yeah i like that
<jbenet>
so essentially, a bytecode
<Blame>
actually, making an assembly language without a fixed "line" size is an interesting problem
<jbenet>
krl: yeah i guess you're right that ethereum had to solve a lot of similar problmes.
<krl>
they have a bytecode
<jbenet>
krl: the focus is vert different though, afaiu ethereum is much more focused on defining languages for contracts
<krl>
a vm basically
<jbenet>
right
<Blame>
it means you could treat is like a stream byte-0 -> length of command
<Blame>
and the command and arguments to it could indicate the length
<jbenet>
also, does anyone here know erlang well?
<Blame>
My advisor keeps wanting me to learn it and I keep avoiding it
<Blame>
so, i should
<jbenet>
Blame: do it.
<jbenet>
the amount of "oh yeah, like this in erlang?" i come across is pretty high
<krl>
oh, erlang seems like a good fit for ipfs impl
<krl>
in the future when all is well
<jbenet>
yeah
<jbenet>
first things first.
<jbenet>
okaayy bringing it back--
<jbenet>
so, (b) tools to make datastructures easier.
<jbenet>
(maybe put discussion on monoids over at ipfs/notes)
<jbenet>
------ for (a) webui,
<jbenet>
we should publish a new version soon.
<jbenet>
what's the status over there with the various bugs, etc?
<jbenet>
cc tperson too if around
<jbenet>
(tperson: are you alive?)
<krl>
#32 remove local storage is blocked on pinning remake
<krl>
the thousands of files hang is still the worst unadressed bug
<krl>
#20
<jbenet>
right
<krl>
no peers shown is fixed (just closed)
<jbenet>
maybe let's fix that and publish a new version? thoughts?
<krl>
Blame: ah, now i understand more what you mean
<krl>
interesting
<jbenet>
Blame: +1
<Blame>
yeah
lgierth has quit [Ping timeout: 246 seconds]
<Blame>
looping can be handled by giving "cmp" a negative value to skip, but a part of the purpose of a language like this is to be able to totally throw away the code you jstu executed.
<Blame>
that way you can essentially treat all fucntion calls like tail-recursion
notduncansmith has joined #ipfs
notduncansmith has quit [Read error: Connection reset by peer]
<Blame>
jbenet: added
<jbenet>
thanks
domanic has joined #ipfs
flugsio has quit [Quit: WeeChat 1.1.1]
<Blame>
and I added a comment showing some example code in that madhouse of a language.
<jbenet>
Blame: thanks thats great! we should keep the discussion going there.