17:22 <CcxWrk> Is apps.sandstorm.io down, or is the javascript just throwing tantrum for me?

17:32 <CcxWrk> I see encryption of grain storage mentioned in how-it-works, is that implemented? I assume that is supposed to work by decrypting on demand, kind of like Lavabit did, right?

20:10 <kentonv> CcxWrk, not implemented unfortunately. But yes, that's the idea.

20:10 <kentonv> apps.sandstorm.io seems up for me

20:12 <CcxWrk> Okay. So it just does not like TorBrowser.

20:12 <TimMc> or vice versa

20:18 <CcxWrk> How would the encryption be implemented? EncFS or similar FUSE filesystem for the data? How is grain data separated from it's code anyway, overlayfs?

20:54 <kentonv> CcxWrk, in the grain, /var contains the grain-specific writable storage, and the rest of the filesystem is the read-only app package contents

20:55 <kentonv> in Blackrock, all the disks are virtual block devices, so we could implement encryption in a lot of ways. I'm somewhat inclined to do the encryption on the storage nodes so that worker nodes never even see the keys.

20:56 <kentonv> (worker nodes are where grains actually run, storage nodes are where the data actually goes to disk)

20:57 <kentonv> on regular Sandstorm, I'm not sure how we'd implement per-grain encryption without some help from the filesystem, since we don't currently have any way to MITM the storage there

20:58 <ccx^xmpp> My ideal would be to store grain data into Tahoe-LAFS, possibly leveraging it's access contols. I know Tahoe has some rough edges around concurrency (and I have some ideas on how to solve it), how does that look in Sandstorm?

21:00 <ccx^xmpp> :P do you have a highlight or what?

21:00 <kentonv> ccx^xmpp, I love Tahoe, but I don't think it's designed to transparently back the storage for a high-transaction database, which is what many Sandstorm apps would expect.

21:00 <simpson> I actually read every single line sent to my IRC client. It's a problem.

21:01 <simpson> kentonv: I have wanted a daemon that is like Tahoe, except that it speaks Capn instead of Foolscap and completely internally handles sharding.

21:01 <simpson> It's on my todo list. It's pretty far down there, though.

21:03 <ccx^xmpp> Sharding as in determining which nodes are responsible for which grains?

21:03 <kentonv> it's possible that only Blackrock will be able to support per-grain encryption. But perhaps we can find a way to bring Blackrock to single-machine servers. The stuff it does with virtual disks means it really needs exclusive control of the machines it runs on, but maybe distributing it as a self-contained VM image would make sense.

21:03 <simpson> Yeah, and also the erasure-encoding part.

21:04 <kentonv> FWIW it could make a lot of sense to use Tahoe-LAFS as "cold storage" for grains that aren't currently running, or for periodic snapshots.

21:06 <ccx^xmpp> Yup, that was my idea. Because I want snapshots for concurrently running grains and merging changes. (basically letting it implement a CRDT)

21:07 <ccx^xmpp> That's currently the only way of doing concurrency on Tahoe anyway: let each node have it's own copy.

21:08 <simpson> I mean, Magic Folders and directories are things today, and they work fine.

21:08 <simpson> I know that you have a desire for a more formally-correct system, and I'm still onboard, and it's still gotta be designed and built.

21:08 <kentonv> you really can't do merges transparently

21:09 <ccx^xmpp> Not outside of the grain itself, it's something each would to do by itself.

21:10 <kentonv> so you mean, the app would need to support a "merge" operation?

21:10 <ccx^xmpp> Yes.

21:10 <kentonv> fair enough

21:10 <ccx^xmpp> simpson: I'm more concerned with moving grains to end-user machines and then using them offline.

21:11 <simpson> ccx^xmpp: Ah, so totally different use-case from mine.

21:15 <kentonv> A magical world where everything is a CRDT would be pretty cool... I think, though, it's probably unrealistic to expect most app authors to implement that (much less get it right).

21:15 <kentonv> so a more realistic model might be one where you "check out" a grain for offline use

21:16 <kentonv> which makes it read-only for everyone else in the meantime, until you check it back in

21:17 <ccx^xmpp> Disconnected operation is a big feature (that I miss almost everywhere), I know.

21:17 <kentonv> TBH even for apps that actually get CRDT / OT right, like say Google Docs, this is probably the right model, because if two people make extensive edits in separate forks it's likely that things are going to collide horribly when they merge later

21:21 <ccx^xmpp> Probably not too great for docs without manual merges. There is a ton of applications that do some kind of syncing nowadays. Mostly PIM stuff like calendars/contacts/todo/emails/notes...

21:22 <ccx^xmpp> Radicale even uses git as a backend.

21:22 <kentonv> ah, good point. I have a bias towards starting with docs as my model.

21:23 <kentonv> (because I've worked with them more)

21:24 <ccx^xmpp> Think more like personal wiki.

21:24 <kentonv> yeah I can definitely agree that the things you listed seem like they'd merge easily

21:26 <ccx^xmpp> Lot of collaborative stuff offers history with reverts anyway (ether*, cryptpad.fr)

21:31 <ccx^xmpp> kentonv: What exactly is the Blackrock you mentioned?

21:31 <kentonv> the cluster version of Sandstorm, that powers Sandstorm Oasis. https://github.com/sandstorm-io/blackrock

21:40 <ccx^xmpp> I see. The other option I see for Sandstorm+Tahoe is passing FURLs into grains (probably via query argument) and letting them manipulate those. I assume that'd need some extra cap added to Sandstorm.

21:41 <kentonv> what's a FURL?

21:41 <kentonv> foolscap URL?

21:41 <ccx^xmpp> I'll probably start writing Tahoe "app" for werc and then looking if it can be Sandstormized.

21:42 <ccx^xmpp> Yeah, but I didn't actually mean that.

21:42 <ccx^xmpp> I mean the URLs that serve as file/directory capabilities.

21:43 <kentonv> yeah, Sandstorm will need some way for the system to be aware when passing an external capability to an app

21:43 <kentonv> usually it's preferred that these things go through the powerbox

21:43 <kentonv> there could be a Tahoe "driver app" which implements Cap'n Proto capabilities based on Tahoe capabilities, handing them out via powerbox interactions.

21:48 <ccx^xmpp> What I'd really like is to have an option while looking at werc to click on "edit in etherpad" and it'd spawn separate grain with that, editing the tahoe file directly.

22:02 <ccx^xmpp> kentonv: Does Sandstorm use cgroups or namespaces? Or is it purely seccomp+chroot?

22:03 <kentonv> namespaces and seccomp, no cgroups. Plan to use cgroups in blackrock eventually for CPU/RAM quota tracking but it's tricky to use cgroups in regular sandstorm since it's hard to avoid interfering with other things on the machine.

22:06 <ccx^xmpp> Yeah, I'm looking into running it on a Linux-VServer box so I figured I might need to spin up a VM with separate kernel for it.

22:08 <kentonv> yeah Sandstorm doesn't work on linux-vserver or openvz since it needs working containerization

22:09 <kentonv> FWIW I highly recommend avoiding vserver and openvz. They are weird forks of Linux and I'm dubious about their security.

22:11 <kentonv> and if this is as bad as it looks, vservers and openvz are going to be screwed: https://lwn.net/SubscriberLink/741878/929c269a8b3c56f2/

22:12 <kentonv> (some people seem to be... rewriting the kernel memory management... seemingly in a hurry... we don't yet know why. It's a huge rewrite and I doubt it's gonna be backported to vserver/openvz very quickly, since they are based on such old forks.)

22:24 <ccx^xmpp> I use VServer because I just don't trust LXC to be actually secure container.

22:24 <ccx^xmpp> And yeah, that will likely be a big bad thing.

22:25 <ccx^xmpp> VServer tracks oldest LTS generally. So we'll see what happens there.

22:27 <ccx^xmpp> If it comes to worst… Sandstorm on FreeBSD/HardenedBSD when? :-)

22:30 <kentonv> hmm, what makes you think VServer is likely to be secure where LXC or Docker are not?

22:31 <kentonv> to me VServer appears to be a similar approach that has received far less scrutiny

22:31 <kentonv> (for the record I wouldn't trust any of these options to provide "secure" isolation)

22:43 <ccx^xmpp> It's been used as PaaS solution for a decade. It's been tracking lxc kernel code where appropriate, shrinking the patchset, but lot of it still isn't there in upstream. Conversely particularly user namespaces had quite lot of security issues by actually adding complexity.

22:43 <ccx^xmpp> vserver uses quite a simple model, it's quite jails-ish

22:44 <ccx^xmpp> I'm not saying it's bulletproof, but lxc never convinced me enough to switch.

22:45 <ccx^xmpp> (And I did't have time to rewrite every piece of software to Capsicum) :-]