13:28 <CcxWrk> TBH Meteor is why I'm not using Sandstorm right now. The point would be to isolate those insane JS and PHP environments an well, Meteor even didn't have a way to build it from source so I can put it on musl system when I looked. Not to mention the amount of code there, I can't deal with TCB that big.

13:30 <abliss> Hm, i wonder how hard it would be to write a new minimal shell for the sandstorm backend that didn't use meteor.

13:31 <simpson> Is the API that the shell is using entirely Capn?

13:34 <xet7> CcWrk: With musl system, do you mean Alpine Linux ?

13:39 <CcxWrk> Among others. I know you can dual-libc on Alpine, but magic mystery binaries are no-no for me.

19:00 <abliss> update: my userspace pty LD_PRELOAD shim has proved harder than I'd gussed/hoped. But, I just got it to support ttyd running `/bin/echo hello world`, so hopefully running `bash` or `screen` won't be too terribly much harder...

19:53 <isd> abliss: links?

19:54 <isd> ...it'd be nice if we could integrate the two libraries...

19:55 <isd> CcxWrk: I wholeheartedly respect that. It bugs me too.

19:56 <isd> abliss: Getting off of meteor would be a long road. I agree I'd like to minimize the TCB though.

19:56 <isd> Some things that would maybe make sense whether we got to the end of the road or not:

19:57 <isd> 1. Start using capnproto for rpc between the client and the server. This would be nicer to work with than Meteor.methods anyway.

19:57 <abliss> (no link yet, still very WIP. I'll try to upload something tomorrow)

19:57 <isd> But note that we'd first need a capnp implementation that runs in the browser.

19:58 <abliss> (integrating the librarie is a good idea , but i won't be learning rust in order to help. i plan to get a proof-of-concept up using just C, then you can feel free to fold it into your rust-built filesystem shim)

19:59 <isd> ...and I think before we start doing major refactorings, I'd want the relevant bits to be ported over to typescript.

20:02 <isd> I kindof also have this fantasy of junking mongo someday. I'm pessimistic about getting there. But either way we should be moving in the direction of hiding the db-specific bits behind an abstraction boundary. SandstormDb is supposed to be that boundary, though stuff needs to be ported over to it.

20:04 <isd> ...I would really like to believe that moving the shell over to not-javascript is realistic, but part of me knows better given the resources we have...

20:06 <abliss> is there a big blocker for capnp in the browser? i expected it would exist by now.

20:06 <isd> There's a typescript implementation that's serialization only

20:06 <isd> The implementation we use is a binding to the C++ library, so can't run client side.

20:07 <isd> I think the "blocker" is just "someone has to do the work."

20:07 <isd> Probably the shortest path to in-browser capnp would be to get the ocaml implementation building with js_of_ocaml

20:08 <isd> Right now it relies on some libraries that use the c ffi. But from what I could tell this isn't a "deep" thing, and shouldn't be that hard to fix.

20:09 <isd> But that leaves us with an ocaml library, and it would add a lot of page weight to pull in all of the ocaml stuff on top of the js we're already shipping.

20:09 <isd> I'd otherwise be thrilled to start adding ocaml to sandstorm :P

20:09 <isd> But it may be worth doing anyway, as apps could use it.

20:09 <abliss> cool, i've been planning to learn ocaml (and its js tooling) for weeks now

20:09 <isd> Bear in mind there are two ocaml to js compilers

20:10 <isd> I suspect getting this to work with bucklescript (the other one) is a much bigger task.

20:10 <isd> ...the one thing that really bugs me about ocaml is how fragmented the ecosystem is.

20:11 <isd> If I were to rewrite sandstorm, I would totally do it in OCaml...

22:40 <xet7> CcxWrk: What would be better than Meteor? I would think that with similar amount of features, amount of code would still be similar.

22:44 <xet7> CCxWrk: What do you mean with "magic mystery binaries" ?

22:46 <isd> xet7: I assume that's in reference to the fact that we're just pulling in the meteor bundle during the sandstorm build, and nobody builds it from source.

22:46 <isd> I think I looked at building meteor from source once and it seemed like a bit of a nightmare.

22:47 <xet7> isd: Is that meteor bundle just Javascript? Is there any compiled C or something?

22:47 <isd> It ships its own copies of node itself and mongodb

22:48 <xet7> isd: Are those not the same official node and mongodb binaries?

22:49 <isd> I'm not sure where they're getting the binaries from. But it would be nice to be building those from source (or pulling them in from a distro I guesss) for reproducibility and auditing purposes.

22:49 <isd> nixos has a package for meteor; I wonder if they do a source build for that?

22:51 <isd> Nope, looks like they're just packaging the binaries.

22:51 <xet7> From what I have read from Meteor, opinions do vary a lot. For some, Meteor is the best framework available, up-to-date, robust auth, etc, and they have looked at a lot of other options. For some, they do not like Meteor. I don't understand why is that.

22:53 <isd> Other than the above about reproducibility, I think most of my other concerns apply to much of the js/npm ecosystem. Honestly meteor itself is not too bad to work with, and has some nice stuff I haven't seen done quite as well anywhere else. But the lack of reproducibility & auditability bugs me, as does the fact that we have 700 ish transitive npm dependencies.

22:53 <isd> (We have a much smaller number of direct dependencies)

22:54 <xet7> Having Sandstorm or Wekan in some other programming language would have similar 700 ish dependencies, with same features

22:54 <isd> That's a lot of possible points of failure. Even with the same amount of code fewer upstream maintainers to have to trust would help me sleep better at night.

22:55 <xet7> How that would not be single point of failure in some other programming language?

22:55 <isd> The norms of package size vary between languages; even if you have a similar amount of machinery, you still have fewer people you have to trust.

22:56 <JacobWeisz[m]> I come from the .NET world, where the dependencies are almost all first party.

22:56 <isd> ...of course, if we had a language that supported proper ocap discipline, we'd have a much smaller TCB by virtue of most of that code not having unrestrained access to things...

22:56 <JacobWeisz[m]> Even the really popular third party .NET things tend to be Microsoft employees.

22:57 <isd> Yeah, it's not that uncommon in some ecosystems to have relatively few dependencies outside the standard library.

22:57 <JacobWeisz[m]> So then JS where you have things like left-pad shunted out to some random dude feels incredibly scary and weird.

22:57 <isd> Yeah

22:57 <isd> It is still true that upstreams would be a single point of failure. But there could be fewer such points.

22:58 <isd> ...again, proper ocaps would limit a lot of the damage that any one dependency could do, but that's something that's harder to come by than just not a bazillion tiny packages to track down.

22:58 <xet7> I don't know do all current Sandstorm or Wekan dependencies exist on other programming languages

22:59 <JacobWeisz[m]> .NET also generally incorporates into first party anything that becomes fundamental. Like for a long time, everyone used a third party JSON library, and now they have a new first party one. Since .NET used to be all about XML back in the day.

23:00 <JacobWeisz[m]> Whereas the JS ecosystem anyone who did it first and keeps it supported tends to retain it, so everything new ends up maintained by someone else.

23:00 <xet7> So is VB.Net being discontinued?

23:00 <xet7> Is it Open Source?

23:01 <JacobWeisz[m]> VB isn't a high priority for future development. But it's still supported and will be for a decade or more. And it's open source now.

23:01 <isd> I don't really think centralizing everything is the ideal answer. But neither is a separate maintainer per function...

23:02 <isd> Before sandstorm got active again, I was working on an ocap-friendly ML dialect. I haven't totally dropped it, though it's taken a back seat.

23:02 <JacobWeisz[m]> I mean, the Meteor framework came with a lot in-house. I feel like that should be the benefit to picking a JS framework.

23:02 <JacobWeisz[m]> They should be providing the standard features you need from a single source.

23:03 <xet7> From what I read, other JS frameworks are worse than Meteor.

23:03 <isd> Yeah, I'm not sure there's really anywhere to go that doesn't involve jumping languages.

23:03 <isd> I'd like to get away from JS, but I don't really see it as realistic.

23:04 <JacobWeisz[m]> I think the more different components are isolated (see Ian's web publishing system grain), the easier it is to at least build newer components in better languages.

23:05 <isd> Frankly I'd like to get away from C++ too; the risk of memory safety bugs worries me as much as if not more than our dependency tree.

23:05 <isd> ...but folks are so desensitized to server software being written in these languages that nobody seems to complain about it...

23:06 <isd> Which reminds me, I should take another whack at that asan build...

23:06 <xet7> What languages have memory safety?

23:06 <isd> Basically anything but C or C++.

23:07 <isd> (I mean, there are usually gaps if you're using the FFI or such, but most languages memory safety bugs for the most part can't happen)

23:13 <xet7> NVidia seems to have selected ADA, not Rust. There was some video about that somewhere.

23:14 <xet7> Some mention here https://blogs.nvidia.com/blog/2019/02/05/adacore-secure-autonomous-driving/

23:14 <xet7> https://www.youtube.com/watch?v=DZSSyWlsb28

23:19 <xet7> Maybe that would help to manage nuclear reactor with Wekan kanban more safely

23:57 <xet7> Hmm, maybe I'll use something more RAD

23:58 <isd> There's nothing quite like decades-old backronyms using slang from the time :P

23:59 <isd> I put together an asan build. It's dynamically linked though, so it probably doesn't actually run on anyone else's box. But I also have a branch with the necessary changes to build it yourself.