07:14 <_habnabit> hi! i don't suppose there's a better channel for talking about capnproto than here? anyway i'm working on a project in rust that exposes a C ABI, but oh my gosh it is so tedious to have to deal with all this unsafety all the time. i'm trying to figure out if it's reasonable/possible to use capnproto for in-process communication using the same general parts as an IPC server would

07:15 <_habnabit> that is, expose a C ABI which is as close to just "make an object and talk capnproto RPC at it" as possible

07:53 <_habnabit> maybe exposing a single function that spins an event loop in a thread and returns an fd pair for talking to it

13:21 <dwrensha> _habnabit: hi!

13:21 <dwrensha> _habnabit: are you using capnproto-rust?

13:23 <dwrensha> _habnabit: can you be more specific about what you mean by "all this unsafety"?

15:26 <CcxWrk> _habnabit: There's also #erights fwiw.

15:31 <CcxWrk> Surely you can do socketpair, some kind of memory sharing would also work (as long as you can make Rust be happy about it), but I'm not seeing the point of additional eventloop.

16:12 <_habnabit> dwrensha, no, i'm talking about exposing a C ABI from rust is hella unsafe

16:13 <_habnabit> CcxWrk, oh okay cool. the event loop is because it looks like all the capnp implementations expect to have promises and such

17:18 <ocdtr_web> I just tagged asheesh_ and kentonv to a Synology/Sandstorm question (I think) in Wefork issues. Though I am not sure if dwrensha might also want to know. https://github.com/wefork/wekan/issues/45

17:53 <CcxWrk> _habnabit: Well, it all depends what the interaction should be. Presumably the thread would have it's own eventloop. Either way, it begs the question of why even thread? (aka WAYTTD)

17:54 <_habnabit> CcxWrk, it really is just trying to reduce unsafe surface area while not making something sufficiently weird that it would be difficult to interact with using existing capnp clients

17:55 <_habnabit> i have a rust library; i want to expose it to non-rust clients. spawning a separate process is possible but has its own problems

17:57 <_habnabit> ah, 'shared memory RPC' is still on the roadmap

17:58 <CcxWrk> Gothcha, so you want to use capnp as library ABI.

17:59 <_habnabit> more or less

17:59 <_habnabit> there's like how openssl has TLS memory BIOs; i guess that's an alternative to having pipes/socketpairs

18:00 <CcxWrk> I wonder if there is any actual advantage to that. I mean, if you share memory space with unsafe code…

18:00 <CcxWrk> And threads rarely make stuff "safer".

18:01 <_habnabit> it's entirely about surface area; more C ABI methods mean more places that you have to reason about the safety of

18:01 <_habnabit> i'm not so worried about the non-rust code accidentally stomping on the rust part of the heap

18:04 <ocdtr_web> asheesh_: I opened a very small Docs PR.

18:04 <CcxWrk> I guess you could make all functions take pointer to capnp buffer on whatever they call it. I mean, whatever they'd be passing to write() in IPC case.

18:04 <CcxWrk> s/on/or/

18:05 <CcxWrk> Do you actually want to run your library in a thread?

18:06 <_habnabit> CcxWrk, yeah, but then i started thinking it seems like i'm duplicating work there anyway, since i'm already going to expose a capnp interface, so why bother making an unsafe C ABI function for every one of these endpoints

18:06 <_habnabit> i honestly don't care one way or another about running in a thread; i'd be fine avoiding that, but i'm just trying to puzzle out how to do the RPC without it

18:06 <_habnabit> especially if shmem RPC isn't done yet

18:08 <CcxWrk> Well, doing it without threads is likely way simpler. Until you discover you need event handling in your library, at which point you'd have to do some messy integration with library user's eventloop.

18:10 <CcxWrk> In which case I'd probably just give the library it's own thread and a socketpair. That is, until you discover you need to spawn subprocesses and need to hook into wait() reaping.

18:15 <CcxWrk> And if you can't ensure the user will wait() with __WNOTHREAD I'd really think more about if you really don't want just another process. You know, they are here for the reason of isolating separate parts of a system. :)

18:17 <dwrensha> _habnabit: it's perhaps relevant that I'm working on porting capnp-rpc-rust to futures-rs (from gj)

18:19 <dwrensha> everything will still be event-loop-centric, but it will be a more standard event loop

18:20 <_habnabit> CcxWrk, isolating the rust into a process is p-robably going to be simplest anyway, because i had already started thinking about how to support non-local persistence as well, and that's going to require opening sockets anyway

18:21 <_habnabit> also it frees me from having to deal with extension modules in python mostly

18:21 <_habnabit> except the capnp python library requires them i think.....

18:22 <_habnabit> dwrensha, yeah that'd be good

18:24 <_habnabit> dwrensha, can it already do RPC over stdin/stdout?

18:28 <dwrensha> on unix we have wrap_raw_socket_descriptor http://docs.capnproto-rust.org/gjio/struct.Network.html#method.wrap_raw_socket_descriptor

18:28 <dwrensha> which probably only works for tcp sockets and unix sockets

18:29 <dwrensha> but I think it would be straightforward to add something similar for pipes

18:29 <dwrensha> ... which I think would work for stdin, on unix

18:34 <CcxWrk> If you spawn your "library" inetd-style, it could be capsicum-compliant. ;)

18:41 <CcxWrk> _habnabit: BTW the folks in #robigalia also know a thing or two about capnp and sharing memory. Next time around you'll need that. :)

18:41 <CcxWrk> (and doing it in Rust)

18:48 <_habnabit> CcxWrk, unfortunately i don't know what else to add to the search query to make 'capsicum-compliant' relevant

18:49 <CcxWrk> http://capsicum-linux.org/

18:50 <CcxWrk> So far the most sensible idea to bring capability security onto unix I've seen. Still, lot of work to remove all the ambient authority.

18:52 <_habnabit> ah this

18:55 <_habnabit> interesting

19:10 <CcxWrk> I've been toying with the idea of making full email stack (and eventually cap-based "groupware") with capsicum and distributed encrypted backend (perhaps tahoe, perhaps not). There's currently lack of any actual use of capsicum besides stuff that can be easily done with seccomp or pledge().

19:16 <CcxWrk> Tahoe-LAFS has currently big issue with concurrent writes. Sandstorm has been nice inspiration in terms of the idea of making isolated app-based capabilities, since I can stack that on top of the storage backend and let the apps deal with concurrency themselves (CRDT-style).