00:38 <dmj`> hannes: do you know if mirage gets linked w/ musl

00:38 <dmj`> well, unikernels created with mirage

00:49 <dmj`> ah, it probably just uses glibc if it uses solo5 right

09:28 <hannes> dmj`: there's no libc involved at all

09:29 <dmj`> hannes: I assume it statically links a c library at some point

09:31 <hannes> no.

09:32 <hannes> there are memcpy, memmove, memcmp, and an alloc implementation, but not a C library.

09:33 <hannes> back in mirage1 days, there used to be a dietlibc in the build process, but it is no longer...

09:33 <hannes> lots of symbols (like getenv) which are used by the OCaml runtime are implemented as stubs: they return NULL or -1

09:34 <dmj`> hannes: hmm, that's cool. So what about Ocaml code that ffi's into c? Does solo5 provide a posix compatibility layer

09:34 <hannes> there is no posix compatibility layer.

09:34 <dmj`> Ah, I see.

09:35 <hannes> some bits and pieces are in C, such as AESNI (and the AES implementation), but these are only using the above mentioned functions...

09:36 <hannes> I believe libgmp (via zarith) is the biggest C library - but this also works without posix

09:38 <dmj`> I see. Sure, so an ocaml application is statically linked with gmp.a and solo5, then runs on Xen (or any other hypervisor)

09:39 <hannes> it is a custom built of gmp.a. and solo5 provides a kvm/virtio backend, while xen is a separate one (thus you won't have solo5 and xen both in the same vm)

09:45 <dmj`> so solo5 let's you target different backends

09:59 <hannes> dmj`: yes, atm kvm and virtio. and there's a prototype port to hypervisor.framework (MacOSX) in a branch

10:03 <kensan> dmj`: There is also a Muen SK port but I am waiting for the Solo5 restructuring work to land.

10:04 <hannes> kensan: how is muen compared to l4?

10:04 <kensan> hannes: A big difference is that Muen is not trying to be a general-purpose microkernel but instead it is a separation kernel.

10:05 <kensan> hannes: There is not dynamic resource management at runtime.

10:05 <kensan> hannes: Basically the sole purpose is to isolate subjects from each other.

10:06 <kensan> hannes: Resource allocation is done at integration time. This includes memory, devices, CPU execution time etc.

10:08 <kensan> hannes: On Muen we use Intel VT-x/VT-d for subject separation. So virtualization is basically our means to isolating subjects.

10:08 <kensan> hannes: It allows us quite fine grained control of what is allowed/exposed to subjects and what is forbidden.

10:09 <kensan> hannes: e.g. access to the timestamp counter (rdtsc) is trapped since we do not want to give subjects access to such a high resolution time source by default.

10:12 <kensan> hannes: Another difference to probably most other kernels is that it is *not* written in C ;)

10:24 <hannes> kensan: where is the PCI network devices handled and how is a virtual NIC provided?

10:24 <hannes> (and thanks for your answers, I was showering)

10:25 <kensan> hannes: Ah I forgot: Muen does not provide a complete IPC interface/API.

10:26 <kensan> hannes: There are what we call events which basically allow a subject to send an interrupt to another subject.

10:27 <kensan> hannes: and then there is shared memory. These basic building blocks enable subjects to implement communication protocols.

10:28 <kensan> hannes: One that we use is "shared memory streams" or channels: unidirectionaly memory with an optional event for signalisation.

10:29 <hannes> kensan: is the dom0 involved in setting up shared memory between guests (I guess it has to)? and so I assume there's a driver guest which then passes all the received packets to all interested other guests?

10:29 <kensan> hannes: regarding PCI nic: in the Mirage/Solo5 demo system there is a Linux subject that has the NIC assigned.

10:30 <kensan> hannes: We have a Linux kernel module which implements a virtual NIC, that uses shared memory to push packets across to other subjects.

10:30 <hannes> kensan: thx.. really need to play around with muen at some point...

10:30 <kensan> hannes: No, there is no Dom0 or privileged subject. On Muen the communication channels and events are also static and defined at integration time in the system policy.

10:31 <kensan> hannes: So the Solo5 unikernel that receives network packets from Linux does not have the Linux itself in its TCB.

10:31 <kensan> hannes: e.g. if you terminate an encrypted connection inside the unikernel.

10:32 <kensan> hannes: obviously the Linux can do DoS by simply not pushing the network pakets across etc.

10:33 <kensan> hannes: But there is no privileged Dom0 that could alter the memory layout of the system etc.

10:34 <hannes> ic

10:34 <kensan> hannes: and since for Solo5/Muen all devices are implemented without the need for a Monitor, there is also no VMM in your TCB.

10:35 <kensan> hannes: It enables the construction of really lean/stripped down systems.

10:36 <kensan> hannes: Compared to Xen or other general-purpose hypervisors the price is of course runtime dynamism since on Muen *all* resource allocation is static.

10:37 <kensan> hannes: that is why it is more suited for building special-purpose system with an embedded style of development where you know your hardware well etc.

10:38 <kensan> hannes: imho it is taking the special-purpose idea that Mirage applies to "VMs" to the encompass the entire system.

10:38 <kensan> s/the//

10:43 <kensan> hannes: Having said that, it does not mean that the range of systems one can implement are restricted to very boring scenarions ;)

10:44 <kensan> hannes: This is a fun example: http://genode.org/documentation/release-notes/16.08#VirtualBox_4_on_top_of_the_Muen_separation_kernel

12:11 <gjaldon__> does using a unikernel running on kvm mean that I need to have a Linux OS installed to run the unikernel?

12:13 <dstolfa> gjaldon__: Mirage runs on a wide variety of things. You don't have to run ukvm, you can run virtio, which should run on any hypervisor supporting virtio

12:13 <dstolfa> I'm freely running Mirage on FreeBSD(though it doesn't have native bhyve support yet, but I'm told it's in the works)

12:13 <dstolfa> KVM-wise, I'd imagine it would run on illumos too

12:15 <gjaldon__> dstolfa: I think I read a press release where they mentioned bhyve support in Mirage 3.0. I may be wrong though

12:16 <dstolfa> gjaldon__: Well, bhyve already is supported, it just goes through VirtIO

12:16 <dstolfa> gjaldon__: Regarding the native support: https://github.com/Solo5/solo5/issues/167

12:18 <hannes> gjaldon__: you'll need some sort of Unix atm to run MirageOS, yes.. for the drivers (and/or look at kensan's description above on how to use the muen sk and contained network device drivers (from a linux, running as guest))

12:19 <hannes> (kensan: sorry, I cannot adapt to your language of subjects yet ;)

12:20 <gjaldon__> The 'virtualization ecosystem' is a new world to me and still figuring things out

12:20 <abeaumont> hannes: could you tell me the name of the author of the forced_lto branch? failed to find it on a quick search

12:20 <abeaumont> hannes: happy bday, btw :)

12:20 <dstolfa> hannes: Oh, is it your birthday? Happy birthday!

12:20 <gjaldon__> happy bday, hannes! :D

12:22 <hannes> abeaumont: https://github.com/ocaml/ocaml/pull/608 is the PR (where a report "tried with the following packages/command line invocation: didn't work" would be appreciated)

12:22 <hannes> abeaumont: it seems you need to pass -lto somewhere...

12:23 <hannes> thx people

12:23 <hannes> and now off to my office

12:24 <abeaumont> hannes: oh... that may be the problem then, i'll check it, thx!

12:29 <gjaldon__> dstolfa: been reading a bit on virtio and was wondering if I understood correctly. virtio provides virtualization for network and device drivers and KVM is the hypervisor? virtio also provides a standard api and many hypervisors work with virtio? what is ukvm?

12:30 <dstolfa> gjaldon__: VirtIO is just an API, any hypervisor can provide it, it works by having these queues called "virtqueues", through which you can send serialized control messages as defined per-driver

12:31 <dstolfa> gjaldon__: The communication is bi-directional and generally works by having a device driver on the guest side, and a userspace device providing information to it on the host side

12:31 <dstolfa> The nice thing about it is that it provides a way to bulk transfer things with 1 context switch

12:32 <dstolfa> So I can push things into the virtqueue and then notify all at once

12:32 <kensan> hannes: No worries. Sorry, its all a little confusing with such terms as subjects, components, partitions, guests, VMs... ;)

12:32 <dstolfa> kensan: I don't have too much time now, but I'd be interested to hear about the performance implications of what you've done :)

12:33 <kensan> dstolfa: That is a tough subject since there is no general statement I could give...

12:34 <dstolfa> kensan: Well, I don't think anyone can give a general statement about the performance of their system

12:34 <dstolfa> I was hoping to discuss some of the specifics, but alas I don't have time right now

12:34 <kensan> dstolfa: Looking at it one would think performance-wise our approach is a no-go. Turns out it can perform quite well if it fits your usecase...

12:35 <kensan> dstolfa: You can always reach me via email.

12:36 <dstolfa> kensan: Sure thing, but if you're going to be around here later on today, I can message you here :)

12:36 <gjaldon__> dstolfa: ok so VirtIO is an api and it uses "virtqueues" for communication between guest and host? how does VirtIO compare to UKVM?

12:36 <kensan> dstolfa: A lot of people focus on the guest state switching which is certainly more costly on Muen since "everything" is inside a VM vs. microkernels where its just ring3-ring0 transitions etc.

12:37 <kensan> dstolfa: For more detailed discussions I actually prefer mail but I should be lurking.

12:37 <kensan> ;)

12:38 <dstolfa> gjaldon__: I don't know how it specifically compares with respect to Mirage, you'd have to ask someone who's worked on it. UKVM essentially implements a unikernel monitor if I've gathered it correctly, which talks to KVM itself. In FreeBSD, this is something like the vmmapi in bhyve. With VirtIO, they'd need to implement the drivers in the unikernel

12:38 <dstolfa> kensan: sounds good :)

12:39 <dstolfa> With KVM itself, they'd probably need to use some kind of paravirt interface(again, don't know for sure), such as hypercalls to communicate from the guest kernel

12:42 <gjaldon__> dstolfa: thank you for the explanations. I appreciate it.

12:51 <hannes> abeaumont: well, the ocaml+lto compiler should actually do this by default (but if I recall you&chris in marrakesh, it didn't do anything)

12:52 <hannes> (there's an lto=1 passed in ocaml_compiler_internal_params, see https://github.com/ocaml/opam-repository/pull/6734/files)

12:54 <hannes> dstolfa, kensan: since this might be of interest to others as well, feel free to use mirageos-devel for discussion :)

12:55 <hannes> there's some solo5 vs xen and MirageOS, as written by mato, at https://pbs.twimg.com/media/C6VQffoWMAAtbot.jpg

12:57 <dstolfa> hannes: Does mato come here often?

12:57 <hannes> yes

12:57 <dstolfa> hannes: Okay, awesome :)

12:58 <hannes> but not during weekends

12:59 <kensan> hannes: I've seen that ;)

12:59 <kensan> dstolfa: Just out of curiosity, what kind of projects are you working on/involved with?

13:00 <dstolfa> kensan: I've been working on a tracing system that allows you to have global state across virtual machines

13:00 <dstolfa> basically, extending DTrace to allow you to trace virtual machines as you do locally

13:00 <dstolfa> But then have global state across them

13:02 <kensan> dstolfa: Ah, sounds interesting.

13:18 <hannes> abeaumont: according to how I understand it, the 4.04.0+forced_lto should enable lto by default... but that seems to be broken... will try to find time to investigate

18:22 <abeaumont> hannes: thanks, will try to take a deeper look at it too

21:24 <argent_smith> hannes: is there a way to use ppxes inside of unikernel code? e.g. lwt.ppx, etc.? including them in packages list in config.ml breaks make depend.

21:29 <hannes> argent_smith: you can inside of a _tags file specify package(lwt.ppx), nothing in config.ml..

21:31 <argent_smith> thanks a ton!

22:20 <gjaldon__> I have my blog as a unikernel(mostly ripped code from the static-website unikernel in mirage-skeleton). I want to deploy it to Google Compute Engine. I already have the compiled output using `ukvm` as the target. how do I go about adding the image to GCE? which of the compiled outputs is the image? I tried googling but couldn't find guides on how to run a

22:20 <gjaldon__> Mirage unikernel on GCE

22:39 <kensan> gjaldon__: My guess is you need to use the virtio target.

22:40 <kensan> gjaldon__: The solo5 readme mentions Google Compute Engine in the "Running virtio unikernels with other hypervisors" section.

22:48 <gjaldon__> kensan: thanks! looks like the solo5 readme has what I need :D

23:27 <kensan> gjaldon__: No problem.

23:29 <kensan> Ironed out some Solo5/Muen issues and now got it to server the Muen project website :)

23:29 <kensan> https://twitter.com/Kensan42/status/841067953089064960