<danielh1234>
Hi. How do you configure Xen to create domains faster. It currently takes 3-4 seconds on my system. I already tried disabling dom0 memory balooning, but it makes no difference.
AltGr has left #mirage [#mirage]
betheynyx has quit [Quit: (Input/output error)]
AltGr has joined #mirage
mort___ has quit [Quit: Leaving.]
seangrove has joined #mirage
miragebot has joined #mirage
<miragebot>
[mirage] yomimono pushed 6 new commits to master: https://git.io/vobdY
<miragebot>
mirage/master e7dd9c7 Mindy Preston: Change source and destination ip and port names to be consistent.
<miragebot>
mirage/master e1a9d6f Mindy Preston: pin all mirage, mirage-types, mirage-types-lwt, tcpip to standardize_record_fields branch
<avsm>
There wasn't much on quality and test in the agenda, but progress has been made this week in terms of infrastructure and containers. There's quite a big array on https://hub.docker.com/r/ocaml/opam/ now (see the README)
<avsm>
Latest Fedora, Ubuntu, OpenSUSE and ARM variations of Alpine are all now there and mostly debugged (some issues left on ARM due to the CPU emulation, when building on x86_
<thomasga>
\o/
<hannes>
do they build mirage-skeleton on a regular basis?
<avsm>
hannes: they do, but the logs go into my personal cron logs. Next steps are to push them somewhere useful, perhaps a Canopy setup
unpurecamelbot has quit [Read error: Connection reset by peer]
<mort___>
(avsm: fwiw— looks like the line for Alpine armhf is duplicated in that page on hub)
<avsm>
thomasga and talex5 have been working on a CI as well, hoping to bring that live during the hackathon
<talex5>
Does this mean Lwt is building on Alpine again now?
<thomasga>
no
<thomasga>
just testing that now :-)
<avsm>
talex5: I'm still not sure exactly how to fix that beyond add conf-ncurses. Should I just go ahead and do that?
unpurecamelbot has joined #mirage
<unpurecamelbot>
I'll be logging this meeting…
<hannes>
avsm: cool! any chance you can enlighten where/whether they'll be able to add more testing for https://github.com/mirage/mirage/pull/544 (or what is needed to get this code movement merged?)?
<thomasga>
but `RUN apk add ncurses-dev` fixes the build
<talex5>
I thought the problem was with `conf-libev`?
dbuenzli has joined #mirage
<avsm>
hannes: they will -- it just unexpectedly blocked on a qemu segfault, frustratingly. Been quite hard to debug. But mort___ now has an ARM64 machine which can build ARM32 natively as well, so this may become easier
<thomasga>
avsm: can this is be added as a depext on lwt directly?
<avsm>
thomasga: yes, I think so. Lwt is the one with the odd configure script so I have no objection to fixing it there.
<thomasga>
talex5: the issue is again discover.ml in lwt
<avsm>
Overall for CI, we have the pieces in place but not the coordination infrastructure. That's the next steps before the hackathon so we can trigger builds more easily.
<avsm>
Anything else for quality and test? thomasga: i noticed new alcotest releases?
<talex5>
thomasga: Lwt's discover.ml is fine Ii think, but the copy in conf-libev seems old and broken.
<thomasga>
not really, it tries to compile a file using libev but the error comes from ocamlc complaining about missing ncurses
<thomasga>
so nothing to do with libev actually
amirmc has joined #mirage
<mato>
tangential depext-related question, is there a depext for "linux kernel headers"? (linux-headers on alpine, linux-libc-dev on Debian/Ubuntu)?
<mato>
Solo5 on ukvm will need this as it needs linux/kvm.h
<avsm>
mato: there is not, but can be added very easily as a `conf-kernel-headers` package
<thomasga>
I don't think so
* avsm
wonders how that would work on OSX
<avsm>
i guess we build in Linux containers on the Mac
<mato>
yes, at least until someone ports ukvm to Hypervisor.framework :)
<avsm>
on my mythical todo list ;-)
<avsm>
Ok, lets save the CI push for the hackathon in 2 weeks then. Next topic
<avsm>
Mirage 3! Over to our release manager yomimono...
<talex5>
thomasga: I can't install conf-libev on Alpine even without Lwt. Installing ncurses-dev does fix it. Lwt's discover.ml works even without ncurses-dev, though (it uses pkg-config and avoids the whole auto-detection mess).
<yomimono>
Yes! I've left quite a long list of things in the call agenda, but I'd really like to draw folks' attention to the bits that say it's not clear to me how to even do them
<yomimono>
(well, and the bits that say "we should do this but it's not done yet" - if any of that looks like something you could do and you've got some time, that would be great too)
<hannes>
the 'specification of version numbers for dependencies' is for inside of config.ml!?
<yomimono>
no
<avsm>
One thing I'm doing to vet these changes is to try to build "real" web services that run under the module signatures. In particular, using ocaml-webmachine and the Slack bindings to run a CI bot. I'm hoping that'll bring out any particularly painful dependency problems.
<yomimono>
I just mean some way of saying that a unikernel needs, say, dns > 0.18.3
<yomimono>
not just dns
<djs55>
or perhaps even that the unikernel needs dns = 0.18.3, so we can guarantee to be able to rebuild it easily in future?
<hannes>
yomimono: yes, what is there is packages: "foo" "bar" "baz"; that should then be "foo=0.18.3"?
<avsm>
I think it would be better to generate an `opam` file instead, and to use OPAM to make reproducible builds. The Mirage file shouldn't overly constrain the library
<yomimono>
avsm: that's the idea I had about it that I liked best
<yomimono>
hannes: I'm not sure I understand?
<yomimono>
djs55: yes, I think that'd be a very useful thing
<hannes>
so you want a MirageOS unikernel to consist of config.ml, unikernel.ml and an opam file, all crafted manually?
<avsm>
hannes: nope, generate the `opam` file during `mirage configure`
<thomasga>
note: if yougenerate an opam file you will "freeze" the deployment model of your unikernel
<talex5>
I've been thinking of doing a "git reset --hard XXX" on opam-repository in my Dockerfiles to get reproducible builds.
<avsm>
it should just be config.ml and the entrypoint module, rest can be fully generated
<yomimono>
thomasga: can you elaborate?
<hannes>
avsm: but from what? I meant we already have a spec of dependencies in config.ml, only names, no versions yet.
<mato>
avsm: where would the source of the version dependencies come from?
<mato>
what hannes said :)
<thomasga>
the only moment you have all the info that you need, is when you do `mirage configure —X`
<thomasga>
with X fully specified
<avsm>
thomasga: yomimono: I think he means that it will select a set of keys (e.g. --xen or --unix) in the `opam` file. We could add a `--name` to mirage configure to give the opam package an identity as a selection of keys
<avsm>
mato: the functoria specs for a key would have to add the constraints. They're where the package selection currently comes from.
<thomasga>
avsm: it's also valid for —dhcp=true/false etc
<avsm>
thomasga: yes all the user set keys need to be stored for reproducibility
<mort___>
is it not possible to specify the versioned name in packages in the config.ml? mirage-http.2.5.2 and so on?
<thomasga>
yes it is too
<avsm>
The hackathon seems like a decent place to put some of these massive API changes together, but we need to prep PRs ahead of time. the master branch is going to be broken for a while otherwise...
<thomasga>
when you generate the opam file, you can "freeze" the version (regarding the current state of opam-repository)
<thomasga>
but you will also make the unikernel less portable as you will have to choose the configuration parameters
<mato>
so we'd end up with a generated opam package named (e.g.) unikernel-<SHA of constraints> or something like that, which would get installed at "mirage configure" time?
<talex5>
mort___: for < and > constraints at least, they get turned into shell redirections when passed to opam.
<mort___>
for reproducibility i presume that one needs also to get the versions of all dependencies
<thomasga>
I'm not sure what you are trying to achieve...
<mort___>
talex5: yes, i was suggesting for exact versions only. (though what about with extra fancy quoting?)
<hannes>
thomasga: hmm, shouldn't it be possible at configuration time to generate a disjunction of all possibilities (thinking about "mirage-no-xen" & bla... | mirage-xen & foo)
<avsm>
talex5: that seems purely like a shell quoting issue :-)
<thomasga>
reproducibility is different than generating an opam file, right?
<djs55>
FYI I've been playing with libuv via fdopen's uwt bindings. It looks like it should be more scalable on Windows and OSX in particular but it isn't an Lwt engine — I'm still working though the implications of that. In particular I've had to avoid using Lwt_unix everywhere, even though I'm targetting Unix (sort of)
<avsm>
there are two steps: generate an opam file to freeze the key selections, and then freeze opam-repository and the OS distribution to get reproducibility
<thomasga>
for reproducability you also need location of dependency source + digest.
<avsm>
djs55: fantastic! An alternative to mirage-unix really helps with cross-platform support
<thomasga>
and selection of dependencies will also freeze your configuration more with OS-specific deps
<thomasga>
so the more you want to add info, the less you become portable
<avsm>
ok, it sounds like we have a lot of pieces in the air at the moment. The hackathon's going to be the most practical place to merge these all into master, as we could test for breakage during the day
<avsm>
i dont think we'll have the `opam` file generation ready for then, so lets defer reproducibility until we can merge some of the bigger API changes
<avsm>
Mindy, your call as to merge order into master branches...
<yomimono>
merge everything! break everything!
* yomimono
is a child with a hammer
<thomasga>
yay!
* mato
was hoping to merge solo5 at the hackathon, but with all the other breaking changes up in the air that might be a bit premature
<avsm>
mato: I'm keen to stage the changes so we emerge with a working master branch. We should coordinate a schedule during the day to get stuff in, build the skeletons and make sure things stay working
<avsm>
I'm happy to sit around doing that
<yomimono>
avsm: we should probably do some planning for test infrastructure to keep that from being too painful, but we can discuss that offline
<avsm>
Ok, lets discuss this more when we get to the hackathon. Meanwhile, it would be most useful if people could review the outstanding PRs ahead of the hackathon. I'm also going to propose a few higher level types that can compile to javascript (web API ones)
<avsm>
*discuss it more when we get to the hackathon part of the agenda, that is
<avsm>
any other PRs that need a look aside from hannes'?
<avsm>
an important note: it can be difficult to work on big features at a hackathon, but an excellent time to request merges. So prepare your trees ahead of time :)
<GemmaG>
Some Pioneer Projects might also be interesting/appropriate
<avsm>
just before i leave, does anyone need hardware brought
<avsm>
like cubies
<avsm>
also put them on that wiki
<avsm>
and if you need wired ethernet etc
brson has joined #mirage
<mato>
wired ethernet is always good
<djs55>
I think I'll be fine with my laptop provided there's wifi
<GemmaG>
I'll get a projector etc sorted for any demos
<yomimono>
If we can be sure to get a lot of temporary wifi codes or whatever from college, that'd be very helpful I think
dexterph has quit [Remote host closed the connection]
<djs55>
yeah I don't have eduroam any more
dexterph has joined #mirage
<GemmaG>
yup - will do
<yomimono>
yay! :D
<mato>
I'd like to do a demo of Mirage/Solo5, will coordinate with djwillia...
<GemmaG>
Great :)
<djwillia>
really looking forward to meeting everyone in person!
<yomimono>
:D
<mato>
djwillia: will you be in cambridge for the whole week?
<djwillia>
yes, i'm flying during the day on sunday and leaving the following saturday
<mato>
great! looking forward to meeting in person :) i'll be arriving in cambridge monday evening, leaving also on saturday
<GemmaG>
Excellent!
<djwillia>
mato: will be great
<mato>
i have to run, gotta go and shift a bunch of servers in 30-degree heat :(
<djwillia>
GemmaG: for the help with accommodation!
avsm has quit [Ping timeout: 250 seconds]
<mato>
see you all later
<djwillia>
see you mato
<yomimono>
later mato - stay cool :)
<thomasga>
what's the next topic?
<GemmaG>
Outreachy - but Gina isn't on the call today
<thomasga>
"how's syslog?"
<yomimono>
she successfully sent us a log message indicating her planned absence :P
<thomasga>
ok :-)
<thomasga>
any other stuff?
<GemmaG>
She is planning on doing a syslog talk at Velocity in Amsterdam in Nov :)
<GemmaG>
Think that's about it, unless there is AoB?
<thomasga>
(I need to run too)
<yomimono>
sounds like it, then :)
<yomimono>
thanks everyone!
<thomasga>
see you all!
<GemmaG>
:D
dbuenzli has quit [Quit: Page closed]
thomasga has quit [Quit: Leaving.]
<djwillia>
thanks and bye!
talex5 has quit [Quit: Leaving]
GemmaG has left #mirage [#mirage]
djwillia has quit [Quit: Page closed]
AltGr has left #mirage [#mirage]
insitu has joined #mirage
yomimono has quit [Ping timeout: 246 seconds]
<engil>
unpurecamelbot: commit done
<unpurecamelbot>
done
<engil>
unpurecamelbot: bye
unpurecamelbot has quit [Quit: unpurecamelbot]
<hannes>
engil! :)
lepoetemaudit has quit [Ping timeout: 250 seconds]
ricarkol has joined #mirage
ricarkol has quit [Client Quit]
mort___ has quit [Ping timeout: 272 seconds]
<reynir>
ahhh I missed hte meeting
noddy has quit [Quit: "it's a bit backwards around here"]
<reynir>
I'm afraid I will not go to the hackathon after all :(
<hannes>
reynir: NOOOO :(
dexterph has quit [Ping timeout: 260 seconds]
<reynir>
yes :(
seangrove has quit [Ping timeout: 264 seconds]
jermar has quit [Ping timeout: 244 seconds]
insitu has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
hannes has quit [Remote host closed the connection]
hannes has joined #mirage
AltGr has joined #mirage
brson has quit [Quit: leaving]
bronsen has left #mirage ["WeeChat 1.6-dev"]
AltGr has left #mirage [#mirage]
insitu has joined #mirage
<smkz>
I am almost done with the NTP client/server implementation for mirage btw
<hannes>
\o/
jermar has joined #mirage
insitu has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
copy` has quit [Quit: Connection closed for inactivity]
seangrove has joined #mirage
dexterph has joined #mirage
agarwal1975 has quit [Ping timeout: 246 seconds]
agarwal1975 has joined #mirage
brson has joined #mirage
andreas23 has joined #mirage
brson_ has joined #mirage
brson has quit [Ping timeout: 244 seconds]
ahf has quit [Ping timeout: 244 seconds]
ahf has joined #mirage
skeuomorf has quit [Remote host closed the connection]
agarwal1975 has quit [Quit: agarwal1975]
agarwal1975 has joined #mirage
insitu has joined #mirage
<smkz>
(I am done with the parts that estimate clock rate/offset, I am working on proper tests for those and once that is done I will write all the needed glue code to make it actually useful)
copy` has joined #mirage
brson_ has quit [Ping timeout: 250 seconds]
brson has joined #mirage
insitu has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
mort___ has joined #mirage
srenatus has quit [Quit: Connection closed for inactivity]