marcan changed the topic of #asahi to: Asahi Linux: porting Linux to Apple Silicon macs | General project discussion | GitHub: https://alx.sh/g | Wiki: https://alx.sh/w | Topics: #asahi-dev #asahi-re #asahi-gpu #asahi-offtopic | Keep things on topic | Logs: https://alx.sh/l/asahi
odmir has quit [Ping timeout: 260 seconds]
<krbtgt>
kettenis: petitboot is a bit of an oddity indeed
<krbtgt>
apparently it's better than forth according to the IBMers i asked who chose it D:
matt6 has quit [Ping timeout: 265 seconds]
odmir has joined #asahi
riker77 has quit [Ping timeout: 260 seconds]
riker77 has joined #asahi
odmir has quit [Ping timeout: 260 seconds]
odmir has joined #asahi
odmir has quit [Remote host closed the connection]
odmir has joined #asahi
odmir has quit [Ping timeout: 240 seconds]
Namidairo has joined #asahi
PhilippvK has quit [Ping timeout: 250 seconds]
phiologe has joined #asahi
gabiruh has quit [Remote host closed the connection]
gabiruh has joined #asahi
marvin24 has quit [Ping timeout: 250 seconds]
marvin24 has joined #asahi
crabbedhaloablut has quit [Ping timeout: 260 seconds]
crabbedhaloablut has joined #asahi
Namidairo has quit [Ping timeout: 268 seconds]
crabbedhaloablut has quit [Ping timeout: 240 seconds]
<arnd>
krbtgt: the Sony PS3 was the original target for petitboot, and it could already boot a linux kernel that would only run on the ps3, so kexec was the easiest way to load a distro kernel
<JTL>
I always foudn it amusing that petitboot kind of originated from the PS3 homebrew scene
<JTL>
found it*
<JTL>
and now it's used on IBM POWER hardware, some ARM boards, buncha other stuff :P
<tmlind>
fyi, there's also kexecboot that is a much simpler solution for folks needing a kexec gui
ponikrf[m] has joined #asahi
herbas has joined #asahi
<maz>
there is also a set of patches that allow grub to run in *cough* userspace and use kexec as a backend. pretty neat for distro kernels.
herbas has quit [Quit: herbas]
raster has joined #asahi
matt6 has joined #asahi
matt6 has quit [Client Quit]
matt6 has joined #asahi
matt6 has quit [Client Quit]
matt6 has joined #asahi
<matt6>
reject FORTH, accept TCL :P
anarsoul has quit [Ping timeout: 245 seconds]
anarsoul has joined #asahi
Bublik_ has joined #asahi
Bublik has quit [Ping timeout: 240 seconds]
agnem has quit [Ping timeout: 260 seconds]
agnem has joined #asahi
JTL has quit [Ping timeout: 260 seconds]
agnem has quit [Ping timeout: 260 seconds]
klaus has quit [Ping timeout: 240 seconds]
agnem has joined #asahi
JTL has joined #asahi
klaus has joined #asahi
<maz>
oddly enough, I really like Forth. I'll have my head checked...
<sven>
lol, grub in userspace :D
<sven>
that sounds, er, fun
<kettenis>
I actually considered doing something like that when porting OpenBSD
<kettenis>
running the OS bootloader as a linux userland process
<kettenis>
but the default petitboot environment doesn't really allow you to do that
<pipcet[m]>
sven: I don't think we want grub in userspace, but I suspect we'd be happier with the device tree mangling in userspace. The main problem kexec has and had is that the old kernel may leave hardware in a bad state, and that's not an issue for us as the first kernel doesn't even need to touch the interrupt controller or anything.
<maz>
that's the patch hacked on to get it a bootloader on kevin.
taziden has joined #asahi
ephe_meral1 has joined #asahi
segher_ is now known as segher
choozy has joined #asahi
jeffmiw has joined #asahi
jeffmiw has quit [Ping timeout: 240 seconds]
maknho___ has quit [Ping timeout: 252 seconds]
maknho___ has joined #asahi
modrobert has quit [Read error: Connection reset by peer]
modrobert has joined #asahi
maknho____ has joined #asahi
maknho___ has quit [Ping timeout: 260 seconds]
maknho has joined #asahi
maknho____ has quit [Ping timeout: 240 seconds]
choozy has quit [Remote host closed the connection]
<pipcet[m]>
do we have any way at all of loading macos if it isn't currently configured to be the boot OS? I tried installing the macos .macho as a custom kernel but could not get that to boot.
<pipcet[m]>
or to emulate it (https://worthdoingbadly.com/xnuqemu2/ doesn't work on actual kernels), or to get a memory image of a running macos system (I don't suppose they have /dev/mem :-) ), or anything like that? Because without a way of doing that, I don't see how all this hypervisor work will ever pay off.
<sven>
uh, by writing code that loads xnu and jumps to it under the hypervisor?
<pipcet[m]>
"writing code that loads xnu" is the problematic part, I fear.
<sven>
i don't
<pipcet[m]>
so which macho would we use? The .kc machos do not have plain AArch64 instructions in them.
<sven>
what do you mean with "not plain aarch64 instructions"?
<pipcet[m]>
running objdump on them produces gibberish :-)
<pipcet[m]>
the data section is fine, the code section...isn't.
rwhitby has quit [Ping timeout: 250 seconds]
rwhitby has joined #asahi
<sven>
i'm not sure what you're doing then but i didn't have any issues
<pipcet[m]>
maybe it's a macos version thing, and older machos work fine.
<pipcet[m]>
but so far the "load xnu and jump to it" part doesn't seem trivial to me. If it is, we should get that done first, I think, because that would give us debugging information even without a hypervisor.
<sven>
why would that give us any additional debugging information?
<pipcet[m]>
memory image of a working macos system? Hardware state while it's running?
<sven>
that's *much* easier with the hypervisor up and running
<sven>
but if you disagree: feel free to explore that direction :-)
<pipcet[m]>
you appear to be saying that it should be trivial to boot macos from, say, Linux, and the difficult bit is getting a hypervisor in. I think it's the opposite.
<pipcet[m]>
maybe we're both partly right and both parts are easy
<sven>
i'm not sure it would be easy from linux but it's going to be relatively easy from m1n1 when almost no hardware is up and running
<pipcet[m]>
why "going to be", though? If it's the easy part we should do it first, to see whether it isn't actually tricky or impossible.
<pipcet[m]>
and we're going to have to do it anyway before the hypervisor would be very useful
<sven>
well, the code is open source. feel free to start working on that :)
choozy has joined #asahi
<pipcet[m]>
yes, I have. And I've run into problems. That's why I think it's hard.
<jix>
hmm to me it seems there are two outcomes when trying this before the hypervisor is complete enough... a) jumping to the macos macho works right away b) it doesn't ... in case of a) there's nothing to do, in case of b) you'd probably want the hypervisor to be able to debug things
<yrlf>
pipcet: we can probably easily chainload MacOS from m1n1 (load it and jump in, basically), but without a hypervisor, that will not really be useful, since it effectively does the same as booting MacOS normally
<yrlf>
with a hypervisor we can look and see what MacOS is doing while it's running
<pipcet[m]>
what would you even jump to?
<pipcet[m]>
the macho does not contain instructions that objdump or the CPU understand
<pipcet[m]>
yrlf: there are plenty of tricks we can play without a hypervisor...
<j`ey>
pipcet[m]: well it must contain arm64 instruction?
<sven>
it does
<yrlf>
as far as I understand it, iboot2 will load the MachO, and jump to the entrypoint (same as it does with m1n1)
<sven>
yup
<yrlf>
there's no reason why the MacOS binary should be different
<sven>
it isn't
<yrlf>
yeah, that makes sense
<j`ey>
but m1n1 doesnt support macho yet, right?
<yrlf>
j`ey: marcan added some stuff
<yrlf>
(e.g. proxyclient/macho.py)
<yrlf>
also, when m1n1 chainloads itself, it loads a macho
<sven>
that's not enough for a xnu macho though
<sven>
it only implements the minimal requirements
<yrlf>
sure, for a kernel image you'd have to extend that, right
<yrlf>
so it does not work out of the box yet, but it's not _that_ far off if I understand this correctly
<sven>
pretty much. it might need some reverse engineering of undocumented macho headers (or however that format works) but i'm willing to bet it's possible to figure those out by staring at them
<sven>
pipcet[m]: so i'm not sure what you're looking at but the latest mac os kernelcache still contains plain aarch64 instructions
<j`ey>
sven: are you using objdump or otool?
<sven>
i, er, coerced an elf objdump to just disassemble everything
<yrlf>
usually, if I just want to disassemble something that the normal tools don't like I use radare2. though the more I learn about that tool the less I want to use it for that
<yrlf>
(I'm pretty sure if you throw the wrong binary at it you can easily get code execution from it)
<sven>
for serious reverse engineering i generally stick to ghidra or ida pro. for quick glances objdump works fine though
<yrlf>
case in point: it segfaults if you try to use it on m1n1.nacho
<yrlf>
*macho
KindOne has quit [Quit: K-Lined]
VinDuv has joined #asahi
zkrx has quit [Ping timeout: 240 seconds]
<marcan>
pipcet[m]: the macos macho absolutely boots as a custom kernel
<marcan>
that is the entire point
<marcan>
the very first test I did to convince myself kmutil worked was patch the darwin version string to say "Linux" and boot it
<marcan>
worked fine (and that also kills any conspiracy theories about apple "blocking" anything for custom kernels)
<marcan>
this is retail darwin, not a xnu source dump build, so everything enabled
<marcan>
I assume the only thing that wouldn't work is SEP-mediated DRM
<marcan>
(and this also means my hypervisor stuff *will* work, modulo a as-yet unconfirmed level of fuckery to deal with corner cases due to macos running in EL1 not EL2, but by definition nothing insurmountable)
<marcan>
and objdump absolutely disassembles it fine, so you must be doing something weird ;)
<marcan>
and yes, what I want to do could be accomplished with kexts, or with a patched FOSS XNU build, or any other number of ways
<marcan>
I chose the hypervisor approach because I liked it initially, and it is proving to have been an *excellent* idea because it also becomes a magical debugger for m1n1 and Linux themselves that anyone will be able to use, without a serial cable. built in USB to serial.
<marcan>
will do more dev in the next 1-2 days; chances are good I'll have serial over a secondary CDC-ACM brought up really soon and that will mean anyone can load Linux kernels and debug m1n1 and Linux (and u-boot or whatever) bring-up with zero special hardware or cables or fuss, which is a big win
<marcan>
(single core initially, multicore will come a bit later, but that's good enough for a lot of work)
<marcan>
sven: ghidra sadly barfs on current kernelcaches, but I'm trying to avoid disassembling anything at this point so I don't care
<marcan>
the only thing I ended up doing in the early days was objdumping the xnu init section to figure out exactly what chicken bits Corellium got from there (and that told me they didn't know what they were doing and had copied useless stuff and added some of their own, so it was a good thing I did it :P)
<marcan>
not planning on ever looking at it again :)
zkrx has joined #asahi
<pipcet[m]>
marcan: excellent news! As I said, on my device the kernel cache does not contain plain aarch64 instructions, and doesn't boot when installed as a custom kernel.
<marcan>
what are you calling the "kernel cache"?
<marcan>
if you mean the kernelcache file, that would be an img4 wrapper built by kmutil; you need to extract the mach-o from it first before feeding it back to it
<pipcet[m]>
yes. it does seem like a valid mach-o to me.
<marcan>
just did it with my installed kernelcache, 11.2.3, 20D91, still yields valid code
<marcan>
let me upgrade to 11.3.1...
<pipcet[m]>
let me boot into macos
<marcan>
going to sleep now but I can confirm this tomorrow
<pipcet[m]>
that would be awesome!
<marcan>
make sure you're looking at the right file (/System/Volumes/Preboot/*/boot/*/System/Library/Caches/com.apple.kernelcaches/kernelcache)
<pipcet[m]>
if it's a new apple trick we should probably work with the older version for now
<marcan>
yes, that shouldn't be a problem
<marcan>
though I'm at a loss as to why they would do anything like this
<marcan>
and either way, they *are* releasing XNU source dumps; it might even be *easier* to do this with such a FOSS build, as those lack hypervisor support
<marcan>
which is good for us
flying_sausages_ has quit [Quit: You just lost the game. Peace Out.]
<marcan>
only reason I can think of why they'd encrypt XNU is to further obfuscate the M1-isms, though that cat is thoroughly out of the bag and if they do so I'm pointing squarely at ARM's lawyers being idiots
<marcan>
I mean they don't even encrypt iOS last I checked, haven't for a long time
<pipcet[m]>
it doesn't look encrypted. repetitive, compresses well. Just not valid insns.
<marcan>
are you sure you're disassembling in little-endian aarch64?
<marcan>
flying_sausages: soon, sorry, I have an update pending but got too caughed up in hypervisor development :)
<pipcet[m]>
well, that's that mystery resolved then.
<marcan>
I'll do a catch-up march post on the upstreaming stuff, then april-may will probably be a combined update about hypervisor goodies since this month is going to be fun
<flying_sausages>
nice, cheers!
<marcan>
*caught (damn, I can't english these days)
<marcan>
also, um, I *thought* booting darwin required more mach-o loading stuff
<pipcet[m]>
cool :-)
<marcan>
I was wrong, because I just chainload.py'ed darwin and it worked, framebuffer and serial output and all (didn't get to the desktop, might not even be the right kernel, but whatever)
<marcan>
actually that isn't even setting up the devicetree and trustcache properly; the version in hv.py does.
<marcan>
wonder if *that* works
<pipcet[m]>
will play with it if it does and I can get at the kernelcache...
<marcan>
ok, HV does not work :-)
<marcan>
which isn't entirely surprising. related to guarded exec I bet (hi sven)
<marcan>
anyway, good night :-)
<pipcet[m]>
good night, and sorry for the confusion.
<sven>
marcan: i never tried loading anything in ghidra. so far i've just used objdump to find the entry register for guarded exec and that's it :D
<sven>
but yeah... let me document the guarded exec stuff for real now
<marcan>
fwiw, this is a far as darwin gets when chainloaded: https://mrcn.st/p/jNkTDWj6 (again this may or may not even be the right kernel for the current iboot2/sfr...)
<marcan>
long delay after the SEP stuff; I did push a bit that was missing but it didn't help
<marcan>
then more delays on the USB stuff
<marcan>
then it panics on pcie
<marcan>
will try tomorrow with the right kernel :)
<pipcet[m]>
...okay, macos lets me have a different image, and that chainloads a little. I must have done something wrong in the recovery terminal
<sven>
ohhh... i messed up with guarded exec stuff actually :D
<sven>
i think it'll work just fine
<sven>
i confused PXN and UXN
<marcan>
oh heh
<marcan>
well, running it under the hv right now just hangs, no output or anything
<marcan>
which suggests an EL1 exception
<sven>
though... it's strange that it doesn't hit an undefined MSR
<marcan>
those are *EL1* exceptions
<marcan>
(because ARM)
<sven>
oh
<sven>
okay, that makes sense
<sven>
yeah, so that's gonna be ugly
<sven>
because you need SPRR enabled in EL2 to make it work in EL1
<sven>
and with SPRR you no longer get rwx pages
<marcan>
probably the next thing I'll do is hook VBAR and add my own exception vector hooks that just hypercall on anything "interesting" like undefined instruction syndromes
<marcan>
and then I can fix things up in EL2 that way
<marcan>
(I *can* hook VBAR writes)
<sven>
but i think you can just fake the guarded stuff
<marcan>
I think there's a good chance just hooking all undefined instruction exceptions will let me do what I want
<sven>
yes
<marcan>
(I hate ARM for not letting me just flip a bit to do that, unless I missed something / cc maz)
<marcan>
(I mean without NV)
<sven>
you'll probably need to emulate the GL1 msrs
<marcan>
yeah
<sven>
you get ELR,FAR,etc._GL1
<marcan>
docs please :)
<sven>
yeah, yeah, working on it now :D
<marcan>
there is that "hook all impdef MSRs" bit though
<marcan>
which is *probably* enough
<marcan>
then I don't need the vbar hook, but conversely I need to emulate all the *others*
<marcan>
https://mrcn.st/p/RA2VE5CV anyway there are your trap bits, and I bet bit 12 is related (trap instructions maybe?)
<sven>
(or GL2 for that matter. that thing is aliased to SYS_ELR_GL2 as well)
<marcan>
makes sense, VHE style
<sven>
yeah
<marcan>
which is also good for us if we need to let macos actually run in this mode
<sven>
so all this crap is disabled from hypervisor.framework
<sven>
but i'm pretty sure you can enable it
zkrx has quit [Ping timeout: 252 seconds]
<sven>
you just need com.apple.private.hypervisor or something like that
<sven>
i'm pretty sure they *want* to run xnu in el1
<marcan>
I'm pretty sure they are planning on making this work at some point in the future
<sven>
yes
<marcan>
also fwiw I don't care *too* much about w^x; I already had a similar problem with EL0 and solved it with a mirror. wouldn't be a big deal doing something similar for call() in the proxy if it comes to that.
<sven>
yeah, so if you don't care about rwx it's very simple to let m1n1 run with sprr as well
<sven>
it kinda already works if you keep the MMU disabled ofc
<marcan>
I only use self-modifying code from python; m1n1's stuff should already be w^x and properly page aligned, I did it properly from the get go
<marcan>
just never bothered enforcing it
<sven>
nice
<sven>
so then we probably just need to move the stack to a rw alias
<sven>
and the malloc stuff
<marcan>
more like move call() to point to a rx alias
<marcan>
I don't think I ever call() anything that is position-dependent
<marcan>
(and if I do I can fix that)
<sven>
oh, so you mean enable SPRR/GXF in call() and disable it before a return? that should work
<marcan>
oh, I just mean enable it whenever I feel like it, and then just keep call() working by pointing it at a rx mirror
<marcan>
while memory is normally otherwise rw
<marcan>
(excepe m1n1 .text)
<marcan>
*except
<sven>
ohh... yeah
<sven>
makes sense
<marcan>
will probably need to extend your pagetable code to support actual page granularity, or otherwise align m1n1 to larger blocks
<marcan>
... thankfully, I already wrote fancier pagetable code for the HV which I can just copy and paste and simplify :p
<sven>
:-)
<marcan>
(or even probably use as-is if I clean it up a bit)
zkrx has joined #asahi
<sven>
okay, let me clean up my current code which just disables the mmu for p.gl1_call and then write some documentation (and/or a blog post because it's kinda awesome how you can figure this all out by (un)educated guesswork)
KindOne has joined #asahi
<marcan>
actually, there is some reason to implement self-reloading for m1n1, which would amount to copying .data from a stash and clearing .bss and jumping to the entry
<marcan>
which is a reason to have the .data image in the mach-o not be the real link address
<marcan>
and then I can align the latter
<marcan>
anyway, ideas ideas
<marcan>
good night, seriously
<marcan>
:p
<sven>
night :D
maknho_ has joined #asahi
maknho has quit [Ping timeout: 240 seconds]
jeffmiw has joined #asahi
riker77 has quit [Quit: Quitting IRC - gone for good...]
<lazarosser[m]>
Is there an eta to when Asahi will be ready?
<clover[m]>
Linux 5.13 I think
* eta
waves
<hrnz>
:)
<lazarosser[m]>
<clover[m] "Linux 5.13 I think"> Will I be able to install on ipad pro?
<clover[m]>
No clue. That would be OP though
<j`ey>
lazarosser[m]: that's not an aim at the moment
<j`ey>
since there probably needs to be a boot exploit found for that
<lazarosser[m]>
That makes sense.
<sven>
it's far from "ready" once 5.13 is released
<clover[m]>
Im fine with my pinetab for now :p
<lazarosser[m]>
Since there is work essentially rewriting the whole stack
<lazarosser[m]>
Does this mean that m1 macs could technically be RYF compliant?
skg has quit [Quit: 〜バイバイ〜！]
skg has joined #asahi
<clover[m]>
Ido that acronym
<clover[m]>
* Idk that acronym
<lazarosser[m]>
<clover[m] "Ido that acronym "> RYF "Respects Your Freedom" it is a thing the FSF likes. It means everything is open source in the hardware. With no proprietary blobs.
<j`ey>
lazarosser[m]: reat marcan's recent tweets about it
jeffmiw has joined #asahi
jeffmiw_ has joined #asahi
jeffmiw has quit [Read error: Connection reset by peer]
matt6 has quit [Quit: Connection closed]
prusnak has left #asahi [#asahi]
jeffmiw_ has quit [Remote host closed the connection]