<marcan>
figured out the sError thing. turns out mapping device memory as nGnRnE isn't enough to guard it, you need to also mark it non-executable...
<marcan>
the M1 decided to speculate an instruction fetch from address 0.
<marcan>
that is very weird, though, because you'd *think* that means it would also break with the MMU off, where there is no such thing as non-executable...
<marcan>
but then again I only saw this behavior, so far, on return from a page fault, which can only happen with the MMU on, so...
<marcan>
anyway
<marcan>
speculation/prediction strikes again
<marcan>
this thing really does punch you in the face if you miss any of these details, moreso than any other CPU I've ever used
<modwizcode>
I feel like that's because most other CPUs aren't quite as speculative and the intel and amd ones have a microcode to make it behave more probably
<modwizcode>
Also I wonder if some of that speculation is disabled unless the MMU is on, like caching.
<marcan>
could be
<marcan>
though the problem happened with caches off too, just with MMU on
<modwizcode>
Yeah but I mean like the whole caching system is disabled with MMU off
<marcan>
sure
<modwizcode>
I'm guess it's a good gate to wait to enable anything that might be buggy until there's an OS capable of dealing with it
<modwizcode>
Anyway how'd you figure this one out?]
<modwizcode>
Everytime you write to more HID registers I get to stub them out in my qemu branch :p
<marcan>
well the L2 subsystem address was basically zero (ignoring the high bits which seem to have a special meaning)
<marcan>
so... it sounded like something was hitting zero
<marcan>
so I tried just unmapping zero and it worked
<marcan>
and then I thought something must be broken because it's nGnRnE mapped and yet something was still hitting it speculatively... in what looked like an instruction fetch path even...
<modwizcode>
I still haven't watched the chicken bits stream, from the registers I've had to poke, what's the L2 stuff pertain to?
<krbtgt>
i wonder if a lot of these... quirks is part of M1's perf crown
<modwizcode>
Did it give you a signal that it was an instruction path?
<marcan>
modwizcode: it was happening on *return* from a page fault
<modwizcode>
I'm more impressed that they can disable all the broken stuff
<marcan>
so... what else would it be?
<modwizcode>
Oh
<modwizcode>
I didn't realize you meant literally... return
<marcan>
like the PC wasn't advancing at all
<modwizcode>
I feel like I've seen that note you screenshotted there
<marcan>
anyway, the L2 error reporting can be disabled, which is what corellium do - presumably because they have bad address faults lurking and they didn't want to bother fixing them
<modwizcode>
Right so I was wondering if they disable that, but thing still work, is it just subtly broken?
<marcan>
well a bad read or write from unmapped address ranges just does nothing, other than report the error
<marcan>
actually a bad read will report a sync error
<marcan>
but a bad write, or speculation, or any of that stuff, is just SError
<marcan>
and those can be disabled in the control register
<marcan>
but that still means you're doing something wrong
<marcan>
and it might break other things
<modwizcode>
wait what is SError? I thought it meant secure until now but then I thought it was sync.
<marcan>
System Error
<modwizcode>
ohhhh
<marcan>
it's an asynchronous exception
<modwizcode>
Yeah okay that makes sense
<marcan>
like MCE on x86
<modwizcode>
your patch setting TTBR1_EL2 faulted under emulator and I realized that I needed to set some id bits to enable the EL2 virtualization extension fully.
<modwizcode>
Do you have a dump of the ID registers handy by any chance?
<marcan>
which ones?
<marcan>
E2H?
<modwizcode>
ID_AA64MMFR[0-3]_EL1
<modwizcode>
and there's a few others I might need but those ones I know I need now
<modwizcode>
arm 64 is such a weird architecture, there's so much crap that's shoved in using the MSR type functionality, and they have crazy names for everything. The TLB flush instruction has insane argumentation. (I think it's the tlb flush one?)
<marcan>
>>> mrs(ID_AA64MMFR0_EL1)
<marcan>
0x12120f100001
<marcan>
>>> mrs(ID_AA64MMFR1_EL1)
<marcan>
0x11212100
<marcan>
>>> mrs(ID_AA64MMFR2_EL1)
<marcan>
0x1201111100001011
<marcan>
and yeah, half the documentation is incomprehensible
<modwizcode>
ty
<marcan>
I used to like ARM, but I think I don't any more
<modwizcode>
I should just pull open the actual architecture reference PDF which is 1000x better but I'm lazy
<marcan>
PPC is way saner, at least the docs
<modwizcode>
Yeah same here lol
<modwizcode>
I like PPC's assembly syntax better
<modwizcode>
Although I've used it less
<marcan>
I'm fine with arm asm
<marcan>
32 and 64
<marcan>
the core stuff I mean
<modwizcode>
Thanks for getting me the registers btw
<marcan>
but stuff like the cache ops etc get pretty ridiculous
<modwizcode>
Yeah I was looking at the MMU enable to make sure I didn't have to touch it and I was like what are these arguments
<davidrysk[m]>
how much of it is ARM and how much of it is Apple's version of it?
<modwizcode>
point of unification, point of coherance sort of make sense, but then you have inner shared, outer shared.
<modwizcode>
Most of it's arms
<modwizcode>
*ARM
<modwizcode>
Apple's stuff is mostly just model specific registers to poke at and disable/enable added functionality they've got from what I can tell, marcan probably knows more
<marcan>
yeah, we're talking about the ARM stuff
<marcan>
Apple's stuff is actually much simpler to understand
<marcan>
I mean the chicken bits are deep CPU feature stuff, so of course nobody knows what those really do
<marcan>
but it's just a pile of enables/disables/knobs for that stuff
<marcan>
the incomprehensible stuff is things like ARM's hypervisor and MMU control registers
<modwizcode>
Oh god ARM's hypervisor thing I thought was going to make sense and it totally doesn't
<modwizcode>
It makes no sense at all
<krbtgt>
marcan: wrt ppc docs, my problem after reading them is IBM bit ordering is the opposite from everyone else
<marcan>
oh yeah, big endian land
<marcan>
but everything *else* is sane
<marcan>
:p
<krbtgt>
marcan: acxtually not
<modwizcode>
Except when PPC decided to be little endian
<krbtgt>
they order bits from least signigicant *bit* on the left
<modwizcode>
oh god they're like the ethernet people
<davidrysk[m]>
what about G5 and all of its idiosyncrasies?
<marcan>
krbtgt: no they don't
<marcan>
they order bits from least significant bit on the right
<marcan>
they just *call* the most significant bit, on the left, bit 0
<krbtgt>
ah right, i mixed it up
<modwizcode>
ARMs docs also do something I find really insufferable which is instead of documenting how enabling one bit that affects others does it, it documents in every other bit how that bit is affected. Which spreads out the meaning of things like the E2H bit into everything. The best part is the E2H bit documentation says "See E2H bit for more details"
<modwizcode>
marcan: do the barriers you added before print_regs calls have visible effects?
<davidrysk[m]>
I've seen someone say that ARM documentation "reads like stereo instructions"
<modwizcode>
I don't know how to interpret that but I'll take it
<davidrysk[m]>
ever dealt with boring dull appliance manuals? :P
<modwizcode>
also is ther eany more details about what SYS_CYC_OVRD does? From m1n1 it seems like it masks interrupts somehow? Why do they have a second place to mask interrupts?
<marcan>
modwizcode: I want to make sure that speculation doesn't mess up ordering of the L2 error status registers; before I added the one after that sometimes it was already SErroring before it *really* returned from the page fault
<marcan>
the wonders of speculation
<modwizcode>
that's amazing
<marcan>
this is low level enough that I don't really mind the shotgun approach to barriers, figuring out the details with all of these system and even undocumented MSRs is not worth it
<marcan>
for Linux using standard stuff we do need to know the specific requirements though
<modwizcode>
Yeah it's basically impossible I think
<modwizcode>
To understand them completely with how much tweaking can be done at a microarchitectural level. Like I said I'm sort of in awe that they had the foresight to have so many of these be tweakable.
<marcan>
the thing is all CPUs are like this
<modwizcode>
I know that all cpus are like this, but knowing what's going to be broken ahead of time is a skill.
<marcan>
Intel and AMD just hide it behind microcode blobs (that contain startup sequences that just poke chicken bits, among other things) and proprietary bringup blobs (AGESA etc)
<marcan>
everyone puts a zillion chicken bits in their cores
<marcan>
the fun thing here is Apple delegates fixing all this stuff to the OS kernel *and* documents the bit descriptions
<modwizcode>
Yeah that's interesting
<marcan>
so we get to see deep CPU design knobs that we usuall don't
<marcan>
+y
<modwizcode>
I mean also though I wonder how much of this complexity the typical arm core even has.
<marcan>
well, judging by the pile of "work around cortex-whatever" errata fixes in the linux kernel...
<modwizcode>
hmm maybe I'm not enough aware of those yet
<marcan>
I'd say ARM have done a much better job of making those chicken bits ahead of time instead of hideous workarounds the kernel needs to apply
<marcan>
probably, anyway
<marcan>
er, Apple
<modwizcode>
I wonder if not poking these bits at all would leave you with a functional core enough to work around it
<marcan>
in some cases, probably
<marcan>
also of note, chickens.py used to set all the chicken bits backwards
<marcan>
and it still mostly worked :)
<marcan>
I only noticed after applying it after the C code ram and noticing it was undoing everything
<marcan>
I had set/clear switched :)
<davidrysk[m]>
oops :)
<marcan>
so a lot of this stuff is not completely obvious
<modwizcode>
LOL. did you see my question about the SYS_CYC_OVRD bits? I'm guessing CYC refers to cyclone, but what's the deal with those and IRQs?
<davidrysk[m]>
a lot of it probably comes up when you're doing more complex stuff
<marcan>
I think that register got introduced with cyclone and then the name stuck
<marcan>
and I think those can be used for IRQ injection maybe?
<modwizcode>
oh interesting
<modwizcode>
example of horrible arm instruction "tlbi vmalls12e1is"
<ldhacker[m]>
What assembly language should I be learning for the M1? Just trying to create a hobby OS that I don’t plan on releasing to learn the OS DEV side of things. Also, any book/reference recommendations?
TheJollyRoger has joined #asahi-dev
<comex>
modwizcode: still beats the arm 32-bit version, mcr p15, 0, r0, c8, c5, 0
<comex>
i mean, at that point, why do you even have an assembler
<modwizcode>
No I like that version!
<comex>
if you're going to have a bunch of magic numbers, why not just hardcode the instruction word
<modwizcode>
The documents have a sort of saneness as to what each register mean
<modwizcode>
*er each subop
<modwizcode>
I used to sort of know it when I was reversing wii stuff
<comex>
disagree :)
<davidrysk[m]>
marcan: any plans to do another stream sometime?
<modwizcode>
Anyway I better get to bed before someone gets mad at me
<modwizcode>
night all
<davidrysk[m]>
night modwizcode
<marcan>
davidrysk[m]: I don't really plan them, depends on what I'm doing :)
<marcan>
but there will be circuit design streams soonish
<davidrysk[m]>
ldhacker: M1 uses aarch64... I'd also recommend any good computer architecture books
<davidrysk[m]>
marcan: cool :)
<ldhacker[m]>
Thank you!
amw has joined #asahi-dev
VinDuv has joined #asahi-dev
<amw>
Is the m1n1 loader / kernel stable enough to try booting on a MacBookAir M1 with out serial cable?
<amw>
If so I was thinking about trying to get this to happen and documenting the setup of the booting
<amw>
If it's not obvious - I don't have a magic USB Type-C serial cable nor likely to get one anytime soon and if MacOS can easily dual boot / recover it would be safe?
<amw>
The framebuffer displays on the screen - there's no input till USB / Keyboard is up but still let's one experiment via printk?
<VinDuv>
AFAIK the boot process is very safe, you can always hold the power button to change the boot device back to macOS; if the recovery partition gets damaged (very unlikely) the Mac won’t boot but can be restored using another Mac with DFU mode (see https://github.com/AsahiLinux/docs/wiki/M1-vs.-PC-Boot)
<VinDuv>
But unless I’ve missed something, m1n1 is not currently able to load a Linux kernel and initramfs from disk; you have to transfer them over serial
<amw>
VinDuv: Ok - Some how I thought the disk was working
<amw>
Would it be possible to concatenate the Linux kernel + device tree to m1n1 ?
kit_ty_kate has quit [Ping timeout: 272 seconds]
<VinDuv>
amw: I guess so, but you’ll have to re-“bless” m1n1 every time you modify it I think
<amw>
VinDuv: Good point - but still really fun for me to get working with on my MBA - perhaps not practical for serious debugging though.
amw has quit [Ping timeout: 240 seconds]
<VinDuv>
of course :)
<marcan>
oh, they left
<marcan>
I was going to say there is no way to brick these machines as long as you don't write to NOR flash
<marcan>
... I have no idea why corellium actually declare the SPI flash in their device tree, unless they have to read anything from it (which I doubt, iBoot should put everything necessary in the device tree) we are definitely *not* doing that
<marcan>
that is a recipe for a bad codepath/script gone rogue bricking machines
<marcan>
anyway, m1n1 will be usable for normal kernel boot soon, as soon as I get SMP up I'll write in appended payload support
<marcan>
right now serial is required
<marcan>
that should be done today
<kettenis>
marcan: some of ARM's own cores do the speculative execution of nGnRnE as well
<Bluerise>
marcan: on some machines, like an APU, it's helpful for firmware upgrades. but on the Macs I wouldn't dare change anything like that, it'd be stupid.
amw has joined #asahi-dev
<Glanzmann>
marcan: Look forward to m1n1 to be able to boot the payload with out usb cable.
<marcan>
Bluerise: leave firmware updates to macos :)
<Glanzmann>
marcan: Have you already all the device drivers from corellium in your kernel or are there bigger changes required?
<marcan>
Glanzmann: I am not merging corellium drivers right now
<marcan>
I am getting the core platform support clean first
<Glanzmann>
I see.
<marcan>
I have no reason to rush things, this isn't a race
<marcan>
they are hacking on things as fast as possible, that is not our goal here :)
<marcan>
I spent yesterday working out what was causing bus errors on m1n1 - corellium just turns off bus error reporting
<Glanzmann>
Yeah, I read that in the backlog somewhere. :-)
<amw>
marcan: When you have m1n1+linux+dtb+initramdisk booting do we need a new partition for it?
<marcan>
no, there is no storage driver yet in my tree anyway
<marcan>
but I would recommend having a secondary macos install you use for linux, instead of messing with your primary one
<marcan>
and then having a separate partition prepared for when the storage driver is in
<amw>
Is there a guide to making a secondary MacOS install? Google?
acelogic has quit [Ping timeout: 265 seconds]
<marcan>
writing documentation is part of my task list now that this is getting usable for people :)
<marcan>
I'll write something up today/tomorrow
<amw>
BTW The m1n1+linux+dtb+initramdisk blob wouldn't need a storage driver as it has the init ramdisk?
<marcan>
yes
kettenis has quit [Quit: leaving]
<amw>
So that means as long as the 1TR can access the bless blob it can be selected in the 1TR mode or something?
<marcan>
if you have two partitions, you can select your OS in the boot picker normally
<marcan>
no need to keep kmutiling, and no need to screw up your macos partition (Apple's updater probably won't be very happy if you keep replacing the macos kernel)
<marcan>
this also means different SEP contexts and such once we get to implementing that
<marcan>
it's the right way to do it
<amw>
ok - using boot picker to boot our blessed m1n1+what ever is the easy way.
<marcan>
yup
kettenis has joined #asahi-dev
kettenis has quit [Client Quit]
<amw>
Lokking forward to it - happy to write up my experiences if that's any help
<marcan>
it'll be helpful for people to follow the process and point out any gotchas :)
kettenis has joined #asahi-dev
<amw>
It's wonderfull learning so many new tools and interesting things - thanks
<Glanzmann>
marcan: I'll try as soon as the instrcutions are up.
TheJollyRoger has quit [Ping timeout: 268 seconds]
<modwizcode>
marcan: no idea if it's been covered but do you expect Linux to have access to store secrets in the secure enclave? Obviously it won't have OS X access but would it theoretically supprot storing linux keyrings and the like?
TheJollyRoger has joined #asahi-dev
<marcan>
modwizcode: that's one of the goals, yes
<marcan>
I definitely want to be able to use it for U2F and SSH keys if possible
<marcan>
I don't see why it wouldn't be
<modwizcode>
Yeah I just wasn't sure if they made the effort to properly support access outside OSX, I think it's probably easier to not do that so who knos
<marcan>
I mean, what's OSX?
<marcan>
it's not like the SE knows we're not OSX
<marcan>
*SEP
<modwizcode>
I mean I guess it knows what got booted though
<marcan>
it knows a mach-o kernel got booted
<modwizcode>
But you're right it doesn't know it's not OS X
<marcan>
I mean I already tested booting a patched darwin (I just changed the version number)
<marcan>
works fine
<modwizcode>
That's reassuring
<modwizcode>
Why does aarch64 have so many damn timers
<kettenis>
virtualization
taziden has quit [Ping timeout: 265 seconds]
acelogic has joined #asahi-dev
taziden has joined #asahi-dev
<vafanlignarde>
Bluerise: if you happen to have one of those YZStudio ZY12PDN USD-PD triggers, there's a FUSB302 chip on those (you still need a type-c breakout and level converter though). Someone recently posted an OSS firmware. I've attempted to get Apple VDM working on top of that code although I've only succeeded with UFP and Get Action List so far: https://git.io/JtBD7
<Bluerise>
Heh, nice
<dhewg>
vs? c++ firmware? what?
<Bluerise>
vafanlignarde: I have a setup with a hummingboard pulse (ptn5110, tcpci compatible) right now
<Bluerise>
which I know kettenis has as well
<Bluerise>
but I might order two... need to think about it. this would allow reducing one device from being complex to being dumb ;) which is an improvement if things work
<Bluerise>
bbl, grabbing food
<vafanlignarde>
Bluerise: neat. are there breakout boards easily available for that chip?
<Bluerise>
vafanlignarde: which one did you get, the one with screw terminal, bare or usb cage?
<Bluerise>
for the ptn5110? haven't seen any :/
<Bluerise>
so far I've seen that one on the i.mx8mq eval kit and the hummingboard pulse
<Bluerise>
oof, if I pay double it'll supposedly arrive in 3 weeks instead of 4-5
<vafanlignarde>
Bluerise: i had a few with screw terminals (ZY12PDN-3) but I found that I needed to desolder the terminals to be able to solder wires for the UART onto the MCU.. the ZY12PDN-1 comes without a screw terminal.
<Bluerise>
Ah.
<Bluerise>
kettenis: I ordered two. Will take a month to arrive at my place, then I'll see how I can flash i
<Bluerise>
so this week I'll build you one with the hummingboard, next month we might not need the hummingboard anymore
<kettenis>
cool
<Bluerise>
and in two months marcan might have serial prototypes :P
<kettenis>
you're taunting him?
<Bluerise>
Nah, just looking forward to a solution that stays on a single pcb, without any weird wires...
<bastilian>
I'm hoping that the M1 to M1 via USB-C will work soon.
<Bluerise>
Hm, does the debug mode reset if you pull the USB-C plug?
<Bluerise>
or does serial continue to be available once enabled?
<j`ey>
bastilian: oh no, ill have to buy another m1 :P
<bastilian>
hehee. :D been there.
<bastilian>
I'm thinking, that at some point there could be a similar solution for non-M1 machines and not as involved as a USB-C PD UART DIY cable, just a proper USB-C (PD) cable.
<bastilian>
Bluerise: I don't know, that may depend on the machine. maybe. I really don't know, i try to piece together things, but my electronics and low level hardware knowledge is limited. ...so don't listen to me rambling. :D
amw has joined #asahi-dev
acelogic has quit [Remote host closed the connection]
<rwhitby>
I'm not aware of any breakout boards for ptn5110. I have lots of different proprietary boards with it on. Fun fact - the discontinued Valve VirtualLink adapter for the Index VR headset has one in it.
<Bluerise>
rwhitby: it's in stock at NXp, 67$ including shipping to .de
<rwhitby>
SWD doesn't have test pads on the WiTRN UPD005 - there is a windows tool which updates it over DFU.