<daurnimator>
andrewrk: to take advantage of Os level batch and async operations, we need to have a work queue type structure.
<daurnimator>
I was working on it back along with the uring work.
<andrewrk>
daurnimator, sounds like a different way to do the implementation, but the API could be the same
<daurnimator>
andrewrk: yeah sort of; but the API can be more advanced; e.g. linux now lets you submit dependant operations: "read from X then write the result to Y"
<andrewrk>
that's neat
<daurnimator>
so yes we can have a simpler read() API on top of it, but it would be on top of the more advanced lower layer
<daurnimator>
linux 5.4 brings timeouts into the equation too, so you can ask the kernel: accept() from this socket in parellel with read() from these 5 file descriptors; when the accept returns read(); but timeout the read after 2 seconds
<daurnimator>
At some point that API looks like: a = q.queueAccept(); b = q.queueRead(); .submit(2); .waitAnyResult([_]T{a,b})
<daurnimator>
at least on linux, only the .submit() actually requires a syscall
<andrewrk>
that's great
<daurnimator>
on windows I can make 80% of it work, the last 20% needs threads. On other OSes we will need threads....
wootehfoot has quit [Read error: Connection reset by peer]
lunamn_ has joined #zig
lunamn has quit [Ping timeout: 252 seconds]
<daurnimator>
trying to manage the read/write buffer ownership for .queueRead and .queueWrite operations was half of the motivation for the fifo work
FireFox317 has joined #zig
FireFox317 has quit [Ping timeout: 240 seconds]
MatrixBridge has joined #zig
MatrixBridge has left #zig ["User left"]
MatrixBridge has joined #zig
MatrixBridge has left #zig ["User left"]
MatrixBridge has joined #zig
MatrixBridge has left #zig ["User left"]
chr-1x has joined #zig
chr-1x is now known as chr[m]
chr[m] is now known as chr-1x
chr[m] has joined #zig
<chr[m]>
test
<chr[m]>
cool matrix bridge is working
chr-1x has quit [Remote host closed the connection]
<chr[m]>
hey so i read through the zig documentation but i'm left with some questions. are there any other resources for learning about zig out there?
<andrewrk>
chr[m], afraid not - the language is still changing, so the main docs are the best bet at this point
<chr[m]>
alright, thanks
<chr[m]>
just trying to get my feet wet. the docs have a lot to say about `for` and `while` but i don't see an example of your basic for loop over a range of integers. is there any kind of range syntax or do i just build it out of variables and continue expressions?
<pixelherodev>
There is no for loop over integers in zig
<andrewrk>
chr[m], generally, when a more general syntax suffices, zig does not provide syntax sugar, unless it encourages better practices
Ichorio has quit [Ping timeout: 245 seconds]
<chr[m]>
okay, cool
<chr[m]>
might be worth having such an example in the docs, i imagine i'm not the only person to have asked this question. it's a good reference point for people coming from another language.
<andrewrk>
agreed
<daurnimator>
chr[m]: that said, we're still thinking about generators
<chr[m]>
i've found generators generally pretty useful in other languages, so i'm happy to hear that
<chr[m]>
oh, the other question i had: `fn foo(a: var) var`
<chr[m]>
i'm seeing stuff like this in the standard library docs but the language docs don't explain what this means
<chr[m]>
it seems to be some kind of type-deduced template magic
<andrewrk>
chr[m], this is a deficiency in the generated docs. sometimes you'll have to look at the source for now
<chr[m]>
hmm
<chr[m]>
but what about e.g. `pub fn tagName(v: var) []const u8`
<chr[m]>
i interpreted that as referring to integers only
<daurnimator>
(FYI to find that out I opened up std/fmt.zig and search for "float")
<chr[m]>
cool
SimonNa has quit [Ping timeout: 246 seconds]
SimonNa has joined #zig
protty has joined #zig
<protty>
daurnimator: the waitAnyResult() sounds similar to the `select()` operation for channels. Could maybe treat any asynchronous function as a oneshot channel? Would make timeouts pretty easy (e.g. `select(async op(), async sleep(timeout))`). Using this, how would you go about cancellation logic?
<daurnimator>
protty: are you talking in Go?
<protty>
mostly in reference to rust futures
<daurnimator>
protty: with that, how would you submit dependant operations?
<protty>
theres another primitive for that `join()` which waits for all given operations to complete instead of just the first one like `select()`
<daurnimator>
huh?
<daurnimator>
protty: e.g. if I want to submit to the kernel: "read from fd 1; *then* write to fd 2" => how would that look?
<protty>
ah, that could be just a normal function which awaits the result. Could maybe do some chaining like `then()` as seen in rust futures or js promises
<daurnimator>
protty: no; this happens *in kernel*
<daurnimator>
you need a way to describe to the kernel what you want to be dependant on what: and then let the kernel do it
<protty>
yes, the building of these operations can be done with the combinators like `then()` and finally submitted to the kernel: similar to what rust iterators do
<daurnimator>
then how would you e.g. cancel an operation?
<andrewrk>
hi protty
<protty>
hey andrewrk
<daurnimator>
`const a = q.queueRead(.{fd = 2, buffer = mybuffer}); const b = q.queueWrite(.{fd = 3, buffer = mybuffer, after = a}); q.submit(2); q.waitForAny(b);`
<protty>
cancellation is the part that im scratching my head at. In rust, its done by dropping (deleting) the computation which does whatever cancel signaling that needs to be done. Zig doesnt have destructors so the same semantics cant apply. The first idea that comes to mind would have to be done manually which isnt too ergonomic
<andrewrk>
protty, did you see the `cancel` keyword that we almost got?
<daurnimator>
for cancellation in the above scheme I was thinking: `q.cancel(a);`
Cucumbas has joined #zig
<protty>
daurnimator: yea, looks like we have the same idea on that part
<protty>
andrewrk: I remember it being a thing for a split moment but dont remember the functionality. Is it still the same as whats in the 0.4 langref?
<andrewrk>
there's no cancel right now, it all would have to be done manually
<andrewrk>
I haven't fully given up on it. I think cancel support in the language would work really well with async function semantics
<andrewrk>
so, if an async function did not explicitly add support for checking the cancel bit, cancel would be identical to await, except it would also run the possible cleanup handling function
<andrewrk>
e.g. cancel would be a suspend point, just like await
<andrewrk>
but then an async function would be able to atomically read the cancel flag bit to see if a cancel request had happened. so if an async function, for example, had some long running loop, it could look for that flag in the loop
<daurnimator>
"how to clean up return value" annotations would be a good improvement in general I thinkj
<protty>
daurnimator: for manual cancel, one idea could be that `waitForAny/select` take a group of types which implement `.cancel()` and `.get()` then provide a type which wraps any async function to provide those functions
<daurnimator>
protty: I don't understand
<daurnimator>
protty: just because I'm waiting for one doesn't mean I want to cancel the operations on the others...
<protty>
for that case, wouldnt a join() be more appropriate than a select()?
<daurnimator>
e.g. I might have 3 things I need to do, and I'm waiting for external data so I can progress on *any* one of them
<protty>
if the 3 things can be done in parallel after the external data is resolved, could be modeled with something like `then(external_data, join(3_things))`
<daurnimator>
protty: I meant 3 different pieces of external data
lunamn_ has quit [Quit: leaving]
traviss has quit [Remote host closed the connection]
<protty>
and you want to wait for all of them to complete? or one to complete without cancelling the others?
<daurnimator>
one to complete without cancelling the others
<daurnimator>
^ that's what I intended .waitForAny to do
<daurnimator>
if I wanted to cancel then I'd do something like: `const x = q.waitForAny([_]T{a, b, c}); if (x != a) q.queueCancel(a); if (x != b) q.queueCancel(b); if (x != c) q.queueCancel(c); q.submit(3);`
<daurnimator>
uh, .submit(2) at the end there
<protty>
believe thats the default behavior of select() then. cancellation when one is resolved is optional but was incorrectly assuming that was the scenario you were talking about
<protty>
*cancellation of the others when one is resolved
<daurnimator>
btw I was thinking: `fn cancel(q: *@This(), op: Operation) !void { const x = try q.queueCancel(op); try q.submit(1); _ = q.waitForAny([_]Operation{x}); }`
<daurnimator>
and similar for e.g. q.read() and q.write() operations that all just wrap their .queue variants
protty has quit [Remote host closed the connection]
protty has joined #zig
<daurnimator>
zig-cache/o/9N2qH3ELa1NeBb2iSyGvyHsl72OCQ93CWdL5Hh5Y9-XN_aK36LHfr2quv5zf2dtm/cimport.zig:39:5: error: declaration shadows primitive type 'u8'
muffindrake has quit [Ping timeout: 246 seconds]
Cucumbas has quit [Remote host closed the connection]
<pixelherodev>
Think I got the OS layer working!
muffindrake has joined #zig
<pixelherodev>
On a different note, it's nice that I no longer need `catch unreachable` after std.io.getStdOut()
<pixelherodev>
... ugh, it's rebuilding musl again (probably because of the Zig upgrade).
<pixelherodev>
Okay, so it needs a bit more work. Seems to be halting at some point - not panicking - not 100% sure what's going on...
Cucumbas has joined #zig
<pixelherodev>
Huh. This is distinctly odd. It's freezing in the ACPICA layer now for some reason, which is *entirely unrelated to the changes I just made*
<Cucumbas>
when exporting functions that are in a struct, the generated C header makes no mention of the struct. I mean, I wouldn't want it to generate a namespace in the C file cause I don't think that's compatible with C89 but maybe there could be a way to include the parent struct name? like as a prefix to every funciton in the generated C header?
<pixelherodev>
I suspected that it was panicking and I broke stack traces, but I double checked and panicking works fine...
<andrewrk>
pixelherodev, you upgraded zig yeah? perhaps a regression
<pixelherodev>
Maybe, but this is in a layer written in C.
<pixelherodev>
So... yeah...
<andrewrk>
ah
<pixelherodev>
Unless it's in the Zig layer *to that C code*
<pixelherodev>
ACPICA requires a bunch of functions to be implemented by the OS itself
<pixelherodev>
e.g. memory allocation
<andrewrk>
Cucumbas, generating .h files is at proof-of-concept stage. probably your best bet for now is to write your own .h files
<pixelherodev>
So it might be one of those functions is encountering a regression
<pixelherodev>
Might not be a coincidence that the last line printed over serial is "Allocated 0x0050 @ 0x0000844b for ACPICA"
<pixelherodev>
Except that function literally returns immediately after printing... whelp. I'll let you know if/when I fix this
doublex__ is now known as doublex_
<daurnimator>
andrewrk: got any idea for a workaround for cImport generating types with names of reserved/builtin types?
<pixelherodev>
Can you change the header?
<andrewrk>
daurnimator, that's a good question
<daurnimator>
pixelherodev: no
<pixelherodev>
Write a C shim?
<daurnimator>
pixelherodev: nope
<pixelherodev>
Why not?
<pixelherodev>
Too complex?
<daurnimator>
pixelherodev: its the linux kernel headers picked up from /usr/lib/modules
<daurnimator>
they change on each kernel upgrade
<daurnimator>
so not as if I can bundle them
<pixelherodev>
Ah
<daurnimator>
this essentially blocks writing a linux kernel module in zig
<pixelherodev>
Yeah, that's problematic
<andrewrk>
daurnimator, are they macros? global variables? function names? fields?
<daurnimator>
andrewrk: the kernel headers do e.g. `typedef char u8`
<Cucumbas>
andrewrk: Ok cool. You know what, I need to read the release notes, thanks.
<pixelherodev>
daurnimator, are all of the types compatible?
<pixelherodev>
Maybe just redefine them, giving them a different name
<pixelherodev>
If it's just types, @cDefine them to change them in the header
<pixelherodev>
The produced definitions should be ABI compatible
<andrewrk>
daurnimator, I think this case could be solved by improving translate-c to detect that these are redundant definitions
<andrewrk>
there is still the general problem to solve, but this would at least unblock this use case
<daurnimator>
andrewrk: can we get around them with @field? => in translate_c.cpp we could detect if the token we're about to emit is reserved and surround it with @field?
<andrewrk>
oh, they're not redundant though. c_ushort != u16, for example
<andrewrk>
the problem isn't referring to it though; the error is about shadowing it. so @field would not work around it
<daurnimator>
andrewrk: what do you mean they're !=?
<daurnimator>
andrewrk: the headers do all the #ifdef-ery to make sure they're correct.
<andrewrk>
literally in zig: `comptime assert(c_ushort != u16);`
<daurnimator>
andrewrk: ah right
<daurnimator>
andrewrk: huh. why is that?
<pixelherodev>
Because it's not always true?
<andrewrk>
because the C integer types have different bit widths based on targets
<pixelherodev>
The types themselves are distinct, even if they're usually (always) semantically identical
<daurnimator>
pixelherodev: isn't andrew saying that its never true?
<pixelherodev>
Right, the types are never equal
<andrewrk>
it matters for the C ABI
<daurnimator>
andrewrk: I thought c_ushort would just be an alias for the relevant u16/u32/etc.
<andrewrk>
that's something that is worth considering. these definitions could move to std lib if that strategy is workable
<daurnimator>
oh interesting, under std.c or something?
<andrewrk>
yeah
<daurnimator>
though at the moment std.c is for libc :P
<daurnimator>
I guess std.target
rjtobin has quit [Quit: Leaving]
<andrewrk>
std.c could be both. c has a global namespace
<daurnimator>
I think separating "C the language" vs "C ABI" vs "libc" is important
chemist69 has quit [Ping timeout: 246 seconds]
<andrewrk>
well we'd need the types in there anyway
<andrewrk>
most libc functions return c_int
<andrewrk>
which would be renamed to std.c.int
<pixelherodev>
....annnnnd stack tracing is panicking now too :(
chemist69 has joined #zig
<daurnimator>
so fun workaround for now: @cDefine("u8", "zig_antishadow_u8");
kotrcka has joined #zig
<andrewrk>
pixelherodev, here's something to try: zig version ae0a219d1f5495acc4d82421fa24d84186c2a40d
<andrewrk>
that is after @as, but before anon container literals
<pixelherodev>
acls?
<andrewrk>
?
<daurnimator>
andrewrk: oh that reminds me. `zig BUILD_INFO` undocumented... and seems to have a hard-coded path to my build location (which is a no-no from a packaging standpoint)
<andrewrk>
daurnimator, if you're making a package, you can remove that feature. it's only used to pass information to the self-hosted compiler
<andrewrk>
sorry; it's only used to pass information to build.zig, for configuring the build of the self-hosted compiler
<pixelherodev>
andrewrk... will do, but I don't think that's related
<pixelherodev>
Stack tracing's been very very fragile this whole time
<daurnimator>
andrewrk: remove it how?
<pixelherodev>
Probably a stdlib upgrade issue
<andrewrk>
pixelherodev, ahh I see.
<andrewrk>
daurnimator, it's unused, if you don't build self-hosted
<daurnimator>
andrewrk: but still: remove it how? is there some cmake option ?
<pixelherodev>
Yeah, it's really annoyingly fragile, and the fact that I in all honesty *barely* understand a lot of the stdlib debugging stuff doesn't help
<andrewrk>
basically it's used to pass information from cmake to build.zig
<andrewrk>
daurnimator, I meant patch it out, if it's causing a problem
protty has quit [Ping timeout: 260 seconds]
emekankurumeh[m] has joined #zig
<pixelherodev>
On a brighter note, got it panicking instead of hanging now :P Forced function call tracing, and apparently there's an unreachable block in there :(
<daurnimator>
andrewrk: at least in the arch linux world, patching out is only considered when the upstream is considered hostile
<andrewrk>
the long term plan is that we ship self-hosted. which has no need for this BUILD_INFO thing
<daurnimator>
but we don't need to worry about that until next release
<andrewrk>
we could also have cmake write a file with this info instead of making it a command in stage1
<emekankurumeh[m]>
i just found a site that has a massive database of windows defines
<pixelherodev>
But as a whole, upstream stdlib works now
<pixelherodev>
The stdlib isn't the problem right now :P
Cucumbas has quit [Read error: Connection reset by peer]
Cucumbas has joined #zig
<chr[m]>
hmm. so i've come across a GitHub issue proposing a feature that i want to advocate for, but the issue is closed. i don't see a way to comment on the closed issue, but i don't want to open a new issue just to present my case, since i'd be more or less proposing the same thing as the original issue. what should i do?
Cucumbas_ has joined #zig
Cucumbas has quit [Ping timeout: 240 seconds]
ur5us__ has joined #zig
_whitelogger has joined #zig
_whitelogger has joined #zig
<pixelherodev>
Huh, now it's working
<pixelherodev>
If i do hbreak *without first trying a normal breakpoint*, it works
<pixelherodev>
Thanks a ton!
<pixelherodev>
Yeah, addresses are all correct now, huge thanks!
<nomadgamma>
:D
<pixelherodev>
Gah - the offset changes whenever I make a slight tweak, so I need to recalculate the offset
<daurnimator>
yay, hello #2231 my old friend
<pixelherodev>
I've come to talk with you again?
<pixelherodev>
Ah - I highly support that proposal
<pixelherodev>
Would come in extremely handy for an existing plugin system I've designed in C
<pixelherodev>
Great. Now even hbreak isn't working again :(
<andrewrk>
daurnimator, if you change that to "ability to set shared object filename" then I'll accept it immediately. problem is that there are many cases where there are more than 1 build artifact, so there is not a single output file name
<daurnimator>
andrewrk: done
<daurnimator>
andrewrk: though I'm not sure how that rename solves the issue: how do you know if you should create the symlinks lib{}.so.x.y and lib{}.so?
<nomadgamma>
pixelherodev, lol fuck
<pixelherodev>
nomadgamma, figured it out, but it's weird
<pixelherodev>
...okay, so for `hbreak` to work, Need to do `file bin/os indomitable` *then* `add-symbol-file bin/os/indomitable CALCULATED_OFFSET`
<pixelherodev>
Then hbreaks work
<nomadgamma>
ahhh, ok that makes sense
<pixelherodev>
Fuck. Uh... this is *worse*
<pixelherodev>
I was right.
<pixelherodev>
There's no panic-within-panic, *it's taking the wrong branch*
<andrewrk>
daurnimator, it sounds like this proposal wants to have an option that overrides that behavior. so you choose the dynamic library filename to output, and zig does nothing fancy like prepending lib or making softlinks
<pixelherodev>
Okay, asm time: "mov 0x6010,%dl ; test %dl, %dl"
<pixelherodev>
Then it `jne`s to the true
<andrewrk>
https://clbin.com/7fZi9 also has expected behavior with -target i386-linux-musl --strip --single-threaded -lc, run with qemu-i386
<nomadgamma>
0x6010 seems really low, is it the same as what's pushed onto stack for print?
<pixelherodev>
Th - wait, that's an *address*... that's... WTF?
<nomadgamma>
wait... it's a constant
<nomadgamma>
...
<pixelherodev>
Yeah, that - that seems to be the address it's given
<nomadgamma>
it's moving a constant into %dl
<pixelherodev>
Wait... wat
<pixelherodev>
No it's not
<pixelherodev>
That's an address
<pixelherodev>
Wrong assembler syntax ;)
<nomadgamma>
ah
<pixelherodev>
If it was a constant it would have a $ prefix
<pixelherodev>
Still, it's being given an absurd address
<pixelherodev>
Yeah, GDB confirms
<pixelherodev>
`print &panicked` shows that address
<pixelherodev>
... whelp, bug found; no clue how to fix thi... wai
<pixelherodev>
wait wait wait
* pixelherodev
bashes head into wall repeatedly
<nomadgamma>
so your code is loaded at low address, so 0x6... is not unreasonable
<pixelherodev>
*My linker script doesn't deal with .data section*
<nomadgamma>
ah
<andrewrk>
you would still expect the value to be a consistent value
<pixelherodev>
Not with that address I wouldn't.
<pixelherodev>
That's probably not even RAM.
<pixelherodev>
Might be MMIO, or belong to BIOS, etc
<pixelherodev>
This is one of the most embarrassing hours I've had recently :P
<andrewrk>
oh, so it could have different values on different reads
<pixelherodev>
The worst part?
<pixelherodev>
*This still doesn't fix panic*
<pixelherodev>
This fixes it jumping to the wrong branch, but the panic function is still borked :P
<pixelherodev>
Wait never mind, my linker script *does* deal with .data
<pixelherodev>
That's - I'm even more confused now
<andrewrk>
what does gdb say the byte of memory is?
<pixelherodev>
0
<pixelherodev>
Wait...
<pixelherodev>
That's...
<pixelherodev>
...
<nomadgamma>
ok so @panicked is at 0x6010, and %dl is the value
<pixelherodev>
Yeah, this is insane.
<nomadgamma>
then it tests %dl
<nomadgamma>
and jumps
<pixelherodev>
Let me do this one last time
<pixelherodev>
... if I can just figure out how to step by one asm instruction...
<andrewrk>
stepi
<andrewrk>
disasm
<pixelherodev>
I have the asm shown, forgot stepi, thanks
<pixelherodev>
Well what do you know. It contains 0xAA
<pixelherodev>
Still no clue why it has that address, but everything else makes sense
<nomadgamma>
do you have an asm dump of the +-15 lines around that point?
<pixelherodev>
Yeah
<pixelherodev>
But again, it doesn't help
<pixelherodev>
The only mistake is the adderss
<pixelherodev>
Just to confirm, globals by default go into .data?
<pixelherodev>
Wait nope, it's in .bss
<pixelherodev>
Good old readelf
<pixelherodev>
s/readelf/objdump
<nomadgamma>
so .bss should be constant and loaded at a specific address
<pixelherodev>
Yeah, and apparently moving .bss before the debug info resulted in it getting a reasonable address...
<pixelherodev>
Except now it's still failing, even with a reasonable address!
<pixelherodev>
0x00123010
<pixelherodev>
Whelp
<nomadgamma>
yea but the global variable shouldn't be in .bss because it's constant
<pixelherodev>
No it's not
<pixelherodev>
It can be set false
<nomadgamma>
I mean .bss is constant
<pixelherodev>
Is it? I thought it was... one sec
<nomadgamma>
oh nvm i am wrong
<pixelherodev>
That's
<pixelherodev>
It shouldn't be in bss though
<pixelherodev>
" statically-allocated variables that are not explicitly initialized to any value"
<nomadgamma>
yea so panicked is global modifiable variable in .bss
<pixelherodev>
It should be in .data
<pixelherodev>
Because it's initialized
<nomadgamma>
hmm
* nomadgamma
shrugs
<pixelherodev>
If I manually set it to linksection(".data")...
<pixelherodev>
Then it works!
* nomadgamma
shrugs more and looks at bottle of whiskey
<pixelherodev>
So, real problem found, this *is* almost certainly a compiler regression
<pixelherodev>
I think something changed causing it to incorrectly view the variable as uninitialized and thus place it in the wrong section
<nomadgamma>
but does it work now?
<pixelherodev>
With that workaround? Yes, sort of.
* andrewrk
checks the IR again
<pixelherodev>
*That* works.
<pixelherodev>
Everything else doesn't.
<pixelherodev>
:P
<andrewrk>
@panicked = internal unnamed_addr global i1 false, align 1
<andrewrk>
it's initialized
<pixelherodev>
So an LLVM error then?
<andrewrk>
what version did you upgrade from? I don't think you switched any llvm versions
<pixelherodev>
I might have
<pixelherodev>
Currently on 9
kotrcka has left #zig [#zig]
<andrewrk>
we've been on 9 since 0.5.0
<pixelherodev>
That's... hmm... maybe a minor release breakage then?
<pixelherodev>
I rebuilt LLVM recently to add more targets I think
<andrewrk>
you upgraded from pre-0.5.0 zig?
<pixelherodev>
No
<andrewrk>
there hasn't even been a bug fix release to 9 yet
<pixelherodev>
... what the bleep
<andrewrk>
pixelherodev, what incorrect section is it ending up in?
<pixelherodev>
bss
<nomadgamma>
also pixelherodev you are amazing, the speed that and huuuge leaps in judgement that you made which somehow intuitively got you to the next step are astonishing, *highfive*
<andrewrk>
that's correct
<pixelherodev>
But it... wait what?
<andrewrk>
bss is for stuff that is initialized to 0
<andrewrk>
it's an optimization because it doesn't actually take up any space in the elf file
<andrewrk>
the elf file is just like, "alright and from here to here is all zeroes"
<pixelherodev>
Ah okay then
<pixelherodev>
Hmm
<andrewrk>
so you are supposed to memset all the bss to zero at init
<pixelherodev>
So maybe it's QEMU not initializing BSS?
<pixelherodev>
... and apparently in release modes a different variable is being optimized out so I can't test it :(
<pixelherodev>
andrewrk, if cast to bool it says false
<pixelherodev>
But after it loaded the value into %dl, %dl contained 0xAA
<nomadgamma>
iirc all the test instructions should be not-equal-to-zero, so zero is false and anything else is true
<andrewrk>
ok, if the memory is 0xaa, that likely means somehow that memory got set to undefined
<pixelherodev>
It `jne` to the true branch though
<pixelherodev>
and `jne` == `jnz`
<pixelherodev>
Right, so my question is, when does Zig mark stuff as 0xAA?
<pixelherodev>
Is it in the executable, or is it at startup? Wait, no, dumb question, has to be in the binary; otherwise it wouldn't happen
<nomadgamma>
and qemu doesn't initialise to 0xAA
<andrewrk>
pixelherodev, zig marks stuff as 0xaa when you set something to undefined
<pixelherodev>
But that's... okay I honestly have no clue when that's happening then.
<andrewrk>
`x = undefined;` or `@memset(foo, undefined, n);`
<pixelherodev>
Okay, then it shouldn't be 0xAA
<andrewrk>
is your stack memory running into your bss memory?
<andrewrk>
some kind of memory overlap situation?
<andrewrk>
@stack = internal unnamed_addr global [16384 x i8] undef, section ".bss", align 16
<pixelherodev>
I highly doubt it... especially given that this is a direct call from kmain to panic
<andrewrk>
does your stack subtract downards?
<pixelherodev>
I mean, that's how stacks work, so yeah?
<andrewrk>
panicked is right above it, it looks like a stack overflow might mangle the `panicked` global
<pixelherodev>
Sure, but... again, that shouldn't be possible
<andrewrk>
if you put `panicked` in data rather than bss and it fixes it, that might be you avoiding clobbering panicked with stack overflow
<andrewrk>
why shouldn't it be possible? zig hasn't solved stack overflow yet
<pixelherodev>
Because that's 16KB of stack.
<andrewrk>
you've only allocated 16K stack memory
<pixelherodev>
One function call does not use even a percent of that
<pixelherodev>
And since only stuff allocated *by kmain* should be in the stack...
<pixelherodev>
Even 1KB is probably more than adequate at the moment
<andrewrk>
test this hypothesis by adding a couple more globals before and after `stack`, initialized to zero. check their memory values at your panic weird behavior
<pixelherodev>
Just for argument's sakes, I'll increase stack size to 32MB
<andrewrk>
oh yeah or that
<andrewrk>
maybe also use gdb to inspect the stack pointer?
<pixelherodev>
...drats.
<pixelherodev>
I think you're right about the cause
<pixelherodev>
... yeah, at 32KB it's fine,
<pixelherodev>
Whelp
<andrewrk>
this is one of the things it's planned for zig to help with, but it's not done yet, obviously :)
<pixelherodev>
Yeah :)
<andrewrk>
you should probably set up some kind of guard page
<pixelherodev>
Yeah, but... well, I didn't bother implementing paging before the jam
<hspak>
I'm trying to read a file line-by-line. What am I doing wrong here? "const line = try io.readLineFrom(try fs.File.openRead("./input").inStream().stream, &buf);" fails with "error: expected type '*std.io.in_stream.InStream(std.os.ReadError)', found '*const std.io.in_stream.InStream(std.os.ReadError)'
<andrewrk>
pixelherodev, we have a stack prober, which I think will be on for you even in freestanding. so if you have a guard page it should be hit
<pixelherodev>
Ah, and that was the bug that was preventing everything from working
<pixelherodev>
I think it's the standard library stuff that's eating up stack, which is why migrating it caused the breakage
<andrewrk>
hspak, I think that function is intended to be used as terminal input, e.g. implementing a repl (although it doesn't yet have any bells & whistles)
<andrewrk>
hspak, for your use case I think you want the functions in std.io.InStream
<andrewrk>
such as readUntilDelimiterBuffer, readUntilDelimiterAlloc, readUntilDelimiterOrEof
<pixelherodev>
andrewrk, thanks a ton! While panic is till broken (yay.), everything else is good to go for the jam!
return0e_ has joined #zig
<pixelherodev>
So as long as nothing goes wrong, I'm good :)
<hspak>
andrewrk: Ah I see, thanks for the pointer!
<pixelherodev>
Anyway, that means I've got a good standard library patch to submit toni - I mean, tomorrow (it's 2AM here :P)
<andrewrk>
and I've got all tests passing in the sentinel-terminated-pointers branch
<pixelherodev>
Nice!
<pixelherodev>
Just need to clean this up and then begin a PR
<andrewrk>
my plan is to spend a couple weeks on PRs/bugs and not working on major changes after this merge
<pixelherodev>
Sounds nice :)
<pixelherodev>
Wait, if bss is default, why does the stack require linksection(".bss")?
<pixelherodev>
I copied that from your kernel I think :P
return0e has quit [Ping timeout: 265 seconds]
<andrewrk>
I don't think so
<pixelherodev>
No yep
<nomadgamma>
wait can you TL;DR that for me? was afk, you overflowed the stack into .bss?
<pixelherodev>
Rather, the updated stdlib required more stack, which overflowed into .bss
<pixelherodev>
AFAICT anyways
<nomadgamma>
ah ok
<andrewrk>
pixelherodev, I think that worked around an llvm issue that was solved in 7.0.0
<pixelherodev>
Ah
<nomadgamma>
surely, as an aside, you use GDT and LDT to set the stack growing down to something which is impossible to overflow
<pixelherodev>
As andrew mentioned, I can use paging to have it basically segfault, but that's not a right now thing
<pixelherodev>
Now, I clean this up and commit locally
<nomadgamma>
maybe give it its own set of pages way up there
<pixelherodev>
Then get back to work on the game itself :)
<pixelherodev>
I don't have paging yet ;)
<nomadgamma>
oh
<pixelherodev>
Doesn't matter where the page *is*, as long as right underneath it is marked read-only
<nomadgamma>
yup
<pixelherodev>
So maybe literally right over .rodata would be good
<pixelherodev>
That means not having to set aside an empty page
<andrewrk>
yeah and then zig's stack probing which is on by default should make sure it pokes the rodata page even if you have a large array in a function stack frame
<andrewrk>
(in debug mode)
<andrewrk>
there's a loooot of safety that release-safe doesn't have :)
<andrewrk>
oops, I mean release-fast
<pixelherodev>
Stack probing?
<andrewrk>
when you have a function that has enough stack memory used that it's larger than 1 page, zig calls some hand-optimized assembly which "pokes" each page in order, until all pages used by the function's stack frame have been poked
<andrewrk>
therefore 1 guard page is all you need, and it will be guaranteed to be hit even if you have a lot of stack variables
<pixelherodev>
Ah, so no risk of jumping over read-only memory and arriving at something writable
<pixelherodev>
That's awesome
<nomadgamma>
so... re-map the stack into some high-address using LDT?
<andrewrk>
of course, even better would be knowing the stack upper bound at compile time and allocating the correct amount of stack space
<nomadgamma>
hmm
<andrewrk>
...which is also planned
<pixelherodev>
That's not implemented yet though
<pixelherodev>
Yeah :)
<nomadgamma>
but LDT mapping isn't something you can do with portability
<nomadgamma>
i guess it only makes sense when you're doing bare-metal stuff
<andrewrk>
pixelherodev, out of curiosity, do you have any function pointer calls, or interrupts that use stack space?
<pixelherodev>
Uh... doubt it?
<pixelherodev>
Oh wait
<pixelherodev>
Actually, probably
<pixelherodev>
function pointers, yes
<pixelherodev>
That's how interrupts are implemented
<pixelherodev>
Other code registers a handler
<pixelherodev>
Then when the associated IRQ fires, the handler is called with relevant info
<andrewrk>
so to use this feature you'd have to annotate all your function pointer calls with the upper bound of how much stack space they might use. safety checks will panic if you ended up calling a function that needed more
<pixelherodev>
Or maybe have a way to require the function pointer to have a maximum upper bound less than X?
<andrewrk>
so at least you'd have your work checked, in debug builds. you'd get a panic message such as, "expected at most xyz bytes of stack, but function required xyz + 10"
<pixelherodev>
(for any arbitrary value of X)
<nomadgamma>
oooh, so you say 1024 is the limit... and there's a compiler feedback at a type level that 1020 + 8 = invalid passed along as a meta-parameter
<andrewrk>
I think these annotations will be worth it though. Consider how much pain you had to go through debugging that
<andrewrk>
people on hosted OS's will be less receptive to it. but zig code is supposed to work everywhere
<pixelherodev>
Still useful there though
<emekankurumeh[m]>
perhaps we could have zig infer that, and when it can't require an explicit bound?
<pixelherodev>
Just not to the same extend
<pixelherodev>
s/extend/extent
<nomadgamma>
I've never had to work out stack frame size outside of bare-metal... and even then if I have to specify it there's something really f'kd up that I've missed
<pixelherodev>
It can be used to reduce the stack size
<pixelherodev>
Relatively useless there though
<pixelherodev>
Nobody really cares if there's a minor waste
<nomadgamma>
what matters more is reducing the number of loads and stores from memory, e.g. C's __restrict is half-way there
<emekankurumeh[m]>
i really wish that `usingnamespace` applied to fields right now
<emekankurumeh[m]>
i'm porting COM definitions and having the time of my life
<nomadgamma>
you're porting COM+ definitions and it's fun?...
<pixelherodev>
Gah - input on kernel is partly borked :( Inconsistent behavior between the two builds = :(
<emekankurumeh[m]>
did i forget the /s?
* nomadgamma
is a veteran Photoshop COM object manipulator
<nomadgamma>
the /s is implicit
<pixelherodev>
Oh wait... I still have it emitting IR, so the build is wrong!
<pixelherodev>
The one good thing to come of this disaster is I know how to GDB it now :D
<nomadgamma>
that's good, being able to GDB stuff is 90% of the problem, the only other problem is the remaining 10% takes 90% of the time
<pixelherodev>
Argh, it seems to be another stdlib change :(
<pixelherodev>
std.fs.File.openWrite is hanging forever apparently.
<pixelherodev>
Or... not...
<pixelherodev>
Weirdly enough, the offset in GDB is correct now
<pixelherodev>
No need to manually adjust it :)
<nomadgamma>
*shrugs* at least GDB is working
<pixelherodev>
... yeah, but the timer's broken apparently :P
<pixelherodev>
Or at least, the frequency is wrong
<pixelherodev>
Probably messed up a calculation when migrating that one
Cucumbas has quit [Remote host closed the connection]
Cucumbas has joined #zig
Cucumbas has quit [Ping timeout: 276 seconds]
<pixelherodev>
@as(u8, '1') should give 49, correct?
<pixelherodev>
Never mind, more weirdness
<daurnimator>
andrewrk: can I do `if (x == undefined)` at comptime?
ur5us__ has joined #zig
<andrewrk>
I'm considering the ability to check for undefined (at both runtime and comptime), but the semantics would be this: `@isUndefined(value: var) bool` - has false negatives, but no false positives
<andrewrk>
the only valid use of it would be in the implementation of a safety feature
<daurnimator>
boooo. translate-c doesn't support duplicated definitions `struct foo {int a;}; struct foo {int a;};` => error: redefinition of 'foo'
<daurnimator>
andrewrk: yeah I was thinking about having an argument to a genetic: `initialise_with: var` => and I wanted to check if they passed `undefined`, and if so, optimize it into a @memset(, undefined,)
<daurnimator>
*generic
<pixelherodev>
Alrighty, everything works mostly. Held together by virtual duct tape, but that's good enough for now
<pixelherodev>
Patch incoming shortly
riba has joined #zig
traviss has joined #zig
<andrewrk>
daurnimator, you can't use the assignment operator?
<daurnimator>
andrewrk: huh?
reductum has quit [Quit: WeeChat 2.6]
hspak has quit [Ping timeout: 240 seconds]
hspak has joined #zig
<pixelherodev>
https://github.com/pixelherodev/indomitable src/zig/std is a to-be-cleaned patched stdlib; I'll clean it and do a PR in the (later) morning; it's 4 AM and I is tireds
<andrewrk>
daurnimator, for your initialize_with thing
<andrewrk>
I did notice that `undefined` being casted to the parameter directly did not have the desired effect. that's worth an open issue
<andrewrk>
oh, that's already an issue. globals don't get set with 0xaa or have client requests
<andrewrk>
solution would be a custom linker script that puts undef globals in a custom section, which startup code before main would set to undefined
ur5us__ has quit [Ping timeout: 245 seconds]
<andrewrk>
but zig is not always in control of the linker script
<andrewrk>
and there's a tradeoff - if you have a large undef array, it might be better to put it in bss (and be zero initialized) to keep binary size down
<andrewrk>
good night
Ichorio has joined #zig
<daurnimator>
zig: /usr/include/clang/AST/Type.h:659: const clang::ExtQualsTypeCommonBase* clang::QualType::getCommonPtr() const: Assertion `!isNull() && "Cannot retrieve a NULL type pointer"' failed.
<daurnimator>
not sure how I triggered that..
FireFox317 has quit [Ping timeout: 276 seconds]
<Sahnvour>
is there a way in zig code to know wether it's being built as an executable or library ? can't find this in builtin
wootehfoot has joined #zig
<daurnimator>
Sahnvour: I don't think so. and why are you asking?
<daurnimator>
Sahnvour: also did you know that you can have an executable that is *also* a library?
<daurnimator>
(ELF PIE is actually a library that the loader executes)
<bgiannan>
works thx
Ichorio has quit [Ping timeout: 245 seconds]
<Sahnvour>
daurnimator: I didn't know
<Sahnvour>
on darwin we unconditionally export a variable that, iiuc, should only be when building a executable (MH_EXECUTABLE)
<Sahnvour>
_mh_execute_header
syscall0 has joined #zig
lunamn has joined #zig
nikoala has quit [Read error: Connection reset by peer]
nrdmn has quit [Ping timeout: 268 seconds]
mmx870 has quit [Ping timeout: 268 seconds]
mmx870 has joined #zig
knebulae has quit [Read error: Connection reset by peer]
mahmudov has joined #zig
protty has joined #zig
Akuli has joined #zig
doublex_ is now known as doublex
protty has quit [Remote host closed the connection]
wootehfoot has quit [Read error: Connection reset by peer]
<pixelherodev>
"fn ohgodwhydidiactuallyimplementthis() void {" I'm clearly in a great mood :P
<mq32>
pixelherodev: what?
<pixelherodev>
I named a function ohgodwhydidiactuallyimplementthis.
<mq32>
neat
<mq32>
what does it do?
<pixelherodev>
It's an entire separate submission in the game :P
<pixelherodev>
s/submission/sub-mission
<mq32>
:D
<pixelherodev>
I was writing and then I jokingly was like "heh, this is a perfect plot hook NO WAIT STOP dangit this is MORE WORK"
<mikdusan>
Sahnvour: I wonder if we need something like `builtin.link_executable = true` to condition the export of _mh_execute_header
<mikdusan>
or an enum `builtin.link_mode`
demizer has joined #zig
<Sahnvour>
mikdusan: that's also my thought, although I'm not entirely sure this is the correct thing to do for _mh_execute_header, if someone knows better
demizer has quit [Quit: WeeChat 2.6]
demizer has joined #zig
demizer has quit [Quit: WeeChat 2.6]
<mikdusan>
if it helps, with clang, creating .o (and therefore .a), or .dylib does _NOT_ define _mh_execute_header; creating an executable _DOES_ define _mh_execute_header
Ichorio has joined #zig
demizer has joined #zig
demizer has quit [Client Quit]
demizer has joined #zig
dbandstra has joined #zig
<andrewrk>
mikdusan, does this mean _mh_execute_header should be part of compiler_rt, rather than exported in std.c ?
<mikdusan>
i can only answer with another question: is zig's compiler_rt something only ever used for executables or can it be manually linked for other link modes
<andrewrk>
it's a dependency that must be fulfilled eventually
<andrewrk>
you can use --bundle-compiler-rt to make zig include it with an object or library
<andrewrk>
one way to describe it would be "external symbols that all zig code assumes will be available to be linked against"
<Sahnvour>
scouting the web it really looks like it's only needed in executables (of type MH_EXECUTE, object files and libraries have another MH_ type)
<mikdusan>
hmm I'd have to consider if creating an { exe, dylib } with clang and using --bundle-compiler-rt
<andrewrk>
what's the problem here btw? even a zig static library wants to use that symbol, so that when it gets linked into an executable, it can look up itself in memory for debug info purposes
<mq32>
if you split up the u24 into u16 and u8 in any order, it works
<mikdusan>
andrewrk: fyi i've confirmed creating an exe with: clang + main.c + libbar.a(from zig code) + manual link to libcompiler_rt.a all works right now; ie: superfluous _mh_execute_header in libbar.a doesn't seem to be issue, this likely means if desired moving _mh_execute_header to compiler_rt.a shouldn't break linking
<andrewrk>
mikdusan, looking at it more, I think how it's currently done is necessary (and actually kinda clever) for how debug info works
<mikdusan>
👍
<andrewrk>
there's a comment above the export in std.c explaining it
fsateler has quit [Quit: ZNC 1.7.2+deb3 - https://znc.in]
protty has joined #zig
riba has quit [Ping timeout: 240 seconds]
reductum has joined #zig
ur5us__ has joined #zig
ur5us__ has quit [Remote host closed the connection]
ur5us__ has joined #zig
ur5us__ has quit [Read error: Connection reset by peer]
ur5us has joined #zig
Ichorio has quit [Ping timeout: 250 seconds]
mahmudov has quit [Remote host closed the connection]
belgin has joined #zig
Cucumbas has joined #zig
Akuli has quit [Quit: Leaving]
dingenskirchen has quit [Quit: dingenskirchen]
dingenskirchen has joined #zig
belgin has quit [Remote host closed the connection]
dingenskirchen has quit [Client Quit]
dingenskirchen has joined #zig
<Snektron>
If the self-hosted compiler is in a working condition, will the stage1 compiler be phased out (like with rust) or will it be maintained?
<andrewrk>
limiting the bootstrapping process to 3 steps is one of the features
<andrewrk>
however more advanced functionality could be phased out. for example, it wouldn't need release builds anymore
<andrewrk>
or the ability to build libc
<Snektron>
good
<andrewrk>
it also would stop being a hybrid of c++ and zig
<andrewrk>
that would become pointless. this hybrid thing is a stepping stone
<Snektron>
Which parts of the current compiler are hybrid?
<daurnimator>
andrewrk: :( my fifo PR already had that fix you just merged in it
<andrewrk>
everything in src/userland.cpp gets replaced by src-self-hosted/stage1.zig
<andrewrk>
daurnimator, your fifo PR is a higher tier of difficulty of merging
<andrewrk>
it shouldn't cause a conflict for you
<daurnimator>
andrewrk: I really don't think its a hard merge... hell we can take out the std.io changes, they were what I thought was trivial
<daurnimator>
I was under the impression that you were just not merging anything into std.fifo until you understood how it worked
<andrewrk>
I'll still consider the std.io changes, but yeah if you made it only fifo files then that would be an easy merge
<daurnimator>
andrewrk: done.
<daurnimator>
(and rebased on top of that other merge while I did it)
<daurnimator>
#3763 contains the pieces I removed.
<daurnimator>
Its mostly a de-duplication. + implementing a TODO
[rg] has joined #zig
[rg] has quit [Client Quit]
<daurnimator>
Snektron: however note that pieces are intended to be taking out of the stage1 compiler (as e.g. zig fmt already has): we only want the bare minimum of cpp code required to build the self-hosted compiler. Next on the chopping block is `zig translate-c`
<andrewrk>
+1
<andrewrk>
can't wait to delete 4,000 lines of C++
<andrewrk>
I think we might want to make a nicer API for building the zig AST though. it's a bit of a pain to append tokens to a buffer *and* build an AST
<andrewrk>
I think that makes it less appealing for contributors
<andrewrk>
on the other hand, it is powerful. it's great having the "warnings" as comments for example
<pixelherodev>
For self-hoosted or stage1?
<pixelherodev>
s/hoosted/hosted
<protty>
how does`suspend` work in the compiler? Does it do liveness analysis and save all the variables that it can observe that are needed?
<andrewrk>
protty, any function that can possibly suspend is generated with all of its local variables spilled into a struct, which is secretly passed as the first parameter
<andrewrk>
currently the implementation is conservative; more than necessary is going into this struct
<pixelherodev>
Isn't that going to be a noticeable performance hit for purely synchronous code though, since it's passing a bunch of stuff around that it really doesn't need to?
<andrewrk>
purely synchronous code doesn't have the possibility to suspend
<andrewrk>
so it won't be generated this way
<pixelherodev>
Ohhh, you mean functions that are ever called `async`?
<pixelherodev>
` any function that can possibly suspend` I think I misinterpreted that :P
<andrewrk>
even calling a function with `async` does not cause it to be generated this way. only if a function has a suspend point (or if you force it with `async fn` which is rarely necessary)
<pixelherodev>
Gotcha, that's neat
<demizer>
hi everyone. Learning zig and got an error I am not sure about. I "error: expected type 'std.fs.file.File', found 'std.os.windows.GetStdHandleError!std.fs.file.File'
<demizer>
i am copying the zasm argument handling code
<demizer>
i suspect it might be do to trying to run on version 0.5.0
<andrewrk>
demizer, the zasm code is based on zig master branch
<andrewrk>
between 0.5.0 and master, std.io.getStdOut no longer has the possibility to fail
<pixelherodev>
Yay!
<pixelherodev>
So for 0.5, it can throw an error which you need to catch
<pixelherodev>
Personally, I just used `catch unreachable` back when it was needed
<demizer>
ah ok, the docs in general are not clear about function return types, and the std lib docs are for master and not 0.5.0
<demizer>
i guess I should just use master
<daurnimator>
demizer: the `xyz!` prefix means its an error union
<demizer>
right, learned that last night after careful reading
<demizer>
thanks for the help gents!
<pixelherodev>
No problem!
cota has left #zig [#zig]
<Pistahh>
what zig pattern to use for some simple mapreduce like thingie? I want a main "thread" receiving some data, passing it to other "threads" which do some long-running calculations on them, passing the data back to the main thread. Once all is done the main thread precesses the results. By "thread" I mean whatever can run stuff on multiple cores, so not necessarily just OS threads
<Pistahh>
I looked into async/await/suspend/resume but I can't see how they could parallelise the calculations