<marler8997>
* Dependency specifications are not validated, leading to incomplete deployment.
<marler8997>
Here are all the problems that other package managers that nix solves:
<marler8997>
you ready for the list :)
<marler8997>
what issues hasn't it solved?
<pixelherodev>
marler8997: I disagree that nix solves them at all.
<marler8997>
for other distributions, they will still have issues, and yes, more work would need to be done to solve issues there as well
<marler8997>
there are techniques to solving issues with dynamic builds... For nix, they are all pretty much solved except this last one
<marler8997>
neither is "better"
<marler8997>
their are pros and cons to static/dynamic builds
<marler8997>
lol...ok try to explain again :)
<marler8997>
I should say, usually it's the "package manager" that tries to handle that
<marler8997>
Every distribution implements their own solution to solve so compatibility issues
<marler8997>
That's why I use Nix, to solve those issues
<marler8997>
are you talking about solving how to find so's as well?
<marler8997>
the issue was that we couldn't implement our own mechanism in our program to locate the loader, reloade aims to solve that, so you can use any tehnique you like to find the loader
<pixelherodev>
marler8997: ... here's the thing. libg... uh what's it called. one sec.
<marler8997>
yeah, I can make an ELF exe with no interpreter, and with so dependnecies, invoke it, and it will reload itself with any linker it wants to find
<pixelherodev>
marler8997: :O
<marler8997>
I got the "reloader" to work...yay!
2020-09-17
<marler8997>
yeah that was clever
<marler8997>
cool, I've been thinking about this problem for years...will be cool if we actually come up with something
<andrewrk>
marler8997, I'm excited about this, feel free to ping me with updates :)
<marler8997>
exe still doesn't have an interpreter
<andrewrk>
marler8997, if you do a proof of concept that does an opengl triangle window and provide a static linux binary that works on Ubuntu, Alpine, and NixOS, that would be big news
<marler8997>
basically, each exe functionally acts as its own dyanmic linker, of course, they aren't full blown linkers, they just find the real linker and call execve again
<marler8997>
which I think it does currently, so that would need to be fixed
<marler8997>
so long as our logic to find the linker doesn't use libc :)
<marler8997>
it's really simple once you think about it, just find the linker at runtime instead of link time...
<andrewrk>
marler8997, interesting! I like where this is going
<marler8997>
then reinvoke yourself with that linker
<marler8997>
andrewk, since your here, after thinking on it the solution I've thought of is just to move Zig's "find linker logic" into the runtime executable
<marler8997>
is_dynamic seems to only apply to libraries, not exes
<marler8997>
I wonder if we would need an option to explicitly disable the interpreter?
<marler8997>
if you come up with a fix, how would I get back to the current behavior? I tried removing the .interp section after the fact but that doesn't seem to work
<marler8997>
pixelherodev, coincidentally, that bug I created is actually helping me experiement with a feature where we don't use an ELF interpreter
2020-09-16
<marler8997>
lol
<marler8997>
I mean, thanks ifruend :)
<marler8997>
fengb: thanks that worked! I can now play the game :)
<ifreund>
marler8997: you pass them to addCSource() iirc
<marler8997>
I'll try that, off the top of your head do you know how to add LLVM options in build.zig? If not Ill figure it out
<marler8997>
but my plan is to port to Zig...which will be super fun :)
<marler8997>
but the C code is pretty portable, I expect most would be modified and cross-compiled
<marler8997>
SNES was a rewrite in assembly as I understand (code was release recently actually)
<marler8997>
is there a way to disable ubsan? to check if that' it?
<marler8997>
ubsan though..thanks for the tip
<marler8997>
I can now go through menus and start the game...still hunting down more
<marler8997>
10 UB so far :)
<marler8997>
I'm sure the cleaned up repos will have fixed these issues :)
<marler8997>
from id-software
<marler8997>
original repo
<marler8997>
not sure what LLVM options Zig is using to detect these instances of UB
<marler8997>
Sorry, "DOOM" source code (not Dune)
<marler8997>
Found around 10 instances of undefined behavior in the Dune source code by compiling it with Zig instead of GCC
2020-09-15
<marler8997>
would be a cool test case, of course, I'd still do the port by hand for educational purposes
<marler8997>
(compiling the doom source code, thinking about porting it to Zig for fun)
<marler8997>
I was thinking of using the generic target to see if that would make the final exe more lenient on unaligned access
<marler8997>
I'm running into issues with illegal instructino because of unaligned access
<marler8997>
ooo, gotta run, thanks for the explanations though
<marler8997>
off the top of my head, I would try to go up my process tree and find the first ELF that uses it
<marler8997>
yeah...how does it do that?
<marler8997>
I did have to think about it for a bit before I realized that's what it was
<marler8997>
well I'm glad you confirmed what I thought it was
<marler8997>
can support.
<marler8997>
> I understand that a fix for this issue might prove difficult. There's no way to set an absolute path to a loader that will work on all distributions. The only fix I can think of is to never use an ELF interpreter. We we need dynamic libraries, we could compile the loader into the final executable...or maybe a better solution would be to include startup code that looks for the system loader that works on as many distributions as we
<marler8997>
I think the linker is setting the interpreter to /lib64/ld-linux-x86-64.so.2, because that is the most common one it can set...
<marler8997>
As I'm filling out this issue, I have a feeling that there is no solution to this
<marler8997>
oh ok, well I'll creating a reproducible issue
<leeward>
Sorry marler8997, I was responding to the Zen thing. I don't know anything about NixOS.
<marler8997>
yeah anyone heard of this?
<marler8997>
hey, when I build on my 64-bit linux NixOS linux machine, the elf interpreter is set to the correct one in the nix store, but when I set an x86-64-linux-gnu target, I get an absolute path interpreter that doesn't exist on NixOS?
2020-08-23
<marler8997_>
before my time
<marler8997_>
wow that's going waaay back
<daurnimator>
marler8997_: I'd rather support NT 3
<marler8997_>
I think we'll be in a much better position to support win9x once we have the stage2 compiler
<marler8997_>
not sure which compilers work on win9x...
<marler8997_>
anyway, see how far you can get compiling LLVM, let us know how far you get
<marler8997_>
interesting...I personally would never elect to use an OS that I don't have source to...of course I have to use Windows all the time, but it's not by choice
<marler8997_>
do you have access to all the source for it?
<marler8997_>
Then...maybe see if you can get cmake to work? :)
<marler8997_>
I would say start by seeing a counselor so you can try to cope with the deep dark hole of self loathing that you're digging yourself into
2020-08-22
<marler8997_>
yeah starting a language with this premise is a much needed thing in my opinion
<marler8997_>
It is infinite from the processor's perspective, because if you ever run out, you get murdered by the OOM...so it's infinite as far as you know :)
<marler8997_>
but then again, I wouldn't say Zig had to add 90% more complexity to handle 100% of errors
<marler8997_>
when it comes to proper error handling, Zig definitely aims for the 100% approach
<marler8997_>
an interesting detail, of course, it depends on how important that "effect" is
<marler8997>
to catch up, is the plan to create a "C Code Generating Backend", then compile the self-hosted zig compiler to C using that backend, then just maintain that going forward?
<marler8997>
the term "liveness analysis" reminded me of Walters "liveness" feature for D, where it implements some Rust-like lifetime features
<pixelherodev>
marler8997: yeah, basically
<marler8997>
oh I see, just an implementation detail for the codegen
<marler8997>
is this going to affect semantics from the programmer level at all, or is it just for the optimizer?
<pixelherodev>
marler8997: no
<marler8997>
liveness analysis...is there any github issue that discusses what this is?
<marler8997>
I'm half tempted to get one of those MNT laptops, but I think I'm going to have to hold out until there's an open source laptop/PC with a RISC-v CPU
<marler8997>
goodness Android is quite a project, we put it on our products and it was my job to get that working...
<marler8997>
leeward, if I remember right that's correct, not android
<marler8997>
I've spent so much time looking for a laptop with mech keyboard, I've given up
<marler8997>
ifreund, I just read that !!!!
<marler8997>
oh yeah that looks cool too
<marler8997>
been seeing articles about them recently
<leeward>
marler8997: Do you mean like raspberry pi open source hardware or Risc-V?
<marler8997>
There's some open source hardware that's been coming out recently, that's exciting
<marler8997>
saw there was a discussion last night about open/closed source. My two cents, the more experienced I've gotten, then more I've realzied how important it is to have full control of your system, to be able to build/reproduce/debug and make changes anywhere in the system. If I have a closed source component, it becomes a rigid black box that I can't fix myself, and now I have to rely on others to support it for me.
<marler8997>
like what's supposed to go in each one
<marler8997>
std.c
<marler8997>
there's also std.os.windows, std.os.windows.bits
<pixelherodev>
marler8997: sorta, yeah
<marler8997>
isn't std.os supposed to be posix?
<marler8997>
a description of the organization at the file/directory structure level
<marler8997>
would take alot of guesswork and investigation work away
<marler8997>
while I'm at it, one thing I would love is a good description of the standard library organization
<marler8997>
yeah I'll wait for his thoughts
<marler8997>
I'm not even sure if Zig wants all the windows declarations in the standard library
<marler8997>
I'm willing to put in work to start working on the standard library, but it will be alot of work and I want to make sure this is the direction zig wants before I start
<marler8997>
So I'm just constantly fixing up my own version, it's getting old having to go back and forth between my app and windows declarations
<marler8997>
I'd say about 70% of the time the function or declaration I need is not there
<marler8997>
*std.os.windows
<marler8997>
std.ow.windows
<pixelherodev>
marler8997: wait, we don't have windows support already?
<marler8997>
if that's what zig wants
<marler8997>
I'd be willing to go through the standard library and start filling it out
<marler8997>
I've been spending too much time maintaining my own version of the windows APIs, going back and forth between developing my apps and maintaining the windows declarations
2020-07-01
<marler8997>
I asked this yesterday but maybe someone knows the answer today, anyone know what a "backwards branch" is when talking about @setEvalBranchQuota?
<marler8997>
what is a "backward branch" when talking about @setEvalBranchQuota?
2020-06-29
<marler8997>
that should handle overflow as well
<marler8997>
timeDiff = newTime -% oldTime
<marler8997>
time checking is a case for using overflow
<marler8997>
mmap is wrapped by PageAllocator in same file
<marler8997>
I don't think it's a big deal either way, it's a standard allocator, meant to serve the general case. If an app wants something more specific they can always roll their own allocator
<marler8997>
yeah...lol
<marler8997>
Yes, ArenaAllocator
<blinghound>
marler8997 is this for an allocator?
<marler8997>
andrewrk, adding superlinear growth to the new version made it faster, but I'm not sure it was as fast as the original. The old version also added more length to the initial allocations, alignment, plus added 150% of what was requested
<andrewrk>
marler8997, oh I didn't realize you reverted the superlinear growth
<marler8997>
I'll make PR to revert to the old version, but just keep the change in to use the full capacity of allocations
<marler8997>
modifying mine to do that as well makes it faster, but I think the old version is still faster than mine with those changes
<marler8997>
old version is also adding the size of the previous buffer to the size of the next buffer, so it makes much larger allocations later on
<marler8997>
old version said, bug me at least 150% of what was requested from me
<marler8997>
new version says, give me at least as much memory as the client requested from me
<marler8997>
I think the problem may be that the old version was allocating larger arenas than the new version
<marler8997>
ok I'm getting the slow speeds even when I move the BufNode back to the start of the buffer
<marler8997>
Going to take my branch and keep all the changes but move the metadata back to the start of the buffer and see what I get
<marler8997>
I'm trying some experiments, my best guess is that storing the metadata at the end of the allocation could be messing up cache predictability
<marler8997>
I was using the wrong branch
<marler8997>
andrewrk, I think I'm seeing the same results now
<andrewrk>
marler8997, interesting, I wonder why we are seeing different results
<marler8997>
ok with page allocator, I think I am getting a consistent slowdown, from .004s to .005s
<marler8997>
I'll try that, one moment though, I think I was testing the wrong branch
<pixelherodev>
marler8997: What about page_alloc?
<marler8997>
Using C allocator as the underyling allocator, compiling with --release-fast and -lc, new ArenaAllocator is consistently 33% faster
<marler8997>
andrewrk, I'm running the benchmark manually and it's consistencly running faster with the latest version of ArenaAllocator
<marler8997>
oh you left it in the comment, I'll take a look
<marler8997>
andrewrk, where is the data on the ArenaAllocator perf regression?
2020-06-28
<andrewrk>
marler8997, it looks like the changes to ArenaAllocator are a pretty significant perf regression
<marler8997>
I'm curious why u29 was picked for the maximum alignment size...
<marler8997>
wasm page size is 64K so it was unlikely that someone would hit this
<marler8997>
fengb, gotcha, well hopefully with that PR it's aligning properly now
<fengb>
marler8997: Ah yeah, at the time I think andrewrk mentioned it’s okay for the page allocator to not align. Although I probably should have added a check and return OOM
<marler8997>
though, it could have been broken before...not sure
<marler8997>
fixing...
<marler8997>
yup
<marler8997>
dang, alignment broke on wasm page allocator
2020-06-27
<andrewrk>
marler8997, ooh it looks like ArenaAllocator could be improved to take advantage of extra capacity. this will make a difference when backing it with PageAllocator
<marler8997>
yes it's ready
<andrewrk>
marler8997, is #5064 ready for merge?
<marler8997>
Zig is free to reorder structs that aren't tagged with packed/extern. So if one field requires more alignment than another, it can reorder them to keep @sizeOf and @alignOf to a minimum
<marler8997>
yeah
<marler8997>
makes me think it would be cool to organize a Zig-only game jam
<marler8997>
andrewrk, looks like valgrind shims malloc_usable_size to return the original size requested to malloc, interesting
2020-06-26
<marler8997>
yeah it was easy to implement malloc_usable_size with the new interface
<andrewrk>
marler8997, oh nice, you integrated malloc_usable_size
<marler8997>
that's a really good idea
<marler8997>
it records execution so you can play it back? wow sounds cool
<andrewrk>
marler8997, brilliant
<marler8997>
andrewrk, I've finished working on new allocator interface PR. It's ready to merge. The last 3 pushes were final touches and all 3 fully passed all CI tests. Please have a look when you find the time.
<marler8997>
going to have to port my wsl-register code to Zig...
<marler8997>
leeward good question, I don't know that one
<marler8997>
WSL1 is just an implementation of the linux syscall interface
<marler8997>
I doubt it would have the same syscall interface as linux
<marler8997>
I'm not too familair with OpenBSD...isn't it a bsd kernel?
<leeward>
marler8997: You're making me sad.
<marler8997>
yeah you could totally run WINE on WSL, I've worked on project that did similar things, even went 3 levels deep in places
<marler8997>
wsl2 is basically a VM, so you can load your custom kernel instead of emulating a kernel syscal interface
<marler8997>
meaning any OS coded for linux should work, not just gnu
<leeward>
marler8997: That's exactly what Wine does in reverse. Don't call it an emulator though.
<marler8997>
leeward true, but it's "emulating" the linux kernel syscall interface
<marler8997>
ZBS could update the timestamp of any file that it copies out of the cache
<marler8997>
this is an issue for Make, because it assume that if an input file has an older timestamp than an output file, then the output file must be up-to-date
<marler8997>
I bet it is hashing the source and seeing it was already built, then it just copies it from the cache which has the old timestamp
<marler8997>
mq32, well I've pretty much finished, now just need andrewrk to review
<marler8997>
m32...lol
<mq32>
marler8997: it was a mistake to watch the allocator issue…
<antaoiseach>
ifreund: marler8997: ah yes, that's right
<marler8997>
if it's a library, usually you just take an allocator from the client/application so you don't need to choose one
<marler8997>
gotcha
<marler8997>
what backends is zir starting out with? x86_64? i386?
<marler8997>
why's that?
<marler8997>
I see ArrayList as an ordered collection of items, I would expect AutoHashMap(T, void) to function be an unordered set of unique items
<marler8997>
so drone must just have some fast and slow machines
<marler8997>
as I think about it, I think usize would have to be 16 bits
<marler8997>
I wonder how Zig's types would look on the 6502...
<marler8997>
also consider the 6502 8-bit processor, with 16K of address space
<marler8997>
actually modern processors boot in that mode
<marler8997>
leeward, yes that's the case for the old 16 bit 8086 processors
<marler8997>
I like it
<marler8997>
I see now
<marler8997>
oh!
<daurnimator>
marler8997: you can't have 2^64 * 2 byte elements on processor that only has 2^64 bytes of memory
<daurnimator>
marler8997: how?
<marler8997>
yes
<marler8997>
all set to zeros right?
<marler8997>
you could have max(usize) of zeros of 2-byte elements
<marler8997>
I don't understand why not?
<daurnimator>
marler8997: max(usize) is the max number of bytes on a machine. if you have a 2 byte element, you can't have max(usize) of them.
<marler8997>
wait I don't think I get it anymore...what if you have an infinite number of duplicate values? Maybe I didn't get it
<marler8997>
you're limiting the type by the "cardinality" of the element type set
<marler8997>
daurnimator, it took me a minute to get what your PR was doing...but now I see, that's pretty clever
<marler8997>
leeward, gotcha, something to explore
<leeward>
marler8997: Right. I'm just suggesting that maybe that common type should be included in the standard library. Let people choose between the bloaty looking code and runtime overhead without writing as much code.
<marler8997>
man my typing is bad today
<marler8997>
so it allows the application to control the trandoff between 'generic bloat' and runtime overhad
<marler8997>
the nice part about using generic readers in the standard lib is that you can always wrap them with a common type for runtime dispatch
2020-06-25
<marler8997>
ok that workds
<marler8997>
you might want to be able to call setenv if you are calling an external library that uses getenv for configuration (this is pretty common in libraries)
<tgschultz>
marler8997: i'm not disputing the utility of the new allocator API, only the degree to which it is responsible for any improvement in speed in CI. Seems I was wrong anyway.
<marler8997>
leeward, yeah they did it just to tease me
<marler8997>
too good to be true
<marler8997>
ok speed boost is gone, I don't know what happened, both dron and azure showed speed boosts but now the latest run doesn't have them