ChanServ changed the topic of #zig to: zig programming language | ziglang.org | be excellent to each other | channel logs: https://irclog.whitequark.org/zig/
<dimenus>
i hate to keep referencing jblow, but he has this concept where he can just run arbitrary functions in the compiler
<dimenus>
and that basically becomes his build system
<andrewrk>
we have compile time function execution
<dimenus>
you have a similar concept going on with zig but it's not quite arbitrary
<andrewrk>
yeah. you can't do things with side effects
<andrewrk>
that opens a whole can of worms
<dimenus>
and i'm imagining a scenario where i want to exactly that
<andrewrk>
such as: running code that corrupts the compiler's RAM
<dimenus>
like stick a precomputed datastructure that was compiled and run into my program when its compiling the rest of the program
<andrewrk>
can you explain your use case?
<andrewrk>
you can do that with the existing infrastructure
<andrewrk>
precomputing data structures
<dimenus>
you could bake a hash table into your program itself
<dimenus>
hmm interesting
<dimenus>
do you have an example of that in the docs?
<dimenus>
i like the case for comptime, a much better printf is a great example
<dimenus>
i'm envisioning the systme where the compiler is basically just pasrsing files its passed in looking for a directive run which might start the build system to pull in the rest of the program
<dimenus>
you're kind of doing that with build_runner and the hardcoded @build
<andrewrk>
it might be different when we self host
<andrewrk>
if the compiler is part of the standard library
<andrewrk>
or maybe we'll keep it like this
<dimenus>
it would also be nice to just be able to invoke the parsing functionality like a library
<dimenus>
and get back a datastructure you understand
<dimenus>
which makes more sense if we're self-hosted
<andrewrk>
actually I don't think we can do that if we self host, because we'll still have C++ code for interfacing with LLVM
<dimenus>
LLVM has a C interface i guess
<dimenus>
so if it's robust enough, you should be able to call llvm from zig then?
<andrewrk>
we'll already be doing that in the compiler
<andrewrk>
but we don't want to have to link the build executable against libc
<andrewrk>
unless we want to expose the compiler as a .DLL
<andrewrk>
but at that point, might as well keep it as a child process like we're already doing
<andrewrk>
what does interpreting the build script (jai) accomplish vs compiling it (zig)
<dimenus>
he also in some video I can't remember talked about how his idea of generics is not duck typing
<dimenus>
but i don't remember why
<dimenus>
or in what video
<dimenus>
=|
<andrewrk>
oh yeah this was a really impressive demo
<andrewrk>
I have plans to support all of these use cases
<dimenus>
it's a very different approach though, like at this point I think there's at least 3 backends
<dimenus>
bytecode / straight x64 output / llvm
<andrewrk>
yeah he got compilation speed very fast
<dimenus>
it's similar to a unity build, you dump everything into one big .o file instead of having separate compilation units
<dimenus>
since serializing out to disk and loading is sometimes the slowest part of a build
<andrewrk>
we do the same thing
<dimenus>
well not serializing, but you konw what i mean
<andrewrk>
but I have plans to automatically break compilation up into multiple cacheable .o files to speed up subsequent builds
<andrewrk>
(in debug builds)
<dimenus>
i know i shouldn't care, but one of my big turn offs from rust was iteration speed
<andrewrk>
why shouldn't you care? it's important
<dimenus>
even with a simple progrma, the compiler took sometimes mujltiple seconds to build a program
<andrewrk>
yeah that's one of my high prio issues for 0.2.0 is proof of concept fast builds
<dimenus>
calling the compiler directly doesn't feel slow
<dimenus>
building from build.zig does i think
<andrewrk>
building from build.zig is slow but we can speed it way up
<andrewrk>
it's slow for very solvable reasons
<andrewrk>
for example, we build builtin.o and compiler-rt.o *twice* for every zig build. once for zig build itself then once for the underlying artifact
<andrewrk>
when we could be building these .o files 0 times, if you've ever compiled anything on the computer before
<dimenus>
in a release do you plan on distributing the code of the std lib or compiling it?
<andrewrk>
distributing the code
<andrewrk>
and the compiler automatically creates a cache
<andrewrk>
so the first time you build anything for a given target, you have to build these 2 .o files
cenomla has quit [Quit: cenomla]
<andrewrk>
I'm also going to follow jblow's lead of making the compiler multithreaded in the self hosted rewrite
<dimenus>
he has a job system but doesn't go into too much detail on it
<dimenus>
i think i've listened to the entire series at least once maybe twice
<andrewrk>
it's a classic pipelining problem
<andrewrk>
it models well to a job system
<andrewrk>
dimenus, what would you work on if zig was more mature?
<andrewrk>
you said a game right?
<dimenus>
i honestly don't know
<dimenus>
i'm not a very good idea man
<dimenus>
i'm much better at coming up with solutions to problems
<andrewrk>
it can be fun to choose a project to work towards
<andrewrk>
and then think of it as a way to discover ways that zig doesn't handle the use case very well, and use that to drive what to work on to improve
<dimenus>
i've written a basic asteroids clone but it was terrible
<dimenus>
i tried to replicate the unity 3d asteroids example without using the engine at all
<dimenus>
so it was pure c & opengl
<dimenus>
i'd eventually like to write a networked game written in vulkan
<andrewrk>
jfo, my coworker showed me a use case for that which convinced me to implement it
<andrewrk>
haven't done it yet
<jfo>
oooooh what was the use case
<andrewrk>
type-safe abstractions over epoll
<jfo>
I love the idea of gaining higher order abstractions at comptime
<andrewrk>
epoll takes a u64 userdata, and then when you get an event from epoll you're given the u64 userdata
<jfo>
I mean, I know you do too :)
<andrewrk>
and we want the API to be more like fn registerEvent(context: var, callback: fn(@typeOf(context)))
<andrewrk>
and with a fn returning a fn, there's a way to achieve this
<jfo>
I think I was reading an issue about this but I definitely like the idea of making it possible to return fns in the same way structs can be returned anonymously.
<jfo>
also I'm almost done with set 1 of cryptopals.
<andrewrk>
yeah. for the first iteration of the feature, it will only support capturing compile-time known values
<andrewrk>
that's cool, did you get past the base64 one?
<jfo>
yeah that was only a component of a larger one
<jfo>
the 6th challenge is extremely error prone and fiddly by design
<jfo>
plus I'm learning the cryptography in conjunction with zig
<jfo>
I've got all the pieces working but the gears aren't lined up yet. won't be long
<jfo>
I'll have a write up for you at some point in addition to my half baked emails
<jfo>
will the `fn -> fn` thing involve addressing closures?
<dimenus>
i started cryptopals awhile ago
<dimenus>
i should do that again
<dimenus>
honestly if i want time for all these projects i should just not be married
<andrewrk>
jfo, I don't understand your question
<andrewrk>
did you see this comment? for the first iteration of the feature, it will only support capturing compile-time known values
_dev_zero has quit [Remote host closed the connection]
_dev_zero has joined #zig
arBmind1 has quit [Quit: Leaving.]
arBmind has joined #zig
arBmind1 has joined #zig
arBmind has quit [Ping timeout: 248 seconds]
_dev_zero has quit [Remote host closed the connection]
_dev_zero has joined #zig
tiehuis has quit [Quit: WeeChat 1.9.1]
_dev_zero has quit [Remote host closed the connection]
_dev_zero has joined #zig
_dev_zero has quit [Remote host closed the connection]
_dev_zero has joined #zig
_dev_zero has quit [Remote host closed the connection]
_dev_zero has joined #zig
_dev_zero has quit [Remote host closed the connection]
_dev_zero has joined #zig
dimenus has quit [Quit: Leaving]
<GitHub54>
[zig] dimenus opened pull request #616: Added DLL loading capability in windows to the std lib. Ensure that t… (master...master) https://git.io/vFMxG
dimenus has joined #zig
arBmind1 has quit [Quit: Leaving.]
_whitelogger has joined #zig
jfo has joined #zig
jfo has quit [Ping timeout: 240 seconds]
<dimenus>
andrewrk: re: your feedback on the PR
<dimenus>
loadlibrary isn't needed at all
<andrewrk>
have you tried it with a .dll that isn't already loaded by zig.exe?
<dimenus>
hmm, yeah that's a good point. but should the stl really be referencing dlls that are not guaranteed to exist?
<dimenus>
kernel32 and user32 will always be part of your process
<dimenus>
when building the compiler, no?
<dimenus>
whether you build in mingw or msvc, kernel32 will always be part of the parent process
<andrewrk>
the DLLs that the compiler depends on are independent from the DLLs you are trying to compile zig code against
<dimenus>
unless.....you're building a cross compiler on linux for windows
<dimenus>
i guess we could if !GetModuleHandle then LoadLibrary
<andrewrk>
that's still not sound
<dimenus>
It's not best practice to try and load system dlls in to your process though
<andrewrk>
the versions of DLLs that zig.exe uses might be different from the versions of DLLs you're trying to compile against
<dimenus>
windows has no strong versioning scheme for dlls though
<andrewrk>
we're going to do the tests in a child process, which at least will have to link against kernel32
<andrewrk>
we can have an exception for kernel32, and use loadlibrary for everything else in that child process
<dimenus>
although, we have the windows sdk path
<andrewrk>
yes, and we want to use the DLLs from that SDK path, not from whatever zig.exe is linked against
<dimenus>
hmmm, this is a hard problem
<dimenus>
the sdk does not distribute system dlls
<dimenus>
only link libraries for those dlls
<andrewrk>
the goal is to find out if a symbol exists, right?
<andrewrk>
sounds like the link libraries can tell us if a symbol exists
<dimenus>
but we're building our own with zig
<dimenus>
rather than relying on the one distributed by the sdk
<dimenus>
maybe building our own kernel32 isn't the right way to go
<dimenus>
nothing in kernel32 depends on libc
<andrewrk>
we're not building our own kernel32
<dimenus>
we are building our own link lib
<dimenus>
that's what the def file is for
<andrewrk>
yeah, but that just says what functions we're trying to call
<dimenus>
that's what an import library is
<andrewrk>
when you cross compile, you don't have any SDK or DLLs
<dimenus>
it creates an import lib that is unverified
<andrewrk>
yes
<andrewrk>
so what we want is optional verification of DLL function calls
<andrewrk>
if zig is (1) building natively and (2) knows the full path of a compatible DLL that we are going to call, then we can enable the verification
<dimenus>
ok, well if you don't have an sdk than you can't cross compile a program that links libc
<dimenus>
ok, so in os.cpp we check if we're compiling natively and if so we use the import lib in the sdk path
<dimenus>
rather than the one provided by zig
<andrewrk>
we also support compiling natively on windows with no sdk installed
<dimenus>
.. as long as you don't link libc
<andrewrk>
correct
<dimenus>
got it, I can make those changes
<andrewrk>
exciting
<andrewrk>
I agree with you, this is a hard problem
<andrewrk>
and it is important to get the details correct on this one
<dimenus>
i initially thought the optional verification felt like a hack
<dimenus>
but since our goal is to get off libc completely
<dimenus>
it isn't really
<andrewrk>
yes exactly
<andrewrk>
I do think it should be DLL based instead of .lib/SDK based
<andrewrk>
maybe we make exceptions for kernel32/user32/system libs
<andrewrk>
but it should also work for e.g. GLFW.dll
<dimenus>
where as here we're defining what the exports are
<dimenus>
but if you're using glfw.dll then the exports are defined by glfw or whoever built glfw
<dimenus>
sorry externs
<andrewrk>
this is why .h files are usually shipped with .dlls
<andrewrk>
yes, this is the fundamental problem that .h files solve
<andrewrk>
and when we declare externs without using an .h file, we are relying on ABI stability
<dimenus>
yeah i think loading extra dlls in the compiler to preverfiy that all the functions exist is overkill
<andrewrk>
dimenus, here's another brainstorming idea: in debug mode, we could add code before main() that does GetModuleHandle and looks for all the functions that are used with GetProcAddress, and panics with a nice error message if not found
<andrewrk>
instead of segfault
<andrewrk>
I think this is a good idea
<andrewrk>
obviously in ReleaseFast mode this safety check is omitted
<dimenus>
getProcAddress isn't a problem in the user's code, it's functions that should be resolved at link-time
<dimenus>
i may be able to screw around with dumpbin to figure out more info
<dimenus>
do you want to just close/delete that PR now and I'll think a bit harder on this?
<dimenus>
I can submit a separate one for the stl additions / fixed test
<GitHub93>
[zig] dimenus closed pull request #616: Added DLL loading capability in windows to the std lib. Ensure that t… (master...master) https://git.io/vFMxG
<dimenus>
andrewrk, as for the video I posted yesterday, he's using the interpreted build file to process messages pumped back from the compiler which can then be used to inject code into the AST
<dimenus>
he used it to demo basically adding a code profiler without having to decorate your original source code at all
jfo has joined #zig
jfo has quit [Ping timeout: 240 seconds]
jfo has joined #zig
arBmind has joined #zig
<andrewrk>
dimenus, I consider that to be an anti-feature
<andrewrk>
zig has no hidden control flow, and injecting into the AST adds a lot of magic hidden control flow
<andrewrk>
here's the deal: when you link against a DLL, that is run-time linking
<andrewrk>
you could replace the DLL with a different DLL that has different symbols
<andrewrk>
the safety feature of checking all the functions that we want to call with GetProcAddress will make sure that the DLL that ends up being available, at runtime, at least has the symbols we are looking for
<dimenus>
so, MS considers 'linking with a dll' to be load time linking
<dimenus>
runtime linking is calling LoadLibrary / GetProcAddress
<dimenus>
i'm still surprised that windows didn't catch the fact that there was no valid symbol in kernel32.dll for our fake proc
<dimenus>
it treats whatever is created in the import lib as the gold standard
<dimenus>
wait, maybe we should just include checking the STL in tests somehow
<dimenus>
because if it's not in the STL, it's not our responsibility to check someone else's dll
<dimenus>
andrewrk, I get your point about being an anti-pattern and it would be hypocritical to do that while simultaneously ranting about templates / dsetructors etc
<dimenus>
but if your compiler knows enough about the code its inserting, perhaps the debugger could still simulate those AST inserts
<dimenus>
it knows the file/line number of every proc, which is why he can click in his profiler and it jumps to the particular spot in emacs
<dimenus>
so by extension it should be possible to have a debugger that can patch the extra procs into the debug symbols
<andrewrk>
on linux we already have such a thing, it's perf
<andrewrk>
it works for any executable, you don't need special compiler abilities
<dimenus>
btw, i'm not advocating that we emulate jai as he's focusing on a specific problem domain
<dimenus>
just discussing ideas
<andrewrk>
that makes sense
<dimenus>
there's also a lot of hidden parameter passing (context, temporary storage etc)
<dimenus>
which hurts understandability
<dimenus>
where as in zig self is explicit and to my knowledge there is no hidden data being passed
<andrewrk>
that is correct
<andrewrk>
the only hidden control flow or data is debug safety, which is removed in ReleaseFast mode