<marler8997>
for most linux systems, finding the loader will be good enough, but if you want to support systems without a loader, I suppose you have to include a loader as well
<marler8997>
on linux it's implemented in userspace, which is the distro, so there are N versions
<marler8997>
windows is just easy because all versions work the same, they all support CreateProcess and the os will handle loading
<marler8997>
or at least...how complicated that loader would be, I'm sure it can be done
<marler8997>
the only potential issue I see is that I wonder if you could make a loader that would work on all distributions...
<marler8997>
all loaders are opt in
<marler8997>
nixos has loaders
<marler8997>
I think it was pixelherodev who was talking to me about systems that are completely statically compiled
<marler8997>
I suppose that is possible
<marler8997>
no loader *mind blown*
<marler8997>
wait, are you talking about elf loader or this vulkan loader thing?
<marler8997>
like, including loader into a static executable?
<marler8997>
but no
<marler8997>
just doom :)
<marler8997>
andrewrk, I actually run headless, everything goes through a remote X server
<andrewrk>
marler8997, we could detect the various window managers. For example on my system xfdesktop is running, and it maps in libGL.so.1
<marler8997>
could also do this: inline for ("\x00\x01\x02...") |x| {
<marler8997>
yeah
<marler8997>
lol
<marler8997>
you gonna make me type all 9 digits :)
<marler8997>
{0, 1, 2, 3, ...}
<marler8997>
var i = x;
<marler8997>
inline for([_]u32{0, 1, 2}) |x| {
<earnestly>
marler8997: Brain Kernighan (the K in awk) did a talk on it which can be found on youtube if you want
<marler8997>
wait, are END and BEGIN patterns? So it's just awk 'pattern block pattern block pattern block..."?
<marler8997>
Oh, so the END is just labeling the last block of code as "run me at the end"
<marler8997>
wow awk is crazy
<marler8997>
actually, now that I've looked at it again I see...seen is a hash table, END means that block of code runs for all lines then once it's done, you loop over your hash table and print the entries
<marler8997>
yeah I don't understand awk enough to understand your awk example :)
<marler8997>
oh!
<earnestly>
marler8997: Instead of using the seen[] trick
<marler8997>
"stick with sort -u instead of using inherent properties of hash tables"/
<marler8997>
huh?
<earnestly>
marler8997: It's the same
<marler8997>
earnestly, you're saying "sort -u" is different than "sort | uniq"?
<earnestly>
marler8997: (In the above case I'd probably stick with sort -u instead of using the inherent properties of hash tables. It's clearer and less surprising)
<marler8997>
which are?
<marler8997>
do you find the time that you spent learning awk has helped your workflows alot? do you use it alot?
<marler8997>
dang you know your awk
<earnestly>
marler8997: In which case replace $NF with $6
<marler8997>
there's other edge cases to though
<marler8997>
its' an optional column, in which case you don't want to get the previous column
<marler8997>
based on maps format, I belive $6 is what we want
<earnestly>
marler8997: Also in the maps format the last column can be addressed via $NF instead of $6, if the intention is actually to extract the last column. If it doesn't matter that it's sorted you can do: sudo awk '/\.so/ {seen[$NF]++} END {for(lib in seen) print lib}' /proc/*/maps
<marler8997>
not sure if there are any others
<marler8997>
anyway, the point here is that's the only kernel mechanism I can think of to query what ".so" files are running in the system, /proc/*/maps
<marler8997>
You can really get so much more done if you know your command-line tools well, always good to learn more
<earnestly>
marler8997: awk is a pattern {action} language
<earnestly>
marler8997: sort has -u, which obviates the need for uniq
<marler8997>
I didn't know awk supported filtering like that
<marler8997>
how do you limit it to the 6th column and filter by ".so" though?
<marler8997>
does awk take filenames?
<marler8997>
I'm not seeing how to make the full, command, can you show me?
<earnestly>
marler8997: the same, just with the pattern matching done in awk itself. Although also bear in mind that awk * or grep * is exposed to ARG_MAX limits
<marler8997>
what's the full command without grep?
<marler8997>
looking at /proc/*/maps could get you there
<marler8997>
andrewrk, that's an interesting idea
<andrewrk>
marler8997, ok so here's where I'm at: Xlib.so: cannot open shared object file: No such file or directory
<marler8997>
custom lookups could be an optional addon if we don't think dlopen's logic doesn't handle all the cases, but I would first start with all the cases dlopen already supports
<marler8997>
if you just pass it a name that is
<marler8997>
dlopen will already check the ld.so.cache file, LD_LIBRARY_PATH, the elf runtime paths, letc
<marler8997>
first, I think you would call dlopen with all possible names
<marler8997>
you call dlopen first, if that fails, then you fall back to custom lookups
<marler8997>
which one are we on?
<marler8997>
wait, finding Xlib.so and the loader are 2 different problems
<andrewrk>
marler8997, are you around to brainstorm some stuff?
<marler8997>
yup, got a PoC working to make sure it would work
<andrewrk>
I think it will! marler8997 did a PoC for using execve with a runtime-detected dynamic linker path and it worked with C code, so the next steps for zig are: (1) PIEs and then (2) detect wayland/x11/other and dlopen the appropriate .so
<marler8997>
what happened to the c_char c_uchar types?
<marler8997>
1,4k tickets, what impression did that give you?
2020-11-20
<marler8997>
just send LemonMan an email with a check :)
<marler8997>
oh is that right?
<andrewrk>
excellent point marler8997
<marler8997>
andrewrk, another idea, instead of using dynamic section or dlopen, you could set LD_PRELOAD with the library list before you call execve on the loader
<marler8997>
with that we could probably remove @cDefine?
<marler8997>
jesus
<marler8997>
just not with NativeTargetInfo
<marler8997>
the zig example works if you comment out "-lc"
<marler8997>
oh wait andrewrk, NativeTargetInfo doesn't work if you comment out "-lc" in build.zig
<marler8997>
are we just going to punt and use dlopen/dlsym for that?
<marler8997>
that's like an "optional" so
<marler8997>
probably was just a bug
<marler8997>
hey it's working now
<marler8997>
let me try it right now
<marler8997>
the reason I think it wasn't working was because I thought it was probably using libc somewhere down the line
<marler8997>
nixos
<marler8997>
did I leave a comment about that or something?
<marler8997>
failing to detect my linker?
<marler8997>
how do you know all the functions you need?
<marler8997>
oh yeah wait a second'
<marler8997>
if we work on the C version, that's less incentive for people to use Zig right? :)
<marler8997>
yean must implement extern weak symbols
<andrewrk>
marler8997, I think I could already PoC this in C pretty easily but I obviously want to show it off in zig instead :D
<marler8997>
very cool
<marler8997>
just check if dlopen is NULL
<marler8997>
wait, wouldn't dlopen be all you need?
<marler8997>
oh nice
<marler8997>
I mean: zig_window_get_fns
<marler8997>
is "static-window" in an so file, or in the exe?
<marler8997>
and if those are the steps, how do you know what all your libraries/functions are?
<marler8997>
then detect populated extern weak symbol, then run dlopen/dlsym?
<marler8997>
then find loader and run execve
<marler8997>
detect NULL extern weak symbol
<marler8997>
run statically
<marler8997>
what are the steps?
<marler8997>
but hold on
<marler8997>
it doesn't use libc though
<marler8997>
zig version had some issues
<marler8997>
oh, the C version doesn't use libc
<marler8997>
I'm not using -lc :)
<marler8997>
ok, so you are basically calling dlopen and then dlsym for all functions?
<marler8997>
you check if they are null or something?
<marler8997>
how do you use extern weak symbols to know if you're running statically or dynamically?
<marler8997>
andrewrk, gotcha, I didn't anticipate the so filenames being different, so yes, you can't rely on dynamic section
<marler8997>
the names of the so files or something?
<marler8997>
"different at runtime"?
<marler8997>
(yeah, redirects may not be necessary, not sure yet though)
<marler8997>
andrewrk, cool what did you find?
<andrewrk>
btw marler8997 switching topics, I iterated on your reloader code a bit and worked out a better solution than the environment variable thing
<marler8997>
and potentially putting the filetree information in another file besides build.zig (which I think is fine)
<marler8997>
so I think you're proposal is really just mine but removing the redirect feature
<marler8997>
if we took the proposal and remove .zig-redirects, users would still be able to modify build.zig (or deps.txt or whatever) to modify the filetree URLS to loca file paths and that would still get tracked by git
<marler8997>
then developers would just modify them directoy in zig-cache?
<marler8997>
so, you're saying we could download a copy of each dependency, probably in zig-cache
<marler8997>
not familiar with npm link
<marler8997>
there was also ideas of having global redirects
<marler8997>
this might be orthogonal to having zig-redirects outside git
<marler8997>
right
<marler8997>
so assume A is the main
<marler8997>
one project is the main project
<marler8997>
project A B C depend on project D
<marler8997>
so would you have to change that file for all repositories?
<andrewrk>
marler8997, I think I can solve the problem of working on dependencies problem in a simpler way than this redirect system
<marler8997>
link?
<marler8997>
I haven't
<earnestly>
marler8997: Have you read gogolinux's paper on package management?
<marler8997>
earnestly, lower down in the document you can assign your trees names
<earnestly>
marler8997: You may wonder to consider namespacing your trees by the version
<marler8997>
it's not an organized proposal, it's more my thought process as I work through the details
<marler8997>
lol
<marler8997>
my idea though is to separate zig libraries/packages from file acquisition
<marler8997>
amounts to the same thing
<marler8997>
that idea applies to my Filetrees example would mean declaring all your filetrees in this declarative file
<andrewrk>
marler8997, one idea that I had about this is that there would exist a strictly declarative (not .zig code) list of dependencies. You would still be able to use build.zig logic to conditionally depend on something, but you would have to declare the *possible dependency* in the declarative file
<andrewrk>
marler8997, yeah I think 95% of the implementation will be in zig code with only a tiny modification needed to stage1 code
<marler8997>
Basically, being able to reference other build.zig files, and implementing file acquisition is all we need I think
<marler8997>
I think I've come up with a very small set of features that accomplish what we need
<marler8997>
I'm working through package manager use cases and what is the minimum amount of features we need for package management
<marler8997>
you didn't happen to look at my filetrees draft did you?
<marler8997>
andrewrk cool, that's where my exploration took me
<andrewrk>
marler8997, the build importing another build thing is something I plan to revisit with during the release cycle where we work on the package manager
<marler8997>
cool
<Xavi92>
marler8997: thanks a lot!!
<marler8997>
zig build-exe --help, there's like 9 --versose-* options you can use, one of them will have what you ned
<marler8997>
--verbose-cc or something I think
<marler8997>
yeah
<marler8997>
this enables using files from anything, including nonzig projects
<marler8997>
yes, with the Filetree's proposal
<marler8997>
so with my proposal, zig-wayland would be a file tree, and the project could access any files from it, including something like protogen.zig
<marler8997>
protogen.zig
<marler8997>
I would say that you could put your Build Object definitions in something other than zig-wayland's build.zig file
<marler8997>
in that case
<marler8997>
got it
<marler8997>
and zig-wayland has implemented BuildStep objects to generate these files correct?
<marler8997>
ok, so the problem you're solving is...somebody needs to generate code for these protocol files
<marler8997>
they look like they are part of the project, not optional things
<marler8997>
these don't look like "build options" to me...are these protocols optional for a project?
<marler8997>
so...for me to understand a bit more...what does zig-wayland do we these protocol files?
<marler8997>
I'm understanding a bit more...I don't see how it would be possible for a project's protocol xml files to be in zig-wayland build.zig file... (in response to your statement " I think it makes most sense for this configuration to happen from the projects build.zig not zig-wayland's build.zig")
<marler8997>
reading...
<marler8997>
but I really want to understand, tackling use case is the best way to test an idea
<marler8997>
I'm trying to understand your use case but I'm having trouble filling details
<marler8997>
I'm sure I'm missing something
<marler8997>
-Dxmlfiles=a.xml,b.xml,c.xml?
<marler8997>
user configuration of packages?
<marler8997>
the JSON would just be the exact data you would find in the builder object
<marler8997>
this means every build.zig file still only gets compiled once, but you can reference the data from any other build.zig file
<marler8997>
so, zig compiles the librarie's build.zig build, serializses the final data to JSON, then the applications build.zig file can access all it's data from the JSON file
<marler8997>
i.e. you have a library in one build.zig file you want to use in another build.zig file
<marler8997>
so the problem is, how do you reference object in other build.zig files correct?
<marler8997>
you could, but zig is much nicer for humans to work with
<marler8997>
My idea right now is to implement serializing the builder object to JSON
<marler8997>
the builder object has everything we need
<marler8997>
now you have 2 ways of adding a package to your local builder object
<marler8997>
right
<marler8997>
I'm trying to approach it from bottom up, what do we absolutely need to get this working
<marler8997>
you can redirect by URL, hash or name, all 3 have valid use cases
<marler8997>
you can also make it project specific, or global
<marler8997__>
there's a proposal that woudl allow us to implement concepts inside the function signature
<marler8997__>
(still watching vid so maybe I'm just no caught up)
<marler8997__>
what's wrong with enable_if?
<marler8997__>
I'm on his first slide with code and feel like throwing my hands in the air
<marler8997__>
I bought a book on modern C++ a few years ago because it was so different
<marler8997__>
lol
<marler8997__>
insane good or insane bad?
<marler8997__>
I mean, I guess I'm ok with closed source as long as it's not closed to me :) But I emailed them about contributing and they just said they weren't hiring...so boo for closed source
<marler8997__>
lol...closed source model like jai? bleh
<marler8997__>
oh cool
<marler8997__>
no problem, what brought this up all the sudden?
<marler8997__>
QUOTE: " think by default it would be reasonable to change the default behavior based on whether we are compiling for native or cross targets. Find the system's dynamic linker at "link time" if we compile natively, and find it at runtime if we are cross compiling."
<marler8997__>
andrewrk, yes, in fact that's exactly what I say in my previous comment before the one I linked you
<andrewrk>
marler8997__, thanks for helping me out on stream! btw what do you think about making this the default when the OS is linux and the target is non-native? there would be the component that changes what the linker does, and then there would be a comptime bool that start code would look at before calling main()
<marler8997__>
usingnamespace is what you want I think
2020-11-13
<marler8997__>
ah, well, zigup is still there for you, you may want to have it around anyway if you are doing zig development because you'll want to be able to have multiple compilers installed and the ability to switch between them
<marler8997__>
0.7.0 was just released, but 0.7.1 is a patch release fixing a few known issues with 0.7.0
<marler8997__>
not sure if you want to wait for 0.7.1 though..not sure how long it will take, maybe a week or so?
<marler8997__>
yeah, it's time for all the package managers to update anyway