2020-11-25

<marler8997> I think LD_PRELOAD is more widely supported
<marler8997> oh...looks like the ubuntu loader doesn't support the "--preload" parameter
<marler8997> I have an ubuntu machine with a monitor attached I can try as well
<marler8997> could be virtualbox doesn't have a GPU
<marler8997> vkEnumeratePhysicaDevices fails
<marler8997> it also fails...which is good I guess :)
<marler8997> trying...
<andrewrk> marler8997, hmm, does this example work? https://github.com/SaschaWillems/Vulkan#Basics this is where I cribbed the code from
<marler8997> andrewrk, scratch my last response (was running with xtrace from glibc). actual X11 xtrace doesn't show much, I see setup/CreateWindows and a couple InternAtom comands
<marler8997> andrewrk, I ran with "xtrace" which said Authentication Rejected: None of the authentication protocols specified are supported and host-based authentication failed
<andrewrk> marler8997, hmm I think it will be interesting to see what failed. perhaps something to do with X11?
<marler8997> got to "opening window"
<marler8997> VulkanInitializationFailed :(
<marler8997> oh headless still getting Vulan Extension not present error, trying on virtualbox
<marler8997> how far are you getting now?
<marler8997> good
<marler8997> checking
<andrewrk> marler8997, the tls initialization problem is solved in the most recent commit I pushed to zig master branch (assuming it passes CI)
<marler8997> so long as you are vigilant with self evaluation and keep yourself to high quality standards you will get better
<marler8997> that's the number of ".." you want
<marler8997> you'll get there eventually if you keep at it
<KIMI> thanks marler8997
<marler8997> yeah, writing a function for you
<marler8997> to get absolute path, you can either allocate, or use MAX_PATH with a global/stack buffer
<marler8997> what is allocating?
<marler8997> to workaround, you could get the absolute pathname, and strip off the last directory
<marler8997> oh..that's why
<marler8997> what OS are you on?
<marler8997> *works for me
<marler8997> "../test.txt" works for now
<marler8997> let me try that
<marler8997> how are you running it?
<marler8997> you must have path or cwd incorrect
<marler8997> FileNotFound
<marler8997> that means the file you are trying to open and read doesn't exist
<marler8997> ok give me a minute
<marler8997> do you just need zig code that reads a file line by line?
<marler8997> also, not sure why you are using fopen
<marler8997> because the code you sent me is missing code I need to compile it
<marler8997> how about just send all your code?
<marler8997> missing definition for fontdata
<marler8997> if you share the zig code you have now, should be easy to help
<marler8997> so if I understand correctly, we need some way to know whether or not we were loaded by ld, so we can know whether or not to initialize tls?
<andrewrk> marler8997, sorry to keep pinging you, I'm using you as a rubber duck and to give little progress updates
<andrewrk> marler8997, I eliminated one thing as a possible cause of failure

2020-11-23

<justin_smith> marler8997: if your goal isn't to write code, this blog post seems to describe the config needed to make it work https://www.jungledisk.com/blog/2017/07/03/live-streaming-mpeg-dash-with-raspberry-pi-3/
<justin_smith> marler8997: it's definitely going to be relatively turn-key on rpi, I wouldn't be surprised if you couldn't just add a copy/paste config to nginx to serve v4l data off some qeury path
<marler8997> I'm thinking of putting together my own baby monitor, first thought I had was to use a raspberry pi, but open to suggestions if anyone has any
<marler8997> but then again, an explicit function might also work in most cases
<marler8997> one case I see for them is initialzing multi-threaded synchronization data structrues (like mutexes) in a library
<justin_smith> marler8997: this is extreme, but it's the kind of thing constructors are often used for, the "invariants" that a lib author has in mind don't always match what I need, especially if I want to make my code testable in isolation
<marler8997> sounds like a problem with the library misusing constructors to me
<marler8997> lol...why the hostility there?
<marler8997> execute constructors would be another one I think
<marler8997> like set envp
<marler8997> oh right, whatever they do in crt0.o
<andrewrk> marler8997, I suspect the next problem we are running into is that we didn't run some necessary start code. for example musl or glibc does some stuff before calling main() to set up threading
<karchnu> marler8997: I saw that, and it's awesome, but I want something usable and stable. Pluto is very much R&D. But I read the logs from time to time. :p
<marler8997> karchnu, SamTebbs is also writing a kernel in Zig: https://github.com/ZystemOS/pluto
<marler8997> night
<marler8997> but hopefully there is an easier way ?
<marler8997> that's a linker script I put together to wrap the libefivar so file a while back
<marler8997> I got a few more minutes in me :)
<marler8997> yeah, definitely a challenge
<marler8997> The only mechanism I've used is a complex linker script
<marler8997> oh geeze
<marler8997> good question
<marler8997> frame #14, libPath address is 0x2d1?
<marler8997> but no good error message :(
<marler8997> right that's what I'm thinking
<marler8997> I bet you're getting farther, I'm running on my headless machine, might need to go to virtualbox which has a display
<marler8997> just fyi, with master gpa is getting me to VulkanExtensionNotPresent
<marler8997> when I switch back to GPA, I don't get that far
<marler8997> I'm using PageAllocator
<marler8997> got to VulkanExtensionNotPresent
<marler8997> woohoo
<marler8997> I'm a bit fuzzy on this, but alignment might be a per function thing as well
<marler8997> we aren't using glibc
<marler8997> trying to remember
<marler8997> there are compiler options for that
<marler8997> freagin stack alignment...
<marler8997> shared knowledge
<marler8997> aha!
<andrewrk> marler8997, I disassembled it: => 0x000000000028a76d <+445>:vmovdqa XMMWORD PTR [rbp-0xa0],xmm0
<marler8997> I don't know what you're trying to do, write a font library? but I would start writing some functions and designing your API
<marler8997> but I don't think this is the struct you want in the first place, so I wouldn't spend any time on that
<marler8997> KIMI, I think you could get it to work if you initialized the whole thing to 0, but I don't think there's a way to just initialize one value, again, that's like trying to initialize 1 byte of a u32 value
<marler8997> I made it public, and added logging, it works and I print a message before returning, but I never execute any code after calling it
<marler8997> I believe you can also reproduce the segfault by calling std.heap.PageAllocator.alloc directly
<marler8997> pastbin has extra lines for context on where I put it in the code
<marler8997> second part?
<andrewrk> marler8997, what's the connection with that second part?
<marler8997> that's like trying to initialize 1 byte of a u32 value
<marler8997> KIMI, I don't think you can initialize an array with just one element
<marler8997> andrewrk, I'm at a stopping point for the night on this, I got debug stack traces to work (see previous comment), but, something weird is going on that seems to be causing a segfault when returning from certain functions, seemingly to do with allocation, see this pastebin for a small code snippet to reproduce the error: https://pastebin.com/ruWHYGag
<marler8997> you said you need to be able to check if it is empty...if you don't have any code, you do you know you'll need to be able to do that?
<marler8997> do you have code to share?
<marler8997> need more context
<marler8997> why do you want to check if it is empty?
<marler8997> you can still have safe code that uses undefined
<marler8997> it means, I'll initialize this later
<marler8997> undefined doesn't necessarily mean you aren't being safe
<marler8997> what was the "yes" to?
<marler8997> so you want undefined
<marler8997> which sounds like what you want
<marler8997> undefined means, don't initialize it yet, I'll initialize it later
<marler8997> why do you not want it to be undefined?
<marler8997> why do you want to set any of it to 0?
<marler8997> why do you want to set the first one to 0?
<marler8997> I'm not sure that's what you want to do though
<marler8997> I believe th C syntax you sent zeros out the entire array
<marler8997> why are you wasting cycles zeroing out that entire array?
<marler8997> How are you using this struct? Do you have code to share?
<marler8997> yeah that's fine
<marler8997> I have a feeling your API is making this overly complex...probably should look at that
<marler8997> his memory is not going to be "consistent" whether or not he uses a slice, that's what he signed up for by avoiding the heap
<marler8997> yes, what you're doing is a bit odd/complex
<marler8997> yeah pfg_ example is better
<marler8997> Something like this: pub fn font_file(self: @This) [*:0]u8 { return self.font_path_bufer; } (TODO: insert appropriate cast)
<marler8997> font_path_buffer: [256]u8,
<marler8997> then KIMI, [256:0] is not what you're wanting
<marler8997> oh thanks pfg
<marler8997> I'm actually not sure what [256:0]u8 means...does that mean it's a 257 byte array with a 0 at the end? or that it's guaranteed to have a 0 somewhere inside it?
<marler8997> he doesn't want heap
<marler8997> third thing, why do you need a default initializer?
<marler8997> second thing, why does it need to be sentinel?
<marler8997> first thing I see, there is a MAX_FILE constant (rather than 256)
<marler8997> but what you're doing looks very odd
<marler8997> I don't have enough context
<marler8997> I'm not sure what you're trying to do so I'm not sure how to help
<marler8997> C doesn't support struct types with default initializers at all
<marler8997> font_file: [256]u8,
<marler8997> not sure what you're trying to do, but can you leave out the initialized value?
<marler8997> which I'm guessing you don't want
<marler8997> if you did it in one statement, you would have to initialize the entire array to 0's
<marler8997> that's probably what you are wanting
<marler8997> font_file[0] = 0;
<marler8997> var font_file : [256]u8 = undefined;
<KIMI_7996__88> marler8997 ty
<marler8997> var font_file = [_]u8 {0};
<marler8997> gdb is just confused because it detects that you are running "ld", so it stops trying to use the symbols from "static-window"
<marler8997> (gdb) file ./zig-cache-bin/static-window
<marler8997> andrewrk, by the way, you can get gdb to load the symbols by reloading the binary file
<marler8997> after PoC, we can insert logic to try to detect which one to use if we need to
<marler8997> probably not
<marler8997> daurnimator, here's the PR with your suggestion: https://github.com/andrewrk/zig-window/pull/4
<marler8997> daurnimator, ah yes that's better
<daurnimator> marler8997: `strings /lib/ld-linux-x86-64.so.2 | grep libc.so` its in there somewhere :)
<marler8997> ok, there's something wrong with GeneralPurposeAllocator, it's segfaulting in its alloc function
<marler8997> we would only need to know the actual file paths to .so files if we can't leverage the .so file lookup semantics of linux loaders
<marler8997> there's too many variations to know for sure what we need or don't need at this point
<marler8997> got to go for a bit, I"ll see if I can narrow down the next seg fault later
<marler8997> oh yeah, more flexible
<marler8997> yeah that works as well
<marler8997> yeah might work
<andrewrk> marler8997, came back to share this idea really quick: we could maybe avoid the --as-needed thing by setting LD_PRELOAD before execve
<marler8997> failing in allocator.reallocAtLeast in ArrayList.ensureCapacity...still narrowing down
<marler8997> but maybe if we add that option we can re-enable pie...still not sure whether pie is needed
<marler8997> we disabled pie
<marler8997> lame
<marler8997> add printf before/after that line
<marler8997> try extensions.append("VK_KHR_surface");
<marler8997> printf debugging
<marler8997> it's a weird place
<marler8997> it's in arraylist
<marler8997> yeah me neither
<marler8997> these are all the "needed" when I linked "dl": libdummy.so.0 libc.so.6 libm.so.6 libpthread.so.0 libdl.so.2 librt.so.1 ld-linux-x86-64.so.2 libutil.so.1
<marler8997> but it looks like we can add "needed" after the fact
<marler8997> I haven't found a way to delete interp section
<marler8997> ah
<marler8997> simple
<marler8997> ah
<marler8997> setgfult in extension.append("VK_KHR_surface")....hmmm
<marler8997> yeah
<marler8997> removing libdummy.so works as well
<marler8997> got further, but still a segfault
<marler8997> coincidentally, created by the nixos guys
<marler8997> love patchelf
<marler8997> *--add-needed (not add-needed)
<marler8997> dlopen gettting populated after this: patchelf add-needed libdl.so.2 zig-cache/bin/static-window
<marler8997> nice
<marler8997> that did it
<marler8997> maybe patchelf --add-needed
<marler8997> is there a way to add "dl" to the dynamic section without adding an ELF interpreter?
<marler8997> but now static-window has an ELF interpreter set, so when the kernel loads it, it doesn't jump to static-window, it goes directly to the loader
<marler8997> is I add "linkSystemLibrary("dl")"...now dlopen get populated
<marler8997> now the ELF interpreter is set
<marler8997> but
<marler8997> ok, so that fixes the issue
<marler8997> yeah that could be
<marler8997> why wouldn't ld be populating dlopen...
<andrewrk> marler8997, just tested. void * __attribute__((weak)) dlopen(const char *filename, int flags); gives the same NULL value
<marler8997> my guess is it's the @extern implementation, but I can see if I can make that work in C to confirm
<marler8997> the dynamic section worked in my example
<marler8997> dlopen might work
<marler8997> these are the 2 problem I worked around by 1. hardcoding the loader (not calling NativeTargetInfo.detect) and 2. using the environment variable
<marler8997> even more interesting....NativeTargetInfo works when we run statically, but not on the second invocation
<marler8997> you're right
<marler8997> no its not
<marler8997> no it's now
<marler8997> oh wait
<marler8997> it's working to detect whether we are loaded by ld
<marler8997> weak extern not working?
<andrewrk> marler8997, interesting, so the weak extern is not working
<marler8997> when I fix that...we get a setgfault somewhere in NativeTargetInfo (probably the segfault I was seeing)
<marler8997> now loader can load/run the exe
<marler8997> and comment out static_window_exe.pie = true
<marler8997> so...first, trick the loader by linking to a dummy so file
<marler8997> you can replace main with an empty body, compile it and try to run it with the loader and it will segfault
<marler8997> pie is not working
<marler8997> it's pie
<marler8997> but we still get a segfault
<marler8997> ok, when I link to an so...ld no longer says it's a static executable
<marler8997> trick the loader
<marler8997> what if we tried linking to a dummy so file?
<marler8997> ok we can rule out the ".interp" section, my working example doesn't have one
<marler8997> at least, something that it *thinks* is static
<marler8997> I have a working example that does this, and now a non-working example. I can look at the differences between the 2 but my guess is that ld won't run a static ELF binary
<marler8997> thinking out loud :)
<marler8997> but you're not using it to load any .so files from your dynamic section
<marler8997> so...the only reason you're actually calling the loader is to get access to the dlopen/dlsym functions
<marler8997> so...ld is saying that static-window is statically linked
<marler8997> still segfault
<marler8997> can simplify, just call loader directly
<marler8997> yeah this is weird, ld is segfaulting
<marler8997> wow NativeTargetInfo is working
<marler8997> so @extern returns whether a symbol has been populated?
<marler8997> it uses normal files :)
<marler8997> fast NVME is also crucial for fast builds like this
<marler8997> AMD Ryzen 9 3900X 12-Core Processor
<marler8997> and...we're built
<marler8997> that's what I was looking for :)
<marler8997> I get 34 errors about using uninitialized variables in zig source code
<marler8997> r u sure?
<marler8997> oh it should?
<marler8997> nix-shell is adding the old version of gcc (9.2)...why
<marler8997> hlolli__, first time running that nix-shell command, only took a few seconds
<marler8997> and use llvm nixos package
<marler8997> I think fastest path to solution is to update nixos
<marler8997> hmmm...getting more uninitialized errors, might need to use a different compiler
<hlolli__> I don't think it takes 3 seconds to enter a llvm shell in nix if it's the first time, the nice thing about nix is the nix store. I bet if you run nix-shell the second time marler8997, it will be 3 seconds or less
<marler8997> fixed that one, just initialized Value, now getting an aliasing error in Attr.h:262
<marler8997> I think it's because I'm on gcc 9.2..too old
<marler8997> oh wow, actually got a compile error: zig_clang_cc1as_main.cpp:521
<marler8997> error...yeah that didn't sound right :)
<marler8997> oh no!
<marler8997> zig 7 seconds
<marler8997> clang 3m41s
<marler8997> have you timed yours?
<marler8997> gotta go fast!
<marler8997> lld is 27 seconds
<marler8997> need to upgrade to new ryzen 9 5950x
<marler8997> 6 minutes to build llvm...I thought it was faster :(
<marler8997> I'm on 20.03...don't think it has 11
<andrewrk> marler8997, nix-shell -p llvmPackages_11.clang-unwrapped -p llvm_11 -p lld_11
<marler8997> I suppose I can try that at the same time
<marler8997> ah
<marler8997> meh, I like to be able to change the llvm source if I need to
<marler8997> oh?
<marler8997> it probably does, but I'm not sure I know how to use it correctly
<andrewrk> marler8997, nixos doesn't have it already built?
<marler8997> compiling llvm...
<marler8997> ok will build...let's see how long it takes
<marler8997> does seem to exist
<marler8997> Your version isn't available for download on the README
<marler8997> actually with my new PC, I can build LLVM/clang in like 5 minutes if I remember right
<marler8997> lame
<marler8997> it's not custom?
<marler8997> I can download your zig compiler?
<marler8997> oh wait
<marler8997> going to have to get llvm 11 built first...haven't been zig since release
<marler8997> haven't built it yet...
<marler8997> I wanted to start with the simplest example possible, and be able to control how the .so file is built
<marler8997> so test that I can call functions from shared libraries
<andrewrk> marler8997, what's up with libfoo.so in your reloader project? what purpose does it serve?
<andrewrk> marler8997, master branch - I also put the sha1 in the readme
<marler8997> do you have a zig branch you're working from?
<marler8997> I can call into a custom built so file, but didn't get libc working
<marler8997> but...someone doesn't work when I call into libc, I never figured out what that was
<marler8997> my example that doesn't use libc, I don't think it needs to be pie
<marler8997> I used zig cc, what's zig cc's default?
<marler8997> I'll have to look to see if I used pie or not..checking
<marler8997> looking
<andrewrk> marler8997, I published the code: https://github.com/andrewrk/zig-window

2020-11-22

<ifreund> marler8997: here's one https://github.com/oasislinux/oasis
<andrewrk> marler8997, https://i.imgur.com/VUlXsNM.png
<marler8997> did you find the X extension to map a framebuffer?
<marler8997> andrewrk that's cool! I'll have to look into the Vulkan loader myself to see what it's doing
<pixelherodev> marler8997: I don't know of any

2020-11-21

<andrewrk> marler8997, I have vulkan-loader compiled statically and it works
<marler8997> pixelherodev, it was you talking about statically compiled distros right? what was the distros you mentioned?
<pixelherodev> marler8997: assuming that's the smallest possible
<marler8997> the nixos loader is 184K, so I suppose it's reasonable to assume the cost to BYOL (bring your own loader) is 184K of extra disk space per executable