<marler8997>
yeah I was thinking the same, I just thought Number would handle it all for you
<marler8997>
so the json parse will be able to parse any thing that's valid JSON
<marler8997>
in std.json.Value?
<marler8997>
so, replace Float and Integer with a Number that handles everything?
<marler8997>
yeah can't think of any use cases
<marler8997>
that's true
<marler8997>
what do you think about making the number type configurable?
<marler8997>
gotcha
<marler8997>
when would it output a string? It's it's bigger than u64 max?
<marler8997>
I thought it supported arbitrary precision, looking
<marler8997>
andrewrk, looking for direction, should I pursue a change in std.json to support u64 integers?
2021-01-23
<marler8997>
yeah I been lurking
<andrewrk>
marler8997, idk if you noticed but spexguy is live right now working on the win32 stuff
<marler8997>
ah..interesting
<marler8997>
ifruend, I guess I'll have to change my stomping grounds then
<marler8997>
irc doesn't seem to be a talkative these days, is there alot more activity on discord?
<marler8997>
I'm working on the Zig win32 binding generation, and want to make sure my build instructions include all the dependencies that are actually required
<marler8997>
ok whose got a windows box that wants to test they can build something for me?
<marler8997>
I see NtCreateFile
<andrewrk>
marler8997, does it have ntdll stuff in it?
<marler8997>
I'm working on the zig projection now :)
<daurnimator>
marler8997: yep. there's been discussion of it on both discord and here
<marler8997>
anyone else see that microsoft is deploying metadata about the win32 api?
2021-01-15
<marler8997>
and I don't understand how glibc doesn't have proper error handling...this could have taken 10 minutes if glibc would have just logged a proper error message when it found the unexpected data
<marler8997>
off topic, I just finished 4 days of triaging a locale issue. The problem ended up being that I was using glibc 2.25 but the yocto tool to create the archive was newer, which added some new entries to the LC_TIME locale...whoever decided not to version the locale-archive binary has caused the world sooo much pain
2021-01-14
<ifreund>
marler8997, mikdusan: you need to pass -fno-LLVM to use the self-hosted backend
<marler8997>
would be nice to update the docs to indicate it's not supported
<marler8997>
yeah I got the same error, I'll let the reddit user know
<marler8997>
thanks for the info
<ifreund>
marler8997: this functionality was temorarily removed shortly before 0.7.0 in #6250. It's recently been reimplemented though in this PR: https://github.com/ziglang/zig/pull/7111
<marler8997>
I tried the example in the docs that says a ".h" file will be generated, but it doesn't appear to work, no header file to be found? Is it missing some command-line option/build config?
<marler8997>
someone one reddit is trying to compile a Zig library and call it from C, the docs say Zig will generate a C header file? Is that true?
2021-01-13
<marler8997>
interesting
<daurnimator>
marler8997: yes
<marler8997>
so on Mac you bascially have to link libc?
<marler8997>
I actually didn't know it was still in draft...still getting use to dark mode
<marler8997>
good to know
<marler8997>
not passing CI yet
<marler8997>
rebased my "require array format specifiers" PR...and looks like more bugs have been introduced that are missing the "{s}" specifier :)
<marler8997>
wait a second "This repository has been archived by the owner. It is now read-only."
<marler8997>
browsing the Way Cooler code that stopped using Rust...the C code looks so clean
<marler8997>
having fun re-reading the linked articles
<marler8997>
CORRECTION: D and C++ have throw/catch exceptions
<marler8997>
D has throw/catch exceptions
<marler8997>
CORRECTIOn: All 3 languages have operator overloading, so the + operator might call a function
<marler8997>
Rust has operator overloading, so the + operator might call a function
<marler8997>
yeah, another sentence in the same paragraph has as similar issue
<marler8997>
it mentions that Rust has operator overloading, but so does C++ and D
<marler8997>
saw "why Zig over C++/D/Rust" was moved
2021-01-12
<marler8997>
yeah, I was wondering why you would put email fields in xattrs instead of just inside the email file itself...with the filesystem query feature, now I see why you would put them in xattrs
<marler8997>
instead of just somewhere inside the file
<marler8997>
ah...now there's a reason for why they are xattrs
<waddlesplash>
marler8997: there are then also attribute "indexes", so you can run "query" to search for e.g. "media/* files with MEDIA:duration >= ..."
<marler8997>
I'd be interested in seeing what Haiku has to offer, I see there's videos on the website, any you would recommend I look at?
<marler8997>
ah
<waddlesplash>
marler8997: the Be File System, which comes from BeOS, yes
<marler8997>
or does haiku use it's own filesystem format?
<marler8997>
interesting, which filesystem type supports xattrs like that?
<marler8997>
waddlesplash out of curiosity, what sorts of things does Haiku do with the extra xattrs it supports?
2021-01-10
<marler8997>
daurnimator, huh? I tested and it looks like windows doesn't do that
<daurnimator>
andrewrk: marler8997: yes on windows you do a syscall per ".."
<dkibi>
marler8997: thanks, I think I need to understand the semantics of zig better to fully grok this, but it's certainly a "huh" when someone tries to use intuition from other languages
2021-01-09
<marler8997>
var bar = 3; is nonsensical. var foo = .{3,4}; probably works because it's s tuple which CAN hold runtime values even though it actually doesn't. We could modify the lanaguge to make a tuple with no runtime values an error when assigned to a runtime variable.
<marler8997>
yeah thanks for looking into this, there are some subtleties here that are not very straightforward
<marler8997>
that sounds reasonable
<marler8997>
by that I mean, is there a way Zig can avoid having to make the decision
<marler8997>
I think this is the sort of decision I would try to avoid having to make
<marler8997>
and if the file doesn't exist, then we just default to removing it correct?
<marler8997>
I suppose that's true
<marler8997>
this means that for all filesystem calls, we're going to need to get the filesystem attributes for every directory in each path
<marler8997>
this might cause a fair amount of extra overhead on windows...hmmm
<marler8997>
ok fair enough
<marler8997>
why?
<marler8997>
which behavior do we want to choose, windows or linux behavior?
<marler8997>
it performs ".." resolution before symlinks, so the behavior in my PR matches what windows does
<marler8997>
looks like on windowse it's the opposite
<marler8997>
I see in linux they perform symlink resolution first
<marler8997>
and the other question is what's supposed to happen first, symlink resolution or ".." resolution
<marler8997>
good question
<marler8997>
however, for symlinks to work another step needs to be introduced
<marler8997>
to be clear, the logic I added here is the same logic that ReactOS uses to normalize the final path
<marler8997>
yeah definitely
<marler8997>
I could be missing a step where I'm supposed to resolve symlinks before removing "." and ".."
<marler8997>
I pulled the implementation from reactos
<marler8997>
to be honest I don't know much about how symlinks are supposed to work on windows, they are a pretty new thing
<marler8997>
all it does is remove "." and ".." from the string
<marler8997>
andrewrk, that function doesn't touch the filesystem, so symlinks aren't relevant
<andrewrk>
marler8997, also doesn't this implementation perform incorrectly with respect to symlinks?
<andrewrk>
marler8997, what's the difference between removeDotDirsSanitized and std.fs.path.resolveWindows?
<g-w1>
ah marler8997, I think I see what you are asking, I think it is meant to be like cmake where it just builds in your cwd. that is what people discussed in the meeting
2021-01-07
<daurnimator>
marler8997: I don't particularly have an opinion on that.
<marler8997>
why does it make more sense to you?
<marler8997>
I dunno, I'm not saying one is better, I'm asking why CWD is being chosen over build.zig directory
<daurnimator>
marler8997: why would I want the build.zig directory hard-coded into my binary?
<marler8997>
> instead of the build.zig directory?
<daurnimator>
marler8997: what other default would make more sense?
<marler8997>
> hardcoding a relative path of... `.` makes sense (as a default)?... but why though?
<marler8997>
instead of the build.zig directory?
<marler8997>
what's the rationale behind using CWD as default prefix?
<marler8997>
right now the most reliable docs is the source code
<marler8997>
so is the plan to unrevert it as some point?
<marler8997>
I've been updating all my scripts and documentation
<marler8997>
hmmm, so yesterday zig build starting putting binaries in debug/bin, but now they are back in zig-cache/bin?
2021-01-06
<marler8997>
right?
<marler8997>
that can cause the program to call "send" or "recv", and that may produce an ENOTCONN
<marler8997>
for example, when you have multiple threads that have a socket handle, the way to signal the socket to be closed is to call shutdown on it
<marler8997>
ikskuh, I'm looking into removing SocketNotConnected, however, I think there are use cases for it
<marler8997>
documentation seems a little hazy on it
<marler8997>
although, I'm not sure on all platforms and in all cases, it's not possible for send to return ENOTCONN or WSAENOTCONN because of a disconection (i.e. it would be a runtime error in this case)
<marler8997>
ikskuh, ok thanks for clearing that up
<ikskuh>
marler8997: not sure why that decision was made
<marler8997>
ikskuh?
<marler8997>
a *runtime error (not bug)
<marler8997>
why would it be a runtime bug for sendto, but not send?
<marler8997>
not following
<marler8997>
but then why is it a SendToError? would it also be a codebug when returned by sendto as well?
<marler8997>
ikskuh, I believe I followed it but I didn't realize SocketNotConnected was a compile-time error (not a runtime error)
<ikskuh>
marler8997: i assume you didn't see/follow the PR that created the original problem in the first place?
2021-01-05
<marler8997>
you gotta do what you gotta do, balance between dev speed and stability
<marler8997>
my strategy, keep rebasing until someone else fixes the CIs :)
<andrewrk>
marler8997, the most recent failed on says "LLVM ERROR: out of memory" at the bottom
<marler8997>
I do yes, I've looked through the logs before and never saw anything about processes being killed, just that they exit with a non-zero exit code
<andrewrk>
marler8997, I think you have access to the logs right? if you click the red x from master branch commit history
<marler8997>
from a quick glance, it doesn't look like windows has an OOM killer, so how is this behavior on the Windows CI occuring? It kills processes with no errors?
<marler8997>
I'm not too familiar with how OOM works on windows...might need to research that one
<marler8997>
no error message on OOM
<marler8997>
is it possible to fix that issue on the CIs?
<marler8997>
ah
<marler8997>
how did you find out windows CI was running out of memory?
2021-01-03
<marler8997>
I see what you're saying now
<marler8997>
that would fix the issue around string literals/error messages yes
<daurnimator>
marler8997: if there was a distinct String type from just "array of u8", then there would be no debate over how slice of u8 is printed: it would always be as the numbers
<daurnimator>
marler8997: I thought I made an issue for it once but I forgot
<marler8997>
woops spelled your username wrong, daurnimator
<marler8997>
darunimator I didn't understand your comment in the array format specifier proposal
<marler8997>
g-w1, nevermind, I was running the old zig.exe which has not moved because the install dir was changed :)
<marler8997>
g-w1 doesn't look like your PR fixed the issue
<marler8997>
$build_root?
<marler8997>
thanks g-w1
<marler8997>
anyone know how "zig run" may have been broken recently? It's adding the compiler as an extra argument to the running program
2021-01-01
<marler8997>
to avoid the heap allocation?
<marler8997>
andrewrk, are you wanting us to duplicate the RtlDosPathNameToNtPathName logic inside the Zig standard library?
<andrewrk>
marler8997, the reason is mostly holidays but I'll look now
<marler8997>
not being able to include ".." in pathnames on windows...I can't believe it's been broken for so long
<marler8997>
any reason why this PR that fixes a truly egregious error hasn't been merged yet?
<marler8997>
true, I'm also pretty limited since it takes a while for my PRs to get looked at :)
<marler8997>
there's just so much to do, it's endless
<marler8997>
probably should submit a PR to fix that
<marler8997>
looks like the default one doesn't do anything for a Windows application (with no console)
<marler8997>
sweet
<marler8997>
almost null :)
<marler8997>
(@intToPtr(fn() void, 1))();
<marler8997>
can you also provide a default segfault handler?
<marler8997>
root.panic
<marler8997>
pivoting...should the root.panic handler be called on a null-reference error on windows?
<marler8997>
like Declaration.name, StructField.name, etc
<marler8997>
the strings inside TypeInfo
<marler8997>
I just assumed they were since string literals are
<marler8997>
it is guaranteed to be null-terminated correct?
<marler8997>
it worked, but, looks like decl.name isn't declaring that it is null-terminated
<marler8997>
ah...so it should work with decls as well
<andrewrk>
@marler8997, @field is just a.b syntax where b is a comptime string
<marler8997>
is there a decl equivalent for @field?
2020-12-27
<marler8997>
congrats on fixing the deadlock...not an easy task
2020-12-26
<marler8997>
very common for calling windows functions
<marler8997>
it's a common practice for many languages to do path conversions on the stack this way
<marler8997>
justin_smith yes
<justin_smith>
marler8997: is PathSpace a fixed size struct?
<justin_smith>
marler8997: you are returning a stack allocated value, I'm actually surprised zig lets you do this
<marler8997>
maybe it is saving it...interesting it looks like the compiler isn't able to optimize the return value from a local variable, so it has to store the return value twice on the stack and copy it
<marler8997>
so the path return value isn't saved to a local variable, it's passed directly to a function...so I want to make sure Zig is still reserving that memory, but it doesn't really look like it is in the assembly code
<marler8997>
trying to figure out if Zig is handling this correctly, it's a function that converts a UTF8 path to a UTF16 path, but the path is returned as an array value
2020-12-24
<marler8997>
Hey, I'm working on windows path issues, it looks like Windows has some functions that are exported from ".exe" files (not always dll files). How do you think the Zig compiler should handle that?
2020-12-17
<marler8997>
I'm hesistant to say anything about a language solution to prevent these types of errors, as I said earlier it's a very deep topic
<justin_smith>
marler8997: even then, when you make a new thread, all visible heap is now shared, I don't think what you are describing is straightforward with threads (but it might be straightforward with peer processes that only share file handles)
<marler8997>
the only way I can think to do something like that is with some sort of "pointer coloring" or borrow checker
<justin_smith>
marler8997: wouldn't an interprocess stack reference be a special case of stale stack references which can already exist in one thread?
<marler8997>
specifically for the Zig compiler
<marler8997>
*interthread
<marler8997>
so you're saying we should try to avoid interprocess stack references?
<marler8997>
ok, so then you're saying we don't think a thread will exit without killing all the threads?
<marler8997>
what would it do in that case?
<marler8997>
OOM
2020-12-16
<marler8997>
if a thread dies, it's stack goes away, so any thread that has a reference to it's stack will be using freed memory
<marler8997>
for example, say the main thread has allocated the ThreadPool metadat on it's stack...should allocate it on the heap instead to avoid this?
<marler8997>
in the interim though...with the language we have today, do you we should avoid threads accessing memory that lives on the stack of another thread
<marler8997>
woooo, that's a deep topic
<marler8997>
andrewrk, once we enable multithreading a new class of errors will be enabled. What do you think about threads having access to stack memory from other threads? Should we try to avoid this?
<marler8997>
see my comment on the PR, I haven't finished reviewing all the code though but I found that bug from a partial review
<marler8997>
andrewrk I see another bug
<andrewrk>
marler8997, interesting, it looks like the new thread pool stuff deadlocked on aarch64-linux
<marler8997>
an enhancement would be to select on the thread handles and the reset event, so we could handle when a thread exits without notifying the wait group
<marler8997>
if we're keeping threads alive, then we need waitgroup
<marler8997>
if that's the case then what I'm saying is nonsensical
<marler8997>
based on my understanding, that's after we call thread.wait() on all the worker threads, but I'm not 100% sure
<marler8997>
my deadlock suggestion about removing AutoResetEvent could be wrong, I'll have to review the code to be sure
<marler8997>
I haven't manually reviewed the code yet though, that was just the issue discovered by running the program
<marler8997>
the only other issue is the deadlock issue, but fix is simple, remove the AutoResetEvent from the wait group, and instead of calling wait, std.debug.assert(wg.count == 0)
<marler8997>
yeah it only optimizes the cold shutdown path as far as I can tell
<marler8997>
potential optimization is to access is_shutdown through an atomic
<marler8997>
don't you just love how 24-core cpus expose all the multi-threaded bugs :)
<marler8997>
in runWorker I moved the check inside the mutex and now I seem to be able to run indefninitely
<marler8997>
it's not being synchronized, it either needs to be checked inside the mutex, or accessed through an atomic
<marler8997>
self.is_shutdown
<marler8997>
FOUND IT
<marler8997>
interesting, I added a log at the beginning of runWorker, the thread that is keeping the process alive doesn't even print that
<marler8997>
I can reproduce more easily by decreasing the number of tasks as well
<marler8997>
and the main thread exits, and this last thread keeps the main process from exiting
<marler8997>
but it never runs a task
<marler8997>
I can see via strace that it gets created and is waiting for an even by waiting on the futex
<marler8997>
well, it never gets a task at least
<marler8997>
the next bug manifests by a thread neve waking up
<marler8997>
andrewrk, (I actually modified the code to call exit to reproduce that bug yes)
<marler8997>
ok I've narrowed down another bug
<marler8997>
if it's not, then it means a thread has exited abnormally, so instead of deadlocking we can exit
<marler8997>
after we're done running, instead of calling wg.wait()...we check that the counter is 0
<marler8997>
ok I think all we need to do is remove the reset event from the WaitGroup
<marler8997>
the main thread already waits for all threads to exit from the loop that calls "wait" on all the threads
<marler8997>
it looks like the fix here is just to remove the wait group
<marler8997>
I don't see this as acceptable at all
<marler8997>
but having an intermittent failure builtin to the code is horrible, it results in intermittent deadlock failures that are impossible to debug
<marler8997>
I mean, it seems to cause the thread to exit (not the whole program)
<marler8997>
I'm not sure
<marler8997>
so you just hang
<marler8997>
and the main thread stays alive
<marler8997>
sometimes std.debug.warn is causing the program to exit without calling wg.exit
<marler8997>
in his example, all we do is call std.debug.warn...which apparently will cause this deadlock
<marler8997>
it means whenever you get a crash, instead of an error the compiler will just hang
<marler8997>
ugh that's a horrible experience
<marler8997>
the only way I know of to avoid that is to use thread joining
<marler8997>
the second bug is just a problem with wait group, basically, with a wait group, if any worker exits without calling WaitGroup.done, then we are in deadlock
<marler8997>
he figured out the first bug
<marler8997>
yeah
<andrewrk>
marler8997, did you manage to get in contact with protty?
<marler8997>
the implementation needs to be able to detect when a worker has exited whether or not it got a chance to call wg.done()
<marler8997>
so if a worker asserts or exits abnormally, it won't call wg.done() and then we are deadlocked
<marler8997>
the implementation depends on each worker calling wg.done()...
<marler8997>
so I think this is an issue with the design
<marler8997>
my last deadlock took 492 invocations
<marler8997>
I produce it by running it in an infinite loop with: while true; do zig run thread_pool.zig; done
<marler8997>
looks like there's another issue, a deadlock
<marler8997>
looks like kprotty has fixed it on my machine
<marler8997>
I'm trying to root cause
<marler8997>
I'm getting a segfault on my 24 core machine with the thread_pool.zig code
<marler8997>
yo
<andrewrk>
marler8997, yo
2020-12-15
<marler8997>
it's the frontend
<marler8997>
yes, but the problem is the backend (Zig's backend is in C++)
<marler8997>
in Debian
<marler8997>
and I'm pretty sure they have a dlang package
<marler8997>
but wait a second, Dlang would have this same problem of trust
<marler8997>
so I suppose having a handwritten Zig compiler written in C would have a use case here
<marler8997>
pixelherodev I'm not coming up with a solution
2020-12-14
<marler8997>
I'll just get a nueral network to create a proof in lisp :)
<marler8997>
Back in college I did alot of this Lisp variant called ACL2 that allowed you to write mathematical proofs in Lisp.... :)
<marler8997>
well the solution I proposed earlier is at best incomplete
<marler8997>
thanks :)
<marler8997>
sure, but I'm still not sure we can't solve the CBE case, I'm formalling it a bit right now to be sure
<pixelherodev>
marler8997: there's existing *binaries* of C compilers considered trustworthy
<marler8997>
pixelherodev, what?
<pixelherodev>
marler8997: you stumbled on exactly why all source has to go through C
<marler8997>
you have to start somewhere of course
<marler8997>
pixelherodev, yes I realize that :)
<pixelherodev>
marler8997: if you assume the tools are trustworthy, sure - but keep in mind that you specified "trusted compiler" as a prerequisite
<marler8997>
I'm not saying it's solvable in the general case
<marler8997>
sure, but it's still solvable in certain cases
<pixelherodev>
marler8997: ah, but what if it optimizes differently?
<marler8997>
pixelherodev, the obvious way is to take a trusted compiler and regenerate the binary from source
<marler8997>
thinking...
<pixelherodev>
marler8997: if I give you a random binary, and its source, how would you prove them equal?
<marler8997>
so where does this statement come from ""prove code generated by program equals inputs to program"
<marler8997>
leeward yeah, but I'll assume we can't do that here to get through pixelherodev's arguments
<marler8997>
I'm not familiar with this problem
<pixelherodev>
marler8997: let's make this a bit worse (paranoid level 9000!): what if cat,objdump,readelf,hexdump,etc are lying to you?
<marler8997>
pixelherodev oh?
<marler8997>
ok so you're calling this situation equivalent to others by treating CBE as a binary
<marler8997>
in regard to trust I'll assume it is 100% equivalent here
<marler8997>
pixelherdev, in many ways yes
<marler8997>
if it's equivalent to this very difficult problem then it's not going to be solved simply like this
<pixelherodev>
marler8997: CBE *is binary*
<marler8997>
This situation sounds unique because not many other tools have a CBE like this, I'm operating under the insight that this may produce a unique situation with a fairly simple solution
<marler8997>
I'm not sure that's true though
<marler8997>
you're saying the Zig situation is equivalent to others that have not been solved, if that's true then sure
<pixelherodev>
marler8997: and nobody's figured out a solution
<pixelherodev>
marler8997: because you're far from the first person to try this
<marler8997>
because it would have big implications? Why is that?