<marler8997>
I mean, I don't think we need a proof of concept for this
<marler8997>
so in `build.zig` @import("buildoptions")
<marler8997>
noam, yeah generating something like that is what I hthought of
<marler8997>
they don't get passed to main.zig
<marler8997>
those aren't comptime, their runtime
<marler8997>
g-w1, yeah I'm not familiar with any concept like that
<marler8997>
so the best I can do is try to implement it and see what happens
<marler8997>
Unfortunately I don't know enough about comptime to really know whether limiting build.zig to comptime for it's configuration would even be viable, or a good idea, but seems like it could be
<marler8997>
the problem of missing imports goes away if all out build-configuration is provided at comptime, because now we can put our @imports inside if blocks that are known at comptime
<marler8997>
yeah we're talking about the details of a proof-of-concept
<marler8997>
now we're exploring something in between the 2
<marler8997>
mikdusan, pretty much yeah, but it's based off another proposal to make build.zig comptime only so it is sandboxed
<marler8997>
g-w1, build.zig passes comptime parameters to main.zig?
<marler8997>
mikdusan, that's where your "exec" function is called
<marler8997>
and how does it pass comptime args to it?
<marler8997>
so build_build.zig is the one that actually builds build.zig
<marler8997>
need more context for that one
<marler8997>
or with "zig build run ARGS", ARGS would be runtime correct?
<marler8997>
would the target name also be comptime?
<marler8997>
I don't think it's a big deal, but I'm not sure yet
<marler8997>
the big downside here is that we need to recompile build.zig with every combination of command-line arguments
<marler8997>
I suppose it's actually different because the logic now executes at runtime
<marler8997>
yeah I think you're right, though I'm not sure how different that is
<g-w1>
marler8997: heck this can be even simpler. all we need to do is support comptime args to build.zig; thats it!
<marler8997>
time to see if DOD is all it's cracked up to be
<marler8997>
congrats on green CI, that's awesome!
<marler8997>
flao, you can remove a level of nesting if you just put your (enable_llvm) around the block above it
<marler8997>
but it's not immediately clear, I would an if { .. } else { ... } to make it very clear
<marler8997>
ok, then the compiler shouldn't analyze the lines below if llvm is missing
<mikdusan>
marler8997: yes some kind of replace-existing-process
<marler8997>
unless, is exec noreturn?
<marler8997>
mikdusan, taht won't compile when llvm is missing because llvm won't have the methods like getLibrariesToLink
<marler8997>
if you're off, then I'll be trying it before then :)
<mikdusan>
marler8997: updated gist. take a peek when u have chance. afk for 10 mins.
<marler8997>
we can try the ziget build
<marler8997>
I'm just not familair enough with comptime to know
<marler8997>
we need to create a simple build_running.zig and see if it works
<marler8997>
Would that work?
<marler8997>
oh, it looks exactly the same as it does today?
<marler8997>
so what does build.zig look like?
<marler8997>
well that clears up one part of my confusion
<marler8997>
so then what was the code you pasted earlier? I thought it was the content of build.zig?
<marler8997>
@import("build.zig").Builder?
<marler8997>
I thought Builder was defined in build.zig?
<marler8997>
I don't understand, what's "execute" and "build" functions for?
<marler8997>
so build is now executed at comptime instead of runtime?
<marler8997>
now I'm the one who isn't understanding :)
<marler8997>
g-w1, I'm still groking your example
<g-w1>
marler8997: does my explanation make sense? build.zig modifies a Builder at comptime and then build_runnner.zig runs it at runtime
<marler8997>
-Denable-llvm is a an option passed to the compiled build.zig executable
<marler8997>
no, it's a runtime value
<marler8997>
*surrounded by an if that uses a runtime value
<marler8997>
mikdusan, you're missing and if (b.option("enable-llvm")) around your block that uses @tryPackage("llvm"). The key here being that the block is surrounded by a runtime value
<marler8997>
mikdusan, looking
<g-w1>
const Builder = struct { exes: []Exe, } const Exe = struct { source_path: []const u8, packages: []Pkg } fn execute(b: Builder) { for (b.exes) |exe| exe.build() } fn build(e: Exe) { os.system("zig build-exe {}", .{e.source_path}); } marler8997 this is how I would do it. then pub fn build in comptimebuild would modify Builder
<marler8997>
I think a path forward here might be to add support for a comptimebuild.zig, and prototype this idea
<marler8997>
*build_runner, not build_running
<marler8997>
pub const MyApp = std.build.Exe("myapp", "myapp.zig"); ??? and then build_running uses reflection to get all the pub decls?
<marler8997>
give me an example
<marler8997>
right, but how do I write this build.zig, what are these things? types?
<marler8997>
what are "all the things" exactly?
<marler8997>
so we don't have a build entry point anymore right?
<marler8997>
I'm still trying to figure out how comptime build.zig looks
<marler8997>
you're right
<marler8997>
noam, that's actually a good point
<marler8997>
if it's viable, I do like the idea of comptime build.zig
<marler8997>
yeah, comptime build.zig would be more sandboxed, though, I'm not intimately familair with all the restrictions on comptime
<marler8997>
I'm still fuzzy on the details for a comptime build.zig
<marler8997>
so that's the crucial question, do we want build.zig to be able to use runtime or not
<marler8997>
@tryImport allows build.zig to make use of runtime
<marler8997>
but are both quite different
<marler8997>
they both solve the bootstrap problem
<marler8997>
ah, is that what congruent is supposed to mean?
<marler8997>
*this would now compile
<marler8997>
this would not compile because androidEnabled is comptime known
<marler8997>
it would be if (androidEnabled) { const al = @import("androidbuildlib"); }
<marler8997>
well, it wouldn't be tryImport anymore
<marler8997>
oh right, yes
<marler8997>
what 2 solutions?
<marler8997>
andrewrk, if/when you see this, have you thought about making build.zig comptime only? Doing this would also be a solution to the build.zig bootsrap problem because it would get recompiled with every set of command-line arguments given to it.
<marler8997>
are you saying we can't implement any logic to select our build configuration?
<marler8997>
with build.zig specifically
<marler8997>
what would need to change?
<marler8997>
so how would build.zig work if it executed at comptime?
<marler8997>
it could be more elegant, but I need to understand it more
<marler8997>
I need to know enough to make a PR
<marler8997>
interesting, I'd have to think on this a bit but this could be a viable solution
<marler8997>
is this a thing?
<marler8997>
what is this dynamic zig?
<noam>
marler8997: no, only for the currently given one ;)
<marler8997>
it does mean you'll have to re-compile build.zig every for every possible set of command-line options
<marler8997>
making all the configuration available at comptime is another solution
<marler8997>
yes that would also work
<marler8997>
so does this mean you wouldn't analyze the @import("androidbuildlib") if the user doesn't enable android with their command-line ?
<marler8997>
interpreting build.zig instead of compiling it...that is an interesting idea, thinking...
<marler8997>
that's not a solution
<marler8997>
nope
<marler8997>
yup
<marler8997>
nope
<marler8997>
in build.zig
<marler8997>
androidbuildlib is a comptime import
<marler8997>
it shouldn't make sense when you first look at it and think about it
<marler8997>
noam, if it makes no sense then you're on the right track :)
<marler8997>
just a little more convuluted, @tryImport with more steps
<marler8997>
g-w1, in that case, it's just as powerful as @tryImport
<marler8997>
you would need some way to know whether it actually failed or not, so you can do if (needsPackageFailed) { // dont call anything from androidbuildlib or you'll get a compile error }
<marler8997>
so when androidbuildlib is missing, what type does @needsPackage return?
<marler8997>
I'm trying to think if it could even work like that
<marler8997>
g-w1, yes that coud work
<marler8997>
we need to be able to build `build.zig` even when androidbuildlib is missing
<marler8997>
yup
<marler8997>
you back now?
<marler8997>
stay with me, I feel like you're close to getting this, what happens to this code when androidbuildlib is missing?
<marler8997>
if (androidEnabled) { const abl = @needsPackage("androidbuidlib"); ... }
<marler8997>
when android is missing, what happens?
<marler8997>
what happens in this example?
<marler8997>
let's compare it to your @needsPackage
<marler8997>
it has to be ?type because whether or not we need the import depends on a runtime value
<marler8997>
?type is the only approach :)
<marler8997>
once you wrap your mind around it, it all makes sense, but it takes a minute
<marler8997>
nope, @tryImport is comptime-known, but we won't know if we actually need it until runtime, which is why we have to handle it being there and not being there
<marler8997>
come on, I know you can get it
<marler8997>
and androidEnabled is a runtime known-value
<marler8997>
we only ever enter this block if android is enabled, if android is disabled, our build still works so long as they don't enable android
<marler8997>
the "if (androidEnabled)" is the point
<marler8997>
the else is not the point of it
<marler8997>
but what's important is that this is all wrapped in the if (androidEnabled)
<marler8997>
it can be a compile error
<marler8997>
anything can go there
<marler8997>
if (androidEnabled) { if (@tryImport("androidbuildlib")) |abl| { ... } else { ... } }
<marler8997>
ok, then it doesn't solve the problem becuase we need a way to not get a compile error
<marler8997>
so basically with @needsPackage, it's exactly the same as @import except it lets you provide a custom error message?
<marler8997>
thinking...
<marler8997>
yeah that's true
<marler8997>
yeah I mean, that's one solution, but @tryImport can be applied to that idea, or the package-manager idea if we want to separate our dependencies into another file
<marler8997>
g-w1, what's your take on this?
<marler8997>
I'll work on that
<marler8997>
andrewrk, yeah that's a good idea because then people can actually understand the problem instead of constatly arguing about whether it is
<marler8997>
I'm more qualifying what I'm saying in case I say something nasty because my emotions were starting to run high there
<marler8997>
Yeah I definitely agree
<marler8997>
but I'm telling you right now, I'm pretty confident your package manager idea of a static configuration file is not going to work, and will eventually need to become more complex, and you'll realize it should have been done in Zig, then you're going to want to support packages in your dependencie.zig file, and then you're back to the exact same problem we have today
<marler8997>
and just to be clear, I'm totally happy with alternative solutions, but it is quite frustrating to discuss this with people who don't even seem to be willing to admit it's a problem, nor make a good faith effort to understand the problem
<marler8997>
but also make sure to consider that dependencies are recursive
<marler8997>
for now
<marler8997>
ok, then that's probably a good place to leave it
<marler8997>
so what do you think about the android example then?
<marler8997>
gotcha
<marler8997>
as opposed to a project's dependencies always being a static non-changing list regardless of the options passed to build.zig
<marler8997>
so the question I would first ask you is, are we agreed that projects should be able to use some logic to change their dependencies?
<marler8997>
@tryImport isn't an off the cuff idea I had, I came up to it by logically reducing the problem
<marler8997>
I feel like I can explain it very simply, the messages I sent earlier should explain exactly why @tryImport is necessary
<marler8997>
this one is especially complex because it has so many tendrils in so many other parts of the future of Zig
<marler8997>
this is less of a problem when the problem you're solving is simple
<marler8997>
yeah I think you're right
<marler8997>
it's quite frustrating for me to see that behavior
<marler8997>
I don't know why some people are immediately passing judgements on this proposal saying things like "I don't like this" when they clearly have not throught about the underlying problem yet
<marler8997>
by the way andrewrk, I think this is the first time someone has actually asked me a question about this proposal
<marler8997>
the bearssl example is less of a big deal because download that repo is small, but the same concept applies to Big repositories
<marler8997>
I'd like to use it, but without @tryImport I can't
<marler8997>
it provides a "linkWithBearSSL" function that adds all the bearssl C sources to the execuatble
<marler8997>
a library with all the stuff in ikskuh's talk in his build.zig file
<marler8997>
anything that you need to be able to build and Android APK
<marler8997>
it's configuring the manifest file, adding application identify, etc
<andrewrk>
marler8997, let's talk about concrete use cases - so what's the example where we have a build.zig that wants to @import("android_sdk") - what is it doing with that import?
<marler8997>
which provides code to let you configure the build to generate an Android APK
<marler8997>
somewhere in build.zig, there is a @import("androidbuild")
<marler8997>
not just the application code?
<marler8997>
did you understand that it was build.zig that has a dependency on Android?
<marler8997>
I left another comment about 8 minutes ago
<andrewrk>
marler8997, I can chat on IRC for a bit
<marler8997>
I don't see why you're not getting that
<marler8997>
well, I think you missed the point on the android example, you said that we would only need to download the android dependency if we need it but that's not the case, build.zig needs the android dependency
<marler8997>
andrewrk are you available to discuss tryImport
2021-02-24
<marler8997>
and is a common Zig pattern to emulator what classes do in other languages
<marler8997>
this is what the Allocator interface does
<marler8997>
this function pointer takes a pointer to the Task struct as the first argument, then you can use @fieldParentPtr to create "sub type" of your Task to add whatever data you need
<marler8997>
one thing you could do, is instead of a function pointer, you could pass a pointer to a Task struct that contains a function pointer
<marler8997>
I don't know enought about async to help much
<marler8997>
a more generalized version of a functionpointer/argument structure
<marler8997>
which would have any arguments passed to the function
<marler8997>
looks like it's using anyframe to encapsulate an asynchronous function call
<marler8997>
you might want to take a look at std/event/batch.zig in the standard lib
<marler8997>
I recall it was either Kprotty or Andrew put together a task queue library, and they may have had a solution for this, maybe I can find it
<marler8997>
for example, you can create a tagged union type like this Variant(i32, usize, []const u8, ...)
<marler8997>
but I would also question whether or not you can actually get the set of known types
<marler8997>
then you have to use a reference
<marler8997>
for example, for all you know the argument could be a struct that has 100 fields...is that right?
<marler8997>
so you don't know the storage size of this struct holding the function pointer and the argument?
<marler8997>
is this function argument going to be a value of a set of known types?
<marler8997>
it's the only solution, asside from using a reference
<marler8997>
tagged union is probably want you want
<marler8997>
It might also help to mention that @tryImport is only necesary for the corner cases where dependencies are dynamic. In most cases, dependencies are static and will therefore not need @tryImport.
<marler8997>
if anyone has trouble seeing why these 3 statements infer the need for @tryImport, let me know and I can walk you through it
<marler8997>
For Statement 3: Android and QT are good examples of why it would be useful to provide libraries to use logic to determine our dependencies
<marler8997>
say I'm using zig-bearssl, and it adds support for android, so now suddenly my project which uses zig-bearssl but doesn't support android has to pull down android even though my project can't even use it
<marler8997>
Statement 2: this was our earlier discussion which shows without this feature, everyone will need to download every possible dependency that a build.zig file might import including the ones they will never use, and keep in mind this also means downloading all your dependencies possible dependencies
<marler8997>
Statement 1: I think everyone will agree on
<marler8997>
ikskuh actually helped along the way, it's a generaliztion of the "prebuild.zig" idea I had, where it allows build.zig to serve as it's own "prebuild" so it can bootstrap itself. This is the same idea he presented when he talked about build.zig building another builder_runner
<marler8997>
in thinking about this problem quite a bit, @tryImport was a result of a reduction of the requirements and the current state of Zig
<marler8997>
Statement 3: the logic we use to determine our dependencies can have its own dependencies
<marler8997>
Statement 2: we want to allow logic to be used to configure dependencies
<marler8997>
Statement 1: build logic is codified in Zig (not BASH or Python etc)
<marler8997>
ok I think I have 3 statements that if people agree are true, shows the need for @tryImport
<marler8997>
the Android example is a good one I think
<marler8997>
we can stick to examples of just using other libraries to configure our build in various ways
<marler8997>
that might be a bit of a stretch and rub people the wrong way because it sounds like it's trying to be a package manager
<marler8997>
i.e. just throwing our hands up in the air and saying that some dependencies just always have to be downloaded...we don't support any logic to determine when to download them
<marler8997>
there's really no way getting around this requirement. Without this feature, we have to make sacrifices no matter what
<marler8997>
yeah
<marler8997>
just talking about the information you're including in your dependnecy data structure, I know the data you've given here is addressing real problems. Things like a package identity, semantic versioning, signing. All stuff we'll need to address with the package manager. But I'm sort of laser focused on the @tryImport thing before I can really weigh in on these details
<marler8997>
but I hope it's clear that all this is completely orthogonal to @tryImport, all @tryImport does is provide a mechanism for build.zig files to handle missing imports. How dependencies are specified or resolved or how the package manager is somewhat related, but ultimately orthogonal
<marler8997>
I'd have to think about the pros and cons of where to put the dependency information, I'm not sure of them at the moment
<marler8997>
looking
<mikdusan1>
marler8997: this is just shooting from the hip. but this is further along in how I see .zig declaring/using packages:
<marler8997>
brb
<marler8997>
is @package() called at runtime or comptime
<marler8997>
hold on, I might be mixing up yours and ikskuh's examples
<marler8997>
-Dbackend=QT isn't available till runtime, so how does it affect a comptime config?
<marler8997>
ok given these idea, how do we tie the command line option -Dbackend=QT to this comptime configuration?
<marler8997>
it's uncanny how much you are adhering to the theoretical example in my "Story Time" comment
<marler8997>
yeah real-time discussion is alot more efficient and allows conversationt to be more focused because of the feedback loop
<ikskuh>
marler8997: well, that was exactly the point of this discussion
<marler8997>
I don't think I've ever seen such a fast turnaround like that :)
<marler8997>
fundamental disagreement solved?
<marler8997>
> i can agree on opting out of "download all dependencies"
<marler8997>
granting the ability to handle missing dependencies does not mean that we are removing support to download all the dependencies
<marler8997>
your use case is safe and sound, this is orthogonal
<marler8997>
ikskuh, please stop attacking this straw man
<marler8997>
that's exactly what @tryImport allows you to do
<marler8997>
ikskuh, of course you can, we're not saying you shouldn't be able to download all the dependencies, we're saying it shouldn't be required if specifically build a variant that doesn't need them
<marler8997>
it sounds like somethine Microsoft would do
<marler8997>
that just seems ludicrous to me
<marler8997>
a perfect example, ikskuh says you MUST download qt even if you're not using it
<marler8997>
to say this use case is invalid is too strong of an argument, even if I agreed with your fundamental point
<marler8997>
what irks me here is you're asserting that your position here is the superior one by not allowing me to support my position
<marler8997>
it looks like we have a fundamental disagreement here
<marler8997>
I'm not ok with that
<marler8997>
that's absolutely ridiculous
<marler8997>
is that right?
<marler8997>
you're saying that the package manager should always download all 5 GB of dependencies for all platforms no matter what, even if you just want to build the linux variant
<marler8997>
it's got like 5 GB worth of dependencies if you combine all of them, but you only need the core project and the linux platform dependnecy, which amount to around 2 MB
<marler8997>
and it can compile for every possible known operating system out there
<marler8997>
so say you have a project like a text editor
<marler8997>
is that correct?
<marler8997>
you're saying it's ok to download all possible dependencies of a build.zig file, but it's not ok to do that for dependnecies that the project itself uses?
<marler8997>
well you seem to be ok with them being handled differently correct?
<marler8997>
let me ask you this question, what if we were talking about the dependencies of the project itself?
<marler8997>
ikskuh, you fetch the dependencies you use
<marler8997>
even if they have ssl completely disabled
<marler8997>
maybe I want to add some Android ssl backend, which requires a large Android SDK project, now everyone has to download that very large project for android
<marler8997>
and if I'm a project maintainer, this is going to encourage me not to add new dependencies because it means that everyone is always going to have to download this new dependency, even when it is only required for some esoteric build configuration
<marler8997>
if I dont' need it, I don't want to fetch it, I don't want every user to have to download every dependency for every possible build configuration
<marler8997>
I don't want to fetch it, that's the point
<marler8997>
when I build the openssl backend, if the import to "zigwin32build" fails, I don't care
<marler8997>
because some dependencies are optional
<marler8997>
it could, but that's unrelated to the fundamental ability for build.zig to handle it at all
<marler8997>
I'm not saying build.zig has to do anything about it
<marler8997>
but that's also orthogonal
<marler8997>
definitely nope
<marler8997>
ikskuh, nope
<marler8997>
again, this has nothing to do with fetching and managing dependnecies
<marler8997>
what does this have to do with the ability of build.zig to detect and handle missing dependencies?
<marler8997>
ikskuh, that is very easily provably false
<ikskuh>
marler8997: this implies that you're building with the exact same configuration
<marler8997>
I think you're making the assumption that if the build system can handle a missing dependency and provide a nice error message, that this also implies that it's not tracked properly as a dependency at all and therefore wouldn't have been downloaded
<marler8997>
again, this is all orthogonal
<marler8997>
of course
<marler8997>
it's an error like "failed to import missing package "foobuild""
<marler8997>
this has nothing to do with fetching or managing them
<marler8997>
we are only talking about the ability to detect and handle missing dependencies
<marler8997>
whether or not a dependency exists has nothing to do with the build systems ability to detect and handle that missinjg dependency
<marler8997>
orthogonal
<marler8997>
of course
<marler8997>
again, it's completely orthogonal
<marler8997>
this has nothing to do with our discussion
<marler8997>
right, because it's easier to maintain everyone's ad hoc solution that gets around the limitation??
<marler8997>
why are we forcing people to comprimise?!?
<marler8997>
when you see a library you want to use, you now have to decide if you want to require everyone to always download it, even if its not being used
<ikskuh>
marler8997: it contradicts maintainability though
<marler8997>
and it's fine if it's hard to do dependencies because it naturally makes people not use them
<marler8997>
downloading all your dependencies might be fine for small projects, and while Zig is in its infancy
<marler8997>
we're talking about the case where you are downloaded dependencies that you don't need and aren't using
<marler8997>
ikskuh, it's fine if you want to cross build, we're not talking about that case
<marler8997>
They are going to come up with "configure" script to run before Zig build
<marler8997>
this solution doesn't scale, and what's going to happen is people are going to come up with clever hacks to get around this limitation
<marler8997>
and all it's dependencies. Every dependency that build.zig depends on now has to be downloaded
<marler8997>
you're also going to need the 80 MB zigwin32 repository, even though you're not even on windows
<marler8997>
it is completely orthogonal
<marler8997>
this has nothing to do with the internet