<pixelherodev>
marler8997: yes, but the only solution that's been devised in the last few decades requires *multiple compilers*
<marler8997>
the question is, if we can prove that the Zig soruce code is equivalent to the CBE, can we solve the problem of creating a trusted Zig compiler from the Zig source?
<marler8997>
hmm, I don't think that was the question
<pixelherodev>
marler8997: you can prove that source == binary, yes
<marler8997>
what was the answer to the question?
<marler8997>
sorry what?
<marler8997>
ok cool, we're on the same page now
<pixelherodev>
marler8997:which one?
<marler8997>
> if we can find a way to verify the CBE is equivalent to the Zig source, then we can solve the problem yes?
<marler8997>
pixelherodev can you answer the other question?
<marler8997>
ok, you could be right this is an unsolved problem, I'll think on it a bit more
<marler8997>
pixelherodev, can you answer the question though?
<marler8997>
this is equivalent to trusting the binary of A C program so long as we trusted the C source and the compiler yes?
<pixelherodev>
marler8997: this is an unsolved problem, so I wish you luck :P
<marler8997>
pixelherodev, correct, if we can find a way to verify the CBE is equivalent to the Zig source, then we can solve the problem yes?
<marler8997>
what I'm trying to do is find a way for the Zig source code to verify the CBE source code
<marler8997>
pixelherodev yes
<marler8997>
yeah I'm missing a piece here
<marler8997>
thinking...
<marler8997>
answered what?
<pixelherodev>
marler8997: you still haven't answered
<marler8997>
pixelherodev I agree
<marler8997>
but first I need pixelherodev to agree with the points, the first one being "The Zig source is trusted"
<marler8997>
it's very simple
<pixelherodev>
marler8997: that was the coreof your argument!
<marler8997>
no it's not
<marler8997>
pixelherodev, yes I agree
<marler8997>
you're not following here, you keep going back to this point, which i AGREE with
<marler8997>
pixelherdev yes
<pixelherodev>
marler8997: sure. and malicious CBEs.
<marler8997>
a compiler doesn't make source malicious, it produces malicious binaries
<marler8997>
source can't be trusted? so then what can be trusted?
<marler8997>
pixelherodev....uh wut
<marler8997>
but the Zig source is trusted yes?
<marler8997>
*is untrusted
<marler8997>
the CBE it untrusted
<marler8997>
I'm saying, a compiler can be untrusted at one point in time, and then trusted after verifying it sometime later
<marler8997>
I didn't say anything to the contrary
<marler8997>
pixelherdev yes
<marler8997>
which is what my proposed solution attempts to do
<marler8997>
a compiler can be untrusted, then then verified after the fact and then become trusted
<marler8997>
it has not been proven to be malicious or non malivious
<marler8997>
pixelherdev, untrusted doesn't mean malicious
<pixelherodev>
marler8997: *if* the input is trusted, you've proven that the output is as well
<marler8997>
but I can't really read why we're talking like this :)
<marler8997>
I asked if the paper I sent was the paper and you said it might be
<pixelherodev>
marler8997: it *doesn't matter* if it's equivalent
<marler8997>
and then we prove that it actually is trusted by proving that it is equivalent to the trusted Zig source code
<marler8997>
it is "untrusted"
<marler8997>
but you're assumign that the compiler is malicious before proving that it is...my sequence relies on an unknown compiler
<marler8997>
pixelherdev yes
<pixelherodev>
marler8997: that's literally my point
<marler8997>
pixelherodev, how does that binary compile the Zig source code into a CBE that is equivalent?
<pixelherodev>
marler8997: the malicious compiler can always produce a binary that's equivalent to `echo 'printf("Hello world!");'`
<leeward>
marler8997: Ok, so what does it purport to do? I'm missing some context.
<marler8997>
but practically it's generated from the Zig source
<pixelherodev>
marler8997: it's *not* though!
<marler8997>
and they have to be equivalent
<marler8997>
leeward, it doesn't matter because the second time we generate it, it has to be generated form the Zig source
<leeward>
marler8997: UZC1 is generated from what source by Zig's CBE?
<marler8997>
I'd have to formalize it a bit more on my end, but I believe I'm using induction to prove it
<marler8997>
pixelherodev, again this is a bit complicated but you need to think about this for a minute
<marler8997>
pixelherdev. yes
<marler8997>
it would be the same if the compiler was written in C
<marler8997>
that's literally the only thing you can do, is verify the original source
<marler8997>
but you can trust that because the Zig source code is trusted
<pixelherodev>
marler8997: fundamentally, that "untrusted compiler" can't even be trusted to *bea compiler*
<marler8997>
pixelherodev yes
<marler8997>
If we find that the new C backend is equivalent to the original C backend code that we generated our Zig compiler from, then that says the CBE is equivalent to the trusted Zig code that we verified
<pixelherodev>
marler8997: for that to work, you have to trust that that Zig binary is in fact translating the zig to C
<marler8997>
Then we can take that Zig binary, and use it to re-generate a new vesrion of the C Backend from the trusted Zig source code
<marler8997>
ok, so we have an untrusted Zig binary
<leeward>
I want to hear the end, marler8997.
<marler8997>
pixelherodev, it's broken because it's not finished yet...lol
<marler8997>
pixelherodev, I'm not done with the sequence
<marler8997>
Then we compiler the untrusted C backend code to create an untrusted Zig compiler
<pixelherodev>
marler8997: it doesn't matter
<marler8997>
Then we have the trusted Zig source code that is handwritten, verified by humans TZC
<marler8997>
Then we have "untrusted C code generated from Zig's CBE" (UZC1 "untrusted Zig C code 1")
<marler8997>
we start with a trusted C compiler (TCC for "trusted C compiler")
<marler8997>
I'll go through the sequence again with more details
<marler8997>
I never said it wasn't
<marler8997>
yes that is true
<marler8997>
The Zig compiler's maliciousness == the maliciousness of the compiler used to build it
<pixelherodev>
marler8997: Even if the source is trustworthy, the output isn't
<marler8997>
pixelherdev yes!
<marler8997>
my solution assumes we can verify the trustworthiness of the handwrittne Zig compiler
<marler8997>
pixelherodev, the Zig source code is trusted though, so if it did produce malicious code then that's fine
<marler8997>
pixelherdev, ok but what is that fact?
<pixelherodev>
marler8997: fundamentally, your solution misses a key fact.
<marler8997>
yes, all of this depends on the trustworthiness of the C compiler
<marler8997>
pixelherodev, I'm not saying I'm right or wrong, I proposed a solution and I'd like to know if it's right or wrong
<marler8997>
and I came up with a way to verify CBE's trustworthiness from the trustworthiness of the Zig source code
<marler8997>
I'm basing the trust on Zig's source code
<marler8997>
that's not what I proposed
<marler8997>
pixelherodev yes
<marler8997>
let's formalize a bit here so we can be sure
<marler8997>
ok let me check your proposed break
<pixelherodev>
marler8997: I did :P
<marler8997>
pixelherodev, all you have to do is come up with a way to break my sequence
<marler8997>
if it changed the source, then the original and newly generated CBE wouldn't match
<marler8997>
that code would have to be in the Zig code as well
<marler8997>
but that "smartness" is encoded in the Zig source
<marler8997>
I said it proves that CBE is equivalent to the Zig code
<marler8997>
I didn't say it did
<marler8997>
we compare the new CBE with the original CBE to verify they are the same, which implies that the original CBE is equivalent to the trusted Zig code
<pixelherodev>
marler8997: yes, *but*
<marler8997>
We use that untrusted Zig compiler to compile the trusted Zig code to a new CBE
<marler8997>
we generate an untrusted Zig compiler
<marler8997>
We're using CBE to verify that Zig's trustworthiness implies CBE's trustworthiness
<marler8997>
we're depending on the Zig source being trusworthy
<marler8997>
pixelherodev, we're not dependign on CBE being trustworthy
<pixelherodev>
marler8997: I responsed quickly because, as I said, this is a known problem with aknown solution
<marler8997>
I'm still thinking about it myself and am unsure about it
<marler8997>
I didn't assume, you just responded very quickly to something that is complex
<marler8997>
pixelherodev yes I agree
<marler8997>
it's not simple
<marler8997>
again, take a moment to look at it
<marler8997>
my scenario assumes CBE isn't trusted
<marler8997>
I didn't say CBE is trusted
<marler8997>
pixelherodev, be careful what I just said is a bit complicated, you may want to take a moment to ponder on it
<pixelherodev>
marler8997: nope
<marler8997>
if they are equivalent, then you by veryfing the Zig code you are transitively also verifying the CBE code
<marler8997>
deploy Zig code and CBE code, during build, compile CBE to make Zig compiler, then compile Zig code to CBE, compare that generated CBE is the same as the committed CBE
<marler8997>
ok what about this
<marler8997>
CommunistWolf gotcha
<marler8997>
so if we made CBE readable then it wouldn't be a problem?
<marler8997>
ok, so the reason is, generated code is harder to read than handwritten code?
<marler8997>
if so, how did they verify the original compilers written in assembly?
<marler8997>
are you saying that all assembly code is untrusted?
<marler8997>
but in any case, assembly is still source that can be audited
<marler8997>
not really though
<marler8997>
so you're saying because it's hard to read?
<marler8997>
so why can't people audit CBE source?
<marler8997>
how do you verify whether new source is trusted or untrusted?
<marler8997>
how do you verify whether new source is trusted or untrusted?
<marler8997>
you're claiming that CBE is untrusted, so that how do you determine whether any new source is trusted or untrusted?
<pixelherodev>
marler8997: according to the Zig project (or, if you prefer, according to Andrew): CBE output is 100% intended to be a binary
<marler8997>
how do you verify any source is trusted?
<marler8997>
it's not 100% binary code, it has aspects of both binary and source code
<marler8997>
how do you verify any source is trusted?
<marler8997>
just like any other source code
<marler8997>
it's source code that you can read and verify
<marler8997>
so
<pixelherodev>
marler8997: but that CBE output was produced *by a compiler*
<marler8997>
what do you mean "tainted"?
<marler8997>
you take your trusted C compiler, compile the code generated by CBE, now you have a Zig compiler
<marler8997>
pixelherdev nope
<marler8997>
pixelherdev, yup :)
<pixelherodev>
marler8997: nope
<marler8997>
pixelherodev, and that Zig compiler can be generated with C source (that is also generated)
<marler8997>
you said that you can't add new binaries, but I have a hard time believing that they never update their toolchains
<marler8997>
and do these toolchains update the C binaries?
<marler8997>
if you disallow generated code then you can't use binaries either
<marler8997>
it's generated source just like binaries
<marler8997>
the contradiction is that you're saying we can't use the CBE in the chain because it's generated code, but binaries themselves are generated, so saying that the chain can't have generated code doesn't hold true
<marler8997>
yes, but that's not the contradiction
<marler8997>
these rules are contradicting each other
<marler8997>
but the chain does have binaries
<marler8997>
right, so by extension, binaries aren't maintained, so you can't use binaries either
<marler8997>
i.e. you're saying you can't use CBE in this chain
<marler8997>
pixelherodev, yes, but you're limiting everything to custom maintained C code
<pixelherodev>
marler8997: because that expands how much needs to be trusted
<marler8997>
how did the original C compiler get introduced into the chain?
<marler8997>
*fengb* why can't you introduce new binaries into the chain?
<marler8997>
there's a distro that doesn't us binaries to bootstrap C code?
<marler8997>
so they are using binaries to boostrap?
<marler8997>
so they are using binaries to bootstrap?
<marler8997>
the next question is...."how do they process C source"
<marler8997>
pixelherodev, I've moved on to the next question
<marler8997>
why would they trust source written in C but not in Zig?
<marler8997>
why?
<marler8997>
which they have solved
<marler8997>
so how do they process C source?
<marler8997>
they can only process C source then?
<marler8997>
they can only process C source then?
<marler8997>
you can submit the Zig source, and generate the CBE
<marler8997>
pixelherodev, sure, but I'm not saying that we submit just CBE to debian
<marler8997>
so in the Zig case, the Zig source code is the source, CBE is an intermediate representation
<marler8997>
and the binary is an output of that
<marler8997>
The GCC source
<marler8997>
The source is what is used for bootstrapping
<marler8997>
that's the distinction
<marler8997>
that's all that matters :)
<marler8997>
for the C code from CBE, we have the source to re-generate it
<marler8997>
it's not the same as having a binary that you don't have source to
<marler8997>
but we're generating it from the Zig source
<marler8997>
pixelherodev yes
<marler8997>
and using C as an intermidiary format
<marler8997>
pixelherodev, by using CBE, we're bootstrapping from the Zig source
<pixelherodev>
marler8997: but most *can* be bootstrapped with a "fully" source chain
<marler8997>
pixelherodev, all programs are technically bootstrapped by binaries if you go down far enough
<marler8997>
So the reason for not using CBE to bootstrap is because of Debian?
<marler8997>
oh gotcha
<marler8997>
who are the rule lawyers?
<marler8997>
what's wrong with using CBE for bootstrapping?
<marler8997>
fengb, not following
<marler8997>
I was perusing another langauge that only supports a CBE, and it commits it's generated C code to a repo for bootstrapping
<marler8997>
and other languages?
<marler8997>
isnt' that what Nim does?
<marler8997>
you're saying we shouldn't use the gnerated code for bootstrapping
<marler8997>
oh wait nvm
<marler8997>
but what about my question?
<marler8997>
fengb yes
<marler8997>
but why would we do that? Instead, why not keep the CBE unmodified? Why do we need to maintain it? If it has a problem, we should fix it in CBE correct?
<marler8997>
to support bootstrap
<marler8997>
I recal the plan to be that once CBE was working, we would perform a one-time generation to convert the compiler to C, then maintain that C compiler independently
<marler8997>
I think my first statement was confusing, let me reword
<marler8997>
so then why is that the plan?
<marler8997>
right
<marler8997>
right
<marler8997>
right, I'm saying why
<marler8997>
yes
<pixelherodev>
marler8997: the generated compiler would be stage1
<marler8997>
...?
<marler8997>
I asked why we wouldn't just use the CBE as part of codegen, why would we maintain it independently but never got an answer
<marler8997>
pixelherodev, yes I recall the plan being a one-time generation to C, then maintaining both backends independently
<marler8997>
pixelherodev, you said you find CBE-ing stage 2 "kinda repulsive after more thinking about it"...what sort of things did you discover are repulsive about it?
<marler8997>
thanks g-w1
<marler8997>
so knowing the day would be helpful
<marler8997>
yeah I'm clicking through each day looking for "pixelherodev" on the web interface
<marler8997>
not sure how to grep the logs...is there a way to download them?
<marler8997>
which day?
<marler8997>
backlog as in the number of bugs/issues?
<marler8997>
what made you think to make an alternative compiler?
<pixelherodev>
marler8997: not much to say just yet
<marler8997>
pixelherdev, share more
<marler8997>
zig build test-stage2 doesn't just build an executable, it builds and runs tests
2020-12-13
<marler8997>
yeah it's one "obvious" way to do things, which is alot weaker than "one way to do things", adding type constraints to the function signature could be seen as more "obvious" than putting then in the function body. And when the type constrains are more "dynamic", that could be when it's obvious to put them in the body
<marler8997>
yeah I agree on those points
<marler8997>
TheLemonMan, would like to hear your thoughts on the subject, share more?
<marler8997>
But it's a good question to ask of any proposal, "without this proposal, what's the current solution? If there is already a way to do this, why are we adding another way?"
<marler8997>
also keep in mind that "one way to do things" isn't an absolute rule, but more of a goal. Using it as an argument doesn't necessarily mean it justifies rejecting every proposal
<marler8997>
a while loop with a counter or incrementing pointer isn't really equivalent, since a for loop covers both cases simultaneously
<marler8997>
you could make the argument that there is no other equivalent way to semantically write a for loop
<marler8997>
Zig clearly errors on the side of minimalism here, which I think is a better side to error on
<marler8997>
However, I think having multiple ways to do things at some point becomes problematic. My intuition tells me there's probably a balance to be had there
<marler8997>
torque, that's a very interesting proposition, I've thought about it myself and haven't really decided one way or the other
<marler8997>
so do we want to maintain one way to add type constraints, or do we want to make it easier to generate documentation?
<marler8997>
an ideal documentation generator in this case should analyze the function body and add type constraints to the documentation, which would be harder that having then in the signature in the first place
<marler8997>
yes it makes it harder to generate documentation
<marler8997>
that's not an equivlent comparison
<marler8997>
if the type checking code is inside conditions, then you couldn't put it in the signature anyway
<marler8997>
torque, yes that's a good observation, but having it at the top of a function body has nearly the same effect
<marler8997>
if we added support for that in the signature, now we have 2 ways of adding type constraints
<marler8997>
torque, one question I recall andrewrk asking is why putting type constaints in the signature is better than putting them in the body
<marler8997>
but then again, solving this problems helps users of generic code, not the generic code itself, so I take my last point back
<marler8997>
so I suppose verifying types inside function bodies like we have now aligns with Zig's goals
<marler8997>
on the other hand, solving this issue makes "generic code" more tractable, which actually isn't a goal of Zig
<marler8997>
the problem is that you're adding code whose sole purpose is to provide better error messages, that code needs to be maintained just like all other code. So if we can solve this problem without requiring all this extra code then we've removed a whole class of possible bugs by removing the need for this special error message code
<marler8997>
^
<marler8997>
and the anytype(predicate) idea is possible today with a wrapper function
<marler8997>
which isn't a huge problem, but there may be a solution that works just as well that doesn't have that problem
<marler8997>
that solution has the problem that the predicate needs to be kept in sync with the implementation
<marler8997>
maybe a good enough solution would just be to add a compile error "NOTE" about the generic type whenever one is involved
<marler8997>
This is the exact same problem that C++ concepts solves, alot of complexity introduced just to create sane error messages
<marler8997>
basically, if (compile_error) ... if (inside_generic_function) ... if (error_involves_anytype_parameter) report_compile_error_and_generic_type_error
<marler8997>
the only unsolved part about it is that I'm not sure how the compiler could tell if it's a problem with the generic type passed in, or with the code using it...the compiler could report both issues though
<marler8997>
a change to the compiler that can backtrace compilation errors to the signature of generic functions and reason about whether the error could be with the type passed in by the caller
<marler8997>
I have an idea that might solve (or at least help) the problem of not using type constraints but still getting good error messages
2020-12-12
<marler8997>
I don't understand why that argument is needed though
<marler8997>
worked
<marler8997>
yeah I'll try that and if it's correct maybe I'll update the wiki
<marler8997>
example is C:\zig\lib....but if it's zig source shouldn't it be the src directory?
<marler8997>
it says it's the path to the zig source code...so is that supposed to be a subpath to my zig repo somewhere?
<marler8997>
Trying out the new windows build option w/out MSVC, but I'm confused by the --override-lib-dir argument
2020-12-07
<DrDeano>
marler8997 it is for going from a FileNode/DirNode to Node
<marler8997>
DrDeano I'm curious what your use case is for using @fieldParentPtr on a tagged union, do you have code to share?
2020-12-04
<marler8997>
oh yeah I think you're right
<ifreund>
marler8997: as far as I can tell the minutes there are for ZSF meetings not about language spec meetings
<marler8997>
s/Andres/Andrew/ I've done that multiple times now...weird
<marler8997>
I made that example because in order to gauge the proposal's usefulness, the biggest thing Andres needs to be convinced is to 'see the alternative'. So I wrote the alternative so we could see the ugliness
<marler8997>
ikskuh, yeah ok looks like we're on the same page with the issues that proposal is trying to solve
<marler8997>
gonz_ what version of windows do you use?
<marler8997>
gonz_, it builds on latest zig but I haven't finished the Windows implementation...it depends on openssl (to download HTTPS) but I got a tip that on Windows I could use SChannel, since you've asked for it, maybe I'll look into finishing the windows implementation
<gonz_>
marler8997: Does zigup build on latest (or even recent) zig?
<ikskuh>
heya marler8997. yeah, that was kinda my point. writing that by-hand for each interface with 100 times the same code is error prone and hard to maintain
2020-12-03
<marler8997>
dch checkout zigup if you want a tool to manage them