<marler8997>
today, all the allocators have to change to support that, with alloc.zig, you only have to change one function
<marler8997>
because all the allocators re-use the same code to implement the runtime interface, I think this speaks to why a composable interface works so well
<marler8997>
small change
<marler8997>
the only change would be the MemSliceAllocator implementation
<marler8997>
related, but this can work with or without the resizeFn change
<marler8997>
it's orthogonal
<marler8997>
MemSliceAllocator is glue for converting a SliceAllocator to mem.Allocator
<marler8997>
SliceAllocator is glue for a "slice-based" api
<marler8997>
Almost :)
<marler8997>
The runtime allocator interface is still useful, especially if your a function that accepts an allocator
<marler8997>
well like I said this is not a replacement
<marler8997>
Policy-based metaprogramming, now that's something to learn!
<marler8997>
But would seem very weird if you haven't done alot of metaprogramming
<marler8997>
Coming from a decade of D metaprogramming, this is par for the course :)
<marler8997>
True
<marler8997>
self: @This()
<marler8997>
passing blocks back and forth is a different interface than passing slices back and forth
<marler8997>
yes
<marler8997>
interface style?
<marler8997>
The more common reason being that you create an allocator that wraps another allocator, and the wrapping allocator can decide where to store the underlying bloc, making it apart of it's own block or putting it somewhere helse
<marler8997>
for example, maybe you want to use a BlockAllocator directly, you could choose to store the Block on the stack if you want
<marler8997>
the reason for having a custom Block type is so the storage can be decided by the caller
<marler8997>
did you read the updated doc where I explain?
<marler8997>
becaue you don't value code reuse?
<marler8997>
the mmap allocator in heap.zig is 250 lines of complex runaround code, using a composable interface, the MmapAllocator in alloc.zig is probably 10 lines of code with a handful of functions
<marler8997>
fengb: I think you need to take a look at heap.zig, and compare it to alloc.zig
<marler8997>
I should say, your implementation can use as much or as little of the interface
<marler8997>
The fail allocator only implements 1 function
<marler8997>
But your interface can be a simple or as complex as you want
<marler8997>
I find both to be important
<marler8997>
you pay for what you use
<marler8997>
but what's great about this is you can use as much or as little of it as you want
<marler8997>
but rather than having to re-implement the same things over and over, this library allows the same logic to be re-used over and over because everything is composable
<marler8997>
and many implementation had subtle difference and I'm sure I found some bugs
<marler8997>
When I was going through heap.zig, I saw the same functionality implemented over and over again
<marler8997>
We do the best we can, but we also don't need to implement everything
<marler8997>
I'm working through all the use cases and trying to come up with the best interface, I find that using the "single-responsibility" principle is key, but it's still hard
<marler8997>
my interface is much different from him
<marler8997>
I would expect most applications to create an allocator and use it directly, and for functions to continue to use the runtime allocator interface
<marler8997>
and applications can choose whether to use a runtime allocator, or use an allocator directly
<marler8997>
I already have it working with mem.Allocator
<marler8997>
it is
<marler8997>
It would allow us to remove most of the code in "heap.zig" with compile time allocator expressions
<marler8997>
keep in mind, this isn't replacing mem.Allocator
<marler8997>
and if you want alignment, you would do: var callocator = makeSliceAllocator(makeAlignAllocator(CAllocator { }))
<marler8997>
with optimization that should literally compile to raw calls to malloc/free with no runtime overhead
<marler8997>
var callocator = makeSliceAllocator(CAllocator { })
<marler8997>
You'll want to start using a runtime interface if you have alot of different allocators and want to avoid generic/template bloat
<marler8997>
It uses less memory than a runtime interface
<marler8997>
It has no function pointer to track
<marler8997>
It uses less memory than a runtime allocator
<marler8997>
The CAllocator is one of the allocators
<marler8997>
Nope
<marler8997>
but this interface was very different from the interface the application cares about
<marler8997>
how this interface came to be was I started with the slice interface, and kept thinking of things that different allocators would need to know about each other
<marler8997>
it can be any size
<marler8997>
each allocator defines it's own block
<marler8997>
no
<marler8997>
which takes a block allocator, but exposes slices
<marler8997>
Or the SliceAllocator
<marler8997>
You can wrap it in an AlignAllocator, which will track both the underlying Block type, and an extra variable to track the length requested by the caller
<marler8997>
Say you have an allocator that doesn't support alignment
<marler8997>
For example
<marler8997>
The Block type defins what every block allocator needs to track every individual allocation
<marler8997>
"needs" == Block
<marler8997>
if you're block allocator only needs pointers and lengths, then the SliceAllocator becomes a no-op
<marler8997>
actually no
<marler8997>
sorry did that answer your question?
<marler8997>
it's dynamic at compile-time
<marler8997>
What I mean by comptime is that it's a compile-time defined interface
<marler8997>
That's the beauty of met-programming
<marler8997>
There's no overhead because it's all done at compile time
<ikskuh>
marler8997: i still don't see the necessity for two layers of allocation management
<marler8997>
But if you're just an app that wants to use slices, then you don't care about these details
<marler8997>
This is the information you want to know if you have allocators interacting with each other
<marler8997>
Does it need any extra data per allocation, that's the big one
<marler8997>
You can use that level to know what the block allocator does, does it track the length? does it support resize in place? Can it deallocate everything at once? etc
<marler8997>
the "block" level is the low-level view from the allocator's perspective
<ikskuh>
marler8997: i don't understand exactly what you're trying to do with the separation of block- and slice allocators
<marler8997>
instantiating a compile-time allocator is creating a runtime allocator
<marler8997>
i.e. makeSliceAllocator(makeAligneAllocator(makeMmapAllocator()))
<marler8997>
which is exactly what I did in my PR. I implemented the current allocator interface by composing a few block allocators
<marler8997>
I can implement any runtime allocator interface with my compile-time allocator interface under-the-hood
<marler8997>
runtime allocators is a subset of compile-time allocators
<marler8997>
I've taken some of his ideas, but the actualy implementation I've made is pretty different
<marler8997>
the "block allocator" attempts to capture all the operations an allocator can implement, which allows you to implement efficient abstractions that know exactly how the allocator behaves and how to use it properly
<marler8997>
they all have different memory requirement, they all support different operations, they all have different semantics
<marler8997>
the second is that not all allocators need the same thing
<marler8997>
the first big difference is that this is a compile-time interface rather than a runtime functio pointer interface
<marler8997>
yeah that's a good question to ask
<marler8997>
You can compose any number of block allocator, you can wrap them with a logger, wrapper them with an "aligning" allocator, wrap multiple allocators with a fallback
<marler8997>
It allows composability and code reuse
<marler8997>
the only requirement is that it contains a ptr field
<marler8997>
each allocator defines it's own Block type
<marler8997>
right
<marler8997>
it's what an allocator uses to track it's allocations
<marler8997>
I'm working on a generic composable allocator library, potentially for the standard library
2020-04-22
<marler8997>
sorry for the noise, thanks for the tips
<marler8997>
Then a wrapper type won't have to indicate support by conditionally defining member functions, it can just use the boolean interface as well
<marler8997>
Actually I think what I'll do is have the underlying type define a boolean for each function that it implements. That way, it MUST explicitly indicate whether it supports all functions which will prevent errors from typos
<marler8997>
I have struct Foo, and I'm creating a struct Bar that wraps Foo, and uses @hasDecl to check whether it should wrap N functions...
<andrewrk>
marler8997, `pub usingnamespace switch (foo) { a => struct {}, b => struct {} };`
<marler8997>
hmmm, I should have clarified that they were instance functions
<marler8997>
it takes a different code path based on whether or not the struct implements that function
<marler8997>
so it uses @hasDecl(T, "foo")
<marler8997>
the caller needs to know whether or not the function can be called at comptime
<ikskuh>
marler8997: don't use it?
<marler8997>
got a quick question, is there any way to include/exclude a struct function based on a comptime condition?
2019-12-02
<marler8997>
to fix issue 3816 we'll need to encode the correct type for the sentinel value (rather than using `var`)
<marler8997>
or stderr
<marler8997>
But it brought up another issue, that fatal error messages in lld don't get printed to stdout
<marler8997>
I don't know if you say earlier, but I figured out the linker issue
<marler8997>
You're PR fixes tests that my PR adds, so I'll have to wait for it
<marler8997>
I'll wait for this PR to be merged before completing mine
<marler8997>
I see, you've modified it to evaluate each field individually
<marler8997>
I can't just use ir_implicit_cast to convert the whole IrInstruction of TypeInfo to a ZigValue...I need to handle the sentinel field special, does that sound correct?
<marler8997>
so I should try to reach into the IrInstruction for type info for the sentinel value
<marler8997>
right
<marler8997>
should I create a temporary instruction to perform the cast?
<marler8997>
but we also don't have an IrInstruction in order to cast it to the correct type
<marler8997>
so we're left with a ZigValue that's not necessarily the right type
<marler8997>
so when we cast it, it doesn't resolve the sentinel value correctly
<marler8997>
but I don't think we're able to represent the sentinel type correctly
<andrewrk>
btw marler8997 #3787 has some code that messes with that, you might take a peek
<marler8997>
we implicitly cast the comptime const value to a TypeInfo
<marler8997>
when converting the type_info to the type
<marler8997>
got a question for you andrew...
2019-12-01
<marler8997>
problem was in get_const_val, need to check that sentinel pointer is both null and whether the optional value is also null
<marler8997>
yes found it!
<marler8997>
I can use grep/investigate myself, just wondering if anyone has pre-existing knowledge that can help :)
<marler8997>
LLVMSetSection looks promising
<marler8997>
already tried, not there
<marler8997>
yeah been doing that
<marler8997>
sorry, not the zig language, the compiler
<marler8997>
does anyone know if zig has any control over the sections in the elf file, if so, where that would be?
<marler8997>
does zig calculate the size of the object file sections somehow?
<marler8997>
I think when a string literal gets put into a cst section (when it's length is divisible by 4, 8, 16, 32, etc), it's adding 1 to the size of the section for some reason
<marler8997>
I think I know what's going on. I saw a section with 2 entries, one was a string literal and the size was off by 1
<marler8997>
a weird section, contains a bunch of strings that are 16-bytes wide
<marler8997>
entry size is 16 but the full size is not divisible by 16
<marler8997>
actually I mean, ".rodata.cst16" (not cst8)
<marler8997>
it's the ".rodata.cst8" section in the "build.o" object file
<marler8997>
any idea what goes in there?
<marler8997>
the size of the SHF_MERGE section
<marler8997>
so the size of the section header is not divisible by 16, only divisible by 8
<marler8997>
sh_size is 2840 and entSize is 16
<marler8997>
which you can't do, it returns - ESPIPE
<marler8997>
I noticed using strace that LLVM is trying to call lseek on stdout though
<marler8997>
what's also weird is that I don't get an error message
<marler8997>
we were storing the sentinel value as a regular type, my change changed it to an optional type
<marler8997>
yeah, must be something to do with how the sentinel value is translated to llvm?
<andrewrk>
marler8997, I'm surprised that your patch affected the linker in any way
<marler8997>
*InputFiles.cpp
<marler8997>
lld just calls "fatal" which ends up calling _exit
<marler8997>
hitting line 519 of InputFile.cpp with my sentinel change
<marler8997>
took me a while to figure out there is a difference between exit and _exit though
<marler8997>
to figure out what's goin on, I wrote a custom _exit implementation, asserted an error and used gdb to get a stack trace
<marler8997>
yeah makes sense
<marler8997>
it's weird because I'm not getting an error message
<marler8997>
did you know that the lld tool calls _exit on error?
<marler8997>
been having a heck of a time triaging an issue
<daurnimator>
marler8997: no it was moved
<marler8997>
fs.Dir.cwd()
<marler8997>
I hope I'm not being too annoying with these questions, it's definitely helping me go alot faster
<marler8997>
k, I got it
<marler8997>
oh!!
<marler8997>
I assume you're interning more than just these 6 values
<marler8997>
the Intern struct in all_types.hpp
<marler8997>
compiler runtime
<marler8997>
lol
<marler8997>
vs what I'm assuming is runtime interning
<marler8997>
hardcoded interning
<marler8997>
should it be apart of those?
<marler8997>
I think there may be 2 levels of interning, there are 6 special ZigValues that are hardcoded in the "Intern" struct
<marler8997>
you're saing that is correct?
<marler8997>
and the sentinel value is currently not optional
<marler8997>
it's creating a ZigValue
<marler8997>
so in the init_const_str_lit function
<marler8997>
right
<marler8997>
so the 0 will be implicitly converted to a ?u8
<marler8997>
you just said that the field is always an optional type
<marler8997>
what?
<marler8997>
but you just said it's u8 zero, not optional u8 zero?
<marler8997>
so then I'm confused, it's not always supposed to be optional?
<marler8997>
> it's always supposed to be optional
<marler8997>
should I intern an optional u8 zero for string literal sentinel values?
<marler8997>
thanks!
<andrewrk>
marler8997, init_const_str_lit calls get_array_type with g->intern.for_zero_byte()
<marler8997>
think I fixed most of it, having a hard time figuring out where the sentinel value is set for string literals
<marler8997>
that's supposed to implicitly convert I think yes
<marler8997>
after my fix I get this: error: expected type '[*:0]const u8', found '*const [14:0]u8'
<marler8997>
ok I think I gotcha
<andrewrk>
marler8997, if you're looking at the typeinfo related code, who person who contributed it converted the IrInstruction into a ZigValue too early
<marler8997>
of course I could be doing this wrong, trying to cast the sentinel value to it's optional type, but I only have a ZigValue where I"m looking, maybe I need to find out where it's analyzing IrInstruction?
<marler8997>
I see you can use ir_implicit_cast to go from IrInstruction to ZigValue, is there a way to implicit cast one ZigValue to another?
<marler8997>
ok that makes sense
<marler8997>
is it supposed to be dynamic, or should the compiler check the type before it instantiate an instace of TypeInfo.Pointer?
<marler8997>
sometimes it's optional, and sometimes it isn't
<marler8997>
it looks like TypeInfo.Pointer is storing a dynamically typed value
<marler8997>
hey I'm trying to figure out the right way to fix this sentinel issue
<marler8997>
[*]align(N) is a type superset of [*]align(M) where M is a multiple of N
<marler8997>
in that case you just wouldn't use zig's type system to ensure alignment of pages
<marler8997>
thanks that makes sense
<marler8997>
is it just always 0 or something?
<marler8997>
what is alignment supposed to be for array types?
2019-11-28
<marler8997>
which is good
<marler8997>
not as much as other languages
<marler8997>
it does quite a bit of implicit conversion
<marler8997>
I'm surprised it doesn't implicitly convert
<marler8997>
works
<marler8997>
what's the right way to convert an array pointer to a single-value pointer?
<marler8997>
is there a plan to add the sentinel value to TypeInfo?
2019-11-26
<marler8997>
yes, I've been poisoned by the Dark side
<marler8997>
derp
<marler8997>
thank u!
<marler8997>
ooooooo
<marler8997>
--library c (not z)
<marler8997>
I'm just running zig run --library z test.zig
<marler8997>
I'm not specifying -lC
<marler8997>
any idea about the lld error about missing -lC?
<marler8997>
now makes sense why I can't find the exec call :)
<marler8997>
ah I see
<andrewrk>
marler8997: zig doesn't shell out to anything, not cc, not lld
<marler8997>
0.5.0+cb38bd0a1
<marler8997>
master
<mikdusan>
marler8997: old version of zig or master?
<marler8997>
also, when I use --verbose-link, it shows a command with "lld" but it looks like it's actualy running "cc" (when I look at strace, since lld is not a program in my PATH)...is this an error?
<marler8997>
zig run --library c test.zig and zig run --library C test.zig give the same result
<andrewrk>
marler8997, -lc vs -lC
<marler8997>
maybe it's just my environment? does this work for anyone else?
<marler8997>
in any case, I hope LLVM remains as a backend option
<marler8997>
__builtin_add_overflow, that's funny, I just patched a couple tools that were using that
<marler8997>
actually zig could sort of be self-hosted if it added support to compile c++ code as well :)
<marler8997>
would zig's version also be meant as a shared framework like LLVM, or just zig-specific?
<marler8997>
llvm has hundreds of thousands of hours of development time and a large ongoing support effort. it's developed to be shared amongst any language. We'd have to make a much better version to justify losing all that shared effort I think
<daurnimator>
marler8997: but whenever there's something we want that LLVM doesn't have: rather than adding it upstream to LLVM, we are quite happy to implement it ourselves to get closer to the long term goal.
<daurnimator>
marler8997: long term yes; short term no
<mq32>
marler8997: "llvm is an implementation detail"
<marler8997>
zig is trying to get rid of llvm as a dependency?
<mq32>
marler8997: to get rid of LLVM as a dependency in the long term
<marler8997>
andrew why create an assembler? Does LLVM not have good ones?
2019-11-14
<marler8997>
great, is it still the plan to remove varargs? Do we know the timeline for that?
<marler8997>
wow, when did zig implement anonymous structs?!?
2019-10-30
<marler8997_>
and the code inside the "suspend" block is the "resume code"???
<marler8997_>
once a thread is "resumed", I would guess control flow resumes after the suspend block
<marler8997_>
yeah I don't think I understand what "resume" means....
<marler8997_>
hmmm, so then wouldn't suspend; foo(); be equivalent to suspend { foo(); } ???
<andrewrk>
marler8997_, because it denotes where the function will get resumed. consider that as soon as you call epoll_ctl it can be resumed
<marler8997_>
so...what's the purpose of the code being inside the suspend block? isn't it equivalent to just running it before calling "suspend"?
<marler8997_>
ah, loop.waitUntilFdReadable
<marler8997_>
so, when acccept returns EAGAIN...how does the socket get added to epoll?
<marler8997_>
still grokking...
<marler8997_>
oh wait, you have strace
<marler8997_>
is os.accept4 what causes the suspend?
<marler8997_>
a nice example, but I don't understand it since I can't see the full stack of the function calls
<marler8997_>
which means the only way that I"ll learn it is by seeing examples and using it
<marler8997_>
yeah I've even watched your YouTube vides and I still have a hard time...it's just such a foreign concept, it's hard to relate it to anything that currently exists
<andrewrk>
marler8997_, re: async/await stuff, I really should do some more documentation and blog posts on the subject. i'm pretty sure I invented something new here that no other language has ever done, and the semantics are sound, but of course nobody knows how it works because it's not something that exists elsewhere.
<marler8997_>
it seems structs are the answer to everything, namespace, tuples, slices? I guess C had it right all along :)
<marler8997_>
having it defined as a struct means the app will have access to all the info that a struct provides
<marler8997_>
there's been a few times I've needed and/or wanted to the know the internal structure of a slice in D
<marler8997_>
I really need to see more of this async/await stuff in action to wrap my head around it
<marler8997_>
daurnimator: would that allow you to arbitrarily resume a function at any point?
<daurnimator>
marler8997_: `it` could encapsulate a @Frame containing the context...
<andrewrk>
marler8997_, I think that information could be available as an iterator field
<marler8997_>
with a format function, you can use the position inside your function to keep track of what's been printed and what's to come
<marler8997_>
iterator might not work as well, since you'd have to track extra context between each call
<marler8997_>
hmmmm, interesting idea
<marler8997_>
goodness using overlapps...
<marler8997_>
we had to write a library that filtered all networking calls to either the default windows stack, or an external network stack
<marler8997_>
I wrote a "shim" layer to the winsock API
<marler8997_>
if (builtin.os == Windows) std.os.windows.listen(sock, backlog) else if (builtin.os == Linux) std.os.linux.sock.listen(sock, backlog) ... etc
<marler8997_>
but if all platforms use the same API, you shouldn't need platform ifs to use that common api
<marler8997_>
in which case you would need platform ifs
<daurnimator>
marler8997_: note that some platforms allow all sorts of listen filtering
<marler8997_>
all platforms conform to the same API
<daurnimator>
marler8997_: why not? when only some platforms support it....
<marler8997_>
if you want to pass in a backlog, you shouldn't need to start having platform "if"s
<marler8997_>
the point is to make the platform-independent abstraction orthogonal to zig's "flavor" of the api