<FromGitter>
<dscottboggs_gitlab> @stronny at a high level it looks like some worthwhile additions to the current fibers API. I'm kinda confused by your naming conventions but I think I get the gist. ⏎ ⏎ Why not write some tests and docs and try to get it merged?
alexherbo24 has joined #crystal-lang
alexherbo2 has quit [Ping timeout: 260 seconds]
alexherbo24 is now known as alexherbo2
alexherbo27 has joined #crystal-lang
alexherbo2 has quit [Ping timeout: 260 seconds]
alexherbo27 is now known as alexherbo2
<FromGitter>
<smalls12> I have some C bindings that return an integer that represents the number of times I need to loop over something ⏎ I can't seem to find a way to loop over a range
<FromGitter>
<Blacksmoke16> `.each`?
<FromGitter>
<Blacksmoke16> er `num.times { |i| i }`
<FromGitter>
<smalls12> hmm thought I tried that
<FromGitter>
<smalls12> user error I guess
<FromGitter>
<smalls12> I do get this error ⏎ `Error: undefined method 'each' for UInt32`
<FromGitter>
<Blacksmoke16> `.times` sorry
<FromGitter>
<Blacksmoke16> to be clear, a `Range` is also a type, which a singular number is not
<FromGitter>
<Blacksmoke16> so you would want to use `.times` to iterate n times
<FromGitter>
<Blacksmoke16> e.x. `(1..5)` would be a range consisting of the members `1, 2, 3, 4, 5`
<FromGitter>
<smalls12> ok that got rid of the compile error
<FromGitter>
<smalls12> thanks :)
<FromGitter>
<eddloschi> Hi! What shard are you guys using for ORM/ODM? I’m trying to convert a rails app to crystal, but choosing a good crystal ORM/ODM is hard
<FromGitter>
<Blacksmoke16> i use Granite personally, depends on what you're looking for and what you'll be doing with it as each has pros and cons
<FromGitter>
<stronny> @dscottboggs_gitlab at the time of commit I was really tired and just got the app to run and be done with it
<FromGitter>
<stronny> I thought I would give pony a try but didn't get to it yet
<FromGitter>
<stronny> now I'm doing a CLI framework lol
darkstardev13 has quit [Ping timeout: 240 seconds]
<FromGitter>
<smalls12> I can't find any documentation on handling c arrays of structures ⏎ I have a library that will provide a pointer to an array of structs, just trying to figure out how to handle it
<FromGitter>
<smalls12> I can workaround it in C itself, but wondering if there is a crystallized way of doing it
<FromGitter>
<stronny> what's the problem?
<FromGitter>
<smalls12> well basically trying to loop through that array
<FromGitter>
<stronny> could you quote the signature?
<FromGitter>
<smalls12> in C it would be something like *(&array_of_structs[i])
<FromGitter>
<stronny> why do you need a `&`?
<FromGitter>
<smalls12> to get a pointer to the element in the array
<FromGitter>
<stronny> array[n] is the pointer already
<FromGitter>
<stronny> or are the scructs malloced?
<FromGitter>
<stronny> so anyway, to initialize an array in Crystal you need a size
<FromGitter>
<smalls12> ok more info
<FromGitter>
<eddloschi> @Blacksmoke16 Last commit on granite was 2 months ago. While it seems to be a good ORM I was hoping to find something more actively developed
<FromGitter>
<smalls12> the C API takes a double pointer to an array and a pointer to a size variable, the C library will allocate said array and size variable
<FromGitter>
<smalls12> @stronny
<FromGitter>
<Blacksmoke16> :thinking: i suppose?
<FromGitter>
<Blacksmoke16> idt thats a reason to roll it out but ok
<FromGitter>
<stronny> right, so after you call the function you'll have to do make a `Array(YourStruct).new(size)`, copy the elements from the allocated memory and free everything allocated in the function
<FromGitter>
<stronny> otherwise you'll have problems with GC, most probably a memory leak
<FromGitter>
<stronny> what are you trying to express here?
<FromGitter>
<smalls12> express ?
<FromGitter>
<stronny> what do you want the compiler to do?
<FromGitter>
<stronny> is CustomType a class?
<FromGitter>
<smalls12> well I am making some C bindings to this API, but I don't want to just expose double pointers out of the crystal API ⏎ I want to inside my bindings convert that double pointer into a crystal array
<FromGitter>
<smalls12> its a type defined in the C API, but I made a type for it inside crystal as well
<FromGitter>
<stronny> okay, is it a class?
<FromGitter>
<smalls12> type being struct in C API
<FromGitter>
<stronny> so basically C `struct` is Crystal `struct` ⏎ however C `struct*` is Crystal `class`
<FromGitter>
<stronny> you don't usually do `Type*` in Crystal
<FromGitter>
<smalls12> hmm
<FromGitter>
<stronny> in other words `Array(String)` === `char**`
<FromGitter>
<stronny> having in mind the sizes of course
Seich has quit [*.net *.split]
Seich has joined #crystal-lang
<FromGitter>
<smalls12> figured it out using the offset from pointer
<FromGitter>
<smalls12> `Array(CustomType*).new(value_cnt) { |i| value + i }`
<FromGitter>
<stronny> I'd still recommend making CustomType a class
<FromGitter>
<stronny> also you should copy the memory, not just the pointers
<FromGitter>
<smalls12> for what purpose though ?
<FromGitter>
<smalls12> oh too fast, in reference to the first thing you said
<FromGitter>
<stronny> after you free the `value` your array will point to invalid locations
<FromGitter>
<smalls12> no, class over struct
<FromGitter>
<stronny> classes are allocated and freed with GC
<FromGitter>
<stronny> structs live in static memory
<FromGitter>
<smalls12> oh structs arent
<FromGitter>
<stronny> you should never take pointers of structs, that's very dangerous
<FromGitter>
<smalls12> in general, or crystal ?
<FromGitter>
<stronny> and of course you should never call malloc yourself
<FromGitter>
<stronny> in Crystal specifically
<FromGitter>
<stronny> `struct*` in C is everywhere, it's normal
<FromGitter>
<smalls12> yea I'm not allocating any memory, the C library operates on creating memory for you, and it is up to you to free
<FromGitter>
<stronny> and because of that you should take care of the bytes in that allocated memory before you free it
<FromGitter>
<stronny> and you should free it as soon as possible
<FromGitter>
<smalls12> I think based off of the design of the C library, I will keep them as structs in crystal so as you said garbage collection doesn't try and free it ⏎ the C library also proves API calls to free that memory and I will expose those into crystal as well
<FromGitter>
<stronny> bu whatever works for you I guess
<FromGitter>
<smalls12> questionable from a crystal point of view ?
<FromGitter>
<stronny> your users will get memory errors
<FromGitter>
<smalls12> but thts arguably just how C works? ⏎ you have to manage memory yourself
<FromGitter>
<stronny> yes, but Crystal is not C?
<FromGitter>
<stronny> it does have a GC
<FromGitter>
<smalls12> should I choose to make use of it
<FromGitter>
<smalls12> don't get me wrong I see your point
<FromGitter>
<smalls12> I could make the C library behave in a more crystallized way
<FromGitter>
<stronny> as soon as you are working against the GC you are fighting the tool, not using it as intended is all
<FromGitter>
<stronny> but again, you do you
<FromGitter>
<smalls12> I see your point, duplicating the memory into the crystal domain is more user friendly
<FromGitter>
<smalls12> i'll try it out, changing some stuff over to class instead of struct and see where it takes me
<FromGitter>
<smalls12> thanks for the info :)
<FromGitter>
<stronny> I think in the end it would work out better
<FromGitter>
<smalls12> 👍
alexherbo2 has quit [Ping timeout: 256 seconds]
dom96 has quit [*.net *.split]
adam12 has quit [*.net *.split]
adam12 has joined #crystal-lang
dom96 has joined #crystal-lang
flips has quit [*.net *.split]
flips has joined #crystal-lang
ur5us has joined #crystal-lang
<FromGitter>
<dscottboggs_gitlab> > I thought I would give pony a try but didn't get to it yet ⏎ ⏎ @stronny pony has some really interesting concurrency features. I wish we had some of the restrictions it offers, like immutable types.
<FromGitter>
<dscottboggs_gitlab> > after you call the function you'll have to do a Array(YourStruct).new(size), copy the elements from the allocated memory and free everything allocated in the function ⏎ ⏎ No, you want `Slice(YourType).new ptr, size` ⏎ ⏎ > so basically C `struct` is Crystal `struct` ... [https://gitter.im/crystal-lang/crystal?at=5e6e931b69387244e3941181]
<FromGitter>
<dscottboggs_gitlab> > of course you should never call malloc yourself ⏎ ⏎ LibC.malloc, yes. Pointer.malloc is GC tracked though
<FromGitter>
<smalls12> thanks for clarifying
<FromGitter>
<smalls12> why Slice over Array ?
<FromGitter>
<dscottboggs_gitlab> just a sec
<FromGitter>
<smalls12> all good
<FromGitter>
<stronny> idk, Array.new allocates its buffer, while Slice.new doesn't, it just wraps the pointer
<FromGitter>
<stronny> I have to say, I'm not an expert, I may very well be wrong
<FromGitter>
<dscottboggs_gitlab> When dealing with C libraries that return an allocated pointer and offer free, I recommend wrapping that API in a class which allocates in `initialize` and frees in `finalize` -- effectively tracking the pointer with the GC. ⏎ ⏎ For example, if you have the following C header: ⏎ ⏎ ``` code paste, see link``` [https://gitter.im/crystal-lang/crystal?at=5e6e97191f0d285eb27f6a82]
<FromGitter>
<dscottboggs_gitlab> sorry for the wall of text, shit. that got out of hand
<FromGitter>
<dscottboggs_gitlab> > Array.new allocates its buffer, while Slice.new doesn't, it just wraps the pointer ⏎ ⏎ Exactly. There's no reason to allocate a new buffer and copy every entry from one to another. Just tell the GC "hey I've got a pointer here". As far as `Slice` over `Pointer`, you get bounds-checking, assuming you give it the right size
<FromGitter>
<stronny> so what's the benefit of `Pointer(YourStruct)` over `YourClass`?
<FromGitter>
<dscottboggs_gitlab> It's not so much that there's a benefit of one over the other as much as it is that they are two separate, incompatible things, as far as I understand. TBH the whole `class` abstraction in crystal is kinda weird and I'd prefer a system where everything was a struct and you could do like `ref SomeStruct`
<FromGitter>
<stronny> one other thing to consider is this: GC allocates chunks of memory, while raw malloc just gives you bytes wherever
<FromGitter>
<stronny> that may lead to fragmentation
<FromGitter>
<dscottboggs_gitlab> is that bad? also doesn't C's malloc align to memory pages usually?
<FromGitter>
<smalls12> I tried the finalize as you say, but crystal won't call that when a object goes out of scope
<FromGitter>
<stronny> if you request kilobytes and more it's not usually a problem, but many small allocations may spread thin
<FromGitter>
<stronny> finalize is called *eventually*, it's not a ++ destructor
<FromGitter>
<dscottboggs_gitlab> > crystal won't call that when a object goes out of scope ⏎ ⏎ No, sadly Crystal does not have RAII style destructors
<FromGitter>
<dscottboggs_gitlab> @stronny interesting. How do C-like programs usually solve this?
<FromGitter>
<stronny> mostly the don't
<FromGitter>
<dscottboggs_gitlab> hah
<FromGitter>
<smalls12> so not much point then having that in the finalize right?
<FromGitter>
<stronny> some employ custom allocators
<FromGitter>
<stronny> C++ is big on this sa you may imagine
<FromGitter>
<dscottboggs_gitlab> I mean, there's a point in as much as there's a point in having a runtime garbage collector at all
<FromGitter>
<smalls12> ah so if I left the malloc'd memory from c be cleared from the crystal garbage collector
<FromGitter>
<stronny> you may call GC.collect directly
<FromGitter>
<stronny> but usually you don't have to
<FromGitter>
<dscottboggs_gitlab> @stronny I've been told by @asterite not to do that
<FromGitter>
<smalls12> fair enough, I'm employing the other option that @stronny suggested where I turn the C structure into a crystal structure
<FromGitter>
<stronny> if you need a deterministic memory model Crystal doesn't give you that
<FromGitter>
<smalls12> as I want to segregate C completely
<FromGitter>
<dscottboggs_gitlab> @smalls12 I would strongly recommend against exposing a C-style malloc/free memory model in a Crystal shard
<FromGitter>
<smalls12> oh im not, as far as I see it
<FromGitter>
<smalls12> my shard will encapsulate it away
<FromGitter>
<dscottboggs_gitlab> ok good
<FromGitter>
<smalls12> using the C library which allocated structs for me, quckly turning them into crystal structure/classes and then freeing what th elibrary gave me
<FromGitter>
<dscottboggs_gitlab> oof, that'll be expensive (computationally)
<FromGitter>
<smalls12> I would agree, but this isnt something that would be performing operations on millions of data sets
<FromGitter>
<smalls12> I guess at the same time I could leave it the way it is, but its less crystallized
<FromGitter>
<stronny> copying memory is fast
<FromGitter>
<stronny> a single string operation would destroy any gain you'd have from not copying
<FromGitter>
<stronny> I'm exaggerating of course but still, optimize the bottlenecks, not what *looks* slow
<FromGitter>
<dscottboggs_gitlab> fair enough I guess
tdog has quit [Ping timeout: 240 seconds]
_ht has quit [Remote host closed the connection]
ur5us has quit [Ping timeout: 260 seconds]
<FromGitter>
<uuwa> Hey guys, got a inquiry about the `finalize` method. I don't really need it but out of curiosity I wrote a dummy class with a simple `puts` in its `finalize` and made an instance of it. There was no output. I tried several other ways of getting some output to the terminal from `finalize` with no success; for example, I wrote a C library with two functions, `mymalloc` and `myfree`, with `myfree` printing something.
<FromGitter>
... I called `mymalloc` in my class constructor and then `myfree` in `finalize`, again with no result. So I guess I don't understand how this method works... would appreciate some clarification.
<FromGitter>
<stronny> as we've just discussed finalize is called eventually
<FromGitter>
<stronny> you may test it with explicit call to GC.collect
<FromGitter>
<dscottboggs_gitlab> and in most cases it's never called at all
<FromGitter>
<dscottboggs_gitlab> even after calling GC.collect
<FromGitter>
<stronny> by the way you should be very careful with finalize
<FromGitter>
<uuwa> @stronny Oh, sorry, I didn't read the previous messages.
<FromGitter>
<stronny> I would avoid it at all costs
<FromGitter>
<dscottboggs_gitlab> @uuwa nbd
<FromGitter>
<stronny> the requirement for no allocation is rather restrictive in Crystal
<FromGitter>
<stronny> yes, I and would avoid it by not using the heap allocated outside the GC
<FromGitter>
<dscottboggs_gitlab> That doesn't really work if you want to interop with C, especially a library which provides init and free functions like in my example.
<FromGitter>
<dscottboggs_gitlab> sometimes the heap is necessary
<FromGitter>
<stronny> I guess when free is also a part of API and it isn't just literally `free()` then I would concede
<FromGitter>
<stronny> however
<FromGitter>
<stronny> instead of relying on finalize I would do a block with ensure and free there
<FromGitter>
<stronny> smth akin to File.open { |fd| ... }
<FromGitter>
<dscottboggs_gitlab> hm. I think it depends on the use-case.
<FromGitter>
<stronny> of course
<FromGitter>
<dscottboggs_gitlab> I could see a long-lived pointer living inside a class, but for context-bound references that syntax makes sense
<FromGitter>
<stronny> I don't see a usecase doesn't mean there isn't one =)
<FromGitter>
<stronny> another problem here is with aliasing. suppose the pointer gets copied
<FromGitter>
<stronny> all these considerations go away as soon as you delegate *all* memory to GC
<FromGitter>
<stronny> of course one set of problems gets replaced with another, but that's a trade-off many would agree on
<FromGitter>
<dscottboggs_gitlab> idk, TBH I don't have a ton of faith in Crystal's GC -- it rarely runs and often doesn't pick up on unused memory. If I'm concerned about memory usage I do what I can to avoid the heap
<FromGitter>
<stronny> right there with you, but allocation control is a really weak side of Crystal
<FromGitter>
<stronny> there was a bug in LLVM that made compilation time exponentially dependent on StaticArray sizes
<FromGitter>
<dscottboggs_gitlab> yeah. A `finalize` on structs, plus copy and move constructors would go a long way
<FromGitter>
<dscottboggs_gitlab> ouch haha
<FromGitter>
<stronny> like SA(8) is alright, but SA(8196) gets compiled for minutes
<FromGitter>
<stronny> I wouldn't be surprised if that's still the case
<FromGitter>
<stronny> but then again
<FromGitter>
<dscottboggs_gitlab> daaammnn
<FromGitter>
<stronny> it's really only a concern for production daemons, and I wouldn't use Crystal anyway
<FromGitter>
<dscottboggs_gitlab> I would (and do)
<FromGitter>
<stronny> for interactive code heap is fine
<FromGitter>
<stronny> well please accept my sincere condolences =)
<FromGitter>
<dscottboggs_gitlab> hahahaha
<FromGitter>
<dscottboggs_gitlab> nothing I depend on to pay my bills but if my crystal scripts don't work neither does my password manager so it's critical enough 😉
<FromGitter>
<uuwa> I'm not experienced on this, but when you say that the GC "rarely runs", wouldn't that cause memory leaks or whatever other serious problem?
<FromGitter>
<stronny> crystal scripts aren't typical production daemons though
<FromGitter>
<dscottboggs_gitlab> yes
<FromGitter>
<stronny> that's not leaks
<FromGitter>
<stronny> more like temporary overuse
<FromGitter>
<dscottboggs_gitlab> @uuwa, yeah, it's like a temporary leak, then it gets cleared up at runtime *eventually*
<FromGitter>
<dscottboggs_gitlab> usually several minutes later
<FromGitter>
<uuwa> I see
<FromGitter>
<dscottboggs_gitlab> it's the same in any other language with a runtime garbage collector except, maybe a little slower
<FromGitter>
<stronny> jdk has several
<FromGitter>
<dscottboggs_gitlab> slower as in less frequent, not as in slower to run
<FromGitter>
<stronny> golang has another set of priorities
<FromGitter>
<stronny> like most things in life GC is a trade-off
<FromGitter>
<stronny> its presense as well as its logic
<FromGitter>
<stronny> boehm isn't the most advances out there, but I guess it could be swapped for something else
<FromGitter>
<stronny> like nothing in Crystal should be dependent on a particular GC implementation
<FromGitter>
<uuwa> Okay, so it's common in other languages. So that is why I once heard that GC's were unpredictable.
<FromGitter>
<stronny> if you need a deterministic memory Crystal doesn't have it
<FromGitter>
<stronny> most people don't though
<FromGitter>
<stronny> @dscottboggs_gitlab someone has too much free time
<FromGitter>
<stronny> I mean it's impressive, sure
<FromGitter>
<dscottboggs_gitlab> hahaha right? still, I'm looking at taking portions of his/her work and porting it to the stdlib. particularly the GC and LibC
<FromGitter>
<stronny> why libc?
<FromGitter>
<dscottboggs_gitlab> because we currently depend on GLibC or Musl LibC which limits our options when porting to other architectures (such as WASM) and ability to produce static binaries
<FromGitter>
<stronny> static binaries are cancer
<FromGitter>
<stronny> I say link against system libs
<FromGitter>
<stronny> is wasm even posix compatible?
<FromGitter>
<dscottboggs_gitlab> sure, then we've gotta package for dozens of linux distros, a few versions of macos, windows, BSD, etc. instead of just saying "this binary will work on any posix system with XYZ arch"
<FromGitter>
<dscottboggs_gitlab> > is wasm even posix compatible? ⏎ ⏎ I don't think so, why is that relevant?
<FromGitter>
<stronny> yes, someone has to package the binaries. trying to avoid it hurts more than helps
<FromGitter>
<stronny> I'm not sure the same stdlib would be able to play well enough with posix vs. non-posix runtimes