<FromGitter>
<codenoid> google wont say if it can be unlimited @fridgerator
<FromGitter>
<fridgerator> no, it cant be unlimited
<FromGitter>
<fridgerator> but you can raise it
rohitpaulk has joined #crystal-lang
rohitpaulk has quit [Ping timeout: 240 seconds]
snsei has joined #crystal-lang
snsei_ has quit [Ping timeout: 240 seconds]
snsei has quit [Remote host closed the connection]
snsei has joined #crystal-lang
snsei has quit [Remote host closed the connection]
<oprypin>
domgetter, please do not exclude some people from being able to help. specify what you want, not provide an example from Ruby (at least I assume that's what it is)
<FromGitter>
<pglombardo> I’m poking around with the GC and `GC::Stats`. I call `GC.enable` in my test app but GC.stats only seem grow. (Started @ ~4MB now at ~45MB process mem size) `bytes_since_gc` only grows too. So what triggers a GC run?
<FromGitter>
<pglombardo> Besides calling it manually which I haven’t tried yet...
<Groogy>
probably when the heap memory reaches a certai nsize
<Groogy>
and I mean you can still keep references to objects which will keep them alive even if you want them dead
<Groogy>
do you make sure to let go of any references of the memory you allocating when testing?
<FromGitter>
<pglombardo> It’s not that I’m having trouble releasing memory - it seems that the GC never runs even after explicitly enabling it (`GC.enable`).
<FromGitter>
<pglombardo> I’m running 0.23 - just upgrading to 0.23.1 now via brew..
<oprypin>
yeah it's not unusual to have it keep growing until a certain size
<oprypin>
i remember 90MB
<Groogy>
oh GC should be enabled by default
<Groogy>
what oyu want is probably GC.collect to force it to collect
<FromGitter>
<pglombardo> Doh I think I killed the process int he 80s ;-) I’ll try again.
<FromGitter>
<pglombardo> Groovy: Cool - I want to see default behavior rather than forcing a run
<oprypin>
try to make an example with more aggressive memory usage
<oprypin>
rather than waiting
<Groogy>
also keep in mind it might be collecting and reclaiming memory, but usually you want to keep a big buffer of claimed memory from the OS
<Groogy>
for performance
<oprypin>
oh yeah
<Groogy>
so you don't need to do unnessecary kernel calls
<Groogy>
so even if it kills objects and reclaims memory, it will not free the memory
<oprypin>
one thing's for sure, it should stop growing at some point
<Groogy>
if it can it should stop growing yeah
<oprypin>
if it doesn't then that's a problem
<FromGitter>
<pglombardo> The numbers in `GC::stats` is what is a bit confusing. `bytes_since_gc` never reduced/only grew which makes me assume a GC run never happened. Same with `heap_size` - only grows.
<Groogy>
well heap_size is just gonna grow for these small sizes since it's more efficent
<Groogy>
but yeah there's not much documentation so no idea what bytes_since_gc will actually return but I would assume the same as you
<FromGitter>
<pglombardo> Another question while that is running in bg. I poked through Fiber.cr & Thread.cr but there doesn’t seem to be a way to get a list of Fibers in a process (yet). Am I wrong?
<FromGitter>
<pglombardo> I see `@@threads` but not sure how to access
snsei_ has quit [Remote host closed the connection]
<Groogy>
pglombardo I checked with a finalize method in a class, the GC seems to be running immedietly
<Groogy>
but like said before the GC doesn't really release memory because there is no reason to
<Groogy>
but the GC is being triggered and reclaiming memory
<Groogy>
but the GC running doesn't mean memory actually gets free'd
<Groogy>
but if your memory profile just keeps expanding then you are probably keeping references to objects that should be dead
<FromGitter>
<pglombardo> That may be possible which I’ll investigate/confirm. But the issue is that `heap_size`, `bytes_since_gc` and `total_bytes` never go down - only increase in value.
<FromGitter>
<pglombardo> `free_bytes` does reset/grow/reset
<FromGitter>
<pglombardo> Just to clarify a bit better. ⏎ ⏎ > but the GC running doesn't mean memory actually gets free’d ⏎ No but I would expect `bytes_since_gc` and `heap_size` to be affected/reset/reduced. [https://gitter.im/crystal-lang/crystal?at=596ca620c101bc4e3a922f93]
<FromGitter>
<pglombardo> 90MB almost there...
<oprypin>
lol don't be so confident about a number i vaguely remember from a shoddy test i did a year ago
<Groogy>
huh trying to build latest shards git version for me have 6 errors in the spec. Donno if that's a archlinux pacman only thing though
<Groogy>
it seems to fails the git commit tests
<FromGitter>
<pglombardo> Here’s an odd one. Removing the `GC.enable` call fixed it. `bytes_since_gc` correctly resets, `GC.stats` numbers looks sane and after running for ~10 mins the process men size is still ~1.8MB.
<FromGitter>
<pglombardo> Could it be that calling `GC.enable` actually disabled the GC?
<FromGitter>
<pglombardo> If I *do* call `GC.enable` - the process memory usage grows continually.
<FromGitter>
<pglombardo> oprypin: even past 90MB which I figured I would try ;-)
<FromGitter>
<bew> and if you call `GC.enable` twice? will it disable and re-enable it?
<FromGitter>
<pglombardo> Oddly no - calling `GC.enable` 2, 3 or more times, always disables the GC with the same symptoms: `bytes_since_gc` and process mem size always increases.
<FromGitter>
<pglombardo> Sorry was stuck on a call. I’l go ahead a file an issue. I don’t think there is a way I can link to this convo?
snsei has joined #crystal-lang
<Groogy>
¯\(°_o)/¯ I'm not on Gitter so not sure how it works
<FromGitter>
<bew> you can click on the time on the right to get a link to a specific message
<Papierkorb>
Just summarize it, much easier for an issue reader to read that instead of a long chat conversation
<Groogy>
also link to the link Oprypin shared as it should make it obvious that the counter goes below 0 and thus becomes true
<FromGitter>
<pglombardo> Ok/agree - I’ll summarize well and only provide the gitter link for background/extended info
Philpax has quit [Ping timeout: 240 seconds]
<FromGitter>
<crisward> @pglombardo The GC does not return ram to OS. This is not crystal. The GC is a separate library 'libgc' - see https://github.com/ivmai/bdwgc/issues/152
<FromGitter>
<crisward> That's probably the answer then. Not the most intuitive interface.
<Groogy>
No but it's up to the crystal team to decide if users have to know boilerplate quirks like this or if they work around it
<Groogy>
we already know the problem
<FromGitter>
<pglombardo> Background on my situation is that I was porting over Ruby code that collected Ruby GC stats and I assumed the GC.enable was appropriate in Crystal. Not a common case I’m sure.
<FromGitter>
<crisward> It's shouldn't be too tricky to store a local var, @@enabled. Then only run enable if it's false, or disable if it's true.
<Groogy>
or query the GC lib "is the GC enabled?"
<Groogy>
tons of way to work around it since it is wrapped in a Crystal native module already
<Groogy>
that is if they decide they want to
A124 has joined #crystal-lang
<Papierkorb>
Besides, isn't all of this a bug in boehmgc?
<FromGitter>
<crisward> Whoever does it, should chuck a few docs on their, it's a bit light on info.
<FromGitter>
<crisward> Not sure if its a bug or a feature. They may want this voting system to allow processes to collectively decide when to enable the GC. Doesn't look like the sort of thing you do by accident.
<Groogy>
nah but they don't own the GC lib I think, so it might be a quirk of that lib (which is by design) that the team didn't know of and maybe don't want to have
<Groogy>
and it's pretty bad if you like call enable too many times, get no assert or error message and to get it enabled again you have to call disable equal amount of times
<Groogy>
that's a pretty bad bug to discover in your application currently running live
<RX14>
it makes perfect sense to me, if you cann GC.disable twice you need to call GC.enable twice to turn it back on
<RX14>
for the reason stated in the docs: to allow them to nest
<RX14>
simply ensure that GC.disable is called every time you use GC.enable and you're done
rohitpaulk has quit [Ping timeout: 240 seconds]
<RX14>
it would probably be ideal if they clamped the value at 0 though
<RX14>
oh, they kinda do, there's an assertion failure if you enable more times than you disable
<FromGitter>
<bew> RX14 if you're talking about the GC_ASSERT, it's disabled on release
snsei has quit [Remote host closed the connection]
<RX14>
well
<RX14>
i don't think it's that big of a deal
<RX14>
we probably don't want to expose GC.enable and GC.disable
<RX14>
only a GC.disable which takes a block
<Groogy>
Oh that would be way better
<Groogy>
and safer
<Groogy>
"we have performance critical code here which will do tons of small allocations" would be a good usecase for that
snsei has joined #crystal-lang
<Groogy>
actually I know some places where I am gonna need that now that I think of it
mark_66 has quit [Remote host closed the connection]
baweaver is now known as baweaver_away
baweaver_away is now known as baweaver
baweaver is now known as baweaver_away
<FromGitter>
<bew> when the GC is disabled, when re-enabling it, will it be able to collect the allocs that happened when it was off?
<Groogy>
I hope so? It seems to still be tracking the info just not running collection cycles
<FromGitter>
<bew> that's cool
<RX14>
the GC replaces malloc
<RX14>
if you disable the GC it acts like a normal malloc
<RX14>
if you enable it, when you run out of memory in your heap the GC tries to collect memory instead of just expanding the heap like a normal malloc
<Groogy>
and the root doesn't change really so all references should still be known to the GC
<RX14>
the roots are given to the GC when the GC decides to collect
snsei has quit [Remote host closed the connection]
<Groogy>
yeah so would be great to have a GC.disable { create_tons_of_tmp_buffers_for_rendering }
<Groogy>
:D
<RX14>
if you were rendering surely you wouldn't have tmp buffers
<RX14>
you'd reuse them
<Groogy>
well I am hyperboling an example, but for instance in Ruby since everything is heap allocated, every single Vector, Matrix etc. would have a chance to trigger the GC
<RX14>
you can bind old malloc if you want
<Groogy>
Also it was a problem I encountered in C# 7 years ago so we had a whole system to track the GC when it triggered so we could optimize for that
<RX14>
and get a pointer back which you can pass to slice
<Groogy>
well the GC.enable and GC.disable is there already, I can just implement the block variant myself
<oprypin>
Groogy, gotta use `ensure` for that btw
<Groogy>
of course incase of exception
<oprypin>
also i really doubt it's actually needed
<RX14>
^
<Papierkorb>
^
<Groogy>
Well we didn't think it was needed in C# :P
<RX14>
if you're writing a game keep allocations out of the main loop
<RX14>
of course that's pretty hard to do
<RX14>
but it's possible
<oprypin>
RX14, :(
<oprypin>
do i have to
<Groogy>
well it's also you know, underlying things like XNA doing allocations you didn't think of etc.
<RX14>
Groogy, huh
<RX14>
well
<RX14>
i'm just saying binding libc malloc is definitely possible
<Groogy>
or just simply put you being sloppy one day because you forgot to drink coffee
<RX14>
and you'd get a nice crystally slice in return
<Groogy>
Why do that when the normal allocation should be good enough with what's already available?
<RX14>
well libgc is probably a slower malloc than libc malloc
<Groogy>
of course this is with the idea that eventually Crystal will support parallelism so that you can have for instance an input or rendering thread that will have downtime but you will rather build up memory usage than drop responsivness
<RX14>
although thats probably never matters
<RX14>
Groogy, yup
<Groogy>
i.e game logic lagging, is fine. As long as UI etc works fine
<Groogy>
also if you know onbeforehand, let's say you are generating tons of immediary buffers for creating a GI texture from ambient probes then you probably want to tell the GC to wait out during that process
<Papierkorb>
But yeah, if Crystal wants to aspire to more than web stuff, it'll need features that those folks need.
<FromGitter>
<sdogruyol> agreed
<RX14>
well, we don't want to aspire to do everything perfectly :)
<FromGitter>
<sdogruyol> no silver bullet :)
<Groogy>
Yeah no, it is just a "nice thing to have"
rohitpaulk has joined #crystal-lang
<Groogy>
it could actually also be a problem because a thing working in C# and not disabling the GC was I learned how to work with the GC instead of against it
<Groogy>
a tool that would be super awesome would be to see average lifetime per class for it's objects to get a sense of where you might be creating unnessecary objects and so on
<Groogy>
but that's more for Crystal to provide hooks for and then an external profiling tool
<RX14>
well actually it would probably be easiest to do this by instrumenting and logging calls to the GC
<RX14>
which could be done on any binary
<FromGitter>
<sdogruyol> It's not even close but I use instruments on mac
<Groogy>
Oh right Instruments, my only thing I like on Mac :P
<RX14>
lol
<FromGitter>
<sdogruyol> I'd like to have a proper profiling tool too :D
<RX14>
i saw someone made a perf GUI though
<RX14>
@sdogruyol instruments *is* a proper profiling tool
<FromGitter>
<r3bo0t> @sdogruyol sorry for noob question, does crystal-mysql provides connection pool for DB connections , where I can set poo_size to handle high concrurency at my application level?
<Groogy>
think he meant a Crystal specific one
<FromGitter>
<sdogruyol> I meant for Crystal
rohitpaulk has quit [Ping timeout: 240 seconds]
<RX14>
it doesn't have to be for crystal
<FromGitter>
<sdogruyol> @r3bo0t crystal-db provides pool size
<FromGitter>
<bew> Ahah nice, I don't have a list, I prefer to find one for each, by mixing several words related to the project, and make it looks pretty..
<jokke>
hi
<jokke>
i'm trying to figure out how to use the xml parse
<jokke>
r
<jokke>
i have an xml with prefixed node names
<jokke>
querying them with an xpath like "//foo:bar" fails
<jokke>
querying them with xpath("//bar", [{ "foo", nil }]) doesn't work either
<jokke>
the result is always empty
<jokke>
i saw no documentation of how to actually pass the namespaces param so i just guessed from the code
<FromGitter>
<johnjansen> @jokke got some xml to look at?
snsei has quit [Ping timeout: 276 seconds]
<jokke>
yeah no with the correct url it seems to work
<jokke>
but maybe i'm not getting xml namespaces... how can i for example check what namespace uris there are
<jokke>
do i really have to hardcode them in my program?
<FromGitter>
<johnjansen> probably not, but it depends what you are trying to do as to how to do it … are you parsing different kinds of xml files? etc
<jokke>
no
<jokke>
just one
<jokke>
but everything is namespaced
<Papierkorb>
You indeed hardcode the namespace URI, which you use to look up their namespace prefix, and that you use to find whatever you need
<Papierkorb>
The uri is the globally unique identifier which will never change
<oprypin>
Groogy, likely so, now that i think about it
<oprypin>
there is no advantage to splitting something in many parts
<oprypin>
actually try to enumerate the advantages. 1) ... uhhh ... wait
<Groogy>
yeah it started with like "Oh it's gonna be nice for anyone to just put dependency on whatever they want"
<Groogy>
then I realized, holy shit interdependencies are gonna be a bitch
<Groogy>
like just gonna add my own serialization classes which gives me a good seperation between a parser and the actual code that serializes (so I can switch between JSON, YAML etc whatever I need without changing the code)
<Groogy>
and the vectors need to be serializable
<Groogy>
and there starts the pain
fazibear has joined #crystal-lang
pwned has quit [Ping timeout: 248 seconds]
<Groogy>
there's nothing like private require to hide symbols from the global namespace? i.e thirdparty modules/classes etc.
<Groogy>
only make it available in the required file
<oprypin>
i dont think there is.
<oprypin>
modules serve for the hiding
<Groogy>
would be nice so bunch of OpenGL methods don't pollute the global namespace :/
<oprypin>
Groogy, but they're supposed to be in a lib or module
<Groogy>
they are
<oprypin>
good then
<Groogy>
but don't want them to be available just because you require "my_framework"
<oprypin>
don't you.. like... come from a c++ background
<FromGitter>
<konovod> there was a discussion about private requires, final response from core team was "please stop asking, maybe will be added in future".
<FromGitter>
<konovod> but you can use something like MyFramework::Private::...
<oprypin>
oh yeah, i forgot about that
<Groogy>
haven't written the bindings myself, though will probably do that later
<oprypin>
bindings to what?
<Groogy>
opengl
<oprypin>
but they exist
<Groogy>
I know
<Groogy>
have them as dependencies right now
fazibear has quit [Ping timeout: 240 seconds]
rohitpaulk has quit [Remote host closed the connection]
<Groogy>
if I want to set a shard dependency to always fetch HEAD, how do I set that?
<oprypin>
Groogy, you mean master? branch: master
<Groogy>
no to always stay up to date and not lock itself to a version with shards.lock
<Groogy>
there's a really nice cppcon video where a guy creates coroutines in C using LLVM coroutines
fazibear has joined #crystal-lang
<oprypin>
how to implement goto in crystal?
snsei has quit [Remote host closed the connection]
<Papierkorb>
inline assembly
<oprypin>
ooooooooooooooooooooooooohhhhhhhhh
<oprypin>
no but there was soem trick with loop and break
<FromGitter>
<straight-shoota> 😄
<Papierkorb>
while true; break; end
<FromGitter>
<johnjansen> wait @papierkorb .. lets see an example of thst
<FromGitter>
<johnjansen> that
<FromGitter>
<johnjansen> cant even type im so excited to see it ;-)
<Papierkorb>
about what? `asm("JMP foo")` ?
<Papierkorb>
Note that it's not "inline llvm ir", it's "inline assembler", the good old platform-specific one
<oprypin>
D:
<oprypin>
i'm determined that crystal would be so much better with all iterators implemented as coroutines
<Papierkorb>
And basically, a yielding method is a coroutine
<oprypin>
no
<Papierkorb>
> LLVM coroutines are functions that have one or more suspend points. When a suspend point is reached, the execution of a coroutine is suspended and control is returned back to its caller. A suspended coroutine can be resumed to continue execution from the last suspend point or it can be destroyed.
<oprypin>
you can save a coroutine and pass it around
<oprypin>
yielding method is just basic inlining
<Groogy>
you can do the same with blocks?
<oprypin>
no
<oprypin>
yielding method is blocks
<Groogy>
yes if you capture the block
<oprypin>
what i mean
<Groogy>
or a Proc
<Papierkorb>
JS ES6 does it too, they call it generators
<Yxhuvud>
I'd assume coroutines would create too much overhead for iterators. I'm more thinking them a tool for implementing worker threads.
<Papierkorb>
oprypin: as I said :P
<oprypin>
Yxhuvud, if implemented properly, coroutines would do the exact same operations as current iterators
<oprypin>
Papierkorb, said what
<Papierkorb>
> inline assebmler
<jokke>
Muss man beim Zoll eigentlich bar bezahlen?
<Papierkorb>
Nö meine nicht
<Papierkorb>
(lol)
<jokke>
Oh sorry, wrong channel :D
<Groogy>
well actually oprypin
<Groogy>
isn't Fibers literally coroutines?
<Yxhuvud>
groogy: the *whole* point of shards.lock is to create reproduceability. You don't get that if you don't lock versions.
<oprypin>
Groogy, no
<Groogy>
Yxhuvud though it's for something in development so would be nice to skip the lock
<oprypin>
Groogy, well... actually they are probably under some definition of coroutines but not what i want
<Groogy>
What specifically is it you want?
<oprypin>
and not what ES6, C#, Python have
<Papierkorb>
Groogy: Basically, oprypin wants to be able to write a yielding iterator method, which can be auto-turned into an iterator. Because easier to write.
<Groogy>
Fibers remember what point they were at and continues from there and also yields
<oprypin>
i want python but compiled
<oprypin>
just like you guys want ruby but compiled LOL
<Groogy>
There is a compiled version of python I believe?
<Groogy>
or well compiled as in Java/#C compiled
<Groogy>
C#*
<oprypin>
no
<oprypin>
they compile certain parts but it's still inefficient because it includes tons of runtime operations
<Papierkorb>
Compile to their VMs bytecode, you mean?
<oprypin>
instead of the interpreter reading the bytecode and deciding to do some inefficient action, the action is compiled into the code. with minimal obvious optimizations
<Papierkorb>
Ruby is getting there too, iirc, there are gems at this point using Ruby 2.3 features doign exactly that
<oprypin>
i dont think that's really the same
<Groogy>
I mean the Java VM is pretty much like the LLVM but JIT instead of precompiled
<Groogy>
There is also the Python made on the V8 engine I think
<Papierkorb>
... erm, LLVM can JIT too
<Groogy>
I know but that's not what we are using it for here
<Groogy>
you also have the... eh Piston?
<Groogy>
don't remember it was called
<oprypin>
that's just an unfinished supposedly more efficient interpreter
<Groogy>
also I need to figure out how the hell this magic works :D
<Groogy>
Structs are on the stack, but still this magically somehow makes it know the correct type of the struct?
<Papierkorb>
mh?
<Groogy>
Coming from C++ it just looks like pure black magic
<Papierkorb>
What line are you referring to?
<Groogy>
since that would just crop the data to a Configuration still and not a AppConfiguration when you would return it from the method
<Papierkorb>
Also, useless `()`
<Groogy>
p @configuration
<Groogy>
yeah just used to so adds them by reflex
<Papierkorb>
I once came from C++ to Ruby, and then to this lang here. Actually, it's not all that magic at all: Type deduction.
<Groogy>
usualyl cleans them out afterwards
<Papierkorb>
Groogy: Also, don't make everything a struct for the sake of it. Configuration has no reason for it to be a struct
<Groogy>
it's just pure data and no functionality in the configuration so wouldn't really benefit from being made a class
<Groogy>
the query to check what the host computer supports that builds an initial configuration if no file exists is a class though of course
<Groogy>
don't worry there's plenty of classes :P just minimalistic things I make structs
fazibear has joined #crystal-lang
<Papierkorb>
structs are not there for minimalism, but for retaining value semantics. By default, go with class. It'll save you tons of trouble on the long run
<Groogy>
well if configuration is by value you can't do "config.graphics.resolution = #new resolution" thinking that will cause everything to update accordingly
<Groogy>
eh add a app. at the beginning of that
<Papierkorb>
You can't do that in any case if there's no writer
<Papierkorb>
*setter, in crystal lingo
<Groogy>
then I need to find some clever way to write to it from the backend
<Groogy>
since it needs to query what's your monitors supported resolution, bitrate etc
<Groogy>
but should be doable
<Papierkorb>
#initialize arguments if done manually, or the auto-generated #initialize if using e.g. YAML.mapping
snsei has joined #crystal-lang
fazibear has quit [Ping timeout: 240 seconds]
<Papierkorb>
As all arguments can be 'set' by name in Crystal, writing it like `Config.new(foo: 1, bar: 2, ...)` makes it not-so-bad afterall, compared to other languages where you can't do it like that (awful Java builder-pattern *shudders*)
<Groogy>
could probably just define it in the backend like GLFWBackend::Configuration < GraphicsConfiguration which would let the backend create it and touch the protected parts
<Papierkorb>
You get readability and inmutability without any further code
snsei has quit [Ping timeout: 258 seconds]
<Papierkorb>
Oh, and the getter macro (and other macros of its kind) don't require you to pass in a symbol or string. `getter foo` works fine, and has the benefit that you can easily extend it with a type to be clearer, right away or later on after figuring out the API: `getter foo : Bar`
<Papierkorb>
Same goes for delegate: `delegate foo, to: @bar` works fine. This doesn't work for non-alphanumeric methods though: `delegate foo, :[], to: @bar` if you want to delegate [], []=, or another operator method
<oprypin>
Papierkorb, regarding "useless `()`" - no, it clearly shows that this here is a function.
<Papierkorb>
As does the leading `def`
<oprypin>
but it's just a word
<Papierkorb>
and the indented body of stuff after the line
<Papierkorb>
And () are just two chars
<oprypin>
that stand out
<Papierkorb>
Not to me.
<Groogy>
I am more on Papierkorb here
<oprypin>
everyone is, formatter enforces it
<oprypin>
doesnt mean it's irrevocably correct though
<Groogy>
anyway bedtime
<Groogy>
gnight!
crack08 has quit [Quit: My iMac has gone to sleep. ZZZzzz…]