qard has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
ephemera__ has joined #crystal-lang
bmcginty_ has joined #crystal-lang
bazaar_ has joined #crystal-lang
foca_ has joined #crystal-lang
RX14- has joined #crystal-lang
johndescs_ has joined #crystal-lang
johndescs has quit [*.net *.split]
foca has quit [*.net *.split]
bazaar has quit [*.net *.split]
bmcginty has quit [*.net *.split]
ephemera_ has quit [*.net *.split]
RX14 has quit [*.net *.split]
johndescs_ is now known as johndescs
foca_ is now known as foca
<FromGitter>
<CaDs> is there any recommended way to deal with very long strings?
<FromGitter>
<CaDs> i understand that loading the whole string on memory is not a good option, but I'm trying to write some code for finding sequences within DNA strings and those strings usually are rather long
<FromGitter>
<braiden-vasco> Hello. I'm new to Crystal. Which package manager do exist?
<FromGitter>
<braiden-vasco> @CaDs Depends on your needs. What you want to do with this data? Display to user, analyze?
<FromGitter>
<CaDs> @braiden-vasco analyze. Basically search for sub-patterns within very long texts
<FromGitter>
<braiden-vasco> @CaDs Independently from where you store your strings, in memory on in files, you need some kind of index to search. First look at this article: https://en.wikipedia.org/wiki/String_searching_algorithm
duane has joined #crystal-lang
DTZUZO has joined #crystal-lang
duane has quit [Ping timeout: 246 seconds]
return0e has joined #crystal-lang
johndescs_ has joined #crystal-lang
johndescs has quit [Ping timeout: 255 seconds]
johndescs_ is now known as johndescs
<FromGitter>
<CaDs> @braiden-vasco yes, I'm aware of that. Currently I'm using an algorithm for parsing those sequences. I was doing some benchmarking and the same algorithm in Ruby is taking 50seconds to process a baseline file, while Crystal version is taking 2.5 minutes. So I'm sure I'm must be doing something wrong, so I wanted to know if there is any special recommended way to deal with long strings in Crystal
<FromGitter>
<braiden-vasco> @CaDs Can I look at Ruby and Crystal code?
<FromGitter>
<bararchy> @CaDs that's interesting , what are you searching for in the genes ?
return0e has joined #crystal-lang
txdv has joined #crystal-lang
<FromGitter>
<CaDs> @bararchy I'm learning about bioinformatics, my goal is to build a tool for searching sequence motifs (https://en.wikipedia.org/wiki/Sequence_motif) and I wanted to take advantage of the raw speed of Crystal for that.
<FromGitter>
<bararchy> Cool , maybe @ArtLinkov can help you, he has masters in Biology and does all of our in-house algorithms
<FromGitter>
<CaDs> hehe yes, I'm following NeuraLegion amazing work closely :D
<FromGitter>
<bararchy> Oh thanks :)
qard has joined #crystal-lang
<FromGitter>
<bararchy> Just off the top of my head, I know we managed to improve speed when following the advice from RX14 which was "minimize intermediate allocations" so if you got an allocation (via select, map, etc.. ) of new objects which you don't really need try and figure a way to do the same without them, for us using `each` and `each_with_index` was very efficient
<FromGitter>
<CaDs> Thanks! I will try that. Current version is just a translation of the pseudocode algorithm to ruby | crystal. I optimize the code following your advice and also the tips listed in this guide https://crystal-lang.org/docs/guides/performance.html
qard has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<FromGitter>
<bararchy> @CaDs sounds like a good plan, let me know if you managed to improve speed
<FromGitter>
<ezrast> @CaDs You have a recursive algorithm that reallocates most of the string on each pass. https://gitlab.com/snippets/1708095
<FromGitter>
<ezrast> Ruby's so mature it probably has a bunch of optimizations for that kind of thing under the hood
<FromGitter>
<ezrast> unless I'm reading your code wrong and pattern_to_number isn't where it's spending most of its time
<FromGitter>
<CaDs> @ezrast you are right, it is the most expensive operation replacing the recursive logic will make the trick. Is still interesting though the way Ruby handles it. Thanks a lot!
<FromGitter>
<bew> and I'm sure it can go even faster with other optimizations :)
<FromGitter>
<bararchy> Would you consider tweeting this? Its really cool
DTZUZO has quit [Quit: WeeChat 2.0]
<FromGitter>
<CaDs> hehe, will do
<FromGitter>
<ezrast> Tested the slower algorithm in Ruby; it's executing in linear-ish time rather than quadratic
<FromGitter>
<ezrast> It must be reusing memory for `prefix = pattern[0..length-2]` where Crystal is doing a full reallocation
<FromGitter>
<straight-shoota> @CaDs you could consider replacing String with a more performance data structure. You don't need to handle UTF8 characters, but accessing string indices
<FromGitter>
<straight-shoota> like `genome[(i-1)..i-2+k]` loops through the string until the given number of characters has been processed.
<FromGitter>
<straight-shoota> You could use `String.byte_slice`
<FromGitter>
<straight-shoota> Or just use a slice directly
<FromGitter>
<straight-shoota> Instead of string
<FromGitter>
<bew> yeah, less copy and string alloc' with slices
<FromGitter>
<straight-shoota> The data is the same all along, you just need to shift the window to look at it. But there is no sense in allocating pieces of the same data over and over again.
<FromGitter>
<CaDs> Yes, thats why I was asking about how to deal with long strings, because I was totally sure that dealing with such a big string allocation was a waste.
<FromGitter>
<CaDs> I will check `String.byte_slice` sounds like something I definitely could use. Thanks!
flaviodesousa has quit [Remote host closed the connection]
DTZUZO has joined #crystal-lang
Groogy_ is now known as Groogy
<Groogy>
Morning! o/
olbat__ is now known as olbat
olbat has quit [Changing host]
olbat has joined #crystal-lang
flaviodesousa has joined #crystal-lang
<FromGitter>
<girng> morning
<FromGitter>
<girng> hello, i have a question about the db module inside spawn
<FromGitter>
<Qwerp-Derp> I finally finished all of the UI bindings for Icosahedron!
<FromGitter>
<Qwerp-Derp> If all the parts of Icosahedron are completely different from the original libui.cr binding, can I claim the entire library as my own?
<FromGitter>
<girng> ``` code paste, see link``` ⏎ ⏎ so.. if i call a `db.query_all` method inside my `handle_new_new_tcp_connection` will it take advantage of being inside a fiber or block the thread?? [https://gitter.im/crystal-lang/crystal?at=5ac5d5792b9dfdbc3a5a0edf]
<FromGitter>
<straight-shoota> that's the
<FromGitter>
<straight-shoota> Point of using fibers... so it w block other fibers
<FromGitter>
<straight-shoota> *won't
<FromGitter>
<girng> so it's kinda of like async code?
davic has quit [Ping timeout: 264 seconds]
marius has quit [Quit: Ping timeout (120 seconds)]
OvermindDL1 has quit [Read error: Connection reset by peer]
OvermindDL1 has joined #crystal-lang
fifr[m] has quit [Ping timeout: 248 seconds]
salvor has quit [Ping timeout: 264 seconds]
badeball has quit [Ping timeout: 264 seconds]
marius has joined #crystal-lang
badeball has joined #crystal-lang
davic has joined #crystal-lang
salvor has joined #crystal-lang
braidn[m] has quit [Ping timeout: 245 seconds]
byteflame has quit [Ping timeout: 245 seconds]
marius has quit [Ping timeout: 276 seconds]
Yxhuvud has quit [Ping timeout: 276 seconds]
Yxhvd has joined #crystal-lang
betofloresbaca[m has quit [Ping timeout: 260 seconds]
marius has joined #crystal-lang
byteflame has joined #crystal-lang
betofloresbaca[m has joined #crystal-lang
braidn[m] has joined #crystal-lang
<crystal-gh>
[crystal] ezrast opened pull request #5921: New logging system (addresses #5874) (master...logger) https://git.io/vxDdX
<jokke>
the dir that breaks it is: DEBUG: /home/jokke/.java/.userPrefs/_!(k![@"k!'`!~!"p!(@!bw"y!#4![!"v!':!d@"t!'`!bg"0!&@!e@"w!'`!ew"0!(k!c!"l!&:!d!"y!'k!bg"n!$0!,w"h!(!!c!"s!'k!}w"h!(@!a@"v!'4!.@"5!'}!a@"s!'`!cw!n!(0=/
<jokke>
...
<jokke>
w... t... f... java
<jokke>
can i somehow escape it?
<f1refly>
disregard my question, ive found an ugly way to work around it
<Groogy>
Firefly you are calling methods
<Groogy>
the compiler doesn't know if the second time you call it if it is going to be nil
<Groogy>
the method might unset it, or whatever so
<Groogy>
you have to fetch the value out in the query
<Groogy>
if desc = ticket.description; do stuff; end
<Groogy>
is what you have to do
<f1refly>
another thing that bugged me for a while now: is there a way to change the standard license crystal uses when initializing projects?
<f1refly>
Groogy: yeah, i figured that. i still forget to think in fibers sometimes
<FromGitter>
<yxhuvud> firefly: Uh, just change hte licence text etc? Or are you suggesting to change the default?
<f1refly>
yxhuvud: changing my personal defaults. id personally never release anything ever with the MIT license, so the first thing i do after creating a project is changing the license to gplv3
<f1refly>
it'd be great to just change a setting somewhere. I think i'll open an issue on github if this functionality doesn't exist
<FromGitter>
<straight-shoota> f1refly, this has nothing to do with fibers
alex`` has quit [Quit: WeeChat 2.1]
<FromGitter>
<straight-shoota> you can't assume a method returns the same value twice
<FromGitter>
<straight-shoota> even without any concurrency
<FromGitter>
<straight-shoota> jokke could you write the breaking path in a way it is not intepreted as markdown in gitter? just wrap in in a couple of backticks should do
<f1refly>
straight-shoota: i can as a human, since this is a getter without anything fancy to it. I see of course why crystal has a problem with it
<FromGitter>
<straight-shoota> what if a subclass overwrites the method? ;)
tilpner has quit [Quit: :wq]
<FromGitter>
<straight-shoota> jokke, in master there is an improved `Dir.glob` which does exactly what you're implementing yourself. see #5179
<crystal-gh>
[crystal] ysbaddaden closed pull request #5896: Refactor out variable name (master...refactor-out-as-variable-name) https://git.io/vx6vB
<f1refly>
im not sure if this answer is acceptable to you, but anyways: you can use the 'stat' command to view the device id a file is stored on
<crystal-gh>
[crystal] ysbaddaden closed pull request #3144: run standard library specs in release mode on travis (master...release_specs) https://git.io/v60o4
<FromGitter>
<straight-shoota> numbers are never stored as strings
<f1refly>
right, crystal uses the gmp for this. i forgot.
<FromGitter>
<straight-shoota> All primitive number types store their value in as many bytes as it says in the name. If that size is smaller or equal to the word size of the target system, I believe it is unlikely there would be a noticeable performance impact.
<FromGitter>
<jburky> I’m attempting to use the HTTP::WebSocket to connect to an endpoint (dev environment) that is secured using a self-signed cert. Naturally, it rejects this cert because it can’t be verified. Is there a way to disable the verification in my development environment? I have been poking around OpenSSL::SSL::VerifyMode to see if there is a global context that is used by HTTP::WebSocket so I can set the value to
<FromGitter>
... “NONE”. General guidance would be much appreciated.
<crystal-gh>
[crystal] RX14 closed pull request #5810: Ensure cleanup tempfile after some specs (master...ensure-cleanup-after-some-specs) https://git.io/vxePT
<wuehlmaus>
anybody here involved in icr? i found something that looks like a bug: "hello".match( /(..)/ ); $1 . Now the $ is eaten and it shows only 1, funny.
<wuehlmaus>
the $1 content is correctly shown.
shalmezad has quit [Quit: This computer has gone to sleep]
Jenz has joined #crystal-lang
qard has joined #crystal-lang
<FromGitter>
<bew> So it works correctly but displays incorrectly?
<wuehlmaus>
it works correctly, yes.
<wuehlmaus>
i did: puts $1
<wuehlmaus>
and then it displayed puts 1
<wuehlmaus>
:)
<FromGitter>
<bew> Ok, basically icr re-print the input to add syntax highlighting, maybe they didn't implement it for the global vars as it's not common
<FromGitter>
<bew> You can open an issue about that on icr's repo
That_Guy_Anon has joined #crystal-lang
<FromGitter>
<straight-shoota> @jburky you could try adding `tls: OpenSSL::SSL::Context::Client.insecure` argument to `WebSocket.new`
<FromGitter>
<jburky> @straight-shoota, thank you very much!!!!
<Jenz>
Hey, I really want to help out with crystal, but as those who've seen me around probably have noticed, im quite a noob; but I still really want to help out. Does anyone know how a noob like me can help out?
<crystal-gh>
[crystal] straight-shoota opened pull request #5925: Move *io* as first argument of method overloads (master...jm/fix/standardize-io-argument) https://git.io/vxyPh
<FromGitter>
<straight-shoota> Jenz, everyone can help. It's especially great if somebody contributes who is not that familiar with the language.
<Jenz>
But the thing is, I dunno what to do?
<Jenz>
the community:newcomer tagged issues look way too boring ...
<FromGitter>
<straight-shoota> Great would be improvements to the documentation (API docs and language reference). Most people who already know how things work, won't even notice that important things are not properly explained
<Jenz>
I might be able to help out with some documentation yes
<FromGitter>
<straight-shoota> there are probably a lot more issues that would be suitable for newcomers but are not tagged
flaviodesousa has quit [Ping timeout: 265 seconds]
<Yxhvd>
Yes, there is especially a lot of things that may seem totally obvious to us with ruby background that can lack documentation
<Jenz>
Do you peeps with ruby background go back to ruby?
heaven31415 has joined #crystal-lang
<Yxhvd>
there is not a lot of jobs available for people knowing crystal yet, you know
<Jenz>
Oh, yeah, I forgot about jobs
<heaven31415>
of course
<Jenz>
Im just a high-schooler programming in my free time
<heaven31415>
everybody here is for lucrative job as crystal dev
<heaven31415>
in my country it is pretty difficult to find any ruby job
<heaven31415>
it isn't very popular
<jokke>
but whenever i need to implement some kind of microservice i argue it should be done in crystal :P
<jokke>
well if it isn't popular it should be easy to find one.
<jokke>
at least here it's super hard to find ruby devs
<heaven31415>
I have seen ruby jobs offers
<heaven31415>
but only for seniors
<jokke>
pf apply anyway
<heaven31415>
but you have a point
<jokke>
they're happy to find anyone
<FromGitter>
<alex-lairan> heaven where do you live?
<heaven31415>
a lovely country named Poland
<FromGitter>
<alex-lairan> many recruiters call me for ruby jobs in Wroclaw ^^" ⏎ ⏎ i'm in france, Paris
<jokke>
heaven31415: oh! so germany isn't far away! come work here!
p0p0pr37 has quit [Ping timeout: 256 seconds]
<heaven31415>
I was thinking about it jokke, however I really don't know German
<jokke>
no problem
<heaven31415>
I live very close to the border
<jokke>
we have many here who don't speak german
<jokke>
latest addition from hungary
<jokke>
so not really a problem :)
<jokke>
frankfurt is super expensive to live in though
<heaven31415>
where do you work?
<heaven31415>
ohh, frankfurt
<jokke>
yeah
<heaven31415>
that is very, very far away from home :D
<jokke>
not thaaaat far awasy
<jokke>
*away
<jokke>
i'm originally from finland ;)
<heaven31415>
why did you move from finland :P?
<jokke>
dunno, grew up there, grew tired of dark winters...
<jokke>
(they're dark here too) :D
<jokke>
so no improvement on that front
<jokke>
my dad's half german so i knew the language and it was an easy choice to make
<heaven31415>
ohh, my friend had very similar issue
<heaven31415>
so I think I can understand what you mean :P
Jenz has quit [Ping timeout: 264 seconds]
<FromGitter>
<girng> hello friends!!
<FromGitter>
<girng> on a string, if you do `bytesize`, what kind of int does it return?
<FromGitter>
<girng> signed or unsigned int?
<FromGitter>
<straight-shoota> signed
<FromGitter>
<straight-shoota> I don't think any method returns an unsigned integer unless explicitly converted
<OvermindDL1>
Something called 'bytesize' returns a 'signed' integer? o.O
<OvermindDL1>
When would you ever get a negative value?!
<FromGitter>
<girng> thank you, i was just curious cuz in godot there has a method StreamPeer.get_u32()
<FromGitter>
<girng> but i need to use get_32(), because i think that's more appropiate when reading TCP stream
<FromGitter>
<girng> not sure if there was a performance issue there, but im pretty sure i shouldn't be reading an unsigned int when crystal is sending a signed one (however it did work)
<FromGitter>
<girng> wait i got those confused. i should be using unsigned right
<FromGitter>
<girng> since its not negative...
That_Guy_Anon has quit [Quit: Leaving]
<FromGitter>
<straight-shoota> OvermindDL1, sure, the possible value range of a byte size is 0+ but that has nothing to do with the data type
<OvermindDL1>
Shouldn't the return type of the function represent the complete representable range of returnable values rather than half of the representable range of the return type never being returned by the function?
qard has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
qard has joined #crystal-lang
<FromGitter>
<straight-shoota> so a UInt32 could hold the complete representable range of a string byte size?
<FromGitter>
<girng> @straight-shoota can i ask u another question plz
<FromGitter>
<straight-shoota> sure, girng
<FromGitter>
<straight-shoota> OvermindDL1, you'll inevitable run into big problems when using unsigned integers in calculations
<FromGitter>
<straight-shoota> that's why they should be avoided unless there is no other way
<FromGitter>
<bew> @straight-shoota, what?
<FromGitter>
<straight-shoota> default integer value in Crystal is Int32 and all methods returning a value that can be reasonably used in calculations return a signed integer
<FromGitter>
<bew> What problems
<OvermindDL1>
UInt32 seems a bit small to represent a string byte size, it should be whatever the platform native word size is
<OvermindDL1>
Or rather the stride size
<heaven31415>
what is a stride sir?
<OvermindDL1>
straight-shoota: Problems being? signed and unsigned integer operations generate the same assembly except for division
<OvermindDL1>
heaven31415: A stride is the width of a representable memory byte range in the CPU cache
<OvermindDL1>
Usually it's the same as the word size (like on x86 systems), but a lot of system types have different sized variants
alex`` has joined #crystal-lang
<FromGitter>
<girng> @straight-shoota when i first started to choose a language for my game server i saw Elixir and thought it would be cooler because it's multithreaded and uses all cpu cores of a server. however, im not familiar with functional programming languages i didn't like the syntax, and i felt crystal was much more easier for me so i chose it. (and i really like it) ⏎ ⏎ with that said, is there something similar to
<FromGitter>
... use all the cpu cores/multithread your application with 1 crystal instance? or, do we need to run multiple crystal instances and act with a in-memory database similar to Redis for shared memory? or, can we s hare memory across multiple crystal instances? what's the best way to utilize all the cores / threads on a server with o ... [https://gitter.im/crystal-lang/crystal?at=5ac659a0270d7d370897c1f7]
<OvermindDL1>
girng: Have to use multiple instances
<OvermindDL1>
Speaking of (as I'm still learning crystal), how do you handle overflow semantics in integral operations? Like how do you do an addition that wraps compared to how do you do an addition that caps?
<FromGitter>
<straight-shoota> issues are not in codegen but type system
<FromGitter>
<straight-shoota> what do you think would be the result of `10_u32 * -1`?
<FromGitter>
<girng> @overmind, thanks i guess it's similar to how nodejs would interact with Redis
<FromGitter>
<girng> with the PM2 manager, running multiple instances
<OvermindDL1>
straight-shoota: I'd expect that to not compile because wtf...
<OvermindDL1>
Yeah the fact it compiles at all is questionable
<FromGitter>
<straight-shoota> why would it not compile?
<OvermindDL1>
Because it is ambiguous whether to use signed or unsigned types, and if auto-conversions do exist, say on the first argument so it is cast to signed, then is -1 underflowed via wrapping (as it apparently is in your example) or is it capped to 0
<OvermindDL1>
That is ambiguous code
<OvermindDL1>
Maybe not in implementation, but in reading certainly
<FromGitter>
<straight-shoota> @girng multithreading is still in the works, but I don't see any ETA on that. Until then, you'll have to run multiple processes and use some sort of messaging between them
<OvermindDL1>
Does crystal really state that it does under/overflowing, or is it UB and that just happens to be what it does by default?
<OvermindDL1>
Where is this documented speaking of which?
<FromGitter>
<girng> @straight-shoota i'm really familiar with Redis. i've used it as a pub sub system for nodejs for years
<FromGitter>
<girng> im sure i can do the same thing with crystal instances to communicate right
<FromGitter>
<straight-shoota> sure
<FromGitter>
<girng> thjat's really cool. i wonder if there is a crystal manager similar to PM2 in nodejs
<heaven31415>
@girng and how will you handle communication between processes?
<Yxhvd>
OvermindDL1: You can create the same number by doing 10u32 - 20u32, and yes it is defined to wrap.
<FromGitter>
<girng> @heaven = PubSub
<FromGitter>
<straight-shoota> for most use cases it's actually not that bad to have no multithreading. when your server doesn't depend on everything being in one process, it will be easier to scale beyond one node if need be
greengriminal has joined #crystal-lang
<FromGitter>
<girng> @heaven, basically redis is a master server for internal pub sub. all the crystal clients and subscribe to it, and then notify other instances to communicate.
<FromGitter>
<girng> (i did it for nodejs before), im sure can do it w/ crystal
<FromGitter>
<straight-shoota> OvermindDl1, I don't know if there is documentation of this
<OvermindDL1>
Yxhvd: How do you do a bound addition or subtraction then?
<OvermindDL1>
Without pre-checking, just using the CPU intrinsics*
<FromGitter>
<girng> local IPC is a little faster i think, there are some examples of doing that and nodejs w/ libraries using windows pipes / unix sockets. im sure this can be possible with crystal too
<OvermindDL1>
And where in the spec is it defined to wrap? I'm wanting to use it a reference to someone. ^.^
<FromGitter>
<girng> but im talking out of my ass so apologizes, just bored. ty @straight-shoota and overmin for help
<heaven31415>
@girng but you won't use them because you know redis and pub sub better?
<OvermindDL1>
Local IPC is quite fast yeah, or use transactional shared memory (can crystal do this) too
<FromGitter>
<jwoertink> Does crystal have some method, or construct that would allow me to take a string that looks like a hash, and make it an actual hash?
<heaven31415>
because I just wanted to test how to create multi-process crystal app and wanted to start with sockets
<FromGitter>
<girng> @heaven, yep just cuz i'm more familiar with redis pub/sub basically
* OvermindDL1
is not a fan of redis
<OvermindDL1>
PostgreSQL actually has a pretty fantastic pubsub built in though
<FromGitter>
<straight-shoota> @jwoertink what to you mean "looks like a hash"?
<OvermindDL1>
^
<FromGitter>
<straight-shoota> yep, Postgres is awesome
<OvermindDL1>
^
<FromGitter>
<jwoertink> I mean literally looks like a hash. `{"a" => 1}`
<FromGitter>
<bew> "can't share memory" why not? It's just a bunch of syscalls, and memory manipulations
<OvermindDL1>
At work I have an ancient Oracle server accessed from inside a PostgreSQL server via the FDW interface and everything is accessed straight through PostgreSQL, it's awesome
<FromGitter>
<straight-shoota> you mean Crystal source code?
<FromGitter>
<straight-shoota> @jwoertink this is not trivial because what types and values could that hash have?
<FromGitter>
<bew> Me? What?
<FromGitter>
<jwoertink> String String
<FromGitter>
<straight-shoota> okay
<FromGitter>
<straight-shoota> then it's easy
<FromGitter>
<girng> server on crystal: ⏎ ⏎ ```var msg = TCPSERVER.get_utf8_string(TCPSERVER.get_32()) ``` ⏎ ⏎ since `bytesize` returns a signed int, i should be using `get_32` instead of get_u32. but a signed 32 can be negative right, but it doesn't matter because `bytesize` will NEVER return a negative number, because it's reading the amount of bytes in a STRING. so i should be ok i think (won't hurt)
<FromGitter>
<jwoertink> Are you suggesting parsing the hash, and then re-creating it manually on the other side?
<OvermindDL1>
What is the 'Crystal standard way' of handling resources (whether memory, file handles, network sockets, whatever) so that they are guaranteed closed/returned at a specific point in execution. Like just using a file handle for example, how would you do these things:
<OvermindDL1>
3. Open a file handle, pass it around to many places and close it when all places are finished with it in an indeterminate order at run-time?
<OvermindDL1>
2. Open a file handle and hold it during the return of the scope, returning it to the parent to close it when their scope ends.
<OvermindDL1>
1. Open a file handle and close it at the end of a scope
<OvermindDL1>
The closing must happen at the specified times even if the programmer forgets to do so explicitly so as to free the handle for other uses
<OvermindDL1>
For example, C++ uses RAII and smart pointers for such things, Rust uses it's ownership system and smart pointers, what does Crystal use?
<Yxhvd>
1 is the simplest, just open the file and provide a block to the open method
<OvermindDL1>
Yxhvd: Example link?
<FromGitter>
<faustinoaq> @bew That was an experiment to use a multithread map on crystal
<OvermindDL1>
faustinoaq: Is there a point to multi-threading it since crystal only runs on a single core anyway?
<FromGitter>
<girng> now that confuses the heck out of me, why would the godot engine have a `get_32()` method, why would someone ever send over a signed byte that's negative in a tcp stream 😄
<OvermindDL1>
I have no experience in Ruby whatsoever except for dealing with some some of it's horror's in server form
<FromGitter>
<jwoertink> thanks. That's what I had too. I was hoping there was already something in place, but I guess not.
<OvermindDL1>
Ah so it opens only inside a given block, that makes 1 easy then
<FromGitter>
<straight-shoota> OvermindDL1 what are use cases for 2 and 3? Until now I haven't found anything that can't be done with `open` + block in Crystal or Ruby
<FromGitter>
<straight-shoota> using blocks really simplifies many things where you would have to return open handles in other languages
<Yxhvd>
2 can simply be implemented similar to way 1 is implemented though, `def autoclosing(fh); yield fh; ensure; fh.close;end
<OvermindDL1>
2 is often used when you want to open a file handle in a special format and you don't want to rewrite the same code all over to open the file handle the same, and thus return it for it to actually be used. You can of course invert the calling there to pass in a callback to handle it though and that would handle most cases, but generally incurs an overhead and makes other things more difficult.
<OvermindDL1>
3 is used way often for file handles that are pipes for multiple readers/writers
Jenz has joined #crystal-lang
<FromGitter>
<girng> @faustinoaq btw, without your crystal on WSL project, it'd be very hard for me to run the crystal sublime thing on windows (auto formatter). APPRECIATE IT sir, a lot.
<FromGitter>
<straight-shoota> in Crystal you don't need a callback for that
<FromGitter>
<straight-shoota> the method for opening the handle would just allow to be passed a block: `open_custom_handle do |handle| use_handle(handle) end`
<OvermindDL1>
A block is a callback, I'm just used to the name `callback` so I tend to use it instead. ^.^
<OvermindDL1>
`block` to describe a callback is very... nondescriptive to me, I'm not sure who came up with it or why why `callback` is actually descriptive...
<FromGitter>
<ezrast> but yeah callbacks are really easy with Ruby/Crystal syntax and are usually inlined in Crystal so they're zero overhead
<OvermindDL1>
Eh, they are easy in most languages
<FromGitter>
<straight-shoota> yeah, but it has no overhead and actually makes things easier because you don't have to pass around open handles and find ways to ensure they get closed
<OvermindDL1>
straight-shoota: How would you handle 3 then?
<FromGitter>
<straight-shoota> that's more tricky, agreed
<FromGitter>
<faustinoaq> @girng You are welcome! I hope RX14 can finish an usable windows port for crystal this year 😉
<OvermindDL1>
Tricky? It's pretty trivial in most languages?
<FromGitter>
<straight-shoota> how so?
* OvermindDL1
primarily comes from C++ though
<OvermindDL1>
In C++ it's by an atomically bound shared_ptr with the file closing function passed in as the destructor function, then copy that shared_ptr around as you need, when the last is destroyed then the file is instantly closed
<Groogy>
Erhm unless you are using templates and functors, callbacks in c++ do not geti nlined
<Groogy>
if that is what you meant with trivial
<OvermindDL1>
std::shared_ptr is a templated class
<Groogy>
(I know I'm horrible jumping in middle of discussion without reading full context=
<OvermindDL1>
Heh
<OvermindDL1>
And checking the assembly you find it has no indirect calls
<Groogy>
Yeah but for instance if you provide in a callback to an API, that function can not be inlined
<Groogy>
Templates are expanded into regular code so the function is known
<OvermindDL1>
Groogy: The optimizer will usually inline such things in non-template calls, but for it to be guaranteed then you need to use templates
<Groogy>
the second the function is not known it can not bei nlined
<OvermindDL1>
This is why C++ lambda's have a unique type per defined lambda
<Groogy>
Eh no there is no way for the optimizer to inline virtual calls or function pointers
<OvermindDL1>
It allows you to place it into a template argument to have no-cost callbacks
<OvermindDL1>
If they are known statically they are
<Groogy>
also if you use std::function it can not inline that either
<Groogy>
No a virtual method means it can be overriden
<Groogy>
it needs to look up the v-table for it
<FromGitter>
<straight-shoota> C++ has shared pointers, Crystal has garbage collection
<OvermindDL1>
std::function is a type wiper, you should *not* be using it for unstored callbacks
<Groogy>
and the v-table is pretty much just acollection of function pointers
<OvermindDL1>
straight-shoota: I'm talking about non-memory resource handling
<FromGitter>
<straight-shoota> so, eventually, the handle will be closed when it is GC'd
<OvermindDL1>
straight-shoota: That is *very* bad when something needs to be grabbing the file when the existing handle is closed though
<OvermindDL1>
'eventually' is not good in resource management
<Groogy>
that's why the crystal way to do it is blocks
<FromGitter>
<straight-shoota> exactly
<OvermindDL1>
Those are scoped-only though, you need a method of resource management that is unscoped
<OvermindDL1>
Groogy: That only works in that scope, I'm talking about something like opening a bidirectional pipe that is stored globally to be used around a program as, oh, IPC
<Groogy>
and it becomes something's ownership of Obj's life cycle
<OvermindDL1>
'most' are yes
<OvermindDL1>
I'm talking about the ones that are not 'most'
<RX14>
for unscoped usages you have to manually call .close
<Groogy>
If you are saving it globally you are doing something wrong
<Groogy>
in Crystal I would do exactly the same
<OvermindDL1>
Groogy: Often you can't control library code though
<Groogy>
the obj in something is responsible for the lifetime
<OvermindDL1>
It's an example primarily. ;-)
<RX14>
at least with FDs instead of memories you're not going to segfault and cause a security issue from a leak or double close()
<RX14>
"memories"
<RX14>
OvermindDL1, we don't have a great answer for unscoped resources I admit
<Groogy>
Well that applies to C++ as well, you can't control library code. You can write your wrap around it (which giving something to std::shared_ptr would be)
<Groogy>
but you would do that in Crystal as well
<RX14>
the best we can do is ask people to try and use scoped as much as possible, hope that people call close, and hope that slow leaks get eaten by the GC
<OvermindDL1>
Groogy: Exactly, hence why you need methods of handling it outside
<Groogy>
and writing your own std::shared_ptr in Crystal is very much possible
<Groogy>
and actually very simple
<OvermindDL1>
File handle is not the only other resource, it's just an example, anything from the similar network handles to perhaps GPU allocated memory (which I highly doubt is managed by the GC) to locking serial interfaces to etc...
<OvermindDL1>
Groogy: Ah, it has an atomic integer?
<RX14>
yeah we have atomics
<OvermindDL1>
Awesome
<Groogy>
Yepp also I do GPU allocated memory
<Groogy>
and I just clear it in the finalizer
<RX14>
disgusting
<OvermindDL1>
A finalizer? That sounds... really... exceptionally bad for GPU memory
<Groogy>
Well I sort of have to right :P You guys won't take care of it
<RX14>
finalizers are for trying to fix up bugs
<Groogy>
No your texture object when it is no longer in use have to clear up it's memory
<OvermindDL1>
Fix up bugs?
<Groogy>
so it tells the GPU it is no longer using the memory
<RX14>
all handles should be manually closed
<OvermindDL1>
Groogy: And how do you deallocate such a texture object when, oh, no other mesh references it immediately?
<RX14>
if it gets to the finalizer GC closing the handle thats a bug
<OvermindDL1>
RX14: Why have finalizer's then?
<Groogy>
The GC collects it?
<OvermindDL1>
Why not just not compile?
<OvermindDL1>
If there is such a bug*
<Groogy>
RX14 eeeeh what?
<RX14>
how the fuck do you not compile?
<OvermindDL1>
Rust has a good method as one example
<RX14>
Groogy, what do you mean ehh what
<OvermindDL1>
C++ can do it as well with attributes but then it tends to be gcc/clang-only
<RX14>
only the GC calls finalizers
<RX14>
structs? no finalizer
<RX14>
finalizers get called at an indeterminate time
<RX14>
which is exactly why they're a last resort
<OvermindDL1>
Indeterminate time's are really really bad though...
<RX14>
not something you should ever rely on
<Groogy>
Eeeeh
<Groogy>
then crystal in itself is going to be really buggy
<RX14>
how
<Groogy>
for prety much EVERY single library
<Groogy>
ever
<Jenz>
What?
<RX14>
no
<Groogy>
For instance Textures i crsfml
<OvermindDL1>
Don't all GC's run finalizers an 'indeterminate' time later though?
<Groogy>
or every single object allocated through crsfml
<RX14>
yes
<OvermindDL1>
Excepting knowably scoped pointers that is
<Groogy>
So any object that coimes from an external library
<RX14>
Groogy, you're meant to be manually closing these handles
<Groogy>
you are supposed to deallocate manually?
<RX14>
for FDs, yes
<Groogy>
THen why do I even have a GC?
<OvermindDL1>
Eh, this is why I tend to hate GC's, not because they are inherently bad or slow but because the languages that use them tend to not have good realtime management for 'other' resources. Memory is not the only resource and GC's are rarely (ever?) real-time.
<Groogy>
Yeah but this isn't FDs
<RX14>
what???
<RX14>
I don't follow your logic
<OvermindDL1>
And if you have a good resource management then that can manage memory as well as any other, thus making GC's useless
<Groogy>
I allocate an object in an external library, I have that handle and do stuff with it
<Groogy>
I allocate A LOT of those
<OvermindDL1>
GPU memory is very much like a file handle, it is a kernel handle, just pointing elsewhere
<Groogy>
you mean I am supposed to manage their memory myself?
<Groogy>
It's not Overmind
<RX14>
OvermindDL1, rust does resource tracking at the expense of programmer annotations everywhere
<RX14>
thats not a route crystal will ever go
<Groogy>
You only get an identifier
<Groogy>
not an actual handle on them emory
<Groogy>
THe identifier dies when the process dies
<OvermindDL1>
RX14: I don't mind it that much to be honest, if Rust yells at me I *know* that I personally am in the wrong and it caught a bug
<Groogy>
and the identifier always starts at 0
<RX14>
OvermindDL1, if it was a route we'd go we wouldn't have a GC at all
<Groogy>
IT's not like FD's at all
<RX14>
OvermindDL1, crystal isn't aiming to be a language for people who care about pointers
<Groogy>
unless you are writing the driver yourself
<OvermindDL1>
I still can't get Rust to be as fast as my C++ code on non-trivial projects sadly...
<RX14>
or borrowing
<Groogy>
Either way RX14 I want that clarified
<Groogy>
if I allocate in crsfml a sprite, you mean I should deallocate that sprite as well?
<Groogy>
andn ot let the GC do it?
<Groogy>
mind you a sprite has no GPU memory or anything
<Groogy>
it's just a logical object
<OvermindDL1>
I'd say it depends on how the sprite points to it's texture
<RX14>
i'd say it depends on what resource it takes up
<OvermindDL1>
^
<Groogy>
IT takes up memory
<Groogy>
memory that the GC is not aware of
<RX14>
having a finalizer to clean up memory allocated in C is fine
<OvermindDL1>
As long as you don't mind it being late
<Groogy>
Second the sprite will have a reference to the texture.
mps has joined #crystal-lang
<RX14>
having a finalizer to clean up a file handle allocated in the kernel is not really
<RX14>
actually fuck
<RX14>
it's all fuzzy
<Groogy>
So you mean it is fine to clear the GPU memory but still have a reference to the texture
<Groogy>
hence having an incomplete object?
<RX14>
what
<RX14>
I'm too tired for this today
<OvermindDL1>
Groogy: To GPU memory you'd probably want to track usages from the sprite to that memory and while any sprites exist that use that memory then you should not clear that memory
<OvermindDL1>
An atomic integer will work well for that
<Groogy>
THat's what the GC already does
<RX14>
RC(T)
<Groogy>
each sprite will tell the GC there's a reference to the texture
<Groogy>
which means it does not get deallocated
<Groogy>
However if you are yourself in charge of that it becomes permissable to clear that memory and have the sprites refer to a texture object that holds no texture data
<OvermindDL1>
I don't generally deallocate an unused texture, rather I keep it around in CPU memory until it is referenced by an active renderable again, upon which it copies it to the GPU again
<RX14>
Groogy, it depends on how cheap the thing is
<OvermindDL1>
At least in C++
<RX14>
if you can only have 1000 sprites then you can't leave the GC to do the work
<RX14>
because you're going to easilly hit 1k sprites before the GC kicks in
<Groogy>
Why even remove it from the GPU though OvermindDL1?
<Groogy>
You have virtual memory manangement these days
<Groogy>
you arej ust doing what the kernel already is doing on the GPU
<Yxhvd>
groogy: a GC will run if it is out of memory normally.No such automatic deallocation will happen automagically if you run out of GPU memory, so you can get situations when you need to run the GC but the GC can't know it if you use finalizers to clean up GPU memory.
<RX14>
a limit of 1k sprites would mean that sprites are a very latency scarse resource
<RX14>
and you shouldn't leave the GC to do it
<OvermindDL1>
Groogy: Because a lot of GPU's, like mine purposefully, are really really old and don't manage that well
<RX14>
I'd say that GPU memory is a scarse resource
<OvermindDL1>
^
<OvermindDL1>
My GPU is ancient
<RX14>
I'd say that file handles are a scarse resource
<OvermindDL1>
My wife's is... not ancient... >.>
<RX14>
but memory is not
<Groogy>
It has to be from like the 90's then
<OvermindDL1>
But if it works on my GPU, it works anywhere
<OvermindDL1>
Groogy: First-gen vulkan support, about 8 years old
<RX14>
Groogy, at the end of the day if it works it works
<Groogy>
Virtual memory management has been a thing for a long time
<OvermindDL1>
Groogy: A lot of drivers do it really badly, and vulkan you need to manage the memory more explicitly
<FromGitter>
<girng> how does a garbage collector work if lets say for a server handling requests? does the memory eventually become free if the server doesn't have a lot of people on the site, or does it stay allocated to crystal for the lifetime of running?
<OvermindDL1>
I'd imagine if it's not running a pool of requests that they'd get deallocated eventually
<OvermindDL1>
But a pool of requests would make sense...
<OvermindDL1>
With an upper bound on pool size based on overall load
<Groogy>
Yxhvd: that's not an argument against cleaning it though. Running out of GPU memory means you are doing something really wrong just saying as well
<RX14>
@girng now you're talking about how the GC uses OS memory, which is a different topic
<OvermindDL1>
Groogy: It's really really easy to run out of 256megs of GPU memory
<RX14>
the crystal (bdw) GC doesn't return memory to the OS
<OvermindDL1>
Wait what? Ever?
<Groogy>
256mb? What?
<RX14>
OvermindDL1, nope
<RX14>
Groogy, lol are you acting like 256mb is insanely small
<RX14>
have you ever used a laptop
<OvermindDL1>
What if I allocate a large 8gig chunk once, do work in it, then dereference it?
<RX14>
OvermindDL1, you'll use 8gb
<Groogy>
also the virtual memory paging is a OS thing, not a graphics card thing
<RX14>
forever
<Groogy>
it doesn't depend on the age of your graphics card
<Groogy>
Yes 256mb is nothing
<Groogy>
Laptops have their entire RAM available
<OvermindDL1>
Groogy: It's a driver thing, you need to trust the driver to page GPU memory in/out, and even then only with OGL, in vulkan you have to be more explicit
<RX14>
wait you don't explicitly manage whats on the GPU?
<RX14>
thats nuts
<OvermindDL1>
^
<FromGitter>
<straight-shoota> ^^
<OvermindDL1>
I manage and pack every byte I can... >.>
<Groogy>
You only manage it if you work with Vulkan
<OvermindDL1>
Or if you want to run fast
<Groogy>
Drivers been managing it for decades
<OvermindDL1>
If you exceed the vram per-frame, your fps drops like a rock
<RX14>
crystal is a terrible language for game programming
<OvermindDL1>
Groogy: Not always well
<OvermindDL1>
*Especially* intel's
<RX14>
OK
<RX14>
3d game programming
<Groogy>
I wasn't saying well
<Groogy>
but it is good enough for whatever a hobbyist is gonna put together
<Groogy>
If you need that extra squeeze you are gonna need a full team working 8 hours a day anyway
<OvermindDL1>
A lot of hobbyist's waste memory like it's going out of style I've found. ;-)
<OvermindDL1>
Not really, you just need to know how the hardware works and keep it in mind
<Groogy>
Well PUBG does as well
<OvermindDL1>
Never seen pubg?
<Groogy>
which I don't get how they manage
<FromGitter>
<straight-shoota> Java is even worse for 3D game programming but there are still games in java
<OvermindDL1>
straight-shoota: libgdx is pretty good for that, and encourages pooling so the GC never runs
<heaven31415>
they even are financial success, who would guess :D
<oprypin>
Groogy, I think CrSFML is the only library for which what you were saying isn't true (maybe only the Textures part)
<Groogy>
No but I have 4gb VRAM at home and somehow PUBG managed to use it up
<Groogy>
and crash
<OvermindDL1>
Java is pretty fast if you manually manage Java's memory so the GC doesn't run... ^.^;
<Groogy>
oprypin of what isn't true?
<OvermindDL1>
Which MC definitely does not do... >.>
<oprypin>
in CrSFML memory allocation is managed by Crystal
<RX14>
GC in games triggers my "bad fucking idea" bell like crazy
<Groogy>
MC is not in JAva anymore
<Groogy>
since they got bought up
<OvermindDL1>
RX14: +1
<RX14>
16ms you have for a frame
<Groogy>
oprypin that's what I meant
<OvermindDL1>
Groogy: MC is still java, MCPC is C++
<Groogy>
I was just trying to get what RX14 meant
<RX14>
and you're just going to trust the GC?
<OvermindDL1>
MCPE?
<OvermindDL1>
Whatever it's called
<Groogy>
just used it as a common language example
<Groogy>
ah yeah MCPC is what I meant
<OvermindDL1>
Which doesn't even run on linux so screw them with a rust spork...
<RX14>
MC is still in java
<OvermindDL1>
rusty*
<RX14>
the java edition is still updates
<OvermindDL1>
^
<RX14>
and anyone who uses mods uses java edition
<OvermindDL1>
1.13 is about to come out? Or did it just?
<OvermindDL1>
I don't keep up anymore, MC's community is nasty... >.>
<RX14>
eh
<RX14>
I don't like calling whole communities nasty
<RX14>
there are nasty parts to it
<OvermindDL1>
'General Community'
<OvermindDL1>
The loud parts
<heaven31415>
eight years old kids?
<RX14>
so lex
<RX14>
lol
<OvermindDL1>
^
<OvermindDL1>
VeryMuchLex
<FromGitter>
<girng> @RX14 bought i thought the whole point of a garbage collector is to free memory im confused
<RX14>
it does free it
<Groogy>
Either way I have 11k frames with just pure OpenGL in my framework on crystal
<RX14>
the problem is there's 2 memory pools
<RX14>
a pool for the program to use
<RX14>
and a pool for the GC to use
<Groogy>
of course if the GC would constantly claim textures then I would be really bad but then something is definetly wrong
<RX14>
the formater is managed by the GC
<RX14>
the latter is managed by the OS
<FromGitter>
<straight-shoota> RX14 the other way around
<RX14>
and the GC frees memory so it can be used again *by the same program*
<OvermindDL1>
No RX14 is right
<RX14>
and not for other programs
<RX14>
OvermindDL1, perhaps i was a bit confusing though
<Groogy>
also even if you do free it back fully the process will still potentially keep the memory depending on the OS
<OvermindDL1>
former instead of formater though*
<RX14>
lol
<OvermindDL1>
Heh
<FromGitter>
<girng> i see. so crystal has specialized memory allocated to it, that's will be there
<OvermindDL1>
girng: It's a pretty standard GC
<Jenz>
It has a good name at least
<RX14>
not unmapping memory isn't really standard
<OvermindDL1>
The GC has a name?
<Jenz>
If you'd just drop the 'GC'
<RX14>
bdwgc
<FromGitter>
<straight-shoota> OS manages GC memory pool?
<RX14>
we just use bdwgc OvermindDL1
<OvermindDL1>
RX14: I've seen some other languages that don't
<RX14>
which is an off the shelf GC
<RX14>
written in C before I was born
<Groogy>
A thing I discovered, even if I did free back memory, the process kept it and it fucked up the Steam inter-process calls sincei t was past the 2gb limit of 32bit
<OvermindDL1>
BohemGC is pretty bad so I'd doubt that
<RX14>
bdw = Boehm-Demers-Weiser
<Jenz>
Oh ok
<OvermindDL1>
'Pure BohemGC'*
<RX14>
OvermindDL1, it's mark sweep lol
<FromGitter>
<straight-shoota> but it works
<OvermindDL1>
Ah, wow, that is a very basic mark-n-sweep GC...
<OvermindDL1>
Why that one instead of another?
<RX14>
because it's the simplest GC
<FromGitter>
<straight-shoota> and it works pretty for many things
<RX14>
you just s/malloc/GC_malloc
<RX14>
and call it a day
greengriminal has quit [Quit: This computer has gone to sleep]
<Yxhvd>
it is very simple to integrate and it is conservative, which at this point is necssary
<FromGitter>
<girng> @RX14 so its best to have an isolated server just for a game server because the ram will be all isolated to crystal. if i have it running on aserver with other systems, like a forum, or redis, it's not a good idea because that ram could all be allocated to the crystal instance under load and not returned back to OS. so it's best to keep one server separate.
<FromGitter>
<girng> if i got that right?
<Jenz>
Im completely at loss when people are talking about these low-level things
<OvermindDL1>
Does Crystal compile via LLVM? If so LLVM has some *fantastic* GC mark point integration for better GC's
<RX14>
OvermindDL1, the GC was chosen when crystal has no gc at all
<RX14>
and any GC is better than no gc
<OvermindDL1>
True...
<RX14>
no gc meaning no free
<RX14>
btw
<OvermindDL1>
Yep
heaven31415 has quit [Ping timeout: 240 seconds]
<Jenz>
I'll gues I'll have to buy that really expensive book on low-level programming
<RX14>
OvermindDL1, yeah we compile via llvm
<Yxhvd>
I would be suprised if integrating llvm gc wouldn't be accepted if someone made a pr that covered all points
<RX14>
but you still need quite a bit of compiler support
<OvermindDL1>
RX14: Should definitely add LLVM's GC support points in then! It allows for pluggable GC's of a great order. :-)
<RX14>
and you still need to write the damn gc that uses that support
<OvermindDL1>
Even if they go unused 'right now', get that in the codegen early!
<RX14>
OvermindDL1, I'm not a GC export
<RX14>
expert*
<Jenz>
Do we have any?
<Jenz>
(expert)
<RX14>
any sources on how the LLVM gc support works?
<OvermindDL1>
Official docs has a section, I can find it if wanted?
<Groogy>
Also hey if you are into games we are having a campaign now where you can get one of my games for free. https://forum.paradoxplaza.com/forum/index.php?threads/crusader-kings-ii-is-currently-free-on-steam-and-there-is-a-sale.1086965/
<Groogy>
if anyone want to give it a try
<OvermindDL1>
Or arbitrary pointers
<RX14>
OvermindDL1, so from reading the LLVM doc, the only actual LLVM support is replacing the load and store functions
<OvermindDL1>
Groogy: Linux support?
<Groogy>
Yepp of course
<OvermindDL1>
RX14: Correct, that is the *BIG* thing needed for proper arbitrary GC support
<RX14>
the rest is seperate from LLVM
<Groogy>
All of our games since 2012 have full native linux support
<OvermindDL1>
And fences
<OvermindDL1>
Yep
<RX14>
OvermindDL1, but surely that's super simple?
<OvermindDL1>
GooNice
<OvermindDL1>
RX14: Not as much as you'd think
<OvermindDL1>
That is why LLVM added those specific instructions
<Yxhvd>
groogy: I'm still waiting for a victoria I can run though :(
<OvermindDL1>
They change execution ordering
<OvermindDL1>
Which is really important for GC use
<OvermindDL1>
CPU pipelining execution ordering*
<Groogy>
Eh I fixed the crash on steam if that's the one you mean?
<FromGitter>
<girng> can i write a game engine in crystal
<Groogy>
or do you just mean "Not be a horribly broken game"?
<FromGitter>
<girng> using openGL bindings or something?
<Groogy>
:P that one is harder
<Yxhvd>
groogy: I mean on linux :P
<Groogy>
ah
<Groogy>
if we do a next one it will be on Linux ;)
<Groogy>
@girng yes you can
<Groogy>
I've written everything natively in Crystal
<Groogy>
well almost, 90% I guess
<RX14>
OvermindDL1, but if the replace load and store then can't they just be subbed out in the codegen simple-as?
<FromGitter>
<straight-shoota> Groogy, I just secured the free CKII. Steam was so nice to inform me of the offer =)
<OvermindDL1>
RX14: The thing about LLVM's specific load store is you have to define gc roots and fences as well, this is very hard to get right since CPU pipelining can sometimes cause a pointer to exist before the memory or vice-versa, which if the GC ran right at that time then *bugs*
<oprypin>
girng, "crusader kings" is not in crystal if that's what you're thiinking
<Groogy>
Nice! LEt me know what you think @straight-shoota
<OvermindDL1>
The codegen's know how to handle the gc load/stores/roots properly for their arch's
<Groogy>
Oh yeah the games I've worked on before is not in Crystal, crystal is what I work in to relax
<RX14>
OvermindDL1, this is what makes a single thread nice for GC
<RX14>
you know that you're in C when you do the GC
<RX14>
so all the roots are on the stack
<OvermindDL1>
Lol, yeah with a single thread it is less of an issue. ^.^;
<RX14>
slow though
<FromGitter>
<girng> @groogy hey, cna u do me a favour and take a look at Godot repo. what kind of bindings do they use to write to screen? OpenGL 3.0 i think? im not entirely sure
<RX14>
OvermindDL1, bdwgc just does signal handlers lol
<RX14>
for multiple threads
<RX14>
iirc
<OvermindDL1>
Heh, yeah and it missing some global arguments as well and it misses pointers that are tagged sadly
<Groogy>
OpenGL doesn't ahve different bindings for different versions... it's hard to... eh just write in a short sentance but yeah..
<OvermindDL1>
It fails in a lot of cases
<Groogy>
THey probably use OpenGL
<OvermindDL1>
Which if the codegen of crystal doesn't do any of that, no worry
<Groogy>
If you want example of how to bind with OpenGL I can show you
<RX14>
I've never actually looked at the crystal codegen
<Groogy>
all here is my backend code for opengl
* OvermindDL1
definitely prefers vulkan over ogl
<Groogy>
Vulkans takes too much effort to just get setup
<OvermindDL1>
Though no older hardware with it unless using a translation layer
<OvermindDL1>
Not really?
<OvermindDL1>
There are fantastically awesome vulkan frameworks out now
<OvermindDL1>
At least for C++
<Groogy>
Yeah I went through Valves own tutorial for it and it took me way too much
<OvermindDL1>
Rust is getting a good one in the `gfx` library
<FromGitter>
<girng> @Groogy wow.. you wrote all that?
<OvermindDL1>
Valve's tutorials are super low level... ^.^;
<Groogy>
Well I was trying to work directly on the direct level
<Groogy>
yeah
<Groogy>
@girng yepp
<Groogy>
It's all zlib license as well so people are free to steal as long as they credit me ^^
* OvermindDL1
really needs to move stuff from bitbucket to github...
<OvermindDL1>
Only have 101 repo's on github, a *lot* more on bitbucket... >.>
<OvermindDL1>
I don't like posting unfinished code though, but I've been trying to fix that the past few years
<OvermindDL1>
Even unfinished code is occasionally useful to others
<Groogy>
OvermindDL1 essentially I wanted no intermediary layer between my crystal code and the graphics etc.
<Groogy>
to prove a point sort of
<Groogy>
becuase it is impossible to do feasably with performance in Ruby
<RX14>
OvermindDL1, I put everything on github
<RX14>
even my infastructure
<OvermindDL1>
I really hate ruby... I don't know the language, but the VM is utter abysmal garbage... >.>
<RX14>
it's nice
<Groogy>
Which I tried even with C libraries in Ruby. Wrote a whole Glue library for it
<FromGitter>
<straight-shoota> why bitbucket? :D
<RX14>
private repos presumably
<FromGitter>
<girng> @groogy well, im not that advanced as you in crystal and bindings so ima just star your project and contribute if i can. no reasonf or me to re-invent the wheel
<OvermindDL1>
I've used bitbucket since before github's days, plus free private repo's
<OvermindDL1>
They were mercurial primarily at the time
<OvermindDL1>
I *still* have my home SVN server running
<OvermindDL1>
I really *need* to get my code off of that before it's *12* year old hard drive dies...
<Groogy>
also shit I should be working so I can go home D:
<FromGitter>
<straight-shoota> for private repos I use gitlab. it's awesome. But certainly hasn't been around as long as bitbucket^^
<OvermindDL1>
I want to like gitlab, but holy-hell-it-is-*SLOOOOW*
<z64>
for private stuff i've really been enjoying keybase's free git stuff
<OvermindDL1>
It's a fantastic little interface for GPG keys
<Vexatos>
I have never used it for anything other than signing git commits
<OvermindDL1>
Vexatos: I use the chat and file stuffs daily
<OvermindDL1>
stuff*
<FromGitter>
<girng> so its like discord but encrpyted?
<Vexatos>
no
<Vexatos>
it is not a chat client
<OvermindDL1>
No, it's a GPG client
<OvermindDL1>
It just happens to do more too
<Vexatos>
It allows you to easily encrypt messages
<FromGitter>
<girng> their screenshots literally show people chatting
<OvermindDL1>
All encrypted with *your* GPG key
<Vexatos>
which you then send over [chat client of choice]
<OvermindDL1>
girng: It's a popular feature
<FromGitter>
<girng> ok well i'd rather stick to gitter
<FromGitter>
<girng> i like this channel
<Vexatos>
OvermindDL1, by files you mean the encrypted git repo?
<OvermindDL1>
girng: Entirely distinct things
<OvermindDL1>
It's chat is 1-on-1 fully GPG encrypted stuff
<OvermindDL1>
Vexatos: No, just the file stuff to share files with friends
<Vexatos>
absolutely overkill encryption
<OvermindDL1>
Mostly Factorio modpacks as of late... >.>
<OvermindDL1>
Very much so yes
<OvermindDL1>
But convenient
<Vexatos>
like, waaaaaaaay overkill
<OvermindDL1>
Heh, yep
<Vexatos>
using GPG for chat messages is like using police escort to buy groceries
<OvermindDL1>
And a 4096-bit GPG key to boot!
<FromGitter>
<girng> seems out of my league lol
<OvermindDL1>
girng: Mostly is, it's just to confirm authenticity
<Vexatos>
OvermindDL1, I never used the keybase software >-< do you just "keybase encrypt infile outfile" or something?
<OvermindDL1>
girng: basically with my public key I can prove I am who I say I am with greater accuracy and trustability then a few lifetimes worth of the universe in computational power
<OvermindDL1>
Vexatos: I use their CLI client primarily, their desktop GUI for chat mostly
<Vexatos>
As I said, I only ever used it for signing git commits using the verified key
<OvermindDL1>
I really should sign my commits... >.>
<FromGitter>
<girng> @overmindDL1 so basically, if i were to create a username OvermindDL1 i can't claim its you
<FromGitter>
<girng> bcz i don't have a authorised key
<OvermindDL1>
girng: Correct, and that's not just on keybase, that is *anywhere*
<Vexatos>
OvermindDL1, I only do it on one repo I care about
<OvermindDL1>
I can 'sign' a message with my private key, and you can confirm that it was only me that made that message by using my public key
<FromGitter>
<girng> oh, i see it's use case then
<OvermindDL1>
Or you can encrypt a message with my public key and only I can decrypt it with my private key
<RX14>
Vexatos, yeah it's like a police escort - you can still get shot
<Vexatos>
Or apparently you can encrypt a file in a way only the recipient can decrypt it >_>
<FromGitter>
<girng> all the trolls and impersonation bs online nowadays, seems like a cool feature
<Vexatos>
with relative easy
<OvermindDL1>
Vexatos: Exactly the same thing yep
<Vexatos>
ease*
<OvermindDL1>
Encrypting and signing works in binary, not text, so it works on anything
<OvermindDL1>
girng: GPG/PGP keys are *OLD*, it's many decades old
<OvermindDL1>
Very well tested technology
Jenz has quit [Quit: leaving]
<RX14>
still not secure against certain usecases though
<RX14>
like chat
<OvermindDL1>
HTTPS encryption uses a similar setup but with more negotiation protocols, and less strength
<RX14>
keybase doesn't use pgp/gpg for chat
<OvermindDL1>
4096-bit that I use is way way *WAY* overkill
<RX14>
it creates a custom key signed by your key
<OvermindDL1>
Yes that^
<RX14>
and then uses that for chat
<OvermindDL1>
It just creates 'another' key just for those two users yes
<RX14>
because you really want forward secrecy
<RX14>
and shit like that
greengriminal has joined #crystal-lang
<OvermindDL1>
Yep
<RX14>
still good though
<RX14>
I keep my subkeys on a yubikey
<RX14>
and sign all my git commits
<RX14>
and use it for ssh
<OvermindDL1>
I've never touched a yubikey as of yet, how useful is it?
<RX14>
it's super convenient
<OvermindDL1>
I keep intending to look in to it
<OvermindDL1>
How do you use it, what exactly does it do?
<RX14>
OvermindDL1, it's really useful for having secure access to your ssh keys on any computer
<RX14>
you start the gpg agent in ssh agent mode
<Vexatos>
Not more secure than a USB stick btw
<RX14>
set the ssh auth sock
<RX14>
and then ...
<RX14>
Vexatos, what no?
<RX14>
the keys don't leave the yubikey
<OvermindDL1>
RX14: I often use a chromebook to SSH home, would it work with that?
<RX14>
OvermindDL1, probably
<RX14>
as long as the gpg agent could run
<RX14>
and like
<OvermindDL1>
chromebooks make such awesome remote thinclients
<RX14>
usb works
<RX14>
then it's fine
<OvermindDL1>
RX14: Eh, it's pretty closed down so I'd be unsure
<RX14>
i'm not sure how linux chroots work re: usb in chromebooks
<RX14>
OvermindDL1, yeah
<RX14>
i'd google it
<RX14>
someone's sure to have tried
<OvermindDL1>
If it can be a chrome extension that uses the HTML5 USB API then it should
<RX14>
the good thing about yubikeys it that it can do everything at once
<RX14>
you have a GPG smartcard, U2F device, OTP generator in one
<RX14>
and you just wear it all day
<OvermindDL1>
Heh
<Vexatos>
AND THEN YOU LOSE IT
<RX14>
lanyards are good
<OvermindDL1>
Lol, that is what I'd be worried about
<RX14>
i've never lost mine
<OvermindDL1>
I'm often carrying almost nothing
<RX14>
but then I keep mine on a lanyard inside my shirt
<OvermindDL1>
So carrying something would be eh...
<FromGitter>
<girng> lanyards made me socialize more for some reason
<Vexatos>
you'd forget it at home :⁾
<RX14>
you get into the habit Vexatos
<RX14>
it's like wallet and keys
<Vexatos>
losing a yubikey is suuuuper annoying :P
<RX14>
you rarely forget them
<RX14>
Vexatos, so is loosing your keys
<RX14>
besides you should only store subkeys on them
<RX14>
so if you loose it you just revoke, buy a new one, and go again
<OvermindDL1>
^
<Vexatos>
you don't "just" buy a new one they are not that cheap :P
<OvermindDL1>
What's their price?
<OvermindDL1>
?google yubikey price
<DeBot>
OvermindDL1: Nothing known about google.
<RX14>
like £40
<OvermindDL1>
Oh, no google bot here?
<RX14>
no
<RX14>
yeah about £40
<OvermindDL1>
~$80's pretty hefty
<RX14>
so you definitely dont want to loose it
<RX14>
$80??
<RX14>
the USA isn't that shit
<OvermindDL1>
$ are not doing well currently
<RX14>
more like $50
<RX14>
$55
<OvermindDL1>
Wait a month... >.>
<RX14>
51.77
<RX14>
but how often do you loose your keys OvermindDL1
<OvermindDL1>
I rarely carry my keys
<RX14>
oh
<FromGitter>
<girng> ok so i have another question about memory @straight-shoota can i ask?
<RX14>
how do you get into your house?
<OvermindDL1>
I live in a tiny town in the middle of the high-plains/desert that is flat on a Level3 backbone line
<OvermindDL1>
Electronic locks
<OvermindDL1>
My house is *very* wired up...
<OvermindDL1>
Not much else to do around here
<RX14>
>flat on a Level3 backbone
<RX14>
priorities
<OvermindDL1>
Mmmm tasty 8ms latency
<OvermindDL1>
You have no idea how much I love that
<RX14>
8ms to where?
<RX14>
I get 4ms to london at uni :3
<OvermindDL1>
google.com, about 12ms to my canada webserver
<RX14>
$ ping 8.8.8.8
<RX14>
64 bytes from 8.8.8.8: icmp_seq=1 ttl=55 time=5.50 ms
<RX14>
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
<RX14>
64 bytes from 8.8.8.8: icmp_seq=4 ttl=55 time=5.38 ms
<RX14>
64 bytes from 8.8.8.8: icmp_seq=3 ttl=55 time=5.39 ms
<RX14>
64 bytes from 8.8.8.8: icmp_seq=2 ttl=55 time=5.45 ms
<OvermindDL1>
23ms to my california webserver, that's better than it normally is
<RX14>
I'm going to miss uni
<OvermindDL1>
8.8.8.8 is usually cached *very* locally, I get about 2-6ms to it
<OvermindDL1>
Ooo I should try 1.1.1.1
<RX14>
OvermindDL1, it's a server in london
<RX14>
OvermindDL1, 1.1.1.1 is about the same
<OvermindDL1>
Yick, 21ms to 1.1.1.1
<RX14>
actually 1.1.1.1 is ~4.9
<OvermindDL1>
They are not on L3 here it seems
<Yxhvd>
64 bytes from 8.8.8.8: icmp_seq=1 ttl=60 time=1.83 ms
<RX14>
whaa
<OvermindDL1>
RX14: Do not I'm well over a thousand miles from any 'real' servers, no such thing as anything next door for me
<RX14>
where do you live Yxhvd
<RX14>
OvermindDL1, fair enough
<Yxhvd>
stockholm. 8.8.8.8 exists all over the place and I think they have one here
<RX14>
the UK's pretty compact
<OvermindDL1>
RX14: His ISP is caching it, Google has 8.8.8.8/8.8.4.4 cache's with a *LOT* of ISP's
<OvermindDL1>
That's why it's not a great test
<RX14>
OvermindDL1, i'm still used to DSL itself being ~5ms lol
<OvermindDL1>
Except to test your ISP's interconnections
<OvermindDL1>
Heh
<FromGitter>
<girng> how many clients do u think 1 instance of crystal can handle on a digitial ocean VPS? just a tcp server that has a couple chat rooms? no tick rate game loop or anything
<FromGitter>
<girng> would be 100, 1000, 2000, 10000 users? where is the ballpark range at??
<OvermindDL1>
girng: From my testing about 60k connections per second that just open TCP, send one packet, and close
<Vexatos>
hm maybe I should install keybase :U no clue what I'd ever use it for :P
<FromGitter>
<girng> @overmindDL1 oh wow u actually did tsesting? 60k is really a good number
<RX14>
@girng it really depends on what you're doing and how you program it
<RX14>
60k is lowend lol
<Yxhvd>
64 bytes from 1.1.1.1: icmp_seq=4 ttl=60 time=0.985 ms <- heh.
<RX14>
Yxhvd, ...
<RX14>
this makes me sad
<FromGitter>
<girng> lol iunowhat to compare the number to
<OvermindDL1>
girng: It was on my local server, C++ was hitting well over 600k
<OvermindDL1>
Almost 700k
<RX14>
OvermindDL1, crystal can do 100k HTTP requests per second
<RX14>
so 60k sounds low
<RX14>
did you use --release
<RX14>
?
<OvermindDL1>
RX14: Yep, it's part of those tests on that repo I linked a couple nights ago
<RX14>
sounds low
<OvermindDL1>
If you can make them faster, *please* do
<RX14>
thats 100k per core btw
<FromGitter>
<girng> i have a basic tcp server up using spawn (fibers). and users can join a chat room (30 players per chat room) and send a message. THAT'S it
<RX14>
so
<RX14>
yeah
<RX14>
sounds low
<OvermindDL1>
girng: Yeah that's nothing
<RX14>
@girng you're probably going to be limited by how many chat messages you get
<RX14>
not how many connections
<FromGitter>
<girng> got it
<RX14>
actually no
<RX14>
30 players per room?
<RX14>
that just makes it O(n)
greengriminal has quit [Quit: Leaving]
<RX14>
that'll scale forever
<RX14>
i'd say 500k/server
<RX14>
easy
<FromGitter>
<girng> yeah chat rooms can't be bigger than 30 players, when a new player joins and if that 1 chat is full it'll create Chat-Room-2 and so on
<FromGitter>
<girng> or whatever one has less than 30, the lpayer will go into. you get it
<FromGitter>
<girng> wow that's a lot...
<FromGitter>
<straight-shoota> can you install keybase CLI only? don't need GUI on a VM server
<FromGitter>
<girng> well i don't think i will need to run multiple instances of crystal then, because i doubt i will ever get players that exceed those numbers
<Yxhvd>
RX14: well, it will stop scaling once every message that get to the server should be sent to every other listener.
<FromGitter>
<girng> @overmind hey can u re-link that repo with those tests again? sorryi think i missed it
<OvermindDL1>
The current benchmarker in that repo is built as a single-core crystal benchmarker, so it cannot saturate most of the servers, not even the crystal servers, so ignore the current measurements in the readme
<OvermindDL1>
But it is easy to run them yourself, I use `wrk` for that, and I've made a simple script to automate it too if you want it and have `wrk` installed
<OvermindDL1>
straight-shoota: yep, a cli version can be installed, useful for headless servers and such
<OvermindDL1>
Don't recall the magical incantation off hand though
<OvermindDL1>
Maybe just run the installed with `DISPLAY=''`?
<OvermindDL1>
But basically just do, oh, clone it then run `make crystal cpp rust` to build all crystal, cpp, and rust servers, then using my script you can just do `<path-to-script>/stats.exs crystal cpp rust` to test them all using `wrk`
<FromGitter>
<girng> damn, very interesting thanks a bunch!
<OvermindDL1>
They are very very stupid servers, designed to be as such
<OvermindDL1>
They do no real work so it's not a real benchmark of any form, but it is a fun little thing to work on on occasion. ^.^
<FromGitter>
<girng> yah i just wanted a general feel/idea and honestly far more than i expected which is awesome
<OvermindDL1>
I find it funny that even elixir is beating nodejs when nodejs is fully clustered and all. ^.^;
<OvermindDL1>
Also go's fasthttprouter library is wow-fast
<OvermindDL1>
Beats all the rust ones
<OvermindDL1>
Barely, but still
<OvermindDL1>
If only go-the-language was not so horrible... ^.^;
<OvermindDL1>
C++ still slaughters it though
<Vexatos>
Didn't even know keybase had a GUI app
<OvermindDL1>
Vexatos: It's a nice interface to all their functionality
<OvermindDL1>
Oh hey, node's clusterpolka is new, it is beating elixir, nice
<FromGitter>
<girng> @overmind i was trying to write a small tcp server, but ended up not and rather use crystal instead because much easier for me. but i also heard if using C++ tcp server, u have to use threads or some kind of async / non-blocking thing and im not advanced at all to code that in c++. so again, i just went back to crystal
<FromGitter>
<girng> all roads lead me back to crystal for some reason.
<OvermindDL1>
girng: Oh a C++ web server is *so* easy, go to the root path in that repo, then go to the cpp directory, take a look
<OvermindDL1>
The cpp_evhtp server is entirely multi-threaded, defaulting to `<num-of-cores>-1`
<OvermindDL1>
girng: Check libevent's docs
<OvermindDL1>
libevent is the root TCP library it uses, evhtp is just a very thin HTTP layer on top of libevent
<OvermindDL1>
Plenty of docs on libevent out there
<OvermindDL1>
Look at the cmakelists.txt file for how to include and use it all
<OvermindDL1>
Can just depend on libevent straight instead of libevhtp
<OvermindDL1>
Or keep it, you get libevent too when you depend on libevhtp and the http stuff might be useful for you too
<OvermindDL1>
As you can see in the `main.cpp` file, there are both calls to evhtp and to libevent straight, it's a really *thin* layer on top of libevent
<OvermindDL1>
I'm amazed at how slow python was in the tests... Ruby is even worse though, much much *MUCH* worse
<OvermindDL1>
That is why I didn't add it to my tests, it is just *so* abysmally slow
<FromGitter>
<girng> why did elixir rank 8?
<OvermindDL1>
I'm running the `wrk` tests over just python and ruby, let's see how that goes
<OvermindDL1>
girng: Look at the table, it performs really fast
<OvermindDL1>
Elixir performs better under bad load though, which I did not test, it might even be in first place in the really bad cases
<OvermindDL1>
It is, by *far*, the best interpreted language
<OvermindDL1>
Like it puts python and ruby and all to absolute shame
<OvermindDL1>
Even nodejs with it's JIT can't even keep up
<OvermindDL1>
Except 1 of the frameworks, and only that one. :-)
<OvermindDL1>
The Beam VM (Elixir/Erlang) is fantastically designed, it just will-not-die
<FromGitter>
<girng> yeah i eally wanted to use elixir i tried it for WEEKs trying to get a hang of it. i jsut can't do it w/ functional languages
<FromGitter>
<girng> it's too confusing for me soi gave up. i tried though
<OvermindDL1>
Functional is so easy, like stupid-easy, immutability and all makes everything so trivial to reason about
<OvermindDL1>
You do have to 'unlearn' some things though if programmed elsewhere first
<FromGitter>
<girng> yes, i posted for help and got it really fast. couple posts actually, but even then i still didn't understand what they were telling me lol
<OvermindDL1>
Ask more questions then! They are super helpful!
<FromGitter>
<girng> for example, if you are using it for a master game server. oyu need to have a "player class". which is super eeasy on crystal. but on elixir, it seems like there is no such thing of classes or whatnot, everything is just tossed around like functions it's just so confusing to me
<OvermindDL1>
Ew classes, don't do OOP stuff, no matter your language
<OvermindDL1>
OOP has a niche it is good at, almost no one uses it for that though
<OvermindDL1>
In Elixir you'd have a player data storage, whatever you want, probably a struct, unless it needs to be doing things constantly, in which case it would be a process, but regardless it all goes through a Module interface
<FromGitter>
<girng> yeah there it is "player storage"
<FromGitter>
<girng> they told me about that
<OvermindDL1>
Generally just a 'struct'
<OvermindDL1>
Which is just a map in elixir
<OvermindDL1>
If you want speed you'd use a record but those are harder to use and harder to introspect so most avoid them overall
<OvermindDL1>
If you need speed you usually write a NIF (native interface function) to C++ or Rust or whatever (maybe crystal if crystal can make C API libraries?)
<FromGitter>
<girng> but wtf how is there storage, when everything is tossed around like a function where do MUTIPLE players come into play? for example Player #2, duplicating the struct? but i thought structs are immutable
<OvermindDL1>
You 'carry' it with you. :-)
<OvermindDL1>
Generally you put it in a process, and the process manages it
<OvermindDL1>
A BEAM process is not an OS process, think of it as a super greenthread
<Yxhvd>
no need to use that in crystal,we have structs builtin.
<FromGitter>
<girng> @overmind yeah but how do you do references?
<OvermindDL1>
Elixir has structs built in, just: defstruct :blah, :bloop, something_with_a_default: 42, etc...
<FromGitter>
<girng> that confuses me too
<OvermindDL1>
How do you 'want' to reference it?
<FromGitter>
<girng> if i want to reference to player 2's socket object, so i can send him a pm if player 1 is messaging him..
<OvermindDL1>
Oh for a socket I'd use the built-in pubsub
<FromGitter>
<girng> there's no way to store a reference to him
<OvermindDL1>
It'll outperform any other pubsub system you could possibly use
<OvermindDL1>
Just store a name or user id or something is the easiest
<OvermindDL1>
The pubsub handles the linking
<OvermindDL1>
Even across different hardware servers
<FromGitter>
<girng> :D
<FromGitter>
<girng> ima give it a nother try just because im talking about it
<OvermindDL1>
Lol, don't hesitate to ask questions. ;-)
<OvermindDL1>
That's how you learn after all!
<FromGitter>
<girng> i kinda understand it more after u told me "carry it around" that does make since in a functional world
<johndescs>
hum, am I really dumb? "crystal tool expand test.cr" just writes me the usage…
<OvermindDL1>
girng: In the BEAM VM world you hold 'shared' state in a process, basically a greenthread you send messages to/from, the basic concept of which is a GenServer
<OvermindDL1>
If it's trivial pure data then ETS is very useful, think of it as a built-in local Key/Value database
<FromGitter>
<girng> i like that
<FromGitter>
<girng> maybe i do like functional programming
<OvermindDL1>
mnesia is the 'global' (multi-node/hardware) key/value database built up using many local ets setups
<OvermindDL1>
Pure immutable functional programming is really *really* easy, just...different
<OvermindDL1>
It makes the code super easy to reason about since nothing surprising is happening
<OvermindDL1>
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h
<OvermindDL1>
Blehg
<OvermindDL1>
Doesn't come with it's own development files, that's fun
<FromGitter>
<girng> @overmind so, making a struct of a player class. and if a new player joins, store a hash/key of structs but for player #2. then, if player 2 wants to send a msg to player 1, loop through the array of structs to find player 1, and then access its socket property to `send` the message?
<RX14>
OvermindDL1, ruby dep management is really fucking annoying
<RX14>
luckilly it's largely solved in crystal
<RX14>
everything's project-local
<Yxhvd>
developent headersnot being included is the fault of the distro you use though
<OvermindDL1>
girng: Are you using the Phoenix webserver? If so it comes with Phoenix.PubSub already
<RX14>
yeah
<OvermindDL1>
If so I'd use that
<RX14>
also uhh
<OvermindDL1>
Yxhvd: ubuntu, not surprising
<RX14>
well thats the rule with debian
<RX14>
it never has dev headers
<OvermindDL1>
Yep, split it all!
<RX14>
and the packages are always out of date
<RX14>
:)
<johndescs>
Yxhvd: thx :) no I got "no expansion found" but this sound more like if something happens
<RX14>
I use arch btw
<FromGitter>
<girng> @overmind nope, just regular tcp echo GenServer
<OvermindDL1>
Not unless using updated PPA's
<OvermindDL1>
girng: Just single-node?
<OvermindDL1>
Single hardware server*
heaven31415 has joined #crystal-lang
<FromGitter>
<girng> yeah one instance
<Yxhvd>
johndescs: you may have to add whatever file you have that requires everything for it to work. :/
<RX14>
the thing with that chat with rooms setup is you can just shard so easily
<RX14>
so it truly is infinitely scalable
<FromGitter>
<girng> @overmind i heard elixir utilizes all cores on 1 server with 1 instance, true?
<OvermindDL1>
girng: Elixir comes with the `Registry` module, it has a single-node pubsub built in, setup a process (GenServer) per player and connect them via Registry. :-)
<RX14>
@girng most languages do
<OvermindDL1>
girng: yes, the BEAM VM is naturally many-core'd
<RX14>
well yeah
<RX14>
bem is better
<RX14>
because your programs are poarallel by default
<RX14>
wow typing is hard
<OvermindDL1>
Okay, ruby is installed, running tests comparing it to python...
<FromGitter>
<girng> what's the reason for pubsub when i just need to loop through a players array to notify someone is sending a private message in-game?
<RX14>
#crystal-lang has become #elixir-lang today...
<OvermindDL1>
girng: You can if you want, but it also makes it even more trivial to let 'other' things communicate with them too
* OvermindDL1
is a big fan of named process registration
<OvermindDL1>
But you can loop if you want
<OvermindDL1>
Or use a map to look up directly in a single GenServer or so
<FromGitter>
<girng> ohh, so not mandatory
<johndescs>
Yxhvd: add to what? currently just trying a single file with a macro, for debug
<OvermindDL1>
Not at all
<OvermindDL1>
You can do things very low level and simple in Elixir/Erlang, or use modules that add all sorts of useful little capabilities and helpers
<OvermindDL1>
Oy flask sucks... >.>
<FromGitter>
<girng> @overmindDL yeah, a map. that's what i want... i just want a way to store the players that are connecting. they need properties however. current_experience, level, username, user_id, etc. which a STRUCT would be perferct for right. i can have a map with structs? then loop em?
<Yxhvd>
johndescs: then you probably get the line numbers wrong
<FromGitter>
<girng> so basically like an array in crystal but with classes. but in elixir it's a struct inside a map
<OvermindDL1>
girng: Yep, almost any type can contain any other type, it's a dynamically typed language
<johndescs>
I've put 1:1 first, then where the macro starts, then no other idea…
<OvermindDL1>
Even a struct is just a map with hardcoded names for easy lookup
<FromGitter>
<girng> ok, well the more i try to compare to how it acts like in crystal, the more i understand it
<johndescs>
ho, it has to be at exact position where the macro is used, now it works! thanks :)
<OvermindDL1>
The basic Type's on the BEAM VM are integer (infinite sized), float (64-bit), lists, tuples, maps, PID's (Process ID), ref (a globally unique reference), fun (a function reference), a binary, and atoms (think 'Interned String' from OOP languages)
<OvermindDL1>
I think that's all
<OvermindDL1>
Everything is made up of those
<OvermindDL1>
Good grief, Ruby's Rack tests are going sooooo sloooooooooow
<FromGitter>
<girng> there are no pointers right?
<OvermindDL1>
Correct
<FromGitter>
<girng> and, if i remove a struct from a map (if a player disconnects). memory is free'd?
<OvermindDL1>
Very 'soon'
<OvermindDL1>
The GC on the BEAM VM is... unique, since everything is split up into non-sharing processes, each process has it's own memory and it's own GC and it tends to remain tiny, and since all data can only contain even older data, it makes collecting super-fast and non-blocking
<FromGitter>
<girng> ok cool. so like crystal we don't have to worry about memory management
<OvermindDL1>
It's a GC very very turned to the dynamics of the language, that is why it often outperforms even Java's GC, even though it is a functional interpreted language (the actual execution is not super fast of course, but eh)
<OvermindDL1>
It's GC*
<FromGitter>
<girng> just make sure to keep delete player from array if they disconnect
<OvermindDL1>
girng: Honestly I'd just put the player data and the socket in the same GenServer, and if the socket dies then let it die too, then just store PID's in ETS or the Registry
<OvermindDL1>
I.E. I'd start by making a GenServer called 'Player' or whatever
<OvermindDL1>
And a defstruct inside that to define what data to hold
<OvermindDL1>
Here, this is not quite right at all, but think of a GenServer as a 'Class' in OOP terms
<OvermindDL1>
And 'start'ing it is like instancing the class
<OvermindDL1>
You then carry around the PID of it, which you can think of as a pointer
<FromGitter>
<girng> can i think of it as a reference
<FromGitter>
<girng> ok overmind ima try it again. and i should prob join the elixir gitter/chat too. do they have one?
<OvermindDL1>
A reference *is* a pointer
<OvermindDL1>
So yep
<OvermindDL1>
girng: They do but I'm not on them, I'm mostly just on the forum (and IRC)
<OvermindDL1>
Most are probably on the Elixir Discord though
<FromGitter>
<girng> i appreciate your time.. helping a random stranger
<FromGitter>
<girng> very nice of you
<OvermindDL1>
I am in the elixir discord but I rarely get messages on it since I'm always in the terminal... ^.^;
<OvermindDL1>
girng: No problem, it's fun to do. :-)
<OvermindDL1>
If I didn't find it fun, I wouldn't be a teacher at times. ;-)
* OvermindDL1
works at a college as the sole programmer + occasional teacher
<FromGitter>
<girng> Well, you explain is really well.. especially since im coming from the OOP background
<OvermindDL1>
Yep, rack is not dying when it is told to die
<OvermindDL1>
Why the hell would a unix process not die when it is told to die....
<OvermindDL1>
It's forking and the original PID is dying! Wtf?!
<OvermindDL1>
Friggin stupid ruby...
<OvermindDL1>
Does anyone know how to kill puma when it's forking away from my control, it's not even responding to killall -9 puma... I may need to iterate the process list, blah...
<havenwood>
OvermindDL1: And Ctrl-C or sending it INT doesn't kill it?
<OvermindDL1>
Indeed
<havenwood>
OvermindDL1: I guess we should be in #ruby.
<OvermindDL1>
Specifically I can't ctrl-c it because a script is running it
<OvermindDL1>
Rather stdin is closing
<OvermindDL1>
And when something is not a daemon, then when stdin closes it should *die*
<OvermindDL1>
That's the unix way
<OvermindDL1>
And I'm obviously not passing in a `--daemon` flag...
<OvermindDL1>
The C++/Rust/Go/Elixir servers are all following the unix way, why is ruby stupid...
<Yxhvd>
does the config file tell it to start a pumactl service perhaps?
<OvermindDL1>
There should definitely not be a service
<OvermindDL1>
And I didn't write it, it's the above linked repo
<FromGitter>
<girng> @overmind so https://gist.github.com/adnils/5c91b51e97c792a7c9e5 if you look here the `spawn` is kind of how crystal's spawn works? but it's calling `listen(socket)`, which is recursion. how come it doesn't error out? is it because the `listen(socket)` will only return true if a new connection has arrived? and if it's true, it will re-call `listen`? but what if no new connections are coming in, it would return
<FromGitter>
... false and exit out of that function. how does it know to call it again?
<OvermindDL1>
girng: spawn is similar, but it sets up a full distinct context with it's own memory and all, just like an OS process (but internal)
<OvermindDL1>
And what would it error out?
<OvermindDL1>
For note, ':gen_tcp.accept(socket)' waits until a connection comes in, so it pauses there
<FromGitter>
<girng> nvm, not error. my bad. i mean, if no connections are coming it would exit right because the last line in elixir is the return value right? if no connection it'll be false, so it jumps out of that function so how does it know its going to be called again
<OvermindDL1>
No? It's not exiting there, you have an infinite loop
<FromGitter>
<girng> yeah but doesn't that infinite loop break if no connections are coming in?
<OvermindDL1>
No? Why would it?
<FromGitter>
<girng> because `listen(socket)` would be false, jumps out of that function
<OvermindDL1>
Why would listen(socket) be false? I'm not seeing anywhere that returns false?
<OvermindDL1>
The start just calls listen, listen then waits for a connection, when it gets it then it spawns off a new process to handle that new socket, then listen just calls listen again and it repeats waiting for an incoming connection again
<OvermindDL1>
It's creating a new process on every incoming connection, processes are *super* cheap on the BEAM VM
<OvermindDL1>
The new process is just echoing back all data it receives until the socket is remotely closed, then it just returns without doing anything. :-)
<OvermindDL1>
But it 'waits' at line 9 until a new connection comes in
<OvermindDL1>
When it comes in it spawns a new process at line 10, then calls itself (listen) again at line 11. :-)
<OvermindDL1>
It's just an infinite loop, never returns. :-)
<OvermindDL1>
Except the spawn on the BEAM VM will distribute the processes among all cores on the machine
<FromGitter>
<girng> @overmind ohhh. got it. iuno why i was thinkingit owuld be returning false. i was just hinking if no connections are coming in something should happen
<OvermindDL1>
Nope, it just waits. :-)
<OvermindDL1>
Think of each 'process' on the BEAM VM like an OS process, it has a `main` function, does whatever it wants, pauses, whatever, and when it returns from that main then the process exits. :-)
<OvermindDL1>
So it's like having thousands or millions of tiny little itty bitty cheap OS processes
<FromGitter>
<girng> funny how the crystal examples kinda makes me understand how it works more than the elixir example hahahaha
<FromGitter>
<girng> but i do getit now
<OvermindDL1>
Hehe, you have to do recursion, that's the difference there. ^.^
<OvermindDL1>
For note, GenServer wraps the basic process 'loop' like listen is there with a defined API, should always use it. :-)
<OvermindDL1>
It's erlang instead of elixir (elixir just compiles to erlang though), but the website Learn You Some Erlang For Great Good is awesome to teach the concepts. :-)
<OvermindDL1>
Then you can apply those to elixir (just a different code syntax is all, plus Elixir has macro's)
<FromGitter>
<girng> so i can think of the GenServer as the "TCPServer" in crystal
<FromGitter>
<girng> That's be fair?
<FromGitter>
<girng> it's a namespace that lets me call methods
<OvermindDL1>
Hmm no, a GenServer is more like a `spawn`'d process but with an API of functions to call on it
<OvermindDL1>
A TCP Server (if you want speed, use the cowboy library for that) just runs 'inside' a genserver
<FromGitter>
<girng> I see
<FromGitter>
<girng> I really like this example i linked i want to learn from this
<FromGitter>
<girng> so, i assume the genserver handles all the TCP connections and disconnects, and free's memory when they are disconnected? they do that all internally like crystal's TCPServer
<FromGitter>
<girng> so, in the `recv` function, is where i can start appending a player struct to a map, and start removing a player struct from a map (if closed), and do all the message handling based on player's input (private message, join lobby, etc)
<OvermindDL1>
Nope, GenServer is just a loop with an API to handle messages, that's all
<OvermindDL1>
And don't think about memory, it vanishes when a process dies
<FromGitter>
<girng> wait really??
<OvermindDL1>
en-bulk, instantly
<OvermindDL1>
No GC needed
<FromGitter>
<girng> WTF
<OvermindDL1>
Most processes, like the echo one and listen one, don't use any memory to begin with, just a single word on the stack anyway
<OvermindDL1>
Like I said, the memory model fits the language design to a T
<FromGitter>
<girng> so when a tcp socket closes, all the data it has is gone because its process is gone?
<OvermindDL1>
It would absolutely not work in something like crystal
<OvermindDL1>
girng: Yep
<OvermindDL1>
Instantly
<OvermindDL1>
No waiting on a GC to process it
<FromGitter>
<girng> but, if i were to store a map of player structs, i still need to remove that struct from the map if they disconnect though right?
<OvermindDL1>
Which is easily done via monitors
<OvermindDL1>
You can 'monitor' a process, and when it dies (like that echo process dies when it's socket is closed) then you get a message telling you that, and thus you can clean up
<FromGitter>
<girng> i could put the code in the recv function tho right?
<FromGitter>
<girng> if i wanted?
<OvermindDL1>
Things like Registry do that automatically
<OvermindDL1>
Yes
<FromGitter>
<girng> seems liek they have a lot of signal systems
<FromGitter>
<girng> that emit stuff when stuff happens
<OvermindDL1>
Everything for a specific purpose
<OvermindDL1>
No point recreating anything unless you are learning how it works
<FromGitter>
<girng> wait i think i have aproblem
<OvermindDL1>
?
<OvermindDL1>
I have a 1+ hour long meeting at work here in 20 minutes for note, so I'll vanish for a bit about then
<FromGitter>
<girng> if lets say i have a large `recv` function handling a lot of player actions, say a bunch of code. this means, this code is duplicated each time a new user is connected?
<OvermindDL1>
It doesn't execute until a specific message comes in of course
<FromGitter>
<girng> i mean if 5 users connect they all have their own beam process with all that `recv` code right?
<FromGitter>
<girng> that's gonna be a lot of code duplication isn't it
<OvermindDL1>
They get their own process, but code is compiled once, what do you mean duplicated?
<OvermindDL1>
It's just like running a program on the OS, you compile the code once, but you can run it many times and they all share the machine code in ram (at least on linux)
<FromGitter>
<girng> i figured the code is in their own process too, but it spawns a new listen each connection so i figured it makes another copy
<OvermindDL1>
Code is not data (this is not LISP ^.^), it only exists once. ^.^
<FromGitter>
<girng> yeah.. hmm
andrewzah has joined #crystal-lang
duane has quit [Ping timeout: 264 seconds]
<FromGitter>
<girng> total mindf*-word there. if the code inside recv function gets run in its own process, it has to have its own seperate code right? or does it just refer to the code that is compiled once and uses that in its process?
<OvermindDL1>
Does it get it's own separate code on the OS if you run a process multiple times (at least in linux)? Nope, they all share the same chunk of read-only process RAM (copied on write, which you can't do on the BEAM, thus always shared)
<OvermindDL1>
Processes on the BEAM VM are *tiny*, a few hundred words on average
<OvermindDL1>
Also, wow, rails is... wow... all of ruby's stuff is horrible...
<OvermindDL1>
Some of the python things are not anywhere near as bad as I expected. ^.^
<FromGitter>
<girng> i understand i think. i keep overthinking what a new BEAM process does
<OvermindDL1>
It's just a local context of execution
<OvermindDL1>
Basically a standalone mini-stack and instruction pointer
<OvermindDL1>
Super tiny
<OvermindDL1>
Unlike a thread, which tends to be multiple kilobytes in size at the smallest
<FromGitter>
<girng> overmind will u be back later??
<FromGitter>
<girng> after your meeting
<OvermindDL1>
Yep, after the meeting
<FromGitter>
<girng> cool!
<OvermindDL1>
I'm headed to it in a couple more minutes, probably should go now. ^.^;
<FromGitter>
<girng> thanks so much =]
<OvermindDL1>
No problem. :-)
<OvermindDL1>
Functional programming is well worth learning, I generally recommend OCaml for that, but the BEAM VM with it's process system is *way* unique, nothing like it in any other language, it is a fascinating way of programming. :-)
<oprypin>
geez this channel blew up
<OvermindDL1>
Heh, sorry... ^.^;
<FromGitter>
<girng> we elixir channel now boys. jk
* OvermindDL1
is out to meeting
<FromGitter>
<girng> soem reason i draw parallels to elixir and crystal, no idea why but i do lol
shalmezad has quit [Quit: Leaving]
<FromGitter>
<faustinoaq> OvermindDL1 @girng I guess
<FromGitter>
<j8r> a new member of crystal-lang will be helpful to help @RX14 and other casual committers like makenowjust, @straight-shoota and cie to have their PR merged
<FromGitter>
<j8r> I think by the way it's only useful for games multithread
<FromGitter>
<j8r> Or desktop
<FromGitter>
<j8r> If you need to scale you application, you will probably need to do so in multiple machines, right? It's simpler to have lots of single thread instances that communicates with APIs with this architecture
<heaven31415>
probably, right!
<FromGitter>
<j8r> for sure there are pros and cons for both. What projects are you aiming for?
<heaven31415>
I was thinking about games
<heaven31415>
I think it would be easier to have naked threads
<FromGitter>
<j8r> yes, multi thread will be well suited
<FromGitter>
<j8r> but i think it's wiser to think for this optimization way after. Lots of games are mono-thread and perform very well
<heaven31415>
there is problem with music
<heaven31415>
I think
<FromGitter>
<j8r> You can do all of this with fibers
<FromGitter>
<j8r> and handle all async stuff with them
<FromGitter>
<j8r> through channels
<heaven31415>
you have point, that you can just create conccurent access
<OvermindDL1>
@girng And back
<FromGitter>
<girng> @overmind WB! I'm still here =]
<FromGitter>
<girng> how did your meeting go
<crystal-gh>
[crystal] RX14 closed pull request #2368: Add `Etc` module with methods for fetching user information (master...feature/etc_module) https://git.io/vVeV6
<OvermindDL1>
Long long, rushing my end of day
qard has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]