return0e has quit [Remote host closed the connection]
return0e has joined #crystal-lang
alex`` has quit [Quit: WeeChat 2.7]
alexherbo2 has quit [Remote host closed the connection]
dwdv has quit [Ping timeout: 265 seconds]
_whitelogger has joined #crystal-lang
_whitelogger has joined #crystal-lang
ur5us has joined #crystal-lang
oprypin_ has quit [Quit: Bye]
oprypin has joined #crystal-lang
oprypin has quit [Client Quit]
oprypin has joined #crystal-lang
oprypin has quit [Quit: Bye]
oprypin has joined #crystal-lang
dwdv has joined #crystal-lang
ur5us has quit [Ping timeout: 260 seconds]
_whitelogger has joined #crystal-lang
_whitelogger has joined #crystal-lang
return0e has quit [Read error: Connection reset by peer]
return0e has joined #crystal-lang
Yxhuvud has quit [Read error: Connection reset by peer]
alex`` has joined #crystal-lang
alexherbo2 has joined #crystal-lang
return0e has quit [Remote host closed the connection]
return0e has joined #crystal-lang
Yxhuvud has joined #crystal-lang
_whitelogger has joined #crystal-lang
Nicolab has joined #crystal-lang
<Nicolab>
Hello :)
ht_ has joined #crystal-lang
FromGitter has joined #crystal-lang
<FromGitter>
<stnluu_twitter> thats the doc i’ve been reading, though from this pull request, it mentioned that the macro is only executed with “the function is called” https://github.com/crystal-lang/crystal/pull/6108
<FromGitter>
<stnluu_twitter> > we don't want to execute @type right away: we want to defer it until the method is executed, so it works like a macro method
<FromGitter>
<Blacksmoke16> right, verbatim, afaik, works the same as `\`
<FromGitter>
<Blacksmoke16> id imagine at compile time when the compiler sees that method is getting called somewhere
<FromGitter>
<stnluu_twitter> hmm, cool. thanks for answering my random questions. i’m just curious and like to know indepth of my compiler/language works :)
<FromGitter>
<Blacksmoke16> if you remove the `#` you'll see the 5 when when macro expands
<FromGitter>
<Blacksmoke16> otherwise it doesnt print since it didnt expand yet
<FromGitter>
<stnluu_twitter> cool. seems like the compiler will take multiple passes until the macro expansion is needed?
<FromGitter>
<Blacksmoke16> :shrug: im not too familiar with the exact internals
<FromGitter>
<stnluu_twitter> haha same. just wanna double check and make sure i’m not paying runtime penalty (similar to runtime reflection in other languages)
Nicolab has quit [Ping timeout: 252 seconds]
Nicolab has joined #crystal-lang
<Yxhuvud>
Oh, the gateway was down. That explains why it has been so quiet here the last couple of days :)
<FromGitter>
<Blacksmoke16> say i have a module `Athena::Config` that contains common configuration stuff to share between projects
<FromGitter>
<Blacksmoke16> should the other projects define their config under that component's namespace, or the common `config` namespace
<FromGitter>
<Blacksmoke16> i.e.
<FromGitter>
<Blacksmoke16> `Athena::Config::Routing` versus `Athena::Routing::Config`
<FromGitter>
<Blacksmoke16> im thinking the former to keep all configuration together
<FromGitter>
<Blacksmoke16> documentation and such that is, esp when you can build docs for installed shards as well...
<FromGitter>
<Daniel-Worrall> You could then *also* do macro magic to define them in `Athena::Config::Component`
<FromGitter>
<Daniel-Worrall> but the first alone is how I'd expect/desire
<FromGitter>
<Blacksmoke16> i think that makes more sense, since it would be config specific to that component
<FromGitter>
<Blacksmoke16> which would leave the `Athena::Config` namespace free for stuff that all components can use in their configs
<FromGitter>
<btihen> I have a question about threads -- if I would like to efficiently send lots of messages to other objects would this approach work or is it too simple (the messages are immutable structs) - I am not sure where the best place to Fiber.yield is: ⏎ ⏎ ``` code paste, see link``` ⏎ ⏎ also if the reciever objects were to forward this message to more objects in a similar manner (I'm assuming `Fiber.yield`
<FromGitter>
... only affects the Fibers it spawned - not all fibers. If it releases all fibers - what is the correct way to trigger an objects own fibers? I find the examples with sleep not clear to me when dealing with objects forwarding to objects, potentially forwarding to objects. ... [https://gitter.im/crystal-lang/crystal?at=5e09028204a1900497cffca8]
Nicolab has quit [Quit: Leaving.]
<FromGitter>
<stnluu_twitter> it depends on what `post_message` does. is it doing a bunch of IO?
<FromGitter>
<stnluu_twitter> or does each `receiver` have their own channel and post_message simply sends it over to their channel in memory?
<FromGitter>
<btihen> Each user has a method which forwards to its various IO objects (sessions - the sessions do real IO). I thought maybe I should be using channels, but I am not sure how they would help and the best way to manage the channel connections. would I need to build a channel for each object and associate that say in a hash and send the message to the channel without spawning to each object. (Sorry - I'm reasonably
<FromGitter>
... confused)
<FromGitter>
<btihen> @stnluu_twitter
<FromGitter>
<stnluu_twitter> without knowing more of the problem you’re trying to solve, what you have seems fine. or the alternative is to put the “spawn” inside `post_message` and do what you need
<FromGitter>
<stnluu_twitter> maybe i’ll have better suggestion if you describe your use-case, rather than nitpicking on a specific implemenation without context :)
<FromGitter>
<stnluu_twitter> you want Fiber.yield OUTside the loop. otherwise it’ll block until each receiver finish doing its things
<FromGitter>
<stnluu_twitter> however, be careful with captured variables (there’s a difference between `spawn do .. end` and just `spawn` macro)
<FromGitter>
<randiaz95> "<th scope='row'>" + i+1 + "</th>" returns table header with 01 11 21 etc. not the sum, lol...
<FromGitter>
<randiaz95> Sorry for another js rant..
<FromGitter>
<stnluu_twitter> does wrapping (i + 1) inside parentheses do the right thing?
<FromGitter>
<randiaz95> yep
<FromGitter>
<btihen> @stnluu_twitter - Thanks. I am trying a few mini-projects to understand how to use threads and channels. From the docs where the only thing that happens is sleep - but I am interesting in knowing when to use channels and when not to. When to fiber.yield (thanks for the answer). And when channels are appropriate - what is the best usage pattern. So my mini-experiment is that I am building a fake chat-room
<FromGitter>
... that has users - who can 'join' multiple rooms. Each room and each connection has a session that prints (at the moment I am not doing anything special with the sessions - just pretending that they are various tcp connections on various devices - all belonging to the single user.
<FromGitter>
<randiaz95> that is the solution
<FromGitter>
<stnluu_twitter> @btihen Ah! There really isn't a right a wrong way, or any written rules of when you should use a channel and when you shouldn't. In general, if you have all of your input up front, just calling a function is the right thing to do. This function may spawn (i.e. fire and forget) a fiber if you want things running concurrently.
<FromGitter>
<stnluu_twitter> If you need to communicate things between different fibers, for example, you want to get the result or a signal after a spawned fiber completes, this is when message passing, or channels come into play
<FromGitter>
<stnluu_twitter> So for your use case, it really depends on your design and guarantee. I'll list two designs here, with different approaches:
<FromGitter>
<btihen> @stnluu_twitter - ah - so using channels is for when a send and response is required to continue for example with a credit card check (it must clear before purchase is allowed) or an authentication must be approved before joining a group would be allowed. the `spawn` macro for methods looks helpful (cleaner). - ah - so if I am using the spawn block within an iterator - i might be only getting the last value
<FromGitter>
... inside the fibers. So the solution is either to use the proc or the spawn macro - otherwise I'm probably not doing what I think with my loop!
<FromGitter>
<stnluu_twitter> kind of
<FromGitter>
<stnluu_twitter> in your case with the credit card check, if you *must* wait for the result before proceeding, you don't need to spawn a fiber, even if that check involves expensive IO.
<FromGitter>
<stnluu_twitter> fibers are NOT threads
<FromGitter>
<stnluu_twitter> so you're not blocking a thread. if you `checkCreditCard` function does IO, the underlying OS green thread will switch to other fibers, and come back to it once the IO finishes
<FromGitter>
<stnluu_twitter> the OS green thread does not get blocked, even though your checkCreditCard function takes a long time
<FromGitter>
<stnluu_twitter> the fiber/function is blocked
<FromGitter>
<stnluu_twitter> back to the chatroom use-case:
<FromGitter>
<btihen> Ah - so firing a standard IO - is a full OS thread and that just gets skipped while waiting for a response. So a channel wouldn't really help in this case when it is a 'print' job or 'tcp' or something like that.
<FromGitter>
<stnluu_twitter> umm, not quite. let me try again
<FromGitter>
<btihen> A fiber on the other hand is blocked until the spawning fiber gives it the go-ahead ?
<FromGitter>
<stnluu_twitter> unless you compile with a special flag, the WHOLE binary only has one OS green thread.
<FromGitter>
<stnluu_twitter> imagine there's a queue. the first thing in the queue is your main fiber.
<FromGitter>
<stnluu_twitter> when you spawn a fiber, another thing gets added to the queue.
<FromGitter>
<luis_salazar_twitter> Hi! Is there any rails-like parameterize method for string?
<FromGitter>
<stnluu_twitter> @btihen for IO functions, the standard library jsut implement do_work in a way that will just fire the IO call to the OS, and return immediately. the next do_work call will check with the OS and see if the data is ready
<FromGitter>
<stnluu_twitter> does that makes sense?
<FromGitter>
<btihen> OK - so if I have an expenseive IO 'print' and a message in memory (which hopefully triggers some expensive activity). If the IO is first, it will just get outsourced to the OS and return to the queue. But from the loop idea - it looks like the message that triggers an expensive calculation would block the main thread while the IO doesn't since it got 'outsourced'
<FromGitter>
<stnluu_twitter> correct. in other words `spawn do someVeryIntensiveCPUWork() end` will not block your current loop, but then someone else will get delayed if they come after that fiber
<FromGitter>
<Blacksmoke16> @luis_salazar_twitter not that i know of
<FromGitter>
<stnluu_twitter> @btihen there's no magic to this. at the end of the day, some CPU will still need to do your bidding. the only difference is that spawning a fiber lets you defer the work.
<FromGitter>
<stnluu_twitter> now, in your example, if `post_message` is doing IO--aka blocking the fiber that calls it, then maybe `spawn post_message` is a good option (assuming that's the design you want)
<FromGitter>
<btihen> @stnluu_twitter somehow I had the impression that crystal is good at concurrency and parrellism (or at least is designed for that). So the only way for the next job to not get massively delayed would be to send it off to another thread and that isn't the default setup. Basically, I thought crystal could use all (or most of my cpu magically - reducing the blocking happening in the fibers and even when it wasn't
<FromGitter>
... on different CPUs it was very good at switching between all tasks in a fast lightweight manner like elixir.
<FromGitter>
<stnluu_twitter> Crystal has concurrency, not parallelism :)
<FromGitter>
<stnluu_twitter> (not unless you compile with the experimental multi thread flag)
<FromGitter>
<stnluu_twitter> https://crystal-lang.org/reference/guides/concurrency.html ⏎ ⏎ > At the moment of this writing, Crystal has concurrency support but not parallelism: several tasks can be executed, and a bit of time will be spent on each of these, but two code paths are never executed at the same exact time.
<FromGitter>
<stnluu_twitter> @btihen now, back to the chat room example and how/when you want to use channels.
<FromGitter>
<stnluu_twitter> @btihen one subtle side effect of `spawn post_message(...)`, especially if you run in multi thread mode, is the `receiver` might be getting messages in random order, because fibers are not guaranteed to run in the order they spawn
<FromGitter>
<btihen> Ah its still experimental - Ok - I thought it was already mainstream. So in anycase, in my chat-room. spawn post_message is propbably the most approprite. Message delivery timing isn't important - just that it gets delivered. Yes I'm currious about the channel usage. Thanks - you are very helpful!
<FromGitter>
<stnluu_twitter> This is where channels come in
<FromGitter>
<stnluu_twitter> from your loop, you just call `receiver.ch.send(full_message)`, this is just a CPU operation
<FromGitter>
<stnluu_twitter> the `@ch.receive()` above will block the spawned fiber, but if there's no message, the thread will just move on to another fiber
<FromGitter>
<stnluu_twitter> this way, the messages will be processed in the order which this `User` object received the messages
<FromGitter>
<btihen> hmm - cool. Now channels are starting to make sense as to why I would use them. They are a little like a message mail system. cool - thanks for the sending from the room too.
<FromGitter>
<stnluu_twitter> hope that helped a bit :)
<FromGitter>
<randiaz95> interesting...
<FromGitter>
<stnluu_twitter> for `def post_message`, I recommend having a "Room" object, or a handful of room objects withtheir own channels and a loop doing the same thing. this prevents each user from needing to keep track of other users
<FromGitter>
<stnluu_twitter> (and keep the messages from the room in order)
<FromGitter>
<randiaz95> 1) o
<FromGitter>
<stnluu_twitter> @btihen precisely. if you think of each part of your design and its own entity with a mailbox, it's a lot easier to reason about.
<FromGitter>
<watzon> Yeah channels are ways to communicate between fibers (and threads, although iirc they're not thread safe yet)
<FromGitter>
<btihen> Yeah - a lot - thanks - in particular - I couldn't figure out the benefit of channels - except some theory and usage was unclear while interacting with real objects! all mcy clearer now. I'll go off and see if I can figure out the best way for the user to have multiple sessions associated with the channels and how the
<FromGitter>
<stnluu_twitter> There is a bit overhead with message passing like this, but overall it's pretty moot
<FromGitter>
<watzon> Yeah it's tiny
<FromGitter>
<btihen> i'll see if I can copy your notes - and play with it tomorrow. I need to get to bed soon. Thanks so much!
<FromGitter>
<stnluu_twitter> @watzon I thought they were supposed to be thread safe with the -Dmt_preview flag?
<FromGitter>
<stnluu_twitter> @btihen cheers!
<FromGitter>
<watzon> Well the mt_preview flag adds multithreading, but last I checked channels weren't completely thread safe
<FromGitter>
<watzon> I could be wrong though
<FromGitter>
<stnluu_twitter> @btihen two more things ⏎ ⏎ 1) if you have one channel at the room level, make sure the message has a "sender" field that you would skip broadcasting to itself, or upon receiving a message, the user skips `post_message` for the ones come form themselves. otherwise you'll get an infinite loop. ⏎ 2) if your messages are big, use `class`, not `struct`. Otherwise the copy penalty is huge
<FromGitter>
<stnluu_twitter> @watzon let me check the blog post again
<FromGitter>
<btihen> @watzon - what does it mean that they are not thread safe yet? Currently if my message is a Struct - isn't that immutable? If I send a variable - I guess that is mutable - are their plans to do something like pony lang with reference capabilities? or some other 'semi-auto' thread safety mechanisms? Or are their plans to build an actor model like elixir?
<FromGitter>
<stnluu_twitter> @btihen the channel itself is not threadsafe, is what @watzon meant.
<FromGitter>
<watzon> As far as I know reference capabilities and actor model aren't on the road map, as much as it could be nice to see
<FromGitter>
<stnluu_twitter> https://crystal-lang.org/2019/09/06/parallelism-in-crystal.html @watzon ⏎ ⏎ > The channels are able to send and receive messages through different worker threads without changes in the API and should be used as the main method of communication and synchronization between fibers.
<FromGitter>
<stnluu_twitter> i **hope** this means they changed the impl to be thread safe
<FromGitter>
<stnluu_twitter> and just keep the APIs
<FromGitter>
<watzon> Hmm awesome. It seems like that's what it would mean
<FromGitter>
<btihen> > @btihen two more things ⏎ > 1. if you have one channel at the room level, make sure the message has a "sender" field that you would skip broadcasting to itself, or upon receiving a message, the user skips `post_message` for the ones come form themselves. otherwise you'll get an infinite loop. ⏎ > 2. if your messages are big, use `class`, not `struct`. Otherwise the penalty for copying messages between
<FromGitter>
... fibers is huge ⏎ ⏎ 1) Ah good points - I was using struct since it is immutable (but yes - the copy does pay a penalty). I hadn't thought about the inifinite loop possiblity - i'll do that - I already have the sender and reciever fields - so ill protect against a loop! [https://gitter.im/crystal-lang/crystal?at=5e0916c8fd580457e792030a]
<FromGitter>
<watzon> Or at the very least they should be soon
<FromGitter>
<randiaz95> I will make my own little chat server tomorrow, do you guys want to share code on GitHub after?
<FromGitter>
<watzon> Sure, it's always nice to see how other people are progressing with Crystal
<FromGitter>
<watzon> And it's good to get feedback
<FromGitter>
<stnluu_twitter> im still busy with my dumbass macro-abusing RPC language @randiaz95
<FromGitter>
<stnluu_twitter> 😆
<FromGitter>
<randiaz95> LOl
<FromGitter>
<watzon> Oooh @stnluu_twitter I need to hear about this lol
<FromGitter>
<randiaz95> He is writing an RPC language without a lexer
<FromGitter>
<stnluu_twitter> @watzon haha. the idea is, i want an RPC language without writing my own parser/code generator.
<FromGitter>
<watzon> Well that's interesting
<FromGitter>
<stnluu_twitter> I'm doing this by abusing the fact that, the languages I use all have good macro support. so all I need to do is implement those macros in each language I support, and let the language compiler handle the code generation (via macros)
<FromGitter>
<randiaz95> your doing it over more than one language?
<FromGitter>
<stnluu_twitter> Crystal & Nim (compiles to JS & C)
<FromGitter>
<randiaz95> Are you wrapping all the compilers with one super compiler?
<FromGitter>
<stnluu_twitter> @randiaz95 nope. don't need to. all i need is a `myrpc_service.macro.txt` ⏎ ⏎ and from each respective language, just `#include "myrpc_service.macro.txt"` to paste it in as source code
<FromGitter>
<stnluu_twitter> @watzon i have only completed the macros & implementation for serialization. the service generation is easier
<FromGitter>
<stnluu_twitter> in this case, what will likely happen is actually the Nim/C++ code will `#include "myrpc_service.cr"` because Crystal doesn't have that "paste" capability
<FromGitter>
<randiaz95> Very cool I need some time to understand it lol
<FromGitter>
<btihen> @randiaz95 - cool - I'll do the same (share my current and hopefully improved by @stnluu_twitter - version) at `github/btihen` - tomorrow.
<FromGitter>
<stnluu_twitter> "someone" should blog about channel/message passing design, with examples. i've seem these same questions across languages, from Crystal to Nim to Golang
<FromGitter>
<btihen> @watzon ⏎ ⏎ > As far as I know reference capabilities and actor model aren't on the road map, as much as it could be nice to see ⏎ ⏎ RefCaps are a pain (for me at least) to keep track of though. I know they are much faster than locks and copying immutable data - but wow - at least pony's version requires a lot of thought and mind space (at least initially). But I do like the actor model in general.
<FromGitter>
<randiaz95> I also have questions about production that I wish could be answered, but I think things like this require a mentor.
<FromGitter>
<btihen> @stnluu_twitter - with the code samples I will write / improve and hopefully @randiaz95 - maybe we can make an examples link from the docs you sent me. With some explainations as to why they are useful in the case shown.
<FromGitter>
<stnluu_twitter> 😍
<FromGitter>
<randiaz95> there is hope in humanity
<FromGitter>
<randiaz95> We are working on improving ourselves and make things that are useful to humanity lol.
<FromGitter>
<randiaz95> yay!
<FromGitter>
<stnluu_twitter> speak for yourself
<FromGitter>
<stnluu_twitter> lol
<FromGitter>
<randiaz95> :D
<FromGitter>
<randiaz95> is there any way to get the type of data in a string in crystal?
<FromGitter>
<stnluu_twitter> has anyone ever written a macro, that wraps around a block of other macros? like: ⏎ ⏎ ```my_big_macro param1, param2, do ⏎ sub_macro p1, p2 ⏎ sub_macro p3, p4 ⏎ end``` [https://gitter.im/crystal-lang/crystal?at=5e091dcb07a1a67d1d7d8d85]
<FromGitter>
<randiaz95> say input "1" -> int "2.01" -> float "something" -> string
<FromGitter>
<stnluu_twitter> typeof(x).to_s ?
<FromGitter>
<stnluu_twitter> actually no. if you input "1" it's just a string ?
<FromGitter>
<randiaz95> Right, essentially I want a TypeSniffer function that works like: ⏎ ⏎ TypeSniff("1") and returns Int
<FromGitter>
<stnluu_twitter> basically, i want `my_big_macro` to generate a class, which wraps around the sub_macros
<FromGitter>
<stnluu_twitter> @randiaz95 not that i know of
<FromGitter>
<stnluu_twitter> where do you get your input from?
<FromGitter>
<stnluu_twitter> can you do TypeSniff(1) instead of TypeSniff("1")
<FromGitter>
<stnluu_twitter> ?
<FromGitter>
<stnluu_twitter> or let's step back. what are you trying to solve? :)
<FromGitter>
<randiaz95> I am trying to get the type of a column from a CSV
<FromGitter>
<randiaz95> in python I would do pd.read_csv("filename")["colname"].dtype
<FromGitter>
<randiaz95> what I have though is a large string with comma separated values inside of it..
<FromGitter>
<randiaz95> Oh never mind, my api gateway is in python, I can essentially figure that out in the python and then pass it to the micro service.
<FromGitter>
<stnluu_twitter> ah. i'm pretty sure under the hood it just tries to parse through different known types
<FromGitter>
<stnluu_twitter> there aren't that many native types in CSV. integers, real numbers, strings, date maybe?
<FromGitter>
<randiaz95> right,
<FromGitter>
<randiaz95> whole numbers, decimal numbers, strings, date and primary key would be what I want.
<FromGitter>
<stnluu_twitter> someone somewhere a few years ago wrote that dtype implementation. it could be you!!
<FromGitter>
<randiaz95> which I wouldn't know how to handle that difference between primary key and int yet
<FromGitter>
<stnluu_twitter> what is a "primary key" in CSV
<FromGitter>
<stnluu_twitter> first column? ID? all values are unique?
<FromGitter>
<randiaz95> yes
<FromGitter>
<stnluu_twitter> there you go. those are you if statements :)
<FromGitter>
<randiaz95> well not first column per se
<FromGitter>
<randiaz95> lol
<FromGitter>
<randiaz95> haha Luu
<FromGitter>
<randiaz95> I feel like writing all the code today anyways
<FromGitter>
<randiaz95> I am just used to finding libraries :D