vitallium has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<FromGitter>
<kingsleyh> @christopherzimmerman Yes I’ve been following V for a while - I’m writing a language in Crystal that compiles to V
_whitelogger has joined #crystal-lang
<FromGitter>
<dscottboggs_gitlab> I don't understand this error message https://carc.in/#/r/8cbx
coderobe has joined #crystal-lang
<Yxhuvud>
Methods defined in the toplevel scope also behave like that. What happens if you move the macro definition inside the struct?
<FromGitter>
<dscottboggs_gitlab> That compiles, but what's the point of that? I need to use it in multiple structs or...oh, I could include a module to do it
<FromGitter>
<Daniel-Worrall> Weird, it also does it for ::template
vitallium has joined #crystal-lang
Creatornator has joined #crystal-lang
Creatornator has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Creatornator has joined #crystal-lang
woodruffw has joined #crystal-lang
<FromGitter>
<stnluu_twitter> Does anyone want to give my very tiny shard a quick code review? :-) It’s a small lib that lets you manage concurrent spawns https://github.com/snluu/commander ⏎ ⏎ For context, I noticed that in the TechEmpower benchmark for Crystal, we could use some concurrency to speed up the multi-query test for Crystal significantly. Couldn’t find a built-in way to manage concurrency, so I plan to use my
<FromGitter>
<tenebrousedge> why would it not be? This is the whole point of having higher-order iterators at all. You should need to justify using lower-order constructs, which was why I asked about it. Using `times` is no better than using a `for` loop.
<FromGitter>
<stnluu_twitter> and why is a for loop bad for this scenario?
<FromGitter>
<dscottboggs_gitlab> sometimes a for- or while-loop is appropriate. what would he iterate over in this case? The results are in a Channel, not an enumerable
<FromGitter>
<tenebrousedge> on line 16, 1024 should be assigned to a constant
<FromGitter>
<tenebrousedge> `for` loops are never appropriate
<FromGitter>
<stnluu_twitter> because?
<FromGitter>
<dscottboggs_gitlab> 😒
<FromGitter>
<tenebrousedge> `times` is appropriate if there are no alternatives
<FromGitter>
<tenebrousedge> `for` can always be wrapped in something more sensible
<FromGitter>
<stnluu_twitter> let’s back up a bit. why is for loop not appropriate?
<FromGitter>
<dscottboggs_gitlab> well, for loops are never appropriate in crystal because it doesn't have it 😂
<FromGitter>
<stnluu_twitter> you mean like the while loop that comes with the core language?
<FromGitter>
<tenebrousedge> `while` loops are usually better wrapped in `Iterator` as well
<FromGitter>
<stnluu_twitter> @Boggs LOL
<FromGitter>
<tenebrousedge> if you want to write code that doesn't use higher-order iterators, go write C
<FromGitter>
<stnluu_twitter> then you might not want to use the Crystal stdlib
<FromGitter>
<dscottboggs_gitlab> or python, or go, or rust, or java, or javascript, or literally any language that isn't ruby and crystal
<FromGitter>
<dscottboggs_gitlab> I mean, don't get me wrong, chainable iterators are nice when you're working with a data structure that well supports it, but I don't see the harm in using it if it's what's there
<FromGitter>
<straight-shoota> @tenebrousedge I might have missed something, what's your argument `#times` vs. `#map`?
<FromGitter>
<tenebrousedge> if you don't have a data structure that supports that, then write support for it and use that
<FromGitter>
<tenebrousedge> `times` is a low-level construct that should only ever be used when no higher-order constructs are available
<FromGitter>
<tenebrousedge> using low-level constructs when higher-level constructs are available is stupid
<FromGitter>
<stnluu_twitter> 1) that's an opinion. i somewhat agree. ⏎ 2) available to do the job*; in this case, i don't have anything to iterate over in this case. what am I using "map" with?
<FromGitter>
<dscottboggs_gitlab> I guess that's fair but so is trying to shoehorn a high-level construct into a situation where simple iteration over `n` channels will suffice
<FromGitter>
<tenebrousedge> if it's not available for this purpose, use `times`
<FromGitter>
<tenebrousedge> which is why I asked
<FromGitter>
<straight-shoota> `#times` is the same layer of abstraction as `#map`
<FromGitter>
<stnluu_twitter> lulz
<FromGitter>
<tenebrousedge> `times` does not have a useful return value
<FromGitter>
<stnluu_twitter> > `times` does not have a useful return value ⏎ ⏎ you don't always need a return value. sometimes you just wanna do something N times
<FromGitter>
<straight-shoota> Exactly. They're different methods, obviously.
<FromGitter>
<straight-shoota> And are used for different tasks.
<FromGitter>
<tenebrousedge> `times` is indeed appropriate for situations where the return value is not useful
<FromGitter>
<straight-shoota> Obviously you can use them interchangeably since `x.times` is pretty much equivalent to `(0..x).map`
<FromGitter>
<dscottboggs_gitlab> ☝️ January 5, 2020 12:34 PM (https://gitter.im/crystal-lang/crystal?at=5e121e3f2af31b403d22dc9e) ⏎ ⏎ TBH idk. I was going to say a buffered channel but what I really think you need is some kinda dynamically buffered channel, like Array is to StaticArray
<FromGitter>
<dscottboggs_gitlab> does that make sense?
<FromGitter>
<stnluu_twitter> no lol
<FromGitter>
<dscottboggs_gitlab> then again, I'm kinda guessing and you should probably just benchmark it
<FromGitter>
<stnluu_twitter> > not sure if it’s better to default to a buffered channel or unbuffered channel ⏎ ⏎ the general use case for the `Commander` shard would be for IO scenarios. if I use an UNbuffered channel for the result, it will go back to the fiber that runs `#collect`
<FromGitter>
<stnluu_twitter> which might not be a bad thing
<FromGitter>
<stnluu_twitter> re map vs times, sounds like everyone is on the same page. lol
<FromGitter>
<stnluu_twitter> hmm, now that i think about it, an UNbuffered channel might be better, less abstraction on top
<FromGitter>
<vitallium> Hi. Does Crystal have something like `--dry-run` in RSpec for running (listing) specs but without executing them?
<FromGitter>
<stnluu_twitter> @dscottboggs_gitlab i tried benchmarking, for the cases I could think of the delta is pretty negligible
<FromGitter>
<straight-shoota> @stnluu_twitter buffered vs. unbuffered depends on whether dispatch and collect run asynchronously
<FromGitter>
<tenebrousedge> crystal's built-in spec is pretty bare-bones
<FromGitter>
<straight-shoota> When you're dispatching LOTS of tasks, and `@result_ch.send` blocks, the entire fiber is kept in memory until the results are collected.
<FromGitter>
<stnluu_twitter> @straight-shoota can you elaborate more? dispatch and collect will generally be called from the same fiber (at least that is the main use-case). Though, dispatch will always spawn a new fiber that `send` into said channel
duane has joined #crystal-lang
<FromGitter>
<straight-shoota> Oh yeah, it's not so much an issue whether they're called from the same fiber. Actually, calling dispatch after collect is going to break.
<FromGitter>
<stnluu_twitter> :nod: collect will close the channel and any dispatch call after will throw
<FromGitter>
<stnluu_twitter> you do have a good point about the whole fiber being in memory, though
<FromGitter>
<stnluu_twitter> vs. having a `Dequeue(T)` overhead
<FromGitter>
<straight-shoota> Yes, but it only closes *after* all results have been collected. As long as there's still a task pending, you can still dispatch.
<FromGitter>
<stnluu_twitter> true
<FromGitter>
<straight-shoota> The performance overhead is negligible. It's really about memory usage. Releasing the fibers as soon as possible is better than keeping them around.
<FromGitter>
<stnluu_twitter> In that case, I might want to let the caller specify a default capacity for `@result_ch`, then
<FromGitter>
<stnluu_twitter> it was late last night and I just hardcoded it to 1024 and went to bed ^^
<FromGitter>
<tenebrousedge> if there's a sensible reason for 1024 being the value, it should be assigned to a constant. Otherwise it should be a parameter
<FromGitter>
<stnluu_twitter> @straight-shoota I don't expect dispatch & result to be called by two different fibers. so I'm okay with closing the channel AFTER collecting the result.
<FromGitter>
<straight-shoota> Yes definitely. For small numbers it doesn't matter much, but when you're dispatching thousands of fibers, it's going to pay off. But you would need to know beforehand, how many results to expect, at least the order of magnitude.
<FromGitter>
<vitallium> @tenebrousedge Thanks but I need this for built-in spec. Will create a workaround in this case.
coderobe has quit [Quit: '); DROP TABLE users;--]
<FromGitter>
<stnluu_twitter> @straight-shoota on that note, it would be useful to have a language construct that prevents an object from being copied into a different fiber :)
<FromGitter>
<straight-shoota> @vitallium You can probably quite easily hook into the spec implementation.
<FromGitter>
<stnluu_twitter> basically the reverse of "Sync/Send" in rust, where if you have that annotation/signature you can't be used in a different "thread"
<FromGitter>
<straight-shoota> Btw. you should probably consider adding an extra collect fiber which asynchronously receives results from the results channel and puts them in a dedicated data structure that can grow dynamically.
<FromGitter>
<straight-shoota> Buffered channels are not very practical for this.
<FromGitter>
<stnluu_twitter> that's a good idea
<FromGitter>
<straight-shoota> I also noticed that `#collect` is going to raise when a task returns `nil`.
<FromGitter>
<straight-shoota> I think it would be good to support nil results.
<FromGitter>
<stnluu_twitter> it does support nil actually. collect should only raise if the exception is explicitly set
<FromGitter>
<stnluu_twitter> let me add a spec
<FromGitter>
<stnluu_twitter> any easy way to wait for a fiber to finish without adding yet another channel to signal back to the calling fiber?
<FromGitter>
<straight-shoota> > any easy way to wait for a fiber to finish without adding yet another channel to signal back to the calling fiber? ⏎ ⏎ You could technically use a flag and wait for it. But a channel is probably better.
<FromGitter>
<stnluu_twitter> > Btw. you should probably consider adding an extra collect fiber which asynchronously receives results from the results channel and puts them in a dedicated data structure that can grow dynamically. ⏎ ⏎ on another thought, the current impl of `collect` basically does that in the calling fiber.
<FromGitter>
<stnluu_twitter> any reason you're recommending having a dedicated fiber?
coderobe has joined #crystal-lang
<FromGitter>
<straight-shoota> It needs to run concurrently with the dispatched fibers
<FromGitter>
<stnluu_twitter> doesn't the main fiber run concurrently with the dispatched fibers in this case?
<FromGitter>
<straight-shoota> Yes
<FromGitter>
<stnluu_twitter> then maybe i'm mising something. not sure why we would need to spawn a dedicated fiber to collect
<FromGitter>
<straight-shoota> That's a simple case. In a real world application, it's likely that the dispatches don't happen more or less simultaneously followed instantly by a collect.
<FromGitter>
<stnluu_twitter> i see what you mean. let me think more on that front. in that case, i *think* that a Buffered `@result_ch` would mitigate most of the memory footprint
<FromGitter>
<stnluu_twitter> assuming it's large enough
<FromGitter>
<stnluu_twitter> @straight-shoota is `typeof(T)` expensive?
<FromGitter>
<stnluu_twitter> aka is it worth doing that up front
<FromGitter>
<straight-shoota> typeof is evaluated at compile time, so no cost
<FromGitter>
<stnluu_twitter> nice
<FromGitter>
<stnluu_twitter> what I have there to support nil result is "pragmatic Crystal" right?
<FromGitter>
<straight-shoota> You can just compare `T == Nil` but in a macro
<FromGitter>
<stnluu_twitter> 👍
<FromGitter>
<btihen> Hello, nice discussion (wish I understood more). Regarding Channels - I found an odd case that I don't understand. ⏎ when sending tons of messages - I get channel exeptions when I switch between async, sync, async. ⏎ ⏎ ``` code paste, see link``` [https://gitter.im/crystal-lang/crystal?at=5e122887809bc77efef3ae49]
<FromGitter>
<btihen> When I send a ton of async then sync all is ok - but not the reverse - is this related to the Channel Buffering? ⏎ ⏎ ``` code paste, see link``` ⏎ ⏎ This works - first doesnt [https://gitter.im/crystal-lang/crystal?at=5e12289f316c1473271eeacc]
<FromGitter>
<j8r> dumb questions, what's the diff between `typeof()` and `#class`?
<FromGitter>
<tenebrousedge> `Array(Int32)` vs `Array`
<FromGitter>
<btihen> yes it is the `spawn` line. Oddly - this only happens when I switch back and forth between spawn and not spawn. I can do as many sets of channel.sends as I want -- as long as they are all sent the same way. (
<FromGitter>
<stnluu_twitter> read my last message. sounds like that is what you're running into
<FromGitter>
<stnluu_twitter> remember, there is only one green OS thread in your case. the underlying runtime scheduler maneuver around different fibers for best CPU usage.
<FromGitter>
<stnluu_twitter> what i *think* is happening: ⏎ ⏎ if you only do `spawn send`, the scheduler will go to your `listen_for_message` loop immediately when you send, and then eventually run `spawn close`
<FromGitter>
<stnluu_twitter> if you do `send (none-spawn)` only, the scheduler, same thing
Creatornator has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<FromGitter>
<stnluu_twitter> when you mix `spawn send`, the scheduler might go back to the main fiber, perform `send (non-spawn)`, which the channel is blocked/full at this point because it's not buffered. so the scheduler goes back to the main fiber again, execute the `spawn close`
<FromGitter>
<stnluu_twitter> just educated guess. you might want to write out log messages to see what's happening
<FromGitter>
<btihen> Yeah -cool ! I just saw your message. Even a small buffer of 2 fixes the problem. I'm guessing the buffer size depends on how often I switch sending methods.
<FromGitter>
<stnluu_twitter> though, fwiw, the call pattern introduced by your stressed test is a bit abnormal
<FromGitter>
<btihen> yeah - I figured - it seems the simplest is to use spawn for all the communication with the other fibers - I had the fewest errors doing that. I was just try to stress my code and see what happens and understand concurrency better. I'm still in the 3-yr old testing (build and knock over, build and knock over). Thanks
<FromGitter>
<stnluu_twitter> 👍
<FromGitter>
<stnluu_twitter> have fun! :D
<FromGitter>
<stnluu_twitter> I'm trying to run the techempowered benchmark locally (with Postgres), but for the life of my I can't understand why `pick_available` from `DB::Pool` would throw an "Enumerable::Empty" error: ⏎ ⏎ ``` code paste, see link``` [https://gitter.im/crystal-lang/crystal?at=5e124bfb809bc77efef4acb3]
<FromGitter>
<stnluu_twitter> pick available just simply calls `@idle.first?`, which should just return `T | Nil`
<FromGitter>
<tenebrousedge> @stnluu_twitter the `?` should definitely avoid that error. Can you link to where that's defined? I'd like to try to write a minimal test case
<FromGitter>
<stnluu_twitter> also, there's a strange behavior in the connection pool
<FromGitter>
<stnluu_twitter> after a good amount of request, the pool seem to be just gone
<FromGitter>
<stnluu_twitter> the idle pool that is
<FromGitter>
<stnluu_twitter> and subsequent requests timeout waiting for available pool, even though the code is using `db.using_connect`, which guarantees calling `conn.release`
<FromGitter>
<stnluu_twitter> thats what im debugging at the moment
<FromGitter>
<tenebrousedge> hmmm, can't reproduce with a minimal test case
<FromGitter>
<stnluu_twitter> let me publish my changes and you can clone them
<FromGitter>
<stnluu_twitter> oh, you need postgres setup tho
<FromGitter>
<dscottboggs_gitlab> your whole class needs to be a within the macro
<FromGitter>
<dscottboggs_gitlab> you could also put `{% begin %}` before the class and `{% end %}` after it to acheive the same effect I believe
<FromGitter>
<stnluu_twitter> @tenebrousedge Kai, are you able to repro?
<FromGitter>
<tenebrousedge> installing vagrant
<FromGitter>
<stnluu_twitter> oh haha, i thought I you "a big repro"
<sorcus>
dscottboggs_gitlab: i get error "Error: mapping argument must be a HashLiteral or a NamedTupleLiteral, not TupleLiteral", with `{% begin %}` i get "Error: for empty hashes use '{} of KeyType => ValueType'"
<FromGitter>
<dscottboggs_gitlab> what happens if you put the `class Test` definition before `class Bar` in the file?
<FromGitter>
<dscottboggs_gitlab> gotta go out for a bit sorry
<sorcus>
dscottboggs_gitlab: Oh, yes. My fault. This works now.
<sorcus>
dscottboggs_gitlab: Thank you soooo much. *HAPPY*
<FromGitter>
<tenebrousedge> @stnluu_twitter ah, well. That seems to be the peccant point then
<FromGitter>
<stnluu_twitter> thanks for your help
<FromGitter>
<tenebrousedge> anytime
watzon has left #crystal-lang ["ERC (IRC client for Emacs 26.3)"]
Creatornator has joined #crystal-lang
<FromGitter>
<dscottboggs_gitlab> np @sorcus
HumanG33k has joined #crystal-lang
<FromGitter>
<stnluu_twitter> aight, fingers crossed, maybe we can show off Crystal performance compared to other languages (right now we are losing to Dart in the multi query test). https://github.com/TechEmpower/FrameworkBenchmarks/pull/5370 ⏎ ⏎ I only updated for the Kemal framework. if you have time it's a fun exercise to update for the other frameworks too
Creatornator has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<FromGitter>
<j8r> imo this isn't a big deal
<FromGitter>
<j8r> if a language is fast and light enough, it's good
<FromGitter>
<j8r> what's matter the features which a language provide
<FromGitter>
<tenebrousedge> the people who are not so enlightened look at benchmarks to judge languages, and some people even choose to use a language based on how fast a language is. Languages with many people working on them also tend to have more people working on speed improvements, so good performance can (in a sense) breed better performance
alex`` has quit [Quit: WeeChat 2.7]
<FromGitter>
<j8r> it's always good, sure, but not the most important
<FromGitter>
<j8r> considering a set of languages "fast enough"
<FromGitter>
<j8r> nowadays, there is plenty of choice
<FromGitter>
<j8r> productivity is what will bring more people
uu91 has quit [Remote host closed the connection]
<FromGitter>
<stnluu_twitter> it's not a big deal, but like @tenebrousedge said, if you want more people contributing to a language, you gotta show off all of its axis', from readability, aesthetic, performance, package management, ecosystem. etc.
<FromGitter>
<stnluu_twitter> and on the other hand, the specific use-case that i was "fixing up", is a real world use case where you wanna fire off many concurrent IO calls
<FromGitter>
<stnluu_twitter> i like the techempowered benchmark because the test cases are pretty "real life"