ChanServ changed the topic of #crystal-lang to: The Crystal programming language | | Fund Crystal's development: | GH: | Docs: | Gitter:
<postmodern> kind of wish there was a .not_nil! that accepted a block. The crystal compiler can't infer that `unless @ptr.nil?` implies @ptr is not `Pointer | Nil` within the `unless`.
<FromGitter> <Blacksmoke16> `@ptr.try { |ptr| }`?
<postmodern> awesome, exactly what i needed
<postmodern> oprypin, just curious if you have a link handy for how you test the #finalize method?
<FromGitter> <> postmodern, u mean like unittests? i don't, lol
<FromGitter> <> u can print something to the screen if you just want to try it out
<postmodern> was thinking i'd probably have to use mocks/stubs, but then that gets complex.
<postmodern> can you call other instance methods from within #finalize ?
<FromGitter> <Blacksmoke16> i dont see why not, its just an instance method
<FromGitter> <> postmodern, the answer to *all* such "can you" questions is the same as "can you do it from within #foo ?"
<FromGitter> <> the method is completely normal and the state of the object is completely normal. *and then* the memory deallocation happens
deavmi has quit [Ping timeout: 272 seconds]
<repo> hey, quick question: are HTTP::Clients thread safe?
<FromGitter> <Blacksmoke16> i doubt it
<repo> mh
<repo> that's a bummer :D
<repo> what's the constant for the number of crystal threads again?
<FromGitter> <Blacksmoke16> `CRYSTAL_WORKERS`
<FromGitter> <> repo, why not just create a new client every time
<repo> because i want to reuse the client
<repo> thanks Blacksmoke16
<FromGitter> <> repo, what do you mean lol, "i want" is not a reason
<repo> tls handshakes are slow
<FromGitter> <> oo does it reuse established tls connection? nice
<FromGitter> <> anyway, it's super not thread safe
<repo> sure
<repo> no prob. will just create one for each thread then
<FromGitter> <> you could also use a pool
<FromGitter> <> not sure if HTTP::Client is fiber-safe
<FromGitter> <Blacksmoke16> its not
<FromGitter> <> only one request has to be done at a time
<FromGitter> <> but it doesn't need to be fiber safe
<FromGitter> <> yes a pool is the right solution. it doesn't need threads at all
<repo> is a pool something built into crystal?
<repo> or do you generally just mean a thread pool
<FromGitter> <> no
<FromGitter> <Blacksmoke16> to be clear threads and fibers aren't the same thing
<FromGitter> <> repo, threads are only a distraction, although sure, in a roundabout way you *can* arrive to some solution using them
<repo> right
<repo> yeah
<FromGitter> <> u need a pool of clients and so a pool of fibers running them
<FromGitter> <> CRYSTAL_WORKERS is irrelevant in the same way cuz it's about threads
<FromGitter> <> j8r ( i know a pool and I'm looking at and i have no idea what i can do with it
<repo> but shouldn't my pool size be CRYSTAL_WORKERS?
<repo> something like this?
<repo> uh, still @@
<FromGitter> <> CRYSTAL_WORKERS is irrelevant in the same way cuz it's about threads
<FromGitter> <> Crystal can drive 100 connections at the same time doesn't matter if it's 1 thread or 4 threads
<repo> orypin: how would you set the pool size then?
<FromGitter> <> just how many concurrent connections you want!
<repo> mhm
<FromGitter> <> CRYSTAL_WORKERS == 1 FYI
<FromGitter> <> repo, it really really looks like you need @clients = {} of {String, Bool} =>
<repo> looks cool, will give it a try
<FromGitter> <> Dude no neet to reinvent the pool repo ;)
<FromGitter> <> It is *very* basic though
<repo> yeah :D
<repo> didn't know about that shard though
<FromGitter> <> It is quite new yes. There is also crystal-db which has a more advanced one
<repo> ah cool it lazily creates the items
<repo> what's the default idle size?
<repo> "Note that if an exception was raised in the block, the resource won't be added back. This behavior is used to prevent adding an invalid resource, for example a connection causing an IO error."
<repo> ah sweet
<FromGitter> <> i think it instantiates all N connections from the start
<repo> that's really useful. thanks!
<repo> ok
<FromGitter> <> That's right
<FromGitter> <> I prefered a safe by default behavior, vs fast and unsafe.
<repo> about right?
<FromGitter> <> What's missing is a logic of resizing the pool in case of bursts, to not keep a big pool
<repo> ok
<FromGitter> <> I find hard to read `@clients = {} of {String, Bool} => ::Pool(HTTP::Client)` ^.^'
<repo> yeah you're right
<FromGitter> <> well that's what it is, what's hard?
<repo> hm would you consider returning the yield return value from get?
<repo> now i need to write a wrapper around it in another instance
<FromGitter> <> oprypin ( it looks like a Array/Hash/Tuple mix, not used to
<FromGitter> <> Also why a hash repo?
<FromGitter> <> *<repo>* hm would you consider returning the yield return value from get? ⏎ seems like a good idea to me
<FromGitter> <> oprypin ( I read: "Hash of String, Bool" then "=> Pool(HTTP::Client)"?
<FromGitter> <> def get(& : T -> U) : U
<repo> yeah ^
<FromGitter> <> ah j8r ( you're right
<FromGitter> <> Hash( ).new syntax would be clearer
<repo> yeah
<repo> i changed it to Hash( ).new
<FromGitter> <> Sounds good
<FromGitter> <> There is a class_getter still sticking around
<FromGitter> <> Generally, if you can, better avoiding those for testability
<repo> mhm how come?
<FromGitter> <> For example using a getter in a class passed around
<FromGitter> <> Another little thing is you can avoid 2 hash lookups in with_client_for
<FromGitter> <> 1) Fetch the value to a var 2. If nil, mutex.synchronize and add the value to the hash
<repo> hm but wouldn't it be possible to read from the hash while another fiber was writing to it?
<repo> thus resulting in a race condition
<FromGitter> <Blacksmoke16> thats what `mutex.synchronize` is for
<repo> yeah sure
<repo> ah now i get it
<repo> yeah you're right
<FromGitter> <> Only needed on writes
<FromGitter> <> Because the fetched Pool has already a mutex mecanism to prevent concurent use of HTTP::Clients
<FromGitter> <> Hum, not 100% sure, a race condition might happen when the hash is written and read at the same time?
<FromGitter> <> Seems unlikely
<repo> yeah that was my fear
<FromGitter> <> Anyway do you really need a hash?
<FromGitter> <> The domains and tls varies?
<FromGitter> <> No need for the ||= in your code. I guess you're right, putting the fetch and add on a mutex will be safer...
<repo> yeah it varies
<repo> i'm writing a service for on the fly image scaling from arbitrary (white listed) urls
<FromGitter> <> The hash can only grow though...
<repo> yeah but in any sane environment you will white list a few urls (or patterns)
<FromGitter> <> If this urls are known at compile-time, you could avoid all this mutex/logic thing
<repo> they're not :)
<repo> also i allow patterns
<repo> something like `/^.*\.my-domain\.com$/`
<FromGitter> <> I'm afraid of having an app growing in memory
<FromGitter> <> Anyway, premature optimization for this thought
<repo> :)
<repo> i guess
<repo> it turned out pretty neat though. handles caching of both originals and conversions based on cache control and expires headers and the params of the conversions
<repo> needs an s3 or minio though
<FromGitter> <> Anyone tried ?
<FromGitter> <> We could have a Crystal binary which runs everywhere, providing the C libs use Cosmo too.
<repo> that's pretty crazy
deavmi has joined #crystal-lang
aquijoule__ has joined #crystal-lang
aquijoule_ has quit [Ping timeout: 246 seconds]
f1reflyylmao has joined #crystal-lang
f1refly has quit [Ping timeout: 264 seconds]
chachasmooth has quit [Ping timeout: 264 seconds]
chachasmooth has joined #crystal-lang
avane has quit [Quit: ZNC -]
chachasmooth has quit [Ping timeout: 240 seconds]
chachasmooth has joined #crystal-lang
deavmi has quit [Ping timeout: 272 seconds]
deavmi has joined #crystal-lang
_ht has joined #crystal-lang
deavmi has quit [Ping timeout: 246 seconds]
deavmi has joined #crystal-lang
f1refly has joined #crystal-lang
f1reflyylmao has quit [Ping timeout: 246 seconds]
f1refly has quit [Quit: bye fags]
f1refly has joined #crystal-lang
deavmi has quit [Ping timeout: 272 seconds]
deavmi has joined #crystal-lang
postmodern has quit [Quit: Leaving]
<raz> sounds too good to be true. but if it works, that would be amazing
<FromGitter> <djberg96> confused about ECR.embed and relative paths
<FromGitter> <djberg96> I've got a project with `src/` and it uses `ECR.embed` with templates located under `src/templates`
<FromGitter> <djberg96> i originally did `ECR.embed("./src/templates/check_sizeof.ecr", io)` thinking the "./" would make it relative to its calling location, but that doesn't seem to work if a dependency tries to use it
<FromGitter> <djberg96> what's the right way? use *DIR*?
<FromGitter> <djberg96> nm, figured it out
Xeago has joined #crystal-lang
deavmi has quit [Read error: Connection reset by peer]
deavmi has joined #crystal-lang
<FromGitter> <roduquen> Hi guys, someone is using cryomongo ?
<repo> hey!
<repo> raz: yeah i am
<repo> uh roduquen
<repo> quick question: is it possible somehow to free up a crystal object without having to wait for GC?
<FromGitter> <naqvis> free up in what sense?
<FromGitter> <> you usually not have to do this
<repo> i allocate a lot of memory really quickly and it seems the gc can't keep up
<repo> can i somehow use two shards with the same name? e.g. alias them or something?
<repo> i use j8s/pool but also ysbaddaden/pool
<repo> the latter is a dependency of crystal-redis
<FromGitter> <Blacksmoke16> are the types not namespaced?
<FromGitter> <Blacksmoke16> nope, you're out of luck
<repo> :(
<repo> that's really annoying
<FromGitter> <Blacksmoke16> mhm, this is why namespacing stuff is useful
<FromGitter> <naqvis> you can consider using `WeakRef` and invoking `GC.collect` to request GC to collect the garbage
<repo> naqvis: mh ok but that would result in a complete GC run, right?
<FromGitter> <naqvis> true
<FromGitter> <naqvis> if you need selective control then you better use custom allocator and de-allocator
<FromGitter> <naqvis> but that definitely beats the purpose of GC
<FromGitter> <naqvis> for memory intensive operations, you can consider having custom allocator and de-allocator
<FromGitter> <> btw why do you *need* this repo?
<repo> naqvis: how would that work?
<FromGitter> <Blacksmoke16> the other pool lib is a dep of a redis shard
<repo> jrei: i'm converting images, which requires me to load the entire image into memory
<FromGitter> <> @Blacksmoke16: you mean the redis shard depends on the pool of crystal-db? Yeah
<FromGitter> <> why do you say that?
<repo> it conflicts
<FromGitter> <> repo: ok, but let the GC deal with this
<repo> jrei: the process get's killed
<FromGitter> <Blacksmoke16> repo, can you not do that and use `IO`s?
<FromGitter> <> you know sometimes a GC is even faster than malloc/free a lot
<repo> Blacksmoke16: i wish
<repo> but no
<FromGitter> <Blacksmoke16> rip
<FromGitter> <> not sure it is because of the GC...
<repo> yeah sure, could be something else
<FromGitter> <naqvis> > *<repo>* @naqvis: how would that work? ⏎ ⏎ Take a look at `GC` module and methods of interest to you would be `malloc_atomic` and `free`
<FromGitter> <> try calling GC.collect in strategic places if it imrpoves the situation
<repo> how long would a GC run usually take?
<repo> i understand it's a very unspecific question
<repo> i'm just looking for a ballpark figure
<FromGitter> <> depends of how many objects, and how much work it has to do
<FromGitter> <> that's just to know if it is because of the GC
<FromGitter> <> maybe you're reaching the max memory of a crystal program
<FromGitter> <> AFAIK there is one
<repo> jrei: not during a single conversion
<repo> it happens when a lot of them come in
<FromGitter> <naqvis> AFAIK Boehm GC used by Crystal is MT enabled and GC takes up all other cores except 1 (which is used by Crystal main thread - assuming you are running crystal in ST mode), so whole GC process should be quite fast
<FromGitter> <> repo: > Pause times below 1/10th of a second are often the case, and applications with relatively small live sets (or slowly changing live sets, for generational collector) can obtain pause times below 1/100th of a second.
<FromGitter> <naqvis> thing you should consider is the scope of variables your program is holding to reference to memory
<FromGitter> <naqvis> as GC won't be collecting memory to which there is still a valid reference
<repo> that's clear
<FromGitter> <> bdwgc is compiled with `--enable-large-config`
<FromGitter> <naqvis> so you were suggested to consider use `WeakRef` or similar approach
<FromGitter> <> maybe in your case using multiple app instances will be better than multi-thread...
<FromGitter> <> one app per core, more so if the app is relatively-stateless
<FromGitter> <> and have a Load Balancer in front
<FromGitter> <> it will be more stable (MT still in preview), faster, and higher availability in case of a crash
_ht has quit [Remote host closed the connection]
<FromGitter> <rukkiddo> I want to generate secure session ids for my cookies, I am planning something like sha256 of ip + username + timestamp + (random string or uuid)
<FromGitter> <rukkiddo> is this secure enough?
<FromGitter> <Blacksmoke16> prob not
<FromGitter> <Blacksmoke16> could just generate a JWT or something, kill 2 birds with one stone
<FromGitter> <Blacksmoke16> ah nvm, missed last uuid arg, that would help
<oprypin> Blacksmoke16, depends if that string is static or not. seems like the suggestion is static. cuz if it's random every time then what's the point of all the other parts lol
<FromGitter> <Blacksmoke16> i should hope it wouldnt be static
<FromGitter> <Blacksmoke16> but yea, good point
<FromGitter> <rukkiddo> it wont be static, I thought I would add it to make sure it is unique
<FromGitter> <rukkiddo> i actually read from stackoverflow that php does it like this
<FromGitter> <rukkiddo> let me rephrase
<FromGitter> <rukkiddo> i mean random string is not static
<FromGitter> <rukkiddo> other parts supposed to make it unique
<FromGitter> <> just uuid will be enough
<FromGitter> <> the collision probability is fairly small, unless you're designing a system at a big scale
<FromGitter> <rukkiddo> probably i will never have a collision since my system is not big
<FromGitter> <rukkiddo> but still it is bugging me
<FromGitter> <rukkiddo> uuid already takes timestamp into calculation right? it already will be unique
<FromGitter> <rukkiddo> I didnt know there are verisons of uuid actually
<FromGitter> <rukkiddo> not all versions have timestamp
<FromGitter> <rukkiddo> "the probability to find a duplicate within 103 trillion version-4 UUIDs is one in a billion. "
<FromGitter> <rukkiddo> I will still make the concat, better safe than sorry lol
<FromGitter> <renich_gitlab> Why doesn't this work? ⏎ ⏎ ```x = [] of Array(Int64 | String | Nil) ⏎ x << [nil, ""]``` []
<FromGitter> <renich_gitlab> ... on v0.35.1
<FromGitter> <> @renich_gitlab: cuz `[nil, ""]` is type `Array(String | Nil)` which isn't `Array(Int64 | String | Nil)`
<FromGitter> <renich_gitlab> Ah, damn...
<FromGitter> <> `[nil, ""] of (Int64 | String | Nil)` is type `Array(Int64 | String | Nil)`
<FromGitter> <> @renich_gitlab:
<FromGitter> <renich_gitlab> How can I make this work then? ⏎ ⏎ ```x = [] of Array(Int32? | String) ⏎ x << [nil, ""] ⏎ x << [1, "Something"]``` []
<FromGitter> <> like i said in my last message
<FromGitter> <renich_gitlab> Oh
<FromGitter> <> if it confuses you less: `Array(Int64 | String | Nil)["", "nil"]`
<FromGitter> <> interestingly, this is invalid: `[] of [] of Int32? | String`
<FromGitter> <> I never licked the `of` syntax
<FromGitter> <> j8r ( ok but u mistyped that
<FromGitter> <> `Array(Int64 | String | Nil){"", nil}`
<FromGitter> <> right!
<FromGitter> <> In my mind `[] of String | Nil` can be `Array(String) | Nil` or `Array(String | Nil)`
<FromGitter> <renich_gitlab> This doesn't work either: ⏎ ⏎ ```x = Array(Int32 | String | Nil){"", nil} ⏎ x << [nil, ""] ⏎ x << [1, "Something"]``` []
<FromGitter> <> @renich_gitlab: it's the exact same code
<FromGitter> <> ```code paste, see link``` []
<FromGitter> <> dude, we meant that's `x << Array(Int32 | String | Nil){"", nil}` ;)
<FromGitter> <renich_gitlab> OK, that worked. Why do you have to say: `of Int32? | String` every time?
<FromGitter> <> yeah same as using `[] of ...`
<FromGitter> <> If I where you, I used a generic
<FromGitter> <renich_gitlab> What do you mean?
<FromGitter> <> like `alias T = Int32 | String | Nil`, then `x << [nil, ""] of T`
<FromGitter> <> same, just less verbose
<FromGitter> <> (personal take)
<FromGitter> <Blacksmoke16> I think he meant alias
<FromGitter> <Blacksmoke16> Not a generic
<FromGitter> <> yes
<FromGitter> <> sorry
<FromGitter> <> anyway, there is an issue around AFAIK