<FromGitter>
<j8r> Not experienced enough, I heard Redis could be used as a basic queue
<FromGitter>
<Blacksmoke16> Yea, that's what mosquito uses
oddp has quit [Ping timeout: 256 seconds]
<FromGitter>
<Blacksmoke16> dont have anything set in stone, just brainstorming architecture ideas
<FromGitter>
<s0kil> @j8r In stdlib, you referring to WebSocketHandler?
<postmodern>
does crystal shards embed the shard versions into the resulting binary? wondering if there's a way to audit that info?
<FromGitter>
<Blacksmoke16> into the binary you build with `shards builds`?
<FromGitter>
<Blacksmoke16> build*
<FromGitter>
<rmarronnier> Hey all, ⏎ I must have missed something : ⏎ ``` puts Time.new ``` doesn't work anymore and neither do ```puts Time.now```
<postmodern>
Blacksmoke16, correct. Say some shard has a vulnerability and an advisory is published. I installed a binary build with crystal+shards, could I audit whether the binary contains the vulnerable shard?
<FromGitter>
<Blacksmoke16> Time.local
<FromGitter>
<Blacksmoke16> Or Time.utc
<FromGitter>
<rmarronnier> Thanks :-)
<FromGitter>
<Blacksmoke16> To be clear shards build is essentially just a wrapper around crystal build
<FromGitter>
<Blacksmoke16> So I would think it would be better to check the shards and or lock file of the version of the binary you have
<postmodern>
Blacksmoke16, could I somehow get the full or relative path of the shards to infer the shard name-version?
<FromGitter>
<Blacksmoke16> I doubt it
<postmodern>
or could we change shards build embed the shards.lock file?
<FromGitter>
<Blacksmoke16> You could, I don't think that should be default behavior
<postmodern>
Blacksmoke16, it would make identifying vulnerable crystal binaries easy, which is something enterprise environments typically insist on
<postmodern>
if it was possible to embed the shards.lock, i could port bundler-audit and ruby-advisory-db over to crystal
<FromGitter>
<Blacksmoke16> but wouldnt it be better to check the version of binary. i.e. `./my_app --version` says `0.1.2`
<FromGitter>
<Blacksmoke16> you would know what it was built with (for the most part) by checking the related files for that project
<FromGitter>
<Blacksmoke16> Versus coupling the binary with shards
<postmodern>
Blacksmoke16, this is the same problem Electron apps face. Each time Google Chrome or some of the JS libraries they vendor in has a vulnerability/advisory, the apps have to publish their own advisory and alert their users, which multiplies the work.
<postmodern>
if you can identify which installed crystal binaries contain vulnerable shards, this gives more piece of mind than depending on the binary maintainer to publish an advisory and reach out to their end users
<FromGitter>
<Blacksmoke16> But how would knowing what libs a binary was built with help? A new version of the application would need to be released anyway, which would result in a new binary
<postmodern>
Blacksmoke16, yes, it would help you identify what's vulnerable, instead of wondering whether a binary was or wasn't vulnerable.
<FromGitter>
<Blacksmoke16> More so a better reason to use version constraints
<FromGitter>
<Blacksmoke16> I mean in a perfect world you go look at the lock file for the version of the application you're using and find out
<FromGitter>
<Blacksmoke16> imo this is more of a shards build type of problem. I.e. that you're building something with a known vulnerability
<FromGitter>
<Blacksmoke16> dont really need to know what libs a binary includes as that can easily be determined from the application at the specific version of the binary
<postmodern>
that's a good start
<postmodern>
post-facto auditing would calm the worries of CISOs when trying to pitch crystal to them. "just run this tool on the binaries"
<FromGitter>
<Blacksmoke16> Does go or anything have something like it?
<postmodern>
a discussion on twitter shows that it's possible with rust's cargo, which got me thinking
<FromGitter>
<didactic-drunk> How do you audit closed source binaries for open source shards vulnerabilities? I'm with @postmodern on this one. Better and easier auditing is helpful rather than trying to match arbitrary binaries to a particular build source which may not be available.
f1reflyylmao has joined #crystal-lang
f1refly has quit [Ping timeout: 244 seconds]
<postmodern>
also how do you audit binaries when the shard advisory was released afterwards, and the binary maintainer is MIA and won't publish an alert to their users to upgrade.
<FromGitter>
<Blacksmoke16> Wonder if llvm has something like docker labels
<postmodern>
oh there's an idea
<postmodern>
my other wish, which none of the other languages support, is delta-patching of the compiled in "modules"
<postmodern>
i suspect that might not be possible with languages that support Generics, and you basically have to recompile the whole thing
sagax has quit [Quit: Konversation terminated!]
<FromGitter>
<ejstembler_gitlab> Hi. I have an array of structs (`Array(Post)`) which I'm saving to Redis (`redis.set("recent-posts", recent_posts, 60 * 5)`) which works fine. If I get that back out of Redis it's a String. Is there a way to deserialize that back into my `Array(Post)`?
<FromGitter>
<ejstembler_gitlab> Never mind. I got it to work with YAML serialization/deserialization. Though I do wonder if there's a more efficient way of doing this?...
_whitelogger has joined #crystal-lang
_whitelogger has joined #crystal-lang
_whitelogger has joined #crystal-lang
<FromGitter>
<galvertez> so it's more like trying to crash the compilation when a constant isn't defined in a child class
<FromGitter>
<galvertez> abstract classes don't work for this, and i don't want it to be done with a method because then that's a method just to access a constant. but the constant *absolutely should* be defined in all child classes because it will be used to define behavior in the parent class
<FromGitter>
<galvertez> so it isn't really magic, exactly
<FromGitter>
<galvertez> to be specific, i am trying to write up a system whereby crystal can serialize instances in binary and send them over the wire. but i want it to be possible for other languages to be able to de-serialize the instances as well, in case there is a comparable library
<FromGitter>
<galvertez> so the class needs a constant defined so that it can indicate its type by writing it's type number to the `IO` first. next, it can write the instance variables in a particular order, but the order can't be known until you have determined the type
<FromGitter>
<galvertez> if we had a good protobuf library i'd be using it, but the one we have isn't that great, and it seems dead at the moment and i don't have time to write my own
<FromGitter>
<galvertez> i guess i should just `abstract def` and be done with it :)
<FromGitter>
<galvertez> OTOH i could just insist that users `ParentClassFromMyLib.register(ChildClass1, ChildClass2, ChildClass3)` or something like that
<FromGitter>
<galvertez> and that `register` would be a macro that, when called, would define the method for the parent class for parsing
<FromGitter>
<galvertez> then there's no magic, it's all explicit
yxhuvud has quit [Remote host closed the connection]
yxhuvud has joined #crystal-lang
_whitelogger has joined #crystal-lang
oddp has joined #crystal-lang
postmodern has quit [Quit: Leaving]
_whitelogger has joined #crystal-lang
deavmi has quit [Read error: No route to host]
deavmi has joined #crystal-lang
<FromGitter>
<Blacksmoke16> `recent_posts.to_yaml` would prob be better
<FromGitter>
<Blacksmoke16> er thats actually what `dump` does, just an alias i guess
<FromGitter>
<phykos> In Ruby there is `Thread#new` and `Thread#join`, in Crystal what should I use?
<oprypin>
it would be good to ask what functionality you need and especially the goal you have in mind.
<FromGitter>
<wyhaines> @phykos The functional equivalent of `Thread#new` is `spawn`. For waiting for completion, the typical idiom is to use a channel, as illustrated in the link that @oprypin just gave.
<FromGitter>
<phykos> oh k
<FromGitter>
<phykos> maybe I can create something like pthread bindings for Crystal
<FromGitter>
<phykos> it is a good idea?
<oprypin>
phykos, no, combining that with almost any part of crystal stdlib it will horribly break
<FromGitter>
<wyhaines> There is full threading support, but it is still at this time considered experimental. Compile your code with `-Dpreview_mt` and it will enable it. A lot of the standard library isn't currently threadsafe, though, so test test test test.
<oprypin>
that's of course assuming you actually need threads
<FromGitter>
<j8r> `sizeof()` can't be executed in macros?!
<oprypin>
j8r, hmmm yea i guess not
<FromGitter>
<phykos> > There is full threading support, but it is still at this time considered experimental. Compile your code with `-Dpreview_mt` and it will enable it. A lot of the standard library isn't currently threadsafe, though, so test test test test. ⏎ ⏎ k
<oprypin>
wanna try to look for a workaround
<FromGitter>
<j8r> Maybe someone has an idea on this: basically, I have a list/tuple of integers floats
<FromGitter>
<j8r> From this, I need to convert this to Bytes
<FromGitter>
<wyhaines> @phykos If your workload is largely IO bound, you may be surprised by how much throughput you can get even from unthreaded Crystal. One of the niceties of threading with Crystal, though, is that you don't have to learn a special syntax or write special code. It's just Fibers. So, you create new threads/fibers with `spawn`. Ideally, you handle communication between them with a Channel. It's not the Ruby model,
<FromGitter>
<j8r> Hum... I guess I could... Being use in the websocket handler, I is necessary to share the same IO for multiple parallel connections of course
<oprypin>
whatever works
<FromGitter>
<j8r> I meant, the same IO can't be shared for multiple connections
<FromGitter>
<j8r> Ok, I will do that
<FromGitter>
<j8r> however, that not really suited if I want to send 8 bytes and IO::Memory is like 1024 in size?
<FromGitter>
<j8r> btw I can just use `Slice[range]` after
<oprypin>
@j8r: why would it be like 1024 in size
<FromGitter>
<j8r> because lot of data, like loading a game chunk, which would be a lot bigger than just a position the player
<oprypin>
@j8r: you probably have some per-connection state object already, just add an IO to that
<oprypin>
j8r: clear method lets you use an io like it's fresh
<FromGitter>
<j8r> yep
<oprypin>
but actually the memory is still reserved internally, yes
<FromGitter>
<j8r> Each connection will have its own `IO::Memory` buffer, looks good
<oprypin>
so while a previous usage might be bigger, it won't be visible to you as long as you clear the io
<FromGitter>
<j8r> yep, I meant avoiding to sent unnecessary trailing `0`
<oprypin>
i don't understand why that would be there
<oprypin>
there's nothing unnecessary there, no custom counting is required
<oprypin>
as long as you don't use sizeof manually
<FromGitter>
<j8r> I see that's not really like slice
<FromGitter>
<j8r> that's nice, the bytesize is reset but not capacity
<FromGitter>
<j8r> `IO::Memory` is great, too bad we can't do properly by directly writing to the WebSocket IO.
<oprypin>
yea
<FromGitter>
<j8r> I try to understand why `WebSocket::Protocol::StreamIO` does not behave like `IO::Memory`
<FromGitter>
<j8r> that's their `#write` implementation, they differs
<FromGitter>
<j8r> note also that each write to the IO results to a `memcpy`
<FromGitter>
<j8r> not a big deal for now
twosecslater has left #crystal-lang [#crystal-lang]
twosecslater has joined #crystal-lang
twosecslater has left #crystal-lang [#crystal-lang]
twosecslater has joined #crystal-lang
HumanG33k has joined #crystal-lang
deavmi_ has joined #crystal-lang
deavmi has quit [Read error: Connection reset by peer]
DTZUZU has quit [Read error: Connection reset by peer]
<FromGitter>
<Blacksmoke16> and what was the code?
<sorcus>
Blacksmoke16: But i'm not sure a little about code, because i don't investigate a much time to this.
<FromGitter>
<Blacksmoke16> also how do you guys handle development with docker? Have a main `Dockerfile` for building production, then a like `Dockerfile-dev` that just mounts and watches for changes?
<sorcus>
Blacksmoke16: I know, i know, this looks ugly :-D
<FromGitter>
<Blacksmoke16> id figure out if its actually worth it
<FromGitter>
<Blacksmoke16> more unsafe code versus bit more memory usage for a lot cleaner code seems like a worthy trade
<sorcus>
Blacksmoke16: It's not only about memory usage, but performance issue.
<FromGitter>
<Blacksmoke16> right, but you can still add 23 million `BigInt`s per second
<sorcus>
Blacksmoke16: New version of my program work 10x faster, than old...
<FromGitter>
<Blacksmoke16> so is that actually a bottleneck?
<FromGitter>
<Blacksmoke16> fair enough
<sorcus>
Blacksmoke16: Maybe yes, maybe not. I must make program fast as possible X-)
<sorcus>
Blacksmoke16: But sadly part is i can't show full code. :-(
<sorcus>
Blacksmoke16: But slowest part of my program is a calculation of points in secp256k1...
<FromGitter>
<manveru> is there some way to pass signals from my process to the one within `Process.run`? seems like it just ignores it...
<jhass>
Process#kill does not work you say?
<FromGitter>
<Blacksmoke16> isnt it deprecated in favor of like `process.signal :kill`
deavmi has joined #crystal-lang
<jhass>
right, the dash docset still isn't updated :/
<jhass>
looked at the wrong docs
<FromGitter>
<wyhaines> Yeah. Process#kill is deprecated.
<sorcus>
Hmmm... Crystal on Alpine is outdated?
<FromGitter>
<Blacksmoke16> no? what imagine are you using
<FromGitter>
<Blacksmoke16> image
<sorcus>
Hmmm...
<sorcus>
Alpine Linux v3.12
<FromGitter>
<Blacksmoke16> oh, not using the docker image?
<sorcus>
Blacksmoke16: It's LXC image.
<FromGitter>
<Blacksmoke16> ah
<FromGitter>
<Blacksmoke16> might have to be on edge of `0.35.1`
<FromGitter>
<Blacksmoke16> for*
<sorcus>
Blacksmoke16: Oh, ok. I will try it tomorrow.
<sorcus>
Good night :-)
<FromGitter>
<manveru> jhass: the problem is that: I start a program using `Process.run`, I then hit ctrl-c to stop both the crystal program and its child... but nothing happens until i issue a manual kill on the child...
<FromGitter>
<manveru> so i assume that somehow crystal is ignoring SIGINT, and also not forwarding it to child processes, for whatever reason...
<FromGitter>
<Blacksmoke16> could maybe try like
<FromGitter>
<manveru> yet i can't make that behavior happen in a normal `crystal eval`...