<FromGitter>
<parruda> Hey guys. I have Crystal in production at https://faastruby.io. The Crystal program runs a child process witten in Ruby and communicates with it via unnamed pipe. The Ruby script is less privileged, and the data the Crystal program receives is parsed with `message = Message.from_json Base64.decode_string(ruby_output)`. I am trying to follow the guideline “treat less priviledged processes as malicious”.
<FromGitter>
... Could a malicious process exploit the JSON deserializer or the Base64 decoder with an arbitrary message?
<FromGitter>
<parruda> It is more like lambda meets heroku
<FromGitter>
<parruda> It is a FaaS platform, but because it is dedicated to ruby there are lots of things you can do to make the developer’s life easy without compromising flexibility
<FromGitter>
<parruda> And I am using Crystal wherever I need speed
<FromGitter>
<parruda> I actually plan to soon allow people to also deploy crystal code and run it as functions as well
<FromGitter>
<proyb6> @parruda Nice!
Raimondi has joined #crystal-lang
Raimondi has quit [Ping timeout: 240 seconds]
<FromGitter>
<rishavs> @parruda thats great! congrats and best of luck on taking the business forward.
<FromGitter>
<DanilaFe> not doing AoC today - good luck everyone else!
Renich has quit [Quit: Renich]
Renich has joined #crystal-lang
Renich has quit [Remote host closed the connection]
Renich has joined #crystal-lang
<yxhuvud>
todays was fun!
tdc has joined #crystal-lang
tdc has quit [Client Quit]
<FromGitter>
<j8r> @parruda if it's possible to modify the script, yes
<FromGitter>
<j8r> And other users that can create processes, that have the right to send data to the namedpipe
<FromGitter>
<j8r> To prevent this, allow only user & group owner to access to the file
<FromGitter>
<j8r> And only add the user of the Crystal and Ruby program to the group
<FromGitter>
<DanilaFe> now that I haven't done today, I'm not sure if I'm going to continue ¯\_(ツ)_/¯
<FromGitter>
<anamba> @yxhuvud @DanilaFe oh it's fun all right, but apparently i missed a few corner cases. fortunately, i wrote a method to print the whole state. unfortunately, HOLY COW IT'S ENORMOUS
<FromGitter>
<DanilaFe> well you've gotten farther than me :D
<FromGitter>
<yxhuvud> The only corner case I had showed up on the example input. YMMV.
<FromGitter>
<yxhuvud> though the failure didn't show up, but the problem did.
<FromGitter>
<yxhuvud> for me, that one just worked.
<FromGitter>
<DanilaFe> hm. this doesn't actually seem difficult to implement. Oh well though
<FromGitter>
<yxhuvud> 80-ish lines for me.
<FromGitter>
<yxhuvud> but I know many had problems, I started an hour late and still got place 321 on part 2.
<FromGitter>
<anamba> i did not really come up with an elegant solution... so, lots of corner cases :(
<FromGitter>
<anamba> really fun to see it work though
<FromGitter>
<anamba> *almost work
azuri5 has joined #crystal-lang
ashirase has quit [Ping timeout: 250 seconds]
ashirase has joined #crystal-lang
<FromGitter>
<parruda> @j8r what do you mean by modify the script? In my case, the ruby script could send any arbitrary payload through this pipe, and it will be parsed by b64 and JSON.
<FromGitter>
<anamba> @yxhuvud @DanilaFe finally fixed all the corner cases
<FromGitter>
<DanilaFe> congrats!
azuri5 has quit [Ping timeout: 250 seconds]
<FromGitter>
<j8r> @parruda I mean this Ruby script is trusted, or not?
<FromGitter>
<j8r> If supposed to be, it might be modified
<FromGitter>
<anamba> @yxhuvud @DanilaFe thank god the part 2 on this one was trivial. i was afraid it was going to be about water pressure
<FromGitter>
<parruda> @j8r not trusted. The ruby scripts are functions uploaded by users. The returned value from the function gets piped to crystal, who delivers the result to the caller
<FromGitter>
<parruda> I’m wondering if a malicious ruby script could return a certain payload that would exploit the b64 string decoder or the JSON parser.
<FromGitter>
<parruda> Buffer overflow or something like that. That’s not really my area of experience so forgive my inability to explain
<FromGitter>
<j8r> It's possible
<FromGitter>
<j8r> There is a max nesting protection
<FromGitter>
<j8r> But an attacker may find another way to break this
<FromGitter>
<j8r> You could add a JSON check to avoid really big documents
<FromGitter>
<parruda> So do you think the problem could be mostly at the JSON level? Not every request is processed by it, but they are all b64 decoded. Users may return files from their functions for example
<FromGitter>
<j8r> or yes the b64 encoder too.
<FromGitter>
<asterite> there shouldn't be any problem
<FromGitter>
<j8r> What happen they process 10GB of data?
<FromGitter>
<asterite> unless maybe decoding a huge string, but for that you'll already have that problem in the Ruby side I think
<FromGitter>
<asterite> @j8r that will happen, Ruby will probably die first
<FromGitter>
<j8r> yes
<FromGitter>
<j8r> defining cgroups to limit cpu/memory/disk/whatever can mitigate this type of problems
<FromGitter>
<j8r> and having a least two instances of the crystal application with health checks, in case of a Ruby trick to send enormous data
<FromGitter>
<girng> omg vscode so annoying. doesn't SAVE MY TABS!
go|dfish has joined #crystal-lang
DTZUZO has joined #crystal-lang
azuri5 has joined #crystal-lang
tilpner has quit [Disconnected by services]
azuri5 has quit [Quit: azuri5]
<FromGitter>
<straight-shoota> @girng it should
<FromGitter>
<parruda> Thanks a lot guys!
<FromGitter>
<proyb6> @j8r It will take years for Ruby to optimize their JIT too
<FromGitter>
<girng> @straight-shoota not sure :/ i have like 8 files open (6 .cr, and 2 .json), then when i re-open vscode (restart computer), the tabs are all gone
<FromGitter>
<proyb6> @asterite I think it would be useful if Crystal could know how much free memory is available, are there any?
<z64>
it's not part of the API (stdlib), its part of the compiler/language
<FromGitter>
<kinxer> Gotcha. I'm trying to grok the difference between `typeof(x) == SomeClass`, `x.class == SomeClass`, and `x.is_a? SomeClass`. I'm pretty sure that the last one is different because it will return true if the class of `x` is some subtype of `SomeClass`, but I'm not sure on the distinction between the other two.
<FromGitter>
<kinxer> Is it runtime class (for `#class`) versus compile-time class (for `typeof`)?
<FromGitter>
<j8r> @MrSorcus @book needs to be defined in initialize or class level
akaiiro has joined #crystal-lang
<FromGitter>
<asterite> You can do `property! book : Book`
<FromGitter>
<asterite> or also `property bool : Bool?`
<FromGitter>
<asterite> sorry: `Book?`
<FromGitter>
<asterite> depending on your needs
<FromGitter>
<MrSorcus> Hmmm, really cool. Thanks you guys. 👍 🆒
azuri5 has quit [Quit: azuri5]
azuri5 has joined #crystal-lang
Dreamer3 has joined #crystal-lang
Raimondi has joined #crystal-lang
Raimondi has quit [Ping timeout: 240 seconds]
azuri5 has quit [Quit: azuri5]
non-aristotelian has joined #crystal-lang
virkony has joined #crystal-lang
azuri5 has joined #crystal-lang
<FromGitter>
<jwoertink> Is it always a performance benefit to use a char instead of a string (where applicable) i.e. `"somestring".delete('/')` vs `"somestring".delete("/")`?
<FromGitter>
<jwoertink> Or is there a case where you shouldn't do that?
<FromGitter>
<r00ster91> @jwoertink yes that's actually faster because then internally in LLVM the char will represented as an integer which is more efficient than a string
<FromGitter>
<j8r> @jwoertink note if you want to perform multiple delete or other string manipulation, it's more efficient to do a `String.build` with an `each_char`
<FromGitter>
<j8r> This avoids multiple iterations
<FromGitter>
<jwoertink> ah, good tip!
<FromGitter>
<asterite> I'm actually not sure there's a performance gain when passing string vs char, and in some cases char might be slower (when you need to convert it to a string or when you need its UTF8 bytes). But benchmarking can help to see which one is faster in each case
<FromGitter>
<jwoertink> I did a benchmark example on what I was looking at, it did say char was faster, but it was almost negligible
<FromGitter>
<jwoertink> But I was more thinking of just general practice.
<FromGitter>
<j8r> usually Char are more efficient than String - in doubt, pick Char
<FromGitter>
<j8r> Furthermore Char is a struct and String is a class - less pressure to the GC
alsm has joined #crystal-lang
<alsm>
Hey, probably a simple one, how can I slice an array like a[1:3]?
<alsm>
ahh, .. got it
<FromGitter>
<asterite> @j8r that's true, but literal strings are allocated in read-only memory inside the program
<FromGitter>
<asterite> (well, little difference)
<alsm>
another similar one, can I range to the end of an array without specifying the end index? Like 2.. but that obviously doesn't work
<FromGitter>
<bew> Hehe, this will work soon (2.. will probably be merged!), for now you can do 2..-1
<FromGitter>
<j8r> outside performance, it's more logic to choose the Char type if we want to manipulate a character and not a sequence of characters...
<FromGitter>
<bew> @r00ster91 i think you can't.. I'd say, the lexer is not the place for this logic (color or not)
<FromGitter>
<bew> @r00ster91 One way to solve this I think would be to move that logic to the place where the exception must be printed, and add a new field in the exception class for "informative text" (>> that would be colored)
<FromGitter>
<oguzbilgic_twitter> @bew @j8r :( Thank you guys, I will check them out
Raimondi has joined #crystal-lang
go|dfish has quit [Ping timeout: 240 seconds]
go|dfish has joined #crystal-lang
akaiiro has quit [Remote host closed the connection]
<FromGitter>
<straight-shoota> RX14 I found the errors in HTTP server spec
<RX14>
good
<FromGitter>
<straight-shoota> at least I hope I found all of them :D
<FromGitter>
<straight-shoota> running `lsof -p #{Process.pid} -i TCP -a` after each spec helps a lot ^^
<RX14>
but it'll be reusing a different port each time right?
<FromGitter>
<straight-shoota> yes
<FromGitter>
<straight-shoota> one of the issues was that `HTTP::Server#bind_tcp` (and `#bind_unused_port`) creates a TCPSocket and tries to add it to the server. When the server is already closed at that point it raises, but the socket is not closed.
<FromGitter>
<straight-shoota> at least that was easy to fix
<RX14>
does it actually fix the specs?
<FromGitter>
<straight-shoota> there are other issues as well, specs are leaking open filedescriptors for different reasons
<FromGitter>
<straight-shoota> I fixed them
<RX14>
but I still dont understand the mechanism for the specs failing
<FromGitter>
<straight-shoota> now I'm working on making the fix look nice
<FromGitter>
<straight-shoota> I don't understand it completely either
<FromGitter>
<straight-shoota> Maybe the patch will help shed some light
<FromGitter>
<straight-shoota> Anyway, I'm also adding a helper to ensure the server (running in a different fiber) shuts down correctly. This is communicated through a fiber.
<FromGitter>
<straight-shoota> Is it possible to query the status of the channel, whether a message is waiting without consuming it?
<RX14>
yes
<RX14>
.full? and .empty?
<FromGitter>
<straight-shoota> Don't seem to work... they're both `true`
<FromGitter>
<straight-shoota> which looks like a contradiction?
<FromGitter>
<straight-shoota> unbuffered channel
<FromGitter>
<straight-shoota> `def full?` is `@has_value || @receivers.empty?`
<FromGitter>
<straight-shoota> shouldn't that be `@has_value && !@receivers.empty?` ?