jhass changed the topic of #crystal-lang to: The Crystal programming language | http://crystal-lang.org | Crystal 0.18.2 | Fund Crystals development: http://is.gd/X7PRtI | Paste > 3 lines of text to https://gist.github.com | GH: https://github.com/crystal-lang/crystal | Docs: http://crystal-lang.org/docs/ | API: http://crystal-lang.org/api/ | Logs: http://irclog.whitequark.org/crystal-lang
Philpax has joined #crystal-lang
tomchapi_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
Oliphaunte has quit [Read error: Connection reset by peer]
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
tomchapin has joined #crystal-lang
onionhammer has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Ping timeout: 276 seconds]
zodiak has joined #crystal-lang
zodiak_ has quit [Ping timeout: 246 seconds]
tomchapin has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
pawnbox has joined #crystal-lang
franzejr_ has quit [Remote host closed the connection]
pawnbox has quit [Ping timeout: 244 seconds]
Oliphaunte has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
Raimondii has joined #crystal-lang
Raimondi has quit [Ping timeout: 240 seconds]
Raimondii is now known as Raimondi
pawnbox has joined #crystal-lang
pawnbox has quit [Ping timeout: 240 seconds]
Oliphaunte has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 240 seconds]
Oliphaunte has joined #crystal-lang
pawnbox has joined #crystal-lang
pawnbox has quit [Ping timeout: 240 seconds]
soveran has joined #crystal-lang
soveran has quit [Ping timeout: 276 seconds]
Oliphaunte has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
pawnbox has joined #crystal-lang
pawnbox has quit [Ping timeout: 260 seconds]
Oliphaunte has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
pawnbox has quit [Ping timeout: 244 seconds]
pawnbox has joined #crystal-lang
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 276 seconds]
soveran has joined #crystal-lang
soveran has quit [Ping timeout: 244 seconds]
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
Oliphaunte has joined #crystal-lang
soveran has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 250 seconds]
snsei has quit [Remote host closed the connection]
snsei has joined #crystal-lang
snsei has quit [Read error: Connection reset by peer]
snsei has joined #crystal-lang
snsei has quit [Remote host closed the connection]
snsei has joined #crystal-lang
qard has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
snsei has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 260 seconds]
qard has joined #crystal-lang
jmoriau has joined #crystal-lang
Philpax has quit [Ping timeout: 244 seconds]
crack08 has joined #crystal-lang
bjz has joined #crystal-lang
bjz has quit [Ping timeout: 244 seconds]
snsei has joined #crystal-lang
Oliphaunte has joined #crystal-lang
gettalong has joined #crystal-lang
snsei has quit [Ping timeout: 244 seconds]
gettalong has quit [Read error: Connection reset by peer]
gettalong has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 276 seconds]
<gettalong> Hi everyone! I was looking at Crystal this weekend and wanted to see how it performs in comparison to Ruby.
<gettalong> There are some benchmarks out there that have Crystal in the league of C. I also found a benchmark of the JSON library at https://github.com/kostya/benchmarks
<gettalong> This looking good I tried implementing a PDF lexer in Crystal to test whether it performs that good for my use case.
<gettalong> Using the Json lexer/parser as example I came up with a solution that compiled and I ran some tests.
<gettalong> The problem is now: The compiled Crystal program is not that much faster than the Ruby solution, ~9s for Crystal and 12s for Ruby...
bjz has joined #crystal-lang
<gettalong> The code is available here https://play.crystal-lang.org/#/r/122g
<BlaXpirit> gettalong, run it with --release; change `class` to `struct`
<gettalong> Since this is my first time writing Crystal it probably (or surely) looks awful but that is not my main concern which is performance. Any tips or hints on how this code could be made faster? Or are there any docs out there which deal with parsing ?
<gettalong> Thanks BlaXpirit - I compiled with "crystal compile --release src/hexapdf.cr", so this is taken care of.
<BlaXpirit> gettalong, consume_float seems dangerous, because integers can overflow. best to store it as string then convert probably
<BlaXpirit> gettalong, have you tried changing the small classes to `struct`?
<gettalong> I have changed the classes to structs, no improvement
<gettalong> Also, the code in consume_float etc. is taken from the Json library of Crystal and adapted to use UInt8 instead of Char, so it should probably be alright (if I didn't introduce any stray bugs)
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<BlaXpirit> :(
<gettalong> My Ruby version uses the StringScanner class for parsing a binary string. What I noticed is that if I don't use the StringScanner in Ruby but a similar approach as Crystal does with JSON (or what I did in the Crystal version of my code), it is twice as slow, probably because I need to loop so much in the Ruby code.
<gettalong> However, I thought that this should be no problem in Crystal because it gets compiled, so everything should be roughly at the same level
bjz has joined #crystal-lang
<BlaXpirit> gettalong, your code is good btw. but you did forget this part https://github.com/crystal-lang/crystal/blob/6f016c1e3/src/json/lexer.cr#L281
<gettalong> Thanks for you input, BlaXpirit! The code is in there but I removed the comment
<BlaXpirit> oh, I'm sorry, it really is there
<gettalong> Btw. just reading my 21MiB test file byte by byte is 14x faster in Crystal than in Ruby but that is something I expected
<gettalong> I think I will dig some more into existing files like json.cr that do this kind of parsing and see if I can get my code faster
Philpax has joined #crystal-lang
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 258 seconds]
<jhass> parse_until_whitespace_or_delimiter looks a bit dangerous, it allocates a string with almost each call. Do we actually do something similar in the JSON lexer?
<jhass> could use char's in LITERAL_STRING_ESCAPE_MAP
<jhass> 445 should be a .delete if anything
<jhass> parse_name looks optimizable too
<gettalong> parse_until_whitespace_or_delimiter needs to create a string because this string is then needed to identify the keyword (parse_keyword) or the PDF name (parse_name) - I don't know how this could be done without allocating the string
<gettalong> jhass: line 445: like this: str = str.delete("\0", "\t", "\n", "\f", "\r", " ") ?
<jhass> no just like in Ruby
<jhass> .delete("\0\t\n\f\r ")
<jhass> but the list of keywords is finite and known at least, no?
soveran has quit [Remote host closed the connection]
<gettalong> jhass: thanks! Yes, the list of keywords is finite and known
<gettalong> but would that help? We would still need to read each byte and generate a string accordingly
<jhass> well for starters you could seed the string pool with them
<jhass> then they'll be static from the data section and not permanent heap allocations
soveran has joined #crystal-lang
<jhass> but in theory you could build up a big nested case/when that returns literals or symbols
<jhass> it might be possible to generate such with a macro, I'm not sure
jmoriau has quit [Quit: http://www.kiwiirc.com/ - A hand crafted IRC client]
franzejr_ has joined #crystal-lang
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 252 seconds]
<gettalong> jhass: thanks for your input! I will look into that. One question though: How would pre-seeding the string pool help with heap allocations? Wouldn't I still need to build the string beforehand with MemoryIO or String::Builder before calling string_pool.get?
<jhass> yes that'll actually probably not make it faster just lower on memory
<jhass> well and thus a bit less GC work, so there's that
trapped has joined #crystal-lang
<gettalong> okay, good to know - thanks
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
gettalong has quit [Quit: gettalong]
Oliphaunte has joined #crystal-lang
bjz has quit [Ping timeout: 244 seconds]
Oliphaunte has quit [Ping timeout: 244 seconds]
bjz has joined #crystal-lang
Davy_CC has quit [Ping timeout: 264 seconds]
Davy_CC has joined #crystal-lang
pabs has quit [Ping timeout: 264 seconds]
pabs has joined #crystal-lang
ssvb has quit [Ping timeout: 244 seconds]
pawnbox has joined #crystal-lang
endou_ has quit [Ping timeout: 252 seconds]
endou has joined #crystal-lang
snsei has joined #crystal-lang
gettalong has joined #crystal-lang
snsei has quit [Ping timeout: 250 seconds]
franzejr_ has quit [Read error: Connection reset by peer]
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Remote host closed the connection]
snsei has joined #crystal-lang
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 240 seconds]
snsei has quit [Remote host closed the connection]
gettalong has quit [Quit: gettalong]
ponga has joined #crystal-lang
<RX14-> I'll probably work on a multipart/form-data interface for the standard library based on the existing shard tonight.
<FromGitter> <sdogruyol> @RX14 that would be awesome
RX14- is now known as rx14
<rx14> Yeah, your shard doesn't have a high-level interface really
<FromGitter> <sdogruyol> which one are you talking about?
gettalong has joined #crystal-lang
<rx14> You made the multi part shared right?
<rx14> Or is there two
<rx14> Shard*
<FromGitter> <sdogruyol> actually i failed at making one :P
<rx14> Oh, it doesn't work?
<FromGitter> <sdogruyol> it's not working
<FromGitter> <sdogruyol> guess i'm not good at writing parsers :/
<rx14> Ah well I'll sort it out somehow
<FromGitter> <sdogruyol> rx14 that'd be really great
ponga has quit []
Oliphaunte has joined #crystal-lang
no0p has quit [Ping timeout: 240 seconds]
no0p has joined #crystal-lang
SolarSailor has joined #crystal-lang
<rkeene> Any dependency manager that depends on Git is broken :-/
<jhass> shards happily takes a path option
soveran has quit [Remote host closed the connection]
<dom96> rkeene: why?
<rkeene> dom96, People who build packages do so from hosts that have no network access (to prevent external influences from changing the package)
<dom96> ahh, so you mean package managers that can *only* work with Git are broken.
pawnbox has quit [Read error: Connection reset by peer]
pawnbox has joined #crystal-lang
SolarSailor has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
SolarSailor has joined #crystal-lang
SolarSailor has quit [Client Quit]
<rx14> rkeene: surely if there is no network access then there is no way of downloading dependencies at all, do you manually place them? If you're doing that why use a package manager at all?
pawnbox has quit [Remote host closed the connection]
Raimondii has joined #crystal-lang
pawnbox has joined #crystal-lang
Raimondi has quit [Ping timeout: 240 seconds]
<rkeene> rx14, Right, that's the point, the dependencies have to be downloaded, packaged, and installed separately. If you use Git to store them then it's much harder to get a serializable object which can be used to create a package and verified to be the same thing later on (like, 10 years into the future)
SolarSailor has joined #crystal-lang
gettalong has quit [Ping timeout: 264 seconds]
Philpax has quit [Ping timeout: 276 seconds]
<rx14> How would you propose to do it?
pawnbox has quit [Ping timeout: 272 seconds]
SolarSailor has quit [Client Quit]
Raimondii is now known as Raimondi
pawnbox has joined #crystal-lang
<rx14> Besides, you don't have to package all the dependencies in crystal, in fact there's not much point
<jhass> Using git actually avoids having to reimplement data verification in a custom package format, one could even easily reuse its signature support
<jhass> a shards package that fetches the repos for offline usage into a well defined site directory should be fairly straight forward to implement too
snsei has joined #crystal-lang
soveran has joined #crystal-lang
soveran has quit [Changing host]
soveran has joined #crystal-lang
<rkeene> jhass, For the hundreds of other packages, it's simple -- verify the SHA256 of the serialized archive (tarball, zip, etc) that holds a particular release
<rx14> Why is that harder than verifying the HEAD of a git repo?
<rkeene> The HEAD of the git repo won't be the same now as in 10 years, when you may be recompiling the package
<jhass> shards.lock contains the git sha or git tag in question
snsei has quit [Ping timeout: 276 seconds]
<rx14> Yes but that's like complaining the sha256 of the latest yarn all release won't be the same
soveran has quit [Ping timeout: 244 seconds]
<rx14> Of course it won't, because there's been a new release
<rkeene> Right, but you don't have any network access when building the package
<jhass> you don't need to. git is not SVN
<rkeene> rx14, But we don't want to want to use the "latest" release, we want to use the same release every time, exactly the same.
<rx14> Exactly
<jhass> git checkout <sha>
<rx14> So you check out the same sha
soveran has joined #crystal-lang
<rx14> And yojre guaranteed to get the same sourcr
<rx14> If you're worried about the commit disappearing then mirror the repo
<rkeene> I do mirror all my sources, on multiple levels -- which is another thing that is harder with git, since it's not a serialized format...
<rkeene> And it's more difficult to do a shallow checkout of Git repo that is not HEAD, it may be fixed now but it never worked for me the the past
<rkeene> (and git-archive is completely broken -- it generates archives that have differing contents)
<rx14> Git remotes are simple directories
ssvb has joined #crystal-lang
<rx14> You can easily vendor in all the deps by including the libs dir in your tarball
<rkeene> That just means that I'm doing the verification work manually, and most likely wrong then.
<rx14> Huh?
<rx14> You verify once, when you first build that package revision, then to make it reproducable just tar the whole dir
<rkeene> Verifying that the upstream sources are unmolested from release-to-release
<rkeene> Right, but my software will have multiple releases
<rkeene> Over time
<rx14> How is verifying they are unmolested harder with git?
<rkeene> And the upstream should not change over that time, unless I verify that I'm getting it from the same entity
<rx14> So you store the sha of the dependency's directory
<rkeene> That's not harder with git, that's harder with a package manager that uses git -- since it may try to "help" by updating things, which is the wrong thing to do.
<rx14> But it doesn't touch anything unless you run it
<rx14> And shards doesn't do thag
<rkeene> Yes, that would be the manual verification I was talking about
<rkeene> But I WOULD run it to fetch additional dependencies as the software developed
<rx14> Its a simple bash for loop
<rx14> Yes but its likely all the dependencies updated anyway due to the nature of crystal's development
<rx14> Early and stuff gets updated quick
<rx14> Even when its not generating a sha of a directory is a one-ljner in bash
<rkeene> And now, of course, I have all those changes in my version control system since they are part of my release
<rkeene> I'd rather have external applications be separate packages and use the distribution package manager to assemble the parts... It's less of a problem with Crystal since it lacks "gems" (from Ruby, which are terrible) and Python's terribly terrible distutils -- but several applications built into a distribution may need to be tied to the same version of several packages to interoprate, if those dependencies are only managed upstream then you are managing only the l
crack08 has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<jhass> Crystal doesn't build shared objects
<jhass> it's an all source compiler
<rkeene> It doesn't matter for this, the shared code doesn't have to be in the form of a shared object
<jhass> it does matter for compatibility considerations
<rkeene> For example, if you have multiple applications talking to an SQLite3 DB, they need to be using compatible versions of SQLite3 (which is usually included inline and not linked as a shared object)
<rkeene> Since newer versions of SQLite3 can create databases that older versions cannot read
<rx14> but you can deal with that on a case-by-case basis, most crystal packages won't need to be version-locked like that
<rkeene> The need for same version remains the same because there are external factors, you need to interoperate via external resources such as the filesystem
<rkeene> But the need isn't on a per-package basis, it's on an integration basis -- the need doesn't exist in either package, just in the system that happens to incoroprate them both and expect them to operate on the same datafiles
<rx14> but it does, because most crystal shards don't operate in that way
Redfoxmoon has quit [Ping timeout: 244 seconds]
qard_ has joined #crystal-lang
qard has quit [Ping timeout: 260 seconds]
Raimondi has quit [Ping timeout: 240 seconds]
Raimondi has joined #crystal-lang
tomchapin has joined #crystal-lang
tomchapin has quit [Quit: Textual IRC Client: www.textualapp.com]
gettalong has joined #crystal-lang
tomchapin has joined #crystal-lang
Redfoxmoon has joined #crystal-lang
Oliphaunte has quit [Remote host closed the connection]
soveran has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
soveran has joined #crystal-lang
Redfoxmoon has quit [Read error: Connection reset by peer]
Redfoxmoon has joined #crystal-lang
Oliphaunte has quit []
<FromGitter> <k-solutions> Hi All,
<FromGitter> <k-solutions> Is it possible to do something like: ⏎ ```def test(*arr) ⏎ yield arr[0] ⏎ end``` ⏎ in Crystal [https://gitter.im/crystal-lang/crystal?at=5768399fa82a6b0079a774c9]
<BlaXpirit> @k-solutions sure, why not, but what are you getting at?
<FromGitter> <k-solutions> sorry wrong snipet
<FromGitter> <k-solutions> more like it ⏎ ```def test(*arr) ⏎ yield *arr ⏎ end``` [https://gitter.im/crystal-lang/crystal?at=57683a06f0528c4c5bbae30d]
<BlaXpirit> yes, in latest version that is probably possible
<FromGitter> <k-solutions> 18.2 ?
<BlaXpirit> yes
<FromGitter> <k-solutions> thanks
<BlaXpirit> >> def test(*arr); yield *arr; end; test(1,2) { |a, b| pp a, b }
<DeBot> BlaXpirit: a # => 1 - more at https://carc.in/#/r/123d
<FromGitter> <k-solutions> aha :)
<FromGitter> <k-solutions> that is exactly what is needed
Redfoxmoon has quit [Read error: Connection reset by peer]
Redfoxmoon has joined #crystal-lang
Redfoxmoon has quit [Read error: Connection reset by peer]
Redfoxmoon has joined #crystal-lang
soveran has quit [Remote host closed the connection]
Redfoxmoon has quit [Read error: No route to host]
Redfoxmoon has joined #crystal-lang
Redfoxmoon has quit [Changing host]
Redfoxmoon has joined #crystal-lang
soveran has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
ruslux has joined #crystal-lang
Oliphaunte has joined #crystal-lang
rx14 is now known as RX14
Oliphaunte has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
tomchapin has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
willl has joined #crystal-lang
ruslux has quit [Ping timeout: 250 seconds]
<crystal-gh> [crystal] asterite pushed 1 new commit to master: https://git.io/vor0T
<crystal-gh> crystal/master be4143f Ary Borenszweig: Merge branch 'release/0.18'
soveran has quit [Remote host closed the connection]
<crystal-gh> [crystal] asterite pushed 1 new commit to master: https://git.io/vorub
<crystal-gh> crystal/master 352eb73 Ary Borenszweig: Merge branch 'release/0.18'
<crystal-gh> [crystal] asterite pushed 1 new commit to master: https://git.io/vorzu
<crystal-gh> crystal/master aa1d7f2 Ary Borenszweig: Merge branch 'release/0.18'
<crystal-gh> [crystal] asterite closed pull request #2876: Fix the order of output in "no overload matches" error (master...no-overload-error) https://git.io/voaYG
<travis-ci> crystal-lang/crystal#be4143f (master - Merge branch 'release/0.18'): The build passed. https://travis-ci.org/crystal-lang/crystal/builds/139035193
greengriminal has joined #crystal-lang
Oliphaunte has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
<travis-ci> crystal-lang/crystal#95faf73 (release/0.18 - Fixed #2877: uses correct llvm module for typedef metaclass): The build passed. https://travis-ci.org/crystal-lang/crystal/builds/139035222
soveran has joined #crystal-lang
<FromGitter> <taylorfinnell> hello :) is there a library to allow monitoring of a directory? ie: an event is fired if that directory contents changed
gettalong has quit [Quit: leaving]
<FromGitter> <sdogruyol> @taylorfinnell https://github.com/f/guardian
<FromGitter> <taylorfinnell> ahh i just am bad at searching watch and guard are good queries
<FromGitter> <taylorfinnell> thanks guys!
soveran has quit [Ping timeout: 244 seconds]
<FromGitter> <taylorfinnell> i now fear i just searched poorly, but are there any ui\graphics libs? not for games but more of for a small gui, more specifically an icon in the system tray
<BlaXpirit> @taylorfinnell maybe just gobject
<BlaXpirit> @taylorfinnell https://github.com/Fusion/libui.cr search for 'ui' doesn't find it
<FromGitter> <taylorfinnell> that looks like it may be perfect
<FromGitter> <taylorfinnell> the github says "(well, forget about Windows for now, this is Crystal)" is Windows cross compilation not really supported yet?
<BlaXpirit> crystal can't run on windows, no
<FromGitter> <taylorfinnell> gotcha, thanks again guys
<BlaXpirit> sigh, this is such a minimal binding
<BlaXpirit> @taylorfinnell i really doubt that this supports system tray. gtk may be good though
<travis-ci> crystal-lang/crystal#557397d (release/0.18 - Formatetr: preserve runs of '#' and whitespace. Fixes #2866): The build passed. https://travis-ci.org/crystal-lang/crystal/builds/139039519
<RX14> what magic is used to stop the modification of slices created by string.to_slice?
<jhass> none?
<RX14> well, somehow it doesn't work
<RX14> wait, if it's a string literal it won't be writable memory will it
<jhass> yup, was just about to mention
<jhass> ends up in the binaries rodata
<RX14> so strings aren't always immutable
<jhass> technically not
<RX14> sad
<RX14> oh well
Oliphaunte has quit [Remote host closed the connection]
<jhass> I mean we could probably use some mmap or whatever it is call to mark the memory read only
<RX14> i was trying to work out if MemoryIO(String) copied the string or not and fell into a rabit hole of testing stuff heh
<RX14> icr is super useful
<jhass> but 1) portable solution to that 2) maybe we want to stop copying on String.new(Slice) someday
<BlaXpirit> RX14, I find crystal play much more convenient. always better to have a real editor window than repeating stuff line by line
<RX14> yeah well they're different
<BlaXpirit> you can still type line by line in crystal play and still get the outputs
<RX14> oh wow why isn't carc.in hosted crystal play
<BlaXpirit> but you can also make a tiny edit without re-typing the whole thing
<BlaXpirit> RX14, that gives much more strain
<RX14> worth it
<BlaXpirit> well not worth it for jhass's personal server :D
<RX14> depends how big it is
<BlaXpirit> RX14, crystal play can literally send a compilation command after every key stroke
<RX14> yeah
<RX14> i guess
<jhass> also carc.in is older than crystal play ;)
<RX14> i realise
<jhass> and actually agnostic to the language or even command in theory
<jhass> I mean I have a ruby and GCC sandbox there too, just because
<RX14> oh, lol icr was hiding the segfault that happens when writing to a slice
<jhass> oh and I have no dedicated IPv4 for the box and proxying websockets, meh
<RX14> well what does it have if not an ipv4 address
<jhass> s/slice/rodata/
<RX14> jhass, yeah thats what I meant
<jhass> an ipv6 adddress and a http proxy for v4
<RX14> huh
<RX14> can the http proxy not proxy websockets
<jhass> no idea actually
<RX14> nginx can
<jhass> anyway, all way more effort than worth it
<RX14> yeah i guess
<RX14> yeah crystal play does kinda kill my cpu
<RX14> the multipart implementation's coming along fine
<RX14> it's going to be a little tricky handling streaming support
<jhass> cool
greengriminal has quit [Quit: Leaving]
<RX14> i'll have to make an IO wrapper which wraps read and finds the boundary
<jhass> yeah I think we already have some of those in the HTTP layer
<RX14> yeah
<RX14> but for chunked not multipart
<RX14> actually, is streaming support even possible
<RX14> because
<RX14> there are multiple body blocks
<RX14> if you call spawn in the block handler
<RX14> then it could require the IO to be read in multiple places
<RX14> which is of course impossible
<RX14> it's possible to stream the chunks as they come in but not the data inside each chunk
<jhass> mmh
<jhass> I guess we also need some API for the user to recognize each chunk
<jhass> might be better to have them explicitly wrap it?
<RX14> luckilly i seem to be able to reuse HTTP.parse_headers_and_body
<RX14> jhass, well i was planning for the user to register a callback that takes headers and body
<RX14> then call start on the parser
<RX14> this is for multipart
<RX14> the form-data bit can be a small wrapper on the former
<jhass> and then expect them to channel out the data?
<RX14> what do you mean by channel out?
<jhass> idk for example in case I want to put them into a single DB record in different columns for each part with a single INSERT
<jhass> I just feel it's generally easier and more straight forward to custom build a callback API on top of a pull API than vice versa
<RX14> well
<RX14> the callback api seems like the lowest-level primitive to me
<RX14> you can build an API that just returns a hash on top of it
<jhass> so start will block until done and you rely on spawning fibers to run the callbacks?
<RX14> for example HTTP::Multipart::FormData.parse(HTTP::Context) => {"field" => ["value", "value"] }
<RX14> jhass, no start will block and block for the handlers
<RX14> if clients want to call spawn in them that's their choice
<jhass> mmh
<crystal-gh> [crystal] opes opened pull request #2883: Remove instance variable error for abstract classes (master...issue_2827) https://git.io/vorXV
<jhass> what if I only want to read the first part and discard the rest?
<jhass> without even consuming the IO?
<RX14> hmmn
<RX14> we could call parser.close
<RX14> which sets an instance variable
<RX14> which is checked after the handler
<RX14> but i just don't think it's the job for a pull parser
<RX14> seems a bit of a pain to keep state
<jhass> parts << parser.part_io.read while parser.has_part? seems just more straight forward to me
<jhass> parser.skip_part
<RX14> it's a lot more complex on the implementation side
<jhass> I'm not sure I see why
<RX14> the code doesn't read declaritively any more
<RX14> well
<RX14> i mean
<RX14> i guess it should be easy enough
<jhass> :)
<RX14> damnit
<jhass> I have my troubles too to break out of the callback pattern
<RX14> well i have to go to sleep, so i'll push to github and work on it again tomorrow
<jhass> all that JS made it so natural
<RX14> it is slightly simpler code
<RX14> tbh
<jhass> btw could go as far as making it Enumerable and have an Iterator
<RX14> you need to wrap it all in a state machine
<jhass> well yes
<RX14> which i find ugly
<RX14> compared to procedural parsing code
<jhass> it's usually also faster though
<RX14> i wouldn't say so
<jhass> or tends to lead to faster client code at least
<RX14> maybe
<RX14> unless you perform weirdness
<RX14> like my IRC parser monstrosity
<RX14> heh
<jhass> <3
<jhass> well sleep over it :)
<RX14> my irc parser basically returns an object which lazilly parses/memcopies parts of the message when getters are called
<RX14> the code's wacky
<RX14> but yes, night
<jhass> I see
<jhass> Enumerator is such a wonderful pattern to turn an internal iterator into an external one, sad it's so slow
tomchapin has joined #crystal-lang
<RX14> Why is it slow?
<RX14> Oh, enumerable gosh I'm tired
<jhass> no Enumerator, it works by wrapping the internal iterator into a fiber and channeling out each iteration with an unbuffered channel
<jhass> Enumerable is actually very fast since yield is inlined
<RX14> Is this in the stdlib?
<jhass> it's not, because slow ;)
<jhass> it's why we have all these Iterator's
<jhass> Enumerator is a ruby idiom
tomchapin has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tomchapin has joined #crystal-lang
<RX14> I thought crystal's iterator was ruby's enumerator
<RX14> In confused of the difference between the two
<jhass> Iterator reimplements the iteration in a way that allows external iteration
<jhass> Enumerator simply wraps an internal iterator implemntation into an external iteration interface
<jhass> to the user of a specific method however, Iterator is indeed the same as Enumerator
<jhass> they solve the same task
<RX14> Glancing at the docs it seems the only difference is that iterator is lazy
<RX14> I didn't know ruby's enumerator wasn't honestly
<RX14> Enumerators in C# were lazy
<jhass> Ruby's is too, well depending on how you define lazy
<jhass> but there's Enumerator#lazy for either definition you'd like
<RX14> I'm still really confused but I'm going to sleep now
<jhass> yes, didn't want to keep you awake, night!
<|2321> I like that the 0.18 blog summary is just :-)
<travis-ci> crystal-lang/crystal#f5f5705 (release/0.18 - Fix the order of output in "no overload matches" error): The build passed. https://travis-ci.org/crystal-lang/crystal/builds/139040513
<travis-ci> crystal-lang/crystal#aa1d7f2 (master - Merge branch 'release/0.18'): The build passed. https://travis-ci.org/crystal-lang/crystal/builds/139040518