<FromGitter>
<faustinoaq> @codenoid BTW, How much memory do you have? 118GB seems to require huge resources, Have you read your big json file using other languages like Go or Rust?
<FromGitter>
<codenoid> my laptop ram just 8GB π , i'll read file content separately, thank you
<FromGitter>
<faustinoaq> @codenoid no problem π
<FromGitter>
<jdhorwitz> Evening all. Experienced Dev new to Crystal and excited to get involved
<FromGitter>
<aisrael> So... I stripped down both the Crystal code and Go code yesterday down to just Gzip deflate and Crystal still takes almost 20x as much time as Go: https://github.com/aisrael/go-csv-gz-test/tree/gzip_only
<FromGitter>
<megatux> Hi, anyone using Crystal with *asdf* version manager? I'm unable to use the *shards* command.
<FromGitter>
<megatux> > $ shards β > No such command in 0.25.0 of crystal
<FromGitter>
<crash2burn_twitter> anyone know if theres a cache, or something else weird with shards that would cause it to not install the latest commit on my repo (and when i set the commit hash in the shard.yml, i get an error saying it doesnt exist..)
<FromGitter>
<Daniel-Worrall> there is a global shard cache, yes
<FromGitter>
<aisrael> @faustinoaq Overall it'd speed up things I'm sure, but seeing as how the bottleneck seems to be in Gzip decompression and CSV & Time parsing... I *will* try it out though out of curiosity
<FromGitter>
<crash2burn_twitter> thanks so much @faustinoaq
<FromGitter>
<S-YOU> Which is the fastest implementation? @aisrael
<FromGitter>
<aisrael> Is there any way to get the fiber "id" or some other identifier? For tracing purposes?
<FromGitter>
<aisrael> @S-YOU I haven't tried the Rust version but so far Go > Crystal > Ruby/Python
<FromGitter>
<S-YOU> And Go is like 27s?
<FromGitter>
<aisrael> If serial, Go is ~1 minute, Crystal 28 minutes (consistent with 20x slower gzip and 8x slower CSV+Time parsing)
<FromGitter>
<S-YOU> You forgot the fact that Crystal can call C code or write assembly π I need to learn gzip inflate first, good for this weekend.
<FromGitter>
<aisrael> Of course. I just checked and, indeed, the `Gzip::Reader` is written in Crystal. Maybe there could be an option to use an existing C Gzip/Zlib library? That, or profile and optimize reader.cr (https://github.com/crystal-lang/crystal/blob/master/src/gzip/reader.cr)
<FromGitter>
<aisrael> Alsoβ @faustinoaq I must be doing something wrong when I use `spawn` and `channel.send`/`.receive` the Crystal implem still takes as much time as the serial version
akaiiro has quit [Ping timeout: 248 seconds]
rohitpaulk has quit [Ping timeout: 256 seconds]
wontruefree has joined #crystal-lang
jetpack_joe has quit [Ping timeout: 255 seconds]
aarongodin has quit [Ping timeout: 255 seconds]
aarongodin has joined #crystal-lang
jetpack_joe has joined #crystal-lang
wontruefree has quit [Ping timeout: 265 seconds]
jwaldrip has quit [Ping timeout: 240 seconds]
thi_ has quit [Ping timeout: 256 seconds]
beizhia has joined #crystal-lang
mroth has quit [Ping timeout: 256 seconds]
jeromegn has quit [Ping timeout: 276 seconds]
ilovezfs_ has quit [Ping timeout: 240 seconds]
Majost has quit [Ping timeout: 256 seconds]
andersh has quit [Ping timeout: 256 seconds]
jetpack_joe has quit [Ping timeout: 245 seconds]
aarongodin has quit [Ping timeout: 260 seconds]
OvermindDL1 has quit [Ping timeout: 245 seconds]
avdi has quit [Ping timeout: 256 seconds]
danzilio has quit [Ping timeout: 276 seconds]
wontruefree has joined #crystal-lang
akaiiro has joined #crystal-lang
akaiiro has quit [Remote host closed the connection]
DTZUZO has quit [Ping timeout: 248 seconds]
wontruefree has quit [Quit: bye]
rohitpaulk has joined #crystal-lang
andersh has joined #crystal-lang
jeromegn has joined #crystal-lang
thi_ has joined #crystal-lang
aarongodin has joined #crystal-lang
OvermindDL1 has joined #crystal-lang
mroth has joined #crystal-lang
ilovezfs_ has joined #crystal-lang
danzilio has joined #crystal-lang
avdi has joined #crystal-lang
<FromGitter>
<conradwt> Are there any plans to bundle the `shards` command with `Crystal` language?
jetpack_joe has joined #crystal-lang
jwaldrip has joined #crystal-lang
aarongodin has quit [Ping timeout: 276 seconds]
jetpack_joe has quit [Ping timeout: 245 seconds]
avdi has quit [Ping timeout: 276 seconds]
jeromegn has quit [Ping timeout: 256 seconds]
jwaldrip has quit [Ping timeout: 255 seconds]
thi_ has quit [Ping timeout: 256 seconds]
danzilio has quit [Ping timeout: 265 seconds]
mroth has quit [Ping timeout: 256 seconds]
OvermindDL1 has quit [Ping timeout: 256 seconds]
andersh has quit [Ping timeout: 276 seconds]
ilovezfs_ has quit [Ping timeout: 276 seconds]
<FromGitter>
<bew> what do you mean "bundle" ?
aarongodin has joined #crystal-lang
jetpack_joe has joined #crystal-lang
Majost has joined #crystal-lang
Majost has joined #crystal-lang
Majost has quit [Changing host]
danzilio has joined #crystal-lang
OvermindDL1 has joined #crystal-lang
thi_ has joined #crystal-lang
mroth has joined #crystal-lang
ilovezfs_ has joined #crystal-lang
avdi has joined #crystal-lang
jeromegn has joined #crystal-lang
andersh has joined #crystal-lang
<FromGitter>
<bararchy> @conradwt it was bundled, in the sense that crystal deps update would call shards, but was decided to decouple them
<Yxhuvud>
there seems to be some regression in play
dragonkh has joined #crystal-lang
snsei has quit [Remote host closed the connection]
<FromGitter>
<bendietze_twitter> @bararchy thanks to our nr 1 two rounds ago and 3, 4 other articles i decided to have a look on Crystal. So, because of murphys law - i am sorry. I will try out the others next π
<FromGitter>
<bararchy> ΧΧΧ
gewo has joined #crystal-lang
<FromGitter>
<bararchy> :)
<FromGitter>
<bendietze_twitter> :)
<FromGitter>
<kingsleyh> I guess the Crystal team is small and not all full time so it must be much harder to keep up with speed and other improvements in other languages - when languages like Rust and Go have much bigger full time dev teams and are sponsored by big companies. So I think Crystal is doing a great job even competing in this space. This is how I feel when working on SushiChain - a small indie type blockchain going up
<FromGitter>
... against the likes of Ethereum etc. I'd love to help support Crystal if SushiChain becomes profitable or we do a good ICO. Since SushiChain is written in Crystal and depends on it being maintained into the future. Keep bringing the Awesome :)
<Yxhuvud>
@kingsleyh: odds are something has become slower rather than that the other systems have become much faster.
dragonkh has quit [Quit: My iMac has gone to sleep. ZZZzzzβ¦]
<FromGitter>
<bendietze_twitter> Me too... There is now way to leave Crystal, the last weeks show me, the concept fits exactly my needs :) And at those benchmarks - we are at 0.25 ;) some concepts
<FromGitter>
<bendietze_twitter> are not written completely but i am sure its the right way :)
<FromGitter>
<bendietze_twitter> But, its to say, that benchmarks are good. If all languages were at 1.0 with same roadmap, its easier to follow were is room for improvement :) so, not to say, benchmarks are bad - they are morely good to have :)
DTZUZO has joined #crystal-lang
<FromGitter>
<aisrael> I seem to keep having trouble with `shards` lately. `shards update` doesn't seem to be picking up the latest commit in `develop` branch. Is this intentional?
<FromGitter>
<aisrael> (As a workaround, I've had to use `branch: hash`)
<FromGitter>
<bararchy> @aisrael There was regression in new shards
<FromGitter>
<bararchy> 1) 25.1 should be released soon with multiple fiex
<FromGitter>
<aisrael> Gotcha. Thanks!
<FromGitter>
<crisward> @bararchy those benchmarks you point too are a bit unfair. The node version is using pm2 to run one instance per core, so 8 instances. It looks like kemal is just using 1. Not sure about the others. Perhaps kemal should have a built in flag to tell it to spawn multiple instances.
<FromGitter>
<aisrael> Anyway, I have my workarounds with `shards` and `File.extname()`
<FromGitter>
<bararchy> @crisward hm.. it's strange because @tbrand is making those and he is a Crystal dev, I didn't think he will miss something like that
<FromGitter>
<bararchy> or maybe all benchmarks are "as it comes"
<FromGitter>
<bararchy> if pm2 supports using all cores, then Kemal should have this option too
<FromGitter>
<aisrael> βοΈ I like to do my comparisons/benchmarks "out of the box" first, then "optimized/equalized" later on
<FromGitter>
<aisrael> Whatever happened to the `Time::Formatter` classes? :(
<FromGitter>
<bendietze_twitter> About searching the API for missing details - i think, it could be a benefit to have a system that searches in release notes too - but how to organize such a view? Maybe an overall search site :) Just as an idea for future purposes
<FromGitter>
<tbrand> @faustinoaq Yay! Crystal is really awesome! π We are welcome your contribution for the project. https://github.com/SushiChain/SushiChain Give us your feedbacks!
<FromGitter>
<fgimian> @tbrand Your code reminds me why I love Crystal so much, great job. Just a little grammatical suggestion in the README: β β ```These channels are bridged to each other. β β π£ We welcome you to join! π£``` [https://gitter.im/crystal-lang/crystal?at=5b2a413c467bd7268c1f989c]
<FromGitter>
<fgimian> open and close VS Code, and now I have three
<FromGitter>
<fgimian> Sublime Text kills the process correctly
hightower2 has joined #crystal-lang
<FromGitter>
<S-YOU> TFB results are out for last run, https://www.techempower.com/benchmarks/#section=test&runid=f26afdb3-7c19-4c20-84de-228b10434f84&hw=ph&test=plaintext β Thanks @faustinoaq for suggesting to submit h2o.cr there, not the best, it was quite good result.
hightower2 has quit [Ping timeout: 245 seconds]
<FromGitter>
<faustinoaq> @S-YOU π π
<FromGitter>
<vlazar> are TFB benchmarks that unstable or it's really faster?
<FromGitter>
<S-YOU> I think h2o is not really tuned for benchmarks
<FromGitter>
<S-YOU> Like function inlining
<FromGitter>
<S-YOU> But I don't really know about why h2o is slower, but I know why h2o.cr is not 7M/s xD
<FromGitter>
<faustinoaq> @vlazar @S-YOU I think that's because db and plaintext tests are mixed, maybe we have same issue on crystal-raw
<FromGitter>
<S-YOU> There is 3 non inline functions call to h2o library for every requests, If I modify h2o library to try to recompile myself, hopefully I could inline, but it is kind of not really use case, but a benchmark game xD
<FromGitter>
<faustinoaq> We should split tests on two groups (json/plaintext) and (queries, fortunes)
<FromGitter>
<faustinoaq> @S-YOU TFB is already a benchmark game XD
<FromGitter>
<faustinoaq> Well, at least we have a good warrior now πͺ
duane has joined #crystal-lang
<FromGitter>
<faustinoaq> We should create tests like `crystal-json`, `crystal-plaintext`, `crystal-db`, `amber-plaintext`, `kemal-plaintext`, etc
<FromGitter>
<faustinoaq> ASP.NET and JVM frameworks are already doing that π
<FromGitter>
<S-YOU> I suspect libreactor a bit, because I get 1M+/s on single core, on VirtualBox, which is impossible unless, libreactor sending response indiscriminately.
<FromGitter>
<faustinoaq> Other frameworks are already doing that `ulib-plaintext_fit`, `aspcore-mw-utf8json`, `ulib-json`, `ulib-json_fit`, etc
<FromGitter>
<S-YOU> So, basically it is not really event based
<FromGitter>
<faustinoaq> Maybe mixing all test in one app (like we're doing right now db/plaintext) have some kind of overhead, or LLVM optimization issues
<FromGitter>
<S-YOU> Is there a way to separate? I thought there is a single port you can get.
<FromGitter>
<faustinoaq> @S-YOU Wow, 1M+/s on single core, What crazy things is libreactor doing? π
<FromGitter>
<S-YOU> If you just keep sendin g, it is possible
<FromGitter>
<faustinoaq> @S-YOU I guess we need to have multiples test (folders)
<FromGitter>
<S-YOU> Those are standard C socket options
<FromGitter>
<S-YOU> It may be inside library, so you cant see there
<FromGitter>
<S-YOU> h2o.cr is mostly translation of h2o, so basically it is same :)
<FromGitter>
<S-YOU> Faster because Crystal can be tuned to no overhead :)
<FromGitter>
<vlazar> hehe
<FromGitter>
<aisrael> @S-YOU Nice! So, maybe we should provide some way to call βnativeβ C libs when possible for performance reasons. Like, a `gzip-native` shard that can drop in replace stdlib?
<FromGitter>
<S-YOU> I don't think so @aisrael, interface are differents. But it is just gzopen, gzread calls, you get uncompressed one.
<FromGitter>
<S-YOU> If you need, I could upload a zlib bindings
<FromGitter>
<S-YOU> Or may be someone did it already
rohitpaulk has quit [Ping timeout: 256 seconds]
<FromGitter>
<S-YOU> I am personally not a fan with abstracting every IO
snsei has joined #crystal-lang
<FromGitter>
<S-YOU> Abstracting IO in Crystal means attaching to Fibers, Schedulers, LibEvent's eventloop, Stack Switches, GC, etc.... β When I decompressing the file or writing to STDOUT, I need none of those.
<FromGitter>
<S-YOU> Good, I could tuned not to use those π
<FromGitter>
<faustinoaq> I just realize devs do a lot of crazy stuff on TFB, from embeding whole http servers to full customized json parsers and a lot of raw customized db libs
<FromGitter>
<S-YOU> Yeah xD
Jenz has joined #crystal-lang
<Jenz>
What makes this not compile?: `getter somestruct : SomeStruct = uninitialized SomeStruct`
<FromGitter>
<S-YOU> You can't assign on getter, do you?
<Jenz>
@faustinoaq: Thanks, I know its unsafe, but need it for some bindings
<FromGitter>
<faustinoaq> Jenz π
<Jenz>
After think a bit about it, I don't get why `uninitialized` behaves differentely to other values in this scenario
return0e has joined #crystal-lang
<FromGitter>
<S-YOU> make uninitialized as global method, problem should be solved.
<FromGitter>
<S-YOU> hmm, may be not, it is attached to compiler
<FromGitter>
<bcardiff> @Jenz I think the issue there is that on the macro expansion the uninitialized is used as a method (and have parens) which should not
<FromGitter>
<S-YOU> Yeah
<FromGitter>
<bcardiff> Feel free to submit an issue about that.
<Jenz>
Nope, its not that
<Jenz>
I tried to manually type it without the parens, same error
<FromGitter>
<bcardiff> But probably you can use `getter!` macro to allow lazy initialized property
<FromGitter>
<S-YOU> zapcc does not work with C, as far as I know.
That_Guy_Anon has quit [Remote host closed the connection]
<FromGitter>
<S-YOU> but it is not really related I guess.
DTZUZO has joined #crystal-lang
<RX14>
oops
<RX14>
I accidentally rm -Rf'd my ~
<Jenz>
Damn
<Jenz>
Well, I accidentaly rm -rf'd my / from an installation media (usb)
<Jenz>
once
<Jenz>
Couple of months ago
<RX14>
oops
<RX14>
i have a 12 day old snapshot though
<FromGitter>
<faustinoaq> RX14 I hope your crystal on windows changes are safe π
<RX14>
i dont keep code in my ~
<RX14>
and my windows branch is up to date with my fork anyway
<FromGitter>
<faustinoaq> Oh great!, Well, I use `~/Projects` to host all my code, What setup you use?
vikaton has joined #crystal-lang
<RX14>
/data/ subvolume for most of my stuff
<RX14>
~ is too messy for me
<RX14>
too many applications use it
Jenz has quit [Ping timeout: 264 seconds]
<FromGitter>
<faustinoaq> Yeah, Well also I use BTRFS and do snapshots (subvolumes) sometimes, although, I backup my files in the cloud and external disks too π
<FromGitter>
<faustinoaq> Hey @veelenga I see you updated https://github.com/crystal-community/jwt Great work, Thank you for maintaining shards on crystal-community! π π π―
wontruefree has joined #crystal-lang
dragonkh has joined #crystal-lang
<pwned>
yesterday somoene asked about ActiveSupport.Hash.from_xml
<pwned>
maybe the above is useful so I'm putting it out there
<FromGitter>
<veelenga> @faustinoaq yup, @vladfaust reported the issue.
duane has quit [Ping timeout: 268 seconds]
dragonkh has quit [Remote host closed the connection]
rohitpaulk has joined #crystal-lang
wontruefree has quit [Quit: bye]
wontruefree has joined #crystal-lang
wontruefree has quit [Client Quit]
wontruefree has joined #crystal-lang
snsei has quit [Remote host closed the connection]
duane has joined #crystal-lang
rohitpaulk has quit [Ping timeout: 248 seconds]
Jenz has joined #crystal-lang
zachk has joined #crystal-lang
zachk has joined #crystal-lang
rohitpaulk has joined #crystal-lang
Jenz has quit [Ping timeout: 240 seconds]
rohitpaulk has quit [Ping timeout: 248 seconds]
Jenz has joined #crystal-lang
pwned has quit [Ping timeout: 245 seconds]
<FromGitter>
<MKRhere> When you do something like `File.write(<>, <>)` if the path doesn't exist, I'd like it to be created (like `mkdir -p`). Any ideas?
<FromGitter>
<MKRhere> Currently it (correctly) raises `Errno`
<FromGitter>
<pedromsilvapt> I'm new to Crystal. Was trying to create some simple bindings for a small C library. I have the `libwren.a ` file in a `lib/` folder on my project. And in the crystal file `@[Link("wren")]`. But when compiling I get `cannot find -lwren`. How could I tell crystal to look in the local `lib/` folder?
<FromGitter>
<sancarn> Does anyone know if optionally autocasting is possible? Most of the time I get a type error at compile time, which makes sense, but it'd be nice if I could only optionally have compile errors xD https://carc.in/#/r/4c8g
<FromGitter>
<pedromsilvapt> @faustinoaq setting the environment variable worked. Thanks! :) Btw, if working with shared libraries, would I need to set the variable every time I run the program? No way to embed that configuration in the binary?
<FromGitter>
<LVMBDV> he means implicit casting, not explicit casting runtime
<FromGitter>
<LVMBDV> it is not and it would kinda nullify the point of the soft typing system
<FromGitter>
<faustinoaq> Well, AFAIK that's not possible on crystal π
<FromGitter>
<LVMBDV> for good reason too
<FromGitter>
<faustinoaq> @pedromsilvapt I guess you will need to create a script to setup environment vars or use `--static` linking (require static libs, `.a` files)
Jenz has left #crystal-lang [#crystal-lang]
duane has quit [Ping timeout: 260 seconds]
<FromGitter>
<pedromsilvapt> I tried compiling with the static flag. The app compiles and runs fine. But the compilation does emit a couple of warnings. `warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking` and the same thing about `Using getprotobynumber`. Can this mean trouble when running the app on different machines? Or can I just ignore
<FromGitter>
... them?
<FromGitter>
<sancarn> Probably a better question, anyone know if we can use ENV variables inside macro expressions? `{{ENV["MY_VAR"]}}`https://carc.in/#/r/4c98
qard has joined #crystal-lang
<FromGitter>
<faustinoaq> @pedromsilvapt Try using musl or alpine:
<FromGitter>
<faustinoaq> musl works pretty nice to generate static binaries π
<FromGitter>
<pedromsilvapt> I will take a look at musl then. Thanks again for the help. π
<FromGitter>
<faustinoaq> @pedromsilvapt I prefer to use dynamic linking though, commonly I create a script wrapper to execute my binary β Even crystal uses a script wrapper when you compile it from source
qard has quit [Quit: qard]
<RX14>
crystal uses the script wrapper to set up the environment
<FromGitter>
<straight-shoota> at least in the API docs for 0.25.0 and master
<FromGitter>
<MKRhere> AHA!
<FromGitter>
<MKRhere> I was on 0.24.2
<FromGitter>
<MKRhere> Wonderful
<FromGitter>
<straight-shoota> It's not very sophisticated search feature and still has many rough edges, but in general, it works
<FromGitter>
<jwoertink> Hey all! I just wanted to bring up a topic that sort of bugs me about the crystal community. Something I notice more around Crystal than any other language I've dealt with. I get that we use English as our common tongue, and English isn't native to a ton of people that use Crystal. One thing we need to keep in mind is *how* we say things, especially in PRs, commits, and the Issues section. There are a lot of
<FromGitter>
... people that tend to use more abrasive and/or off-putting tone in how they respond to others. This is something that can easily make a newcomer feel less welcome.
<FromGitter>
<jwoertink> Thankfully it's not everyone, and there are a TON of really great and helpful people in this community which makes it still so great to be a part of
<FromGitter>
<jwoertink> I just feel this is a really important topic that we all need to be aware of. :)
<FromGitter>
<jwoertink> Thanks π
<FromGitter>
<MKRhere> I'm pretty new to Crystal, and don't have any experience with Ruby either π So far, the community has been welcoming. I'm thankful for that as well. And thanks for sharing Jeremy ^^
<FromGitter>
<pedromsilvapt> @faustinoaq I understand. And I usually prefer dynamic linking as well. But for this use case, I'd actually prefer static linking (hence why I was trying to use the .a file originally). For smaller utility apps, like the one I'm working on, I kind of like it to be a little more self-contained, honestly. But I don't know what I'll use in the end. Right now, I'm still testing the waters, particularly with
<FromGitter>
... crystal, so I might change the linking later.
<FromGitter>
<j8r> of course @jwoertink , even a π without friendly explications can be a cold welcome for newcomers π
<FromGitter>
<jwoertink> Sweet! Well welcome @MKRhere!
<FromGitter>
<j8r> Linking depends of how you ship your software to the end user.
<FromGitter>
<pedromsilvapt> @j8r Yes, I understand that. For now, the app that I'm building is just for my personal use. Like I said, i'm mostly testing the waters with crystal, and coming from web dev, and learning how to work with it and with native code in general.
<FromGitter>
<j8r> In this case static linking vs dynamic doesn't add anything - agree!
<FromGitter>
<MKRhere> I need to find the current user's home directory. Does Crystal provide an API for this? If not, any workarounds?
<FromGitter>
<j8r> static is very often used when we release the application
<FromGitter>
<MKRhere> As in: `/home/user/`. I'll take a Linux-only answer as well
<FromGitter>
<j8r> IIRC there is a PR about this, still not merged