<Papierkorb>
The uncompressed file is 95GiB, so yes I do want to read that thing line by line :)
<FromGitter>
<barisbalic> Anyone else find that the `rhysd/vim-crystal` causes vim to slow to a crawl?
mgarciaisaia1 has quit [Quit: Leaving.]
<FromGitter>
<crisward> Getting error 'Index out of bounds' when setting value of array. Is there any easy way to work around this? Some kind of function to pad the array up until that index?
CompanionCube has joined #crystal-lang
<FromGitter>
<crisward> Found array.fill - looks close I think
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<FromGitter>
<andreaTP> @maiha_twitter :+1: cool lib! both Try and monadic, probably monadic could be expanded to other cases, like options, lists etc?
<TheGillies>
well and also monadic
<FromGitter>
<maiha_twitter> Yup, I found one or two monadic libs for Crystal. Please wait ...
<FromGitter>
<andreaTP> Promises and Futures map concurrent flow of programming, just blocking is simply avoiding this
<RX14>
but when you block in crystal you only block the current fiber, other fibers can continue
<RX14>
so a single promise isn't really useful
<RX14>
it just makes your code more complex for no gain
<RX14>
promises are however very useful for modelling fan-out and fan-in concurrency
<FromGitter>
<andreaTP> Usually you use in one library a single way of concurrency, so futures can abstract for certain cases over fibers...
<RX14>
and currently the single way of concurrency is fibers and channels
<RX14>
Promises would simply be an abstraction over that
<FromGitter>
<andreaTP> Sure
<RX14>
in fact I think the parallel macro is nicer
<RX14>
in some cases
<RX14>
for example `job1result, job2result = parallel(job1(query), job2(query))`
<RX14>
it spawns job1 and job2 in fibers
<RX14>
and then waits for them both to finish and collects results
<RX14>
a `result = first(db(query), cache(query))` would also be nice
<RX14>
however stopping fibers cleanly is quite a challenge
<RX14>
the power of promises is being able to deal with collections of promises and promises of collections though
<RX14>
i'm not sure if there's a better, blocking, abstraction which could be used though
<RX14>
as we don't have the limitations on blocking that for example the JS community has before they took up promises
<RX14>
essentially CSP is all about realising that blocking code is is the cleanest code to read and write, then making the language make blocking code as efficient as non-blocking code by utilising loads of small fibers of blocking code and coordinating them with channels
<FromGitter>
<sdogruyol> divide and conquer that is
<RX14>
indeed
<RX14>
fibers spawning fibers spawning fibers make light work
<RX14>
I think an Enumberable#parallel_each could be quite nice
Ven has joined #crystal-lang
<RX14>
same with Enumerable#parallel_first
<FromGitter>
<sdogruyol> first we need to get multi-thread support for that?
<RX14>
well
<RX14>
it would just spawn a fiber for each
<RX14>
oh, and parallel_map
<FromGitter>
<sdogruyol> hmm
<RX14>
probably add a jobs: named argument too
<FromGitter>
<sdogruyol> if that's an IO bound stuff makes sense otherwise i dont see the advantage of having that atm
<RX14>
yes
<RX14>
most things are IO bound
bjz has joined #crystal-lang
<RX14>
and when we get to true parallelism later it becomes useful for that too
<FromGitter>
<sdogruyol> i agree with that
bjz_ has quit [Ping timeout: 250 seconds]
pawnbox has quit [Remote host closed the connection]
<Papierkorb>
RX14: I ported most of the importer to crystal. It's not really crystal-y (uargh), but it's ~2.5x faster than Ruby
<RX14>
thats cool
<RX14>
how is it un crystal-y?
<RX14>
also even if ruby's JSON parser is as fast as crystal's the marshalling to ruby objects will be a bottleneck
<Yxhuvud>
I'd bet there are instances where the generational GC of ruby may make some operations faster in Ruby, but apparently parsing json wasn't that. :)
<Papierkorb>
In Ruby I just used a Hash to map columns -> values and then push that into the PSQL dump writer which does some PSQL-specific things and then writes that as TSV data. That work's fine in Ruby. In Crystal though .. well ..
<Papierkorb>
This is my favorite line: `model = Hash(String, Int32 | JSON::Any | Time | String | BigFloat | Range(BigFloat, BigFloat) | Nil).new`
<RX14>
oh...
<Yxhuvud>
x_x
<RX14>
yeah you need some JSON.mapping/DB.mapping in there I think
<RX14>
oh well actually probably not DB.mapping
<Papierkorb>
If it was a real crystal project I'd have classes to not mess with hashes
<RX14>
crystal-db is excellent for queries
<Papierkorb>
Too slow, COPY is much much faster
<RX14>
however i think crystal-db needs a better INSERT story
<Papierkorb>
Mh, I'm wondering if it's viable to just use the `rerun` tool and then do `crystal run`. That's essentially what I do in ruby with roda/sequel
<FromGitter>
<mjago> Should `bin/crystal spec/std_spec.cr` work? `bin/crystal spec/std/int_spec.cr` works fine (OSX)
<Papierkorb>
What's the easiest way to mirror this ruby code in crystal? `[ lat, lon ].pack("DD").unpack("H*")` Basically, store two Float64's in binary in HEX representation
<FromGitter>
<bcardiff> if they are represented in binary is more like a cast what you want, hence that method. Or I understood the format wrong. I am not used to pack/unpack in ruby.
<Papierkorb>
bcardiff, the ruby snippet turns two Float64's into binary first, then stores it in hex, as in, "\xAB\xCD" => "ABCD"
<RX14>
so you want to take the binary representation of a Float64 and display it in a hex tring
<RX14>
?
<Papierkorb>
yes
<RX14>
well I would first turn the Float64 to a slice
mgarciaisaia has joined #crystal-lang
<RX14>
Slice.new(pointerof(float), sizeof(float))
<RX14>
then use the hex methods of slice
<RX14>
sorry
<RX14>
Slice(UInt8)
<Papierkorb>
Oh, I was looking for a method called #to_hex
<Papierkorb>
Next thing is, I can't find anything to do unicode normalization, is that correct?
<RX14>
carc.in is broken for me
<RX14>
isn't iot broken in firefox or something silly like that
<crystal-gh>
crystal/master 2de5577 Konstantin Makarchev: JSON::ArrayBuilder#push_raw method
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<Yxhuvud>
Ok, this is weird design. Array#product(something_enumerable) will produce the set operation of combining each element with all elements in arr. Set#product will not - instead it will multiply all elements with each other.
<Yxhuvud>
(oh, and array#product will also multiply if given none or a numeric argument).
<BlaXpirit>
Yxhuvud, that's what thoughtless pull requests do
mgarciaisaia has joined #crystal-lang
<FromGitter>
<asterite> I actually find it OK. Different overloads. One is cartesian product, one is product of elements...
<FromGitter>
<asterite> Thought I wouldn't mind renaming one of the products to cartesian_product
<RX14>
yes
<RX14>
I think that's my favourite solution
<Yxhuvud>
I guess my main issue is that it doesn't exist for Set currently :)
<FromGitter>
<asterite> But you are aware that Set#product(Set) will return Array, right?
<FromGitter>
<asterite> I'm not against Enumerable#cartesian_product(Enumerable), but it will return an Array
<Yxhuvud>
currently? where is that defined?
<Yxhuvud>
or are you talking about a possible implementation? That'd be fine - Array(Tuple()) would make sense IMO.
<RX14>
+1
pawnbox has quit [Remote host closed the connection]
<Yxhuvud>
asterite: though reading the code I see it sort of conflicts with that reuse flag you added the other day. hmm.
pawnbox has joined #crystal-lang
Philpax_ has quit [Ping timeout: 256 seconds]
<RX14>
overflow checking seems like quite an involved topic regarding performance
<RX14>
I think doing what rust has done and only doing overflow checks on non-release builds is the best way
<RX14>
with the addition of always-check and always-overflow possibilities too
<FromGitter>
<martinium> So to learn Crystal you need to essentially learn Ruby?
<RX14>
no
<RX14>
i wouldn't say that's true at all
<FromGitter>
<martinium> I don’t know Ruby and am attempting to learn Crystal by reading the docs and looking for tutorials
<RX14>
there's a strong design connection between the two languages
<RX14>
it's like C and C++ except less so because C++ is a near superset
<FromGitter>
<martinium> the syntax is almost identical as well as many methods etc
<RX14>
the syntax is identical but the semantics are not
<FromGitter>
<martinium> yeah I’ve noticed some differences already
<RX14>
crystal is almost too close that rubyists expect the semantics to be the same
<FromGitter>
<martinium> yeah but I’m glad it isn't
<RX14>
so not being encumbered by ruby expectations is somewhat of a benefit
<RX14>
i don't think crystal's learning resources are so dire you need to learn ruby though
<FromGitter>
<martinium> what would you say is the most comprehensive crystal learning resource at present?
<RX14>
the official docs for the syntax
<RX14>
the api docs for the stdlib
<RX14>
and then practics
<RX14>
practice*
<FromGitter>
<martinium> yeah that’s essentially what I have been doing.
<FromGitter>
<martinium> the stdlib is modern
<FromGitter>
<martinium> saved me a ton of time
<FromGitter>
<spalladino> I wouldn't neglect reading the source itself also as a learning resource: since most of crystal is written in crystal (unlike ruby, where you stumble upon C much too often), it's a good place to learn how things are done
<RX14>
indeed
<FromGitter>
<martinium> I was actually browsing the source also
<FromGitter>
<martinium> many of the documentation examples are embedded in the source files
<FromGitter>
<martinium> there were little things I had to look up such as what @ and @@ meant
<FromGitter>
<martinium> instance vs class variables
<FromGitter>
<martinium> now I know
<RX14>
yes, the API documentation is generated from source
<RX14>
and @@ vs @ are in the language docs
<FromGitter>
<martinium> I need to rad through it all instead of googling
<RX14>
the stdlib isn't close to 100% documented so you will have to venture into the source quite often
<RX14>
yes, I would recommend taking the time to read it all
<RX14>
instead of trying to work it out as it comes up
<FromGitter>
<martinium> I will be doing that
<FromGitter>
<martinium> learning this and nim at same time
<FromGitter>
<martinium> both very promising
<FromGitter>
<martinium> I have been waiting for high-level languages with everything we love that compile to native binaries
<RX14>
i've had a look at nim but it doesn't really interest me
<FromGitter>
<martinium> I value performance and think everyone should especially as mobile devices need every ounce squeezed out to preserve battery life etc
<FromGitter>
<martinium> it’s a more imperative systems language
<FromGitter>
<martinium> don’t think it has classes
<RX14>
it has UFCS which can get the job done of classes somewhat
<FromGitter>
<martinium> yeah I was reading about its OO patterns
<FromGitter>
<martinium> similar to smalltalk
<FromGitter>
<martinium> but yeah I want to get good enough with crystal where I can accomplish some tasks I have in mind
<FromGitter>
<martinium> also contribute libs and docs
<RX14>
i find the best way to learn programming languages is with a project in mind
<RX14>
and contributions are always welcome :)
<FromGitter>
<martinium> my problem is my projects in mind are pretty complex for a beginner
<RX14>
well, so are mine, typically
<FromGitter>
<martinium> one of the reasons I am loving the stdlib is because it has http built right in as well as json
<FromGitter>
<martinium> I want to be able to write wrappers for rest API's
<RX14>
the first projects I built in my first and second programming language were really quite large
<FromGitter>
<martinium> but that alone lacks documentation in the wild for many languages
<RX14>
yeah, crystal's JSON is really nice
<FromGitter>
<martinium> lots of ideas I have
<RX14>
with JSON.mapping
<FromGitter>
<martinium> I haven’t used JSON.mapping yet
<RX14>
it's pretty much the best way to parse JSON
<FromGitter>
<martinium> that is used when you are creating your own JSON or when you want to map certain keys to specific data types?
<RX14>
it can be used for reading and writing JSON
<RX14>
you create a class for each json object and then call RootObject.from_json(json)
<FromGitter>
<martinium> yeah I will have to dig in to that to learn it well.
<FromGitter>
<martinium> I have programming fundamentals pretty much down. Just need to get used to libs, patterns common usage etc
<FromGitter>
<smarr> @maiha_twitter perfect! thanks, that's it :smile: I saw the `forall` note in the release notes, but didn't make the connection. Thanks!
<RX14>
@smarr I find it hard to believe that java almost always wins. Did you happen to create your initial implementation in java and then port it to the other languages?
<FromGitter>
<asterite> One thing that could be improved is that, for example, Vector in som in Crystal is implemented on top of Array, so then you have like three indirections... in Java it's implemented on top of Object[]. The equivalent in Crystal would be Slice, not Array. That might speed up things a bit
<RX14>
indeed
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<RX14>
especially as crystal's array is resizable
<RX14>
actually i don't think it matters
<RX14>
for just access/set
<RX14>
it should have the same performance as Slice
<FromGitter>
<asterite> There are also many nilable types... hmm...
<FromGitter>
<asterite> I'll try to see what can be done
<RX14>
so many unions
<FromGitter>
<asterite> In any case it doesn't matter, Crystal is closest to Java so probably it's as fast or faster... it's just too many changes to the benchmarks for me to do :-(
<Yxhuvud>
Vector2D, Vector3D, RbtEntry and Collision should probably be structs. That would help out the GC somewhat
<Yxhuvud>
@smarr
<RX14>
indeed
<RX14>
however @smarr might object saying that he's trying to measure language implementation
<RX14>
including GC
<Yxhuvud>
Then I'd respond by arguing he should write ideomatic code.
<RX14>
but thats not the point if you read the readme
<RX14>
there's plenty of banchmarks comparing idiomatic code
<RX14>
if you write the exact same implementations in multiple languages you can compare the languages runtimes
<Yxhuvud>
RX14: read point #2 of that readme..
<RX14>
hmm
<RX14>
the question is whether you are trying to benchmark to compare or benchmark to improve
<RX14>
benchmarking to compare would want to most idiomatic basic implementation
<RX14>
benchmarking to improve would hit on all the weak points
<Yxhuvud>
but it also tells to use only those abstractions. I guess it is up to the maintainer of the project to decide.
<RX14>
which is in crystal the GC
bjz has joined #crystal-lang
dtzu has quit [Ping timeout: 265 seconds]
<adam12>
Excited to see what crystal-db brings to the rest of the ecosystem.
bjz has quit [Read error: Connection reset by peer]
mgarciaisaia has quit [Quit: Leaving.]
vivus-ignis1 has joined #crystal-lang
vivus-ignis has quit [Remote host closed the connection]
vivus-ignis1 is now known as vivus-ignis
matp has quit [Excess Flood]
<RX14>
indeed
<RX14>
i feel that crystal-db really fits crystal where maybe some of the other DB libraries do not
vivus-ignis has quit [Ping timeout: 265 seconds]
BlaXpirit has quit [Quit: Bye]
FromGitter has quit [Remote host closed the connection]
FromGitter has joined #crystal-lang
BlaXpirit has joined #crystal-lang
bjz has joined #crystal-lang
bjz has quit [Client Quit]
soveran has joined #crystal-lang
soveran has quit [Remote host closed the connection]
soveran has joined #crystal-lang
soveran has quit [Remote host closed the connection]