<FromGitter>
<bigtunacan> "Crystal Weekly summarizes every news of the Crystal community in a short, focused newsletter. You can subscribe to it and even see the past issues on the web."
<FromGitter>
<fridgerator> cool
<FromGitter>
<fridgerator> I don't know whatsup w/ the website, maybe @sdogruyol would know
greengriminal has quit [Quit: Leaving]
pduncan has joined #crystal-lang
snsei has joined #crystal-lang
<FromGitter>
<bigtunacan> Where does the Crystal stdlib/api live on github?
snsei has quit [Remote host closed the connection]
snsei has joined #crystal-lang
statikowsky has joined #crystal-lang
statikowsky has quit [Ping timeout: 245 seconds]
snsei has quit [Remote host closed the connection]
muhlisbc has joined #crystal-lang
muhlisbc has quit [Quit: AtomicIRC: The nuclear option.]
<FromGitter>
<piyushranjan> I have a valid json (validated using online validators and got VALID JSON (RFC 4627)) which results in a crash (I have try, catch around json parsing)
<FromGitter>
<sdogruyol> Meanwhile crystalweekly.com is up again
mark_66 has joined #crystal-lang
<FromGitter>
<bararchy> @sdogruyol Nice ! I really love your updates !
statikowsky has quit []
<FromGitter>
<drosehn> iirc, the error you're seeing is because your JSON.mapping defines a field which does not exist in the JSON that you're reading, and it also has 'strict:true' defined.
statikowsky has joined #crystal-lang
statikowsky has quit [Client Quit]
<FromGitter>
<drosehn> Apologies if that doesn't make a lot of sense, but I woke up just a minute ago, and much of my brain is still asleep.
statikowsky has joined #crystal-lang
<FromGitter>
<drosehn> so, for instance, you have a field `v` defined as `v: String | Int32`, but in your data there is an entry which does not define `v`. So instead you need to define it as `v: String | Int32 | Nil`.
<FromGitter>
<drosehn> Although based on the error message, I'd guess that your field is named `#` instead of `v`, and that doesn't seem likely to my (tired) brain.
<FromGitter>
<drosehn> I'll say that in my experience with JSON.mapping (*which amounts to one single program*), it does work very well. However there are a few cases where a programming error in your program will generate an error message which does not make sense when you first see it. I even wrote up one PR thinking that JSON.mapping had an error in it, when in fact the error was in the JSON data that my program was reading,
<FromGitter>
... when compared to the JSON.mapping I had defined.
unshadow has joined #crystal-lang
unshadow has quit [Client Quit]
<FromGitter>
<piyushranjan> @sdogruyol I am doing JSON.parse over this json.
<FromGitter>
<drosehn> @piyushranjan -> I hope my comments make some sense, but I have to go back to sleep now.
Philpax has joined #crystal-lang
<FromGitter>
<piyushranjan> @drosehn Thanks for the message. JSON.mapping is not something that I have done. This is an existing JSON. I am parsing it using JSON.parse
<FromGitter>
<bew> @l3kn u right, interesting implementation, with many handy methods, maybe in the end I'll use sth like this
Ven_ has joined #crystal-lang
Ven_ has quit [Ping timeout: 245 seconds]
Ven_ has joined #crystal-lang
Ven_ has quit [Ping timeout: 260 seconds]
Ven_ has joined #crystal-lang
Ven has joined #crystal-lang
Ven is now known as Guest32458
<FromGitter>
<l3kn> @bew https://github.com/jon-hanson/parsecj might be a good inspiration, too. Do you plan on implementing all stuff from the original parser combinator paper?
<FromGitter>
<l3kn> Not sure if the performance would be great, but I'd love to have something like that available in crystal
Ven_ has quit [Ping timeout: 240 seconds]
<FromGitter>
<bew> thanks! I'm working on a cool parsing lib for crystal, it has been in my head for some month now, I've read a lot of papers on parsing in general, currently I'm just experimenting combinator parsing, to see how it's going with crystal
<FromGitter>
<bew> but my lib will not hopfully be tied to a parsing technic
<FromGitter>
<bew> I've seen the benefits of combinator parsing, and I'm experimenting, that's it for now
<FromGitter>
<bew> So I'm trying to get `haydari` parsing lib to compile with recent compiler version, I have a type error that shouldn't be one IMO, I tried to make a minimal version of it: https://carc.in/#/r/2304
mark_66 has quit [Remote host closed the connection]
Ven_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Ven has joined #crystal-lang
Ven is now known as Guest4882
Guest4882 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
onionhammer has joined #crystal-lang
Ven_ has joined #crystal-lang
onionhammer has quit [Ping timeout: 240 seconds]
<wontruefree>
can you deconstruct an array like `* [1, 2, 3]`
<FromGitter>
<werthen> Hey guys, what's the best way to contribute to crystal with little compiler knowledge? Any documentation or something that needs writing or...?
<wontruefree>
which I think I just figured out in a different way
<wontruefree>
so like
<wontruefree>
`Tuple.new(Array.new(100, 0))`
<Papierkorb>
werthen, Docs (See what you're missing, and then improve it), helping out here (or anywhere else), and/or creating and maintaining shards. You can also work on the standard library!
<Papierkorb>
werthen, for inspiration, you could look around the stdlib, and identify parts you care about and/or know about. Then check existing code, improve tests (try to break it!), and/or contribute additional functionality
<Papierkorb>
werthen, you can do of course the same with shards: Usually, authors don't bite :) Just build stuff in Crystal, and be on the look-out for what is missing.
<FromGitter>
<straight-shoota> I just noticed the keywords `deconstruct` and `splash` don't yield any results in the Gitbook :P
<RX14>
@werthen the stdlib is quite easy to get started in as it's all in crystal
<FromGitter>
<werthen> @fridgerator consider yourself hooked up
<FromGitter>
<fridgerator> yeah! thanks!
<FromGitter>
<straight-shoota> I'm 100 :star: ^^
<FromGitter>
<fridgerator> yes! thank you
<FromGitter>
<sdogruyol> :tada:
wontruefree has joined #crystal-lang
<wontruefree>
is there something like ObjectSpace in crystal
<FromGitter>
<bew> wontruefree I don't think so, what do you need to do?
<wontruefree>
trying to profile something
<wontruefree>
I am using benchmark report
<wontruefree>
but I am looking for memory allocation
<wontruefree>
I did an Array.new with Ints and got way more objects then I expected
<FromGitter>
<akzhan> @wontruefree GC is not part of Crystal project. It is separate https://github.com/ivmai/bdwgc one. Anyway I should ask author for required APIs. Binding is easy, already has GC.stats
xaxisx has joined #crystal-lang
<FromGitter>
<akzhan> *You
<wontruefree>
yeah I was just doing `puts GC.stats.total_bytes `
<wontruefree>
I would like to know if there are any tools in the language for profiling this
<wontruefree>
it is a little hard to figure out
_whitelogger has joined #crystal-lang
wontruefree has quit [Quit: Is gone... Just plain old gone]
miketheman has quit [Ping timeout: 240 seconds]
miketheman has joined #crystal-lang
jadams has joined #crystal-lang
wontruefree has joined #crystal-lang
Kug3lis has joined #crystal-lang
<Kug3lis>
Hi, I am feeling bit stupid I have a incoming JSON structure which I map to object and have property which is coming string binary and I want to convert it to hex representation. I just did like string.bytes.map(&.to_s(16)) which converts to completely different stuff than original JS library
<Kug3lis>
maybe I am doing something incorect?
splitty__ has quit [Ping timeout: 255 seconds]
<FromGitter>
<bew> maybe `String#hexstring` ?
<FromGitter>
<bew> huh sorry doesn't work
<Kug3lis>
yeah if it was so easy :(
<FromGitter>
<bew> there is `Slice#hexstring`
<FromGitter>
<bew> and `String#to_slice`
<FromGitter>
<bew> not sure that's what you're looking for
<Kug3lis>
I think something is wrong with parsing json, because I just dumped json to file and tried to parse with php same shit incorrect value :/
<Kug3lis>
TheGillies its better than parsing yml file to get version in code
<Kug3lis>
bew I think its related that string is in binary encoding
<FromGitter>
<bew> TheGillies it's for accessing the version at runtime, to respond to cli like `program --version`
<FromGitter>
<akzhan> Better to provide Shards API to get my own shard info.
<FromGitter>
<bew> Kug3lis yes indeed, maybe you'll need a custom JSON parser for that binary string
<Kug3lis>
Yeah I am already writing converter for it just trying to deal now with encoding have zero ideas :D
<FromGitter>
<bew> especially for `\u####` chars
<Kug3lis>
\u its standard json char encoding which are not in utf something
<FromGitter>
<bew> encoding is utf-8 by default, I don't think you should worry about it
<Kug3lis>
yeah I understand that, but from that JS code I see that strings is added to buffer as binary and then converted to hex but how that parsing in binary is done I have zero idea
zipR4ND has joined #crystal-lang
<FromGitter>
<bew> maybe your final type (the one you convert the string to) should be simply `Bytes` (which is a `Slice(UInt8)`, and `UInt8` is the equivalent of a C `char`)
<FromGitter>
<bew> so you end with binary data
<FromGitter>
<bew> a crystal `String` should contain only valid utf-8 chars
<Kug3lis>
I think I know whats problem faceplam :(
<Kug3lis>
JSON::PullParser read_string....
<zipR4ND>
hey all, I'm getting this kind of output deep down the rabbit whole: https://pastebin.com/raw/9XwDQafd can't easy reproduce atm, but can anyone give me a hint of what's happening to find a solution=
<zipR4ND>
?
<FromGitter>
<bew> yes, instead of writing a converter, write a custom json pullparser for your banary type
<Kug3lis>
Hmm, I compared string result after parsing json and from nodejs they both same so something is bad with my code trying to convert into hex
<FromGitter>
<bew> what kind of input & output do you have/need ?
<FromGitter>
<akzhan> anyway json should be utf-8 encoded by RFC
<Kug3lis>
i have binary string need to generate hex representation
<FromGitter>
<akzhan> write it to json as base64, for example.
<FromGitter>
<akzhan> and binary is not compatible with json at all. write your own modified json parser
<Kug3lis>
I cant do anything to what I receive its protocol
<oprypin>
zipR4ND, an LLVM error message almost always means that there's a bug in Crystal, so yeah, just gotta try to reproduce and report it
<Kug3lis>
I think something is bad with the way I try to convert I think, I need to change something with encoding because I tried to do the same with php and get the same result as in crystal. in JS it adds that string to Buffer with encoding binary I dont even imagine whats happening inside there...
<FromGitter>
<bew> <troll>that's JS, you probably don't want to know </troll>
<oprypin>
Kug3lis, what are you actually trying to do?
<oprypin>
i tried to follow the conversation but i can't get to the core of it
<Kug3lis>
I am writing webtorrent client in crystal so I could create Tracker because u know crystal Websockets Rocks
<Kug3lis>
communication is done under Websockets with JSON
<zipR4ND>
@oprypin I know, its just my code is pretty complex and it's really hard to reproduce this issue because I have no Idea why or where it is happening. I narrowed I so far that I now know that the problem seems to be two classes including one module. I on class includes the module and the other two inherit from that one the bug is gone. the module itself makes extensive use of macros (a nested combination of inherited, extended, included,
<FromGitter>
<akzhan> but said that it is not valid: offer_id => pack("H*","96e8df9bea4feec88bc1b6a83658442a70b23316")
<Kug3lis>
mm?
<FromGitter>
<akzhan> your offer_id field have invalid bytes for utf encodings.
<Kug3lis>
yeah I see
<FromGitter>
<akzhan> So only way is to create your own not-json parser that supports invalid strings
<FromGitter>
<bew> or use a custom type with a custom json::pullparser
<oprypin>
no
<FromGitter>
<bew> why not?
<oprypin>
Kug3lis, do you really have to try to transfer this gibberish through json?
<oprypin>
i tried to reproduce this result in Python and it also just makes no sense
<Kug3lis>
yeah it's hash but I am going to create issue on webtorrent side because its not valid UTF-8 value going to json, and JS has no problem with it
<FromGitter>
<akzhan> @bew pull parser saved raw value sas srtings as i see
<oprypin>
they treat first 256 unicode codepoints as byte numbers
<oprypin>
which is idiotic btw
<FromGitter>
<akzhan> Yest another approach - to read all file as bytes and represent all invalid sequences as any escape. Parse JSON. and convert these escapes to Bytes.
<FromGitter>
<piyushranjan> @sdogruyol I found a pattern to replicate the "expected Hash for # (key : String), not Nil (Exception)" with JSON.parse. Justa little context: What I am trying to do is to parse a bunch of json files. All the files are large but they are valid json files. When I try to parse them one by one in a loop I encounter this error with a specific file. However, if I try to parse just this file, I do not find this
<FromGitter>
... issue
<oprypin>
Kug3lis, what should be happening is that hexstring should be in json in the first place (or base64)
<oprypin>
if they used base64 it would literally take less bytes
<Kug3lis>
Yeah, thats what I wrote in ticket, but as its already accepted so in question if they invest time trying to fix this...
snsei has quit [Remote host closed the connection]
<FromGitter>
<piyushranjan> I am guessing that JSON.parse is creating a mapping and when I try to parse another json which is not consistent with previous json, it craps out. Questions is, is there any way I can destroy json variable here at that end of every iteration ?
Renich has quit [Quit: Renich]
pduncan has quit [Ping timeout: 260 seconds]
snsei has joined #crystal-lang
<oprypin>
piyushranjan, I think your assumption is incorrect
<FromGitter>
<piyushranjan> yes I am effing stupid
<FromGitter>
<piyushranjan> it is something else
<FromGitter>
<piyushranjan> really sorry for the botheration
<crystal-gh>
[crystal] wontruefree opened pull request #4463: Spec suite speed increase (master...spec_speed) https://git.io/vHsSL