<FromGitter>
<bcardiff> but nowadays with some of the new macros it should be easier. And I know there is another sample of ast dump, but I can't find it right now.
mgarciaisaia has left #crystal-lang [#crystal-lang]
snsei has joined #crystal-lang
Sija has quit [Ping timeout: 250 seconds]
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 276 seconds]
crystal-lang673 has quit [Ping timeout: 250 seconds]
soveran has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
soveran has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 272 seconds]
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
pawnbox has quit [Ping timeout: 264 seconds]
pawnbox has joined #crystal-lang
snsei has quit [Remote host closed the connection]
mark_66 has joined #crystal-lang
snsei has joined #crystal-lang
snsei has quit [Ping timeout: 240 seconds]
hangyas has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 264 seconds]
pawnbox has joined #crystal-lang
unshadow has joined #crystal-lang
unshadow has left #crystal-lang [#crystal-lang]
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 244 seconds]
<crystal-gh>
[crystal] ysbaddaden opened pull request #2959: SecureRandom: use getrandom (linux) and /dev/urandom (master...std-securerandom-getrandom-urandom) https://git.io/vKIXp
mark_66 has quit [Quit: Leaving.]
mark_66 has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
hangyas has quit [Ping timeout: 240 seconds]
Oliphaunte has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 260 seconds]
ysbaddaden has joined #crystal-lang
<ysbaddaden>
hello, I can't compile the master branch with Crystal 0.18.7 : "Error in ./src/fiber.cr:26: this 'initialize' doesn't explicitly initialize instance variable '@stack_top' of Fiber, rendering it nilable"
<jhass>
hang on, let me try
<ysbaddaden>
if I return to a52faf2be21f8d3d82df46d0a09ae1d3475d4c34 (right before bd01fb2a3b41f2762ee131938d3f18349bdb327e that replaces ifdef for the flag? macro) then it's fine
<jhass>
travis seems fine though
<ysbaddaden>
I can compile master now that I compiled a52faf2be21f8d3d82df46d0a09ae1d3475d4c34 but 0.18.7 couldn't
<ysbaddaden>
btw, why `if !flag?(:without_openssl)` and not `unless flag?(:without_openssl)`
<jhass>
eh no reason, better :)
<jhass>
*unless is better
bam0 has joined #crystal-lang
bam0 is now known as alsm
<alsm>
is this; io.write(Slice.new(buffer.to_unsafe, buffer.size)) the right/best way to take an Array and write it to an io?
<ysbaddaden>
that depends on the array
<ysbaddaden>
it must be an `Array(UInt8)`
<alsm>
it is
<ysbaddaden>
or you'll have to compute the size (eg UInt32 => buffer.size * 4)
<alsm>
thanks, is definitely an Array(UInt8)
<alsm>
and it isn't modified after I take the pointer
<ysbaddaden>
I guess it's okay. A StaticArray or Slice would be safer / faster, but we can't always use one
snsei has joined #crystal-lang
soveran has quit [Remote host closed the connection]
hangyas has joined #crystal-lang
<jhass>
alsm: where does it come from though?
snsei has quit [Ping timeout: 276 seconds]
<alsm>
jhass: where does what come from? The Array?
<ysbaddaden>
@alsm: you could use a `Slice(UInt8).new(@size + 4)` instead
<ysbaddaden>
jhass: I have Crystal 0.18.7 [68783f1] (2016-07-03) from the official Ubuntu debs
<alsm>
I didn't see << as a function on Slices, don't you have to add data to a slice by index?
<jhass>
ysbaddaden: only difference would be that your crystal 0.18.7 was compiled with 0.18.6 while mine was compiled with the 0.18.7 tarball, I'd be surprised if that makes a difference though
<ysbaddaden>
other difference: it's not the same git sha1
Oliphaunte has joined #crystal-lang
<ysbaddaden>
you may write to the IO directly and multiple times, instead of building the array
<jhass>
ysbaddaden: eh well, mine was just built outside a git repo
<alsm>
I had thought about that too, as you can probably guess it's a network protocol, I wasn't sure about when the io might flush and getting packets fragmented.
<jhass>
it's TCP no? the OS should do the right thing for you
<alsm>
it is, I should probably do it that way and see if I have any problems
<jhass>
alsm: you can also make PROTOCOL_NAME a tuple
<jhass>
and I would consider remaining_length -> write_remaining_length and pass it the io
Oliphaunte has quit [Ping timeout: 258 seconds]
<alsm>
thanks
<ysbaddaden>
No problem to compile master with 0.18.7 on Alpine. That's weird.
<ysbaddaden>
and now I don't have any problem with 0.18.7 on Ubuntu. That's even weirder :sob:
<ysbaddaden>
btw IO#read_bytes and IO#write_bytes are nice to read/write any Int or Float (16, 32, 64), I use it extensively for the HTTP2 protocol
ysbaddaden has quit [Ping timeout: 250 seconds]
soveran has joined #crystal-lang
soveran has quit [Changing host]
soveran has joined #crystal-lang
Sija has joined #crystal-lang
Sija has quit [Ping timeout: 250 seconds]
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
ssvb has quit [Remote host closed the connection]
Sija has joined #crystal-lang
pawnbox has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
Raimondii has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
trapped has joined #crystal-lang
Raimondi has quit [Ping timeout: 240 seconds]
Raimondii is now known as Raimondi
pawnbox has joined #crystal-lang
Raimondii has joined #crystal-lang
Raimondi has quit [Ping timeout: 240 seconds]
ysbaddaden has joined #crystal-lang
Raimondii is now known as Raimondi
snsei has joined #crystal-lang
<Sija>
hi, any1 could help me with type overloading issue?
soveran has quit [Remote host closed the connection]
<jhass>
ysbaddaden: btw you should get a proper client and hang out here more :P
<ysbaddaden>
bad idea, I'd spend even less time on paid work :D
<jhass>
Sija: you have to downcast the things again
<jhass>
Sija: .first.as(TextWidget).value = "foo"
<jhass>
or better avoid adding them to an array
<ysbaddaden>
yup, you have an Array(Widget) and Crystal can't infer what subclass you have at what position
<Sija>
i was hoping for some meta-programming magic ;)
<ysbaddaden>
also Widget#value would then return an Union(String|Time|Bool) and you definitely never want that
Oliphaunte has joined #crystal-lang
mbarbar has joined #crystal-lang
globalkeith has joined #crystal-lang
Raimondi has quit [Remote host closed the connection]
Raimondi has joined #crystal-lang
<Sija>
ysbaddaden: and if i DO want set_value(value) argument to be Union(String|Time|Bool) so it will work for any of these but fail with RuntimeError when it's invoked on incompatible type?
<jhass>
removing the type restrictions in the childs should work I think
<ysbaddaden>
I guess
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
<Sija>
jhass: works gr8, thanks!
snsei has joined #crystal-lang
Oliphaunte has quit [Remote host closed the connection]
<paulcsmith_>
How do you handle debugging staging or prod servers? For example, if I want to debug why emails aren't going out I might use a Mailer class to send an email to myself to see if an error is thrown (bad API key for example) or if some other strange issue is happening. Without a REPL/console I'm not sure how I would do that. How do people handle this right now?
Philpax has quit [Ping timeout: 252 seconds]
alsm has quit [Ping timeout: 244 seconds]
<jhass>
if at all yet, I would guess installing the compiler there, having the source code there and writing little scripts that can be run with crystal foo.cr inside the project context
<paulcsmith_>
jhass Ah yeah the script things makes sense. A bit less convenient, but definitely doable! (less convenient because I use Heroku so I'd need to actually push and deploy to add a script, no ssh)
<paulcsmith_>
How would you use crystal eval? I'm not sure I understand that part :S
<jhass>
well it's just a compiler command
<paulcsmith_>
Oh I didn't realize that was a compiler command
<jhass>
takes the argument, writes it to a temporary file, compiles the file, runs the resulting executable
<paulcsmith_>
Oh that's pretty cool!
<paulcsmith_>
`crystal eval "puts 1 + 1"` prints 2. Saweet. Thanks jhass. That is probably going to be quite helpful!
<paulcsmith_>
Much easier than adding a file, commit, redploying, etc. :)
<jhass>
:)
mbarbar has quit [Ping timeout: 272 seconds]
trapped has joined #crystal-lang
mgarciaisaia has joined #crystal-lang
mgarciaisaia has left #crystal-lang [#crystal-lang]
snsei has quit [Remote host closed the connection]
Raimondii has joined #crystal-lang
mark_66 has quit [Quit: Leaving.]
<RX14>
jhass, you know we made that assumption that we made that if read didn't return slice.size it failed?
<RX14>
if the stdlib has read_fully then that assumption must not always be true
Raimondi has quit [Ping timeout: 240 seconds]
<jhass>
mh
<jhass>
I vaguely recall something about read_fully being redundant these days
<jhass>
and being from back of the days of non-evented IO
<RX14>
ah
<RX14>
well
<jhass>
might be wrong though
<RX14>
i really don't feel like adding another change to my pull request :/
<jhass>
asterite: do you remember the story of read_fully?
<asterite>
read can return less than what was requested
<asterite>
write must write the whole slice
<jhass>
ah, meh
<asterite>
read_fully reads fully
<jhass>
RX14: well, should be swap read with read_fully then
<jhass>
asterite: unless the stream closes I guess?
<RX14>
read_fully throws
<asterite>
it raises if there's not enough data to fill the slice
<RX14>
yes
<RX14>
and it returns
<RX14>
but if it returns then it always will return slice.size
<RX14>
which is weird
<asterite>
it should probably return nil
<RX14>
yeah
<asterite>
or, the return value is not important
<RX14>
if the return value is undefined then just make it nil
<jhass>
I wonder if "read until slice full or stream closed" shouldn't be the standard semantic of read
<RX14>
^
<RX14>
it would make a lot of things easier
<jhass>
and we should have a "read_nonblock" or whatever that does "fill the slice with the available data"
<jhass>
which can also be "nothing"
<RX14>
yeah... i don't know any mmore
<RX14>
read can't return nothing
<asterite>
read is read_nonblock
<jhass>
and I'm arguing that might be an unintuitive default
<RX14>
except for the nothing
<RX14>
read can't return 0 or it's EOF
Raimondii is now known as Raimondi
mgarciaisaia has joined #crystal-lang
jokke has quit [Quit: WeeChat 1.5]
jokke has joined #crystal-lang
tomchapin has joined #crystal-lang
tomchapin has quit [Client Quit]
snsei has joined #crystal-lang
tomchapin has joined #crystal-lang
<RX14>
read_fully is pointless because the EOFException it throws doesn't tell you how much of the buffer is filled
<RX14>
so you know you reached the EOF somewhere in the buffer
<RX14>
but not where
Oliphaunte has quit [Remote host closed the connection]
paulcsmith_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<asterite>
RX14: the use case of read_fully is when you expect N bytes, and not getting N bytes is an error from which you can't recover, and it doesn't matter how many bytes you could read
<asterite>
if you don't need to read exactly N bytes, you should use read
<RX14>
hmmn i guess
<RX14>
but then you have to reimplement most of read_bytes
<asterite>
what are you coding?
<RX14>
just to get the semantics where if read bytes != slice.size you're at the EOF
<RX14>
and those are really useful semantics
<RX14>
asterite, the bounded stream for multipart
<asterite>
so the idea is that you give the IO a string boundary and it stops at that point?
<RX14>
yep
<RX14>
surprisingly hard task
<asterite>
wrapping another IO, right?
<RX14>
yep
<asterite>
the bounday has to start in a new line?
<RX14>
nope
<RX14>
it's arbitrary bytes
<asterite>
why not use gets("delimiter")
<asterite>
there's gets(String)
<RX14>
because when the io is multiple gigabytes
<RX14>
and can be a png
<RX14>
that just doesn't cut it
<RX14>
png is not valid utf-8
<asterite>
oh, I see
<asterite>
good point
<RX14>
nor do you want to store 10gb in memory
<RX14>
when you can use a bounded IO and just write it to file
<RX14>
or do whatever
paulcsmith_ has joined #crystal-lang
Oliphaunte has joined #crystal-lang
soveran has quit [Remote host closed the connection]
snsei has quit [Remote host closed the connection]
snsei has joined #crystal-lang
mgarciaisaia has left #crystal-lang [#crystal-lang]
<RX14>
you cannot read over the delimiter if you only read delimiter.size bytes
<RX14>
because the delimiter is fixed length, you either have the delimiter at your location, or you have it in the future, but havent read enough into the buffer
<asterite>
Ah, good point
<RX14>
I wonder, maybe the most efficient implementation is the simplest
ysbaddaden has quit [Ping timeout: 250 seconds]
<RX14>
asterite, your implementation is also not constant memory
<RX14>
because MemoryIO isn't cleared when it's finished being used
<RX14>
it just has bits added on every partial match
<asterite>
Ah, yes, forgot to do that
<RX14>
it actually allocates 2gb of memory in my benchmark
<RX14>
because its matching so often
<asterite>
any reason why you don't use Benchmark.ips ?
<asterite>
I'll try to fix that so we can run the benchmark again
<RX14>
because thats iterations per second
<RX14>
not MB/s
pawnbox has quit [Remote host closed the connection]
paulcsmith_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<jhass>
that one has examples where the last part does not have the delimiter
<RX14>
jhass, uhh read the ABNF
<jhass>
yeah, but still read the example above
<RX14>
where is this?
paulcsmith_ has joined #crystal-lang
<RX14>
by the way
<jhass>
oh, it's meant to be ignored?
<jhass>
ugh
<RX14>
yeah thats the epilogue
<RX14>
"The key insight in this algorithm is that if the end of the pattern is compared to the text, then jumps along the text can be made rather than checking every character of the text. The reason that this works is that in lining up the pattern against the text, the last character of the pattern is compared to the character in the text. If the characters do not match there is no need to continue searching backwards along the pattern. If the character in the
<RX14>
text does not match any of the characters in the pattern, then the next character to check in the text is located n characters farther along the text, where n is the length of the pattern. If the character is in the pattern then a partial shift of the pattern along the text is done to line up along the matching character and the process is repeated. The movement along the text in jumps to make comparisons rather than checking every character in the text
<RX14>
decreases the number of comparisons that have to be made, which is the key to the efficiency of the algorithm."
<RX14>
oh oops
<RX14>
wrong clipboard
pawnbox has joined #crystal-lang
<RX14>
asterite, by the way, MemoryIO.clear does need to reallocate memory because MemoryIO looses knowledge of the original allocation size of the Pointer, so has to reallocate back up to size to fit written bytes
<RX14>
so basically all the IO::Delimited implementations can handle about 1gbps
<RX14>
which is enoguh for me
Raimondii has joined #crystal-lang
<jhass>
:)
<RX14>
(n cores)gbps when we implement parallelism
<RX14>
how will it work anyway?
A124 has joined #crystal-lang
<RX14>
manually creating threads or a thread pool for fibers
<RX14>
or both?
Raimondi has quit [Ping timeout: 240 seconds]
Raimondii is now known as Raimondi
ysbaddaden has joined #crystal-lang
hangyas has joined #crystal-lang
pawnbox has joined #crystal-lang
Oliphaunte has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
ysbaddaden has quit [Ping timeout: 250 seconds]
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
hangyas has quit [Quit: hangyas]
pawnbox has quit [Ping timeout: 276 seconds]
ysbaddaden has joined #crystal-lang
triangles has joined #crystal-lang
<ysbaddaden>
RX14: I'm not sure what you're implementing: is this the specific multipart/form-data for HTTP (RFC7578) or the generic MIME multipart for emails (RFC2045 to RFC2049)?
<RX14>
ysbaddaden, generic MIME
<RX14>
then i'm going to build form-data on top of that
<RX14>
a light wrapper of the underlying multipart parser that interprets the required headers
<RX14>
probably will have a callback architecture, but i'm not sure
<RX14>
it's hard to deal with large file uploads without resorting to what I would call hacks: saving incoming data to a temp file and presenting that
<RX14>
so i'll just use callbacks
<RX14>
if you want more control, like enumerating fields you can use the underlying multipart parser
<ysbaddaden>
writing large file uploads to a tempfile sounds like a requirement to avoid bloating memory
<RX14>
ysbaddaden, not if you expose it as an IO
<RX14>
and read it right from the socket as it's being processed
<RX14>
then the application can choose to do whatever it wants to it
<RX14>
including writing to a file for later processing
<RX14>
sending to another api/service over the network
<RX14>
whatever really
<RX14>
it's a much much better API
<ysbaddaden>
that depends on the IO, if it's a socket then it's fine... but HTTP/2 requires to load all received DATA frames (otherwise it would prevent receiving multiplexed frames) so the incoming IO may not be a socket
pawnbox has joined #crystal-lang
<RX14>
so you're saying http/2 is designed so that consumers can't set the pace of recieving data without impacting all the other users of the stream?
<ysbaddaden>
that being said, I'm using an IO.pipe so now it's a socket, but it consumes 2 file descriptors for each stream which is overkill
<RX14>
or is it just that you have to buffer each DATA fram
<RX14>
not just stop recieving half way through
<RX14>
because thats not much of an issue
<ysbaddaden>
it's multiplexed: you receive frames and must read them, otherwise you can't read the next frames, thus coming back to a pipeline (and killing all the HTTP/2 potential)
<RX14>
but it's not like tcp where you can say stop sending this stream?
<RX14>
or pause
<ysbaddaden>
for incoming (client -> server) ? no you can't control the flow
<RX14>
thats stupid
<ysbaddaden>
for outgoing (client -> server) the client (or proxy) can set control-flow to limit usage of some streams of the connection
<ysbaddaden>
sorry I meant server->client
<RX14>
wait client->server is different than server-> client in http/2?!?
<RX14>
what were they thinking
<RX14>
why not just implement the same protocol each way and treat it as a bidirectional data pipe
<RX14>
where some operations are only supported by certain sides
pawnbox has quit [Ping timeout: 244 seconds]
<RX14>
http/2 was a mistake
<ysbaddaden>
maybe I just said something stupid, RFC7540 doesn't say anything about client/server for WINDOW_UPDATE frames, so maybe the server can limit the client flow-control for each stream
<RX14>
i assume http/2's windows work kind of like tcp's windows
<RX14>
or do they not
<ysbaddaden>
sigh, I guess I'm far from done with the implementation of HTTP/2, even from just a server standpoint, not even thinking of client :fearful:
<ysbaddaden>
basically, a stream window size is only 65KB by default, and this must be increased by the receiver throught window_update frames so the sender can continue sending data
<RX14>
in that the sender knows the recipients remaining window size
<ysbaddaden>
yeah
<RX14>
65kb is a ok size to keep in memory I would say
<RX14>
keep a static 65kb buffer per stream
<ysbaddaden>
yeah, I just miss a writer/reader MemoryIO so I could skip the IO.pipe and the 2 fds
<RX14>
and issue window_update to refill that as its read
<RX14>
ysbaddaden, uhh memoryIO can read and write
<ysbaddaden>
you can't pipe data
<RX14>
what do you mean?
<RX14>
oh
<ysbaddaden>
because it doesn't remember both the reader and writer position
<RX14>
wouldnt you use 2 memoryIOs
<RX14>
plus a pipe of crystal stream seems to be better for this
<RX14>
pipe or*
<ysbaddaden>
I guess there is a nice solution... the IO.pipe just works for now and I have other things to beat (I totally missed the "server must send window update frames" part)
<RX14>
i think the nice solution is to use a curcular buffer
<RX14>
and have an IO that can read a circular buffer
<ysbaddaden>
good idea!
<RX14>
see jhass's IO::Delimited for a circular buffer implementation
<RX14>
and IO on top shouldn't be too hard i would think
<RX14>
well he's implemented a ring i'm not sure if it blocks like a true circular buffer should
<ysbaddaden>
I'll keep that in mind
<ysbaddaden>
it's getting late, I'm gonna doze off :)
<RX14>
a true blocking circular buffer would for sure be welcome in the stdlib
paulcsmith_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<RX14>
one where you can write on one end and recieve on the other like they were an IO pair but with a configurable size buffer and dome in crystal
<ysbaddaden>
a bunch of niceties would be welcome when it comes to binary protocols and IO (even thought there already are a lot of niceties)
<RX14>
also for gods sake can someone implement to_slice on Array, even StaticArray has it lol
<ysbaddaden>
eg: endianness on slices, set default endiannes on IO and slices ...
<ysbaddaden>
Array are mutable!
<ysbaddaden>
an Array#to_slice would have to copy the whole array
<RX14>
hmmn
<ysbaddaden>
StaticArray are immutable, so we can have a Slice from its pointer/size
<ysbaddaden>
but we can't with Array
<ysbaddaden>
hence we only to_unsafe which is *truly* unsafe (any change in the Array after getting the pointer -> potential segfault)
<RX14>
nah it wouldn't segfault
<RX14>
the memory readion from the GC would still be valid
<RX14>
Array doesn't resize the memory allocation iirc
soveran has quit [Remote host closed the connection]
<RX14>
but still, I find myself needing a slice over an array too often