pawnbox has quit [Remote host closed the connection]
<jhass>
ruslux: sorry, got to see something that's self contained and actually reproduces
gettalong has quit [Ping timeout: 260 seconds]
jeromegn has quit [Remote host closed the connection]
jeromegn has joined #crystal-lang
hangyas has joined #crystal-lang
pawnbox has joined #crystal-lang
rolha has joined #crystal-lang
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
rolha has quit [Ping timeout: 252 seconds]
<ruslux>
jhass: I understood the reason, when I write struct Nil IN my module I shadow built-in struct Nil. I think struct ::Nil use built-in struct, not shadow.
<jhass>
ruslux: that's clear
gettalong has joined #crystal-lang
<jhass>
I'm not sure how that ends up being relevant/a conflict though
<jhass>
did you expect to be able to extend built in primitives inside your namespace only that way or so?
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
FromGitter has quit [Quit: Shutting down]
Philpax has joined #crystal-lang
splattael has quit [Quit: Connection closed for inactivity]
Oliphaunte has joined #crystal-lang
A124 has quit [Quit: '']
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #crystal-lang
Oliphaunte has quit [Ping timeout: 260 seconds]
A124 has joined #crystal-lang
perks has joined #crystal-lang
perks has quit [Client Quit]
<crystal-gh>
[crystal] asterite closed pull request #2875: Spec: show time in human friendly way in `--profile` (release/0.18...feature/spec_human_time) https://git.io/voaJx
<crystal-gh>
[crystal] asterite pushed 1 new commit to master: https://git.io/vooFO
<asterite>
jhass: yes, that spec is flawed. I'd change it to see if it matches any of the expected outputs, as I said before
<asterite>
I won't cancel the 0.18.3 release or tag for that failed spec, though
<asterite>
everything works fine, the compiler isn't flawed, just the spec
<asterite>
I hope 0.18.3 will be the last release in that branch
<asterite>
Then we should think how to do those conditional checks... maybe in macros one should be able to easily compare versions. We so have a SemanticVersion class in the std
<BlaXpirit>
asterite, i often see people complaining here that their specs fail
Oliphaunte has joined #crystal-lang
<BlaXpirit>
last time that was with libxml2, now it'll be even more of an outrage. no, i'm kidding of course, but still not nice
<asterite>
I guess the solution is to implement the whole standard library in crystal, so all dependencies will be there
<BlaXpirit>
is there some kind of `select` to notify me when either a socket or a channel are ready for reading? I really don't see a way to properly implement socket communication. currently I have one fiber that receives from the socket and I write to it in another fiber but surely that can't be good
<asterite>
well, mmm... I didn't release 0.18.3, I'll wait to see if travis gives the OK and then I'll run the scripts again to release 0.18.4, so there won't be a 0.18.3
<jhass>
<3
<asterite>
BlaXpirit: as far as I know, waj wanted to have a select primitive that would work on channels, io, timeout, etc., I don't know how much of that is possible though
<BlaXpirit>
well it needs to be based on OS descriptors or something
<asterite>
I think in go the way to do it is to use channels (like anything else)
<jhass>
BlaXpirit: given OpenSSL sockets, relying on libevent and just having a fiber blocked on a read should actually be bit more reliable
<jhass>
select(2) utterly fails on OpenSSL wrapped sockets
<jhass>
even in Ruby
soveran has quit [Remote host closed the connection]
<BlaXpirit>
jhass, wait, I'm not sure I understand you correctly
<jhass>
I don't see a real issue with writing to from another fiber at the same time
<BlaXpirit>
the trouble starts when I want to reconnect
<BlaXpirit>
one fiber is in the process of recreating the socket and another might still send to the old one or something
<jhass>
well the reader should wake up as you close(2)
<RX14>
and the the user of the parser would handle that header itself
<jhass>
so if it's not
<jhass>
we can't make a string out of it
<RX14>
it's not what?
<jhass>
base64 encoded and it is binary data
<RX14>
can't we?
<jhass>
we can't
<RX14>
so...
<jhass>
String has to be valid UTF-8
<RX14>
shit
<RX14>
so how is the body in http a string?
<jhass>
convenience
<RX14>
what if someone sends a binary png image in a HTTP post
<jhass>
there's body_io
<RX14>
then does it break?
<RX14>
oh
<RX14>
i guess we'll have to return Bytes then
<jhass>
or can we make an IO wrapper that stops reading on the boundary?
<RX14>
we covered this, it would be way too complex
<jhass>
so you .next.read, .next.read
greengriminal has quit [Quit: Leaving]
<jhass>
is it?
<RX14>
we would have to create some sort of strange IO wrapper which allowed multiple io "cursors" into the request
<RX14>
because multiple fibers can be reading from the same multipart request at the same time
<jhass>
mmh
<jhass>
I wonder if we can find an API that prevents multiple instances
<RX14>
so we have to read the whole multipart section into memory
<RX14>
the only way would to do some special IO magic which reads the request into a buffer as it comes in, then creates bounded MemoryIO instances for the in-memory buffer
<RX14>
but thats bad because then the whole request has to be in-ram anyway
<RX14>
which kinda defeats the point of streaming somewhat
<RX14>
plus i'm not up for that implementation
<jhass>
yeah no, that'd be bad
<jhass>
still thinking whether we can find an API that prevents multiple instances
<jhass>
we could document that you can't share a parser instance among fibers
<RX14>
we could just use a lock on the whole thing
<jhass>
and then do something like .read_part do |io|
<RX14>
actually
<RX14>
no
<jhass>
yes I wouldn't lock, just document don't share
<jhass>
I mean sharing an IO and reading from it in multiple fibers probably already gives potentially weird and unpredictable results
<jhass>
so it's not unseen in the language
<RX14>
thing is there is no way that we can allow the client to use the IO after calling "next" again
<RX14>
because we need to move the "cursor" to find the next block
<jhass>
yes
<jhass>
hence the idea of the block based API there
<jhass>
instead of next
<RX14>
it's a situation where the IO is only valid in this fibre in this block
<RX14>
and thats super easy to break
<jhass>
mh, perhaps external iteration is not worth it after all
<jhass>
.each_part do |io| and then pass the io whereever you want it inside the block
<jhass>
and break if you want to stop consuming
<RX14>
i really don't want to have to deal with bounded IO streams
<jhass>
we pseudo close the io after the block so if it's collected for use after the block they just get closed IO errors
<RX14>
it seems a bit difficult
<RX14>
jhass, i guess
<jhass>
is it? IO's implementer interface for reading is just read(Slice)
<jhass>
you read that slice from the underlying IO
<jhass>
check if it has the boundary
<jhass>
truncate it before the boundary if it has
<jhass>
and pseudo close the IO
<RX14>
and perform weirdness if the boundary is on a buffer boundary
<RX14>
jhass, do you want to work on it if I give you commit access to my shard?
<jhass>
I'll think about it, can't say yet. Not today
<RX14>
hmmn
<RX14>
i guess we can add a streaming interface later
<jhass>
I have the feeling there's a smart solution to this, I just don't see it yet
<RX14>
yeah
paulcsmith_ has joined #crystal-lang
<jhass>
continue consuming byte per byte while the end looks like the start of the boundary perhaps?
<jhass>
keep the extra read stuff in a buffer as big as the boundary
<jhass>
prepend on the next read if the boundary wasn't found
<RX14>
well
<RX14>
i would read the underlying IO into a buffer
<RX14>
scan it for "\r\n"
<jhass>
that kinda defeats the point of a streaming API
<jhass>
the part could be gigabytes
<RX14>
by buffer
<jhass>
think uploading some VM image or whatever
<RX14>
i mean small buffer
<RX14>
like
<RX14>
8kb
<RX14>
if we found \r\n then calculate if the end will leave the end of the buffer
<RX14>
then we can create a new slice with guaranteed to be enough space for the whole delimiter
<RX14>
and read the rest into that
<RX14>
then we can strcmp it
<RX14>
but your way sounds somewhat the same approach
<jhass>
I think so
<RX14>
so read into buffer
<RX14>
iterate buffer
<jhass>
would need to scan the whole user provided buffer too anyway
<RX14>
well
<RX14>
yes
<jhass>
yes
<RX14>
but do we read into the user provided buffer then scan and return the correct length
<RX14>
because that's technically correct
<jhass>
yes and we need to store the rest for the next read
<jhass>
which may be a new pseuod IO
<jhass>
sigh
<RX14>
but i'm saying the supplied slice might get beyone the boundary
<RX14>
in the clice
<RX14>
we just return the correct length
<RX14>
and thats technically correct and easier and less memory copies
<RX14>
but a little messy
<jhass>
the thing is that we can't seek back
<jhass>
so we have to keep the cut off part for the next read
<RX14>
ahh fuck
<jhass>
I wonder how slow reading byte by byte would be
<jhass>
given the underlying IO is buffered anyway
<RX14>
gets does it that way
<RX14>
it reads a utf8 byte and appendsit to a memoryio in a loop
<jhass>
so perhaps not too terrible, at least for an initial implementation
<jhass>
yes and we could just write into the user slice
<jhass>
and watch out for the boundary while doing that
<RX14>
yep
<jhass>
if we find the boundary, we consume it
<RX14>
yep
<jhass>
if we find something that looks like it for the first few bytes
<RX14>
if we find \r\n
<jhass>
and exceed the user request bytes while doing that
<RX14>
read the rest into memoryio
<RX14>
and then compare it
<jhass>
we store the rest in a buffer max as big as the boundary
<RX14>
if it fails dump to slice
<jhass>
and prepend that to the next read
<RX14>
somehow
<jhass>
I think a byte by byte reader could work, yeah
<RX14>
but i dislike the API
<RX14>
it's not really safe
<jhass>
it's not exception free
<RX14>
neither is my crrent one
<jhass>
we do can guard against bad reads/bad sharing
<jhass>
each iteration gets its own pseudo IO that has a closed state
<RX14>
hmmn
<jhass>
a struct anyway
<RX14>
yes
<jhass>
if @closed we raise
<jhass>
and we close it after the block
<RX14>
yes...
<jhass>
so if someone stores it
<jhass>
they can't read it
soveran has quit [Remote host closed the connection]
<RX14>
that does sounds possible
<RX14>
still a pain in the ass though
<jhass>
it is admittedly more complicated that just slurping all into memory and calling the user, yes ;)
<jhass>
but hey, let's read bigger files than RAM available ;)
<RX14>
yeah
<RX14>
*sigh*
<jhass>
thank you for keeping up with that btw
<RX14>
we can build pretty much any API on top of that one
<jhass>
yes
<RX14>
keeping up with what?
<jhass>
me, largely, but also the task
<RX14>
we seemed to understand each other for the most part
paulcsmith_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
th___ has joined #crystal-lang
<RX14>
:)
th___ is now known as Garbaz
<jhass>
<3
Garbaz has quit [Client Quit]
<crystal-gh>
[crystal] asterite pushed 1 new commit to master: https://git.io/vo6ul
<crystal-gh>
crystal/master bab00a9 Ary Borenszweig: Added `Thread.current`, and store current fiber in Thread
<jhass>
:o
<BlaXpirit>
:O
<jhass>
well that means no fixing generics this cycle then
<jhass>
I guess
<RX14>
damnit
<BlaXpirit>
and this is gonna be a long cycle
<BlaXpirit>
problems cropping up all throughout stdlib. mmmm
<RX14>
i need to have another go at fixing power_assert.cr
<jhass>
the *args, and *T stuff might have helped somewhat in that, dunno
<RX14>
I sorta liked being singlethreaded
<RX14>
it meant i could forget about threadsafety for the most part
<BlaXpirit>
hey, maybe there will be a flag
<RX14>
yeah...
<jhass>
well, if so you'd want it to be runtime :/
<RX14>
we all know how that goes
pawnbox has joined #crystal-lang
<jhass>
or perhaps we could have a Crystal::ThreadPool.size= or so
<RX14>
yeah
<jhass>
would leave the programmer the choice to make it compile or runtime configurable
<RX14>
would be... interesting if the stdlib detected size = 1 and disabled thread safety
<RX14>
for that 1% extra performance
<RX14>
would also be completely useless and impractical
<RX14>
and a terrible idea
<jhass>
:D
pawnbox has quit [Ping timeout: 250 seconds]
<jhass>
I'm actually not sure how much thread safety we should do in stdlib
<RX14>
well
<RX14>
collections should be threadsafe
<RX14>
otherwise wtf
<jhass>
I think they aren't in a lot of languages?
<RX14>
they should be
<jhass>
if the synchronization mechanism is reasonably easy to use and unnoisy I'm actually not fully opposed to leaving it to be the programmers responsibility
<jhass>
but didn't make my mind up yet
<RX14>
i would hope that if thats true then the default threadpool size is one
<jhass>
interestingly I'd currently even value IO thread safety higher than collection thread safety atm. Can't really nail a reason, just a strange feeling
<RX14>
because all the langauges with non-threadsafe collections have explicit threading
<RX14>
instead of boom you're rescheduled to another core now
<RX14>
well
<jhass>
well our unit of concurrency is a fiber
<RX14>
yeah
<jhass>
don't concurrently access from two fibers
<jhass>
which thread the fiber runs on, who cares
<jhass>
this isn't that different from explicitly controlling the threads IMO
<jhass>
if you have only a single fiber ever anyway
<RX14>
well
<jhass>
you'll never have a concurrent program
<RX14>
yes
<BlaXpirit>
RX14, really doubt anything will be threadsafe, the point is you just use channels for communication :/
<jhass>
even if that fiber hops threads
<RX14>
but most lnaguages don't make creating fibers so easy
<jhass>
yes, I see that it can be surprising if you're inside a HTTP::Server handler for example
<jhass>
it might be hidden
<RX14>
yes
<jhass>
as said, not fully made up my mind yet
<RX14>
who knows when you're going to pop out in another fiber
<RX14>
it'll get messy
<RX14>
because of yield
<RX14>
yield makes it too easy
<BlaXpirit>
RX14, you don't mean the keyword yield probably, i was about to object
<jhass>
yes, user side synchronization would need to be as easy
<RX14>
BlaXpirit, i mean the yield keyword
<jhass>
array.synced.each
<BlaXpirit>
RX14, but it actualy does not do anything related to threads
<BlaXpirit>
it's just a way to rearrange your code, everything is inlined
<RX14>
yes but i'm saying that it's too easy for the method you're calling to yield in a different fiber
<RX14>
and you not know
<jhass>
BlaXpirit: the point is that if you do array.map! do end in two fibers
<RX14>
i know how yield works :)
<BlaXpirit>
don't even need that, IO will make it jump all the time
<jhass>
that might happen at the same time
<RX14>
BlaXpirit, ?
<jhass>
well yes, you actually could have that problem already if you'd do IO inside a array.map! or similar
<jhass>
never occurred to me, interesting
<BlaXpirit>
nothing really interesting here, this will be a huge mess just like typical threading
<jhass>
that also means with by default synchronized collections that situation would easily deadlock, mmh
Oliphaunte has quit [Remote host closed the connection]
<jhass>
ah no, nvm
<RX14>
see
<RX14>
KISS
<RX14>
no concurrency
<RX14>
it's better
<jhass>
and then you end up with fork and IPC, ugh
<BlaXpirit>
async/await is the right way to do concurrency, but whatever
<RX14>
we don't even have any sort of promise type
<RX14>
and honestly
<jhass>
oh actually we do I think
<RX14>
async/await is one way to do it, CSP is another
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<jhass>
well actually there's public toplevel API to it
|2321 has quit [Quit: Connection closed for inactivity]
<asterite>
we'll tackle threads and parallelism first, then generics and other things. On the internet whenever someone talks about Crystal they say "this language is so nice, too bad it runs on a single thread". No one complains about generics not working well... probably because they don't use generics, at least not with complex relationships (which doesn't mean that it shouldn't be fixed, just maybe that multiple threads is more visible and import
<asterite>
ant, because it will also define how APIs will be, and that will probably be a breaking change, but fixing generics is backwards compatible)
<RX14>
i feel way way more people complain about it not being on windows than being single threaded
<RX14>
thats what I would consider my number one priority
<asterite>
implementing windows is also backwards compatible
<RX14>
it's also very important
<BlaXpirit>
yeah windows is #1 by far
<RX14>
i know 5 or 6 people who would love to use crystal, but they need windows support
paulcsmith_ has joined #crystal-lang
<asterite>
if we implement windows people will write more programs that aren't prepared for parallelism, and once we add that there will be more broken programs than now
<BlaXpirit>
bleh they're always gonna be broken, it's too easy to make a mistake
<asterite>
I think parallelism is the #1 priority right now, because it will break many things... I'm thinking probably kemal will break, for example
<BlaXpirit>
almost everything using fibers will break, I guarantee it
<BlaXpirit>
so, sure, good point
<RX14>
yay lets make crystal harder to program in :/
<BlaXpirit>
but if there was an announcement "XXX is never gonna be implemented", the case where XXX = Windows support would turn me off the most and I'd immediately leave
<BlaXpirit>
just saying
<RX14>
nodejs gets along just fine being slingle threaded
<RX14>
they make a feature out of it
<RX14>
and i kinda like it
<BlaXpirit>
RX14, well yes, it's pretty good, and their paralelism model makes sense because at least it guarantees thread-safety
<BlaXpirit>
crystal does not currently have thread safety even though it runs in one OS thread
<RX14>
well of course it guarantees thread safety
<RX14>
they don't have threads
<RX14>
or fibers
<BlaXpirit>
sorry, I said parallelism - bad, bad! I mean to say concurrency
<RX14>
they have to deal with reentrancy safety though
Philpax has joined #crystal-lang
<RX14>
i just get flashbacks to CME hell in java
pawnbox has joined #crystal-lang
<RX14>
i could never get power_assert.cr to work in anything higher than 0.15
pawnbox has quit [Ping timeout: 244 seconds]
bjz has joined #crystal-lang
paulcsmith_ has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
paulcsmith_ has joined #crystal-lang
bjz has quit [Read error: Connection reset by peer]
soveran has joined #crystal-lang
paulcsmith_ has quit [Client Quit]
Oliphaunte has joined #crystal-lang
soveran has quit [Ping timeout: 244 seconds]
jsownz has quit []
bjz has joined #crystal-lang
<RX14>
yesss
<RX14>
i got it working
<jsownz>
vhosts /Cloaks
Philpax has quit [Ping timeout: 276 seconds]
<ssvb>
RX14: if you don't care about threads, then you don't need performance, and if you don't need performance, then you can just use Ruby
<RX14>
ssvb, that's not true
<RX14>
in any way
<RX14>
i can get more prformance from crystal on 1 core than from ruby on 8
<RX14>
and i don't use crystal just for the performance
<ssvb>
and I can get even more performance with C++ and OpenMP
<RX14>
i'm sure you can
<RX14>
crystal is my personal favourite balance between performance and developer productivity
<RX14>
ruby is too slow, and probably actually less productive in some cases because of it's dynamic nature
<RX14>
C and C++ are more performant but hurt my productivity
bjz has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<ssvb>
Ruby is fast enough for many things, especially if the bottleneck is in the standard library
<RX14>
did you read any of what I just said about type safety?
<ssvb>
but when it is not fast enough, then the performance usually does matter
Oliphaunte has quit [Remote host closed the connection]
Oliphaunte has joined #crystal-lang
<xaxes`>
Typing matters.
bjz has joined #crystal-lang
bjz has quit [Client Quit]
<RX14>
I fixed power assert! The one library keeping me on 0.15.0, finally I can update all my things
<RX14>
Wish more people used that library instead of the ugly default spec assertions
<RX14>
Groovy has power assertions as part of the language, and it works beautifully, I would advocate for power assertions to be merged into crystal myself, at least the spec library