<headius[m]>
not sure of the resolution but there's only two failures and they seem to be related to concurrent-ruby
<headius[m]>
olleolleolle: are you looking into the failures?
<olleolleolle[m]>
That's narrowing it down, very helpful! I'm not at a computer now. And, no I don't think I will - don't currently have JRuby in my critical path these days. New job.
<headius[m]>
olleolleolle: hopefully we or someone will have time to look into that... I know it can be frustrating for library authors when things aren't working as expected
subbu is now known as subbu|away
<headius[m]>
olleolleolle: looks like we're both poking at it
<headius[m]>
it's something causing JRuby's timeout executor to get terminated, so it rejects future requests
<headius[m]>
it also failed locally for me on Java 8 and 11
<headius[m]>
down side of fixing the most bugs is introducing the most bugs
<headius[m]>
enebo: what say you
<enebo[m]>
yes
<enebo[m]>
although I would like to make sure we try some other hitters like rb-inotify
<enebo[m]>
My main goal in the idea of a .12 is to get the known ickies bigger production users are hitting in 11.1
<headius[m]>
some of this has been improved lately on that end of things
<enebo[m]>
9.3 as a major ruby compat level means we will have some pressure and having a more solid 9.2 will help us not force too much change
<headius[m]>
I recently landed commits that attempt to shut down all threads, which should prevent the inotify thing
<enebo[m]>
I really thought 9.1.11 was that but a few more fixes and we should be good?
<headius[m]>
this after I learned that CRuby has been doing a kill/join on all threads at shutdown since forever
<headius[m]>
enebo: there will always be a few more fixes... I think .12 is valid mostly because it has been months since .11.0
<headius[m]>
and I don't think we're ready for 9.3 yet
<headius[m]>
(load service at a minimum)
<enebo[m]>
well we have some weird errors and not just workaround stuff
<headius[m]>
yeah
<headius[m]>
unavoidable bugs
<enebo[m]>
having AIIOBE and stuff is my main meaning for few fixes
<headius[m]>
yeah let's start gathering fixes for .12 and try to work with jsvd on the AIOOB
<enebo[m]>
we have thousands of things we could include :)
<enebo[m]>
yeah
<headius[m]>
I will do a merge from 9.2 to master so we're clean for some cherry-picking
<headius[m]>
and then I'll fix this on 9.2 branch
<enebo[m]>
ok cool
<headius[m]>
olleolleolle: I updated your PR with explanation
<enebo[m]>
I am still continuing with parser just because I am actually making progress again
<enebo[m]>
but I will fix the persist bug unless you happen to fix it
<enebo[m]>
I think I will just synchronize on the IRReader instance because how many peopel are concurrently loading pre-compiled files
<headius[m]>
merge appears to be trivial
<headius[m]>
no go ahead and fix that bug
<enebo[m]>
yeah I bet
<headius[m]>
I don't know enough of details to fix it right now
<enebo[m]>
I could also kill sharing a byte buffer but who knows what other stuff in shared
<headius[m]>
I do have a suggestion: put the stuff needed for lazy compile into something atomic
<headius[m]>
and then only remove it once the compiled IR has been installed
<enebo[m]>
The basics of the bug is loading the same IR at the same time so a synch is not horrible
subbu|away is now known as subbu
<headius[m]>
locks probably won't be a bottleneck but you could avoid it just using atomic
<enebo[m]>
we use a closure we store on IC
<headius[m]>
don't really even need to spin on the atomic operation... just make sure the transition from uncompiled to compiled is atomic and this should go away
<headius[m]>
I would guess it's clearing buf before it finishes installing the IR?
<enebo[m]>
well it is two things using a shared buf[] for reading in a tempoary string
<enebo[m]>
It is not specifically the overall file which is at issue
<enebo[m]>
it is just the assumption two threads would not try and use the same byte[] for temporary decoding (performance motivated)
<enebo[m]>
ultimately we do not want both to actually happen at the same time
<headius[m]>
ok
<headius[m]>
sure, I'm sure whatever you come up with will be lower overhead than the actual compile anyway
<enebo[m]>
but this is not about a specific IRXXX being made but just an implementaiton detail which is not usable by two threads
<headius[m]>
doing it atomically and allowing two threads to possibly compile at the same time just might avoid some bottlenecking at the lock
<enebo[m]>
If I use a synch it will be much less overall work
<headius[m]>
but I know we have state issues with concurrent changes to IR
<headius[m]>
yeah I'm sure
<enebo[m]>
since it will just realize on second thread entering it is done and return
<enebo[m]>
otherwise it will be doing a lot of extra processing to just overwrite with same data in another instance (which it does today)
<headius[m]>
sync fix would be fine with me
<headius[m]>
I don't know all the state involved so I leave that up to you
<enebo[m]>
ok
<enebo[m]>
but I am continuing with parser for now
<enebo[m]>
I think for 9.3 we try and advertise how we killed most known issues
<headius[m]>
agree
<headius[m]>
we should ask for the most broken libs and see how much we can get running
<headius[m]>
I think there's a bunch of failures in async for example that probably expose fiber bugs
<enebo[m]>
it may also be worth doing a spec/mri test pass and really just go big on cmopat
<enebo[m]>
not to drag it out but I would love to reduce some tagging
<headius[m]>
I can think of a few other libraries we should start running tests for to get super green
<headius[m]>
yeah I agree
<headius[m]>
we have not been too bad though with spec updates
<enebo[m]>
and that work generally is not massive just time consumbing to fix 1000 things
<headius[m]>
only a handful of tags every time
<enebo[m]>
yeah mostly weird corners
<headius[m]>
we're better on 2.6 support than we were on 2.5 support at release too
<headius[m]>
there's only a few things missing
<enebo[m]>
which typically affects no one since they are things like mismatched error cases
<enebo[m]>
yeah
<headius[m]>
I do want to try to land some tiered indy usage for 9.3 also, so we can stop requiring the flag to get peak perf
<headius[m]>
that has come up a lot more recently for some reason
<headius[m]>
and eregon's point still sticks in my head: people use --dev for startup anyway
<enebo[m]>
I still have to unbreak inliner
<headius[m]>
I don't think we have any known cases where indy is slower now
<enebo[m]>
yeah I don't know
<enebo[m]>
we do have reports of it being slower for warmup still
<enebo[m]>
warmup though has been the only real reason
<headius[m]>
right
<headius[m]>
I think that's what we're down to, which some tiering would help mitigate
<headius[m]>
really the only thing left to turn on is method calls
<enebo[m]>
yeah and tiering also allows inlining to fit in as well
<enebo[m]>
which I want to re-add on master
<enebo[m]>
profilingcallsite is broken but it is not even indy friendly
<headius[m]>
I can see in my head exactly how to do a "lightweight" indy bindiing to DynamicMethod.call that has a counter to switch to optimized on some heuristic
<headius[m]>
the heuristic is the icky part
<headius[m]>
same problem as JIT
<enebo[m]>
yes
<headius[m]>
hmm
<enebo[m]>
the jit timing option needs some tuning and that could help
<headius[m]>
I could also try running the method binding logic in a tight loop and reduce the overhead of optimized indy path
<headius[m]>
yeah
<enebo[m]>
if we can figure out how to determine it at runtime at least
<enebo[m]>
OMGZ this parsing error reporting is weird
<enebo[m]>
stuff like << is reported as << vs "<<" which it is in yyNames
<enebo[m]>
but if it is a single character they will report < as '<'
<headius[m]>
hah
<enebo[m]>
why? I don't get it. It is a lot of code and I am not even sure it is from MRI and not bison
<headius[m]>
I wonder what language Ruby is written in
<headius[m]>
seems like a C-ism to me
<enebo[m]>
but it must be MRI since why would bison add extra \" around strings...then remove it on a parsing error reporting
<headius[m]>
or like an old pre-1.9 ism when strings and characters weren't both strings in Ruby
<enebo[m]>
They could just not emit it in the first place as bison
<headius[m]>
yeah I don't see why it would be bison
<enebo[m]>
but then they implement their own printf logic for finding "%s" in a format string
<enebo[m]>
That is super weird...why?
<headius[m]>
haha
<enebo[m]>
I assume there is a reason to add dozens of lines vs using a standard function of C
<headius[m]>
yeah beats me... you'd expect %s would be pretty standard across all printf impls
<enebo[m]>
/* Avoid sprintf, as that infringes on the user's name space.
<enebo[m]>
Don't have undefined behavior even if the translation
<enebo[m]>
produced a string with the wrong number of "%s"s. */
<enebo[m]>
so it does look like partially bison? I have no idea
<enebo[m]>
'C' stands for 'cursed'
<enebo[m]>
It was fine for 20+ years but it is time to move on
<headius[m]>
branch is merged
<enebo[m]>
coolio
<enebo[m]>
olleolleolle: thanks for helping out
<headius[m]>
mrnoname is continuing his learning of C... we were debugging some code last night he wrote that reads in json and parse some data out with libjson
<enebo[m]>
lopex: are you still living? I have not heard you in a week
<headius[m]>
he wants to learn Rust but I think it's worth him learning C first
<enebo[m]>
well it will definitely imprint in him how much C does not do for you
<headius[m]>
heh that's for sure
<headius[m]>
he has been talking about C++ but I'll try to steer him to Rust instead
<enebo[m]>
be sure to make him change it to use libpthread
<enebo[m]>
then port the same thing to rust
<headius[m]>
hah yeah, here's your challenge: parallelize it in C
<enebo[m]>
although serde + serde_json processing will be about 4-5 lines of code in Rust for the method which does the work
<olleolleolle[m]>
<enebo[m] "olleolleolle: thanks for helping"> Ah, a pleasure. You have SUCH cadence, here! Like a bicycle team, racing forwards.
<headius[m]>
olleolleolle: if you know of other libraries with failures on JRuby let us know... a JRuby bug report is fine
<enebo[m]>
olleolleolle: that is the nicest way of ever describing how much I type in chat :)
<headius[m]>
conversely if you know important libraries that are green, we should consider adding to CI
<olleolleolle[m]>
Your CI?
<olleolleolle[m]>
Or, adding a JRuby target to theirs?
<headius[m]>
yes, I have been adding some important libraries to our CI
<headius[m]>
definitely want JRuby in their CI too but in our CI it guarantees we won't regress
<olleolleolle[m]>
Epic-level move.
<headius[m]>
like this timeout thing
<headius[m]>
I added sequel first because damned if that library doesn't hit a million edges
<enebo[m]>
for me I feel we get more value in high usage and also highly problematic ones
<headius[m]>
jeremyevans: kudos on that once again
<headius[m]>
ok well that's a bit of good news then
<headius[m]>
this only affects unreleased jruby-head
<olleolleolle[m]>
Perhaps: sinatra
<headius[m]>
sinatra's a good one
<headius[m]>
rubygems/bundler/rake would be good too but their test suites spin up hundreds of subprocesses
<headius[m]>
clearly any Rails modules that are green should go in CI too
<headius[m]>
not sure any major ones are totally green just yet though
<olleolleolle[m]>
headius: That means that Mike Perham's question "can we find a way to JRuby latest release always, without changing the CI?" led to us finding this issue, due to `jruby` -> jruby-9.3.0.0.-SNAPSHOT
<headius[m]>
olleolleolle: yup you got it
<headius[m]>
good to find this before release, would likely have broken lots of other minitest users
<headius[m]>
enebo: I will fix on master and we'll let 9.2 go on not shutting down the executor?
<olleolleolle[m]>
So, a gem: minitest with minitest/autorun ?
<headius[m]>
yeah hmm that brings up how to regression test this
<olleolleolle[m]>
(RSpec is a lot of tortured code, too.)
<enebo[m]>
ok
<headius[m]>
enebo: I think I need to introduce another list of non-Ruby shutdown hooks for this
<headius[m]>
olleolleolle: pry
ur5us has joined #jruby
<headius[m]>
that would definitely keep us honest
<headius[m]>
enebo: there are other things that could go in such a list so it's not just a one-off
<headius[m]>
enebo: other option would be registering this as a finalizer
<headius[m]>
wow I realize now we're also running finalizers in a terrible place... after ThreadService.shutdown
<olleolleolle[m]>
gem: Faraday (hell of test suite, we currently don't have the bandwidth to deal with compat with everything and at the same time untangle what's our mistakes and what may be JRuby interactions with other gems we work with)
<olleolleolle[m]>
* gem: Faraday (hell of test suite, we currently don't have the bandwidth to deal with compat with everything and at the same time untangle what's our mistakes and what may be JRuby interactions with other gems we work with; so we removed JRuby from the build matrix, I think)
<olleolleolle[m]>
* gem: Faraday (hell of a test suite, we currently don't have the bandwidth to deal with compat with everything and at the same time untangle what's our mistakes and what may be JRuby interactions with other gems we work with; so we removed JRuby from the build matrix, I think)
<headius[m]>
seems legit
<headius[m]>
if you have time to open an issue for it we can get it in the pipeline
<olleolleolle[m]>
We're using GitHub Actions. Perhaps they nowadays have something akin to allow_failures?