<JasonRogers[m]>
headius / enebo ... I have an app that is an executable JAR created with Warbler. Running in 9.2.8.0 or 9.2.9.0 both produce a PID from `Process.pid` that is inaccurate. We use this in our logs for tracing purposes. Am I wrong in expecting that to produce the actual PID of the process?
<enebo[m]>
Jason Rogers: It does ask posix for a pid so it should be correct unless native support is not loaded. You can verify if it properly loads via -Xnative.verbose
<enebo[m]>
This works for me: jruby -e 'p Process.pid; gets'
<enebo[m]>
I do not think warbler and running within an app server should change the result but that is a difference
<JasonRogers[m]>
That works for me too. But running as an executable JAR doesn't.
<JasonRogers[m]>
I'm putting together a Gist for a min. viable example.
<enebo[m]>
Cool. Yeah if I had to guess I would say for some reason native is not loading and it is falling back to pure Java mode which could end up with "numbers"
joast has joined #jruby
nirvdrum has quit [Ping timeout: 260 seconds]
<JasonRogers[m]>
<enebo[m] "Cool. Yeah if I had to guess I "> I'm failing to produce a small example. So, it must be something that's off in our application. I've just tried my small example with 9.2.8.0, 9.2.9.0, and 9.2.10.0 and they all produce the correct PID. :/
<JasonRogers[m]>
<JasonRogers[m] "I'm failing to produce a small e"> Ah! It's the installation of Oracle's Java 8 on our RedHat 7 servers.
shellac has quit [Read error: Connection reset by peer]
<headius[m]>
Try -Xnative.verbose and see what it says
subbu|afk is now known as subbu
Antiarc has quit [Ping timeout: 255 seconds]
<JasonRogers[m]>
<headius[m] "Try -Xnative.verbose and see wha"> Sorry, was in a meeting. headius , I've set that in `JRUBY_OPTS`, but I don't see any output with respect to it. Maybe I'm not understanding how to set that flag.
<headius[m]>
Hmm that should do it but the output may be going somewhere else
<headius[m]>
There is another possibility... Have you been able to get a proper pid on this system before?
<JasonRogers[m]>
Ah!
<JasonRogers[m]>
my fault
<JasonRogers[m]>
I needed to set `-Djruby.native.verbose=true`
Antiarc has joined #jruby
<JasonRogers[m]>
success
<JasonRogers[m]>
I think I've run into this before and forgot about it.
<klobuczek[m]>
I built 9.2.9, jruby points to it, but when I run the test with it refers to gems in another rvm
<enebo[m]>
klobuczek: ah you need to 'rbm use system' or it will use whichever gems rvm happens to be pointed at
<enebo[m]>
rvm
<klobuczek[m]>
that looks better
<klobuczek[m]>
I think the performance problem is around calling java. 9.2.8.0 -> 9.2.10.0 is 400% increase
<klobuczek[m]>
I cannot do that comparison between 9.2.9.0 and 9.2.10.0 because nearly all specs fail very soon becuase of the java 11 bug in 9.2.9.0
<klobuczek[m]>
If I replace most of the java libs with corresponding C libs I can only get 150% increase from 9.2,9.0 -> 9.2.10.0
<enebo[m]>
klobuczek: so it is tough to test because of 9.2.9 java integration regression but it is possible the problem occured between .8 and .9 since you could never test .9?
<klobuczek[m]>
yes, if it is related to java 11 then this could be true
<klobuczek[m]>
we were testing 9.2.9.0 on java 8 and didn't experience any slowdown vs 9.2.8.0
<enebo[m]>
klobuczek: another question I am not sure I remember...is there something which requires Java 11 for this to happen or that your app requires to run your tests?
<enebo[m]>
klobuczek: if this can run to completion on Java 8 I somewhat expect a perf regression to still exist
<klobuczek[m]>
now we could go back to a place where it did not require java 11 and test jruby-9.2.10.0, but that's far fetched and a hassle
<enebo[m]>
perhaps you can run in 9.2.8.0 with --sample flag and with 9.2.10.0
<enebo[m]>
not sure if we will see it show up that way but it is worth a shot
<enebo[m]>
or also perhaps --profile which will give a more Rubyish view
<klobuczek[m]>
or we work with the 150%
<enebo[m]>
klobuczek: we definitely want to help you but this indicates something which is probably affecting other jruby users too
<klobuczek[m]>
how did expect to fail the wrong running tets git bisect? Witth `timeout`?
<klobuczek[m]>
* how did expect to fail the wrong running tets git bisect? With `timeout`?
<klobuczek[m]>
* how did expect to fail the wrong running tets in git bisect? With `timeout`?
<klobuczek[m]>
* how did expect to fail the long running tets in git bisect? With `timeout`?
<enebo[m]>
I expected you to time each run and if it took longer than old time then you would mark it as bad
<enebo[m]>
Which arguably would take a long time of running
<klobuczek[m]>
that would be very manual, I'll try timeout command which automatically returns non zero
<enebo[m]>
klobuczek: another quick thing you could run is 9.2.8 with -Xlog.exceptions and 9.2.10 with same option and see if we have a big change
<enebo[m]>
exceptions have explained massive perf changes in the past
<klobuczek[m]>
that's a jruby or java option?
<enebo[m]>
-X is jruby options but -Xlog.exceptions to JRuby is -Djruby.log.exceptions=true as Java option
<headius[m]>
hmmm
<headius[m]>
are you using `java_method` in your app?
<headius[m]>
that is one thing that changed and may have more overhead, due to the way it searches for the method you want
<headius[m]>
I may have assumed too much by thinking that people typically cache the result of that call
<headius[m]>
it's a linear search now where it was a hash lookup before
<headius[m]>
for a regression this large I'd expect sampling to make it pretty clear
<headius[m]>
klobuczek: another way you could sample would be with visualvm ... there's a built-in sampling profiler you can attach to a running process
<headius[m]>
and still another way would be to use Java Flight Recorder
<klobuczek[m]>
yes, we do use java_method
<klobuczek[m]>
and we are not caching it
<headius[m]>
if that's on a hot path it could definitely be a problem
<enebo[m]>
admittedly this is a fairly ugly fix but in order to properly fix this is pretty major
<headius[m]>
ugly but I suppose if it works it's fine
<headius[m]>
use a string switch
<enebo[m]>
we do not pass any additional tests with this largely because we emit more than we should overall...like we emit trace events in the tracehandler itself
<enebo[m]>
ah yeah I will change that
<enebo[m]>
I keep forgetting we are at that level
<headius[m]>
yeah I saw that
<headius[m]>
it would be "nice" to get more of the MRI set_trace_func tests passing
<headius[m]>
most seemed to fail because we emit more events than expected
<headius[m]>
klobuczek: Maybe you can try a build of JRuby with that commit reverted?
<headius[m]>
it did fix a bug but perhaps not one you were hitting
<enebo[m]>
headius: I did not add Thread method at all
<enebo[m]>
headius: so I kept excluding those
<enebo[m]>
perhaps it fixed one of those
<enebo[m]>
I need to examine if there are any actual differences in that method before I add it
<enebo[m]>
could be trivial though
<headius[m]>
it might be possible to do a quick fix on the thread version by just adding a thread field to the event hook
<headius[m]>
only dispatch if context.getThread == hook.getThread
<headius[m]>
I don't quite understand why there's a Thread#add_trace_func but no global version
<headius[m]>
klobuczek: in any case I will work up a fix for that perf regression
<enebo[m]>
so one difference is thread is passed?
<enebo[m]>
HAHAHA lol
<enebo[m]>
MRI 2.6 event name is the same string and is not frozen
ur5us has joined #jruby
<headius[m]>
what do you mean?
<headius[m]>
they changed it in 2.6?
<enebo[m]>
ignore me...it is different
<headius[m]>
can do
<enebo[m]>
So they, like us, create a new mutable string for every event
<enebo[m]>
I guess this is set_trace_func I wonder if Tracepoint at least switched to a symbol
<enebo[m]>
I would argue this should be a frozen string and then can be the same instance
<headius[m]>
Tracepoint is symbols I'm pretty sure, with the fixed name
<headius[m]>
which probably was fixed because you can't do symbols with a dash in them without quotes
<headius[m]>
```:'c-call'```
<headius[m]>
well, and also the inconsistency
<headius[m]>
looking into java_method perf fix now
<headius[m]>
klobuczek: FWIW if you can cache you'll save some overhead in any case, even with the optimization
<headius[m]>
java_send will also go down this new path but clearly java_send is the slowest way
<headius[m]>
hmm
<headius[m]>
perhaps java_send and java_method should have another level of caching
<headius[m]>
actually cache the Method object too
drbobbeaty has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
nirvdrum has quit [Ping timeout: 265 seconds]
jeremyevans has quit [Ping timeout: 265 seconds]
<klobuczek[m]>
how about regular java non overloaded calls by name?
<klobuczek[m]>
* how about regular non overloaded java method calls by name?
lucasb has quit [Quit: Connection closed for inactivity]
<headius[m]>
Regular calls should be fine because they cache the selected method. Non-overloaded methods will be direct
<headius[m]>
The change to java_method means it searches all methods
<klobuczek[m]>
and if let jruby select the overloaded method at runtime by actual parameter types? Should this be avoided?