<headius[m]>
kalenp: sorry nobody got back to you, I was off most of yesterday
<headius[m]>
kalenp: you don
<headius[m]>
kalenp: you don't work with Freaky do you? We have been trying to sort out a json issue for him also
<headius[m]>
boc_tothefuture: Hmm this could be due to LoadError hiding some exceptions... it wraps them and unless you can rescue it you can't figure out what the original was
<headius[m]>
this particular error looks like it can't find a file though... could that file be missing or inaccessible?
<kalenp[m]>
I don't believe I do. johnphillips31416 is my coworker who may have brought this up before.
<headius[m]>
Freaky: did you ever open an issue for this?
<headius[m]>
kalenp: last we chatted about it Freaky was working to narrow it down as it seemed intermittent
<headius[m]>
I was suspecting character encoding issues in the passed in json
<kalenp[m]>
character encoding is the best we can come up with as well
<headius[m]>
the issue seemed to be that it didn't manage to pull any tokens during parse and the error shows unexpected token with the entire json
<headius[m]>
which as far as I could tell would only be something odd about encoding
<kalenp[m]>
we see that the "unexpected token" shows the entire rest of the json when it fails. but we also see that sometimes it partially succeeds, consuming at least the first part of the json
<kalenp[m]>
* we also see that the "unexpected token" shows the entire rest of the json when it fails. but we also see that sometimes it partially succeeds, consuming at least the first part of the json
<Freaky>
headius[m]: I didn't, seemed to be an OpenJDK 15 issue
<kalenp[m]>
definitely also seeing the intermittent behavior
<Freaky>
interesting to see it on 11
<Freaky>
and permanent
<kalenp[m]>
we recently updated from 8 to 11, and that's when we started seeing our json issues
<Freaky>
I'm seeing it as a one-off early in the process, it either happens fairly quickly or not at all
<Freaky>
and on static strings, not just payloads from services
<headius[m]>
kalenp: we were down to wondering if it was some bug in JDK jit because it didn't seem to tie to anything else
<headius[m]>
either of you tried rolling back to an earlier json?
<Freaky>
I switched to jrjackson
<kalenp[m]>
I don't think we've seen it on explicitly static strings, but pretty close, json files that we package in the jar, so that still works to take the user-input factor out
<Freaky>
I threw in a tiny static JSON.parse('{ ... }') as a test and saw it trigger in that almost immediately
<Freaky>
food, bbl
<headius[m]>
kalenp: so this is an open mystery and we have no solid leads right now
<headius[m]>
if you just recently started seeing this, were there any updates that might have brought in a new json?
<kalenp[m]>
I'll double-check the timing, but I believe the json library update doesn't correlate with when we started seeing our issue
<kalenp[m]>
our best correlation is the java 8 -> 11 switch, which fits with your "jdk jit bug" theory, as much as I hate blaming lower-level systems
<kalenp[m]>
Confirmed: We updated the json gem from 2.2.0-java to 2.3.0-java back in May of last year, but we didn't start seeing these issues until this fall.
<headius[m]>
we can review commits between there perhaps
<kalenp[m]>
is there an open issue on the project github for this?
<headius[m]>
not yet
<boc_tothefuture[>
headius: It wasn't the actual error I was trying to debug, more about is there a way for me to configure so that the information (line number where its failing) shows up in the exception, like it does in STDOUT. In this system, people don't really have access to STDOUT which makes debugging tough.
<headius[m]>
kalenp: open something on jruby and provide info you have
<kalenp[m]>
will do
<headius[m]>
boc_tothefuture: ahh I think we would need to look into how the embed stuff is handling that... perhaps the original exception is not being used as the cause of the EvalFailedException?
<boc_tothefuture[>
They get the offending line from Jython with roughly the same error, but not JRuby..
<boc_tothefuture[>
And it could be in some way how I have configured the engine, but I didn't see anything I could change.
<headius[m]>
well let's see
<headius[m]>
EvalFailedException is a pretty straightforward subclass of RuntimeException and does appear to pass cause through
<headius[m]>
boc_tothefuture: I think the difference in that python example must be that jython includes the file and line number in the exception message
<headius[m]>
probably because they don't manipulate the stack trace like we do to include Ruby lines
<headius[m]>
we generally try to match the format of the error messages in CRuby, which don't include the file or line where it happened (that would be in the stack trace)
<boc_tothefuture[>
Ok. So, I guess one option would be for me to redirect STDOUT to the logger?
<boc_tothefuture[>
or I guess that would be STDERR
<boc_tothefuture[>
but I haven't checked
<headius[m]>
yeah why it is going to stdout/stderr I am not sure
<boc_tothefuture[>
Isn't that what ruby does?
<headius[m]>
well it does when you run something at a command line and it bubbles all the way out and kills the process
<headius[m]>
that may be the same logic here but we shouldn't print it to stdout if it is being rethrown for something else to handle
<boc_tothefuture[>
the stack trace for the errors show up in STDOUT (or maybe STDERR I don't actually know).
<headius[m]>
yeah I am looking to figure out how/why that happens
<headius[m]>
boc_tothefuture: I don't see anything in the 223 or embedding logic that would be printing that trace out
<headius[m]>
not yet anyway
<headius[m]>
it definitely looks like JRuby exception output though
<boc_tothefuture[>
let me go make sure I didn't do anything crazy...
<headius[m]>
nothing in embedding should be calling through there
<headius[m]>
the top level in your example error above is `openhab/dsl/group.rb:5`, perhaps whatever is calling that logs errors using stderr?
<boc_tothefuture[>
hmm.. like catching all exceptions?
<boc_tothefuture[>
let me check for any puts statements
<headius[m]>
yeah
<boc_tothefuture[>
nope, no pp or puts statements in any of my code
<headius[m]>
hmm
<headius[m]>
I am looking for anywhere else we output this format
<headius[m]>
aha I may have it
<headius[m]>
well there are other places that call this same error printing but not sure which one might be affecting your calls
<headius[m]>
what do you call on the 223 engine to get that failure above?
<headius[m]>
boc_tothefuture: yeah no leads here... the 223 engine impl does not appear to call logic that would print this exception but I am still poking around
<boc_tothefuture[>
sorry didnt' see the previous question.
<boc_tothefuture[>
hmm, what do you mean what do I call in it? what is the code or how is the engine invoked?
<headius[m]>
only went into master as part of cleaning up test output
<boc_tothefuture[>
Yeah, this is on 9.2.14.0
<headius[m]>
so getting back to your issue, I think logging a stack trace would get you line numbers, but we may want to add something to the embedding logic that includes some file+line info in the message
<headius[m]>
I don't see that we are doing anything wrong per se, we just don't include file and line information in the message because it should be in the trace
<boc_tothefuture[>
well, except I don't own the core platform.. i can submit a PR of course, but we end up in a situation where one JSR223 implementation (Jython) has the offending line and Jruby does not.. At this point they have chosen not to log the stack trace, so would need to see if they would be open to that.
<boc_tothefuture[>
yep, I get what you are saying from the matching the MRI perspective.
<boc_tothefuture[>
Wonder if in a JSR223 perspective, it does make more sense to include it.
<boc_tothefuture[>
In the mean time, is there anything I can do globally to catch the stack trace so I can log it from Ruby?
<headius[m]>
in a JSR223 perspective I could see us enhancing that message
<headius[m]>
if you need to log it for some debugging purpose there are noisy ways
<boc_tothefuture[>
what would be the noisy way?
<headius[m]>
jruby.log.backtraces=true will log every backtrace generated
<boc_tothefuture[>
And should I open a PR for enhancing JSR223 exception logging with the offending line number?
<headius[m]>
we use it to track swallowed exceptions
<headius[m]>
since they can be expensive
<boc_tothefuture[>
oh sorry.. I mean, is there a way for me inside of Ruby to hook in when the exception is happening so I can redirect to the framework log?
<headius[m]>
there's also jruby.log.exceptions=true but it is only the exception type
<headius[m]>
there is not a general purpose way to always hook exceptions getting raised
<boc_tothefuture[>
Ok, so probably the best option is for me is to open a PR.
<headius[m]>
a PR to enhance the message would be good yes
<headius[m]>
enhance the message or enable/disable logging full exceptions somewhere