<travis-ci> jruby/jruby (master:d394c27 by Charles Oliver Nutter): The build has errored. https://travis-ci.com/jruby/jruby/builds/206389553 [197 min 20 sec]
travis-ci has left #jruby [#jruby]
travis-ci has joined #jruby
sagax has joined #jruby
<byteit101[m]> I've been mucking around, and realized the jvm verifier doesn't like finally blocks around super calls. The frame will look weird, but I don't think I can avoid that
<byteit101[m]> enebo: Success on the scopes! I also added some frames that I think are good, but I don't see them in stack traces yet. the java super() is un-catchable, and for that reason all super() calls won't have a frame around them
<byteit101[m]> And running 50+ times I don't see any JIT errors, did you expect that?
<byteit101[m]> enebo: oops, when called with `.new(5, hi)`, `super` returns `[[5, "hi"]]` from interpret, whereas `super(5, "hi")` returns `[5, "hi"]`. I expect the latter, or nil to use the passed in args (actually, should I keep that, or trash it?)
Antiarc has quit [Ping timeout: 264 seconds]
Antiarc has joined #jruby
<kares[m]> there's a few open questions raised in the PR's description - those might be worth a discussion ...
den_d has quit [Excess Flood]
den_d has joined #jruby
<enebo[m]> byteit101: Hmm I may need a little more. This seems to work: https://gist.github.com/enebo/af5645fab36f65799aae153e10a4415d
<enebo[m]> I did notice if I try to 'p' the result of this I get a ClassCastException so I am unsure if something else is at play
<enebo[m]> That seems a separate issue if I switch from zsuper to super it still happens
<enebo[m]> byteit101: sorry I missed the fact that your example was provided some how :)
<enebo[m]> b
<enebo[m]> 17%v_8 = instance_super(%current_module, *(unsplat)args(0:0), %v_0 ;n:initialize, t:SU, cl:true)
<enebo[m]> byteit101: ^ So our callargs are not a simple list but something which needs to unsplat. I think I can fix this fairly quickly. ExitableInterperterContext.getArgs just needs to notice this and appropriately unplat vs just assume they are a list of individual operands
<headius[m]> enebo: ahorek pointed out that jnr-constants has not had a release in a few months and has new constants and updated constants for several platforms, so I will do one more round with that and upstream (I think only 2 or 3 jnr projects total)
<headius[m]> otherwise I guess the PR changes look ok?
<enebo[m]> headius: yeah I think so. I do not know what windows did for uid/gid on stat before but that was my only question
<headius[m]> I believe it was basically the same fallback but I broke it in jnr-posix trying to make the java logic do more
<headius[m]> when I was trying to get Apple Silicon working without FFI
<headius[m]> enebo: sonatype staging seems to have gone down in the middle of releasing
<headius[m]> if I can't get jnr-constants and upstreams released in the next hour I will just go without jnr-constants update in the PR
<headius[m]> ok that time it worked
<headius[m]> everything seems to be flowing correctly now
<headius[m]> yeah sonatype is working but mega slow today
<headius[m]> enebo: hey wiki.jruby.org does not redirect to GH wiki anymore
<enebo[m]> headius: weird. I just licked that in chat and it went there
<headius[m]> hmm
<headius[m]> yeah it did for me too
<headius[m]> but typing it didn't
<enebo[m]> byteit101: there may be other issues but this is pretty much the solution to dealing with ordinary and special arguments: https://gist.github.com/enebo/a950dccf08914ef51d85d8f085169435
<headius[m]> I got an nginx error the first time but can't repro now
<headius[m]> maybe just a glitch
<headius[m]> ok jnr-constants and friends are out, 9.2 PR is ready to push in a bit once things have some time to propagate
<enebo[m]> coolio
<headius[m]> JNR stuff is merged with the updated constants
<headius[m]> concurrent-ruby started failing but they seem to break HEAD once in a while and I don't see how it could be related to JNR
<headius[m]> it is marked as allowed failure on master but not on 9.2
<headius[m]> enebo: there is also this still out there but it is that Nix environment and may end up being purely an env fix
<headius[m]> in any case it needs to be run through a docker to show the effect and it is not a showstopper for anything at the moment
travis-ci has joined #jruby
travis-ci has left #jruby [#jruby]
<travis-ci> jruby/jruby (jruby-9.2:ccd4cb0 by Charles Oliver Nutter): The build was broken. https://travis-ci.com/jruby/jruby/builds/206683424 [174 min 29 sec]
<headius[m]> I'm not sure what to make of this failure
<headius[m]> it seems to fail consistently now but I am not sure how JNR update could be related. I will rerun a pre-PR job and see if it also fails
<headius[m]> It appears to pass... I will have to dig into this
<byteit101[m]> enebo: pasting that change in works for me. Any idea on the stack frames in caller and raise?
<headius[m]> of course it doesn't fail locally 🙄
<headius[m]> and it passed this time in CI
<headius[m]> love those concurrency tests
<travis-ci> jruby/jruby (jruby-9.2:ccd4cb0 by Charles Oliver Nutter): The build was canceled. https://travis-ci.com/jruby/jruby/builds/206683424 [173 min 47 sec]
travis-ci has left #jruby [#jruby]
travis-ci has joined #jruby
<headius[m]> ok I dunno. I get other unrelated failures locally but my env is completely different from CI and they don't appear to test on macOS
<headius[m]> and it now passed after re-running on 9.2 branch
<enebo[m]> byteit101: you mean that they do not appear within them?
<enebo[m]> the native initialize you create
<byteit101[m]> at RUBY.trytrace
<byteit101[m]> # missing: RUBY.initialize
<byteit101[m]> at RUBY.<main>
<byteit101[m]> java stacks "work" fine
<byteit101[m]> class ...; def initialize; trytrace; end; def trytrace; raise ...; end; end.new
<enebo[m]> Yep this makes sense. We actually have two mechanisms for this
<enebo[m]> Here let me find the code ... we examine our Java stacktrace looking for patterns
<byteit101[m]> headius: when you get a chance, I'd like some feedback on the issues in https://github.com/jruby/jruby/pull/6422#issuecomment-734996219 Hoping to start to wrap up this PR this weekend
<byteit101[m]> Oh, and a general question: how do I get the current, non-superclass method defined?
<headius[m]> like not using searchMethod because that will get super?
<byteit101[m]> Ok, that's the good method? I know I saw two of those with similar names and similar javadoc and wasn't sure which did what
<headius[m]> yeah well searchMethod will use internal caching and look up parents and such
<headius[m]> if you just want to see if the method is defined on the target class itself it should be in the Map returned by getMethods
<headius[m]> there's some other complexity for prepends and such but that is the class's individual method table
<byteit101[m]> retreiveMethod is the other one
<headius[m]> ah yeah
<headius[m]> which just does getMethods.get
<byteit101[m]> It has the same javadoc, and says "Cache superclass definitions in this class."
<byteit101[m]> which is why I wasn't sure
<enebo[m]> byteit101: if we call a method called INTERPRET_METHOD in Java which just calls the ExitIableInterpreterEngine it will then ask the frame setup for initialize and it should show up
<headius[m]> ok that is wrong
<headius[m]> retrieve doesn't cache anything and is just a shortcut for looking up on this class only
<enebo[m]> so your code can just call a method on ExitableInterpreterEngine which calls interpret() and I think it will do the right thing
<enebo[m]> that new method is name INTERPRET_METHOD (I just went for a run and my head is not quite working again)
<enebo[m]> The only wrinkle would be if we are not pushing a frame for initialize but I think we are
<byteit101[m]> enebo: I added ThreadContext.push/popBacktrace and context.preMethodFrameOnly/popFrame
<byteit101[m]> That was my guess last night for what to do, but it didn't work
<enebo[m]> for pure-ruby backtraces that is right I think. For random Throwable for an exception I think we did it out of the java stacktrace
<byteit101[m]> > that is right I think
<byteit101[m]> Unfortunately that's the code that I generated the failing example above 20 minutes ago with
<enebo[m]> byteit101: if you are saying a java generated error is not showing it in the backtrace that is not enough
<byteit101[m]> No, this was via `raise "test"` and `puts caller`
<enebo[m]> We may use java exceptions for both of those
<enebo[m]> You know I have not seen this past a cursory review once headius updated this to use StackWalker
<enebo[m]> but I believe not calling through an INTERPRET_METHOD is why it is not seeing those two split intialize in the stack
<enebo[m]> you can look at BacktraceData.constructBacktrace and see how we look for mangled names in JIT and a few special strings for interpreted code
<headius[m]> yo
<headius[m]> what is the issue, I have not been following along
<headius[m]> push/pop backtrace are the right way to inject a ruby frame that must also be paired with one of the known interpreter entry points
<byteit101[m]> see message from almost 30 minutes ago, :58
<enebo[m]> this generated method calls new IR interp directly but raise/caller are not showing initialize() in the backtrace
<headius[m]> the trace logic uses the presence of frames like INTERPRET_METHOD to consume a Ruby frame for the trace
<enebo[m]> I believe all he needs to do is call through a method called INTERPRET_METHOD
<headius[m]> if you do one but not the other it will be off
<headius[m]> yeah there is a list of known names it uses in the trace generator
<enebo[m]> so instead of Exitablebleh.interpret() just make a new method Exitablebleh.INTERPRET_METHOD and have that call interpret
<headius[m]> frame and class must match
<headius[m]> there are the method names but above it you can see the classes it expects
<headius[m]> when class and method name match it knows this will push/pop interpreted frame
<enebo[m]> oh phooey I forgot about INTERPRETED_CLASSES
<byteit101[m]> Hmm... tried that, no dice
<enebo[m]> INTERPRETED_CLASSES.add(DefineModuleInstr.class.getName());
<enebo[m]> you need to add a line for ExitableIntepreterEngine like that in FrameType
<byteit101[m]> is `preMethodFrameOnly` correct?
<headius[m]> depends what the method needs
<enebo[m]> byteit101: yeah and the pushDynscope
<headius[m]> frame + scope
<enebo[m]> headius: this is fine it is only startup interp
<headius[m]> then you will want to push both
<enebo[m]> yeah he already is pushing dynscope
<headius[m]> see my last link along the non-protocol path
<headius[m]> pre and post methods in InterpretedIRMethod push frame unconditionally and scope conditionally
<enebo[m]> yeah
<headius[m]> push
<byteit101[m]> How should split methods be demarcated?
<headius[m]> pushing an extra scope is not a huge issue but pushing too few obviously will break
<enebo[m]> headius: for proper split he always should
<byteit101[m]> I chose <init:2> arbitrarily
<enebo[m]> we do not eliminate scopes at startup
<enebo[m]> byteit101: but it should be initialize in finsihed PR since the use does not know what it means
<headius[m]> well I guess that is an open question... we can do whatever we want
<headius[m]> :2 looks like a line number
<headius[m]> init/2 or something might be clearer
<enebo[m]> but it is raising in def initialize() it just happens to be internally split
<headius[m]> yeah
<enebo[m]> If this feature works then it is just code in initialize which raised
<enebo[m]> but for debugging it right now I see some value :)
<headius[m]> so this is just a question about how we encode the fact that it is actually a split method
<headius[m]> sounds like you have the trace working properly now though
<enebo[m]> I do not really see a user visible use for this unless the feature is buggy
<headius[m]> it will be unlike any other Ruby trace but if you are extending a Java class you are already in strange country
<enebo[m]> I guess we could add something at the end but then we run the risk of some ruby lib out there expecting some format
<headius[m]> that is a possibility yes
<enebo[m]> My main take though is who other than us would know what it could mean. Although that might have value
<headius[m]> I don't care too much about format or whether it is there or not
<enebo[m]> I feel like in most cases it will be pretty obivous since it will include file/line
<headius[m]> if it works ideally you won't have to know that there are actually two separate halves
<enebo[m]> unless the super is on same line as pre and post calls
<byteit101[m]> A tricky bit of note: exceptions from within the super call won't have this trace on them, even if from ruby code
<byteit101[m]> But sounds good, I'll just go back to "initialize" once I'm done cleaning up
<enebo[m]> I was thinking about that as well maybe we could mangle JIT style since this is a generated method
<enebo[m]> meh ignore this for now. Let's solidify what we have
<headius[m]> byteit101: yeah because they are technically coming from Java
<headius[m]> we could add post-processing if it becomes a problem... Java calls used to post-process every exception that meant we were catching and rethrowing on every edge
<byteit101[m]> and there is no way to put an try/finally around super calls in the jvm
<enebo[m]> ultimately you would hope to see initalize() in that backtrace though but we can change how we do this
<headius[m]> ah yeah that too, so it would have to be reprocessed elsewhere during the split initialize
<enebo[m]> in fact we could just JIT mangle and not do the interp match at all?
<enebo[m]> then all of the Java and the calls to interpret would have initialize() in them
<enebo[m]> but I think for the sake of progress we should fill out and debug right now
<byteit101[m]> yes. Speaking of that, I'd love (both?) your feedback on this comment https://github.com/jruby/jruby/pull/6422#issuecomment-734996219
<headius[m]> JIT mangle would be more reliable and still be in the Java trace, yes
<headius[m]> but if it is already there in some form we already can see it
<headius[m]> JIT mangling for trace just avoids the cost of pushing a backtrace frame
<headius[m]> I admit the format I came up with is arcane and not documented though
<headius[m]> > Matching non-java_signatureed methods to java methods
<headius[m]> matching only arity would preclude metaprogramming from handling additional arities, so that is one thing
<headius[m]> I'm not entirely sure... does the legacy logic generate all arities and signatures for a given name?
<byteit101[m]> No, `Two tests show that the current behavior is a fail-through of 2 then 1:`
<byteit101[m]> If it matches arity, just that one. If it fails, then all
<enebo[m]> sorry dinner prep
<headius[m]> I don't think we need to do more than current unless you have a use case for it
<headius[m]> ahorek: it has been rolled into RG and now ships in stdlib from 2.6 on
<headius[m]> but it should be treated like a default gem and get picked up
<headius[m]> updates should get picked up I mean
<headius[m]> that may not be happening properly... should see a specification for it under lib/ruby/gems/shared/specifications/default
<headius[m]> hmm
<headius[m]> yeah I think this needs to be in default gem stuff
<headius[m]> I see it is not there now
<ahorek[m]> yeah, it should choose the installed gem if exists
<headius[m]> I am trying adding to the list of default gems in lib/pom.rb
<headius[m]> it works ok running `bundle` but the require mechanism should pick up new gem as well
<headius[m]> it is annoying that RG install also installs bundler now, I am thinking we should not version stdlib/bundler and just install the right one as default gem
<headius[m]> ahorek: open an issue for this quick and if my change works I will push a PR
<ahorek[m]> sure
<headius[m]> $ jruby -rbundler -e 'puts Bundler::VERSION'
<headius[m]> 2.1.4
<headius[m]> confirmed the stdlib one is still 1.17.3 also
<byteit101[m]> > I don't think we need to do more than current unless you have a use case for it
<byteit101[m]> No use case, I was just surprised at it
<headius[m]> it is risky to override everything too because we may pick up signatures that were not intended or that are expected to be internal
<headius[m]> assuming this is green I will merge right away
<headius[m]> good catch
<ahorek[m]> headius: thanks for a quick fix! It works exactly as expected.