Sigyn has quit [Quit: People always have such a hard time believing that robots could do bad things.]
Sigyn has joined #jruby
<headius>
aargh, why are properties still not sorted
_whitelogger has joined #jruby
<headius>
oh right we need a polyglot release
jmalves has joined #jruby
jmalves has quit [Read error: Connection reset by peer]
jmalves_ has joined #jruby
jmalves_ has quit [Ping timeout: 260 seconds]
<lopex>
always something
jmalves has joined #jruby
<headius>
enebo: so about this --module-path change
xardion has quit [Ping timeout: 244 seconds]
<headius>
I'm still feeling icky about disabling all verification
<enebo>
yeah I am not sure about how evil it is or isn't at this point
<headius>
I have an additional change locally that will use the appcds archive if present
<enebo>
it is really too bad we cannot continue to selectively disable it
<headius>
even with verification on that's now the fastest startup
<headius>
but it's not something we can ship easily
<enebo>
yeah and it is version fragile
<headius>
I have asked a few times about portability and that's a big open question right now
<headius>
apparently for Java 12 they want to start shipping JDK with a pre-build CDS archive, so there's some work happening
<enebo>
I somewhat assumed it was a bit mmap so I figured it would break fairly often
<enebo>
ah well that is nice if true
<headius>
problem I have at the moment with these bash script changes is that if we move JRuby out of bootclasspath and don't disable verification, it's considerably slower than current Java 8 startup
<headius>
I'm just wondering which problem is the bigger problem
<enebo>
since you have that check can't we maintain old launch options for 8 and otherwise use this changed logic for 9+
<enebo>
-d check
<headius>
well considerably slower = like 0.2s
<headius>
0.2-0.4 maybe
<headius>
my patch does maintain old logic for 8
<headius>
i.e. bootclasspath
<enebo>
wait so you mean without appcds 9+ we are slower with new logic but it removes the warnings
<headius>
there's the three configs... 8, 11 with appcds, 11 without appcds
<enebo>
at this point I am more worried about why this even works to remove warnings although the warnings themselves is a huge thorn
<headius>
so 11 without appcds is what people would get out of the box until generating jruby.jsa
<enebo>
It is icky no matter how it is diced though
<headius>
well my current theory is that whatever logic they have for doing those warnings will always fire for anonymous modules
<enebo>
Warmup time is extremely important but those warnings will just piss everyone off
<headius>
or rather it always fires for things from old classpath
<enebo>
the devil we know is startup
<headius>
new module path observes the silencing flags
<headius>
yeah
<headius>
I am also reluctant to do this but jruby bash script could warn when starting on 11 without jruby.jsa
<enebo>
and as you somewhat are pointing out if we can make a non-fragile appcds setup we will get that slowdown back plus a bunch
<headius>
we have options here to get people much better startup but I don't know the right way to present it to them :-\
<enebo>
so we should consider these are separate issues
<headius>
ok sure
<enebo>
1. warnings vs slowdown
<enebo>
2. appcds
<headius>
well maybe three issues... warnings, slowdown or not versus verification, and appcds
<headius>
warning-wise...
<enebo>
I think the warnings will massively turn off users so getting rid of them may be worth the temporary solutions until 2 is solved problem
<headius>
we need to start moving stuff into module-path either way
<headius>
so I think doing that is a no-brainer
<headius>
I don't know that this gets rid of all warnings but I have not seen any while running jruby this way
<headius>
since all our java code is in jruby.jar and now part of module path
<enebo>
verification is only an issue in the sense we could decide to stop but today 8- we can verify external loaded libraries
<headius>
right, and more of a concern for me is that turning it off globally also turns it off for bytecode we and other libs generate
<enebo>
for 9+ I guess it is all or nothing proposition so safe bet would be to always enable
<headius>
which is the vector that scares me a bit
<enebo>
I am not so sure verification is important but people on the internet tell me it is scary
<headius>
yeah
<enebo>
and I can see it could be used as an attack vector but I wonder how that looks in reality
<headius>
right, I have not found any info on that
<enebo>
I am not worried about consumers grabbing artifacts out of the mavosphere getting hacked
<headius>
at this point I feel like for 9.2.1's Java 11 support we should not turn off verification
<headius>
there's only going to be a tiny fraction of users on 11 for this round anyway
<headius>
it was just released like two weeks ago
<enebo>
That is certainly possible but it seems so weird to me. I wonder if that has ever happened in the history of maven repos
<headius>
there are some people on 9 or 10 but still tiny
<enebo>
yeah I am totally unconcerned about this today vs a year from now
<enebo>
being a little slower on startup for 11 but working is pretty reasonable place to be
<headius>
if I push my changes in, the script will work like this on 9+
<headius>
* -classpath and -Xbootclasspath become --module-path
<enebo>
but do people use 9 or 10 much?
<enebo>
I am sure someone does
<enebo>
but I feel like most people were waiting it out for 11
<headius>
* if lib/jruby.jsa AppCDS archive is detected, it adds Java flags to use it
<enebo>
I wish we could posthook or add something when it does not properly load but that is maybe not even possible
<headius>
yeah I don't know how to detect that without a separate run
<enebo>
My biggest fear with appcds over time is no one ever loads and it just makes our startup even worse
<headius>
verify-appcds might be a good comman to add
<headius>
right
<headius>
I've asked @redestad from Oracle to clarify what causes the appcds archive to be rejected
<enebo>
I am thinking if some shutdown hook to jruby runtime if we could know then we could just set some file on shutdown to regenerate or something
<headius>
he's part of the group that has been working on startup improvements in 10, 11, 12
<headius>
yeah I was thinking about auto-generating possibilities too
<enebo>
failing but running and perhaps warning or even regenerating
<headius>
I don't know if there's a way even at runtime to know if it loaded appcds
<enebo>
yeah implicit background generation would be best but who knows if that is realistic
<headius>
there's nothing in standard JDK classes obviously
<enebo>
I don't like to complain (well I do actually) but new features of systems should always be fucking reflective
<headius>
agree
<enebo>
I do get it too. This is probably not something they think about from how we use Java
<enebo>
like you are running a prod server and you generate this once but it is a highly controlled JVM env
<enebo>
That seems to be the golden path but with that said as written someone will grab a newer JVM and that old JSA will just mess with their startup for 2 years before someone realizes it is not helping them
<enebo>
anyways I am distracting now
<enebo>
for 11 today I say we should enable verification and use module path
<enebo>
for soon hopefully we can figure out a "robust" appcds component
<enebo>
but as you said the number of 11 users is not that high yet so working trumps a small slow down in startup
<enebo>
and we can get rid of annoying warnings
<enebo>
for 11 I mean 9+ but both 9 and 10 are dead to me mostly already
<enebo>
This new fast train JVM schedule will be kind of tough to deal with
<enebo>
Best case it creates a culture where Java people upgrade more often but I have my doubts
<headius>
yeah screw 9 and 10
<headius>
I'm not going to focus on the intermediate non-LTS versions in any major way, other than perhaps to leverage new features internally
<enebo>
yeah I still wish stuff like var was a compiler plugin so we could use it without having to move everything forward
<enebo>
"use it" == older javac/jvm
jmalves_ has joined #jruby
jmalves has quit [Ping timeout: 276 seconds]
jmalves_ has quit [Ping timeout: 244 seconds]
<headius>
heh so about those warnings
<headius>
this does appear to silence ours...but there's warnings from jopenssl.jar still
<headius>
I think the way to fix that is to also use module APIs to load exts from jars
<headius>
rather than just classloader APIs
<headius>
org.jruby.ext.openssl.SecurityHelper (file:/Users/headius/projects/jruby/lib/ruby/stdlib/jopenssl.jar) to constructor java.security.cert.CertificateFactory
<headius>
I'm not sure why jossl isn't using the SecurityHelper based in JRuby though
<headius>
that would eliminate this
<headius>
kares: ^^
<headius>
enebo: I've updated my branch with the bash script changes and pushed an updated jruby-startup gem...can you try it out?
<headius>
headius/module_path_launcher is the branch
<headius>
we could also preinstall jruby-startup gem
Eiam has joined #jruby
<headius>
ok
<headius>
so those changes are merged
<headius>
they do seem to silence warnings from classes in jruby.jar but classes in extensions still warn, like openssl.jar
<headius>
kares: if we can get a jossl release that uses JRuby's SecurityHelper I think that warning would go away too
jmalves has joined #jruby
jmalves has quit [Ping timeout: 246 seconds]
<kares>
headius: hey! what?
<headius>
hi there!
<kares>
you do not have the same SecurityHelper :)
<headius>
so this does something more?
<kares>
lot of reflection around javax.crypto stuff
<headius>
ahh
<headius>
ok, so until we tweak JRuby to load exts via module subystem I think this warning will stand
<headius>
I'm not sure what that will look like even
<kares>
can't we add opens?
<kares>
oh we can not since still anonymous, right?
<headius>
in this case we'd have to add opens to all unnamed modules
<headius>
since the jossl jar does not get a proper module
<kares>
we could put it in a module ... if that makes sense
<kares>
I mean the ext parts
<kares>
its a jar in the end
<headius>
yeah
<headius>
we load it via URLClassLoader so I'm not sure how that fits into modules
<headius>
I think we'd have to use module APIs to load a module jar
<kares>
aaah, right and how's your module-path thing going?
<headius>
I merged the changes to bash script to master
<headius>
on 9+ now it will use module-path and not bootclasspath
<kares>
so no warnings?
<headius>
this gets rid of the warnings but since JRuby is now being verified it will be slower unless you do appcds stuff
<kares>
for jruby core
<headius>
yeah seems so
<headius>
I have not seen any from jruby core
<kares>
ah than jossl will get all the warnings :)
<headius>
most libs won't have any warnings either...jossl just digs deep
<headius>
not sure the best way to deal with jossl then
<kares>
yeah
<kares>
me neither
<headius>
I guess I need to try loading it as a module
<kares>
but I'll keep that in mind
<headius>
I have no idea how we're supposed to add new jars to system under modules
<headius>
might be via module layers or something...I just started reading about those today
<headius>
it's going to be a long tail getting this stuff working