sgeorge has quit [Remote host closed the connection]
sgeorge has joined #jruby
<lopex>
headius: fixed another issue, and rereleased joni
_whitelogger has joined #jruby
sgeorge has quit [Remote host closed the connection]
jmalves has joined #jruby
jmalves has quit [Ping timeout: 252 seconds]
sgeorge has joined #jruby
sgeorge has quit [Ping timeout: 240 seconds]
sgeorge has joined #jruby
sgeorge has quit [Ping timeout: 252 seconds]
shellac has joined #jruby
drbobbeaty has joined #jruby
drbobbeaty has quit [Ping timeout: 272 seconds]
blallo has joined #jruby
<blallo>
Hi. I was trying to compile puppetserver from source on debian buster. I have some JRuby error related to something I don't understand: dpaste.com/28H0DGC
sgeorge has joined #jruby
jmalves has joined #jruby
sgeorge has quit [Ping timeout: 260 seconds]
drbobbeaty has joined #jruby
shellac has quit [Quit: Computer has gone to sleep.]
<kares>
blallo: this is RubyGems installing a required gem
<kares>
which cointains a weirdly named file: test_utils/resources/ᚠᛇᚻ.conf
<kares>
its the hocon version 1.2.5 maybe they have an update on that - seems like test code
<kares>
your FS might have issues with such names
<kares>
or Java/JRuby reading them if you seen and ls the file just fine
<blallo>
kares: thanks, I solved
<blallo>
It was a locale issue
<blallo>
LANG=en_US.UTF-8
<blallo>
but LC_ALL=C
<blallo>
solved with a `export LC_ALL=$LANG` :)
shellac has joined #jruby
jmalves_ has joined #jruby
jmalves has quit [Ping timeout: 245 seconds]
sgeorge has joined #jruby
jmalves_ has quit [Remote host closed the connection]
sgeorge has quit [Remote host closed the connection]
sgeorge has joined #jruby
<kares>
blallo: najs!
<headius>
lopex: ok
<headius>
blallo: nice find
<lopex>
headius: no idea what's with that spec:regression
<lopex>
F
KeyJoo has joined #jruby
<lopex>
headius: oh, and should jdk8 be able to release still ?
<headius>
yeah should be fine
<headius>
oh wait
<headius>
joni?
<headius>
you should release with 9 since it has been modularized
<headius>
if not it will exclude the module-info
<headius>
I have not found a way to prevent releasing on 8
<lopex>
headius: something's wrong wtih activations, on jdk8 in complains about module info
<lopex>
*it
<lopex>
headius: so exclusion on activation might work ?
<eregon>
headius: Hello, so joni/jcodings broke on master?
sgeorge has quit [Ping timeout: 246 seconds]
<lopex>
yep
<lopex>
eregon: now only spec:regression didnt finish
<eregon>
headius: at some point we were running on TravisCI with pre-gate checks and I think we didn't too many flakes. Most of them are caused by bad tests or actual bugs, rarely the infra
<eregon>
(for TruffleRuby)
<eregon>
we didn't get*
<eregon>
it's a bit hard to start, but once it's enforced it can't be broken so easily and there are less and less flakey tests
<lopex>
eregon: oh, I must have looked at mri only then
<headius>
I've seen hangs in travis, maven failures that make no sense, and sometimes a test with all good output that travis reports as failed
<headius>
I have been quarantining unreliable tests
<eregon>
I think Maven unreliability exists outside TravisCI, I really hate Maven TBH
<enebo>
I have only had maven issues outside travis when network is flaky but it is very rare locally for me anyways
<eregon>
hangs in Travis are typically real hangs, just more likely to reproduce on a less powerful system with less cores
<enebo>
and I am not a person who like defending Maven :P
<enebo>
eregon: you guys run MRI tests yet?
<eregon>
Yeah, we do, even in parallel with -j4 recently :)
<enebo>
eregon: cool. there is a lot of gold in there
<eregon>
Might be worth trying to run them in parallel for JRuby too, might speed TravisCI a bit, I could almost get a 4x faster with -j4
<eregon>
(in our infra, not TravisCI)
<enebo>
eregon: you know I don't remember ever trying but if I did it would have been over a decade ago
<eregon>
enebo: much less descriptive and hard to follow tests, but definitely covers a lot of edge cases too and regression tests
<enebo>
eregon: but that is a good suggestion since I spend some amount of time waiting on test:mri
<enebo>
eregon: yeah it is almost all edge cases
<eregon>
basically no gentle test to start implementing sth, but many edge cases, useful once there is almost complete impl of that functionality
<eregon>
ruby/spec can also run in parallel, but usually the absolute gain is lower as they run faster than MRI tests
<eregon>
Still quite nice to run all specs with MRI in 10-20 seconds :) (JRuby/TruffleRuby take a lot longer with startup though)
subbu is now known as subbu|afk
<headius>
eregon: where are your travis runs these days?
<eregon>
TruffleRuby uses GraalVM's internal CI nowadays, no longer TravisCI (it's much faster)
<enebo>
eregon: how do the non-Oracle contributors see?
<eregon>
we tell them what fails, or fix it ourselves
<enebo>
eregon: ok
<eregon>
the Graal repo also has a TravisCI as a smoke test so it's faster for contributors to see problems
<eregon>
so it is possible to have TravisCI always green for a Ruby impl, but it requires to run the gate before merging, and so no direct commits to master :)
<enebo>
sounds fun
xardion has quit [Remote host closed the connection]
xardion has joined #jruby
sgeorge has quit [Ping timeout: 244 seconds]
<eregon>
it's much more relaxed when merging instead of pushing directly to master as one can be pretty sure nothing broke :)
shellac has joined #jruby
<enebo>
well it is certainly nice to have confidence but I have seen one too many projects be way too dogmatic about this and force CI runs for any commit whether it changes code or not
<enebo>
tbh if CI took a minute it would be fine but in a long CI run it sucks to stack up
<eregon>
yeah for doc changes, you could just force merging without waiting for CI if you want, but we just ran CI, took around 30-40min I think and that was good enough
<eregon>
question of habit mostly I think
<enebo>
eregon: oh yeah ability to override is the important bit
<enebo>
eregon: I know projects which cannot merge without being green and passing rubocop and 100% mandatory reviews and coverage must be same level of higher
<enebo>
on paper is sounds nice
<eregon>
there is a setting on github to make the green gate mandatory, but still let project admins override or not
<enebo>
eregon: yeah which is important exception
<eregon>
I'd encourage anyone to try for large projects, I find it much nicer in practice even though a bit weird at first.
<eregon>
Probably something that needs to be tried to realize what it brings, hard to describe in words
<eregon>
unrelated but fun: JRuby allows to serialize Mutex it seems :p
<eregon>
[exec] Marshal.dump raises a TypeError if dumping a Mutex instance FAILED
<eregon>
[exec] /home/travis/build/jruby/jruby/spec/ruby/core/marshal/dump_spec.rb:571:in `block in <main>'
<eregon>
[exec] but no exception was raised ("\x04\bo:\x12Thread::Mutex\x00" was returned)
<eregon>
[exec] Expected TypeError
<enebo>
eregon: I have done it in the past
sgeorge has joined #jruby
<eregon>
enebo: what did you think of it?
<enebo>
eregon: It was fine so long as the time to finish was ok and it is also a factor of how many people are stacking up
<enebo>
eregon: to me the dogmatism of forcing it on a short run is not that important to correct though either. It is fine though. It is the practical limits of time where it sucks.
<enebo>
eregon: then having override definitely helps but then it is not known to be green
<enebo>
eregon: I think on the other end of the spectrum if it is always red it is easy for it to get away from you
<eregon>
We don't force rebasing each PR against the latest master before merging in TruffleRuby currently, so passing CI once per PR is all that's needed
<eregon>
And with that and 5 people working on it it's not been a bottleneck so far
<enebo>
eregon: that especially helps with PRs
<eregon>
So I'd guess it would be fine for JRuby, given there aren't dozens of PRs per day
<headius>
I have no problem turning on the gate once we are reliably green
subbu|afk is now known as subbu
<headius>
I've been trying to fix that in between keeping issues under control and trying to do perf work...spread a little thin unfortunately
<enebo>
I somewhat do but knowing we can exempt a build makes it more appealing
<eregon>
and once pre-merge gate is enabled, fixing flakey tests has an obvious priority as it blocks everyone
<enebo>
eregon: we will take your suggestions under advisement. Thanks for raising them.
<headius>
eregon: is there a reason you're pushing for this?
<enebo>
lunch
<eregon>
headius: I'd like to know the CI is green and ensure specs are always passing on JRuby, rather than having to check previous builds.
<eregon>
It would also mean nobody would regress on specs or tests without knowing about it.
<eregon>
And from experience with pre-merge CI, I find it a huge change in how the project is more reliable, and how much nicer it is to contribute to it (if it doesn't build for me, it's an actual bug, not just the latest master commit which broke).
<eregon>
MRI also has a flakey CI, and they suffer from it, having to revert random commits because they might have cause a CI failure.
<eregon>
enebo, headius: thanks for considering it
shellac has quit [Quit: Computer has gone to sleep.]
sgeorge has quit [Remote host closed the connection]
sgeorge has joined #jruby
shellac has joined #jruby
<lopex>
eregon: but their flakiness is somewhat different than ours ?
<lopex>
is it the machine load and thread interoperability timings ?
<headius>
I have suspected load issues for some of ours but I have no way to prove it
<lopex>
headius: my ssd sometimes freezes for a couple of seconds, and then I see cache flush timeouts in dmesg
<headius>
nice
<lopex>
headius: so maybe similar things happens on cpu cache levels lol ?
<lopex>
under load
<headius>
there's a number of things that could cause it in a heavily virtualized env
<lopex>
so in a gist this pre-gate does what ?
sgeorge has quit [Remote host closed the connection]
sgeorge has joined #jruby
<eregon>
lopex: avoids breaking master and making sure all specs/tests pass before they get merged in master
<lopex>
aah
sgeorge has quit [Ping timeout: 250 seconds]
<lopex>
eregon: btw repetition_tags.txt from TR should work as well
shellac has quit [Quit: Computer has gone to sleep.]
shellac has joined #jruby
sgeorge has joined #jruby
subbu is now known as subbu|lunch
<lopex>
oh it doesnt
<lopex>
but why
sgeorge has quit [Remote host closed the connection]
sgeorge has joined #jruby
sgeorge has quit [Ping timeout: 252 seconds]
shellac has quit [Quit: Computer has gone to sleep.]
<lopex>
ok, we removed too many warnings I guess
shellac has joined #jruby
sgeorge has joined #jruby
subbu|lunch is now known as subbu
sgeorge has quit [Read error: Connection reset by peer]
sgeorge_ has joined #jruby
shellac has quit [Quit: Computer has gone to sleep.]
<nirvdrum>
headius: IIRC, the lazy rubygems works by replacing require and intercepting LoadErrors. But if there's a legitimate LoadError, it just keeps trying to load.
<nirvdrum>
A bug to be fixed, for sure. But if that is the case, we're failing to load something else.
<eregon>
nirvdrum: that's not true, it, on purpose, does not handle LoadError while loading the full rubygems, for simplicity
<eregon>
otherwise we need to track recursion in require and that's not fun
<nirvdrum>
eregon: Sure. I raised this as a potential issue when you added it. That spot can get into an infinite loop.
<eregon>
it cannot, since rubygems redefine require
drbobbeaty has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<nirvdrum>
Unless that fails...
<eregon>
I guess you're saying the same thing as me, just a different way
<nirvdrum>
I think it's probably the right trade-off. But because it can get into this infinite loop, it's not immediately clear what the issue is.
<nirvdrum>
Anyway, it's worth looking into the issue headius noted.
<eregon>
I mean that stack overflow can only happen while loading full rubygems itself, so it's fairly contained code
<nirvdrum>
Maybe my analysis is wrong.
<nirvdrum>
Right.
<eregon>
the issue above is most likely that rubygems/defaults/operating_system and rubygems/defaults/ruby need special treatment, because those do fail as LoadError during loading RubyGems, even on MRI
shellac has quit [Quit: Computer has gone to sleep.]
<headius>
eregon: GC-sensitive test
<headius>
might also be there are enough live threads from other specs to tip that test over the edge sometimes, in which case a higher threshold would be warranted
<headius>
it's only supposed to test that the majority of enumerators are getting cleaned up
<headius>
I'll make similar tweaks to the fiber test...that one has additional GC calls and adjustment of the thread pool size
shellac has joined #jruby
<headius>
enebo: we could have this deploy jruby-dist and then use that for running the remaining tests
<headius>
dist doesn't include all the tests though
shellac has quit [Quit: Computer has gone to sleep.]
<headius>
hmm bit of a potential race though...and it would need logic to know how to fetch correct snap
<headius>
docker image might be better
<headius>
enebo: still seems to be deploying on every build, I'll try the stages
<lopex>
headius: ifonly this would have more images http://plash.io/
<lopex>
lol there's even travis example for it
<headius>
lopex: it probably wouldn't be hard to build and push a snapshot docker image somewhere, and then use that for the test runs, but it seems heavy-handed
<lopex>
headius: btw, is there a limit of brancher per projects for travis