pawnbox has quit [Remote host closed the connection]
<GitHub116>
[jruby-openssl] kares pushed 2 new commits to master: https://git.io/vXcmt
<GitHub116>
jruby-openssl/master 7d13c15 Dmitry Ratnikov: some ~ explicit security manager checks for jruby/openssl...
<GitHub116>
jruby-openssl/master b71026d kares: [test] less warning noice with -w
<GitHub114>
[jruby] kares closed pull request #853: Have test_openssl.rb be explicit about security checks it adds (master...security-manager) https://git.io/vXcmW
<enebo>
kares_: you ever see problems with AR connection_pool tests
<enebo>
kares_: It seems there is a race present in Rails 5
<kares_>
enebo: hey! you mean from Rails test?
<enebo>
kares_: yeah
<kares_>
nope - haven't run those for very long
<enebo>
kares_: I finished base_test but then I go to wider tests I get tons of pool problems
<enebo>
kares_: and when I isolate to just those tests I get interrmittent failures.
<kares_>
I gave up trying to run AR tests basically the whole 1.3 AR-JDBC life-line
<enebo>
kares_: I will look into a bit more
<kares_>
it seemed just tremendous work
<kares_>
they usually do MR style concurrency and threads
<enebo>
kares_: I run all of test_base and only have two small encoding errors
zacts has quit [Quit: WeeChat 1.6]
<kares_>
very najs!
<enebo>
kares_: but our internal tests are really tough to maintain because AR/Arel/… keeps changing their APIs
zacts has joined #jruby
<enebo>
kares_: so I would like to be able to run at AR tests moving forward. I think if I get this pooling stuff working we will get some really interesting test feedback
<kares_>
yep I recall such annoyances - depending on test run they were effecting each other
<kares_>
but that was probably AR 3.2 or 4.0 last time I checked
<kares_>
enebo: well you'll see what's more pain in the end :)
<enebo>
kares_: but basic statements seems fine
<enebo>
kares_: yeah I guess so
<enebo>
kares_: for sqlite3 for Rails 5.1 we should onlty have about 100 lines of ruby left
<enebo>
kares_: so I am hoping we can do the same for test code maintenance
sandelius has joined #jruby
<kares_>
would be great to run AR I just fear it won't be easy to manage in the end since its 2 repos
<kares_>
enebo: sounds good but Rails commiters will need to be more careful
<kares_>
very little usually care about JRuby or know AR-JDBC :)
<kares_>
but we'll see its definitely worth a try
<enebo>
kares_: you know when I look at this code I can see PR from thedarkone and dantonio so I think some work has been put into making JRuby work with this
<enebo>
kares_: which is why I am a bit surprised to see a race
<kares_>
on the other hand Rails should be mature by know - touching adapter code is probably rare
pawnbox has joined #jruby
<kares_>
enebo: yy the pool should work better
prasunanand has joined #jruby
<kares_>
in 5.0 there was a change I recall
<kares_>
(tested with JRuby)
<kares_>
but I think they did not run the suite much
<kares_>
they did some benchmarking with external scripts
<enebo>
kares_: yeah I saw the PR which had that in it
<kares_>
so I think you're on the right track that the Rails tests need tuning
<enebo>
kares_: sounds like it should be ok but perhaps someone tweaked code later and broke it
<enebo>
the test which fails seems totally ok so it has to be the pool
<kares_>
enebo: what does it say?
<enebo>
well it either says connection is taken by another thread or that the thread has already been leased
<enebo>
so basically same thread which took connection then trying to close will not work out
tcrawley is now known as tcrawley-away
<enebo>
They seem to cache on Thread hash identity
<enebo>
so I guess I will need to print some stuff out and see what is wrong
<kares_>
oh ok - sounds like they miss some connection cleanup in Thread.new somewhere
<enebo>
hmm
<kares_>
... or the main thread
<enebo>
I wonder … do we have Java thread poooling any more?
shellac has quit [Quit: Computer has gone to sleep.]
pilhuhn_ is now known as pil-afk
djellemah_ has joined #jruby
claudiuinberlin has quit []
subbu|lunch is now known as subbu
lucasallan has quit [Read error: Connection reset by peer]
<enebo>
kares_: what is exec_query_raw for? Is it an extra JRuby feature?
shellac has joined #jruby
<headius>
kares_, enebo: I thought rails had moved to using the new "truly thread-local" thread-locals because of problems with fiber locals
<headius>
fiber locals and thread locals in the presence of fibers don't behave logically to me in MRI either
<enebo>
headius: I do not follow Rails development and only see what I see
<enebo>
headius: pool appears to use @map[Thread.current] = shit
<headius>
nirvdrum: what did you need yesterday?
<headius>
enebo: yeah that will probably have issues under a thread
<enebo>
headius: the coordination of how it stores/removes from the pool and how it stores thread on the connection itself seems to be issue
<headius>
er under a fiber
<enebo>
headius: if fiber uses a pool for sure :)
<headius>
fiber uses a thread pool but no fiber state is pooled
<enebo>
headius: there is a commen on the identity function that JRuby can monkey patch that method for fibers
shellac has quit [Quit: Computer has gone to sleep.]
<enebo>
headius: so in theory thought was put into this without addressing it
<enebo>
headius: but so far as I can see no fibers are involved in this particular failing test
<nirvdrum>
headius: Are you aware of anyone on Solaris using JRuby?
pawnbox has quit [Remote host closed the connection]
<nirvdrum>
I got access to a SPARC machine to update jnr-constants and was surprised to see a new platform generated. Apparently FFI hasn't used the name "sunos" in quite some time, but that's all jnr-constants has.
<headius>
nirvdrum: we definitely get bug reports on Solaris
<nirvdrum>
Anything related to native, do you recall?
<headius>
and I knew of someone doing packaged installs of Redmine + OpenSolaris a couple years back
<nirvdrum>
I have a sneaking suspicion they're falling over to the fake constants.
<headius>
those might have been linux-style usespace
<enebo>
nirvdrum: we got a couple in last year because of libcrypt I thought
claudiuinberlin has joined #jruby
<headius>
and still outstanding issues with flock
<headius>
because I can't figured out the blasted fcntl struct
<nirvdrum>
But the files generated in jnr-constants are only 3 years old.
<headius>
well I wouldn't be surprised if it's out of date...anything we can do to update that stuff would be good
<headius>
we used to have CI on solaris but that was years ago
<nirvdrum>
Okay. I'm mostly just confused at the moment.
<nirvdrum>
headius: It looks like you generated the sunos files. I'm assuming you ran the rake task, but not sure how you wound up with that name. It's going 3 years back, so you probably don't recall.
<nirvdrum>
I just don't want to break anything on anyone.
<headius>
yeah I don't know...just ran what was there
<headius>
it would probably get it from something like uname
<enebo>
there is more than one solaris distro now too right?
<enebo>
maybe one still uses sunos?
<nirvdrum>
enebo: I could buy that maybe IlluminOS (?) might be different. But FFI code explicitly handles anything with "sunos" or "solaris" in it and emits "solaris" as the normalized name.
<enebo>
nirvdrum: I have no idea…throwing random crap ideas and seeing if you buy it
<nirvdrum>
Heh.
<nirvdrum>
enebo: I was hoping you'd look at it and tell me I was reading the code wrong.
thedarkone2 has joined #jruby
<nirvdrum>
headius: You're holding out on me ;-) I found a comment from Sept. 19 saying you have a Solaris VM (that flock issue).
<nirvdrum>
Anyway, if you could do a sanity check for me at your convenience, that'd be helpful.
<headius>
yeah I do
<headius>
or at least I think it works
<headius>
what do you need me to run?
<headius>
just the generator?
<nirvdrum>
Yeah.
<nirvdrum>
I think the libc part is fine. It's the constants I think are out of whack.
<nirvdrum>
I'll look at setting up an OpenBSD VM to fill in those values.
<nirvdrum>
And I guess NetBSD, too?
<headius>
hah...it resumed right back to my flock exploration
<headius>
yeah so here's the thing
<nirvdrum>
I'd be surprised if there's really variation on this stuff between FreeBSD, NetBSD, and OpenBSD.
<headius>
we had someone interested in helping us set up a bunch of VMs for all these projects so we could consistently generate everything
<headius>
that never materialized but it's what we need to do
<headius>
I'm thinking vagrant might have images for almost everything
<headius>
at least x86
<nirvdrum>
Okay. That's what I was asking about in that issue.
<nirvdrum>
Aethenelle, I think?
<headius>
yeah
<nirvdrum>
He was working on some KVM-based system. He sent me a link a couple years back, but I never got around to looking at it and have since lost it.
<headius>
yeah kvm is an option but also requires control of the physical machine
<headius>
so we couldn't easily automate tasks using those images in the cloud
<nirvdrum>
Gotcha.
<headius>
for generating locally, it would probably be fine as long as you're running under xen all the time
<headius>
maybe that's what people do
<nirvdrum>
EC2 & DigitalOcean both support FreeBSD now. That could be part of an option. I think Joyent does Solaris.
<headius>
yeah joyent is the big one for sure
<nirvdrum>
But I don't have access to anything Power or ARM.
<headius>
right...there are power and arm clouds out there but it's still early days
shellac has joined #jruby
<nirvdrum>
Ostensibly your Solaris VM is x86. I was running on SPARC. I'd be curious if there at platform differences there, too.
<nirvdrum>
Constants, that is. I hope endianness is dealt with elsewhere.
<headius>
ahh yes, that's certainly possible
<headius>
I assume Oracle has sparc clouds...joyent might too
<headius>
nirvdrum: generate:platform?
<nirvdrum>
Yeah.
<headius>
uname is SunOS btw
<headius>
yeah it generates under src/main/java/jnr/constants/platform/solaris
<headius>
hmm
<nirvdrum>
I get SunOS for uname, too.
<headius>
hmm
<nirvdrum>
I can push the solaris ones easily enough. It's more of an open question as to whether anyone is using the sunos ones. I can't really see how, unless it's ancient (I haven't gone further than 'git blame' in the ffi repo).
<headius>
yeah this is a mystery
<headius>
I'm going to try generating with 1.7
<headius>
maybe we started reporting host_os differently
<nirvdrum>
Did you run the rake task with a JRuby installation?
<headius>
yes
<nirvdrum>
Okay. I've been using MRI.
<nirvdrum>
I'll stop doing that.
<headius>
shouldn't matter obviously
<headius>
that's it
<headius>
1.7 reported host as SunOS
<headius>
$ jruby-1.7.26/bin/jruby -v
<headius>
jruby 1.7.26 (1.9.3p551) 2016-08-26 69763b8 on Java HotSpot(TM) 64-Bit Server VM 1.8.0_60-b27 +jit [SunOS-amd64]
<headius>
$ jruby -v
<headius>
jruby 9.1.6.0-SNAPSHOT (2.3.1) 2016-11-03 38eaba9 Java HotSpot(TM) 64-Bit Server VM 25.60-b23 on 1.8.0_60-b27 +jit [solaris-x86_64]
<headius>
so that explains that
<headius>
not host_os though...that seems to be the same
<headius>
hmmm
<nirvdrum>
And the Rakefile uses FFI::Platform::OS
<headius>
hmm
<headius>
ah
<headius>
ok
<headius>
so we probably fixed that in jruby's FFI to normalize to solaris
<headius>
yeah org.jruby.platform.Platform.OS is "sunos" in 1.7 and "solaris" in 9k
<headius>
in any case it seems like we normalized this to solaris, which matches MRI
<headius>
so I think we should commit the solaris constants and deprecate the sunos ones
<nirvdrum>
Okay. Well, I can keep the sunos files around in case someone on 1.7 decides to update jnr-constants for whatever reason.
<headius>
yeah definitely
<headius>
we won't be modifying it to report "solaris"
<nirvdrum>
I just couldn't find a way that they were being used at all.
<headius>
nice find!
<headius>
this may explain my issues with fcntl in fact
<headius>
I never thought to check if it was actually using the right constants
<nirvdrum>
Thanks for looking into it. When I come across things like this, my first reaction is "what did I screw up?"
<headius>
yeah, I've been trying to get the jnr stuff updated and active, because it kinda bitrotted after wmeissner left
<nirvdrum>
Oh, probably. Fcntl constants are all over the place and the fake ones are just monotonically increasing.
<headius>
so this is just more cleanup
<headius>
I'll have to revisit flock then
<nirvdrum>
I still don't really know what the difference between jffi and jnr-ffi is :-/
<headius>
jffi is just the native call layer
<headius>
the binary bits we have to ship per-platform
<headius>
some of the arg wrangling lives in there too but jnr-ffi does a lot more of that
<headius>
jffi = java binding to ffi, mostly
<nirvdrum>
Okay.
<nirvdrum>
I assume there's some intrinsic value to having them split?
<headius>
I assume
<headius>
:-D
<headius>
we have rev'ed jnr-ffi many more times than jffi, so that's something
<headius>
you can't really use jffi without jnr-ffi though, so that value is dubious
<headius>
enebo and I have often mused that jnr should just be one big project
<headius>
I even started putting together a jnr-all package at one point
<nirvdrum>
That'd be interesting.
<nirvdrum>
It's a bit tedious updating jnr-constants, then jnr-posix, then jruby.
<headius>
for sure
<enebo>
“It’s a fucking nightmare!” (tm)
<headius>
at least they all are on the sonatype release train
<enebo>
hahaha
<enebo>
whoo whoo
shellac has quit [Quit: Computer has gone to sleep.]
<enebo>
I do admit sonatype does prevent a lot of problems but not this nested dep being out of sync on
<nirvdrum>
Did you guys see that thing I was working out with mkristian?
<nirvdrum>
I very much wanted to murder something.
<headius>
hmm
<nirvdrum>
Apparently Travis has a custom maven settings file that for some reason they haven't been able to update. But it overrides the snapshots repository maven looks at. And it's completely wrong.
<headius>
looks like jdantonio has some vagrant boxes under ruby-concurrency domain
<headius>
at least a solaris 11 one is there
<nirvdrum>
One of the servers in rotation is a codehaus one that hasn't been online for 18 months or so.
<nirvdrum>
Another is an apache repo that only mirrors apache projects.
<headius>
nirvdrum: ugh that's lovely
<nirvdrum>
And the net of this is builds were failing because the jnr-posix snapshots couldn't be retrieved.