<JulesIvanicGitte>
which means that first, there’re 2 users (req/s) during 2 minutes
<headius[m]>
got it, that should get it pretty warm
<JulesIvanicGitte>
yes I think so :)
<headius[m]>
So you logged all those singletons and they're all coming from that same place in activesupport?
<JulesIvanicGitte>
the logged singleton I showed you are not part of the benchs
<JulesIvanicGitte>
it’s just the kubernetes health check
<headius[m]>
oh ok
<JulesIvanicGitte>
it’s the simplest case I found
<JulesIvanicGitte>
maybe there’s also a lot of singleton classes during our benchs but it’s harder to see
<headius[m]>
So then CPU usage...is the JRuby server really using up a core (at least)?
<JulesIvanicGitte>
I suspected that this behavior in our health check endpoint could be something interesting
<headius[m]>
Just trying to discover if JRuby is perhaps blocking on something, rather than being a CPU bottleneck
<headius[m]>
it could be
<headius[m]>
I'm feeling around in the dark here unfortunately
<kares[m]>
sounds like a concurrent bench but its really very little info with screenshots ...
<kares[m]>
than there's 5 during 4 mins
<headius[m]>
yeah so the first round of things I'd do personally would be to see if CPU is being used like I expect (if it's partly idle or not 100% it could be blocking) and look at GC logs to see if there's some issue with memory use
<headius[m]>
JRuby ought to be using many cores if this is really hitting server hard with more than 1 concurrent user
<kares[m]>
yeah it will pbly need a deeper look, maybe even some isolated profiling
<kares[m]>
5 users -> AR conn pool default is 5 so that should be still okay, unless there's other threads using up connections besides request threads
<headius[m]>
Jules Ivanic (Gitter): Basically we need to understand if this is saturating CPU or overloading GC before we can really dig deeper, otherwise there's lots of possible issues
<headius[m]>
ugh I have not fixed that failure yet have I
<JulesIvanicGitte>
my current Puma is configured to boot 128 thread
<JulesIvanicGitte>
> Basically we need to understand if this is saturating CPU or overloading GC before we can really dig deeper, otherwise there's lots of possible issues
<JulesIvanicGitte>
I’m running some benchs to show you the CPU usage
<JulesIvanicGitte>
it’s not 100% full
<headius[m]>
ok
<headius[m]>
well that's a sign
<kares[m]>
that config is pretty much confusing -> its mostly due forking servers, you can just set size: xx in database.yml
<headius[m]>
kares: just to point out, this is rails 4.2.11.1 in case you didn't see that trace above
<kares[m]>
that should work like a charm :)
<kares[m]>
altough AR-JDBC might be slightly slower than C AR
<JulesIvanicGitte>
AR-JDBC 1.3.25
<headius[m]>
Ok looking for your CPU results
<headius[m]>
I think a run with some GC logging would be helpful next
<headius[m]>
if CPU is not getting saturated and you expect it should, GC could be a culprit...something creating too many objects, heap too small, etc
<headius[m]>
you're not going to get much gain out of having eight times as many threads as cores
<JulesIvanicGitte>
agree
<headius[m]>
I don't think it should hurt anything but it makes this harder to investigate
<JulesIvanicGitte>
I can put `nbCores * 2`
<JulesIvanicGitte>
if you prefer
<JulesIvanicGitte>
or anything else
<headius[m]>
how about we focus on one thread, one concurrent user, and see what it looks like?
<headius[m]>
Make sure we can get good throughput with one thread first
<JulesIvanicGitte>
good idea
<headius[m]>
which will also make profiling a heck of a lot simpler
<headius[m]>
we'll figure this out...just need to rule things out for a bit
drbobbeaty has joined #jruby
<headius[m]>
concurrently I'm trying to fix this puma keepalive thing, but I don't think it's related
<JulesIvanicGitte>
> but I don't think it's related
<JulesIvanicGitte>
I don’t think so too because keepalive is deactivated between our Nginx and Puma
<JulesIvanicGitte>
I have a meeting. I come back in 1 hour :)
<headius[m]>
hmm
<headius[m]>
I'm not going anywhere
<headius[m]>
or rather I'm going somewhere very quickly but I'll just be working
<JulesIvanicGitte>
😄
<JulesIvanicGitte>
you have wifi in the plane ?
<headius[m]>
indeed
shellac has quit [Quit: Computer has gone to sleep.]
shellac has joined #jruby
umair has quit [Ping timeout: 256 seconds]
lucasb has joined #jruby
<JulesIvanicGitte>
Soo
<JulesIvanicGitte>
sorry to come back that late
<JulesIvanicGitte>
My company decided that the mesured performances were good enough to test a put in production of the JRuby version of our app
<headius[m]>
Well, I'm glad we're not completely out of the game yet 😀
<JulesIvanicGitte>
The decision also comes from the fact that I’m quitting this company at the end of the month and I’m perceived here as the “JRuby expert” because I’m working on this migration since 6 months
<JulesIvanicGitte>
They prefer to put in prod while I’m still here :)
<JulesIvanicGitte>
so, let’s put that in prod ! :)
<headius[m]>
Hopefully we can get perf up where it should be before you leave!
<JulesIvanicGitte>
it would be wonderful :)
<JulesIvanicGitte>
that let us 2 days this week and 1.5 days next week :)
<rdubya[m]>
weird, quoting doesn't attribute who wrote it in the first place... enebo posted it yesterday
<headius[m]>
rdubya: hi there
<headius[m]>
I am of two minds, which probably isn't helpful
<headius[m]>
depending on a well-maintained library that we don't have to hack or tweak is obviously far less work for us
<headius[m]>
And if this is interesting enough to ship support for MSSQL in the first place, that gem probably can maintain a critical mass of contributors to stay maintained
<headius[m]>
The other half of me says we've relied on external gems many times and then the author goes away and we have to adopt it anyway
<headius[m]>
so that's the balancing act
<enebo[m]>
I think tis is why #1 has the most value to me. We will have a clean line of separation but still have flexibility to change it if need be
<headius[m]>
I am still catching up a bit here, but this other PR duplicates some code from the mssql gem?
<headius[m]>
if that's the case then it's a simpler decision...who do we trust to have the cycles to maintain that code more
<rdubya[m]>
yeah, it looks like a bunch of it is basically copied over from the sqlserver-adapter codebase
<enebo[m]>
My main stance now is getting a clean line between our arjdbc code and this adapter code and how easy we can make tracking changes to that sqlserver-adapter project
<enebo[m]>
At this point forking it with minimal patch set gives us that and the freedom to change stuff without needing to wait on someone to accept a PR immediately or reject it outright
<enebo[m]>
but for this to work we need to still try and get as much back upstream as possible
<headius[m]>
what does the gem use for the actual DB connection?
<headius[m]>
I actually don't know what the gem is
<enebo[m]>
I am sure rdubya knows more but he told me that they have trans dep to native c ext for actual connection
<headius[m]>
I think there's some synchronicity here that could answer all our questions
<headius[m]>
seems to me like ar-sqlserver should add jdbc with our help
<rdubya[m]>
yeah, we went back and forth in that pr and the maintainers basically said that they didn't want to deal with jruby unless it was minimal work
<rdubya[m]>
i posted in their gitter channel on monday and haven't heard back though
<headius[m]>
enebo: so is this what you mean by figuring out where that line is?
<headius[m]>
so we'd help them isolate the native tds bit behind some interface and we'd maintain the jdbc version of that interface
<headius[m]>
they wouldn't have to maintain anything unless that interface needed to change
<enebo[m]>
well no I just meant the line is really clear as to what to look at when something changes in mysqlservr-adapter and our codebase VERSUS tracking 2k of partially copy/pasted code from that project
<enebo[m]>
headius: that is strat #2 them releasing is #3
<rdubya[m]>
they have to release a java version of the gem which it didn't sound like they were particularly interested in
<headius[m]>
"It's based on a private sqljdbc4 wrapper gem"
<headius[m]>
what does that mean
<rdubya[m]>
the person that created that pr had their own gem pulling in the jdbc library
<rdubya[m]>
apparently they didn't know about the jdbc-mssql gem that does the same thing
<headius[m]>
"Now I don't have any personal interest or time to spend writing or maintaining this stuff. I have no projects today that depend on the activerecord-sqlserver-adapter"
<enebo[m]>
headius: it should be pointed out rdubya has already done the work to make arjdbc work with this gem
<headius[m]>
this sort of comment tips me the other way...he coul drop this thing tomorrow
<headius[m]>
enebo: yeah I'm just trying to understand what we want to go where
<headius[m]>
rdubya: ah ok thanks...so that's a non-issue
<enebo[m]>
well regardless of his desire to continue supporting it he is now and it is largely a finished project sans bugs which all projects have
<enebo[m]>
and we use that code in all possible cases
<headius[m]>
so I agree with his (1), the gem for CRuby could just depend on tiny_tds. Having a better later that's beneath this (something sort of like, say JDBC!) would be nice but I can understnd not wanting that complexity unnecessarily
<enebo[m]>
so in the case he quits and we adopt #1 for example then we would need to fix bugs in that forked repo
<headius[m]>
what do you guys prefer at this point?
<enebo[m]>
if we accept the current PR we definitely are fixing all bugs
<enebo[m]>
I wish kares was in this conversation
<enebo[m]>
At this point I think #1 makes the most sense even though it means a friendly fork of that adapter
<enebo[m]>
I think barring any strong objection from kares we should do #1 and rdubya seems willing to support that fork but I will also add myself to it
<headius[m]>
oh you had a list of four options...I have trouble reading this :-)
<enebo[m]>
yeah a synopsis is #4 is jesse PR
<headius[m]>
well if we're maintaining our own fork that's not very different from copying code in
<headius[m]>
hell it could vendor the contents of his gem as part of build
<enebo[m]>
#3 is getting them to release java tinytds-ish thing but not change sqlserver-adapter
<headius[m]>
same thing
<enebo[m]>
#2 is changing sqlserver-adapter to release jruby
<rdubya[m]>
ideally we keep the fork clean so that commits to the main gem can be easily merged into the fork
<enebo[m]>
well it is a huge difference that we can just fetch and see what changed
<headius[m]>
ah he does own tiny_tds too
<enebo[m]>
how do we track changes with #4
<headius[m]>
enebo: maintenance wise it's no different
<headius[m]>
we'd pin to a version and have to fetch and see what changed
<headius[m]>
but doing that via a git fork would be easier I agree
<enebo[m]>
ok yeah this is the main value of it
<enebo[m]>
in most cases we just merge when we decide to do next release and it will be two pushes no no explicit code merging
<headius[m]>
right
<enebo[m]>
the #4 PR would mean possible issues with code drifting from original project
<enebo[m]>
The downside is releasing an extra gem
<headius[m]>
physical copy within ar-jdbc repo would require the most steps to stay up to date
<headius[m]>
#4
<enebo[m]>
but to me that is not really a big deal since the updating will be simpler
<enebo[m]>
yeah and fwiw we don't know how much of that code is identical now much less a year from now
<enebo[m]>
although we could certainly copy that whole project into ours too
<enebo[m]>
I feel like that would be more challenging via git to deal with though
<headius[m]>
so if we did #1 we'd be "done" for the moment, and could put some extra work into finding a common subset of both copies we can both depend on
<headius[m]>
enebo: I'd picture something sort of like how we do rubyspec, but it's still cumbersome
<enebo[m]>
well #4 folks can help support #1 soln too
<rdubya[m]>
yeah, I would hope that they are like me and would be happy just to have an "official" solution to help with
<enebo[m]>
yeah our internal spec copy is not a model I think we would want for a real dep vs convenience for testing
<rdubya[m]>
seems like there are a number of people that have expressed interest but none have take the time to bite off completely supporting it
<headius[m]>
I'm on board with #1 if we can work to eliminate duplication in the jdbc fork
<rdubya[m]>
but hopefully if it works and its just bug fixes that are needed, more people would be able to contribute
<enebo[m]>
and the thing which should not be lost here is our plan is to upstream as much as we can to the main adapter project to keep our delta small
<enebo[m]>
which will partially involve getting some relationship/trust from that adapter author
<headius[m]>
right
<enebo[m]>
at this point we should engage kares to make sure we are not missing something important
<headius[m]>
yeah I agree
<rdubya[m]>
yeah, it would be good to hear from him
<enebo[m]>
rdubya: can you work through a fork and see if you can get it packaged enough to do with arjdbc release
<enebo[m]>
rdubya: like removing the tinytds transdep and whatever small patches are needed?
xardion has quit [Remote host closed the connection]
<rdubya[m]>
yeah I can start looking at it, that's basically what I've been doing for the work I've done so far
<enebo[m]>
I sort of imagine you must have done most of that already :)
<enebo[m]>
otherwise you would have no results
<enebo[m]>
I partially assumed though you were not doing the gem creation part of it
<rdubya[m]>
do we want to release the fork as the new version of the activerecord-jdbcmssql-adapter? or should it be a separate thing?
<rdubya[m]>
yeah at this point I just have a local copy that I've been commenting pieces out of
<enebo[m]>
yeah I was thinking about that as well
<enebo[m]>
I mean arjdbc could be the dep ala how it works for mssql-adapter project
<enebo[m]>
in same way as tinytds is
<enebo[m]>
That seems logical but then we used to have ar-jdbcmssql-adapter as part of our arjdbc releases
<enebo[m]>
The two projects likely are in lockstep regardless though so perhaps it should be whatever is simpler or less code
<enebo[m]>
well hmm maybe that is not true
<enebo[m]>
naming it activerecord-jdbcmssql-adapter and using arjdbc as trans dep would mean any fixes makes would mean it could be independently released unless we actually need to change any Java code
<enebo[m]>
the original project makes and we merge into the fork that is
<rdubya[m]>
yeah
xardion has joined #jruby
<enebo[m]>
rdubya: This is really cool. mssql will rise again!
<rdubya[m]>
lol
<headius[m]>
just to be clear, it would be jdbcsqlserver
<headius[m]>
not jdbcmssql
<rdubya[m]>
so you think it would be better to leave jdbcmssql capped at rails 4.2 support?
<rdubya[m]>
or have it include this fork as a dependency?
<enebo[m]>
How does rails hook in the adapter
<headius[m]>
well do we even have anyone using sql server on JRuby right now?
<headius[m]>
the invariant we've maintained with the other databases is that the config stays the same
<enebo[m]>
on older versions no doubt we do somewhere :P
<enebo[m]>
not on rails 5 though
<rdubya[m]>
we do, and there's been a number of other people asking for support for it
<headius[m]>
and if they're using activerecord-sqlserver-adapter then ours would be jdbcsqlserver
<enebo[m]>
but how does railsties/whatever translate adapter name to writing a Gemfile
<enebo[m]>
I say we want it to fit that convention
<headius[m]>
for JRuby it adds jdbc
<headius[m]>
there's not much more than that
<headius[m]>
right?
<rdubya[m]>
from what I can tell you specify the adapter name in your config
<headius[m]>
changing config from mssql to sqlserver isn't a big deal...unless you want to support both drivers
<rdubya[m]>
the current jdbcmssql version supports both
<headius[m]>
if JRuby isn't a huge ask but I just wanted to bring that point up
<headius[m]>
I mean like `if JRuby`
<rdubya[m]>
it actually recommends using sqlserver as the name of the adapter
<headius[m]>
then it's mostly a question of consistency
<headius[m]>
if we have depreacted mssql name we should deprecate it in the gem name
claudiuinberlin has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<headius[m]>
dep reacted
<enebo[m]>
rdubya: I don't have a strong opinion on this really...if people have to change from mssql to sqlserver to update to 52+ databsde.yml on rails then so be it
<enebo[m]>
rails 5 involves some other changes already
<rdubya[m]>
we could do like I've seen some other gems do and release a 1.4.99 or something that only adds a message about the name being changed
<headius[m]>
this whole 50+ exercise has been about reducing our diff
Donation_Drive has joined #jruby
Donation_Drive has quit [Remote host closed the connection]
<rdubya[m]>
enebo: yeah with all the other changes that you have to make when upgrading to rails 5, changing the database.yml file a bit is probably minor
<enebo[m]>
rdubya: yeah agreed
<kares[m]>
hey all, I did try to read the conversation ... still catching up and I will be leaving in a bit
<kares[m]>
honestly I would see as least resistance just going forward with AR-JDBC
<headius[m]>
I have to have airplane lunch now, bbiab
<kares[m]>
but than there's no one to maintain it ... so I am leaning towards the general consensus
<kares[m]>
you guys reach
<kares[m]>
esp. since there's now a precedent case with the Oracle adapter
<enebo[m]>
kares: and we will be maintaining the fork but have the benefit of being and to fetch and diff when they do change that adapter
<kares[m]>
that being said the maintainers aren't super welcoming us like Oracle did ;(
<kares[m]>
enebo: not sure about that ... AR-JDBC might be less work
<kares[m]>
you will start hitting issues and having to do work-around due the tiny_tds API
<enebo[m]>
kares: rdubya already went through the exercise and it is like 200 lines of code
<kares[m]>
yeah I've seen that I'm just saying its designed around tiny_tds and some features will end up adding more and more complexity
<kares[m]>
but hey its worth the experiment
<kares[m]>
if someone is willing to maintain it
<enebo[m]>
For me the biggest takeway is that mssql-adapter is largely in maintenance mode and not getting many changes past simple bug fixes. The dynamics of larger things breaking us in the future are not as likely
<enebo[m]>
kares: yeah and if the experiment fails we control the gem name and move it into arjdbc
<kares[m]>
rdubya: haven't looked but it should be possible to add jndi support?
<rdubya[m]>
using that project would help us semi-easily get all the way up to rails 5.2 support
<kares[m]>
easily right?
<rdubya[m]>
vs having to figure out all the changes ourselves for the other 2 versions, right now both prs only target 5.0
<enebo[m]>
kares: dr-itz will figure it out :P
<rdubya[m]>
@kares based on what I can tell, it should just work, if we support it with the abstract classes, but I don't have a setup to test it
<kares[m]>
kk no problem, just asking
<enebo[m]>
I was partially kidding
<kares[m]>
and we also partially agreed I believe :)
<enebo[m]>
haha
<kares[m]>
one concern might be performance but that would need someone to look into
shellac has quit [Ping timeout: 264 seconds]
<kares[m]>
since MS-SQL is doing query rewrites on top of arel sql generation I doubt its 'fast'
<headius[m]>
Oracle adapter is exactly the case I had in mind
<kares[m]>
to begin with
<headius[m]>
But those guys actually deploy a product with jruby
<kares[m]>
headius: and we also had numbers for Oracle
<kares[m]>
despite their Java scripting it was doing fine compared to AR-JDBC
<kares[m]>
altough that might have been the case of poor work on AR-JDBC's end
<enebo[m]>
At some level we have to consider that we were willing to drop support for this altogether. Not being as fast but working is better than deciding we cannot support it at all
<headius[m]>
Good point
<kares[m]>
yy its just a note from me ... I do not want to block any effort here
<rdubya[m]>
yeah at this point, not having it is keeping us from upgrading to rails 5
<rdubya[m]>
so I'm happy with "works"
<kares[m]>
certainly welcome rdubya's work ... its awesome someone looked into how difficult it is to support JRuby on top of the adapter
<kares[m]>
we should ask the PR author of the AR-JDBC MS-SQL work to try rdubya's work
<kares[m]>
or maybe easiest for them to try will be once we release (e.g. a pre-release)
<kares[m]>
and see how they feel about it
<kares[m]>
that would be a good indicator -> and they will know better if they miss some JDBC features
<rdubya[m]>
yeah, right now I have a bunch of things hard wired between projects so I can run the tests, so not exactly easy for them to test it out, but I would like to know their thoughts on the approach
<enebo[m]>
kares: sounds good to me we want their help too
<kares[m]>
enebo: rdubya I think this work towards the jdbcmssql adapter should go into a separate repo
<kares[m]>
not AR-JDBC, right?
<rdubya[m]>
yeah, I was thinking it would be a forked repo so it would be separate
<kares[m]>
sounds good to me ... we could create a fork under jruby org once name is negotiated :)
<kares[m]>
maybe just a swap: activerecord-mssqljdbc-adapter ... altough slightly weird but I am not much of an english poet
<rdubya[m]>
we can go with activerecord-jdbcsqlserver-adapter, I mainly asked about reusing the name so that people that are using the current version would know that there was a version that supported rails 5 available
<kares[m]>
actually the native name is activerecord-sqlserver-adapter
<kares[m]>
rdubya: exactly I just realized that :)
<kares[m]>
so its different from the old JDBC MSSQL adaper name
<rdubya[m]>
I'll throw a comment in the other PR with our general plan and see if they have any concerns/support