<kares>
ooh I see: there's package org.postgresql.jdbc
<kares>
yeah that is definitely a class-loading problem
<mengu>
exactly
<mengu>
so, i've tried putting all the jars
<kares>
we haven't tested jndi much but this is ugly anyways
<mengu>
activerecord-jdbc.jar
<mengu>
adapter_java.jar
<mengu>
jruby-core-9.2.5.0-complete.jar
<mengu>
jruby-stdlib-9.2.5.0.jar
<kares>
not sure what to do - except for resorting to reflection
<mengu>
postgresql-42.1.4.jar
<kares>
rdubya: hey, you might know more about possibly working around a package org.postgresql.jdbc in arjdbc's ext ^^
<mengu>
yeah, the final error i got was could not find method load_java_part in ArJdbcModule
<kares>
that is cause you moved the jar containing the ext part
<mengu>
i guess i'm the only person who tried jndi with psql :D
<mengu>
i didn't move it out
<kares>
in theory you could get it to work if you move just the one class from the ext jar to be at the same location as the PG driver
<mengu>
the generated war also has it, and tomcat has it as well
<kares>
no ideas otherwise - sorry I do not have time to look in detail but rdubya might jump in with more ideas how to avoid that class
<mengu>
all the jars above ^^ are in $CATALINA_HOME/lib
<kares>
mengu: yes you're likely the only one doing AR 5.x with PG using JNDI
<kares>
this worked in <= 4.x but was heavily refactored since
<kares>
just skip jndi for now or set a Java connection pool (e.g. Hakari) manually at your end
<mengu>
yeah, i initially used the latest postgresql.jar and then got the error thought maybe i should use the version in jruby lib
<mengu>
hmm
<mengu>
so tomcat uses hikaricp
<mengu>
and my database.yml connects to that?
<mengu>
well, if i skip the jndi, i got connection problems :D
<kares>
no TC does not use Hakari it has its own pool - Tomcat JDBC or DBCP
<kares>
but there are ways setting these up by hand - binding a DataSource to JNDI
<mengu>
yeah, but i can use hikari with TC as well
<kares>
Hakari was just a suggestion
<rdubya>
hey mengu, kares: I had to do that because it was the only way I could get the information to build out the return object
<kares>
database.yml stays as is but you need to do a setup before the app boots - it might get messy but should work
<mengu>
ActiveRecord::Base.establish_connection to hikari or tomcat's pool?
<mengu>
rdubya: i believe so, and there are no problems whatsoever when you are not using jndi
<kares>
rdubya: so no other options except reflection than, right?
<kares>
... if only there was some other way getting the Field info needed wout a "hack"
<mengu>
kares: but no matter what jdbc pool i connect to, i believe i'm going to get the same error anyway, no?
<kares>
mengu: if you setup the pool by hand from within the app it should work
<mengu>
or my final resolution is to use pgbouncer
<kares>
you won't have it configured with TC but instead manually load .jar and setup the pool
<mengu>
i'll give that a try
<kares>
than establish_connection as you wrote - you do not have to bind to JNDI you can do a establish :data_source => configured_hikari_ds
<kares>
but it needs to happen before Rails connects
<kares>
I believe PG bouncer doesn't work with the JDBC driver (on Java/JRuby)
<kares>
or feel free to try avoiding that ext class (if you know some Java) and submit a PR
<mengu>
i am thinking of workarounds
<mengu>
i definitely would like to contribute
<mengu>
i mean i can mvn package :D
<kares>
+1
slyphon has joined #jruby
slyphon has quit [Ping timeout: 240 seconds]
jmalves has quit [Ping timeout: 240 seconds]
slyphon has joined #jruby
slyphon has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<headius>
g'day
<headius>
kares: The cases I fixed probably aren't problems, since they use Java APIs that should be doing that
<kares>
talking embedded nulls?
<rdubya>
headius: good morning, saw the article in forbes, congratulations on the award!
<headius>
kares: yeah
<headius>
but we do call native APIs and these paths might eventually be going directly into one of them
<headius>
so other than making tests pass it seems like a good idea to do these checks
<headius>
rdubya: thanks :-D
travis-ci has joined #jruby
<travis-ci>
jruby/jruby (master:8249863 by kares): The build was broken.
travis-ci has left #jruby [#jruby]
<kares>
okay - good point, will keep that in mind (whether a string/byte-list might have originated from a native)
<kares>
yay CI now works - no fork notifications
<mengu>
rdubya: do you have any suggestions for me? what should i check? i'd like to contribute if i can figure out a solution
slyphon has joined #jruby
<rdubya>
mengu: sorry I haven't looked at that stuff in quite a while, it looks like I needed the OID and modifier to look up the active record type object
<rdubya>
since they are indexed by postgres' oid value
xardion has quit [Remote host closed the connection]
xardion has joined #jruby
<mengu>
rdubya: got it
shellac has joined #jruby
<headius>
ar-jdbc ships a class in org.postgresql.jdbc package?😳
shellac has quit [Quit: Computer has gone to sleep.]
Aethenelle has joined #jruby
<mengu>
headius: yes
slyphon has quit [Read error: Connection reset by peer]
slyphon has joined #jruby
<headius>
rdubya: can you elaborate more on why this is needed? Obviously it's to get access to getField but why does that have info we can't get another way?
<headius>
I also notice that the getField method is protected, not package-visible, in the javadoc mengu linked, which is probably the source of the error
<headius>
mengu: but you say it's fine outside of jndi?
<mengu>
exactly
<mengu>
when the postgresql driver for tomcat and jdbc adapter's driver are on the classpath, things go south
<headius>
and this is the same pg jar that ar-jdbc would use itself?
<headius>
same version etc
<mengu>
exactly the same version
<mengu>
i dug it out from the warble generated jar :)
<headius>
ok
<headius>
I guess reflection is the only other way to do this
<mengu>
what i also did is, i ran mvn package on so i have activerecord-jdbc.jar, i got adapter_java.jar, jruby-core-9.2.5.0-complete.jar, jruby-stdlib-9.2.5.0.jar as well
<headius>
I'm not sure this would compile against the protected version of getFields
<headius>
it mentions package visibility in that file
slyphon has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<mengu>
and probably that's why rdubya generates a class under org.postgresql
<mengu>
and actually i think adding all these jars to tomcat's lib dir kinda worked
<mengu>
but then just another error occured
<mengu>
related to load_java_part method
<rdubya>
headius: this was the only way I could find to get the field data for a generic query. I'm definitely not strong in Java though so there may be another way to get the protected data
<rdubya>
mengu: yeah that's why I had to put it in there, its a hack to make that method accessible
<headius>
well I'm confused, don't these other methods provide access to what you need?
<mengu>
makes sense
<rdubya>
which other methods?
<headius>
I see most of the Field methods exposed via the PgResultSetMetaData methods
<headius>
what do we need out of Field exactly?
<rdubya>
OID and mod
<headius>
ok I see that now
bleda has joined #jruby
<headius>
and those are indeed not exposed by the metadata
<headius>
can we get them to add them? :-D
<rdubya>
that would be ideal, at this point I unfortunately don't have any bandwidth to focus on it
<headius>
what does the Oid get us? I see other accessors for information about the Oid
<headius>
not all public again though
<rdubya>
we use it to look up the type converter on the ruby side
<headius>
hmm
<mengu>
so what do you guys think would be a faster solution to this issue? kares suggested maybe to initialize the activerecord connection directly through hikaricp before rails does
<headius>
that's outside my expertise
<headius>
sounds like it could work
<rdubya>
yeah, sorry, I don't have any experience with jndi
<headius>
I'm not sure I understand why it doesn't work with jndi though...it clearly sees the class we created but it's denying access
<headius>
this error should only come up if visibility of the target has changed since the code was compiled
<headius>
but if that's the case I'd expect it to always fail
<headius>
kares: do you know why this fails?
<headius>
enebo: or you...I'm wondering if we can infer the contents OID and Mod through other means
<headius>
like, could we query for it directly? Infer it from the available accessors in the metadata?
<headius>
I'm going to get back to the stdlib/test upgrade for now
<enebo>
oh man I need to read back a ways :)
Puffball has joined #jruby
shellac has joined #jruby
shellac has quit [Quit: Computer has gone to sleep.]
slyphon has joined #jruby
shellac has joined #jruby
bleda has quit [Ping timeout: 272 seconds]
subbu is now known as subbu|lunch
shellac has quit [Ping timeout: 244 seconds]
shellac has joined #jruby
<mengu>
i'm gonna go ahead and create an issue on pgjdbc project on github
shellac has quit [Ping timeout: 272 seconds]
subbu|lunch is now known as subbu
<enebo>
forgot to read back until now. I did not know we did that either, but I can only guess the "extra" class is not loaded from the same classloader. Perhaps now java 9+ (if that is what mengu is using) won't allow this?
<mengu>
i'm on jdk8 i believe
<mengu>
yup
<mengu>
java version "1.8.0_112"
<enebo>
ok so not that...
<enebo>
I would still feel classloaders are to blame somehow...they always are :P
<mengu>
that's my opinion as well
<mengu>
i also wonder the reason behind protected
<enebo>
mengu: but definitely open an issue...if there is info and we really don't have a way to get it then we should definitely get that ball rolling
<mengu>
we've got a project that we've sold to enterprise customers
<enebo>
It is easy to imagine someone just assumed no one would need it
<mengu>
and we are enjoying jruby
<mengu>
but the app is under heavy load
<mengu>
we've got conn pools to blame there
<mengu>
then i've decided to give it a go
<mengu>
failed miserably :D
<enebo>
mengu: so if you did build and package just that class into pg adapter then you may or may not realize whether this will solve your particular problem
<mengu>
what i'm trying now is compiling pgjdbc package, modifying protected modifiers to public
<enebo>
mengu: we obviously want it fixed either way but I would not want this to limit your ability to see if this will fix your problem
<enebo>
mengu: yeah good. At least you can move forward a bit then
<mengu>
like, they could really assume nobody would need those methods public
<mengu>
i believe so
<enebo>
yeah I find most people are a little too conservative on what they think people will use
<mengu>
alright
<mengu>
now i got something else ActiveRecord::JDBCError (Cannot unwrap to org.postgresql.PGConnection):
<mengu>
let's give tomcat's conn pooling a shot first, before hikari
<enebo>
mengu: as kares said when rails 5 work was done there was a lot of changes to replace supporting many versions of rails/jdbc and we had no tests to verify jndi was working. So we definitely have made changes which maybe will need extra work
<enebo>
mengu: I guess if you can access and catch that JDBCError there is a @jdbc_exception variable with the original exception too
<mengu>
i somehow think this is related to having two postgres drivers on classpath
<mengu>
does it make sense if i get rid of jars packaged in the warble generated war?
<enebo>
mengu: none of this is my expertise. My hacking of ARJDBC has been more about simple JDBC + AR itself
<mengu>
or should i use config.excludes of warbler?
<enebo>
any warbler people here?
<enebo>
mengu: but I would get original exception and not the wrapped just to see if it gives you more clue on what is not working
<headius>
you could probably omit most of the details that are in the ar-jdbc bug and just link to it
<headius>
important bit for them is that we'd like that method made public so we don't have to hack around getting at Oid and Mod
<mengu>
makes sense
<headius>
once you file it one of us can jump on to endorse and clarify
<headius>
I think in the short term we should modify it to use reflection
<headius>
rdubya: is this used at runtime or just on the initial import of the table?
<headius>
if it's just the initial import we can afford a bit of reflection overhead to avoid this error
<headius>
I seem to remember that package visibilty gets managed at a classloader level so if you have two org.postgresql.jdbc packages loaded in different classloaders they aren't considered the same
<headius>
if I'm remembering right that would explain the issue
<headius>
rdubya: I think reflection instead of that shim should work fine for now, right?
<mengu>
headius: probably that was the main issue in the first place
<headius>
reflection we can always force, assuming no modules (java 9) or security manager
<headius>
and pgjdbc would have to be modularized to restrict that, which I doubt they've done
<mengu>
wait
<mengu>
it doesn't let me create an issue
<mengu>
the button is disabled
<mengu>
i cannot be more dumb
<mengu>
i didn't write a title
<mengu>
lol
<mengu>
headius: and how can we fix having two postgresql drivers? one for tomcat and one from jdbc-adapter?