hoodow has quit [Read error: Connection reset by peer]
jrafanie has joined #jruby
hoodow has joined #jruby
hoodow has joined #jruby
hoodow has quit [Changing host]
<headius>
they distribute a tz jar don't they?
<headius>
quarterly tz data
<headius>
asarih was updating it for us at one point
<GitHub144>
[jruby] headius pushed 3 new commits to ruby-2.5: https://git.io/vxzMU
<GitHub144>
jruby/ruby-2.5 d344f52 Charles Oliver Nutter: Eliminate unnecessary abstract class.
<GitHub144>
jruby/ruby-2.5 70379dd Charles Oliver Nutter: Merge branch 'master' into ruby-2.5
<GitHub144>
jruby/ruby-2.5 69ec12c Charles Oliver Nutter: CGI#escape should not escape tilde. #4876
<headius>
kares: in any case, we should try to always ship latest tz data
<headius>
so yes
<kares>
headius: you're right, its there - part of joda-time.jar
<kares>
so this seems redundant
<kares>
remove?
<headius>
remove what exactly?
<headius>
we update the tz data independent of joda, since it is released more often
<headius>
or we don't want to risk joda update
<kares>
its bee updated in 2013?
<kares>
joda-time will have newer data in the .jar
<kares>
so it seemed to me like duplication? to include the org.jruby:joda-timezone as well?
<kares>
if its to stay than it definitely should get updated ...
<headius>
I think it should stay and we should go back to updating it periodically
<headius>
maybe as a release step
<headius>
enebo: ^^
<headius>
we can add an exciting perma-bullet to the release notes for * Updated time zone data
<enebo>
headius: yeah I don't know the risks past dates and timezones sucking
<rdubya>
kares: enebo: I pushed some changes for arjdbc PR that I made last week if you want to take a look. Hopefully it makes more sense now.
<enebo>
rdubya: looking
<enebo>
rdubya: do those comments even match AR adapter source about type map and using execute? I remember this values.first.first
<rdubya>
not sure which comment you mean
<enebo>
+ # Need to use #execute so we don't try to access the type map before it is initialized
<enebo>
in adapter.rb
<rdubya>
yeah, otherwise we try to fetch the type for the column so that we can build the AR::Result and it goes into an endless loop
<rdubya>
just realized that I have a conflict in that PR too so I'll take care of that, shouldn't change much though
<kares>
rdubya: hmm I am actually adding postgresql_version on the native end - extracting from JDBC meta-data on 50-stable
<kares>
is it too much?
<kares>
I mean it should help, but there still others initializing along with type-map
<rdubya>
yeah, that should be fine
<rdubya>
just need to make sure it doesn't run through exec_query
<kares>
haven't thought this thru but to add back jndi support the right way these things would need a proper review
<kares>
or at least be configurable ...
<kares>
but yeah maybe some day ;(
<enebo>
rdubya: I think the impact of this has nothing which hurts mysql or sqlite3. Whether mysql wants to follow suite is a question since they have a native result with the same or very similar small set of methods
<enebo>
you eliminate ColumnData which should reduce some allocation so you more than pay for your result object
<enebo>
The fact the jdbc to ruby methods are now in a specific connection adapter class could yield some duplicated code but as we talked about before if you work on pg support it is all in one place
<enebo>
So I am not very concerned about and DRYness
<enebo>
(if we really wanted to dry we could call to a common helper class with static impls but YAGNI)
<rdubya>
I was planning on updating the other two so we could pull a bunch of that out of the other adapters as well
<rdubya>
we can't really remove it until *all* the other adapters are updated though I guess
<enebo>
rdubya: it is questionable whether sqlite ever should
<enebo>
rdubya: but I have not studied it to know
<rdubya>
I think it has the concept of a result as well, but not 100% sure
<rdubya>
but I think the benefit of them all using a standard interface and the encapsulation it provides
<rdubya>
would make it worth doing
<enebo>
rdubya: if so then yeah mysql has result in their native adapter and it is a reasonably simple type of each, size, and fields (for column type)
<enebo>
I imagine all probably do this at some level even if not in a specific Result instanc3e
<enebo>
rdubya: so that util/ActiveRecord we need to change
<enebo>
you can read my comment before we talk
<rdubya>
i can pull that out if you want
<enebo>
I will just start with my desire. We can just look them up once in initialize() and save them as fields
<enebo>
I don't know if we need to worry about them changing?
<enebo>
kares: ^ we always lookup constants like ActiveRecord all the time
<rdubya>
yeah, i still don't know enough about how jruby <-> java interact to say if its safe to cache it
<enebo>
maybe for dev mode?
<enebo>
does dev mode completely tear down stuff like this? I don't think so
<kares>
enebo: that is fine - more problematic was call-site caching for callMethod
<kares>
but some of it should be in ... hopefully
<kares>
but yeah sure there's ton of room for improvement :)
<enebo>
kares: yeah calls are another issue but we definitely need equivialent to site() like we have in core
<enebo>
I just do not want to codify a static method pattern which will make that impossible
<kares>
you mean for constant caching, right?
<enebo>
yeah
<kares>
ok
<enebo>
This PR change made things no worse except it is a static API which makes changing it later less convenient
<kares>
smt would be useful - also wondered about a 'pattern' previously at some point in jossl
<enebo>
kares: yeah for extension authors a nice mechanism for site caching constants and methods would be very nice
<kares>
okay - trust your judgment I am in a middle of smt and already on low brain activity
<enebo>
kares: ok :)
<kares>
guess we can change pieces as we go - my concern will not show up until I attempt a piece
<rdubya>
I'm more concerned about getting the result stuff in at this point, so I'm good just duplicating the methods that I need access to in there
<rdubya>
and removing the static stuff I added
<kares>
+1
<enebo>
rdubya: yeah other than that static stuff I am happy
<rdubya>
cool, i'll pull that bit out then
<enebo>
rdubya: you even cheapened the allocation a bit
<enebo>
two primitive arrays for columns
<enebo>
vs n ColumnData also i is just i of the array
<enebo>
so one less field in a sense
<enebo>
If columns were in the 100s perhaps making two arrays maybe becomes some extra cost but these will generally be small
<kares>
have seen AR managed tables with 10s of columns
<kares>
its wild out there :)
<enebo>
kares: oh yeah but even 80 column primitive array is not a huge array
<enebo>
kares: finding space for it anyways
<kares>
yy
<kares>
might run some benchmarks later
<enebo>
kares: if there is a simple way to run benchmarks I would love to do that too
<kares>
although not sure if the adapter for PG is in a benchmarkable state ...
<kares>
well just with some AR operations
<enebo>
it is always in a benchmarkable state!
<kares>
some lives under bench/...
<enebo>
:)
<enebo>
kares: oh these are the ancient ones
<kares>
well its quite demotivating if you compare to MRI :)
<kares>
I know but they're good indicator
<enebo>
no that is great
<enebo>
I just mentioned them in passing to rdubya saying we used to have a set
<enebo>
I did not realize we still had them
<enebo>
kares: rdubya did get some nice prepared stmt improvement
<enebo>
kares: maybe we will be pleasantly surprised
<enebo>
although we no doubt regressed performance as well during 5.x work
<enebo>
kares: you just need to make us pure native adapters so we can bypass JDBC!
<kares>
yeah that was amazing
<enebo>
kares: then we can use bytelist all the way through :P
<kares>
never actually approached going that way - so lesson learned
<enebo>
that would easily double perf of simple queries
<kares>
exactly - that's what I had in mind
<enebo>
no more utf16 transcoding crap
<kares>
its possible by going to some native APIs
<kares>
already checked MySQL/JDBC
<kares>
get the encoding raw bytes and map to ruby
<enebo>
Probably the only huge achilles heel of Java
<kares>
possible but time consuming + there's 2 druivers we support somore work ...
<kares>
* so more work
<enebo>
kares: yeah I know I tried in the past with MySQL (aka 2008)
<enebo>
kares: something weird broke with one of the raw methods
<enebo>
so it has been a decade (geezus).
<kares>
that and some Time/Date handling used to be slower on raw AR ops
<kares>
but the Date/Time part should be done
<kares>
also some of it was to account to JRuby's date.rb
<enebo>
kares: but your recent work no doubt will make it better
<kares>
that should also be done - so progress I guess
<enebo>
I heard there will be a new async jdbc coming as well
<kares>
java 11?
<enebo>
I don't know how it fits into anything
<enebo>
I just literal saw someone mention those two words next to each other
<kares>
ah okay - there are attempts
<kares>
there's another driver for PG as well
<kares>
which I naively thought at some point of adding support
<kares>
but than it got wild with PG's type system that Rails did ...
<kares>
although since than the 'official' driver has caught up - it used to be 'slow'
<headius>
enebo: I've heard that before, is there something new?
<enebo>
headius: I was just hoping it was something concrete but I have not checked
xardion has quit [Remote host closed the connection]
<GitHub66>
[jruby] headius pushed 7 new commits to ruby-2.5: https://git.io/vxgJp
<GitHub66>
jruby/ruby-2.5 b4b381b Charles Oliver Nutter: Rogue copy got left behind.
<GitHub66>
jruby/ruby-2.5 9802921 Charles Oliver Nutter: Range no longer rescues exceptions during init <=> call. #4876
<GitHub66>
jruby/ruby-2.5 2e850a5 Charles Oliver Nutter: Add Exception#full_message. #4876.
<headius>
lunch!
justinmcp has quit [Remote host closed the connection]
justinmcp has joined #jruby
shellac_ has joined #jruby
<rdubya>
enebo: just pushed up changes for that PR if you think it looks good I'll try to get it merged and then backport this and the other changes I had with the exec_ methods to the 50-stable branch
<headius>
ugh
<enebo>
rdubya: I will loook
claudiuinberlin has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<headius>
kares: you didn't do anything that might cause an FTP+ssl spec to hang, did you?
<headius>
just started in past few pushes to ruby-2.5
<enebo>
I think in the case of arjdbc it may not matter a ton but it is difficult to know
<headius>
along these same lines I wanted to bring up another idea
<enebo>
nice odin's ax
<enebo>
hammer
<enebo>
heh
<headius>
the pure-indy invoker stuff works well last time I checked it
<enebo>
it makes thunder
<headius>
basically it has a single DynamicMethod subclass that just holds a couple MethodHandles
<headius>
I have never gotten it quite as fast at startup as the generated invokers, so it has not been on
<headius>
but it might be appropriate to turn it on with -Xcompile.invokedynamic
<enebo>
well it will be faster than inheritance searching for a method over and over
<headius>
if I could get it as fast as generated invokers we could simply them but it's maybe 10-20% slower booting right now
<headius>
this also plays into making AOT actually AOT and using newer JDK utils
<enebo>
I think as a nice solution extension authors could love this
<headius>
the newer JDKs have the ability to pre-create method handles in constant pools
<headius>
so if we had all these pieces in place, we'd be able to precompile Ruby, ditch invokers, and pre-optimize all of it using new tools
<headius>
that's what we'd turn into a toolchain for ext authors
<enebo>
yeah
<enebo>
so this looks like some extra unneeded coding
<enebo>
or is his example just showing how it can be hand coded
<enebo>
"Mjolnir is built using pro which is my own build tool, you can download it from github (amazon S3) like this"
<enebo>
no remi no
<headius>
I know
<enebo>
hopefully it is a maven artifact at leastr
<headius>
I always have to fork his projects to use them
<headius>
it's not
<headius>
he refuses to push anything to maven
<enebo>
hahaha ok well it is oss
<enebo>
jruby-hammer
<headius>
I've asked too
<headius>
he's a character
<enebo>
so we call it jruby-hammer and then we define API called screw
<headius>
but yeah all his shit is public domain or MIR
<headius>
MIT
<headius>
ha
<headius>
from his readme it looks pretty simple to use
<enebo>
something about rewriting always feels dirty
<headius>
which means it should be nearly impossible to integrate into maven toolchain
<headius>
yeah I know what you mean
shellac_ has quit [Quit: Computer has gone to sleep.]
<headius>
failing of java
<headius>
no way to do indy
<enebo>
yeah but we would want something which intercepts ordinary callMethod from our types and basically reflects and then makes proper bootstrap
<headius>
right
<headius>
so we'd either start normalizing to a single Mjolnir path or customize it in some way
<enebo>
Well I wonder how much we can help some other projects with this
<headius>
the code may be simple enough we just gank this into our repo and do it our way
<enebo>
I bet nearly all have dyn dispatch
<headius>
ah well if you want to be a good guy
<enebo>
we could renamespace it
<headius>
making this a general-purpose indy-from-java would be great
<enebo>
seems like a descriptor .java explaining which types/method combos and it just rewrites
<enebo>
I hoped this project did that since that would be excellent
<headius>
yeah
<enebo>
Still I guess he did make the good bits
<enebo>
headius: imagine just plugging that into java reflection and rewriting on the fly using profiling!
<headius>
so we get this, we start specializing methods that take blocks (through IR inlining or manually regenerating bytecode) and then profit
<headius>
heh yeah
<enebo>
ah that is interesting
<headius>
we need to revisit profiling for incrementally deeper indy stuff too
<enebo>
I think native calls which calls blocks
<headius>
like initially just bind stuff via DynamicMethod.call until it gets hot enough we want to try the heavier direct binding
<headius>
if that lightweight binding matched CachingCallSite we could start going all indy call sites
<headius>
hell, compile.invokedynamic could still use indy call sites but just simple ones
<headius>
probably not much more overhead than CachingCallSite everywhere
<headius>
better caching
<enebo>
what I like about Mjollnir is the notion we could just write ordinary Java code and then change where we do this
<headius>
right
<headius>
it's cool
<enebo>
So in places which don't matter we just do something slower
<headius>
and with that freedom we could do it many more places
<enebo>
I keep thinking there are limits to how much indy that can be in play
<headius>
at this point, recent 8+, they've done a ton of work to reduce the memory overhead of indy
<headius>
MH trees now just go straight to bytecode very quickly rather than waiting
<enebo>
I am also thinking about warmup though (acutally mostly warmup)
<headius>
once they do they collapse
<headius>
yeah this helps that too
<headius>
but it *is* still generating code at runtime
<enebo>
yeah
<headius>
that's where jlink etc can help
<headius>
you pre-emit the handles
<enebo>
appcds?
<headius>
appcds is in all of them but starts to take advantage of modules and jlink in 10
<headius>
it's only Free in 10+...it's free for dev in 8 and 9
<enebo>
oh heck gotta run for a bit
<headius>
we only want it for dev anyway
<headius>
ok
<headius>
grr...I'm going to have to dissect this
<headius>
and bisect too
<headius>
of course it's my collapsing of RubyException into AbstractRubyException
<headius>
guess I'll re-attempt that
<GitHub52>
[jruby] JasonLunn opened issue #5115: ConcurrentModification error during Tomcat startup https://git.io/vxgs1
<GitHub123>
[jruby] nomadium opened pull request #5116: Range should raise TypeError if necessary method is not defined (ruby-2.5...fix-range-ruby-2.5-should-raise-type-error-if-necessary-method-is-not-defined) https://git.io/vxgG5
<GitHub179>
[jruby] headius pushed 2 new commits to ruby-2.5: https://git.io/vxgZI
<GitHub179>
jruby/ruby-2.5 8e631ca Charles Oliver Nutter: Second attempt removing unnecessary abstract class.
<GitHub179>
jruby/ruby-2.5 d237355 Charles Oliver Nutter: Revert "Eliminate unnecessary abstract class."...
<GitHub151>
[jruby] headius pushed 2 new commits to ruby-2.5: https://git.io/vxgZO
<GitHub151>
jruby/ruby-2.5 ab440ad Charles Oliver Nutter: Merge pull request #5116 from nomadium/fix-range-ruby-2.5-should-raise-type-error-if-necessary-method-is-not-defined...
<GitHub151>
jruby/ruby-2.5 ec02fe1 Miguel Landaeta: Range should raise TypeError if necessary method is not defined...
<GitHub101>
[jruby] headius closed pull request #5116: Range should raise TypeError if necessary method is not defined (ruby-2.5...fix-range-ruby-2.5-should-raise-type-error-if-necessary-method-is-not-defined) https://git.io/vxgG5
<headius>
enebo, kares: I think I'm going to merge 2.5 to master at this point
<headius>
we haven't been green for a while anyway, and we'll just mop up remaining tests on master
<headius>
gotta land this stuff so it can bake
<GitHub171>
[jruby] headius pushed 1 new commit to ruby-2.5: https://git.io/vxgCe
<GitHub171>
jruby/ruby-2.5 be3bd17 Charles Oliver Nutter: StringIO#write takes multiple arguments. #4876
<GitHub75>
[jruby] headius closed issue #4876: Ruby 2.5 Support https://git.io/vbUxG
<GitHub135>
[jruby] headius opened issue #5118: Unimplemented Ruby 2.5 Features https://git.io/vxgaC
<lopex>
headius: String#each_grapheme_cluster and String#grapheme_clusters are done
<lopex>
headius: and landed today
<headius>
oh right you are
<headius>
I'll fix that
<headius>
most of the remaining failures look pretty mundane
<GitHub116>
[jruby] headius closed issue #4458: NotImplementedError: kill unsupported or native support failed to load - works on RHEL5, fails RHEL7 https://git.io/vMxL1
<lopex>
headius: is there a similarly pruned 2.4 issue ?