<GitHub126>
[jruby] eregon pushed 2 new commits to master: http://git.io/vC1BU
<GitHub126>
jruby/master 6e3cca6 Benoit Daloze: [Truffle] We need only a single constMissingNode as it has its own inline cache.
<GitHub126>
jruby/master ba40478 Benoit Daloze: [Truffle] Add a cache limit for const_missing.
<enebo>
klemanski: which version fo JRuby are you seeing this?
mkristian has joined #jruby
<klemanski>
ok i've found something that fixes this: phrase = "sterownik".bytes.to_a.pack('c*').force_encoding("utf-8")
<klemanski>
1.7.19 and 1.9.22
<klemanski>
but it is not new code, it worked before without problem
<enebo>
klemanski: but this must be an issue of some source of multi byte characters
<enebo>
klemanski: I have not seen any in your pastes
<klemanski>
this is not repeating for all phrases for "sterownik" there is problem bot for "terownik" there is none
<enebo>
klemanski: those are all 7bit ascii
<klemanski>
yes
<enebo>
klemanski: you are getting an AIIOBE from that specific string?
<klemanski>
yes i can reproduce this on this
<enebo>
klemanski: can you share a complete snippet…those above pastebins are not crashing are they?
<klemanski>
i will try to extract this
<klemanski>
this string is passed to method that matches it agains regexps and checks which regexp match
<enebo>
klemanski: does the regexp have mbc (multibyte chars) in it?
<klemanski>
they may have because the are build dynamically from db
<enebo>
7bit in ascii and utf-8 are the same so I am confused how we can see this sort of error without on
<enebo>
ah
<enebo>
klemanski: yeah getting exact input which crashes will make this a lot easier
<klemanski>
i will try to deploy this fix for now: .bytes.to_a.pack('c*').force_encoding("utf-8")
<klemanski>
looks like it helps
<klemanski>
and after fixing production i will try extracting sample
<enebo>
klemanski: well if it makes your stuff not crash then it is a bandaid but I would love to fix it for real
<enebo>
klemanski: I can see in the stack it is also coming in from Java land
<enebo>
oh well I think it might be anyways
<enebo>
but if you get a repro we will fix it or help you fix it
<enebo>
time to make a salad
jamiejackson has joined #jruby
<klemanski>
maybe regexp have other encoding and that's problem
colinsurprenant has quit [Quit: colinsurprenant]
<enebo>
klemanski: these bugs typically end up misassociating some string (or string in regexp) as wrong encoding…then we ask for number of chars and it explodes because it is not that encoding of string
<lopex>
numbers ?
<lopex>
klemanski: do you have reduced case ?
<klemanski>
lopex, not yet shareable i'm working on it
blaxter has quit [Quit: foo]
<klemanski>
i will try to finish it from home
klemanski has quit [Quit: This computer has gone to sleep]
<enebo>
mkristian: that is weird. we do not seem to care abuot extension but this does not work without it
<enebo>
mkristian: but if I did a require “test” and uri:classloader was in path it probably does work?
<mkristian>
no idea why the "." works and the jar does not. there should be no difference. but the ".class" extension is not check as far I can see.
<mkristian>
just try to find the list of extensions
<enebo>
yeah we do have different *Resource.java classes
<enebo>
maybe something with that?
<GitHub100>
[jruby] chrisseaton comment on commit 14a7a31: Yeah 500ms.... http://git.io/vC1ye
skade has joined #jruby
<mkristian>
enebo, I need to set -Djruby.aot.loadClasses=true only then the .class files will be searched - totally forgot this
<enebo>
mkristian: oh!!! I forgot too
maleghast has joined #jruby
bbrowning is now known as bbrowning_away
djbkd has joined #jruby
<mkristian>
do you know if this really needed inside the require_relative to first make a real_path and then do all the setup of the new path. if I just replace the filename of given path with the new name. then things work.
<mkristian>
let me gist you what I mean
<enebo>
ok I wondered if only stripping extension in require_relative was enough to fix this
hobodave has quit [Quit: Computer has gone to sleep.]
<mkristian>
I copied it from within the ruby-complete jar and did it wrong
<enebo>
mkristian: ok I think your case will not work if file does not end in / or \
<enebo>
and it relative_arg does not start with / or \
bb010g has joined #jruby
<enebo>
mkristian: I am not sure but I think one boon to relative_relative is that as absolute path it does not need to search LOAD_PATH
<mkristian>
ok this one also works: absolute_feature = File.join(File.dirname(file), relative_arg). the real_path with a relative "file" always assumes it was found via the current directory
<mkristian>
was just about to try something like this. let me see
<enebo>
klemanski: I see tons of ?? which I think is some character data right?
<mkristian>
enebo, Errno::ENOENT: No such file or directory - /Users/cmeier/projects/active/jruby/app/test1
<mkristian>
I did this before actually
<klemanski>
yes
<klemanski>
regexp is build ising Regexp.union
<klemanski>
* using
<enebo>
klemanski: sorry I am rebuilding but I am seeing tons of ? which to me means this gist is unable to display those chars
<enebo>
but I do see AIIOBE: 9
<enebo>
so :)
<klemanski>
? is regexp character
<enebo>
a-z??????
<enebo>
in []
<enebo>
:)
<enebo>
ogrodz[a-z??????]
colinsurprenant has joined #jruby
<klemanski>
i will try to simplyfy this regexp
rcvalle has quit [Ping timeout: 240 seconds]
<mkristian>
enebo, removing the absolute thingy will work. no realpath and no expand_path. even File.exists? on the relative file is false since it is not on the filesystem. what about a
<mkristian>
recue Errno::ENOENT
<enebo>
klemanski: but this ‘ogrodz[a-z??????]’ is that some mbc char after z not rendering for me?
<mkristian>
for the AOT compile case ?
<mkristian>
rescue
<klemanski>
enebo, will check that for sure but it's rather just "?"
<enebo>
ok klemanski so it could have been [a-z?]
<enebo>
klemanski: something is generating these or something
<klemanski>
ok i will check conditions before Regexp.union
<enebo>
mkristian: could we check for classloader/uri in absolute and use relative only for those and not rail via exception
<mkristian>
enebo, OK, but not this evening, well not on my side
<enebo>
klemanski: lopex: I see no mbc in this regexp at or in the phrase
<enebo>
mkristian: gist what you have and I will probably also look at this
<mkristian>
and if File.realpath would return something with uri:class loader: then the rest should work
<mkristian>
without my patch
<lopex>
klemanski: ok, that one blows /(?i-mx:^bram[ay])|(?i-mx:(sterowniki?|automatyka).*\bbramy?)/
<lopex>
klemanski: more reduction: /(sterowniki?).*\b/
<lopex>
even more "x".force_encoding("utf-8"); /x.*\b/
<klemanski>
ok that's really minimal example
<lopex>
klemanski: looks like a joni thing, can you file an issues there ?
<klemanski>
super
<klemanski>
* sure
<klemanski>
:)
rcvalle has joined #jruby
<nirvdrum>
lopex: I noticed that when force_encoding is called with the same encoding the String is using, part of it no-ops, but CoW is triggered and the code range is cleared. Is this an oversight? Or is that required for something?
<lopex>
nirvdrum: hmm, I vaguely remember puttng there modifyCheck
<lopex>
nirvdrum: but yeah, it's an no op
<lopex>
nirvdrum: modify check is needed anyways since the string might be frozen etc
<lopex>
so it sohuld raise
<lopex>
klemanski: got it reproduced that with internal joni api, so it's most likely joni
<enebo>
mkristian: thanks. perhaps we can solve realpath
<lopex>
with regexp and UTF8Encoding
<klemanski>
x = "x".bytes.to_a.pack('c*').force_encoding("utf-8") looks like fix
<mkristian>
enebo, just thinking. realpath cannot work since there is no app/test1.rb file inside the jar. and expand_path you want to act on the PWD for relative paths only the LoadService does interate over the extensions and looks into the uri:classloader:/ space. why does the caller does not have the uri:class loader: prefix ?
<enebo>
mkristian: how about for app/ itself?
<enebo>
mkristian: maybe we can just realpath the parent except for base case of just a filename??? hmm
<GitHub92>
[jruby] kml opened issue #3397: ArrayIndexOutOfBoundsException when utf8 string matched against regexp with with word boundary \b http://git.io/vCMei
<mkristian>
it does exist. but I think we are heading the wrong way. what if we have jar://some.jar on the load_path. the uri:classloader: prefix is there when using the *.rb files but not when using the *.class files
<enebo>
mkristian: so you think we need to make sure the .class appropriately have this same consistent prefix?
bbrowning_away is now known as bbrowning
<mkristian>
enebo, it feels wrong to fallback on "uri:class loader:" to search - it is just a very common case but just a special case
<enebo>
mkristian: at this pount I am unsure what the solution should be :)
<enebo>
mkristian: but you said you have other stuff to do so I don’t want to keep you
<enebo>
mkristian: I will try and learn more about this through experimentation
<mkristian>
cooking + eating
hobodave has joined #jruby
<nirvdrum>
mkristian: At some point I'd like to revisit how to prevent Truffle unit tests running for every task in the truffle-head branch.
<lopex>
klemanski: fixed on master
nateberkopec has quit [Quit: Leaving...]
<klemanski>
lopex, possible to backport to 1.7?
<lopex>
enebo: will there be 1.7 update soon ?
<enebo>
lopex: maybe
<enebo>
lopex: but we can for sure get in a joni fix
<lopex>
enebo: two actually
<lopex>
enebo: multibyte not word bound was also affected
<lopex>
will add tests
<enebo>
lopex: thanks… that was quick :)
<lopex>
enebo: it was easy since singlebyte versions had a fix already
<klemanski>
lopex, while still using 1.7.19 should I remove all \b from regexps?
<GitHub176>
[jruby] nirvdrum pushed 2 new commits to master: http://git.io/vCMk6
<GitHub176>
jruby/master 51cd5fe Kevin Menard: String#force_encoding should fail if the string is frozen.
<GitHub176>
jruby/master 793d175 Kevin Menard: Add spec for String#force_encoding called on a frozen string.
<klemanski>
remove == replace
nateberkopec has joined #jruby
<GitHub124>
[jruby] chrisseaton pushed 1 new commit to truffle-head: http://git.io/vCMky
<GitHub124>
jruby/truffle-head 47d19d5 Chris Seaton: [Truffle] Fix guards for foreign dispatch.
mkristian has quit [Quit: This computer has gone to sleep]
<lopex>
klemanski: yeah nad \B - these will blow when matching at the end of a string
maleghast has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<klemanski>
lopex, thanks for fixing this, now when i know what is the cause of problem it will be possible to fix the app
<GitHub143>
[jruby] lopex closed issue #3397: ArrayIndexOutOfBoundsException when utf8 string matched against regexp with with word boundary \b http://git.io/vCMei
<nirvdrum>
cremes: They're linked from the wiki, in case you ever need to find the link again.
<cremes>
nirvdrum: many thanks
<nirvdrum>
No problem.
Aethenelle has quit [Quit: Aethenelle]
<chrisseaton>
Does JRuby try to start sub-rubies in the same VM? If so, how can I turn this off? Symptom is warnings like 'warning: -J-original argument ignored (launched in same VM?)', where that argument is essential
lanceball is now known as lance|afk
<lopex>
klemanski: no problem
<lopex>
nirvdrum: aprarently I did it in 21fc38a5d51e24d4400e746ccd6eb95b64554386
<lopex>
that modify
<lopex>
nirvdrum: mri uses str_modifiable so modifyCheck for that
<lopex>
if it's still called so
<nirvdrum>
lopex: I added a missing spec for this.
<nirvdrum>
But I only added that check.
<lopex>
nirvdrum: but there's no modify
<lopex>
er, shouldnt be then
<nirvdrum>
lopex: I was going to remove it, but wouldn't changing the encoding in a ByteList constitute a modification?
<nirvdrum>
We can push the modify into the associateEncoding operation.
<lopex>
nirvdrum: oh right
<nirvdrum>
But that method is used by other code.
<nirvdrum>
So it would require a little deeper cleanup.
<lopex>
mri uses flags so that's not a part of cowed data
<lopex>
right
jensnockert has joined #jruby
<cremes>
found another performance issue (with master) but reducing this to a small example looks difficult.
<cremes>
still profiling
colinsurprenant has quit [Quit: colinsurprenant]
<nirvdrum>
lopex: I'm just suggesting we can prevent invalidating until we know that the encoding is actually going to change.
<lopex>
nirvdrum: yes
<cremes>
mri is 5x faster (even with brain dead threading) and rbx is 30x faster (but it SEGVs)
<lopex>
same encoding - not invalidation
mkristian has joined #jruby
<lopex>
nirvdrum: oh, and modify19 aleady does modifyCheck
<lopex>
so it was good anyways
<nirvdrum>
Oh? My bad then.
<mkristian>
nirvdrum, I did play around with it and should have the branch still around. let me see if I can find it
<nirvdrum>
mkristian: That'd be fantastic.
<mkristian>
nirvdrum, https://gist.github.com/mkristian/103403ae15ae9a034912 actually only the two surefire lines are important. not sure if I understood you right but this disables surefire for all but this one profile
<headius>
I added BigD and sqlite3-ffi under popular C exts
<jeremyevans>
headius: cremes: Sequel uses exceptions for validation/hook failures, not sure if you are hitting those
<headius>
that's a possibility
<headius>
jeremyevans: if they're not intended to be caught by users, you can provide an empty backtrace
<headius>
or use catch/throw instead
<cremes>
I’m within 30m or so of having a repro. We’ll have data shortly. :)
<headius>
catch/throw is preferred way to do exception-like stack unrolling
<jeremyevans>
headius: well, changing it would be a big backwards compat issue. Possible in Sequel 5 maybe, but not before
<headius>
are they user-visible exceptions?
<headius>
or just for internals of sequel
<jeremyevans>
headius: Besides, I think exceptions for validation failures are the way to go. AR-style save returns nil/false on failure makes it way to easy to call save and forget to check return values.
<jeremyevans>
headius: User-visible by default
<headius>
ok, that's not unreasonable then
<jeremyevans>
headius: If you want nil/false on validation failure, you can turn on a setting, but it just uses an internal rescue for that
<headius>
the exception cases we dealt with in csv were using exceptions to failover from one converter to another internally
<headius>
massive perf hit you can't do anything about as a user
<headius>
in the nil/false case you could also give it a bogus backtrace btw, since you know they won't see it
<jeremyevans>
I've just switched my largest, oldest app (originally designed in Rails 0.11 or so) from save returns nil to save raises and it resulted in much better, easier to understand code, and caught quite a few bugs
<enebo>
If backtraces are not important threadlocal singleton exceptions will be faster on JRuby but also on MRI
<enebo>
The speedup will be less pronounced on MRI but it should still give something
<enebo>
but if backtraces are important then there’s that too :)
<jeremyevans>
I don't consider validation failures a big performance issue. It's going to be way less processing raising an exception that querying a database, which is what it would do on success
<enebo>
yeah exceptional behavior on a resource
<jeremyevans>
Anyone with truly high performance needs probably shouldn't be using models anyway :)
<enebo>
fwiw no one knows why cremes code is slower…we just always suspect exceptions :)
<enebo>
we do emulate a PPC machine for sqlite3 :)
<headius>
enebo: MIPS
<headius>
but it has a native binding for some platforms
<enebo>
well we emulate a MACHINE
<headius>
INDEED
<enebo>
:)
<headius>
jeremyevans: yeah I don't dig return codes for errors either
<cremes>
jeremyevans: not using models. doing a straight-up @db[:table].import(header, rows, :commity_every => 500)
<jeremyevans>
cremes: Yep, that shouldn't be raising an exceptions
<jeremyevans>
cremes: Could be a jdbc issue if you are using that on JRuby
<jeremyevans>
cremes: IME, the jdbc adapter is slower than the postgres adapter
<headius>
using jdbc has to do marshalling of ruby values to java and back
<headius>
unfortunately :-(
lance|afk is now known as lanceball
<jeremyevans>
gotta run now, drop a note in #sequel if you are having performance issues and you think they are sequel related
<cremes>
here’s the thing… I can get very fast 9k+ inserts/sec with some data on jruby. something I’m doing now is causing it to only insert at a rate of about 100/sec.
<headius>
that's why a sqlite3-ffi would be nice, and we need to get out pg lib for JRuby
<headius>
cremes: wow
<enebo>
cremes: O_o epic
yfeldblum has quit [Ping timeout: 240 seconds]
<headius>
enebo: hey, I finally got approval for Oliver to be out of school
<headius>
where are you planning to stay in Tokyo?
<headius>
I was thinking that little place in Akihabara wouldn't be bad...it was reasonably priced
<enebo>
headius: we have not figured anything out yet
<cremes>
shoot, my attempts at reproducing this are failing. I’m getting between 4k-12k inserts/sec.
<enebo>
cremes: start hacking partfs of import in sequel to time methods with the full script
<headius>
cremes: huzzah! ship it!
<headius>
maybe it's not sequel then
<enebo>
cremes: only run your script with sample data seems to be one answer I am hearing
<enebo>
maybe dirty data raising
<cremes>
argh, very frustrated
* enebo
cannot leave the exception line of thought
<headius>
enebo: how long are you planning to go?
<headius>
it looks like O and I will be in town for about a week after Taipei
kith_ is now known as kith
<GitHub76>
[jruby] nirvdrum pushed 2 new commits to master: http://git.io/vCMhO
<GitHub76>
jruby/master 26fb9e4 Kevin Menard: [Truffle] Replaced some profiles in the `string_copy_from` primitive with guards.
<GitHub76>
jruby/master 87d73d3 Kevin Menard: [Truffle] Added profiles around branches in the `string_copy_from` primitive.
<headius>
enebo: re IntHashMap
<headius>
there's no reason this new lock could deadlock
<headius>
I think there's a bug in IntHashMap
<enebo>
oh
<headius>
it is being hit hard for the first time now that kares used it for JI
<headius>
I'm going to pull in Cliff Click's code and see how it looks
<headius>
it's public domain
<enebo>
sure that sounds good
<enebo>
If we see someone using 9 in prod deadlocking it feels like a blocking problem
<headius>
I think one thread has the lock and is stuck like in the original report, and now a second one is stuck witing for the lock
<headius>
enebo: fwiw this is priority over __FILE__ right now
<enebo>
headius: I think they are both blockers
<headius>
yeah
<headius>
I suppose so
<headius>
damn AOT
bb010g has quit [Quit: Connection closed for inactivity]
<headius>
I always have a heck of a time reading the graph profile
<cremes>
search for “call name” with two spaces in between. I think you want the threads where the Total Time: is over 100secs
mkristian has joined #jruby
<headius>
this isn't telling me much
<headius>
Array#each, Proc#call
<cremes>
heh, you really don’t know how to read it :)
<headius>
yup
<headius>
I don't...contrib copied this format from rbx
<cremes>
this might be slow: Java::OrgSqliteJdbc4::JDBC4Statement#executeUpdate
<headius>
well, that could just be slow because it's hitting DB
<headius>
no?
<headius>
could it be throwing too many threads at the sqlite driver that's a problem?
<headius>
it's not exactly a high-scale DB
<cremes>
sure, but why slow in a multi-threaded app but super fast with a single thread?
<enebo>
contention
<headius>
contention for some lock in the driver
<cremes>
the writes are to separate DBs, so no contention for the DB itself
<headius>
hmmm
<headius>
ok
<enebo>
cremes: but you said you were getting 9k/s for some data
<cremes>
sure, in a single-threaded environment
<cremes>
I can dump this whole script plus some sample data on you if you like. nothing proprietary here.
<cremes>
I’m just stumped
<enebo>
cremes: have you tried just running with 2 then 4 then 8 and seeing if you can see threads falling over
<cremes>
enebo: going to try that next. good idea.
<enebo>
cremes: but if it is contention you would think it would be simple to see with a java heap dump (although god that is probably big :) )
<headius>
cremes: repro for us would be great of course
<cremes>
i’m trying, i’m trying....
<headius>
I'd look at GC and --sample output first
<headius>
if it's not maxing CPU then the problem is memory pipeline or contention most likely
<enebo>
am I reading this write…log_yield is pretty epensive?
<enebo>
I never quite grok this format of profiling
<cremes>
if you all want a different format, tell me which one to produce. happy to oblige.
<enebo>
heh whoops I get this
<enebo>
the log_yield wraps the action
<cremes>
btw, with a single thread handling all DB writes, I am getting about 1500/s. With 4 threads, about 250/sec.
<cremes>
guessing there is some locking contention in the JDBC driver.
<enebo>
77% of this thread is in execute and executeUpdate within a log_yield bloc
<enebo>
cremes: hmmm yeah either the driver or how we our PPC weird-ass sqlite impl
bbrowning is now known as bbrowning_away
<headius>
MIPS
<enebo>
but with that said I don’t think sqlite3 can even do concurrent writes
<headius>
:-D
<enebo>
HAHA yeah I will say PPC for the rest of my life I think
<headius>
probably
<headius>
if it's actually using the MIPS emulator it could be *way* faster
<cremes>
enebo: it can’t to the same file, but I am opening different files/DBs depending on what data I need to write. I keep the file open so I’m not paying the open/close IO price every time.
<cremes>
and only one thread is ever trying to write to a specific file, ever.
<cremes>
other threads might be writing to other databases/files.
djbkd has quit [Remote host closed the connection]
<cremes>
anyway, I have an idea on how to improve my repro.
<headius>
it looks like the driver only uses native sqlite on windows
tcrawley is now known as tcrawley-away
<headius>
cremes: windows or linux, I forget what you said if you said
<cremes>
OSX this time
<headius>
ah
<enebo>
with a 32G heap?
djbkd has joined #jruby
<headius>
nice apple box you got there
<cremes>
64GB for the win :)
<headius>
yeah, pretty sure that's using the JVM MIPS emulator
<headius>
boo
<enebo>
I sure hope these nothing dumb about that driver like using a static field
<headius>
so even your best sqlite3 perf on JRuby might be 4-8x slower than it could be
mkristian has quit [Quit: This computer has gone to sleep]
<headius>
enebo: it certainly could be
<enebo>
headius: because we do better than MRI on one thread
<headius>
yup
<headius>
it could have a single machine emulator that everything's bottlenecking on
<headius>
on Windows the driver includes a native DLL with sqlite3 built in
<enebo>
headius: if adding threads hurt but threads are not contending for the same DB since cremes is opening multiple DBs?
<enebo>
headius: but if he opens 4 sqlite3 dbs it would br running 4 machines
yfeldblum has joined #jruby
<headius>
maybe, maybe not
<enebo>
headius: makes me think something is sharing something not helpful
<headius>
it may not run them in separate machines and lock around calls to the one
<headius>
i'm looking into the code
<headius>
he does have makefiles for the jni on other platforms
<headius>
that's poor form but I wouldn't expect it to be this big a bottleneck
<enebo>
WOW
<enebo>
oh wait only saving the pointer field
samphippen has quit [Client Quit]
<headius>
yeah
<headius>
they should just pass it as a param
<headius>
silly
<enebo>
well yeah arguably wasteful to retrieve the field value but not the big cost unless it has a global lock on java side
<enebo>
we are getting a handle to retrieve db data :)
klemanski has quit [Quit: Leaving]
<enebo>
we could make a ruby byte-based one and kill off the transcoding
<headius>
the lookup of the field could be a hit
<headius>
yeah I know
<enebo>
headius: but the hit is not thread contending
<headius>
doesn't seem to be
<headius>
very little GC happening too
<headius>
two young GC during the main part of bench, that's it
jensnock_ has quit [Ping timeout: 250 seconds]
<headius>
like 0.012s
djbkd has quit [Remote host closed the connection]
<enebo>
they could haveeasily make 2 java methods first being the one the call in NativeDB and second passing the pointer/handle
<enebo>
maybe they did not see a lot of benefit
<headius>
I'm trussing it to see if anything stands out
<headius>
heh
<headius>
well this doesn't look good
<headius>
psynch_mutexdrop 19441
<headius>
psynch_mutexwait 20101
<headius>
highest syscall counds according to dtruss
djbkd has joined #jruby
<headius>
followed closely by
<headius>
psynch_cvsignal 6317
<headius>
psynch_cvwait 6487
<headius>
there's a mutex in here I'm not seeing
<headius>
could be in sqlite3 itself, unsure
<headius>
or JVM
<enebo>
DB is synchronized for all methods
<enebo>
for is re-opening new dbs use same instance it would sync all access
<headius>
yeah unclear if it reuses same instance
<enebo>
headius: going to dinner but check to see ifthere is 1 or n DB instances in a heap dump running that
<headius>
sure
<headius>
I see flags about mutexing in the sqlite parts too
<headius>
like NOMUTEX versus FULLMUTEX
Aethenelle has joined #jruby
<headius>
NOMUTEX would be appropriate for 1:1 thread:connection
<cremes>
headius: if you want it to run longer than 7s in the bench, just duplicate the slow.csv data 2-5x.
<headius>
no problem
<headius>
definitely not looking like a JRuby issue but we'll see what we can do
mdedetrich has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<headius>
my current theory is that the DB sets up sqlite to fully mutex all operations, and it's mutexing against something global
<headius>
I mean the JDBC sets up...
gregoriokusowski has joined #jruby
<xardion>
yeah, this has to be something driver specific. I use multiple threads with the vertica jdbc driver and jruby and have no issues.
<headius>
I wouldn't be surprised if they mutexed this to death to be sure JDBC concurrency was safe
<xardion>
To be fair, I'm not writing that much data, mostly reading.
<headius>
and in this case it's not needed
<xardion>
That said, I'm about to test moving a process that currently uses multiple workers to write into that same db to use multiple threads instead. Then we'll see
<xardion>
it's extremely high volume too
colinsurprenant has quit [Quit: colinsurprenant]
<enebo>
headius: did you try to see how many DB exist?
<headius>
not yet, poking around the code
<enebo>
headius: if only 1 then it is there
<headius>
will do
<xardion>
currently we push about 7k rows/sec
<xardion>
we'll see if it has any effect when I do 4 threads instead of 4 processes
<xardion>
That'll be tomorrow though most likely
<headius>
xardion: what DB?
<xardion>
vertica
<headius>
new to me
<xardion>
It's an HP product.
<headius>
ahh
<xardion>
column-oriented db with a SQL layer on top of it
<xardion>
it's great for doing selects on large datasets, and inserts as well, updates are god-awful slow.