mattwildig has quit [Remote host closed the connection]
samphippen has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
Petesta has joined #jruby
colinsurprenant has quit [Quit: colinsurprenant]
mdedetrich has joined #jruby
Petesta has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
whitby has quit [Ping timeout: 268 seconds]
mdedetrich has quit [Ping timeout: 240 seconds]
kstuart has joined #jruby
bbrowning_away has quit [Read error: Connection reset by peer]
bbrowning has joined #jruby
kstuart has quit [Ping timeout: 244 seconds]
mattwildig has joined #jruby
mdedetrich has joined #jruby
mattwildig has quit []
rsim has quit [Quit: Leaving.]
bb010g has joined #jruby
elia has quit [Quit: Computer has gone to sleep.]
elia has joined #jruby
elia has quit [Quit: Computer has gone to sleep.]
mdedetrich has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
mdedetrich has joined #jruby
mdedetrich has quit [Client Quit]
cprice has quit [Quit: Konversation terminated!]
kstuart has joined #jruby
tenderlove has quit [Quit: Leaving...]
camlow325 has quit []
rsim has joined #jruby
_djbkd has quit [Remote host closed the connection]
rsim has quit [Quit: Leaving.]
nirvdrum has quit [Ping timeout: 240 seconds]
mdedetrich has joined #jruby
e_dub has quit [Remote host closed the connection]
e_dub has joined #jruby
jeregrine_ has joined #jruby
yfeldblum has quit [Ping timeout: 246 seconds]
tjohnson_ has joined #jruby
atambo has joined #jruby
AckZ_ has joined #jruby
olleolleolle has quit [Ping timeout: 240 seconds]
AckZ has quit [Ping timeout: 240 seconds]
tjohnson has quit [Ping timeout: 240 seconds]
jeregrine has quit [Ping timeout: 240 seconds]
atamb0 has quit [Ping timeout: 240 seconds]
AckZ_ is now known as AckZ
tjohnson_ is now known as tjohnson
jeregrine_ is now known as jeregrine
mdedetrich has quit [Ping timeout: 250 seconds]
mdedetrich has joined #jruby
olleolleolle has joined #jruby
_djbkd has joined #jruby
mdedetrich has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
mdedetrich has joined #jruby
pawnbox has joined #jruby
pawnbox_ has joined #jruby
pawnbox has quit [Ping timeout: 240 seconds]
SynrG has quit [Ping timeout: 252 seconds]
nirvdrum has joined #jruby
yfeldblum has joined #jruby
nirvdrum has quit [Ping timeout: 244 seconds]
mdedetrich has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
mdedetrich has joined #jruby
pawnbox_ has quit [Remote host closed the connection]
pawnbox has joined #jruby
thedarkone2 has quit [Quit: thedarkone2]
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #jruby
mdedetrich has quit [Read error: Connection reset by peer]
mdedetrich has joined #jruby
tjohnson has quit [Quit: Connection closed for inactivity]
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #jruby
rsim has joined #jruby
digitalextremist has quit [Ping timeout: 264 seconds]
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #jruby
rsim has quit [Quit: Leaving.]
digitalextremist has joined #jruby
projectodd-ci has quit []
projectodd-ci has joined #jruby
tlarevo has joined #jruby
pitr-ch_ has joined #jruby
pitr-ch has quit [Read error: Connection reset by peer]
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #jruby
pawnbox has quit [Remote host closed the connection]
pawnbox has joined #jruby
tlarevo has quit [Read error: Connection reset by peer]
mdedetrich has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
kstuart has quit [Ping timeout: 260 seconds]
tlarevo has joined #jruby
kstuart has joined #jruby
pawnbox has quit [Ping timeout: 265 seconds]
pawnbox has joined #jruby
bb010g has quit [Quit: Connection closed for inactivity]
dinfuehr has joined #jruby
jensnockert has joined #jruby
mdedetrich has joined #jruby
blaxter has joined #jruby
_djbkd has quit [Quit: My people need me...]
mdedetrich has quit [Client Quit]
jensnockert has quit [Ping timeout: 240 seconds]
jensnockert has joined #jruby
pawnbox has quit [Read error: Connection reset by peer]
pawnbox has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
skade has joined #jruby
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
mkristian__ has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
skade has quit [Quit: Computer has gone to sleep.]
pawnbox_ has joined #jruby
jensnockert has joined #jruby
skade has joined #jruby
pawnbox has quit [Ping timeout: 260 seconds]
dinfuehr has quit [Remote host closed the connection]
jensnockert has quit [Read error: Connection reset by peer]
vtunka has joined #jruby
jensnockert has joined #jruby
mkristian__ has quit [Quit: This computer has gone to sleep]
jensnockert has quit [Read error: Connection reset by peer]
elia has joined #jruby
jensnockert has joined #jruby
mkristian__ has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
projectodd-ci has quit []
sebstrax has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
pawnbox_ has quit [Remote host closed the connection]
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
SynrG has joined #jruby
projectodd-ci has joined #jruby
pawnbox has joined #jruby
tlarevo_ has joined #jruby
samphippen has joined #jruby
tlarevo has quit [Read error: Connection reset by peer]
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
drbobbeaty has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
shellac has joined #jruby
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
drbobbeaty has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
shellac has quit [Quit: Computer has gone to sleep.]
jensnockert has quit [Read error: Connection reset by peer]
vtunka_ has joined #jruby
jensnockert has joined #jruby
vtunka has quit [Read error: Connection reset by peer]
vtunka_ has quit [Ping timeout: 264 seconds]
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
PragTob has joined #jruby
tlarevo_ has quit []
jensnockert has quit [Read error: Connection reset by peer]
shellac has joined #jruby
jensnockert has joined #jruby
vtunka_ has joined #jruby
djellemah has quit [Read error: Connection reset by peer]
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
dinfuehr has joined #jruby
mdedetrich has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
pawnbox has quit [Remote host closed the connection]
jensnockert has joined #jruby
pawnbox has joined #jruby
mdedetrich has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
skade has quit [Quit: Computer has gone to sleep.]
jensnockert has quit [Read error: Connection reset by peer]
shellac has quit [Ping timeout: 240 seconds]
jensnockert has joined #jruby
PragTob has quit [Remote host closed the connection]
temporal_ has quit [Read error: Connection reset by peer]
temporalfox has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
skade has joined #jruby
pawnbox has quit [Ping timeout: 240 seconds]
jensnockert has joined #jruby
djellemah has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
cristianrasch has joined #jruby
jensnockert has joined #jruby
drbobbeaty has joined #jruby
mkristian__ has quit [Quit: This computer has gone to sleep]
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
mdedetrich has joined #jruby
bbrowning has quit [Ping timeout: 244 seconds]
mkristian has joined #jruby
jensnockert has joined #jruby
shellac has joined #jruby
bbrowning has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
mkristian has quit [Quit: This computer has gone to sleep]
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
tcrawley-away is now known as tcrawley
jensnockert has quit [Read error: Connection reset by peer]
yfeldblum has quit [Ping timeout: 240 seconds]
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
dinfuehr has quit [Remote host closed the connection]
dinfuehr has joined #jruby
jensnockert has joined #jruby
dinfuehr has quit [Ping timeout: 250 seconds]
nirvdrum has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
shellac has quit [Quit: Computer has gone to sleep.]
skade has quit [Quit: Computer has gone to sleep.]
skade has joined #jruby
mdedetrich has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
temporalfox has quit [Ping timeout: 244 seconds]
dinfuehr has joined #jruby
temporalfox has joined #jruby
samphippen has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
samphippen has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
<eregon_>
enebo: Hi! Can I update com.headius:options to 1.2 in jruby? I observed quite a few times NPE due to loaded=true but value still null (fixed in 1.2)
eregon_ is now known as eregon
jensnockert has quit [Read error: Connection reset by peer]
shellac has joined #jruby
jensnockert has joined #jruby
nateberkopec has joined #jruby
mkristian has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnock_ has joined #jruby
brauliobo_ has joined #jruby
colinsurprenant has joined #jruby
jensnock_ has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
rsim has joined #jruby
temporalfox has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
mkristian has quit [Quit: This computer has gone to sleep]
enebo has joined #jruby
rsim has quit [Quit: Leaving.]
rsim has joined #jruby
rsim has quit [Client Quit]
tjohnson has joined #jruby
colinsurprenant has quit [Quit: colinsurprenant]
brauliobo_ has quit [Ping timeout: 255 seconds]
kares has quit [Quit: I'm using a Free IRC Bouncer from BNC4FREE - http://bnc4free.com/]
digitalextremist has quit [Ping timeout: 255 seconds]
digitalextremist has joined #jruby
brauliobo_ has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
temporalfox has joined #jruby
mkristian has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
rsim has joined #jruby
jensnockert has joined #jruby
nirvdrum has quit [Ping timeout: 250 seconds]
jensnockert has quit [Read error: Connection reset by peer]
mkristian has quit [Quit: This computer has gone to sleep]
skade has quit [Quit: Computer has gone to sleep.]
jensnockert has joined #jruby
skade has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
tenderlove has joined #jruby
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
whitby has joined #jruby
skade has quit [Quit: Computer has gone to sleep.]
whitby has quit [Quit: I go.]
whitby has joined #jruby
dinfuehr has quit [Ping timeout: 272 seconds]
dfr has quit [Ping timeout: 246 seconds]
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
lance|afk is now known as lanceball
baroquebobcat has quit [Quit: baroquebobcat]
camlow325 has joined #jruby
dfr has joined #jruby
tcrawley is now known as tcrawley-away
jensnockert has quit [Read error: Connection reset by peer]
jensnockert has joined #jruby
skade has joined #jruby
baroquebobcat has joined #jruby
pawnbox has joined #jruby
rsim has quit [Quit: Leaving.]
jensnockert has quit [Read error: Connection reset by peer]
rsim has joined #jruby
jensnockert has joined #jruby
pawnbox has quit [Remote host closed the connection]
rsim has quit [Client Quit]
jsvd has joined #jruby
pawnbox has joined #jruby
rsim has joined #jruby
rsim has quit [Quit: Leaving.]
<jsvd>
hi, I'm seeing a leak in jruby 1.7.{20,21,22} on windows with File.stat: http://imgur.com/a/ksbG2
pawnbox has quit [Remote host closed the connection]
<jsvd>
in a previous test, a loop for ~1h made the process use 3g on windows, while the jvm heap was under 150 mb
pawnbox has joined #jruby
<enebo>
jsvd: we recently post 22 changed the stat we use to be windows-oriented calls and not using the windows posix emulation library
<enebo>
jsvd: 9.0.3.0 does have these changes. If you could verify the same problem with 9.0.3 it would give us a data point
<jsvd>
enebo: version <= .19 wasnt affected by the same problem, I think
<enebo>
hmm
<jsvd>
enebo: yep I'm now testing with 9k, 1 sec
<enebo>
jsvd: mildly scary if true since I think we have released jffi since then
<jsvd>
it's quite easy to reproduce, just a loop of file:stat
jensnockert has quit [Remote host closed the connection]
<enebo>
jsvd: can you create an issue on this.
brauliobo_ has quit [Ping timeout: 264 seconds]
<jsvd>
enebo: yep, just wanted to run past irc first
<enebo>
jsvd: yeah I will try and snoop and see if this is only windows now
brauliobo_ has joined #jruby
shellac has quit [Quit: Ex-Chat]
<jsvd>
enebo: we've only had problems on windows. this is affecting the file input on logstash. people running for a while with lots of files eventually show this mem usage pattern
<enebo>
jsvd: just playing with NMT
<enebo>
I wonder if this can help see JNI alloc’d native memory
<jsvd>
in VMMAP we can clearly see tons of 56kb chunks of unusable memory and their 8kb counterpart off heap
<enebo>
jsvd: another data point might be to use any other api on windows which allocates a FFI Struct in jnr-posix
<enebo>
if we see that leaking then something lower down is not cleaning this stuff up when finalization is happening
vtunka_ has quit [Ping timeout: 260 seconds]
<enebo>
wow I just reviewed the last changes for windows stat and the allocated structs do not live beyond the stat call in jnr-posix itself
<enebo>
the returned stat instance now is a POJO
<enebo>
So perhaps those instances are not cleaning up
<enebo>
but it did make me realize something else rather interesting
<enebo>
We should just use thread locals since these are out structs and we can reuse then per thread
<enebo>
which would sort of leak a single struct per thread forever but it would not constantly be doing native allocs
<enebo>
and in theory if they thread dies it should clean up (which may still be a leak based on this would mitigate this a lot)
<enebo>
jsvd: I guess no one else is chatty today so my above rambling is for you :)
<enebo>
jsvd: thanks
sebstrax has joined #jruby
<jsvd>
haha I follow the reasoning but honestly I'm not familiar with that code base (or libraries) at all
<jsvd>
took me and a few others a few days to find this
<jsvd>
in a sort of desperate way I'm proud, haha.
<jsvd>
enebo: do you see any way of manually freeing this inside of jruby code?
<enebo>
jsvd: yeah I guess the general understanding is that every time we call out of the JVM via JNI (via JFFI) it is expensive and potentially using native memory. By just using a ThreadLocal we will avoid an entire allocation and a JNI callout. It should make stat faster somewhat
jensnockert has joined #jruby
<enebo>
jsvd: the native memory getting alloc’d is in a local variable in jnr-posix stat() method. It is dead by the time the method exits
<enebo>
jsvd: so something is not properly finalizing it when GC finally happens (GC can take a long time to fire based on memory usage in heap)
<enebo>
jsvd: This idea of us making thousands of temorary native structs is a bad idea in general due to GC firing be non-deterministic
<enebo>
jsvd: with that said finalization will happen on a FULL GC and should clean them all up
<enebo>
jsvd: the only way this could not be a leak is if your test app never runs a full gc
<enebo>
jsvd: which is doubtful
<jsvd>
yep, I manually triggered it in visualvm
elia has quit [Quit: Computer has gone to sleep.]
<jsvd>
did nothing, even heapdumps were 21 mb of size, while the process reported 3g working set in task manager (6g commited)
<enebo>
I just did a stat loop in macos and can see no leaking
<enebo>
I ran for like 15 minutes
<enebo>
weird though. Hard to imagine a huge difference in behavior for alloc/free/finalization on windows only
<jsvd>
I'd bet that this is a windows only problem, since the same codebase is used by logstash users on *nix
<jsvd>
and no complains
jensnockert has quit [Ping timeout: 268 seconds]
<jsvd>
and several on windows
balo has joined #jruby
<enebo>
jsvd: I think based on my above polemic on us needing a full gc to clean up native memory I might just change this to use threadlocal variable for these temp structs
<enebo>
jsvd: it will not solve this problem per se but it will go away until you spin up thousands of new threads constantly
<jsvd>
enebo: I'd probably just make the point that this didn't happen on .19, and started on .20
<jsvd>
could be a potential nice investigation point
<enebo>
jsvd: yeah it is a great data point
<enebo>
jsvd: I am hoping to get out 9.0.4 out this week and I want quickest fix possible so I might still make the change I mentioned
<enebo>
but we will circle back and fix this in general via your issue
<enebo>
jsvd: If you are into sleuthing this more you can see if we changed any of our jnr-* dependencies in pom.rb/pom.xml betwen .19 and 20
<jsvd>
enebo: so would this cause the need for a 1.7.23? we're still using 1.7.x on logstash
<enebo>
jsvd: it would show up in both versions
<enebo>
jsvd: we could maybe do a 23 next week but rubyconf is week after and I need some demoware done :)
_djbkd has joined #jruby
<enebo>
jsvd: to more broadly answer the question we need to get a 23 our since we have several other fixes in the branch already
<jsvd>
+1
<jsvd>
I'll check the differences from .19 to .20 to see if there's anything else more obvious, and post in the issue if so
<enebo>
jsvd: thanks for reporting this
nirvdrum has joined #jruby
<jsvd>
enebo: no problem, hopefully the demoware is easy to do ;)
<enebo>
jsvd: need to unbreak our method inliner so maybe? :)
xardion has joined #jruby
drbobbeaty has quit [Read error: Connection reset by peer]
<nirvdrum>
enebo: What's the plan for the 9.0.4.0 release?
<enebo>
nirvdrum: well to address this memory leak and to fix the require’ing files out of jars problem on windows
<enebo>
nirvdrum: with those two fixed then we push out what we have
<nirvdrum>
Right. I mean I thought it was coming out a week ago to address some Window-specific issue in jnr-posix.
<nirvdrum>
Gotcha.
<nirvdrum>
So, "soon".
<enebo>
nirvdrum: yeah JavaOne was busy and we decided to punt
<enebo>
yeah (air quotes) soon
<nirvdrum>
I figured JavaOne was to blame. I just wasn't sure if something else came up.
<enebo>
nirvdrum: yeah communication for me always breaks down when I am at conferences
sebstrax has quit [Quit: Connection closed for inactivity]
bbrowning is now known as bbrowning_away
skade has quit [Quit: Computer has gone to sleep.]
skade has joined #jruby
tcrawley-away is now known as tcrawley
johnjoseph has joined #jruby
elia has joined #jruby
* johnjoseph
waves at codefinger
<codefinger>
yo
<johnjoseph>
alas, no change in behavior under 1.7.19
shellac has joined #jruby
jensnockert has joined #jruby
<johnjoseph>
Also I attempted a simplistic recreation of jsvd's bug on os x and couldn't replicate it. i don't have the memory visualization tool but I just watched total memory used, it didn't go up with the loop running for a couple minutes. looks like in jsvd's replicate it went up to more than 1G in a minute?
<johnjoseph>
* in jsvd's test
camlow32_ has quit [Remote host closed the connection]
camlow325 has joined #jruby
camlow325 has quit [Remote host closed the connection]
baroquebobcat has quit [Quit: baroquebobcat]
elia has quit [Quit: Computer has gone to sleep.]
jensnockert has quit [Remote host closed the connection]
thedarkone2 has joined #jruby
camlow325 has joined #jruby
camlow325 has quit [Remote host closed the connection]
camlow325 has joined #jruby
subbu is now known as subbu|away
yfeldblum has joined #jruby
jensnockert has joined #jruby
bbrowning_away is now known as bbrowning
rsim has joined #jruby
rsim has quit [Client Quit]
yfeldblum has quit [Remote host closed the connection]
<jsvd>
johnjoseph: yep but just on windows, afaict
skade has quit [Quit: Computer has gone to sleep.]
brauliobo_ has joined #jruby
skade has joined #jruby
baroquebobcat has joined #jruby
_djbkd has quit [Remote host closed the connection]
_djbkd has joined #jruby
blaxter has quit [Ping timeout: 268 seconds]
_djbkd has quit [Read error: Connection reset by peer]
_djbkd has joined #jruby
rsim has joined #jruby
<enebo>
chrisseaton: congrats on your phd
<chrisseaton>
enebo: thanks! all done now
<enebo>
chrisseaton: Will you have more hacking time now? :)
<chrisseaton>
now I just need to lose the 20 kg and years of stress it's added to my body and mind
<enebo>
haha
<chrisseaton>
I'm looking forward to concentrating on just one thing, yeah
rsim has quit [Quit: Leaving.]
rsim has joined #jruby
temporal_ has joined #jruby
temporalfox has quit [Ping timeout: 252 seconds]
pawnbox has quit [Ping timeout: 265 seconds]
skade has quit [Quit: Computer has gone to sleep.]
rsim has quit [Quit: Leaving.]
<enebo>
hmmm I just switched to having structs use threadlocal and the leak still happens
<enebo>
Makes me think this might be allocateDirect on wpath never getting freed
<enebo>
the fun continues
bb010g has joined #jruby
whitby has quit [Quit: I go.]
camlow325 has quit [Read error: Connection reset by peer]
camlow325 has joined #jruby
<enebo>
nirvdrum: how does allocateDirect memory get freed
jensnockert has quit [Remote host closed the connection]
rsim has joined #jruby
jensnockert has joined #jruby
subbu|away is now known as subbu
cristianrasch has quit [Quit: Leaving]
<subbu>
chrisseaton, congrats Dr. Seaton :)
mkristian has joined #jruby
cristianrasch has joined #jruby
rsim1 has joined #jruby
rsim has quit [Ping timeout: 240 seconds]
rsim1 has quit [Quit: Leaving.]
shellac has quit [Quit: Computer has gone to sleep.]
skade has joined #jruby
rsim has joined #jruby
yfeldblum has joined #jruby
mkristian has quit [Quit: This computer has gone to sleep]
_djbkd has quit [Remote host closed the connection]
rsim has quit [Quit: Leaving.]
<nirvdrum>
enebo: I need to look up the exact class involved, but somewhere in that crazy hierarchy there's a finalizer that does the free.
<enebo>
nirvdrum: yeah I am trying to figure out what is leaking in particular right now
camlow32_ has joined #jruby
<nirvdrum>
I don't have the jnr-posix source open right now, but I think for the Windows implementation we emulate with a Win32 call. That probably allocates a stat object that we need to explicitly free.
<enebo>
it sure helps to run the right version of jruby while doing this :)
<nirvdrum>
But if it really changed from JRuby 1.7.19 to 1.7.20, I'd diff those two versions of jnr-posix.
<enebo>
nirvdrum: I rewrote stat recently to only use windows APIs
camlow325 has quit [Ping timeout: 265 seconds]
<enebo>
nirvdrum: the only difference in jnr-posix itself was using wstring
<enebo>
nirvdrum: it was using the java string before that
<nirvdrum>
Maybe it's not leaking then and just legitimately using twice the amount of memory?
<nirvdrum>
Ahh.
<enebo>
nirvdrum: no it keeps going up and up
<enebo>
nirvdrum: I might have gotten the leak to stop but I need to let it run a while
<enebo>
nirvdrum: the finalization queue for a loop { stat } is extreme
<enebo>
nope still leaking
<enebo>
maybe slower
<nirvdrum>
I wish there were a way to directly free. It should be easily doable by making that method public and ensuring it's idempotent.
<enebo>
yeah I find that aspect of this library totally confounding
<enebo>
If I could control lifecycle of object I would write everythng around that and I could literally just see that there is no leak
<enebo>
at this point the lack of determinism in the GC firing and causing finalization seems like a big problem to me
<enebo>
This has to be some bug in memory allocation in jffi
<enebo>
There are zillions of 8k private data objects ever growing
<chrisseaton>
When we use JFFI, we always find the different types of allocation confusing and error prone - I'm not surprised there are leaks in code using it
<enebo>
I guess he might be alloc’ing larger segments for allocateDirect to use and some references are causing none of them to ever clean up or something
<enebo>
chrisseaton: wayne was/is a wizard at this stuff but I have always had a really hard time understanding what is happening below the covers
<chrisseaton>
I know in some cases the memory is lazily allocated
<enebo>
chrisseaton: I would have preferred less magic and more explicit life-cycle based calls where I can control alloc/free
<chrisseaton>
Or it'll use a byte[], and then convert to a real pointer if needed
<enebo>
yeah
<enebo>
some of the magic no doubt allows some optimizations by taking that control away
<nirvdrum>
enebo: The other fun part is allocateDirect doesn't always allocate directly.
<nirvdrum>
If under 256 bytes, it allocates out of an internal magazine.
<enebo>
yeah which no doubt is what is happening here
<enebo>
but I am now caching the wstring if same as last time and I am reusing the struct via threadlocal
<enebo>
somehow it is still leaking though
<chrisseaton>
ah yes - block allocation with nothing to compact the blocks
skade has quit [Quit: Computer has gone to sleep.]
<nirvdrum>
enebo: Maybe the Windows PageManager is broken.
<nirvdrum>
You could just use that API directly (cut out stat) to see I guess.
<enebo>
nirvdrum: jnr-ffi 2.0.1->2.0.3 and jnr-jffi 1.2.7->1.2.9
<enebo>
nirvdrum: which one is PageManager in?
<enebo>
nirvdrum: because these were the changes between 12->20
<enebo>
heh 19-20
<nirvdrum>
jffi
<nirvdrum>
Not to be confused with jnr-jffi
<nirvdrum>
Sorry, that one can be confused. The artifact is just "jffi", however.
<enebo>
gotta love the 89k diff from 1.2.7 to 1.2.9
<enebo>
I think things mostly moved around
camlow32_ has quit [Remote host closed the connection]
camlow325 has joined #jruby
camlow325 has quit [Remote host closed the connection]
camlow325 has joined #jruby
<nirvdrum>
I think chrisseaton is correct though. It looks like this thing keeps growing magazines without freeing them.
<enebo>
nirvdrum: well if they are 8k then I totally agree :)
<enebo>
I do not know why but 1.2.7 and 1.2.9 must come from different branches since I see the universe in this diff
_djbkd has joined #jruby
<nirvdrum>
It'd be 2 * pageSize
<nirvdrum>
A 4K page size sounds right.
camlow325 has quit [Remote host closed the connection]
<enebo>
this is way earlier than jffi release 1.2.7 or 1.2.9
<nirvdrum>
enebo: Looking at TransientNativeMemory, it looks like the constructor is supposed to clean up magazines by deleting everything with a phantom reference (whatever that is), but TransientNativeMemory holds references to every magazine in a CHM, so it should always have a strong reference.
<enebo>
nirvdrum: when was that modified last
camlow325 has quit [Remote host closed the connection]
<nirvdrum>
FinalizableReferenceQueue.cleanUpAll() should do the clean up.
<nirvdrum>
I'm not sure. I'm looking at downloaded sources without history.
<nirvdrum>
Ahh, GitHub blame works.
<nirvdrum>
2013.
<enebo>
yeah Idea has a nice feature
<enebo>
yeah a smattering of 2013
mdedetrich has joined #jruby
<enebo>
I guess I should verify the leak really is not happening for me locally with 1.7.19
<nirvdrum>
I'm looking at the downloaded sources jnr-ffi sources within JRuby since I don't have a clone of that set up.
<enebo>
sentinal is grey in the IDE
<nirvdrum>
It's unused as far as I can tell.
<nirvdrum>
It's not even properly set if memory is allocated.
<nirvdrum>
I suspect it's used to keep references.
dinfuehr has joined #jruby
<nirvdrum>
enebo: If it is TransientNativeMemory, it should leak on *nix, too.
<enebo>
nirvdrum: how about macos?
<nirvdrum>
The only two strategies are Unix and Windows.
<nirvdrum>
But this part is OS-indifferent.
<enebo>
nirvdrum: hmm
<nirvdrum>
I can't readily verify, but I can look later.
camlow325 has joined #jruby
<nirvdrum>
Any idea what a phantom reference is?
<enebo>
nirvdrum: there is an explanation between weak/soft/phantom online somewhere
<enebo>
nirvdrum: but I backed off to jnr-jffi on both jnr-posix and jruby and recompiled
bbrowning is now known as bbrowning_away
<enebo>
nirvdrum: appears to still be leaking
<enebo>
nirvdrum: I do see some 4k objects but the leak seems to be with 8k ones
rsim has joined #jruby
<nirvdrum>
Well the Magazine would be 8K I think since it's allocating two pages at a time.
<nirvdrum>
So, while I know next to nothing about phantom references, my 30s reading of the javadocs leads me to believe that Magazine#sentinel should really be Magazine#get.
<nirvdrum>
It looks like this was just partially completed. Maybe something Wayne was working on before he quit.
camlow325 has quit [Remote host closed the connection]
<enebo>
nirvdrum: but if so then we should see these change in some way between 1.7.19 and 1.7.20 timeframe
<enebo>
nirvdrum: but that code I think is older
<enebo>
nirvdrum: I am installing 1.7.19 to make sure it is working
<enebo>
nirvdrum: the other thing is 1.7.20 was first to wstring the path
<nirvdrum>
Did we implement stat differently in 1.7.19, however?
<enebo>
nirvdrum: it used String and not WString which uses a native converter
<nirvdrum>
Maybe it was done in jruby and not jnr-posix.
<enebo>
nirvdrum: but in my current version I end up caching that wstring so it should not be churning
<enebo>
nirvdrum: but who knows perhaps some other wrapped value is getting made in processing the stat results which is causing this or something
<enebo>
running 1.7.19
<enebo>
I see it rising but it maybe needs to go a while to tell for sure
camlow325 has joined #jruby
tcrawley is now known as tcrawley-away
shellac has joined #jruby
<nirvdrum>
Well, caching a WString would keep the entire Magazine it's in live.
<enebo>
nirvdrum: but I am only caching 1 of them
<enebo>
although it might be that the nativeconveerter is getting called over and over and that is what is in the magazine
<enebo>
so caching the wstring is probably not the important bit
<enebo>
ok so 1.7.19 is stable
<enebo>
maybe I can replace the wstring with a byte[] and see if that part of jffi works
<enebo>
So my current theory is that WString converts to native every call and those native bytes are stuffed into magazines and never freed
<enebo>
I was not originally caching and it was still happening so caching is not a cause but my messing around to see if I could get the leak to stop
<nirvdrum>
And you are calling FindClose
<nirvdrum>
I was hoping that might be it.
<enebo>
nirvdrum: well only for one code path
<enebo>
the other path does not retrun a handle (HAHA I hope)
<enebo>
but fwiw that could not explain this
<enebo>
1.7.20 was still using windows compat wstat
<enebo>
my changes were really recent
<enebo>
wstring is common to both though
dinfuehr has quit [Remote host closed the connection]
<nirvdrum>
Well, you could just allocateDirect in a loop and what happens.
<enebo>
I can do lots of things but I will try this quick…the byte[] bindings are working
<enebo>
If this does work then tons of crap is leaking
<enebo>
nirvdrum: but also if this works then I do not understand macos
<enebo>
nirvdrum: I cannot see a leak
<enebo>
hmmm we got a winner I think
<enebo>
yep stable
<enebo>
well double fudge sundae
<enebo>
nirvdrum: so all this native converter stuff is broken
<enebo>
nirvdrum: passing a direct byte[] is fine no doubt because it does not need to convert to native
<enebo>
nirvdrum: which makes me wonder why we don’t do this as a consistent strategy
<enebo>
heh so we don’t see this on Macos because we have no wrapped type like wstring
<enebo>
but someone sees a problem with some jnr-posix api on ubuntu so likely we have some API which is using this and leaking
<nirvdrum>
byte[] can be tricky if you need to pass the memory to a native call.
<enebo>
nirvdrum: why?
mdedetrich has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<enebo>
GC’ing?
<nirvdrum>
How would a Win32 call see into the JVM heap?
<nirvdrum>
I think JFFI will convert to a native buffer somewhere, however.
<nirvdrum>
It's how some of these posix calls work with Java types (e.g., CharSequence).
<enebo>
nirvdrum: ok well perhaps it does convert deep down but it works
<enebo>
nirvdrum: so I don’t know if it is just using a different mechanism or not but I am not seeing leaking
<enebo>
nirvdrum: with all this said I have no confidence in this library as it stands now
mkristian has joined #jruby
<enebo>
nirvdrum: I can fix the Windows APIs to use byte[] to at least make those work for a quicker release but TransientNativeMemory is some scary stuff :)
<enebo>
nirvdrum: I fucked with my own finalizer for Racob for like a week and never completely nailed it
<nirvdrum>
Yeah. And I believe the whole idea behind the magazines is to prevent excessive allocations and memory fragmentation.
<nirvdrum>
What's Racob?
dinfuehr has joined #jruby
mattwildig has joined #jruby
<enebo>
nirvdrum: I make a win32ole package for jruby-wni32ole
<nirvdrum>
Ahh.
<lopex>
nirvdrum: phantom reference is something that always returns null with get, but you can assosiate another object with collected one (to get a mapping without strongly/or even weakly pointing to old one)
<lopex>
the collected mappings/references are accessible via reference queue
dinfuehr has quit [Remote host closed the connection]
<nirvdrum>
I think this sentinel is supposed to be the phantom reference referent.
<lopex>
not sure if that helps but that's my understannding of it
<lopex>
it's just for the mapping
<lopex>
my data -> data that's already been collected
<lopex>
at least cannot be reachable now
mdedetrich has joined #jruby
rsim has quit [Quit: Leaving.]
<lopex>
nirvdrum: also with ref queue you can list objects that are already collected - though I dont know how deterministic is that
<lopex>
afaik I had issues with System.exit
shellac has quit [Quit: Computer has gone to sleep.]
drbobbeaty has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<enebo>
this is somewhat depressing
<lopex>
numbers ?
shellac has joined #jruby
bb010g has quit [Quit: Connection closed for inactivity]
<nirvdrum>
enebo: On the other hand, if it is all TransientNativeMemory, we could potentially reduce JRuby's footprint substantially without a huge amount of effort.
<enebo>
nirvdrum: meaning we should fix this so that the goal of this actually works
<nirvdrum>
Yes.
<enebo>
nirvdrum: it nearly works except for the never freeing pages part :P
<enebo>
nirvdrum: it doesn’t fragment it just never forgets
<nirvdrum>
enebo: I'm testing locally in Linux and it's not cleaning up until the app exits.
rsim1 has joined #jruby
<nirvdrum>
So it at least cleans up on clean app exit.
<enebo>
nirvdrum: what are you using to see it grow on linux?
<enebo>
stat?
<nirvdrum>
I've just set a breakpoint where the pages are freed.
rsim1 has quit [Client Quit]
<codefinger>
chrisseaton: congrats!
mdedetrich has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<nirvdrum>
But that might not be fair if it's async anyway.
<enebo>
this is not simple to follow :)
<enebo>
so hard reference of all magazines and things use memoty from it
temporal_ has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
rsim has joined #jruby
<enebo>
how does it get put into the finalizer reference queue?
<enebo>
Or doesn’t it :)
<nirvdrum>
The magazine does it in its constructor.
<enebo>
but I see no logic to check whether it is really clear
<nirvdrum>
If what's clear?
<enebo>
none of the memory is used anymore
<enebo>
unused might be better word than clear
<nirvdrum>
JFFI's finalizer iterates over the reference queue and calls Magazine#finalizeReferent
<enebo>
yeah I get that
<nirvdrum>
Otherwise it should look to clear data in the TransientNativeMemory constructor. I don't think that's working.
<nirvdrum>
And there's a background thread that should try to clear things. I don't think that's working either.
<nirvdrum>
My Magazine#get was wrong. But I think the Sentinel is still problematic.
<nirvdrum>
Although maybe that's fine, too. This code is gnarly.
<enebo>
hah
<enebo>
to be fair this shit is really hard to do right
<nirvdrum>
Indeed.
<nirvdrum>
I still would like an explicit way to free.
mdedetrich has joined #jruby
<enebo>
yeah perhaps just needs silmulated pointer references which do not actually point at the object
<enebo>
then a memorymanage.free(fake_pointer) will go and remove it from the magazine
<enebo>
unless it has already been removed in which case nothing happens
shellac has quit [Quit: Computer has gone to sleep.]
rsim has quit [Quit: Leaving.]
cristianrasch has quit [Quit: Leaving]
rsim has joined #jruby
skade has joined #jruby
<enebo>
nirvdrum: ok so transient native memort saves the magazines sentinel reference and when all of them are gone the seintial weakref should return null which means it should be ready to wipe out the magazine
<enebo>
nirvdrum: right? I see nothing which is checking that weakref past allocating new memory in current magazine
<enebo>
nirvdrum: seems like some reaper should be rechecking older magazines for this
shellac has joined #jruby
<nirvdrum>
I think the phantom reference system in the JVM is supposed to manage it.
<nirvdrum>
When the sentinel is phantom referenceable, it should enqueue the magazine on the reference queue for clean up.
<enebo>
ah yeah
<enebo>
since this ends up in a reference queue
skade has quit [Ping timeout: 250 seconds]
mkristian has quit [Quit: This computer has gone to sleep]
<enebo>
nirvdrum: ok it is slowly gettting in more focus for me. WeakRef + phantom on sentinal makes me confused though…the phantom will be bound by weak life cycl
<enebo>
This makes me think this is why there is that sentinal field which is unused
<enebo>
current sentinal must have been a hard ref until next magazine was made
<enebo>
which mean current magazine could never ever get cleared but all others would just be phantom references
<enebo>
oh but it was final :)
<nirvdrum>
enebo: I think the idea is the unused sentinel is just a reference is the TransientNativeMemory object.
<nirvdrum>
So when you cease referencing the allocated memory, the sentinel is no longer referneced.
rsim has quit [Quit: Leaving.]
<enebo>
yeah
<nirvdrum>
The weak ref is just to make sure the magazine doesn't keep it hot.
<nirvdrum>
And the magazine keeps track because you can have multiple TransientNativeMemory objects per magazine, each of which would need a reference to the sentinel.
<enebo>
yeah I can see that sentinal is how this actually knows to clear but I am confused about the weak part since it will not probably clear until low on heap memory
shellac has quit [Quit: Computer has gone to sleep.]
<lopex>
enebo: you mean soft ?
<enebo>
lopex: oh I do…crud I need to refresh my memory on this :)
<lopex>
soft works like you described
<lopex>
enebo: but I'm not sure what you mean by weak cycle for phantoms
<lopex>
so just like weak but then edn up in ref queue ?
<lopex>
*end up
<enebo>
lopex: the same object is added as referrent for a phantom reference and then immediately also made weak
<enebo>
lopex: so I guess the weak gets cleared when there are no more references and that them ends up making the phantom marking things for processing in its queue
rsim has joined #jruby
<nirvdrum>
enebo: Well, shit. I think this is actually working.
<enebo>
nirvdrum: yeah after my last sentence I think it makes sense to me
<enebo>
nirvdrum: finally :)
<enebo>
nirvdrum: but then this means something else must be holding a reference
<nirvdrum>
I forgot in IDEA that you need to change the breakpoint type to Suspend -> Thread for it to guarantee to hit the breakpoint when multiple threads are running.
<nirvdrum>
And this cleanup task is in a background thread.
<enebo>
nirvdrum: so the weak gets cleared which unblocks the phantom from cleaning up the magazine
<enebo>
nirvdrum: but if things are not really getting cleared it must be some hard references to the memoty itself
<enebo>
err transient memory
<nirvdrum>
Right.
<nirvdrum>
And while this may be obvious, it only takes one such reference to keep the whole magazine alive.
<enebo>
heh
<enebo>
to ToNativeConverter in WString is a key
<enebo>
it looks totally fine
<enebo>
but however this converter gets called must go through something holding a reference
shellac has joined #jruby
<enebo>
oh god this is indirect :)
<enebo>
allocateDirect in MemoryManager somehow makes it to this allocate in TransientNativeMemory so it could hold a reference
<enebo>
I can see BoundedMmoery holding one but I would nope nothing it holding it past when it is used
skade has joined #jruby
jensnockert has quit [Remote host closed the connection]
rsim has quit [Read error: Connection reset by peer]
<enebo>
nirvdrum: you add ToNativeConverterMarchaller.marshal()?
rsim has joined #jruby
<enebo>
add/at
<nirvdrum>
I don't think I have any commits in jnr-ffi.
<nirvdrum>
So I don't believe so.
<enebo>
nirvdrum: sorry I meant have you gotten to that point
<enebo>
nirvdrum: not did you write it
<enebo>
I think this is called in setting up the call and it will hold a reference to boundedmemoryio which contains the transient native memory
<nirvdrum>
I'm not on Windows at the moment, so I'm trying to reproduce with Linux stat().
<enebo>
yeah you probably will not get it to work that way
<enebo>
nirvdrum: I think structs arenot being alloc’d in this
<enebo>
err by this
<nirvdrum>
Yeah. But I can change that locally.
<enebo>
my struct usage on windows was not causing the leak
<enebo>
it was the WString
<enebo>
which is callong convertToNative and I think something in the gears here must be holding the reference
* subbu
is wondering why enebo is still around today ..
<enebo>
heh
<nirvdrum>
Yeah. Linux stat() does a convertToNative, but it doesn't use allocateDirect.
shellac has quit [Quit: Computer has gone to sleep.]
<nirvdrum>
Changing to allocateDirect, however, does clean up.