18:59 <headius[m]> enebo: does loading a file keep the file open?

19:00 <headius[m]> I am getting specs green on Windows and most failures in :language are due to it being unable to delete temp files it loaded

19:02 <enebo[m]> hmm

19:03 <enebo[m]> I believe there is some reason it could happen for DATA perhaps but we do fully read source into memory generally

19:03 <headius[m]> it seems like ExternalScript.load tries to close the stream but when it tries to delete the file it fails

19:03 <headius[m]> when the spec tries to delete it I mean

19:03 <enebo[m]> I really don't recall anything other than DATA but we should be able to detect that

19:04 <headius[m]> that should only be for the main script though

19:04 <enebo[m]> The opening and the parsing are not all that close together either

19:04 <enebo[m]> yeah so nothing is popping out

19:04 <headius[m]> ok

19:04 <enebo[m]> windows is an excellent platform for noticing this behavior :)

19:04 <headius[m]> that's for sure

19:05 <enebo[m]> but fwiw if we did this on a large rails app and it was like this we would have thousands of open fds

19:05 <headius[m]> yeah I am trying to determine what files remain open

19:05 <headius[m]> this basically writes a file and then `load`s it and then tries to delete it in the teardown

19:05 <enebo[m]> I guess modern unixy systems probably do have higher resource limits so perhaps we are not hitting any ceilings

19:08 <headius[m]> I'll poke around

19:10 <enebo[m]> It does not on linux

19:10 <enebo[m]> jruby -e 'File.open(%q{pom.xml}); gets'

19:10 <headius[m]> that's essentially what these specs are doing

19:11 <enebo[m]> just run lsof -p {pid} and you can see the pom.xml and whatever else java has open file wise

19:11 <enebo[m]> So this appears to be windows only if it is an issue

19:12 <enebo[m]> I guess since those specs exist we knew that but I am doubly sure now :)

19:12 <headius[m]> yeah I need to figure out how to do lsof on Windows

19:12 <enebo[m]> windows io is not native so likely we are not closing within windows code path somewhere

19:12 <enebo[m]> That looks like a duck :)

19:13 <enebo[m]> It quacks like it too

19:13 <enebo[m]> sysinternals had a tool for open files

19:13 <enebo[m]> a million years ago

19:13 <enebo[m]> but it does look like you know it is not closing the file already

19:14 <headius[m]> it certainly seems that way

19:14 <headius[m]> but we do appear to try to load fully any external script

19:14 <headius[m]> so I'm looking at the load logic to see if somewhere here we accidentally leave the file open

19:16 <enebo[m]> I did just verify load specifically as well using lsof

19:16 <enebo[m]> linux is fine

19:16 <headius[m]> ok that's good to know

19:18 <headius[m]> I think I found it

19:18 <headius[m]> LibrarySearcher.ResourceLibrary

19:18 <headius[m]> its load impl calls "prepareInputStream" which fully buffers the file but does not close the real stream

19:19 <headius[m]> this is via FoundLibrary's delegate.load... different on Windows?

19:19 <headius[m]> it's not an ExternalScript at this point, it's a FoundLibrary wrapping a LibrarySearcher.ResourceLibrary

19:21 <headius[m]> starting from here that is: https://github.com/jruby/jruby/blob/6c0f3370f5b550cfd857483e4c82fefff8daf27f/core/src/main/java/org/jruby/runtime/load/LoadService.java#L343

19:22 <headius[m]> works and passes now

19:23 <enebo[m]> nice

19:27 <headius[m]> enebo: https://github.com/jruby/jruby/pull/6357

19:28 <headius[m]> the remaining language failures appear to be a default codepage issue... we do IBM-437 instead of UTF-8 for some reason

19:28 <headius[m]> but good news is that all other language stuff seems ok

20:10 <enebo[m]> headius: it seems ok to me but just for the sake of argument...if we thought original file resource should not go away (although how it is written is amazingly non-determinstic) why would we want that?

20:10 <headius[m]> only 39F and 48E on :core specs

20:10 <headius[m]> why would we want what?

20:10 <enebo[m]> It was so long ago that specs have run that is pretty amazing

20:10 <enebo[m]> original is to stay around until finalized

20:11 <headius[m]> yeah I can't think of any reason since we buffer the whole file before proceeding

20:11 <enebo[m]> I am mostly asking because you made it a draft pr

20:11 <headius[m]> I think it was just overlooked

20:11 <headius[m]> it's just draft because I will likely have other fixes for windows specs

20:12 <enebo[m]> ok...seems well contained as its own in case we discover something later (we won't though)

20:12 <headius[m]> yeah I did not check lineage of this code but I think someone may have thought the is would be closed by the wrapping stream, and it isn't

20:13 <headius[m]> the javadoc for LoadServiceResourceInputStream even says you need to handle closing it yourself

20:14 <enebo[m]> that is a perennial bug of stream I guess too

20:14 <enebo[m]> and this was a general issue and not windows specific

20:14 <enebo[m]> lsof maybe was run after finalization happened to close it

22:00 <headius[m]> well that is an open question I guess

22:00 <headius[m]> I would not have expected this to follow different logic on Windows