thegeekinside has quit [Ping timeout: 272 seconds]
MarcinMielyskiGi has quit [Remote host closed the connection]
GitterIntegrati4 has quit [Read error: Connection reset by peer]
just_another_use has quit [Remote host closed the connection]
BlaneDabneyGitte has quit [Remote host closed the connection]
JesseChavezGitte has quit [Remote host closed the connection]
JulesIvanicGitte has quit [Read error: Connection reset by peer]
KarolBucekGitter has quit [Read error: Connection reset by peer]
kares[m] has quit [Remote host closed the connection]
XavierNoriaGitte has quit [Remote host closed the connection]
lopex[m] has quit [Remote host closed the connection]
liamwhiteGitter[ has quit [Remote host closed the connection]
sillymob[m] has quit [Remote host closed the connection]
CharlesOliverNut has quit [Remote host closed the connection]
TimGitter[m] has quit [Remote host closed the connection]
ChrisSeatonGitte has quit [Remote host closed the connection]
headius[m] has quit [Remote host closed the connection]
RomainManni-Buca has quit [Remote host closed the connection]
MattPattersonGit has quit [Remote host closed the connection]
rdubya[m] has quit [Remote host closed the connection]
ThomasEEneboGitt has quit [Remote host closed the connection]
enebo[m] has quit [Remote host closed the connection]
FlorianDoubletGi has quit [Remote host closed the connection]
cheald[m] has quit [Remote host closed the connection]
jayjlawrence[m] has quit [Remote host closed the connection]
TimGitter[m]1 has quit [Remote host closed the connection]
OlleJonssonGitte has quit [Remote host closed the connection]
GitterIntegratio has joined #jruby
rusk has joined #jruby
lopex[m] has joined #jruby
enebo[m] has joined #jruby
headius[m] has joined #jruby
MarcinMielyskiGi has joined #jruby
liamwhiteGitter[ has joined #jruby
CharlesOliverNut has joined #jruby
KarolBucekGitter has joined #jruby
TimGitter[m] has joined #jruby
rdubya[m] has joined #jruby
ChrisSeatonGitte has joined #jruby
FlorianDoubletGi has joined #jruby
kares[m] has joined #jruby
XavierNoriaGitte has joined #jruby
BlaneDabneyGitte has joined #jruby
JesseChavezGitte has joined #jruby
jayjlawrence[m] has joined #jruby
cheald[m] has joined #jruby
just_another_use has joined #jruby
OlleJonssonGitte has joined #jruby
JulesIvanicGitte has joined #jruby
MattPattersonGit has joined #jruby
RomainManni-Buca has joined #jruby
sillymob[m] has joined #jruby
TimGitter[m]1 has joined #jruby
ThomasEEneboGitt has joined #jruby
shellac has joined #jruby
shellac has quit [Ping timeout: 246 seconds]
_whitelogger has joined #jruby
jasmin has joined #jruby
shellac has joined #jruby
jasmin has quit [Quit: Leaving]
joast has quit [Quit: Leaving.]
subbu is now known as subbu|away
joast has joined #jruby
subbu|away is now known as subbu
xardion has quit [Remote host closed the connection]
xardion has joined #jruby
shellac has quit [Ping timeout: 268 seconds]
_whitelogger has joined #jruby
<byteit101>
Hmm, tricky thing about _open_osfhandle is that it creates a second handle to the file that needs to be closed concurrently with the channel
<byteit101>
headius[m]: Got IO::sysopen returning the correct values on windows, though I still have some issues to verify. Is FilenoUtils a reasonable place for ffi libraries?
<byteit101>
It seems non-ideal to me as I have to route the file flags into ChannelFD
<headius[m]>
byteit101: I'm wondering if maybe we should yank some IO stuff out of jnr-posix and jnr-enxio into a new jnr-posix-io or something
<headius[m]>
Wherever you want to put it for now is fine but we've tried to avoid maintaining separate FFI bindings within JRuby itself if they would make sense in a general library, like jnr-posix
<headius[m]>
looking at your commit now
<headius[m]>
ok I see
<headius[m]>
Questions...
<headius[m]>
Why aggregate openFlags in ChannelFD? It seems to only be used during initialization of the fileno
<headius[m]>
I think it could just be passed through into the fileno logic and not stored
<byteit101>
lazyness. I may be wrong, but I don't think it's a good idea to have two open handles to each file on windows open at all times so I attempted to only create the posixy fd if actually requested,
<byteit101>
hence the overload of getBestFileno(bool)
<headius[m]>
ok so let me understand this...
<headius[m]>
the channel on Windows will have a HANDLE of course...and to get a posixy fd from that you have to call _open_osfhandle, and then both need to be closed when you shut down the IO
<byteit101>
yes
<byteit101>
*if you control the source code, you can not close the HANDLE
<byteit101>
as close(fd) closes the HANDLE
<byteit101>
but java is dealing with that for us
<byteit101>
so we can't avoid close(HANDLE)
<byteit101>
which closes the handle, but leaves the fd open
<headius[m]>
hmm
<byteit101>
fd is invalid as I can see, but takes up resources (2048 max on my win10 machine)
<headius[m]>
btw I looked into how JDK does "select" on Windows...they use the not-so-efficient select that windows provides, rather than WaitForSingleObject or similar
<headius[m]>
fdsets and whatnot
<headius[m]>
that article I linked the other day (old, admittedly) claimed that the windows select is O(n) on the number of descriptors, making it unsuitable for high-concurrency selects
<headius[m]>
what do you mean by "java is dealing with that for us"
<byteit101>
RandomAccessFile.close -> closes the HANDLE
<headius[m]>
the Java channel has a handle, and if we close the fd but not the handle will that handle close or not?
<byteit101>
Oh, hmm
<enebo[m]>
I know enxio is beyond jruby itself but how does MRI do this?
<enebo[m]>
Maybe only using a HANDLE makes more sense
<headius[m]>
so enebo this is basically dealing with Java-managed io HANDLE
<headius[m]>
the ones we get out of open(2) and such will "just work" as enxio (with byteit101 patches)
<enebo[m]>
ah so some semblance of Java APIs can still work with it
<headius[m]>
I think I have that right
<headius[m]>
cracking these with reflection is problematic but if we get the native side working well I think more can move that way
<enebo[m]>
and that open call is not some wacky #define'd thing which is not doing something radically different on windows
<headius[m]>
if I remember right it the msvcrt has some ref counting table of handles so closing the open(2) fd will close the handle
<byteit101>
Yes, this _open_osfhandle is only our problem when using io::sysopen/java channels. if someone passes an already opened posixy fd to IO::for_fd, my enxio patches require no other jruby support
<byteit101>
yes, the docs claim that and I verified that close(fd) invalidates the handle
<headius[m]>
in a way it's almost like our pseudo-fd mappings to channels
<headius[m]>
but when we have a HANDLE we need to do some extra work to properly manage the fd from it I guess
<byteit101>
"To close a file opened by using _open_osfhandle, call _close. The underlying OS file handle is also closed by a call to _close. "
<headius[m]>
honestly I think everyone would be ok with sysopen only working with native files
<headius[m]>
so sysopen has this issue because we need to get an actual fileno right away
<headius[m]>
the mix of native channels and Java channels could be managed better I guess
<headius[m]>
but why can't sysopen just use native API now?
<headius[m]>
it's just a file so select isn't important
<byteit101>
My use case is serial port configuration
<headius[m]>
and open(2) can't open COM1 or whatever
<headius[m]>
is that it?
<byteit101>
File::open("COM5") works with the first set of my patches, and requires no support, but the issue is configuring it
<byteit101>
the configuration apis are all Win
<byteit101>
32 as far as I can see
<byteit101>
and require a handle
<headius[m]>
ahh I see
<headius[m]>
so you would need to get the handle for this anyway
<headius[m]>
which you can't do from a fd
<byteit101>
File#fileno returns a fake fd that can't be munged into a handle easily on MRI + JRUBY
<byteit101>
handle _get_osfhandle(int posixyfd);
<headius[m]>
ok
<byteit101>
MRI returns a real FD that can be configured using _get_osfhandle
<byteit101>
then I can correctly configure it
<byteit101>
and since the fd owns the handle, everything works
<headius[m]>
Ok but your File::open("COM5") should return a proper native channel right?
<headius[m]>
with a normal open(2) call?
subbu|away is now known as subbu
<byteit101>
Err, define proper native channel
<byteit101>
The FD has a handle and not a fd field, so it's a win32 open
<headius[m]>
I guess I'm trying to understand why we need to manage the handle if open(2) works and manages it for us most of the time
<headius[m]>
and you have a way to get the handle from it if necessary for serial port config etc
<headius[m]>
ok hang on
<headius[m]>
FD there means FileDescriptor class I assume
<byteit101>
yes
<headius[m]>
Does your File::open on Windows does not use enxio Native channels?
<byteit101>
it does not
<byteit101>
it uses RandomAccessFile
<headius[m]>
aha
<headius[m]>
Could it?
<headius[m]>
usually when the fileno is important as in this case we just use the enxio path and get native channels out
<headius[m]>
then you'd have a proper fileno from File and could get the handle and do whatever
<byteit101>
That could happen, but you'd probably need to flush out my jnr-enxio change to support async io.
<byteit101>
I was trying to be minimally invasive and maximally lazy :-D
<headius[m]>
so if we could get enxio channels fully working on Windows, with Selector support, that would be good
<headius[m]>
byteit101: fwiw the logic you have here needs to exist anyway because we always try to support non-native as much as possible
<byteit101>
good to know
<headius[m]>
looking at MRI now
<headius[m]>
so if you do this on MRI you'd basically just get the fileno and then use ffi to access handle etc
<headius[m]>
the gymnastics in this recent patch are to support getting fileno out of a java channel that only has handle...ok
<headius[m]>
I am not clear on the reverse relationship from HANDLE to fd... the extra logic to close the fd is necessary because it would leak otherwise?
<headius[m]>
i.e. the relationship only goes from fd => HANDLE
<headius[m]>
yeah starting to come together in my head
<byteit101>
Yes, and then after 2047 of them it starts failing
<headius[m]>
and since we don't necessarily need the fd for all Java-based Windows IO, you have it set up to do that _open_osfhandle lazily and then set a flag to close it
<byteit101>
I tried to avoid that, but when testing I came to that limit
<byteit101>
and yup, that was my goal with that code
<headius[m]>
got it
<headius[m]>
ok
<headius[m]>
I think this is generally correct then
<headius[m]>
I might make the windows-specific stuff be a subclass of ChannelFD though
<byteit101>
that seems reasonable.
<byteit101>
I did verify that after 10,000 open/close cycles, it didn't leak anything obvous and that it still correctly read/wrote data, and that it reused the same fd number. If you have an arduino, I can paste my verification code
<headius[m]>
I trust that you have it working 🙂 I do have some devices around here somewhere though
<headius[m]>
so we can get this and the other patches in for initial support for some native stuff on Windows, and then try to decide how to do async IO well
<headius[m]>
if we must do async IO via WaitFor then we're going to have to deal with this fd+handle duality within enxio
<headius[m]>
it may be possible to use their select but it's all macros for the fdset management
<byteit101>
Cool, yea, that's going to be nasty
<headius[m]>
ok I'll start merging some of this stuff in
<headius[m]>
no reason we can't expand native features for 9.2.8 without going all the way