<mihaid>
Hello, I am working on the issue of sendmsg and recvmsg on branch py3.5 (issue/2527), and I have a problem with rffi. The problem is that I have a complex structure (created by me in C code) that recvmsg needs, but I get a compilation error saying that rffi cannot calculate its size: /tmp/usession-py3.5-71/platcheck_60.c:4575:23: error: invalid app
<mihaid>
lication of ‘sizeof’ to incomplete type ‘platcheck_t {aka struct recvmsg_info}. I am not sure how to fix this, because other structures that I have seen in _rsocket_rffi.py don't do anything different from what I have tried.
<LarstiQ>
what does the 'incomplete type' mean?
<LarstiQ>
seems key to me
<mihaid>
I am not sure, as well. Here is the structure as I declared it in the C code: https://pastebin.com/8Hz5b1sE
arigato has quit [Ping timeout: 268 seconds]
<LarstiQ>
mihaid: does it not know some of the types used in that definition?
<LarstiQ>
mihaid: say socklen_t
<mihaid>
@LarstiQ it should because the C code includes sys/socket.h and _rsocket_rffi has the types socklen_t already specified. Btw here is how the structure is ported to rffi: https://pastebin.com/DGzH3Hc7
<LarstiQ>
mihaid: is rffi including the right headers?
gbutnaru has quit [Ping timeout: 260 seconds]
asmeurer__ has quit [Quit: asmeurer__]
<mihaid>
@LarstiQ Here are the includes: https://pastebin.com/NWdQzpUt Apart from sys/socket.h, I don't think there are any types I don't include and are needed (I also compiled the C code myself first).
<LarstiQ>
mihaid: have you inspected /tmp/usession-py3.5-71/platcheck_60.c?
<LarstiQ>
mihaid: afaics it's the C compiler that is complaining, so the first step is to figure out what it thinks is incomplete
<LarstiQ>
once you have that you can dig for if it's a bug in rffi or a usage error
<njs>
mihaid: sticking a sockaddr into a struct might make sense somehow, but it's pretty funny looking -- traditionally "sockaddr_t" is a abstract placeholder type whose size can vary depending on the instance
<njs>
oh, I see, in the C code it's a sockaddr*, which makes sense. Is CConfig.sockaddr also a pointer, or does it indicate that the object is supposed to be inline there?
rubdos has quit [Ping timeout: 246 seconds]
<mihaid>
@njs the sockaddr* is supposed to take the address returned by recvmsg, which will be 1 object. I also tried changing from CConfig.sockaddr to a sockaddr_ptr, but I have the same issue.
<njs>
so I'm pretty sure dropping CConfig.sockaddr there is not going to work, and something like a "incomplete type" error would be a reasonable outcome
<njs>
maybe you have another issue as well, I dunno
rubdos has joined #pypy
marr has quit [Ping timeout: 240 seconds]
marr has joined #pypy
cstratak has quit [Ping timeout: 260 seconds]
oberstet has quit [Ping timeout: 276 seconds]
adamholmberg has joined #pypy
oberstet has joined #pypy
antocuni has quit [Ping timeout: 260 seconds]
adamholmberg has quit [Ping timeout: 260 seconds]
nimaje1 has joined #pypy
nimaje1 is now known as nimaje
nimaje has quit [Killed (wilhelm.freenode.net (Nickname regained by services))]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Remote host closed the connection]
<arigo>
fwiw, "struct recvmsg_info" does not appear in CPython 3.5 source code, so I'm not sure what this struct is supposed to be
<arigo>
unsure why you need to create this structure in C code, but assuming you do, then the definition of this structure needs to be available from the platcheck_60.c too
<arigo>
(but note that I don't understand why you have this struct, because I'm not finding anything similar in CPython's socketmodule.c)
oberstet has quit [Remote host closed the connection]
oberstet has joined #pypy
slacky has quit [Ping timeout: 276 seconds]
<mihaid>
@arigato I created it to keep thing structured in the C code. What recvmsg returns by default is a even more complex structure (struct msghdr), that would be really hard to make sense of in _rsocket_rffi. From what I have gathered up to now (I might be wrong) the compiler is not aware of the structure I have created because it does not belong in ec
<mihaid>
i ( the external compilation info that existed originally in _rsocket_rffi) but rather to another eci that I have created.
cstratak has joined #pypy
marky1991 has joined #pypy
<mihaid>
@arigo Basically, this would work, I think, if I would be able include the structure in my own ".h" Either that or, I can use this structure only in the C code and simply use the parameters of my implementation of recvmsg() as pointers to the members of the structure. Sorry if this sounds too weird or complicated but I am still getting the hang of
<mihaid>
rffi and RPython -> C communication.
<arigo>
as far as "man recvmsg" says, "struct msghdr" is much less complicated than your structure
<arigo>
anyway, what you need is "ExternalCompilationInfo(post_include_bits=["""..."""])"
<arigo>
this just dumps whatever text into the generated C file, after the #include
<arigo>
as for the C implementation, I have no idea because I don't know recvmsg(), but I'd try to follow CPython's implementation
adamholmberg has joined #pypy
<arigo>
either that, or simply write it as pure-python code using cffi
<arigo>
see module/posix/app_startfile.py for an example of how to use pure-python code based on cffi in order to write a specific function in a "built-in" module
<arigo>
(of course the first step is to implement it as a regular cffi interface, which you can test as usual, and only then you'd move it to the style of module/posix/app_startfile.py)
<arigo>
ah, no, sorry
<arigo>
this is a bad example because it's using ABI, and in this case you need API-level cffi
<arigo>
then I guess you should make a new module with an internal name like lib_pypy/_pypy_socket_internal.py
<arigo>
with cffi in API mode, with lib_pypy/_pypy_socket_build.py or something
adamholmberg has quit [Ping timeout: 240 seconds]
<arigo>
(see lib_pypy/_syslog_build.py for example)
<arigo>
and then when it's done you only need to import and call that module from the built-in methods socket.recvmsg()
<mihaid>
@arigo What I tried with sendmsg (and worked) up to was to write it in _rsocket_rffi, in C, as separate source_module. It does not need complex structures, so I tested it (with a CPython 3.7 server) and it works smoothly. The only issue I encountered with rffi was this structure, but I think I can solve it (if you agree, ofc).
<arigo>
yes, sure, if you can make it work and be somehow portable, that's all we really need
<mihaid>
well, sendmsg and recvmsg are POSIX only, so linux and Mac should be good.
<arigo>
I bet Linux and Mac are quite different
<arigo>
but yes, if both of these work that's enough
<mihaid>
I can also run the tests on Mac afterwards. But I am trying to follow the CPython implementation as closely as possible, so this should work.
<arigo>
good
* arigo
lunch
arigo has quit [Ping timeout: 240 seconds]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 240 seconds]
rokujyouhitoma has quit [Ping timeout: 260 seconds]
cstratak has quit [Ping timeout: 268 seconds]
ELFrederich has joined #pypy
arigato has quit [Read error: Connection reset by peer]
<ELFrederich>
arigato: yesterday you were telling me to only have a single ffibuilder = FFI() and only list a single cffi_module=['SomePackage/bindings_build.py:ffibuilder'] ... what should I do then from that bindings_build.py?... import the 120 modules and pass the ffibuilder into functions within those modules which call cdef?
adamholmberg has quit [Remote host closed the connection]
TheAdversary has quit [Read error: Connection reset by peer]
TheAdversary has joined #pypy
pilne has joined #pypy
commandoline has quit [Quit: Bye!]
oberstet has quit [Remote host closed the connection]
commandoline has joined #pypy
commandoline has quit [Client Quit]
antocuni has quit [Ping timeout: 260 seconds]
adamholmberg has joined #pypy
commandoline has joined #pypy
rokujyouhitoma has joined #pypy
<nanonyme>
ELFrederich, I think pywincffi (as probably many others) handle this by only having a single file call cdefs but instead keep the definitions in separate header files on disk
<nanonyme>
Then import the final built binding into all modules that need it
rokujyouhitoma has quit [Ping timeout: 260 seconds]
arigato has joined #pypy
Rhy0lite has quit [Quit: Leaving]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
rokujyouhitoma has joined #pypy
tbodt has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
lritter has joined #pypy
inhahe_ has quit [Read error: Connection reset by peer]
inhahe_ has joined #pypy
mattip has joined #pypy
<Alex_Gaynor>
arigato: Do you know offhand if there is an x86-64 instructino "XORQ $imm, %reg"?
<arigato>
yes, for general-purpose registers
<arigato>
and only as long as $imm fits in a signed 32-bit
rokujyouhitoma has joined #pypy
<Alex_Gaynor>
arigato: Is there a way to do it that doesn't sign-extend the result?
marr has quit [Ping timeout: 240 seconds]
rokujyouhitoma has quit [Ping timeout: 255 seconds]
<arigato>
the 32-bit limit of immediates exists in almost all instructions, except "movabs $imm, %reg", so the workaround is to load the large constant in a register first with movabs
<arigato>
(movabs == mov, I'm not sure why it's named differently)
raynold has joined #pypy
ELFrederich has quit [Quit: Page closed]
arigato has quit [Quit: Leaving]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 240 seconds]
<kenaan>
rlamy default 36eb150b3c67 /pypy/module/cpyext/: Change signature of PyXXX_Check pseudo-macros from int(PyObject*) to int(void*) (fixes #2596)
Fleche has quit [Ping timeout: 260 seconds]
<mattip>
ronan: about 36eb150b3c67 - nice, thanks
<ronan>
mattip: well, it turned out to be trivial
<mattip>
ronan: yes, the best kind of fix
<ronan>
mattip: fun fact, I had forgotten about the magic void* -> W_Root conversion, so I was quite surprised that it worked
<mattip>
:)
rokujyouhitoma has joined #pypy
mat^2 has joined #pypy
rokujyouhitoma has quit [Ping timeout: 255 seconds]
lritter has quit [Remote host closed the connection]
adamholmberg has quit [Remote host closed the connection]