cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<vhle>
When this function is called, destructor of A will be called automatically.
<vhle>
In Python:
<vhle>
To release A object in Python, I call: mylib_cffi.lib.releaseA(a_handle);
<vhle>
The issue is that sometimes the destructor will run in mThread, which causes a deadlock.
<vhle>
I think this is related to the note in the documentation: " Moreover, the destructor is called in whatever thread PyPy is at that moment, which might be a problem for some C libraries".
<vhle>
Do you have any idea to work around this issue?
<vhle>
(I'm using standard CPython, does the note still apply?)
<antocuni>
vhle: I don't think it is related on the doc line you mention
<antocuni>
that doc is about "ffi.gc(cdata, destructor)", where destructor is a C function which will be called automatically when cdata is released. It is not related at all by C++ destructors
<antocuni>
is is possible that you are simply calling mylib_cffi.lib.releaseA(a_handle) from the mThread thread?
<antocuni>
I don't think that cffi changes the thread on which functions are called under your feet
<antocuni>
(but I might be wrong)
<antocuni>
ronan: ping
<ronan>
antocuni: pong
<antocuni>
I am reading the code in cpyext/cparser.py and I am a bit confused
<vhle>
@antocuni: I will check again to see if cffi changes the thread.
<antocuni>
is it used to parse real headers or to parse "dummy" headers that we use just to avoid writing lltype structs by hand?
<antocuni>
(or both)
<ronan>
antocuni: both
<antocuni>
why do we need to parse the real headers?
<ronan>
parsing the real headers ensures things stay in sync and avoids copy-pasting
<antocuni>
by "real headers" we mean the files in pypy/module/cpyext/include, right?
<ronan>
yes
<ronan>
well, cparser only parses the files in cpyext/parse/ but these are included from the real headers
<antocuni>
ah
<ronan>
TBH, I'm not sure the advantages are worth the complexity of the setup
<antocuni>
so e.g. cpyext/include/datetime.h is NEVER seen by cparser; only parse/cpyext_datetime.h is seen?
<ronan>
yes
<antocuni>
yes, it sounds like a good&cool idea, but I guess it can quickly become a nightmare
<antocuni>
and why did you use a copied&pasted version of cparser.py instead of using directly cffi? I suppose you had to modify something, but I couldn't understand exactly what by looking at the diff :)
<ronan>
I don't remember exactly, but I had to change a few things to support more of the Python headers, and also for rffi
<ronan>
so typedefs and #defines are handled a bit differently
altendky has joined #pypy
dddddd has joined #pypy
<antocuni>
wait, I am confused
<antocuni>
I thought we only parse the headers in cpyext/parse: why do you need to parse the Python headers?
<ronan>
All our headers are copied from CPython, if cparser supports more of it, we can consolidate most of them into cpyext/parse/
<antocuni>
ah, maybe I start to understand it (sorry being slow :))
<antocuni>
so, we started by copying cpython headers into cpyext/include
<antocuni>
then we moved parts of them into cpyext/parse, so that they can ALSO be parsed by CTypeSpace()
<antocuni>
is that correct?
<ronan>
yes
<ronan>
sorry that it wasn't clear
<antocuni>
no problem, I am just trying to understand what this code is supposed to do :)
<antocuni>
I see now why you tried to use it for hpy: indeed, it would be very nice and useful to do something like that. What is the blocker that you encountered when you tried?
<ronan>
there were a few bugs in cparser and the hpy API does things that rffi doesn't support
<antocuni>
like?
<antocuni>
va_list, probably?
<ronan>
yes, and returning a struct
<antocuni>
I think this is might be solvable by using the right typedefs. I.e., if we tell CTypeSpace something like "typedef long HPy;", then rffi will be happy
<ronan>
yes, that would be equivalent to what we're doing now with manual lltype/rffi code
<antocuni>
yes exactly, I was about to write that :)
<antocuni>
in doesn't even have to be the real header: it might be enough to have a header which defines structs&signatures which are "compatible enough" with the real ones
<antocuni>
e.g. we could have "struct _HPyContext_s" containing void* fields, instead of fields with the right function pointer signature
<ronan>
yes indeed
<antocuni>
and possibly generate this automatically by using autogen
<ronan>
yes, it should be regular enough for that
<antocuni>
one more thing which is not totally clear to me
<antocuni>
Foo is an instance of lltype.Struct; why isn't it a rffi.CStruct instead?
<antocuni>
ah no, never mind
<antocuni>
rffi.CStruct DOES return an lltype.Struct
<ronan>
yes
<antocuni>
where is the code which instantiate this struct?
<antocuni>
Foo, I mean
<ronan>
it's in CTypeSpace.configure_types()
<antocuni>
ah, I suppose that probably it is "realize_struct" that eventually does it
<ronan>
well, realize_struct() sets up the struct for being configured, but it's actually created at "self._config_entries[entry].become(TYPE)" in configure_types()
<antocuni>
yes, found it just now
<antocuni>
so basically, we are using cparser to play the role of "ffi.cdef()", and rffi_platform to play the role of ffi.set_source()/ffi.compile(), roughly speaking
<ronan>
yeah
<antocuni>
it might be worth polishing it and move it somewhere into rpython/; it might become the basis for a "rcffi" or something like that
<antocuni>
ideally, sending PR to cffi's cparser so that we can use it as a dependency instead of having our fork
<ronan>
yes, that was what I had in mind
<antocuni>
cool
<ronan>
but IIRC, it required some refactoring in cffi to make it usable in rpython
<ronan>
though maybe monkey-patching would work
<antocuni>
do you remember what was the problem?
<ronan>
not really, but it was related to why I had to use a fork in the first place
<ronan>
I guess we should try to reconverge with cffi to see where the actually required changes are
<antocuni>
yes
<antocuni>
on the other hand, it looks like a biggish project which might drive our attention away from hpy :(
<antocuni>
uh? I get a segfault during translation 😱
<antocuni>
more specifically, during the c database phase
<antocuni>
got it again
<antocuni>
what the hell could it cause a segfault during the database phase?
astronavt___ is now known as astronavt
jcea has quit [Remote host closed the connection]
<antocuni>
arigato: context fields such as h_None cause translation to fail: llapi.HPyContextS are declares them as HPy (i.e., Signed), but during C compilation _HPyContext_s is defined from autogen_ctx.h, where the type "HPy" is actually a struct
<antocuni>
I suppose that the proper fix should be do declare llapi.HPy correctly, but this causes lots of other problems, because I don't think we support passing struct values around in RPython
<antocuni>
another alternative would be to declare the fields in llapi.HPyContextS as "real_HPy_t" or something like that, but it would be quite obscure IMHO
<antocuni>
any brilliant idea?:)
<antocuni>
on an unrelated note, my segfault seems to be caused by f9c2cfcfaafa; no idea what could be, trying to rerun translation in gdb
<antocuni>
however, I am not 100% sure about it. In particular, I had to modify cparser to expose "struct HPyContext_s": without my additions, structs are converted to lltype.Struct only if they are part of a typedef
<antocuni>
moreover, cpyext now fails to import with this traceback:
<antocuni>
which I suppose it is caused by my change
<antocuni>
so, a review would be very welcome :)
<antocuni>
if this approach looks ok, I will proceed by converting the other types back to C. I think it is much much cleaner than having to write them explicitly as we are doing now
<antocuni>
arigato: a review/opinion by you is also welcome :)
mattip_ has quit [Ping timeout: 245 seconds]
mattip_ has joined #pypy
inhahe has quit [Read error: Connection reset by peer]
inhahe_ has joined #pypy
jvesely has quit [Quit: jvesely]
jvesely has joined #pypy
jvesely has quit [Quit: jvesely]
jaffachief has joined #pypy
mikroskeem has quit [Quit: overflow in /dev/null]
Rhy0lite has quit [Quit: Leaving]
jvesely has joined #pypy
mattip_ has quit [Ping timeout: 250 seconds]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
jaffachief has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
oberstet has quit [Ping timeout: 265 seconds]
jvesely has quit [Quit: jvesely]
oberstet has joined #pypy
mattip_ has joined #pypy
dansan has quit [Quit: The C preprocessor is a pathway to many abilities some consider to be unnatural.]
dansan has joined #pypy
altendky has quit [Quit: Connection closed for inactivity]