<kenaan>
mattip buildbot 89139fc90e06 /bot2/pypybuildbot/master.py: add platform-specific factory for aarch64 so that package.py gets the right archive-name
<mattip>
89139fc90e06 seems really fragile - right now the aarch64 packages are uploaded as linux64
<RoadrunnerWMC>
Hello! I'm using cffi to make bindings for an existing C library, but I've run into an issue with callbacks. Hope this is the right place to ask about it
<mattip>
RoadrunnerWMC: this is the right place but most of the "right people" are on European time, so your question may linger a while
<RoadrunnerWMC>
All right, thank you; I'm not in a hurry :)
<RoadrunnerWMC>
You run the test case by running build_cffi.py to build and then test.py to run
<RoadrunnerWMC>
The error I get when I do so is "extern "Python": function _c_code._py_destroy_callback_function_impl() called, but @ffi.def_extern() was not called in the current subinterpreter. Returning 0."
<RoadrunnerWMC>
Environment is Python 3.6 on KDE Neon (which's based on Ubuntu).
<RoadrunnerWMC>
and latest cffi (1.12.3)
<RoadrunnerWMC>
Expected result (which is what happens if you comment out that function) is for it to print "End of test.py" followed by "Destroy callback"
lritter has quit [Ping timeout: 244 seconds]
lritter has joined #pypy
<mattip>
RoadrunnerWMC: you need to make sure obj.__del__ is being called before the interpreter exits
<mattip>
for instance, if I put the call to "obj = wrapper.make_class()" in a function, all works as expected (on cpython)
<mattip>
on PyPy, you need to explicity trigger the deletion since the gc works differently
<mattip>
with something like "del obj"
<mattip>
ronan: something is wrong with collecting tests and -D on windows. If I call
<RoadrunnerWMC>
mattip: hm, I guess that makes sense, but I can't really dictate exactly how people use the bindings, can I?
<mattip>
RoadrunnerWMC: yeah, maybe there is a problem that the callback or a part of it is being gc collected before the object
<RoadrunnerWMC>
mattip: That makes a lot of sense, and also explains why seemingly unrelated changes affect the behavior. I wonder if there's a way to keep the callback alive long enough, then...
<RoadrunnerWMC>
My current ideas are to either make my wrapper Python class into a context manager, or to entirely bypass the C library for this particular destructor callback and implement it in pure Python instead. Or maybe I could just not expose that callback in Python at all, since I don't think it's very necessary anyway
<RoadrunnerWMC>
I'll look into it further tomorrow. Thanks for the help so far, mattip!
<arigato>
RoadrunnerWMC: right, not too surprized that we get strange errors
<arigato>
the object returned by ffi.gc() is freed by CPython at a random time late during interpreter shut-down
<arigato>
at that point, the cffi logic to support ffi.def_extern() is not registered any more, but anyway it's unclear that we can really call arbitrary Python code at that point
<arigato>
if you're using ffi.gc(), you can also try a more direct approach: right now, the ffi.gc() object calls c_class_destroy at the end, which itself calls back pure Python code
<arigato>
the more direct approach would be to use ffi.gc() with a lambda that both calls c_class_destroy and then (before or after) calls some custom code that you pass as argument to make_class(), for example, if it is not None
<arigato>
of course, that's assuming you need that at all; if you don't really need it then it's even better. The exact time destructors are called is slightly undefined on CPython and very undefined on PyPy
<arigato>
(note that recent releases of cffi also support using an ffi.gc() object in a "with" statement, and force the destructor to run at the end)
<ronan>
mattip: the warning is annoying but rather meaningless, --result-log is still here in pytest 5
jcea has joined #pypy
<RoadrunnerWMC>
arigato: thanks for all that info! I think I'll just not support that callback, since it doesn't translate well at all to Python and it's not really necessary. (If somebody wants to run some code after they're done with that object, they can easily do so manually; the C library I'm wrapping is completely synchronous)
<arigato>
ok!
<RoadrunnerWMC>
While I'm here, I'll ask another question that Google wasn't too helpful about: how do people generally compile their package for a variety of OSes? Is there some standard relatively automated way to do that? CI services, maybe?
<arigato>
I think so, but I don't really know myself
<RoadrunnerWMC>
That link doesn't DNS-resolve for me, but it looks like they have their own CI server going, which would imply that existing free CI services are insufficient for this. That's a bit unfortunate if so, but thanks
<arigato>
cryptography might be a bit of a special case; maybe others can point to some free CI service
<RoadrunnerWMC>
That's true; I should probably ask people involved with some other packages that use C code (like Pillow). Thanks again
<mattip>
RoadrunnerWMC: azure pipelines has linux, windows, macos
<mattip>
soon it will be integrated more tightly with github as microsoft owns both
<mattip>
ronan: ok, more worrying is the failure to collect any tests on win32 and -D
<ronan>
mattip: I'm looking into it
<ronan>
the conftest for _vmprof is doing something wrong
<mattip>
cool, let me know if you need anything
<RoadrunnerWMC>
mattip: looks nice at a glance; I'll probably give it a try. Thanks
<mattip>
ronan: perhaps because the testrunner/app_level_tests.py and testrunner/runner.py collect per-directory,
<mattip>
so the _vmprof pytest_collect_directory function is only specific to _vmprof,
<mattip>
where the -D uses pytest to do all the work, so it collects _vmprof together with the other directories
ekaOlogik has joined #pypy
<ronan>
mattip: the issue is that _vmprof/conftest.py silently crashes test collection
<ronan>
but I don't know what is the right way to do what it tries to do
<mattip>
could ask on #pytest how to get conftest.py to skip only its directory
<kenaan>
arigo py3.6 4006ceea6169 /pypy/module/imp/importing.py: Remove the only usage of open() in the py3.6 core parts (as found in the branch py3.6-sandbox-2). Also wrap the raw ...