cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<mattip>
about issue 3107, the point of _Py_Finalizing is a sentinel to mark when inside space.finish()
<mattip>
comparing Py_Finalize_Ex in cpython3.6.9 (in Python/pylifecycle.c) to space.finish(), we are missing much of that code
<mattip>
running gc.collect till nothing changes, calling the equivalent of PyImport_Cleanup, dumping some statistics (that we probably don't collect in the first place)
<mattip>
that is only about gc.collect(). What about PyImport_Cleanup(), PyInterpreterState_Clear(interp)? Since we don't do any cleanup, we could just set "_Py_Finalizing=NULL" and be done
<mattip>
maybe with a XXX comment in space.finish()
Ai9zO5AP has joined #pypy
i9zO5AP has joined #pypy
Ai9zO5AP has quit [Ping timeout: 240 seconds]
i9zO5AP has quit [Quit: WeeChat 2.5]
antocuni has joined #pypy
<arigato>
shunning: ah, you mean converting the regular dict to a "module-dict" when we pass it to exec()? maybe
<arigato>
shunning: can you open an issue about that so we don't forget?
<arigato>
mattip: wrote a comment on that issue
<arigato>
mainly, is it a good idea to ignore _Py_Finalizing, or will he want instead something closer to the `atexit` module
<arigato>
fwiw, I see that there is a flag space.sys.finalizing in py3.6
<YannickJadoul>
Tests are currently failing because the auditwheel version is too old (2.0), and doesn't include this PR (included in 2.1): https://github.com/pypa/manylinux/pull/293
<YannickJadoul>
Fixing it might just be as easy as just creating a new docker image from the existing Dockerfile (since the base pypa docker image has been updated meanwhile), if anyone has access to the pypywheels account on dockerhub
jvesely has joined #pypy
rabysh has joined #pypy
antocuni has joined #pypy
<tos9>
Wow, amazing
<tos9>
Well done, I had that on my todo list forever but wasn't able to make any progress on it
* tos9
will look at your change
<mattip>
antocuni ^^^
<YannickJadoul>
tos9: Well, now that we finally have `manylinux2010` in cibuildwheel, it's a lot easier
<YannickJadoul>
And this is still only the Linux, ofc, but any feedback/help is welcome :)
<antocuni>
YannickJadoul: I have access to dockerhub's pypywheels account
<antocuni>
what do you need me to do?
<YannickJadoul>
antocuni: The Docker images would have to be rebuilt, I think, to get an update of auditwheel
<antocuni>
probably _minimal_curses was not available on the py3k branch at that time
<arigato>
maybe, or maybe you're right and the goal was to run on CPython too (that commit doesn't seem to have the __pypy__ check yet)
<antocuni>
if anybody is going to modify our version of pyrepl, I strongly suggest to get in touch with blueyed (who lurks here) and try to have a single version of it
<antocuni>
because currently we have many slightly different forks here and there: we have pypy's pyrepl on defaul and py3.x branches which IIRC are different; then we have bitbucket's pyrepl, and finally github's pyrepl :(
<arigato>
I guess the two versions are already pretty diverged
<blueyed>
antocuni: at least bitbucket's should be out-of-date, given that we moved it to github.
<blueyed>
I suggest getting it merged in pypy first, and then see how it compares to pypy/pyrepl on github.
<blueyed>
There is at least one pending PR for me for pypy where I've tried to forward fixes etc, without any response/feedback.
<blueyed>
pypy/pyrepl on github works ok nowadays for py2/py3 with pdbpp for me.
<YannickJadoul>
antocuni, tos9: Except for AppVeyor and Travis's being waaay behind on the build queue, seems like building manylinux wheels works perfectly, now :)
<YannickJadoul>
Now we 'just' need to add things for Windows and macOS
<YannickJadoul>
Fun fact: even manylinux1 wheels are being built, somehow
jvesely has quit [Quit: jvesely]
<tos9>
Hooray!!!
<tos9>
YannickJadoul: Do you have an example of how to set that up somewhere? (Is it as simple as using cibuildwheel 'normally'?)
<YannickJadoul>
tos9: Once that PR would get merged, yes!
<YannickJadoul>
I've got two new build tags: pp271-manylinux_pypy_x86_64 & pp371-manylinux_pypy_x86_64, to be selected/deselected with CIBW_BUILD and CIBW_SKIP
<YannickJadoul>
And then an option CIBW_MANYLINUX_PYPY_X86_64_IMAGE that defaults to using pypywheels/manylinux2010-pypy_x86_64
<YannickJadoul>
(Though it would be even better if the PyPy binaries would be part of PyPA's manylinux images?)
<svanheulen>
in the cffi documentation it says you "can use literally ... at various places in the cdef(), and the missing information will be completed with the help of the C compiler." with no further explanation of how to do that. can anyone give me some details?
jacob22_ has joined #pypy
<cfbolz>
svanheulen: it's probably easiest to look at some of the examples
<svanheulen>
cfbolz: so for the passwd struct why do they have one of the memebers defined?
<cfbolz>
svanheulen: the defined one is the one that can be accessed from python
<cfbolz>
but there might be others, that the python code doesn't care about
<cfbolz>
that's what the ... is for
<svanheulen>
ah so you still have to define things that you need to access. what about if the one i want to access is in the middle of the struct? do i need to define everything up to that point? or can i have a "..." before and after it?
<cfbolz>
svanheulen: not sure, I think it should work before and after
<svanheulen>
ah ok, thanks for the info
<cfbolz>
svanheulen: I always with for a few more examples in the docs too ;-)
<mattip>
s/with/wish/
<cfbolz>
yep
svanheulen has quit [Quit: WeeChat 2.6]
jvesely has quit [Quit: jvesely]
shunning has joined #pypy
<shunning>
I'm back :( it seems like for small workloads, the hack works. For bigger workloads which we had no issue using pypy for python 2.7, those trampoline dict accesses are back. I wonder again if there is anything changed from python2 to python3?
<shunning>
I can see for some of the functions, there are trampoline dict accesses inside the trace, but for other functions there are no trampoline dict accesses
shunning has quit [Remote host closed the connection]
<cfbolz>
shunning: unfortunately I don't have time to look in detail atm. However, I wanted to point out that it might be interesting for you to look at *unoptimized* traces to see whether there's a big difference there
<cfbolz>
The unoptimized traces are the basis of inlining decisions
<cfbolz>
You can see them by looking at the jit-log-no categories
<kenaan>
rlamy default ef97a01d2e01 /pypy/interpreter/test/test_function.py: Fix uses of raises()
<shunning>
Thanks Carl, hope everything goes well with you
<shunning>
It's just seem to me that the limit of how many function the JIT can inline in a single trace is limited
<shunning>
Right now I have a dynamically compiled function with 100 function calls inside, and the first thirty inlines well without the actual dict lookup. However, after that, each function call turns into ll_call_lookup_function_trampoline__v1434___simple_call__function_.