cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<vstinner>
hello. i wrote a PEP introducing incompatible changes in the Python C API to hide implementation details. your early review is welcome, especially on the sections related to PyPy :) i plan to propose it to python-dev tuesday: https://www.python.org/dev/peps/pep-0620/
<vstinner>
mattip: hello. migrations always take slow. it's a long term goal
<vstinner>
sorry, migrations are always slow and take several years to complete
<vstinner>
mattip: the question if i forgot other C CAPI which must change. i recall that someone here asked to deprecate tp_finalize API (PEP 442). i like tp_finalize, so i will not be the sponsor for such deprecation :)
<vstinner>
i also recall a request here to deprecate APIs which are too close to the Unicode implementation (PEP 393), but I'm scared by this API :-( the unicode API is HUGE and half of it is deprecated
<vstinner>
INADA-san just wrote a PEP to schedule the removal of the deprecated Python <= 3.2 APIs
<vstinner>
my second question is if it's correct to say that PyPy can allocate Python objects on the stack, and move it (to the heap) if needed
<cfbolz>
vstinner: it's a bit more complicated than that
<cfbolz>
vstinner: we don't allocate these objects at all, their fields live in registers and on the stack
<cfbolz>
it's not like an object has anything resembling its heap layout, just on the stack
<vstinner>
"their fields live in registers and on the stack" that's what I call "allocate objects on the stack" :)
<cfbolz>
well no
<vstinner>
in CPython, I don't how it would be feasible currently to allocate an object on the stack, even if we can prove that its lifecycle is exactly the function
<cfbolz>
eg if their fields are constants they take 0 space
<cfbolz>
so "allocate on the stack" is kind of a weird term
<vstinner>
cfbolz: i don't think that it's worth it to into such level of details in my PEP, so I just removed the two sentences about allocating objects on the heap or on the stack ;)
<vstinner>
cfbolz: if i undertood what you say, in fact, you don't allocate an object, it's not an object. there are just variables allocated on the stack
<cfbolz>
now we are getting philosophical ;-)
<vstinner>
in CPython, all objects are PyObject and must be allocated on the heap. there is no other way
<cfbolz>
yes, we "explode" the object, in a sense
<vstinner>
cfbolz: lol
<cfbolz>
vstinner: how about this: 'Objects can be allocated on the stack (or even not at all), rather than always having to be allocated on the heap.'
<vstinner>
cfbolz: if it works for you, it works for me :-D
<cfbolz>
it's an acceptable compromise ;-)
<cfbolz>
vstinner: thanks for doing this
<cfbolz>
vstinner: note that the "not at all" part is one of the main reasons why pypy is fast on arithmetic
<vstinner>
mattip wrote "it will be years before we can simplify the cpyext layer": right. but removing PySequence_Fast_ITEMS() & disallow &PyTuple_GET_ITEM(0) block access to PyObject** array and so it might become possible to only convert accessed objects of a list to a PyObject*, rather than converting the whole list, no?
<vstinner>
i'm not talking about the list strategy stores numbers directly as numbers, but a general PyPy list storing "PyPy objects"
<vstinner>
i understood that currently, cpyext always converts all items of a list when the C API is used
<cfbolz>
yes, sounds plausible. I am not super deep in cpyext though
<vstinner>
i chose to not take care of borrowed references in this PEP, since I understood that cpyext already solved the problem, it's not a major performance bottleneck, and it would require to modify a lot of code for "little benefit"
lritter has joined #pypy
<mattip>
vstinner: there is a list conversion from an internal representation to a cpyext representation whenever we need to iterate
<mattip>
we could theoretically convert them one at a time when indexed, but it is easier and maybe even faster to just convert all at once
floppydisk has joined #pypy
<vstinner>
mattip: "it is easier and maybe even faster to just convert all at once" ok
<vstinner>
mattip: what would in the C API would avoid any need for conversion?
floppydisk has quit [Remote host closed the connection]
dddddd has quit [Ping timeout: 260 seconds]
<mattip>
as soon as we need to return a PyObject from a sequense or dict, we are stuck
<mattip>
sequence
<vstinner>
mattip: because PyObject* can be dereferenced directly, right?
<mattip>
HPy provides a partial answer. The better answer is to not directly write C-API code, always go through a tool like SIP, Cython
<mattip>
then at least, given enough work, we could think about a non-CPython backend
kanaka has quit [Ping timeout: 260 seconds]
<arigo>
I think that HPy has the potential to go all the way towards an efficient API for PyPy, as long as we're clear that it's always possible to write C code that wouldn't be as fast as Python code in PyPy because objects can't be virtual in HPy
<arigo>
but for most kinds of code that we'll want to write in C anyway, virtual objects wouldn't help much I believe
<arigo>
apart from making "virtual" arguments and return values, which is the job of an argument-clinic-like solution
<vstinner>
mattip: in pratice, Cython still consumes the Python C API and was affected by multiple changes of my PEP 620 :)
<vstinner>
mattip: I agree that the Python C API must not be used directly anymore. it's even stated in the official documentation which suggests to use cffi or Cython
YannickJadoul has joined #pypy
<vstinner>
i'm not sure why cpyext would still have to immediately convert a list item into a PyObject if the PyObject structure becomes opaque. it could be done later, on demand, when a function which still requires a concrete PyObject structure is called, no?
<vstinner>
obviously, the long term goal is to remove all ways to access directly into a PyObject :)
<mattip>
well, that requires people to use the pypy-specific API and for them to know they have a list of ints.
<mattip>
what do they do if they have an arbitrary list?
<antocuni>
mattip: the idea is that HPy_AsSequence_xxx can return "NULL" to signal that it doesn't support that specific protocol
<mattip>
understood. It requires a level of commitment to non-CPython APIs. Will that happen in practice?
<antocuni>
well, it's an hpy-specific API, yes. The idea was to be used e.g. by cython
<antocuni>
this way if you are using pypy and pass a list-of-integers to a function written in Cython, it will be able to access the items without forcing them
<mattip>
ok, thanks
<antocuni>
np :)
<mattip>
it seems the test_interp_semaphore code is failing since the windows version of semlock_acquire reaches into interp_time.State
<mattip>
to pull out the interrupt_event (via get_interrupt_event)
<mattip>
but that has not been initialized, time.moduledef.startup is never called in the untranslated test
<mattip>
can someone remind me what is supposed to trigger a call to module.startup in untranslated tests?
<mattip>
it seems it should be triggered by "import time" in the test, but that does not seem to trigger time.startup()
<ronan>
mattip: 4f0e384e7817 looks OK, though I'm not sure why bz2 doesn't just use the regular leakfinder
<arigo>
mattip: in wb-before-move, in the graph you posted it's reported that trunk is a bit faster than wb-before-move at running richards, but I guess that's noise, because on my laptop it's the opposite (by a small margin too)
<arigo>
so I guess we're happy and merge now?
dddddd has joined #pypy
<mattip>
I'm happy :)
<arigo>
OK
<arigo>
merged
<mattip>
got it. This test is weird: I need to manually call space.getbuiltinmodule('time') since it is not a real AppTest
jcea has quit [Quit: jcea]
jcea has joined #pypy
wooster has left #pypy [#pypy]
YannickJadoul has quit [Remote host closed the connection]
YannickJadoul has joined #pypy
YannickJadoul has quit [Ping timeout: 240 seconds]