cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<vstinner>
a colleague wants to compare PyPy performances without/with -fno-semantic-interposition
<vstinner>
in Fedora, pypy is linked to libpypy
<vstinner>
CPython is way faster (like 10% faster) with gcc -fno-semantic-interposition, since 99,9% of CPython code lives in libpython, and there are many function calls from libpython to libpython
<vstinner>
-fno-semantic-interposition helps LTO
<vstinner>
hello arigo & cfbolz
<mjacob>
vstinner: you could try to benchmark non-jitted pypy, where i'd expect the differences to be bigger, as all of the run code is compiled by the C compiler
<vstinner>
mjacob: do you mean disable the JIT?
<vstinner>
like "pypy3 --jit off"?
<cfbolz>
vstinner: we didn't investigate lto in a while
<mjacob>
vstinner: either disable the JIT with flags, or compile without JIT
<cfbolz>
But it doesn't help the jit generated code, of course
<vstinner>
thrnciar_: ^^
<vstinner>
thrnciar_: do you know if PyPy is built with LTO?
<mjacob>
vstinner: i'd expect that, for both non-jitted and jitted pypy, the speedup will not be large, as pypy sets the symbol visibility to hidden more aggressively than cpython
<vstinner>
mjacob: CPython changed the visibility to hidden by default in Python 3.9 :) but even with that, CPython does tons of function calls to itself
<vstinner>
mjacob: basic example, checking if there is an exception "if (PyErr_Occurred())" is a function call to libpython. without -fno-semantic-interposition, it couldn't be inlined because the "semantic interposition" requires to go through PLT indirection
<vstinner>
to support LD_PRELOAD
<vstinner>
and it's the same for all function calls to libpython
<vstinner>
by the way, https://pyperf.readthedocs.io/ project moved under the PSF organization on GitHub: https://github.com/psf/pyperf which makes it more independent of myself :) previously, it was in /vstinner/ "organization". also, it got two more maintainers ;)
<mattip>
it calls both with "--jit off" and without
<vstinner>
mattip: oh ok
<mattip>
yeah, but as you point out it is alot of work to set the baseline for the benchmarks
<mjacob>
vstinner: PyErr_Occurred() is not hidden, is it?
<vstinner>
mjacob: it's part of the public C API, and Python consumes its own public C API
<mjacob>
vstinner: that's less the case in pypy
<vstinner>
mjacob: (well, about this exact function, I added a private _PyErr_Occurred(tstate) to Python 3.9. it takes a Python thread state argument, and this function is a static inline function)
<arigo>
vstinner: also, no-semantic-interposition has no effect on pypy unless you're talking about the cpyext C extension module compatibility
<arigo>
we use systematically "visibility hidden"
<arigo>
you can see with objdump that libpypy-c.so exposes almost no symbol apart from the thousands of Py*() compatibility symbols
<vstinner>
"libpypy-c.so exposes almost no symbol" oh ok
<mattip>
searching "lto" on the issue tracker, we discussed this in the past. Maybe compilers have improved since this last one
<mjacob>
vstinner: it could of course still be the case that a visible symbol is in the hot path, so it's still worth trying, but don't expect a 10%-ish speedup
<vstinner>
mjacob: the first time I tried LTO, I got tons of compiler bugs :) like crashes in gcc
<vstinner>
mjacob: honestly, it's *way* better nowadays (in 2020)
<vstinner>
mjacob: one week ago, sometimes reported a compiler crash in Python bug tracker. he was using Ubuntu 16.04. the fix is to upgrade Ubuntu to get newer GCC :)
<vstinner>
clang is also better than gcc on LTO
<vstinner>
too bad, PYPY_IRC_TOPIC=1 feature doesn't work in pypy3 :-(
<vstinner>
i was checking if there an env var to control PyPy JIT (to turn it off)
<mattip>
vstinner: works for me? Are you using one of our downloads or a different provider?
<arigo>
"of course", the following multiline regexp: {.*}|//.*?$
<arigo>
matches stuff line {hello} or everything from // to the end of the line
<arigo>
but "of course" if there is a line ending in "// {"
<arigo>
then which of the two rules would match?
<arigo>
(assuming there is a "}" on a following line, and "." is allowed to match newlines)
<arigo>
the answer is: it's the second rule, i.e. "//.*?$", eating the "{", because that rule happens to start before the first rule in the string
<arigo>
that means my attempt to split this regexp in two parts is doomed
<arigo>
...in really two regexps, I mean
<arigo>
for whatever order you run the two regexps, there is a counter-example string that is matched differently
<arigo>
(the counter-examples are "// {" like above, and "{ // }")
jacob22_ has quit [Quit: Konversation terminated!]
jacob22 has joined #pypy
<vstinner>
mattip: i added "pyperformance is not tuned for PyPy yet: use the PyPy benchmarks project instead to measure PyPy performances." note to pyperformance documentation until someone tunes pyperformance for PyPy ;)
epsilonKNOT_ has quit [Quit: ZNC 1.7.5 - https://znc.in]
jeroud has quit [Ping timeout: 272 seconds]
epsilonKNOT has joined #pypy
phlebas has quit [Ping timeout: 264 seconds]
samth has joined #pypy
phlebas has joined #pypy
fryguybob has quit [Quit: Lost terminal]
samth has quit [Ping timeout: 256 seconds]
phlebas has quit [Ping timeout: 256 seconds]
otisolsen70_ has quit [Ping timeout: 240 seconds]
samth has joined #pypy
phlebas has joined #pypy
jeroud has joined #pypy
tsaka__ has quit [Remote host closed the connection]
tsaka__ has joined #pypy
tsaka__ has quit [Remote host closed the connection]
tsaka__ has joined #pypy
Dejan has quit [Quit: Leaving]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
marvin has quit [Remote host closed the connection]
marvin has joined #pypy
otisolsen70_ has joined #pypy
otisolsen70_ has quit [Remote host closed the connection]
tsaka__ has quit [Ping timeout: 264 seconds]
otisolsen70 has joined #pypy
marvin has quit [Remote host closed the connection]
marvin has joined #pypy
lritter has joined #pypy
otisolsen70 has quit [Quit: Leaving]
oberstet has quit [Remote host closed the connection]
Ododo has joined #pypy
<Ododo>
Hello. is it possible somehow to compile a 32 bit cffi extension on a 64 bit machine ? ..
<tumbleweed>
if you're executing a 32 bit python, and the compiler available is 32bit, I think that's what you'd get
<tumbleweed>
easy answer: do it in a 32bit chroot
<Ododo>
yeah .. it's looks like something that could be feasable but the easy way is tempting..
<tumbleweed>
(or container / whatever)
<Ododo>
ok. it's mainly because i was looking for a way to distribute the package and that the basic cloner of the repository could just build it from a 64bit machine without having to create a chroot by himself.
<Ododo>
(build a 32bit .so)
<Ododo>
but a good readme is also ok
vstinner has left #pypy [#pypy]
<tumbleweed>
they'll need a 32bit toolchain and python
<tumbleweed>
so, either you have to distribute those
KindTwo has joined #pypy
<tumbleweed>
or just point them at an existing distribution
KindOne has quit [Ping timeout: 272 seconds]
KindTwo is now known as KindOne
<Ododo>
yep. ok. i see, thanks.
Ododo has quit [Quit: leaving]
tsaka__ has joined #pypy
tsaka__ has quit [Ping timeout: 258 seconds]
asmeurer has joined #pypy
tsaka__ has joined #pypy
inhahe has quit [Read error: Connection reset by peer]