<antocuni>
I just managed to build my first manylinux wheel on pypy
<antocuni>
the good news is that it seems to be working out of the box
<cfbolz>
nice
<arigato>
is there a bad news?
<antocuni>
not really
<antocuni>
the annoying news is that the default pip we ship doesn't support manylinux2010
<antocuni>
so, what I have done is this: I ran manylinux's official docker image, installed pypy-7.1.1 and built the python-manylinux-demo wheels
<antocuni>
after running auditwheel repair, I got two wheels: one targeting manylinux1 and one targeting manylinux2010
<antocuni>
the manylinux1 can be installed by the stock pip we provide
<antocuni>
to install manylinux2010 you need to pypy -m pip install -U pip setuptools wheel
<antocuni>
the another kind-of-annoying news, is that apparently pypy DOES support manylinux1. I never managed to build those with the old manylinux docker images (mostly because pypy itself didn't run on such an old image), but now everything works
<antocuni>
which probably means that we could have used manylinux1 wheels for years, but we just didn't know how to do
<antocuni>
now I wonder whether I can upload those wheels to pypi
<fijal>
I put vmprof.com back up
<fijal>
not sure if anyone cares
<fijal>
but we should probably remove the necessity for the server
<antocuni>
fijal: thanks, it's good to have it working again
<fijal>
but I guess I can't be bothered
<antocuni>
but yes, it would be good to be able to use vmprof without it
<fijal>
I think we kinda fucked up what we thought people care about in something like vmprof
<antocuni>
as you see the wheel name is something like: wheel_example-0.7.1-pp271-pypy_41-manylinux1_x86_64.whl
<mattip>
ahh, cool. I can't access the page, but that is good enough
<antocuni>
pp271 means "pypy2 v7.1", while I think that "pypy_41" is the ABI tag
<antocuni>
actually, maybe it is even "too" good. As it is now it means that you need a different wheel for every pypy version
<mattip>
right, somehow when I built them I missed the pypy271 thanks
<mattip>
well, we should be careful when we bump minor versions, which is why I released 7.1.1 and not 7.2
<antocuni>
yes but not every minor release breaks cpyext compatibility (in fact, I think that most don't)
<mattip>
unfortunately the next release will have to change that since we touched the header files
<mattip>
but we can discuss it when we start a release cycle
<antocuni>
so, in an ideal world, the wheel would be "pp2-pypy_XX", and we update XX when we change cpyext
marky1991_2 has joined #pypy
<antocuni>
but yes, I don't think it is strictly needed to do it now, it's an optimization
marky1991_2 has quit [Remote host closed the connection]
marky1991_2 has joined #pypy
<antocuni>
I think that a good plan is to create an official "manylinux2010-pypy" docker image, which is based on manylinux2010 but contains also all the recent PyPy releases, so that people can just pull from it to build their pypy wheels
<tos9>
+1
<antocuni>
mattip: how likely it is to convince numpy to build and upload pypy wheels?
<mattip>
getting closer. My next numpy task is to revise our wheel building system, so will start in a month or so
<mattip>
once that happens, scipy uses the same wheel building system, so that should be a no-brainer too
<antocuni>
how does the current building system work? Do you docker pull from quay.io/pypa/manylinux1_x86_64?
<antocuni>
cool... having official numpy and scipy wheels will be a major improvement, and also a good marketing to convince other projects to build their own wheels :)
<mattip>
we build with OpenBLAS, so it is complicated, we need a version for each platform we support
<mattip>
he told me out of the top 1000 packages 230 have c-extensions, and we successfully build 210 of them
<antocuni>
sounds cool
<antocuni>
is he publishing the wheels anywhere?
<mattip>
no, it was just an experiment
<mattip>
for a single platform (macOS)
<antocuni>
well, the only reasonable future-proof way to go is to have each project responsible to build their wheel
<mattip>
... and convince them to use CFFI :)
<antocuni>
good luck with it :)
<antocuni>
well actually... I am not even sure that CFFI is the proper solution everywhere. Sometimes you DO want to operate on Python objects in C, like iterating over Python list or returning a Python dict
<antocuni>
good luck doing that with CFFI
<antocuni>
I often wonder whether we should offer a PyPy-friendly C API to do that
<mattip>
a few days ago someone was here, even fixed a parser bug, only to discover python-libusb uses ctypes which makes pypy slower than cpython
<antocuni>
yes, ctypes should die, I agree
<mattip>
and I got OpenCV to build on pypy2.7, they use numpy for data exchange but could use PEP 3118 buffers or CFFI
<antocuni>
I suppose that eventually we will have to special case numpy arrays and provide our fast implementation for that instead of relying on the default one
<antocuni>
they are used everywhere, there is no chance that EVERY project migrates to something else
<mattip>
yes, but to get fast indexing we would need that hacky stuff we were experimenting with
<mattip>
the higher functions could be intercepted by the new __array_function__ protocol
<antocuni>
yes sure
<mattip>
it is annoying that we do have the container, ufuncs, linalg, dtypes, and it all works well, but 95% is not 100%
<antocuni>
btw, another good news: to compile manylinux pypy wheels, you need to upgrade your pip/setuptools/wheel. But to install them from pypi, the stock pip we provide is good enough
<mattip>
we could upgrade that
<antocuni>
"we do have the container, ufuncs, linalg, dtypes": are you referring to micronumpy?
<mattip>
yes
<antocuni>
yes, ideally I would like a system in which we can improve micronumpy incrementally to add new features, and use the stock numpy for the features we don't provide
<antocuni>
but it's more complicate than that
<mattip>
right
<antocuni>
in particular, we need to think how it interacts with the numpy C API
<antocuni>
mattip: do you know what it is needed to upgrade the pip we ship with pypy?
<mattip>
99% of the use of the numpy c api is through cython, we should concentrate on cython
<antocuni>
I didn't know that, but I trust you. In that case yes, it is much easier
<mattip>
AFAIK it is simply a zip file somewhere that ensurepip unzips
<antocuni>
indeed, in lib-python/2.7/ensurepip/_bundled
<mattip>
ahh, wheels. So even easier
<antocuni>
technically you were right, they are "zip file somewhere that ensurepip unzips" :)
speeder39_ has joined #pypy
<mattip>
nice chatting, gotta get going
<antocuni>
see you
kenaan has joined #pypy
<kenaan>
mattip default 2680d2fce15d /pypy/doc/how-to-release.rst: document the issues around major.minor.micro release version numbers
ruda_porto has joined #pypy
mattip has quit [Ping timeout: 245 seconds]
moei has joined #pypy
mattip has joined #pypy
andi- has quit [Remote host closed the connection]
andi- has joined #pypy
antocuni has quit [Ping timeout: 258 seconds]
marky1991 has joined #pypy
marky1991_2 has quit [Ping timeout: 252 seconds]
marky1991_2 has joined #pypy
marky1991 has quit [Ping timeout: 244 seconds]
speeder39_ has quit [Quit: Connection closed for inactivity]
mattip has quit [Ping timeout: 252 seconds]
Rhy0lite has quit [Quit: Leaving]
speeder39_ has joined #pypy
mattip has joined #pypy
<jacob22>
It might be a good idea for PyPy to adveritise help for projects with C integration to have better interfaces at conference sprints.
ssbr has joined #pypy
marky1991_2 has quit [Remote host closed the connection]
marky1991_2 has joined #pypy
marky1991_2 has quit [Remote host closed the connection]
ruda_porto has quit [Ping timeout: 258 seconds]
marky1991_2 has joined #pypy
marky1991_2 has quit [Remote host closed the connection]
marky1991 has joined #pypy
forgottenone has quit [Quit: Konversation terminated!]
<tos9>
whee, travis does not support python on macos
mattip has quit [Ping timeout: 248 seconds]
mattip has joined #pypy
marky1991 has quit [Ping timeout: 245 seconds]
marky1991 has joined #pypy
Guest50 has joined #pypy
speeder39_ has quit [Quit: Connection closed for inactivity]
<mattip>
jacob22: do you mean we would help them design/migrate codebases?
xcm has quit [Ping timeout: 255 seconds]
Guest50 has left #pypy [#pypy]
dannyvai has joined #pypy
xcm has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
adamholmberg has quit [Remote host closed the connection]
<tos9>
Did 7.1.1 change pyrepl things
adamholmberg has joined #pypy
mattip has quit [Ping timeout: 248 seconds]
marky1991 has quit [Ping timeout: 246 seconds]
marky1991 has joined #pypy
<jacob22>
mattip: Right. I think may projects know fairly little about it. Many just look at Python C extensions or ctypes and do what seems to be easiest.