cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
Rhy0lite has quit [Quit: This computer has gone to sleep]
rubdos has quit [Ping timeout: 240 seconds]
rubdos has joined #pypy
jcea has quit [Quit: jcea]
petronny has joined #pypy
Taggnostr has quit [Ping timeout: 260 seconds]
camelCaser has quit [Ping timeout: 256 seconds]
camelCaser has joined #pypy
Taggnostr has joined #pypy
xcm has quit [Remote host closed the connection]
oberstet has joined #pypy
xcm has joined #pypy
dddddd has quit [Ping timeout: 260 seconds]
otisolsen70__ has quit [Quit: Leaving]
xcm has quit [Ping timeout: 240 seconds]
<mattip>
pillow will be providing pypy wheels for linux64, macos in the July 1 release
xcm has joined #pypy
<antocuni>
mattip: wonderful!
<antocuni>
we should do a blog post
<mattip>
maybe a blog post like "popular packages are now shipping PyPy wheels, but not for windows. Wouldn't it be nice to have someone sponsor work on PyPy 64 bit windows?"
<antocuni>
maybe, but are we sure to have someone willing to do the work in case we get the money :)
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
jacob22_ has quit [Quit: Konversation terminated!]
jacob22 has joined #pypy
todda7 has joined #pypy
YannickJadoul has joined #pypy
xcm has quit [Read error: Connection reset by peer]
xcm has joined #pypy
lritter has joined #pypy
xcm has quit [Remote host closed the connection]
Rhy0lite has joined #pypy
xcm has joined #pypy
lritter has quit [Quit: Leaving]
lritter has joined #pypy
thrnciar_ has joined #pypy
thrnciar_ has quit [Ping timeout: 240 seconds]
thrnciar_ has joined #pypy
dddddd has joined #pypy
xcm has quit [Read error: Connection reset by peer]
xcm has joined #pypy
<YannickJadoul>
This might be an important failure, when trying to release a 3.7 alpha?
<mattip>
YannickJadoul: that is a pypy-only file, and the test seems bogus to me
<mattip>
especially since we seem to have solved the pip/setuptools incompatabilities: cibuildwheel works, conda packaging works ...
<YannickJadoul>
mattip: Alright, then. Just going through some test failures, and it somehow seemed like something would be wrong with the installation, or so
<YannickJadoul>
OK, great :-)
<mattip>
well, we could just delete that file
BPL has joined #pypy
<mattip>
YannickJadoul: I think I will merge the py3.7-x-dev-option branch. It passed the new test, right?
<YannickJadoul>
Euhm, yes. I can quickly check again
<mattip>
I just remember getting burned with the last change to app_main, but this one seems less intrusive
<mattip>
although adding an import also seemed non-intrusive :)
<mattip>
I guess we could kick a buildbot build, then download and check it outside a pypy checkout
<YannickJadoul>
The relevant one from PyPy's test suite does
<YannickJadoul>
Some other have fail on `assert _sre.MAGIC == MAGIC, "SRE module mismatch"\nAssertionError: SRE module mismatch`, but that was also the case before the changes
<YannickJadoul>
I also tried translated, and I think that worked as expected
<jaraco>
Hello PyPy folks. I’m working on https://github.com/pypa/setuptools/pull/2143, which attempts to merge setuptools and distutils, but I’m currently blocked on PyPy compatibilty issues.
<jaraco>
I’ve investigated one incompatibility but there are probably several others.
<jaraco>
I’d really like some help producing a merged distutils includes the PyPy customizations.
<jaraco>
I tried checking out the distutils code from around the same time as a recent merge into PyPy’s distutils.
<mattip>
What customizations do we have?
<jaraco>
good question ...
<mattip>
the py3.6 branch's lib-python/3/distutils should be a copy of the cpython 3.6.9 Lib/distutils
<jaraco>
I’m not sure I’ve chose the right distutils base or pypy commit from which to get a minimal but complete diff.
<cfbolz>
mattip: it sounds plausible that we have some patches in there
<mattip>
the msvc ones are backports from later versions of cpython in order to use more modern compilers
<mattip>
we need that to compile with msvc2019
<jaraco>
That suggests I should probably be targeting a later distutils. Are you saying that distutils for PyPy is a copy from CPython at 3.7.x (for py3.7 branch) but some changes were backported from later Python versions? It may prove difficult to disentangle those changes.
<mattip>
right now the active branches of pypy are default, which corresponds to python2.7, py3.6 which corresponds to cpython 3.7.6, and
<mattip>
py3.7 which corresponds to cpython3.7.6
<mattip>
sorry, py3.6 which corresponds to 3.6.9
dddddd__ has joined #pypy
<mattip>
comparing py3.6 to cpython 3.6.9, I get that these files differ
<mattip>
let's start from the beginning. What is the goal?
<jaraco>
Setuptools wishes to adopt distutils such that distutils can be removed from the stdlib.
<mattip>
without any version/implementation tweaks for distutils?
<mattip>
or will there be a separate versions of setuptools for 3.6, 3.7, 3.8, 3.9?
<jaraco>
It should include those tweaks and support PyPy across many versions.
<jaraco>
Including CPython 3.5+ and PyPy for Python 3.6+
dddddd_ has joined #pypy
<mattip>
would it make sense to merge the cpython 3.5+ version into setuptools without pypy
dddddd__ has quit [Ping timeout: 260 seconds]
dddddd has joined #pypy
<mattip>
then in a second PR add in the bits required for pypy, like splitting sysconfig into sysconfig_cpython and sysconfig_pypy
<mattip>
(since we have no makefile)
<jaraco>
I want to merge the latest version (from CPython 3.9) with select back ports so that it has the latest improvements and behaviors and can readily replace that version.
<mattip>
makes sense. Still I think two (or more) PRs will be needed for such a big change
dddddd__ has joined #pypy
dddddd_ has quit [Ping timeout: 265 seconds]
<jaraco>
I think I’m attempting something very similar to what you’re suggesting - by choosing an older commit from the distutils history (https://github.com/pypa/distutils/commit/5f468e19b9daea4441ceba0eaa8c83d02ab8b15d), and applying the PyPy changes to that, I’m effectively merging with a recent ancestor of the PyPy changes.
<jaraco>
It sounds though like I might have an easier time if I chose an earlier copy of PyPy’s distutils.
<jaraco>
One before the backports of MSVC.
dddddd has quit [Ping timeout: 264 seconds]
<mattip>
no, that is not two PRs
<mattip>
so it seems you want to do it all at once. The differences you will need to take into account are at least these three basic things:
<mattip>
- we want to use MSVC2019
<mattip>
- we have a different sysconfig
<mattip>
- sometimes distutils leaks references "s = open(f).read()"
<jaraco>
I don’t necessarily want to do it all at once. I want to first merge CPython distutils and PyPy distutils, probably at the older version where PyPy is. Then I want to merge in changes from CPython 3.9, then I want to merge in backports to support Python 3.5, then I want to merge that into Setuptools.
<jaraco>
The `pypa/distutils` project contains the history of distutils as found in CPython, but without the rest of CPython.
<jaraco>
In `afbcc9a6fe9d`, you’ve backported msvc distutils from CPython; was there anything in that commit that wasn’t taken verbatim from CPython?
<mattip>
not on purpose
<mattip>
unless it has a PyPy comment
<mattip>
nope, no comments
<mattip>
I would suggest a different order: leave PyPy for last (merge 3.9, merge backports for 3.5, then PyPy) since the PyPy changes will require a different strategy for sysconfig
<jaraco>
I’ve completed the first two steps, except I can’t merge them into Setuptools because Setuptools tests on PyPy fail.
<jaraco>
It is a fairly complicated endeavor, and it wasn’t until today that I realized there was another divergence of the code in PyPy (though in retrospect, makes sense).
<mattip>
but 2143 is a merge from pypa/setuptools/distutils into pypa/setuptools/master. How does jaraco/setuptools-merge relate to these two?
<jaraco>
So what I’m aiming to do is use some branch of pypa/distutils to adopt the changes from PyPy, but without losing CPython compatibility or features from CPython 3.9.
<mattip>
I don't think you can simply "diff" between two versions and be done. We will have to look at the changes one at a time
<jaraco>
pypa/setuptools/distutils is a merge of pypa/setuptools/master and pypa/distutils/jaraco/setuptools-merge.
<mattip>
So pypa/setuptools/distutils has distutils that works on CPython 3.5 - CPython 3.9 ?
<jaraco>
Yes.
<jaraco>
It has a setuptools+distutils.
<jaraco>
but pypa/distutils/jaraco/setuptools-merge has only distutils.
<jaraco>
So if we can apply the PyPy changes (cherry-picked or otherwise) somewhere along the pypa/distutils tree and then merge that with pypa/distutils/master, I’ll have what I need to then merge that into the jaraco/setuptools-merge branch.
<mattip>
ok. Let me try to make a PR to jaraco/setuptools-merge, it will take a few days
<jaraco>
Oh, that would be fantastic!
<jaraco>
I’ll keep an eye out and check back. Don’t hesitate to ping me if I can help.
<mattip>
where exactly is that branch?
<mattip>
nvrmind, got it
<jaraco>
You can send a PR instead against master if that’s easier.
<jaraco>
Note that in master, there’s no `distutils` folder; everything’s in the root.
<jaraco>
(that was an unfortunate consequence of how I extracted the history from CPython)
<mattip>
ahh, no tests whatsoever
todda7 has joined #pypy
<jaraco>
There aren’t, or if they are, they’re detacted from any runner.
<jaraco>
I’m planning on relying primarily on setuptools’ tests. If you’re able to work without tests, feel free to provide an approximate solution and I can refine it after running it through the setuptools test suite.
<jaraco>
If that proves inadequate, I may consider extracting the tests from CPython as well.
<mattip>
the big problem is sysconfig. Do you have a preferred strategy for dealing with it?
<jaraco>
I haven’t looked at it enough to know what the challenges are, much less have a strategy. I noticed that PyPy splits into sysconfig_(cpython/pypy). Will that approach be sufficient to support both?
<mattip>
I am looking. We may be able to get by with a single file, but it may have IS_PYPY scattered around
<mattip>
not having tests is problematic. How will I know if my changes work?
<mattip>
it seems too hard to do like this. I go back to my first proposal: merge #2143 into a branch of setuptools (not master), without PyPy, then we can iterate on that.
<mattip>
i.e. remove pypy from CI in that branch. Then we can start a new PR with PyPy re-enabled
<mattip>
and eventually merge it to master
<jaraco>
In that case you could make a PR targeting the distutils branch of Setuptools.
BPL has quit [Quit: Leaving]
<jaraco>
Does that make sense? Use that branch as a base and target the changes to that branch.
<jaraco>
Going afk. Mention me if you need my attention.
<YannickJadoul>
Is it normal that something like '\N{GREEK CAPITAL LETTER DELTA}' doesn't work untranslated?
todda7 has quit [Ping timeout: 240 seconds]
<YannickJadoul>
(alright, I got it: {"usemodules" : ["unicodedata"]})