cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
altendky has quit [Quit: Connection closed for inactivity]
dddddd has quit [Remote host closed the connection]
jcea has quit [Quit: jcea]
andi- has quit [Ping timeout: 246 seconds]
andi- has joined #pypy
_whitelogger has joined #pypy
_whitelogger has joined #pypy
<bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-linux-s390x/builds/1217 [mattip: force build, release-pypy3.6-v7.x]
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-s390x/builds/1218 [mattip: force build, release-pypy2.7-v7.x]
<mattip> 15 hours for lib-python tests on s390x. http://buildbot.pypy.org/builders/pypy-c-jit-linux-s390x/builds/1217
<mattip> the timeout in testrunner/lib_python_tests.py is 3600, maybe we can shorten it
<mattip> rather than run lib-python tests with our vendored pytest-2.9.2, maybe we should run them in the virtualenv we create anyway
<mattip> antocuni: impressive. On the cpyext-speedup-tests-py36 branch, module/cpyext/test/test_api.py tests took 17 secs
<mattip> vs 44 secs on py3.6
<mattip> total own test time dropped from ~1hr7min to 22min !!
<mattip> there is a problem with build_cffi_imports in translation though, which is maybe why benchmarks failed
i9zO5AP has joined #pypy
xcm has quit [Remote host closed the connection]
Ai9zO5AP has quit [Ping timeout: 260 seconds]
xcm has joined #pypy
xcm has quit [Read error: Connection reset by peer]
xcm has joined #pypy
_whitelogger has joined #pypy
<antocuni> mattip: on cpyext-speedup-tests-py36 from 1hr7m to 22m? 😱
<mattip> yup
<antocuni> that's unexpected... I thought that the speed penalty was paid only for initializing the space, which is done only once
<mattip> I think we run each file in a separate runner. Search for module/cpyext/test/test_api.py on this stdio log
<mattip> module/cpyext/test/test_api.py
<mattip> and see how each file is reported separately
<mattip> so maybe we would be even faster if we ran them all together
<antocuni> ah, I see
<antocuni> and indeed, there are 59 of them, so on average 45s less for each
<mattip> I think I did that because otherwise the tests timed out, but now they won't
<antocuni> ah, it's worth trying them
<antocuni> *then
<mattip> now if I can only remember which test config file does the splitting up
<antocuni> mattip: note that I also got a huge speedup with 317104f1b067. So I think that the total speedup of running a cpyext test compared to when I started is like 10x now :)
<antocuni> there are some failures on the branch, though :(
<mattip> maybe the cache needs a lock around it?
<antocuni> they are about TestMultiPhase, so I suspect it's related to the fact that "self.import_module" no longer puts the module in sys.modules
<antocuni> because it no longer goes through "imp.load_dynamic" now
<antocuni> if I am correct, it should be easy to fix
<mattip> +1
<kenaan> mattip pypy.org[extradoc] fb71ab56fca3 /: release 7.3.0
<mattip> time for a blog post
<mattip> please review, it is based on http://doc.pypy.org/en/latest/release-v7.3.0.html
<mattip> fixed, pypy7.2 was released in October, a little over 2 months ago. For just that short of time, the changelog is impressive
jvesely has joined #pypy
<tumbleweed> lookslike a formatting issue around the manylinux link
<antocuni> mattip: yes, you a probably missing the rst link
<mattip> thanks. fixing
* mattip bbl
<antocuni> mattip: also, maybe it is worth mentioning https://github.com/pypy/manylinux to tell people they can build their own manylinux wheel now?
<antocuni> nice blog post, looks good to me
<tumbleweed> want a 32bit arm build?
<bbot2> Started: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/7911 [antocuni: force build, cpyext-speedup-tests-py36]
<cfbolz> mattip: blog post looks good to me!
<cfbolz> mattip: thanks so much for digging through all the manylinux and portability problems
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
<bbot2> Failure: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/7911 [antocuni: force build, cpyext-speedup-tests-py36]
ronan has joined #pypy
jvesely has quit [Quit: jvesely]
_whitelogger has joined #pypy
Dejan has joined #pypy
<mattip> tumbleweed: 32 bit arm froma debian build? Sure. Which debian?
<mattip> antocuni: sure, I'll try to work it in
<kenaan> antocuni py3.6 4af6d07b98ee /: hg merge default
<kenaan> antocuni cpyext-speedup-tests-py36 8098a1266abb /: close branch to be merged
<kenaan> antocuni py3.6 98474e851cf8 /: merge the cpyext-speedup-tests-py36 branch, which greatly spees up running cpyext tests, especially if you run the...
<kenaan> antocuni py3.6 ead9a40f22ed /pypy/doc/whatsnew-pypy3-head.rst: document this branch
<tumbleweed> whatever kind of build you want. Don't know the build process, though :)
<mattip> ahh, I thought you already had builds handy
<mattip> If possible we would like a machine that can run buildslave to hook into
<mattip> buildbot.pypy.org
<mattip> and run nightly builds
<Dejan> io_uring rocks
<mattip> antocuni: how does this look (about building wheels)?
<Dejan> Looks good
<mattip> Dejan: I added a new paragraph about "If you are a python library maintainer ...". Does it make sense?
<Dejan> let me check
<Dejan> IMHO it does as CFFI is preferred
<antocuni> mattip: it looks good, thank you!
<tumbleweed> mattip: happy enough to donate a VM if that's what's needed
<mattip> the problem with a VM is that translation is 10x slower, so 5-10 hrs instead of less than 1
Rhy0lite has joined #pypy
<mattip> not worth the cost IMO, we haven't had many complaints about a missing 32-bit arm PyPy
<tumbleweed> thore's nothing inherent in virtualization that should cause quite that cost
<tumbleweed> but yeah, fair enough if nobody's complaining
<mattip> maybe I am wrong. how long do your debian runs take?
<mattip> is there a debian 32-bit offering on ARM?
<mattip> Dejan: thanks
<tumbleweed> debian is building pypy on all architectures, so yes 32bit ARM is there
<tumbleweed> looks like we're getting 3hr builds, on one buildd, and 7hr on another. Very different hardware, presumably
<mattip> ahh, so I was remembering the 7hr one. That would be annoying to run 2x daily (default and py3.6)
<mattip> plus testing, which would double that
<mattip> antocuni: it seems multibuild also supports pypy in a docker (they already have a PR for 7.3)
<Dejan> would be nice to update 3.7 branch too and try to build it
<mattip> Dejan: go for it, The branch was updated, anyone can kick the buildbots
<antocuni> mattip: didn't know about multibuild but it's cool they are supporting PyPy
<antocuni> I wonder if there any project actually uploading pypy wheels on PyPI
<mattip> pillow, pyside
<mattip> pysoundfile, but it is cffi based
<antocuni> pyside? Does it work on pypy?
<antocuni> I didn't know, that's very cool
<mattip> ahh, no, my bad. searching "site:pypi.org pp72" misled me
<mattip> Published!
<mattip> cfbolz: could you tweet it on the pypy twitter account
<cfbolz> Yay, congratulations!
adamholmberg has joined #pypy
<mattip> hopefully this will lead to more projects making wheels
<cfbolz> mattip: done. Thank you so much for the work you've been doing!
<mattip> on another topic: we have a cpyext module in pypy/tool
<mattip> It is being imported when pypy/tool/build_cffi_imports.py tries to import pickle.py by the recent changes to speed up cpyext
<mattip> gahh
<mattip> (this is during the last stage of translation, when it builds the imports)
dddddd has joined #pypy
<kenaan> mattip default 4824f6da985b /testrunner/lib_python_tests.py: shorten timeout to help s390x tests finish faster. We should also fix the failures
<kenaan> mattip default 2f00c6d27ef5 /pypy/doc/release-v7.3.0.rst: make release note match blog post: add bit about wheels
adamholmberg has quit [Remote host closed the connection]
squeaky_pl has joined #pypy
<squeaky_pl> Congrats on the release, download page seems to point to 7.2 still for me.
<squeaky_pl> I grabbed 64 bit Linux tarballs and tested them under Window Subsystem for Linux 2 on Windows 10 April 2020, everything seems to work fine on Windows, idle works as well.
<squeaky_pl> Under stock Ubuntu from Windows Store.
altendky has joined #pypy
<Dejan> I've just tried the pypy-c-jit-98354-1608da62bfc7-linux64.tar.bz2 on Fedora 31
<Dejan> works pretty fine
<squeaky_pl> I will probably build portable ones one last time with some differences not to trigger any regression for the users and point out in the repo that official ones are now portable.
adamholmberg has joined #pypy
<squeaky_pl> Dejan, does `python -m idlelib.idle` work fine?
<squeaky_pl> Given that you have X installed or something that speaks X protocol
<Dejan> squeaky_pl, yep, works pretty fine
<Dejan> erm... i was too fast
<Dejan> _tkinter.TclError: named font "{DejaVu Sans Mono} 10 normal" already exists
<Dejan> fails whenever something should be printed out
<Dejan> i tried changing the font - same exception
<squeaky_pl> Ok I reproduced as well on Ubuntu: https://imgur.com/a/Cfo2yh3
<squeaky_pl> my builds have the same problem, I wonder what's that
<squeaky_pl> it pretty looks like some problem in idle itself
<Dejan> CPython one works as expected
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
<mattip> squeaky_pl: thanks for pointing the way for the portable builds
ronan has quit [Remote host closed the connection]
<mattip> indeed, it seems the pypy.org website did not update the site
<Dejan> I wonder sometimes would it be better to have separate cffi packages depending on whether one wants ABI (basically wrapper around dynamic lib loader) or API
<Dejan> the API one seems more heavy as it needs to parse C code
ronan has joined #pypy
<mattip> the page here seems OK, I guess I have to talk to the python-infra people
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Read error: Connection reset by peer]
adamholmberg has joined #pypy
shane has quit [Quit: http://quassel-irc.org - Chat comfortably. Anywhere.]
dnshane has joined #pypy
squeaky_pl has quit [Quit: Leaving]
dnshane has quit [Quit: http://quassel-irc.org - Chat comfortably. Anywhere.]
dnshane has joined #pypy
jvesely has joined #pypy
<kenaan> rlamy py3.6 f574e8884bed /pypy/: Don't swallow the UnicodeDecodeError in one corner case (fixes issue #3132)
jvesely has quit [Quit: jvesely]
<mattip> I forced the website www.pypy.org to update
<mattip> now it is showing the downloads for the release
<arigato> just a random note about arm32: it would be worth to have a buildbot that we use only during release time manually (kind of like I should maybe turn the ppc into such a buildbot too)
adamholmberg has quit [Remote host closed the connection]
<mattip> maybe if we split the buildslave tasks into translation and upload; and tests. That would mean we don't need to run hours of tests that no-one looks at
<mattip> like we used to do for the arm32 cross-translations
<mattip> it might be good to do that in general, then we can make sure the post-translation tests run in a clean environment
<mattip> and would mean we could use travis/azure pipelines for testing since we would not exceed the time limits anymore
<arigato> Dejan: you're wrong, the API version of cffi is more efficient at runtime, whereas the ABI version parses C-like code at runtime
<arigato> in general if we can assume that we have a compiler at installation, there is no reason to use ABI
<tumbleweed> what's the process to add one?
<mattip> "add one" ?
<tumbleweed> buildslaves
<mattip> basically pip install buildbot-slave, "$buildslave create-slave ...",
<mattip> tell use your preferred slave name and password and tasks you wish it to participate in
<tumbleweed> what's most useful to you? manylinux docker slaves?
<mattip> the only thing we are really missing is arm32
<mattip> truthfully, the only one that really gets attention is linux x86_64, all the others we only really look at if there are big problems
<mattip> you can see that if you look at the number of failing tests on pypy2
<mattip> and pypy3
jcea has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
<kenaan> mattip default 5bf1495559a4 /: move build_cffi_imports
<kenaan> mattip py3.6 69267325e0a5 /: merge default into py3.6
mattip has quit [Ping timeout: 265 seconds]
mattip has joined #pypy
<bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-linux-s390x/builds/1218 [mattip: force build, release-pypy2.7-v7.x]
trinityhex has quit [Ping timeout: 258 seconds]
trinityhex has joined #pypy
Rhy0lite has quit [Quit: Leaving]
jcea has quit [Quit: jcea]
webmeister_ is now known as webmeister
trinityhex has quit [Ping timeout: 258 seconds]
created has joined #pypy
adamholmberg has joined #pypy
<antocuni> mattip: sorry for the pypy/tool/cpyext issue :(
<created> Hi.
<created> How do I try compiling pypy again if it failed at the compilation stage?
<created> I tried going to the temp directory and doing nmake, but it doesn't seem to have the right environment and fails early
<created> Oh false alarm - it has the right environment once I setup the externals right
adamholmberg has quit [Remote host closed the connection]
<created> I guess you're all off having christmasses, so you won't see this anyway
* cfbolz waves
<cfbolz> created: much more likely to be a timezone problem, most of us are europeish
<cfbolz> (timewise)
<created> I'm trying to figure out why vmprof doesn't run on windows
<created> Do you know the codebase and can help me a bit?
<created> I'm trying to understand where __vmprof_eval_vmprof is supposed to come from
<cfbolz> sorry, I'm a) not a windows person and b) it's 11:30pm here
<created> It's used in rvmprofc, but not defined or declared anywhere, except as some kind of decorator in rvmprof.py (but I don't quite see how it would end up exporting itself)
<created> I'm actually looking for the lunix perspective, but it being late for you is a different story
<created> (It's even later for me, but that's another story :P)
<created> Perhaps someone else shall see this
<cfbolz> probably tomorrow only