antocuni changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "PyPy: the Gradual Reduction of Magic (tm)"
amaury has quit [Ping timeout: 255 seconds]
drolando has joined #pypy
drolando has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
pilne has joined #pypy
drolando has joined #pypy
drolando has quit [Read error: Connection reset by peer]
<guilherme>
Hi everyone, I'm new to rpython. Is there a way to speedup the translating process? Currently, the translator is taking almost 84 secs in my computer with simple programs.
<simpson>
guilherme: No, that's actually a pretty fast translation. Which language are you implementing?
<simpson>
Nifty. So, as a couple bullet points, PyPy implements Python entire, and it can take 40min or more to build, with quite a lot of RAM usage. Typhon implements Monte, a JS-sized language, and that builds in about 5min without a JIT or 15min with a JIT on my laptop.
ArneBab has joined #pypy
ArneBab_ has quit [Ping timeout: 240 seconds]
<guilherme>
Wow! And the build is done incrementally? Like C/C++ where you only rebuild the files that changed? Or in every change you need to rebuild the entire project?
<simpson>
No, it's whole-program translation, so full rebuild.
jcea has quit [Remote host closed the connection]
exarkun has joined #pypy
<simpson>
But most of the PyPy tests can be run untranslated, and that's the recommended design.
jcea has joined #pypy
<guilherme>
Ok, I'm using the ebnfparse that lives under rpython.rlib.parsing! Do you know if it's possible to use it with python?
<guilherme>
by python I mean a regular python interpreter
<simpson>
Pretty much all RPython code should work fine untranslated, yes.
<simpson>
You should know exactly what RPython does when it translates. First, it imports your program like normal Python code. Then, it looks at what's been imported into memory, and translates *that*.
<guilherme>
:q!
<guilherme>
sorry :)
jcea has quit [Remote host closed the connection]
guilherme has quit []
guilherme has joined #pypy
dddddd has quit [Remote host closed the connection]
guilherme has quit [Remote host closed the connection]
<kenaan>
rlamy py3.5 e170f5f30e32 /pypy/objspace/std/test/test_celldict.py: Add failing test that explains the failure in CPython's test_unencodable_filename()
jamesaxl has quit [Read error: Connection reset by peer]
asmeurer__ has quit [Ping timeout: 248 seconds]
jamesaxl has joined #pypy
Jellyg00se has joined #pypy
antocuni has joined #pypy
marr has joined #pypy
Cheery has quit [Remote host closed the connection]
exarkun has quit [Ping timeout: 256 seconds]
exarkun has joined #pypy
<cfbolz>
antocuni: ping?
<cfbolz>
antocuni: I was wondering, what's the status of the cpyext branches?
<arigato>
I should review and merge at least one of them
<antocuni>
I think there is only one which is active
<antocuni>
which is our capetonian friend, cpyext-avoid-roundtrip :)
<cfbolz>
ok, would be cool to do that before the release
<cfbolz>
right
<arigato>
yes
<antocuni>
the other branch I worked on was cpyexct-method-refactoring, but I already merged it inside avoid-roundtrip
<cfbolz>
antocuni: I had an idea how to make passing arguments to C a bit cheaper, if I implement it would you be able to take a look (trying not to add more to Armin's load)
<antocuni>
yes, sure
<antocuni>
what's the idea?
<cfbolz>
antocuni: right now passing a w_obj to C requires a dict lookup for non-ints, to get the corresponding pyobj, right?
<antocuni>
yes
<cfbolz>
I'd like to improve that for instances of C-defined classes, by storing a reference to the pyobj directly in the w_obj
<cfbolz>
we can't do it for everything, since it would make all objects bigger
<antocuni>
right
<antocuni>
we already discussed something like this with arin
<antocuni>
armin
<cfbolz>
ok
<antocuni>
there is a doc somewhere in extradoc I think
<cfbolz>
where?
<antocuni>
planning/cpyext.txt
<antocuni>
the bulleted point about w_None
<antocuni>
note that by looking at callgrind, this dict lookup never showed up as a bottleneck, because the rest was so awfully slow to hide it :)
<cfbolz>
antocuni: my hacked together Version helps quite a bit on the second half of the benchmarks
<cfbolz>
Those that do method calls
<antocuni>
cool
<cfbolz>
And right, I see the points about being a bit more general for None etc
<arigato>
after improving the horribly slow things, other points stick out. easier to not improve the horribly slow stuff...
<cfbolz>
arigato: haha
<antocuni>
and probably W_TypeObject is the obvious next candidate; it is also already big, so adding an extra cpy_object field doesn't cost much
<cfbolz>
antocuni: right, but is it passed to C a lot?
<antocuni>
I think so, but I no longer remember why :)
<fijal>
Could not fetch URL https://pypi.python.org/simple/perf/: There was a problem confirming the ssl certificate: Can't connect to HTTPS URL because the SSL module is not available. - skipping
<fijal>
ok, how do I build pypy3 with ssl support?
<arigato>
on os/x I guess
<arigato>
who knows
<fijal>
no, baroquesoftware
<fijal>
so linux
<fijal>
well maybe ronan or mattip know
<fijal>
from _pypy_openssl import ffi
<fijal>
ImportError: No module named '_pypy_openssl'
<arigato>
last I tried, no problem
<fijal>
ok, that's a bit why
<arigato>
but that's a long while ago
<fijal>
mattip: ^^^
<arigato>
try to run the build script
<arigato>
and see if and how it explodes
<fijal>
how do I do that?
<arigato>
uh? look for lib_pypy/somethingssl/build.py
<arigato>
it's standard cffi
<fijal>
ah
<fijal>
arigato: no, it's the virtualenv that does not copy the _ssl module together with the rest
dcrosta has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
guilherme has joined #pypy
guilherme has quit [Ping timeout: 264 seconds]
<arigato>
fijal: maybe you should have worded your comment to https://github.com/pypa/virtualenv/pull/1103 like this: it clearly changes nothing for CPython, so if it breaks anything it's at most PyPy
<mattip>
we should rewrite our post-translations tests as a script that downloads the tar.gz and runs tests,
<mattip>
then use it on all the buildbots which would unify the ARM testers with the other ones
tbodt has joined #pypy
tbodt has quit [Client Quit]
<mattip>
so the buildbot would call this script which would live in the repo
<mattip>
and could be tested offline
dcrosta has joined #pypy
tbodt has joined #pypy
tbodt has quit [Client Quit]
tbodt has joined #pypy
<mattip>
ahh, ok, that ensurepip error is an old one from before I did some fixes, whew, I thought I was going crazy
mat^2 has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
tbodt has quit [Client Quit]
kenaan_ has joined #pypy
<kenaan_>
mattip buildbot 46954035f3c6 /bot2/pypybuildbot/builds.py: install virtualenv HEAD to get win32 fix from pypa/virtualenv pr 1103
asmeurer__ has joined #pypy
tbodt has joined #pypy
tbodt has quit [Client Quit]
dcrosta has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
dcrosta has joined #pypy
adamholmberg has quit []
tbodt has joined #pypy
mentalita has quit [Quit: Leaving]
mentalita has joined #pypy
tbodt has quit [Client Quit]
adamholmberg has joined #pypy
kipras`away is now known as kipras
amaury has joined #pypy
tbodt has joined #pypy
antocuni_ has quit [Ping timeout: 263 seconds]
<kenaan_>
cfbolz cpyext-faster-arg-passing 85fea167b9ea /pypy/module/cpyext/: speed up passing some objects to C specifically, passing the instances of classes defined in C ...
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
tbodt has quit [Client Quit]
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
dcrosta has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
tbodt has quit [Client Quit]
<kenaan_>
rlamy py3.5 da0e3f790bdb /pypy/objspace/std/celldict.py: Fix handling of unencodable strings in ModuleDictStrategy
<dmalcolm_>
FWIW: "But the real win was switching to PyPy rather than CPython as the Python interpreter - it turns out this is exactly the kind of job load for which their JIT compilation shines." https://gcc.gnu.org/ml/gcc/2017-12/msg00076.html
amaury has quit [Ping timeout: 272 seconds]
forgottenone has quit [Quit: Konversation terminated!]
<cfbolz>
oh no, not him :-(
aboudreault has joined #pypy
<dmalcolm_>
(sorry)
amaury has joined #pypy
bbot2 has quit [Quit: buildmaster reconfigured: bot disconnecting]
bbot2 has joined #pypy
<kenaan_>
rlamy py3.5 b3b1beda9224 /: Give up on trying to get test_multithreaded_import() to work untranslated
<ronan>
^^^ If anybody understands why the test fails on CPython 3.5, please tell
<mattip>
cfbolz: it would be nice to devise an objective measure for the "goodness" of cpyext-faster-arg-passing
<mattip>
in micronumpy we had tests that looked at traces
<cfbolz>
mattip: ah, you mean as a test?
<mattip>
yes
<cfbolz>
I was aiming for having a benchmark that shows the improvement. The traces of cpyext are a bit too messy to see anything in them, at this point
<mattip>
but those tests were very slow, so I am not sure I want too many of them in cpyext
<mattip>
+1 for benchmark
<cfbolz>
if we add a bench to speed, we can track (at least very big) slowdowns