antocuni changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "PyPy: the Gradual Reduction of Magic (tm)"
yuyichao has quit [Quit: Konversation terminated!]
yuyichao has joined #pypy
jamesaxl has quit [Quit: WeeChat 1.9.1]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
pilne has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
jcea has quit [Remote host closed the connection]
mat^2 has quit [Ping timeout: 240 seconds]
antocuni_ has quit [Ping timeout: 265 seconds]
tbodt has joined #pypy
SunDwarf has quit [Remote host closed the connection]
marr has quit [Ping timeout: 255 seconds]
SunDwarf has joined #pypy
Civil has quit [Remote host closed the connection]
nimaje is now known as Guest94872
nimaje1 has joined #pypy
Guest94872 has quit [Killed (weber.freenode.net (Nickname regained by services))]
nimaje1 is now known as nimaje
marvin_ has quit [Ping timeout: 248 seconds]
Civil has joined #pypy
marvin_ has joined #pypy
pilne has quit [Quit: Quitting!]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
ArneBab has joined #pypy
ArneBab_ has quit [Ping timeout: 268 seconds]
astronavt has joined #pypy
tbodt has joined #pypy
dddddd has quit [Remote host closed the connection]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
dcrosta has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
nimaje has quit [Read error: Connection reset by peer]
tbodt has quit [Client Quit]
tbodt has joined #pypy
nimaje has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
Garen has joined #pypy
abrown has joined #pypy
nimaje has quit [Read error: Connection reset by peer]
nimaje has joined #pypy
dcrosta has joined #pypy
exarkun has quit [Ping timeout: 265 seconds]
exarkun has joined #pypy
tbodt has joined #pypy
tbodt has quit [Client Quit]
tbodt has joined #pypy
tbodt has quit [Client Quit]
astronavt has quit [Quit: Leaving...]
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
abrown has quit [Ping timeout: 260 seconds]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
tbodt has quit [Client Quit]
tbodt has joined #pypy
tbodt has quit [Client Quit]
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
tbodt has quit [Client Quit]
tbodt has joined #pypy
* arigato implements os.scandir() on windows
<pjenvey> and there was much rejoicing
<njs> not by arigato I'm guessing
<pjenvey> definitely not by arigato
<arigato> :-/
<arigato> pjenvey: also, the mmap-for-sharing branch won't be merged for the release
<arigato> it has a not-understood memory impact on running a full translation
<pjenvey> arigato: I saw it was increasing memory usage for translation. pretty weird
exarkun has quit [Ping timeout: 248 seconds]
<arigato> what is the executable "t32.exe" that distutils tries to locate when doing "python -mensurepip"??
<pjenvey> we could make it an optional gc env flag.. but le sigh
exarkun has joined #pypy
<arigato> uh, on windows when distutils installs an "executable script", it prepends a magic line containing "\full\path\to\t32.exe"
<arigato> apparently?
<pjenvey> seemingly some exe from https://bitbucket.org/vinay.sajip
<arigato> right, I actually find it in my installed C:\Python35-32\Lib\site-packages\pip\_vendor\distlib
<arigato> also in C:\Python27\Lib\site-packages\pip\_vendor\distlib
<pjenvey> IIRC setuptools had a similar exe for launching scripts, this must be the same but different
<arigato> but there's no way the same exe can invoke pypy instead of cpython
<arigato> how does it work on pypy2? I guess it doesn't
<njs> arigato: does pypy2 -m virtualenv give you a myvenv\bin\python like it does on posix?
tayfun26 has joined #pypy
<pjenvey> I think it's a hack that reads shebang lines to pretty much emulate posix behavior
<arigato> right now it prevents "pypy3-c.exe -m ensurepip" from completing
<arigato> I have no clue why, but "pypy-c.exe -m ensurepip" completes
<njs> is the README, and describes the .exe's behavior
<arigato> I'm fine with continuing to ignore this exe if we did so far, but I'd like to understand why ensurepip fails on pypy3
<arigato> waaaat
<pjenvey> yea it's a total hack
<arigato> if you do repeatedly "pypy3-c.exe -m ensurepip", the third time it works
<pjenvey> what does the shebang line in question look like I would first wonder
<njs> I have absolutely no idea what ensurepip even does or how it relates to the executable so
<arigato> well, doing "pypy{2 or 3
<arigato> well, doing "pypy{2 or 3}-c.exe -m virtualenv venv" fails on both pypy2 and pypy3
<njs> (I mean I know what it's supposed to accomplish, I just have no idea how it works)
<arigato> I guess it wouldn't be hard to add to the buildbot "try to run this series of commands"
<kenaan> arigo default c1c8d16890dd /lib-python/2.7/subprocess.py: Make virtualenv work on Windows too
<kenaan> arigo py3.5 fd74b9439261 /lib-python/3/subprocess.py: Port of c1c8d16890dd
<arigato> none of the cpython tests work because "import faulthandler" is not here on pypy3-c.exe
<arigato> yes, I know it doesn't work on Windows
<arigato> I didn't know that "import faulthandler" was done unconditionally inside the test suite
<pjenvey> yep
<kenaan> arigo py3.5 3c8c318aa8d4 /lib_pypy/faulthandler.py: Add a dummy faulthandler.py for the CPython test suite
<kenaan> arigo py3.5 cffba7a52fb2 /lib_pypy/_ssl/__init__.py: Add a dummy enum_certificates() function, for now
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
<dstufft> njs: basically the same thing as ``pip install pip``
<dstufft> njs: through some hacks
<njs> dstufft: ah cool
exarkun has quit [Ping timeout: 264 seconds]
exarkun has joined #pypy
mat^2 has joined #pypy
antocuni_ has joined #pypy
mat^2 has quit [Ping timeout: 265 seconds]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
kenaan has quit [Write error: Connection reset by peer]
Jellyg00se has joined #pypy
marr has joined #pypy
Garen has quit [Read error: Connection reset by peer]
Garen has joined #pypy
mattip_away has joined #pypy
<mattip_away> arigato: did you want to commit the scandir changeset?
antocuni_ is now known as antocuni
<antocuni> so apparently my fix fixed translation on s390x but not on ARM? How is it possible? :(
<antocuni> ah wait! How does translation on ARM work? Is it native or cross-translation?
<antocuni> I am checking the CPU by doing 'import platform; platform.machine() in ("i686", "x86_64")'
<antocuni> maybe I should use rffi_platform instead?
<arigato> mattip_away: yes, coming soon
<mattip_away> nice
<mattip_away> to both of you
<arigato> :-)
<antocuni> buildbot says the the builddir is "/home/buildslave/bot_cross_armel/", so it seems it's indeed a cross-compilation
<antocuni> the question now is: how am I supposed to check what is the CPU of the target?
<mattip_away> how does it choose a jit backend?
<antocuni> good question, /me looks
<mattip_away> —platform=arm as a translation arg
<antocuni> there is also jit.backend.detect_cpu, which seems to have logic to handle both cross and non-cross translations
<antocuni> although I have no idea how all this mess interacts with e.g. tests
<antocuni> we assume that this info is in the global state, while it is not at all :(
<mattip_away> tests are not run cross platform
<antocuni> yes, but it's still not a good reason for having this mess
<antocuni> anyway, I suppose it's too late to fix it now
mattip_away has quit [Remote host closed the connection]
kenaan has joined #pypy
<kenaan> antocuni default b1b0f51304ce /rpython/rlib/rvmprof/cintf.py: this method of determining the CPU is broken in presence of cross-translations (like we do e.g. on ARM). Try to ...
jacob22__ is now known as jacob22
<kenaan> arigo py3.5 4bb7cf3fd5a1 /: In-progress: os.scandir() on Windows
<kenaan> arigo py3.5 6deed1172830 /: Enough os.scandir() to pass our own tests on Windows
<kenaan> arigo py3.5 2bd764ec6d22 /rpython/annotator/binaryop.py: Obscure annotation fixes: propagate no_nul a tiny bit more
<kenaan> arigo py3.5 c45c28743766 /: Translation fixes
antocuni has quit [Ping timeout: 264 seconds]
exarkun has quit [Ping timeout: 264 seconds]
exarkun has joined #pypy
oberstet has joined #pypy
jcea has joined #pypy
AndrewBC has quit [Ping timeout: 265 seconds]
raynold has quit [Quit: Connection closed for inactivity]
antocuni has joined #pypy
<antocuni> cool, it seems that ARM translation works again
<antocuni> mattip: ^^^
<arigato> yay
<antocuni> arigato: sometimes I think that we should switch to a more "configure-like" system for rpython
<antocuni> in which we do a configure step which gathers all the info about the target platform (and store them e.g. inside a config.py)
<antocuni> but I know that it's probably a huge project which nobody feels like to do :)
<arigato> yes
<antocuni> since we are talking about ambitious projects which will never be done, another possibility would be to do something like cython, in which the C code produced is always the same, and the platform-specific parts are enabled using #ifdefs
<arigato> ronan: ah, right
<ronan> having the arch in the tag is linux-specific in CPython
<arigato> it's just that the change was wrong
<arigato> ah, ok
<arigato> but does platform.architecture() always return the right thing on linux?
<ronan> not sure
<ronan> probably not
<ronan> the definitions are irregular, they come from configure.ac
<arigato> sorry, .machine()
<arigato> indeed, no
<arigato> platform.machine() returns 'x86_64' even in a 32-bit chroot
<antocuni> arigato: are you talking about pypy or cpython?
<arigato> testing with CPython 3.x
<antocuni> because on the 32-bit chroot which we have on tannit, platform.machine() returns 'i686' for both cpython2.7 and pypy 5.0
<ronan> platform.machine() calls uname
<arigato> right, it's not actually a chroot I'm testing here
<arigato> just the 32-bit version of python that comes with arch linux, I think
<arigato> which lives not in a chroot
<ronan> well, platform.machine() doesn't know anything about its own executable
<arigato> it's rather sad, the failing test you pointed to shows that it's possible for 32-bit libraries to end in "x86_64-linux"
<arigato> which imho defeats the point completely
<ronan> huh, yes, "x86_64-linux-gnux32.so"?!?
<arigato> ah
<arigato> sorry, missed the x32 part
<arigato> so there is still a point, and we should still do the same
<arigato> (we can ignore x32 as an unsupported platform)
* arigato fixes
<arigato> how is it on windows, btw?
<arigato> do the .pyd also get two names depending on 32- or 64-bit?
<ronan> is uses a completely separate logic
<ronan> *it
<ronan> EXT_SUFFIX is just .pyd
<arigato> ah
<fijal> arigato: I wrote the release announcement. Should I wait for something/help with something?
<arigato> we're busy fixing details
<fijal> I guess I can create the 2.7 release branch (and bump the versions etc.)
<fijal> I can see that, that's why I'm asking :-)
<arigato> this particular detail will be fixed in 10 minutes, but I have no clue about how many others there are :-)
<fijal> heh
<fijal> is this the next step to try to get ssl running?
<arigato> I think that ssl should work already
<arigato> ensurepip crashes, but if you just have to try again two more times: at the third time it works
<arigato> I know what the problem is but I have no clear clue about how to fix it
<arigato> it's inside code that gets downloaded
<kenaan> arigo py3.5 92a3e28a4aef /pypy/module/imp/importing.py: Another attempt at this logic
dddddd has joined #pypy
<kenaan> arigo py3.5 1dd2914caff4 /pypy/module/imp/test/test_import.py: fix test
<kenaan> rlamy py3.5 def83ef09f31 /pypy/module/posix/: listxattr returns a list of str, not of bytes
dddddd has quit [Ping timeout: 240 seconds]
<arigato> ah argh
<fijal> pffff
<fijal> (about the ensure pip crashing twice)
<fijal> arigato: should we file a bug report?
<fijal> I'm sure we can get it sorted quickly
adamholmberg has joined #pypy
<arigato> I don't know what the correct thing is
<fijal> "not crash" :-)
<arigato> I'd need to investigate what the "t32.exe" executable really is
<arigato> and how it is used
<arigato> for now, yes, we can suggest that if the "t32.exe" executable is not found, it falls back to doing something sane instead of crashing with an obscure AttributeError
<arigato> or TypeError
<Rotonen> IIRC those are distlib glue to adhere to PEP 397
<fijal> arigato: that's unfortunately the name of the game for those tools
<kenaan> arigo py3.5 934eabf72b5c /pypy/module/imp/importing.py: Still use machine.platform() to know if we're on a x86 or not at all
<fijal> ideally it would not crash like that at all, but for now maybe "it does not crash if you do everything exactly right" is a start
<arigato> fijal: easy: "don't use windows"
<arigato> Rotonen: thanks a lot
<arigato> so we should ideally make the same t32.exe (etc.) executables
<arigato> that would load libpypy-c.dll instead
dddddd has joined #pypy
<Rotonen> arigato: the whole thread is a bit tiresome to read, but does go into some depth on those
<arigato> I'd love to answer "so, not now"
<fijal> arigato: :]
<arigato> that's really functionality that we can report to the future, but ensurepip fails
marky1991 has joined #pypy
<kenaan> rlamy py3.5 0c9668c522b1 /pypy/module/posix/: Fix error reporting from posix.*xattr functions
<arigato> Rotonen: can you help me locate the source code for t32.exe? I'm failing
<Rotonen> i'd suppose it's the visual studio projects from vinay sajip, iirc somewhere on bitbucket or google code or somesuch
<arigato> ah, it's included as already-compiled binary inside the file ./Lib/ensurepip/_bundled/pip-7.1.2-py2.py3-none-any.whl
<arigato> that's great
<LarstiQ> <OutputFile>dist\t32.exe</OutputFile>
<arigato> ok then I have no clue why t32.exe is not simply unpacked from that .whl like it is on CPython (it wouldn't work but nobody tries to run it so far)
<kenaan> rlamy py3.5 e5552dd110c1 /pypy/doc/whatsnew-pypy3-head.rst: document branch
<ronan> numpy still works (with 28 failures, but they're all known issues)
yuyichao has quit [Ping timeout: 248 seconds]
demonimin has quit [Ping timeout: 276 seconds]
yuyichao has joined #pypy
<arigato> aaAaah trying to debug it but if I unpack the .whl to add a pdb.set_trace(), then it works fine
dcrosta has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
tayfun26 has quit [Quit: tayfun26]
<antocuni> arigato: you can start your program using python -m pdb foo.py
<arigato> ok it's because somehow we look for 'pip/_vendor/distlib\t32.exe' in the zip, but the index only contains '\'-separated names
<antocuni> then you get a pdb prompt beofe the program starts
<arigato> antocuni: haha naive
<arigato> :-)
<antocuni> and from there you can use the "break" command to set a breakpoint
<arigato> of course somebody catches the exception to display it nicely
<arigato> antocuni: but thanks anyway for the reminder
<antocuni> np
<antocuni> arigato: but the exception is somehow not displayed?
<arigato> no, it is
<arigato> but in red
<arigato> narrowing the problem to zipimporter
<ronan> arigato: ah, ISTR that when I last hacked on zipimporter, I thought "windows makes my head hurt, I'll just ignore it"
<ronan> (sorry)
<arigato> :-)
<arigato> it's because python files inside a zip have a __file__ with '/' instead of '\'
<arigato> on pypy3 only, on pypy2 it works
demonimin has joined #pypy
<antocuni> arigato: about 934eabf72b5c: hasn't it got the same problem I hit earlier today? I.e., that cross-compiled ARM builds will be confused
<arigato> right, you have a point
<arigato> what's the solution?
<antocuni> I used jit.backend.detect_cpu.autodetect()
<antocuni> it seems to have logic to handle cross-compilation
<antocuni> although I didn't dig to understand how it works
<antocuni> I wonder whether we should move e.g. to rffi_platform, and ensure that we use it everywhere
<arigato> could you fix this too? I'm digging on a different machine right now
<antocuni> arigato: I fixed it blindly and then started the buildbot, I don't have a phisical machine where to try it
<arigato> ...test_zipimport fails on windows on py3.5 precisely because that
<arigato> antocuni: same here
<antocuni> :-/
<arigato> though I think I installed an arm virtual machine some time ago (somewhere else)
<antocuni> I *think* that what buildbot does is to run translation on a normal x86 CPU, but using a cross-compiling SDK
<antocuni> so, no need for an arm virtual machine
<arigato> then, I don't have the cross-compiling SDK I fear
<antocuni> try to login to the buildslave, maybe?
<antocuni> "Cross Translation builder for ARM softfp ARMv7 builds"
<antocuni> no idea how to ssh into it, though
<arigato> same here
<kenaan> arigo py3.5 5a865516f947 /pypy/module/zipimport/interp_zipimport.py: Fix for test_package. This might fix the next issue for "pypy3 -m ensurepip".
<kenaan> arigo py3.5 d8113c2982af /pypy/module/zipimport/interp_zipimport.py: merge heads
realitix has joined #pypy
Jellyg00se has quit [Quit: Leaving]
<kenaan> arigo py3.5 50aa133f3ee9 /pypy/module/posix/: Test and fix (for lib-python/3/pathlib): 'os.symlink' must exist, even if it can raise NotImplementedError on Windows
<kenaan> rlamy py3.5 74f1be327b1d /pypy/: Cache the wrapped code.co_filename: space.newfilename() is expensive
tbodt has joined #pypy
marky1991 has quit [Remote host closed the connection]
realitix has quit [Quit: realitix]
marky1991 has joined #pypy
marky1991 has quit [Ping timeout: 272 seconds]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
mattip has joined #pypy
marky1991 has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
marky1991 has quit [Ping timeout: 264 seconds]
<ronan> pff, there's a segfault in the pandas test suite, but that's because they call PyBytes_AS_STRING on a str
<ronan> also, pandas master doesn't compile, but the last release works
marky1991 has joined #pypy
<mattip> ronan: is that from cython?
<ronan> no, it's C code that was badly ported to py3 AFAICT
jamesaxl has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
antocuni has quit [Ping timeout: 260 seconds]
<tos9> I like the "at the moment" in the exception message
<tos9> Clearly the TypeError version of a "hang up and try again later"
<mattip> nice. The compilation problem seems to be an abuse of PyDateTime_* macros, right?
<ronan> yes, something like that
oberstet has quit [Ping timeout: 256 seconds]
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
tbodt has quit [Client Quit]
oberstet has joined #pypy
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
<pjenvey> 23
tbodt has quit [Client Quit]
<kenaan> mattip cpyext-avoid-roundtrip 5db9d141c474 /pypy/module/cpyext/src/object.c: quiet warnings
<kenaan> mattip cpyext-avoid-roundtrip f428c66de212 /pypy/module/cpyext/: test, fix PySequence_GetItem allows negative indices
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/5173 [mattip: force build, cpyext-avoid-roundtrip]
<mattip> fijal: if default builds pass, I will create a release branch for py2.7-v5.10
mattip has left #pypy ["bye"]
marky1991 has quit [Ping timeout: 264 seconds]
marky1991 has joined #pypy
raynold has joined #pypy
dowwie_ has joined #pypy
dowwie_ has quit [Read error: Connection reset by peer]
dowwie has joined #pypy
tbodt has joined #pypy
<bbot2> Success: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/5173 [mattip: force build, cpyext-avoid-roundtrip]
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
<kenaan> rlamy py3.5 9ae4bd0c4555 /pypy/module/cpyext/: Fix 'errors' arg in PyUnicode_Decode()
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
jamesaxl has quit [Quit: WeeChat 1.9.1]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
oberstet has quit [Ping timeout: 240 seconds]
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
tbodt has joined #pypy
adamholmberg has quit [Ping timeout: 256 seconds]
mat^2 has joined #pypy
<njs> re: accurately detecting platform: pip has some hard-won defensive coding here, including handling of 32-bit chroots: https://github.com/pypa/pip/blob/7bb73f60ad44ce31bb69c8d21b03dd4ee1ee7682/src/pip/_internal/pep425tags.py#L113-L140
<njs> obviously this is stuff that should have migrated back upstream into distutils and platform, but they don't find it any easier to fix the stdlib than you do...
guilherme has joined #pypy
guilherme has quit [Client Quit]