antocuni changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "PyPy: the Gradual Reduction of Magic (tm)"
danieljabailey has quit [Quit: ZNC 1.6.5+deb2build2 - http://znc.in]
<kenaan>
amauryfa py3.6 dda99c39353f /pypy/module/_sre/interp_sre.py: CPython Issue #29444: Add array bound check in group(), because the underlying buffer is mutable. Difficult to tes...
<kenaan>
amauryfa py3.6 5e9a2c6daf17 /pypy/module/_sre/: re.Match.group() now accepts index-like objects.
<kenaan>
amauryfa py3.6 0e57c6df8538 /pypy/module/_sre/: CPython Issue #24454: Regular expression match object groups are now accessible using __getitem__. "mo[x]" is equ...
<kenaan>
amauryfa py3.6 77d216c5b248 /pypy/module/_sre/: CPython Issue #28727: re.Pattern objects created by re.compile() become comparable (only x==y and x!=y operators)
<kenaan>
amauryfa py3.6 3f28eff2fcfc /pypy/module/_random/: Since Python3.6, the seed() call to urandom() has been moved to the _random module.
<antocuni>
ouch: am I wrong or rpython tests are NOT run nightly nowadays?
<fijal>
arigato: the windows pypy3 does not work at all
<fijal>
it starts, but does not import ssl
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 339c7996dc19 /: close merged branch
<kenaan>
antocuni default c30916ebe15f /: merge the fix-vmprof-stacklet-switch-2 branch, which fixes vmprof+greenlet: before, vmprof did not take any samp...
<bremner>
hmm. apparently I screwed up some fonts. oh well.
<arigato>
no, I'm talking about the standard visual studio
<bremner>
ok. no experience with that.
<arigato>
yay, I got 'Europe de l\x92Ouest' in a test now
<arigato>
ah, _init_timezone() of course
<Rotonen>
visual studio code does not really have most of the static analysis tools from the ms ecosystem integrated all too well into it, so visual studio proper is still the way to go for something like pypy
<Rotonen>
nor the debugger for that matter either
<arigato>
thanks. I'll try again later to open pypy in visual studio, maybe this time it won't crash
<arigato>
for now just opening files instead of the whole project seems to work well enouogh
<arigato>
(I guess I'm loosing cross-file references)
<Rotonen>
i'm planning on taking a stab at the windows stuff over the holidays
<arigato>
cool
<arigato>
I'm "only" fighting py3.5 in windows
<Rotonen>
if youcpython windows builds have
<Rotonen>
pff
<LarstiQ>
Rotonen: 24-26, or till 2018? :)
<Rotonen>
if you're having trouble symbolicating your stack traces, you'll have probably bumped into the fact that cpython on windows hardcoded the paths for where it looks for the source code
<arigato>
well, as I said, I stopped trying because VS crashed on me
<fijal>
the windows buildbot is in a bit of disarray
raynold has quit [Quit: Connection closed for inactivity]
<Rotonen>
arigato: it could be you do not have the correct cpython source extracted out to c:\build\cpython\cpython35 and intellisense trips up on itself due to that
<Rotonen>
hmm apparently just c:\build\cpython35
amaury has joined #pypy
<kenaan>
arigo default d5ba3ff15b4c /rpython/: Add FormatMessageW() to get the errors in unicode
<kenaan>
arigo py3.5 72a8c93b5914 /pypy/: Use FormatErrorW() instead of the bogus space.newtext(FormatError()), because FormatError() does not return utf8 at all
<kenaan>
arigo py3.5 b0e0af09b762 /rpython/rlib/: More Windows compatibility to return unicode error messages
<kenaan>
arigo py3.5 20fdafb25341 /pypy/: Fixes for unicode error messages
<arigato>
my guess is that we didn't care about it, and so the buildbot builds pypy3.exe with the same old Visual Studio version
<LarstiQ>
has at least some pypy3 stuff
<fijal>
ok, but that's a bit wrong
<arigato>
"Setting up VS for building ssl on python 3"
<fijal>
ah, python3, not python 3
<fijal>
this is all nonsense and it also does not work, because I don't have a very old version of VS
<arigato>
you usually need the same version of VS, yes
<arigato>
if we want to update the VS we use for pypy3 specifically (which would probably be a good idea), then we need to do that, which is involved
<arigato>
e.g. on the buildslaves we need to make sure to have the correct version installed
<fijal>
ok
<fijal>
maybe we should do that
<arigato>
we can't pick a random version of VS
<arigato>
otherwise windows people will be unhappy because it's not the same version as CPython 3.N
<fijal>
no, but maybe using the same version as cpython will be ok
amaury has quit [Quit: Konversation terminated!]
<arigato>
I suggest that we pick exactly the same version as cpython uses, for exactly the same release number
amaury has joined #pypy
<fijal>
arigato: do you know how to find out what should be the name of the .pyd file btw?
<Rotonen>
cpython 3.5 is officially vs 15, but all that stuff works without issues in vs 17 as well
<fijal>
(like how is it constructed)
<arigato>
additional messes may come from a possible difference between the version for 3.5 and 3.6 and 3.7
<fijal>
let's pick the same one as 3.5
<arigato>
Rotonen: I've heard about it, but it is the case that they are now fully cross-compatible? like run a VS17 .pyd in a VS15 cpython or vice-versa
<Rotonen>
15 -> 17 works, never tried the other way
<arigato>
how the filename is constructed: look in distutils/, I suppose?
<Rotonen>
what's the target audience of that project file - mostly ~5 people from this channel?
<fijal>
arigato: ok, so importlib.machinery.EXTENSION_SUFFIXES agree
<fijal>
but the module still does not import (none of the cffi modules), I wonder why
<fijal>
how do I open a .pyd file directly with cffi?
<arigato>
ffi.dlopen(), if you just want to see if it works
<arigato>
it's unlikely to be usable
<fijal>
right
<fijal>
well, I want to see if "ImportError" is something deeper or not
<fijal>
ok, I can dlopen it but it has no exported symbols
antocuni has joined #pypy
<fijal>
dir(ffi.dlopen(...)) should give me something more than [] right?
<arigato>
no
<fijal>
ok?
<fijal>
so how do I list the symbols?
<arigato>
unless you said ffi.cdef("") first
<arigato>
I don't know how you can list the symbols
<arigato>
there are some non-portable ways to do that, I think
<arigato>
certainly nothing built into cffi
<fijal>
ok, I got it from VS
<fijal>
so it has PyInit__lzma_cffi
<arigato>
and that's it I guess
<fijal>
and _cffi_pypyinit__lzma_cffi
<arigato>
as expected
<fijal>
arigato: do you have any clue where is the code loading that?
<arigato>
$you mean
<fijal>
(code that's supposed to call cffi and do stuff when I type "import _pypy_openssl")
<arigato>
pypy/module/_cffi_backend?
<fijal>
no, the importing part
<antocuni>
fijal: if you want the symbols inside an ELF, I suggest to pip install lief
<antocuni>
I have been playing with it lately, and it seems to be powerful and working good
<kenaan>
arigo py3.5 d79aaf54b0bc /pypy/module/imp/importing.py: Remove platform.machine() from the extension of the CPython- and CFFI-compatible dynamic libraries. I cannot figure ...
<arigato>
fijal: "import _ssl" still crashes though
<arigato>
but that's the next issue
<arigato>
File "C:\Users\arigo\pypy\pypysrc\lib_pypy\_cffi_ssl\_stdssl\__init__.py", line 23, in <module>
<arigato>
from select import poll, POLLIN, POLLOUT, select
<arigato>
ImportError: cannot import name 'poll'
amaury has joined #pypy
yuyichao has joined #pypy
amaury has quit [Quit: Konversation terminated!]
yuyichao has quit [Ping timeout: 252 seconds]
tayfun26 has quit [Remote host closed the connection]
adamholmberg has quit [Read error: No route to host]
adamholmberg has joined #pypy
Arfrever has joined #pypy
yuyichao has joined #pypy
mattip has joined #pypy
dcrosta has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Jellyg00se has quit [Quit: Leaving]
yuyichao has quit [Read error: Connection reset by peer]
<fijal>
yes, so do we do 5.9.1 like matti suggested or 5.10?
<fijal>
or 6.0?
* LarstiQ
is of the 5.10 camp
<arigato>
I think 5.10
<ronan>
definitely not 5.9.1
<arigato>
if we do a beta of pypy2 for cpyext, maybe it would also be a good idea to stick unicode-utf8 in the same beta, because of the same potential breakage
<arigato>
but not in pypy3
* fijal
makes it 5.10
<fijal>
arigato: I think cpyext & utf8 warrant 6.0
<fijal>
so we can do 6.0 beta
<arigato>
ok, so 6.0 beta for pypy2, and still 5.10 for pypy3 to make it clearer that it's behind for utf8?
* arigato
throwing up random suggestions, really
<fijal>
yeah
<kenaan>
fijal release-pypy3.5-v5.9.x cb3d0a043647 /pypy/: bump the versions
<cfbolz>
that sounds pretty confusing, honestly
<ronan>
arigato: I'm not sure what you're suggesting, but I'd rather not get into a state where we can't merge default into py3 for who knows how long
<fijal>
.....
<cfbolz>
shouldn't we just do a "boring" 5.10
<fijal>
yes we should
<cfbolz>
and then release again in february or so
<cfbolz>
or whenever the other things are ready
<fijal>
cfbolz: Im not sure if there is any point in releasing pypy2 5.10 though
amaury has joined #pypy
<arigato>
well, cpyext and utf8 are both kind of ready, but not well-tested
<arigato>
and indeed, with none of these two features, there isn't much news in pypy2
<cfbolz>
arigato: is utf8 ready? eg the integration into py3 is still going to be quite a chunk of work
<fijal>
it all requires work
<arigato>
yes, but it's roughly ready for pypy2
<arigato>
I see ronan's point about merging
<arigato>
maybe merge default into unicode-utf8 and make that the beta release?
<fijal>
how about we do nothing about pypy2 or utf8 and just do the 3.5 release?
<fijal>
(as a start)
<cfbolz>
+1
<fijal>
and then we can think about merging
<arigato>
yes, they are two independent questions so +1
glyph has quit [Quit: End of line.]
* arigato
off
realitix_ has quit [Quit: realitix_]
glyph has joined #pypy
<LarstiQ>
fijal: the point of pypy2 5.10 is to keep it unconfusing and predictable which versions exist for pypy2/pypy3
<LarstiQ>
so personally I'd do boring 5.10 for pypy2/pypy3, and then move both on to 6.0beta
<cfbolz>
tend to agree
<cfbolz>
(but then, I'm not the one doing the work)
<fijal>
yes, fair, but also it's a but of work for no real reason
<fijal>
a bit of work
<fijal>
let's do it that way - I'll try to release 3.5 and see if there is time left
<LarstiQ>
I'm willing to volunteer some hours this month
<fijal>
great, that would definitely make it easier
<fijal>
should I write in the release announcement that it's a boring release, stuff works (more or less) and what is needed?
<fijal>
we should at the very least try to fix installing packages on windows, but I can't seem to be able to get the dev env going at all
* fijal
tries 32bit windows in a vm
<mattip>
I am +1 for a 5.10 pypy2 and pypy3, just to keep them synced
dcrosta has joined #pypy
<mattip>
there is also a small tkinter fix for matplotlib, if anyone ever merges my matplotlib tkagg-cffi branch then it will just work
<mattip>
on 5.10
<mattip>
fjial: where is the issue getting the devenv going?
<fijal>
getting vcvarsbat working
<fijal>
and then some 64 vs 32 bit issues
<fijal>
anyway, I'll install 32bit windows on a VM don't want to care right now
<mattip>
I tweaked platform/windows.py so vcvars should be found :(
<mattip>
and the buildbots both are running on 64 bit windows
jamesaxl has joined #pypy
<fijal>
mattip: yes, fine, but my pypy refuses because sys.machine or sys.architecture or something returns 64bit
<fijal>
so platform does not pick it up
<fijal>
mattip: my point being that while might be a genuine issue, fixing issues on my specific platform is not getting me anywhere
<Cuddlesby>
The loop appears to be doing the same thing, but results in an error.
<Cuddlesby>
If I set the stack limit to 10000 before setting it to anything lower first, it crashes.
<cfbolz>
Cuddlesby: it's not safe to set the recursion limit arbitrarily high (same in CPython) at some point you get crashes
<cfbolz>
you run out of stack space and start writing stack frames into unrelated memory
<Cuddlesby>
cfbolz: Of course, but ulimit reports I have ~8MB stack. And from the docs, 768B is required per call, so that gives me 10922 calls supposedly
<cfbolz>
which docs?
<cfbolz>
cpython and pypy require different amounts of stack per call (and in pypy it also depends on the JIT, so it's not even just a simple number)
<Cuddlesby>
It also doesn't explain why it fails with only setrecursionlimit(10000); f(0) but is fine in the example
<cfbolz>
it even says that that number is very approximate only
<cfbolz>
anyway, what are you trying to achieve?
<Cuddlesby>
cfbolz: Trying to raise the stack limit to handle a set amount of recursion
<Cuddlesby>
On Linux I can use ulimit/resource module but what about Windows?
<Cuddlesby>
I've considered spawning a thread after setting threading.stack_size
<simpson>
Cuddlesby: If this is to fix a production problem, then the correct fix is to rewrite your recursive code to either be iterative with a reified stack, or to use a trampoline/memoizer/etc.
<Cuddlesby>
simpson: I would usually agree with you but that isn't an option in this case
<simpson>
It's been an option every other time I've encountered this problem. What are you building?
<Cuddlesby>
simpson: I know, in fact I already did such a rewrite but it doesn't fit elegantly and it's slow with no chance of speeding it up
dcrosta has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<Cuddlesby>
I essentally have to capture the state in a closure since the stack stores many different things
dcrosta has joined #pypy
<Cuddlesby>
Anyway, I do know what I'm doing. I just want deep recursion like Golang can do. But I want Python. Possible?
<simpson>
Is this an interpreter?
<Cuddlesby>
Kind of
<simpson>
Yes, there's a relatively straightforward mechanical transformation that you can do from closed recursion to open recursion, and then you can stick a trampoline into the open recursion. It's a fun afternoon's work to write the trampoline as a decorator.
<simpson>
It's not free. Python isn't a language for free arbitrarily-deep recursion. (In a certain sense, no language gets it entirely for free!)
<Cuddlesby>
Golang is close to free
<Cuddlesby>
But anyway, I know the stack depth I want. So I just need to know how to set that up
<simpson>
Doesn't Go do it with segmented stacks? So there's a cost, it's just covered for you by the runtime. Ditto with thread-local storage, etc.
<Cuddlesby>
I think Golang does stack copying now. And I said "close". There's as much cost to appending/popping from a list arguably
<simpson>
What's your Windows strategy? So far, I've found a way to alter python.exe to have a bigger default stack, and it seems like there's Win32 API too.
<Cuddlesby>
Like I said, threading.stack_size maybe
<simpson>
Have you tried it and seen whether it works yet?
<Cuddlesby>
Nope
<mattip>
win32 py3.5 can now load c-extension modules, but packaging fails, it tries to rebuild pyd which is being held (?maybe), and NTFS does not allow that