cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
BPL has quit [Quit: Leaving]
Ai9zO5AP has quit [Quit: WeeChat 2.5]
CrazyPython has joined #pypy
zmt01 has joined #pypy
zmt00 has quit [Ping timeout: 245 seconds]
dnshane has quit [Quit: http://quassel-irc.org - Chat comfortably. Anywhere.]
_whitelogger has joined #pypy
dnshane has joined #pypy
inhahe has quit [Read error: Connection reset by peer]
inhahe has joined #pypy
CrazyPython has quit [Remote host closed the connection]
dpn` has quit [Quit: ZNC - http://znc.in]
dpn` has joined #pypy
jcea has quit [Quit: jcea]
andi- has quit [Remote host closed the connection]
andi- has joined #pypy
dddddd has quit [Remote host closed the connection]
ionelmc has quit [Quit: Connection closed for inactivity]
forgottenone has joined #pypy
simpson has quit [Ping timeout: 252 seconds]
altendky has quit [Quit: Connection closed for inactivity]
jvesely has quit [Quit: jvesely]
forgottenone has quit [Quit: Konversation terminated!]
dansan has quit [Ping timeout: 250 seconds]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
inhahe has quit []
inhahe has joined #pypy
<cfbolz> arigato: at hhu there is a vm running called pypybuildbot on hydra, do you have any memory what that was/is for?
forgottenone has joined #pypy
asmeurer__ has joined #pypy
Ai9zO5AP has joined #pypy
asmeurer__ has quit [Ping timeout: 246 seconds]
xcm has quit [Read error: Connection reset by peer]
xcm has joined #pypy
<cfbolz> arigato: seems david's arm cross-compiler
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
lritter has joined #pypy
dddddd has joined #pypy
lastmikoi has quit [Quit: ...]
lastmikoi has joined #pypy
YannickJadoul has joined #pypy
<cfbolz> mattip: what do you think of such a commit:
<cfbolz> (several people on twitter had the problem that their libssl.so was the wrong version compared to what the release wanted)
mattip_ has joined #pypy
<mattip_> cfbolz: looks good, just s/SSL/_ssl/
<cfbolz> right
<mattip_> is that all they needed? The other stdlib modules work?
<cfbolz> mattip_: yes, that's really the only incompatibility that I found people complaining about
<mattip_> fwiw, the buildbot is now ubuntu16.04, so maybe nightlies would work
<cfbolz> mattip_: right
jcea has joined #pypy
<kenaan> cfbolz default cdba165e6a48 /lib_pypy/_cffi_ssl/_stdssl/__init__.py: (cfbolz, mattip around) print instructions how to rebuild the _pypy_openssl cffi module
mattip_ has quit [Remote host closed the connection]
Rhy0lite has joined #pypy
jvesely has joined #pypy
oberstet has quit [Remote host closed the connection]
jacob22 has joined #pypy
jacob22_ has quit [Read error: Connection reset by peer]
forgottenone has quit [Quit: Konversation terminated!]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
xcm has quit [Read error: Connection reset by peer]
xcm has joined #pypy
<arigato> thank you for backporting _pypy_openssl to pypy2
<arigato> cfbolz: maybe attach the error message to the exception, instead of printing it? this way, it's harder to miss (and maybe there's someone who really tries to import and catches the ImportError and doesn't want things printed)
<cfbolz> arigato: yes, but then you don't see the underlying error message
<arigato> well no, python 3 is modern :-)
<arigato> if you raise instead an except handler, you see both exceptions in the end
<arigato> I think
<cfbolz> arigato: ok, but it's also on pypy2 now ;-)
<arigato> on pypy2 you can raise ImportError(str(e) + " explanation")
<cfbolz> ok
<arigato> unsure how you're meant to replace an exception on python 3, if you wanted to do that, btw
<arigato> ...ah, that's still the default, and you have new syntax to raise an additional error but keep the original one: "raise ImportError(..) from e"
<tos9> exception chaining should totally get backported to pypy 2.8
* tos9 ducks
<cfbolz> a lot of things should be, somehow
<cfbolz> but that so much work
<tos9> cfbolz: I mean, to be 67% more serious -- I think the part you're going to have to change probably isn't a ton more is it
<tos9> Like, backporting the syntax probably sounds terrible to have to do -- but even if py2 remembered exceptions that were raised while handling other ones and gave code no way to modify that it'd be nice
mattip_ has joined #pypy
<tos9> (I always wanted to write that for work as just a normal library since ISTM you can do most of what I care about without any changes to the interpreter, but now probably I don't care enough and those apps will just move to pypy3)
jcea has quit [Remote host closed the connection]
jcea has joined #pypy
<arigato> let's release pypy2 supporting python 2.8 on January 1st :-)
<arigato> (I am 100% joking)
<mattip_> since I removed _ssl, pypy/module/interpreter/test/test_appinterp.test_random_stuff_can_unfreeze is failing, it tries to import _ssl
<mattip_> on py3.6 I made the test pass by not importing _ssl, but then the whole thing is nonsense.
<arigato> I can try to figure that out
<mattip_> it's ok, just the idea is to import an interp-level module that imports a mixed module with app-level code
<mattip_> and now we don't have that situation anymore
<mattip_> my question is: mock something, or delete the test?
<kenaan> cfbolz default b6cf80ef9029 /lib_pypy/_cffi_ssl/_stdssl/__init__.py: (cfbolz, arigo around) attach error to exception
<arigato> mock something I believe, because I'm not sure we don't have that anymore
<cfbolz> arigato: I am finding (in my head) tons of fun use cases for the json dict strategy, btw (ie a dict with a fixed set of unicode keys)
<cfbolz> arigato: eg csv.DictReader
<cfbolz> or the python3 opcode BUILD_CONST_KEY_MAP
<arigato> what about our KwargsDictStrategy?
<cfbolz> yes, that could be replaced too
<arigato> there is nothing to win if the dict is all jitted away, but in case it isn't, then it would be a win too
<arigato> but is the json dict strategy good for the purpose of being jitted away?
<arigato> mattip_: I think we have pypy/module/_demo, btw, for mock purposes
<mattip_> +1, thanks
<mattip_> it seems c4b55d31320d works. maybe we should try to refactor the code into a single function
<mattip_> rather than copying str_decode_utf8_impl as unicodehelper.str_decode_utf8
<mattip_> just with a StringBuilder
<cfbolz> arigato: yes, it is virtualized just as easily
mattip_ has quit [Read error: Connection reset by peer]
<cfbolz> ah, there is also https://docs.python.org/3/library/typing.html#typing.TypedDict in 3.8
<cfbolz> arigato: nice!
<arigato> "great" I suppose, though argh, it's another strange piece of the language---you either say "class Point2D(TypedDict)" or "Point2D=TypedDict(...)" which are equivalent..?
<arigato> will only add to the newbie confusion of "subclass" vs "instance"
<cfbolz> arigato: also "instances" of such a TypedDict are just dicts
<arigato> ah, real dicts?
<cfbolz> yes
<cfbolz> ... but they would be a good usecase for the json dict strategy
<arigato> yes
<arigato> so it's a built-in example of: A is a subclass of B, A() is a dict, B()() is a dict
<cfbolz> yes, super strange
<cfbolz> arigato: so the gcc bug was them not using their integer abstract domain correctly?
<arigato> I think so, but I don't understand any of the gcc internals
<cfbolz> me neither, really
<cfbolz> sounds scary all
<arigato> I hope it's a proper fix and not just "we see a bug, by whacking there it doesn't show up any more"---I trust it's fine but I cannot check
<cfbolz> arigato: if it's really the integer domain, it's notoriously hard
<arigato> cpus should only have the unlimited bigints, never something as messy as N-bit integers
<arigato> I'm sure it can be designed in the spirit of complex modern Intel-like CPUs, and sold as a speed benefit because it can compress smaller integers into less than 4/8 bytes
<arigato> burning even more electricity for a little bit more speed
<cfbolz> arigato: hah
<cfbolz> then compilers will start proving complex number theoretic properties for you
<arigato> yay!
simpson has joined #pypy
BPL has joined #pypy
tsaka__ has joined #pypy
<mattip> arigato: around?
<arigato> mattip: yes
<mattip> about separating bytecode file names, cffi so names, and c-extestion file names: any idea how to do that?
<arigato> I think we need someone else's opinion about producing two different patterns of .so files
<arigato> it's probably fine
<arigato> but I really try to stay away from distribution issues so I don't know
jvesely has quit [Quit: jvesely]
<mattip> cffi modules are tied to the regular import process, and so should follow the same syntax as c-extension modules. I don't see how to separate them
<mattip> maybe we could use the "stable abi tag abi3" https://www.python.org/dev/peps/pep-0425/#abi-tag
<arigato> we can look up for files of both patterns
<arigato> the "stable abi tag abi3" for cffi modules is rather confusing imho
<mattip> why? I think that is exactly what you want to declare, the abi does not change with python version
<arigato> right... and then how about pypy2? we keep a different name for no deep reason?
<arigato> or just use the same tag with "abi3" and import that?
<arigato> I don't want to know, but that might make some package manager unhappy
<mattip> I don't see how we get around this on cpython for cffi
<mattip> and on windows there are multiple issues with pip like this one https://github.com/pypa/pip/issues/4445
<mattip> cffi uses setuptools/distutils, how can we distinguish between building a cffi c-extension module and a c-api c-extension module?
altendky has joined #pypy
<mattip> this quote seems relevant "It's just a matter of using py_limited_api=True in setuptools.Extension"
<mattip> ahh, cffi already has a _set_py_limited_api() function, with the comment
<mattip> "Setuptools takes care of ignoring the flag on Python 2 and PyPy"
kbtr has quit [Remote host closed the connection]
forgottenone has joined #pypy
<mattip> which boils down to pypy3 not putting '.abi3.so' into importlib.machinery.EXTENSION_SUFFIXES
<mattip> python2 and pypy2 are a lost cause: it will always use the non-py_limited_abi name
<tumbleweed> mattip: what problem are you trying to solve?
<tumbleweed> without a 2.8, limited_abi seems irreleveant to python 2
<mattip> currently when we build the _audioop cffi module, the name is _audioop_cffi.pypy-41.so on pypy2.7 v7.2
<mattip> and when we build a c-api module the name is the same: _dummy.pypy-41.so
<tumbleweed> what's the issue with that?
<mattip> which has remained constant across too many versions of pypy, so c-api modules built with 7.2 may not work on earlier version
<tumbleweed> ah, right
<mattip> we should be using a full PEP 425 tag, like on pypy3
<tumbleweed> so you want stable ABI for cffi modules
<tumbleweed> but want to be able to bump DEFAULT_SOABI for c-api
<mattip> yes, even on pypy2
<tumbleweed> I had wondered why DEFAULT_SOABI was never being bumped any more
<mattip> it **is** being bumped on pypy3, but not on pypy2
<tumbleweed> I can imagine hacks to make this work, but they'd break tools that expect cpython like behavior
* mattip off to a meeting, back in two hours
<tumbleweed> arigato: BTW, I applied your workaround to the gcc bug. Works well for pypy2, but for 3, getting some interesting test failures that look like more JIT weirdness https://paste.debian.net/1109365/ (the offending line is: return f(*args, **kwds)
<tumbleweed> full log with lots of strange test failures: https://people.debian.org/~stefanor/pypy3_7.2.0+dfsg-1_arm64.build
jvesely has joined #pypy
<tumbleweed> oh, same failure on amd64, nevermind
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
jiffe has quit [Quit: WeeChat 2.3]
jiffe has joined #pypy
YannickJadoul has quit [Quit: Leaving]
dansan has joined #pypy
forgottenone has quit [Quit: Konversation terminated!]
jiffe has quit [Quit: WeeChat 2.6]
firespeaker has joined #pypy
jiffe has joined #pypy
* mattip back
<mattip> tumbleweed: I think my conclusion is something like
<mattip> - pypy2 is negligent in not updating DEVAULT_SOABI
<mattip> - pypy2 cannot use a "limited_api" approach to name the cffi c-extension modules differently than the c-api c-extension ones
<tumbleweed> yeah
<tumbleweed> you really want separate abi mechanisms for the two
<mattip> - pypy3 should be offering a limited_api option which it currently does not
<tumbleweed> but the world only expects one kind of C extension for python
<mattip> there is the limited_api C extension, which has the filename.abi3.so conventions, right?
<tumbleweed> yes. except there is no concept of python implementations in that spec (AFAIK)
<tumbleweed> so the same .abi3.so has to work on pypy and cpython
<tumbleweed> which means you can't rely on any magic in compliation headers (dunno if c-api currently does that)
<mattip> with cpython3.6 and cffi 1.12.3 creates a non-limited_api so _example.cpython-36m-x86_64-linux-gnu.so
<mattip> so at least it seems pypy3 is consistent with cpython3
<mattip> thanks for helping work through this
jvesely has quit [Quit: jvesely]
jvesely has joined #pypy
<tumbleweed> np. I don't think we're that much closer to solutions though :P
firespeaker has quit [Quit: Leaving.]
<mattip> well, there really is no solution for python2 without hacks
lritter has quit [Ping timeout: 246 seconds]
CrazyPython has joined #pypy
Rhy0lite has quit [Quit: Leaving]
CrazyPython has quit [Remote host closed the connection]
<tumbleweed> I think you'd need buy-in from pypa to introduce a new mechanism. And given the EOL of python2, it's hard to imagine people putting work into that
jvesely has quit [Quit: jvesely]
CrazyPython has joined #pypy
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
njs has joined #pypy
xcm has quit [Read error: Connection reset by peer]
xcm has joined #pypy
jvesely has joined #pypy
CrazyPython has quit [Remote host closed the connection]
CrazyPython has joined #pypy
jvesely has quit [Quit: jvesely]
BPL has quit [Quit: Leaving]
jvesely has joined #pypy
RemoteFox has quit [Ping timeout: 240 seconds]
RemoteFox has joined #pypy
CrazyPython has quit []
firespeaker has joined #pypy
jvesely has quit [Quit: jvesely]
jacob22 has quit [Ping timeout: 245 seconds]
jacob22 has joined #pypy