cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<mattip>
matplotlib for pypy is now in conda-forge!
<cfbolz>
wow!
<arigo>
Alex_Gaynor: I reverted the change ffi_prep_closure -> ffi_prep_closure_loc, because some systems run old libffi's without the latter---notably the OS X on Azure are like that
<arigo>
(that's macOS-10.14, so not that old)
forgottenone has joined #pypy
<arigo>
(pff, pytest gives ever less useful error messages sometimes: Azure Pipelines says "Using pytest.skip outside of a test is not allowed. To decorate a test function, use the @pytest.mark.skip or @pytest.mark.skipif decorators instead, and to skip a module use `pytestmark = pytest.mark.{skip,skipif}." and nothing more (no line number or traceback)
<arigo>
and of course I cannot reproduce locally
oberstet has joined #pypy
<mattip>
one hint might be this is on windows, I couldn't find that error in the linux run
<mattip>
and it says "ERROR collecting c/test_c.py"
<mattip>
my guess would be line 74 in the find_and_load_library() function
<arigo>
I think it's the py.test.skip() in this file, which I now tried to replace with pytest.mark.skip(), which involves more hacks because that latter call is not raising an exception so the rest of the module must import correctly, at least
<arigo>
I mean the one at the very beginning
<arigo>
but maybe indeed it's a different one
<arigo>
but yes, no clue why it seems to be on windows only (maybe the windows machines have a different version of pytest)
<mattip>
I ran into this on PyPy, see 18f14f03e1c5
<mattip>
expecially the apptest change
<arigo>
ah, I think it's wrong
<arigo>
pytest.mark.skip() returns an object, it doesn't raise an exception
<arigo>
so if you ignore the result, it won't do anything (I think??)
<arigo>
but yes, it shouldn't matter because all the actual tests have been indented in the "else:" part
<arigo>
well, one more example of pytest deciding to change things for the sake of changing them and breaking stuff around
jkoan-work has joined #pypy
<mattip>
the actual error was supposed to be raised 4 years ago
<mattip>
I wonder what changed now to suddenly expose it?
<arigo>
no clue, but may be related to the fact that azure pipelines started to support pypy
<arigo>
which I precisely don't want, for cffi
<arigo>
we can use "versionSpec" in the azure-pipelines.yml file, but its content is completely undocumented as far as I can see
<arigo>
I'm trying now ``versionSpec: '2.x || 3.x'`` and we'll see what versions it builds
jkoan-work has left #pypy [#pypy]
<arigo>
nope, didn't work
<arigo>
pypy is still included
<rjarry>
pmp-p: I am hoping I can gain some cycles by coding some specific functions in C but these functions would require to manipulate python objects, is that even possible with CFFI extensions?
<arigo>
rjarry: unsure you'd win a lot of performance but you can try: the trick is ffi.new_handle() and ffi.from_handle()
<rjarry>
arigo: this allows passing a python object as an argument for a C function, but I had understood that CFFI functions are called with the GIL released
<arigo>
...or, well, the first step, if you're ready to have a hackish solution, doesn't even involve handles
<arigo>
you need anyway to invoke a callback from C to Python in order to set the item in the dictionary
<rjarry>
is that possible to use the Python.h API into a C function wrapped by a CFFI extension?
<arigo>
not really
<arigo>
so the idea is to pass that callback to C, and that callback should be written as Python code, which should be something simple like "def call(key, value): my_global_dict[key] = value"
<arigo>
(and then you use handles if you want to avoid having 'my_global_dict' being a global)
<rjarry>
I see
<rjarry>
but, the "value" is necessarily a CData object
<rjarry>
I still need to do conversion to python strings and/or other primitive types into the callback?
<arigo>
yes
<rjarry>
I need to think about this
<arigo>
note that this is all assuming that the end goal is a normal python dictionary with normal python strings
<rjarry>
thanks for the suggestion :)
<arigo>
what you may consider too instead is very different:
<arigo>
if you have some dict-like C structure,
<rjarry>
yes, a normal dict with builtin python key/values
<arigo>
you make a wrapper class in python with a custom __getitem__() method and you fetch and convert lazily from the C structure
<arigo>
so you end up with not a normal dict, but instead a Python instance that implements dictionary-like methods
<arigo>
this is very useful if there is a large amount of data, because you avoid the high memory usage and conversion time
<rjarry>
that would work for C -> python indeed
<rjarry>
even if that prevents from making naive copies as with builtin dict objects
<rjarry>
but it would not work for the python -> conversion
<rjarry>
python -> C conversion
<arigo>
indeed
<rjarry>
I'll keep that callback solution in mind though
<rjarry>
maybe that's a good one :)
<arigo>
maybe for that direction, have ffi.cdef() expose two functions like "make_empty_c_dict()" and "add_key_to_c_dict()" and call them repeatedly from python?
<arigo>
depending on whether your C structure is happy with incremental building
<ronan>
arigo: you can use pytest.skip(..., allow_module_level=True)
<arigo>
another -1 for giving a misleading error message then
<arigo>
and sorry, not supported by the older version of pytest I still want to remain compatible with
<arigo>
the question is if there are intermediate versions of pytest that don't have that keyword argument but still crash when pytest.skip() is called at module level, and digging the answer to that seems pointless at this point
<arigo>
(for cffi I mean)
<ronan>
tbh supporting antediluvian pytest versions is self-inflicted pain
<fijal>
supporting new ones as well :-)
<fijal>
I must say that I have been using software intermittently and the neverending upgrades to something that works less so than previous version is really bad
<arigo>
ok WAT
<arigo>
the allowed SYNTAX of python 2.7 CHANGED somewhere between 2.7.6 and recent 2.7.x
<arigo>
as a result, I cannot build pypy using python 2.7.6 on a machine that happens to have it
<arigo>
ronan: would you complain if I revert the changes to make it 2.7.6-compatible again, at least for now? it's "exec(src, d)" that cannot be called in a function using nested scopes
<arigo>
it requires "exec src in d" instead
<ronan>
arigo: which commit was it?
<arigo>
e.g. c067c8c93177
<arigo>
but I'm not saying I want to revert the whole commit
<arigo>
just the few exec calls where that is a problem
<ronan>
yes, it's fine
<arigo>
I'm getting a SyntaxError at import time, so it's easy to figure out, and there are only a few places
<ronan>
that was just for tidying up, and supposedly safe
<arigo>
OK
<arigo>
yeah I guessed
BPL has joined #pypy
forgottenone has quit [Remote host closed the connection]
<arigo>
yes, sorry, I meant in general. I guess the improvement you see might be due to this fix which applies to all pypy3 versions (as well as pypy2 in theory but the file I/O logic is really different there)
<arigo>
and not to any specifically 3.7 or aarch64 change
forgottenone has quit [Ping timeout: 256 seconds]
<YannickJadoul>
Huh, hg repos are still not deleted on BitBucket?
jaraco has quit []
jaraco has joined #pypy
YannickJadoul has quit [Remote host closed the connection]
ebarrett has quit [Quit: WeeChat 2.8]
ebarrett has joined #pypy
Dejan has quit [Quit: Leaving]
<tos9>
ech. Can someone on linux tell me what precision you get out of getmtime on pypy
<tos9>
On macos it definitely seems to be giving less precision than CPython
<tos9>
specifically what does this output for you:
<tos9>
touch foo; for each in pypy3 python3.6; do echo $each; "$each" -c 'import os; print(os.path.getmtime("foo"))'; done