cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<exarkun> lazka: yes I know
dansan_ has joined #pypy
dansan has quit [Ping timeout: 240 seconds]
lazka has quit [Ping timeout: 240 seconds]
lazka has joined #pypy
tbodt has quit [Ping timeout: 244 seconds]
tbodt has joined #pypy
Rhy0lite has quit [Quit: Leaving]
epsilonKNOT has joined #pypy
jcea has quit [Quit: jcea]
speeder39_ has quit [Quit: Connection closed for inactivity]
<mattip> did we implement PEP 538 on py3.7 ?
<mattip> nope, the code above does not warn and prints "ascii" :(
<mattip> another thing to fix before the release
oberstet has quit [Remote host closed the connection]
jvesely has quit [Quit: jvesely]
ctismer has quit [Ping timeout: 260 seconds]
Alex_Gaynor has quit [Ping timeout: 272 seconds]
Olorin has quit [Ping timeout: 240 seconds]
fijal_ has joined #pypy
Olorin_ has joined #pypy
fijal has quit [Ping timeout: 272 seconds]
fijal_ is now known as fijal
ctismer has joined #pypy
Alex_Gaynor has joined #pypy
_whitelogger has joined #pypy
<mattip> exarkun: would it help if we implement PEP 538 and backport it to 2.7, 3.6?
TheNewbie has joined #pypy
tos9 has quit [Quit: ZNC 1.7.2+deb3 - https://znc.in]
tos9 has joined #pypy
epony has quit [Ping timeout: 258 seconds]
TheNewbie has quit [Ping timeout: 240 seconds]
epony has joined #pypy
TheNewbie has joined #pypy
jacob22 has quit [Read error: Connection reset by peer]
jacob22 has joined #pypy
Dejan has joined #pypy
<arigato> also, what kind of code would it break
<exarkun> It _seems_ like the PEP 538 behavior is generally better than the non-PEP 538 behavior to me. "Better" does not mean "completely compatible" so, indeed, I don't know what would break so that's probably a good concern to factor in.
<exarkun> Practically speaking I'm not sure what will actually help me. For CPython 2.7 I had a cffi/Py_FileSystemDefaultEncoding fix that _mostly_ works but it fails for certainly builds of Python (eg static libpython linkage) - which is sort of what you'd expect from such a hacky solution.
<exarkun> I don't know if I can get away with saying "If you want it to work reliably, stop using CPython 2.7 and use PyPy 2.7.something instead."
<exarkun> Or if I really need a general solution where it just works period.
<exarkun> (which is what I have now - by sprinkling hundreds of explicit encode/decode calls throughout the entire codebase)
<arigato> note that if you're thinking about cffi, you can also just call _Py_SetFileSystemEncoding
<arigato> by writing a C extension module
<arigato> (a regular one)
<mattip> does anyone feel like commenting on this issue on pybind11 about why we don't provide PyFrame_FastToLocals and general frame probing?
<mattip> that's the code
<arigato> well, yes, we can't possibly provide a full emulation of the live PyFrameObject
<arigato> this would be completely incompatible with the JIT
<mattip> I'm not sure I understand the motivation. The are wrapping a c++ class with python, and overriding methods in python
<mattip> then why do they need to dispatch via the C-API?
<arigato> str(frame.f_code.co_name) == name
<arigato> this is a huge hack imho and will break if the user defines a random function anywhere with the wrong name
<arigato> beyond that, I have no idea about what pybind11 does or should do
<arigato> if you want to write down a suggestion, it would be to not call PyRun_String(), which requires compilation of the code from the string
<arigato> they could instead write pages of C along the lines of PyObject_CallMethod(sys_module, "_getframe", PyInt_FromLong(1))
<mattip> here is the motivation
<arigato> (but however they write it down, it's going to be somewhat slow on pypy)
<mattip> and as a rendered page
<arigato> sorry, this page is huge
<mattip> ok. I wonder why they don't link the c++ class and the python class - why they have to dig around the frame for it
<arigato> and I don't know what part you're pointing to
<arigato> I really don't know pybind11
<mattip> sorry. The whole thing is about overloading c++ functions from python, AFAICT
<arigato> can you explain or point to the place that explains this "
<arigato> <mattip> ok. I wonder why they don't link the c++ class and the python class - why they have to dig around the frame for it"?
<arigato> sorry
<arigato> can you explain or point to the place that explains this "Don't call dispatch code if invoked from overridden function." ?
<arigato> is that for the super call?
<mattip> dunno. I am just guessing what is going on
<mattip> based on quickly trying to read follow the code out to where it is used
<arigato> is that a horrible hack so that if you're in DerivedPythonClass.f() and you just write 'BaseClass.f(self)', then it invokes the base class instead of calling again DerivedClass.f()?
<arigato> that would sound quite backward to me
<arigato> why would Python code 'BaseClass.f(self)' end up invoking the derived method?
<mattip> look for PYBIND11_OVERLOAD_PURE_NAME on the rendered doc. It says
<mattip> This is required when the C++ and Python versions of the function have different names, e.g. operator() vs __call__
<mattip> sorry, never mind, I will just limit my answer to the "we can't expose the pyframe" and be done.
oberstet has joined #pypy
todda7 has joined #pypy
tsaka__ has joined #pypy
dddddd has quit [Ping timeout: 246 seconds]
todda7 has quit [Ping timeout: 258 seconds]
dddddd has joined #pypy
rubdos has quit [Ping timeout: 272 seconds]
rubdos has joined #pypy
TheNewbie has quit [Ping timeout: 240 seconds]
lritter has joined #pypy
Rhy0lite has joined #pypy
<exarkun> arigato: re _Py_SetFileSystemEncoding, not on Python 2.7 I guess. I wonder if calling that would be any more successful than directly setting Py_FileSystemDefaultEncoding, anyway ... The failure mode I encountered is that I find the symbol and change its value and it has no effect (on some systems).
<Alex_Gaynor> Is it expected that when creating a type with `PyType_FromSpec` (cpytext) if there's no `"."` in the `name` field, that `__module__` with be `importlib.bootstrap`?
jcea has joined #pypy
<arigato> probably not
<Alex_Gaynor> arigato: in https://github.com/PyO3/pyo3/pull/1132#issuecomment-685705493I I'm chaging pyo3 (rust python extensions) to use PyType_FromSpec and it's failing on pypy https://github.com/PyO3/pyo3/pull/1132/checks?check_run_id=1059787749
jvesely has joined #pypy
rfgpfeiffer has joined #pypy
Rhy0lite has quit [Read error: Connection reset by peer]
Rhy0lite has joined #pypy
omnipotententity has joined #pypy
<omnipotententity> I seem to be running into some trouble with what I'm trying to do: I'm trying to pass a callback from python code into rust code, and according the the CFFI docs, the way to do this is to use the @ffi.def_extern() decorator. Except, I was planning on partially currying this function because the purpose of the callback is to resize a numpy array (and the FFI only has the pointer to the data of the array). Is there a way to use
<omnipotententity> a lambda as a callback?
<omnipotententity> (Well, technically, I haven't yet managed to make the API version of the CFFI interface work yet, so I'd be using the old style ABI callback which has the @ffi.callback() decorator, same problem though.)
<tos9> omnipotententity: what happens when you pass one in?
<tos9> CFFI doesn't do anything special there AFAIK so lambda functions are just normal functions same as always, but I've only used def_extern once or twice
<arigato> omnipotententity: basically the answer is No
<arigato> because C doesn't support currying, and the various non-portable libraries to do that all have problems
<omnipotententity> arigato, I'm doing the currying on the python side
<arigato> if you control both sides, though, it's easy to use the standard C way of declaring only one function (and one @ffi.def_extern) and passing an extra "void *" argument
<arigato> sure, but you want to have a variable number of function pointers in C, and that doesn't really work
<omnipotententity> No, I just want to have one function pointer passed into the call.
<omnipotententity> Sorry, I'll make a pastebin so it's a bit more clear
<arigato> by "variable number" I mean a quantity of function pointers that are each different and that is not bounded at compilation-time
<arigato> it may be that in the common case, you only need one such function pointer alive at the same time, but that doesn't help
<omnipotententity> Would doing this be OK? I notice that they mention needing to ensure that the callback might fall out of scope.
<arigato> this is using the deprecated ffi.callback(), which you should not if you aim to be portable
<omnipotententity> That's not an important aim. This is a purely internal tool.
<arigato> well, then OK
<arigato> you need to say "p_callback = ffi.callback(...)"
<arigato> and then pass "p_callback" to the C function
<omnipotententity> excellent thanks.
<arigato> the lifetime issue is only that as long as "p_callback" exists in Python, the callback is invokable from C
<omnipotententity> Then make sure that p_callback doesn't fall out of scope before the callback completes.
<arigato> yes
<omnipotententity> excellent! thanks. Now to figure out the rust side of this.
<omnipotententity> ありがとう
<mattip> uhh, if you are really resizing a numpy array, why not just call realloc in C?
<mattip> or PyArray_Resize from the numpy C-API
<mattip> if you want the error checking
epsilonKNOT has quit [Read error: Connection reset by peer]
epsilonKNOT has joined #pypy
epony has quit [Ping timeout: 258 seconds]
wilbowma has quit [Ping timeout: 246 seconds]
wilbowma has joined #pypy
<omnipotententity> mattip, I didn't think that would work, and I'm calling from Rust anyway.
<omnipotententity> That being said, if it does work, it does make this a lot easier and I'll give it a shot.
<omnipotententity> (But like, how would the numpy array know that its data was relocated via realloc?)
epony has joined #pypy
lritter has quit [Quit: Leaving]
rfgpfeiffer has quit [Quit: http://quassel-irc.org - Chat comfortably. Anywhere.]
pmp-p has quit [Ping timeout: 258 seconds]
pmp-p has joined #pypy
lritter has joined #pypy
danchr has quit [Quit: ZNC - https://znc.in]
danchr_ has joined #pypy
Smigwell has joined #pypy
speeder39_ has joined #pypy
<mattip> numpy reassigns ndarray->data.
<mattip> That can go terribly wrong if there are views of the array out in the wild,
<mattip> so there are checks based on refcount, which PyPy will refuse to do, and a boolean "refcheck" that means
<mattip> "I know what I am doing, resize anyway"
<mattip> omnipotententity ^^^
<mattip> using PyArray_Resize from C is equivalent to the python np.resize() call
<mattip> ,so if you are comfortable using resize in the first place, PyArray_Resize is no different
<mattip> of course, correctly allocating the array when constructing it would be preferable
dmalcolm has quit [Remote host closed the connection]
danchr_ has quit [Ping timeout: 240 seconds]
danchr_ has joined #pypy
dmalcolm has joined #pypy
lritter has quit [Ping timeout: 240 seconds]
lritter has joined #pypy
speeder39_ has quit [Quit: Connection closed for inactivity]
speeder39_ has joined #pypy
gsnedders has quit [*.net *.split]
pjenvey has quit [*.net *.split]
[Arfrever] has quit [*.net *.split]
Kronuz has quit [*.net *.split]
gsnedders has joined #pypy
pjenvey has joined #pypy
[Arfrever] has joined #pypy
Kronuz has joined #pypy
i9zO5AP has quit [Ping timeout: 240 seconds]
i9zO5AP has joined #pypy
lritter has quit [Quit: Leaving]
i9zO5AP has quit [Ping timeout: 240 seconds]
i9zO5AP has joined #pypy
Rhy0lite has quit [Quit: Leaving]
avakdh has quit [Ping timeout: 256 seconds]
avakdh has joined #pypy
speeder39_ has quit [Quit: Connection closed for inactivity]
oberstet has quit [Remote host closed the connection]