danieljabailey has quit [Ping timeout: 250 seconds]
danieljabailey has joined #pypy
danieljabailey has quit [Ping timeout: 250 seconds]
danieljabailey has joined #pypy
danieljabailey has quit [Ping timeout: 268 seconds]
danieljabailey has joined #pypy
themsay has joined #pypy
themsay has quit [Ping timeout: 250 seconds]
dddddd has quit [Remote host closed the connection]
themsay has joined #pypy
jcea has quit [Quit: jcea]
antocuni has quit [Ping timeout: 268 seconds]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
dddddd has joined #pypy
zmt01 has quit [Quit: Leaving]
Zaab1t has quit [Quit: bye bye friends]
_whitelogger has joined #pypy
PileOfDirt has joined #pypy
PileOfDirt has quit [Quit: Leaving]
PileOfDirt has joined #pypy
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
<njs>
if using cffi's from_buffer, is there any way to manually "release" the buffer object?
<njs>
it seems like this design assumes a refcounting gc, which is not what I expect from the pypy devs generally :-)
<mattip>
hmm, maybe it should have an API to allow use as a context manager
<njs>
that's what memoryview does
<mattip>
for something completely different
<mattip>
scipy is running pypy3 6.0 in CI on a 4GB VM, via 'pytest -n3' which starts up 3 pytest instances
<mattip>
the tests randomly crash
<mattip>
I suspect the processes are running out of memory
<njs>
it looks like the current behavior is: on cpython, ffi.from_buffer(bytearray) locks the bytearray's size for as long as the cffi object is alive, so you have to be careful to trick the refcount GC into releasing it at the appropriate time. On PyPy, it doesn't lock the bytearray size; instead it lets you realloc the bytearray, and then the cffi object starts pointing into random memory and letting you mutate it accidentally.
<mattip>
when I set ulimit -m 990 and run without the `-n3`, I can get a consistent crash
<njs>
so it is technically possible to use correctly, but it's a pretty sharp corner in both cases
<njs>
mattip: out of memory should still give some kind of consistent error message, no? maybe if you turn on faulthandler, or manually print the return code?
<mattip>
careful, if you resize the bytearray, the <cdata> object will point to freed memory
<mattip>
for my issue, no it just segfaults
<mattip>
no error
<njs>
it also says "The original object is [...] locked [...] as long as the cdata object returned by ffi.from_buffer() is alive"
<njs>
and both of those comments are true, on pypy and cpython respectively
<njs>
mattip: fun :-(
<mattip>
but setting PYPY_GC_MAX=900MB makes tests run to completion
<njs>
mattip: segfault is a strange way to report OOM... usually linux will overallocate and then invoke the OOM-killer, which sends SIGKILL
<njs>
if malloc for some reason decides not to overallocate, then it could return NULL, and if no-one checks the return value then it might segfault... but (a) I don't know how you'd convince malloc not to overallocate, (b) that would be a genuine bug, malloc's return value should always be checked
<mattip>
I am getting .......Segmentation fault (core dumped)
<mattip>
I didn't bother to check the core
<njs>
mattip: it is possible with some kluges to convince gdb to give you a backtrace in CI
<mattip>
I can do it locally
<mattip>
of course, running under gdb changes something so tests no longer crash
<mattip>
and the core is not actually saved AFAICT