cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "the modern world where network packets and compiler optimizations are effectively hostile"
rokujyouhitoma has quit [Ping timeout: 260 seconds]
ronan has quit [Ping timeout: 240 seconds]
<cfbolz>
mattip: does cpyext use __del__ much?
<mattip>
nope
cstratak has joined #pypy
<mattip>
AFAICT
<mattip>
obviously, something is happening, but not clear what
<mattip>
and it is a bit hard to get a handle on what is going on in a translated test
<cfbolz>
mattip: yes, as usual
<cfbolz>
mattip: it could be the list strategy doing something weird
<cfbolz>
Or the refcounts
<mattip>
list strategy is a good suspect, I tried to refactor it a while back but maybe missed a case
<mattip>
gc.get_objects() is not showing a change between one call to np.array(range(10000)) and ten calls,
<mattip>
even though rss memory changes almost linearly
<cfbolz>
mattip: so it's raw malloced objects
<cfbolz>
They don't appear in get_objects, I think
<mattip>
ok, thanks
<cfbolz>
mattip: I have an unrelated question about cpyext: the mapping from python object do C structure is still done with a dict, right?
yuyichao has quit [Ping timeout: 255 seconds]
<mattip>
AFAIK it is rpython.rlib.rawrefcount._pypy2ob and _pypy2ob_rev, which are both dicts
cstratak has quit [Ping timeout: 276 seconds]
<cfbolz>
Hm
<mattip>
cfbolz: that may be true only for untranslated?
<cfbolz>
Yes, I think one is done by the GC
yuyichao has joined #pypy
<cfbolz>
no, it seems I am wrong
* mattip
off, later
mattip has left #pypy ["bye"]
rokujyouhitoma has joined #pypy
nimaje1 has joined #pypy
nimaje1 is now known as nimaje
nimaje is now known as Guest18276
Guest18276 has quit [Ping timeout: 260 seconds]
lritter has joined #pypy
rokujyouhitoma has quit [Ping timeout: 260 seconds]
<Remi_M>
fijal: about the resop-interpreter: do you actually expect to get a speedup over CPython when interpreting the traces produced by the JIT?
<Remi_M>
It is still a bit unclear to me what we really expect from that project. If it is just going to be a development tool for debugging the JIT, I'm not sure if that is not merely a nice-to-have thing, which will eventually bit rot since nobody maintains it.
<fijal>
Remi_M: maybe?
<fijal>
I don't know
<fijal>
I think it's an experiment worth doing whether or not we'll end up using it
<Remi_M>
maybe it is...
<Remi_M>
seems like for anyone not knowing RPython, however, it will be quite a big project
<Remi_M>
(probably master thesis)
<fijal>
you could in theory write it in C
<fijal>
but maybe
<Remi_M>
do we know of other projects that dis something like that? e.g., using an llvm bitcode interpreter?
<fijal>
maybe hotpy?
<Remi_M>
I'll have a look at that. seems similar indeed
<Remi_M>
just looking at results, apparently hotpy really did achieve significant speedups. so indeed attempting something similar for pypy seems worthwhile
<kenaan>
stevie_92 cpyext-gc-trialdeletion f8d7bb5f29b6 /: Implemented flags for concurrent cycle deletion (Bacon and Rajan 2001) with overflow handling f...
rokujyouhitoma has joined #pypy
ronan has joined #pypy
rokujyouhitoma has quit [Ping timeout: 246 seconds]
oberstet has quit [Ping timeout: 260 seconds]
SunDwarf has quit [Read error: Connection reset by peer]
vkirilichev has quit []
mattip has joined #pypy
<mjacob>
i'm missing context - which problem will the resop interpreter solve?
<Alex_Gaynor>
Isn't the metainterp a resop interpreter?
rokujyouhitoma has joined #pypy
adamholmberg has joined #pypy
rokujyouhitoma has quit [Ping timeout: 248 seconds]
cstratak has quit [Ping timeout: 268 seconds]
ronan has quit [Ping timeout: 240 seconds]
yuyichao has quit [Ping timeout: 246 seconds]
mattip has left #pypy ["bye"]
<Remi_M>
Alex_Gaynor: the metainterp interprets "jitcode" and emits a trace of "resops", no?
<Alex_Gaynor>
Remi_M: pfff, yes, you're right
<Remi_M>
mjacob: unclear if it solves anything :) . it's portable and debuggable, and it may help understanding the effectiveness of optimisations better
yuyichao has joined #pypy
Rhy0lite has joined #pypy
ronan has joined #pypy
Remi_M has quit [Quit: See you!]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 258 seconds]
asmeurer has joined #pypy
<fijal>
mjacob: I think it could be good for experimenting with optimizations, maybe
cstratak has joined #pypy
marky1991 has joined #pypy
<mjacob>
well, i would argue that you should find a convincing use case before developing something which will bitrot ;)
<mjacob>
how would the interpreter useful in experimenting with optimizations?
<bendlas>
hm, that's not announced on the home page ..
<LarstiQ>
hmm
<LarstiQ>
ah no
<LarstiQ>
bendlas: sorry, that's 5.8.0, I got confused by the 1 in the filename
<LarstiQ>
ok, so then it's something NixOS specific maybe
<bendlas>
might be
<LarstiQ>
bendlas: or, could you try that portable binary too?
<LarstiQ>
to rule out any differences there
<bendlas>
on it ...
<bendlas>
i need to compose a fhs env to be able to do that
<LarstiQ>
fhs?
<LarstiQ>
ah, so ldd can lod libgcc_s and friends?
<bendlas>
yes, in nixos, everthing is stored and statically bound within /nix/store/
<bendlas>
packages can't reference /bin, /usr or anything like that
* LarstiQ
nods
<bendlas>
but I can build an FHS chroot, where they can
<bendlas>
and indeed, the portable binary raises OSError as expected ...
<LarstiQ>
bendlas: the portable binaries minimize what they reference, but there's still a bit in /lib
<LarstiQ>
bendlas: aha!
<LarstiQ>
bendlas: want me to file that bug still, or wait till you've dug more?
<bendlas>
No, that's a ticket for the nixos tracker. Thanks for making me check with the portable binary!
<bendlas>
but still, however exactly we managed to mess this up, it's interesting, that pypy would silently fail, but I'll check back when I have a ticket
<bendlas>
LarstiQ: I'm still wondering how to debug this ... mattip mentioned something about running untranslated tests
<LarstiQ>
bendlas: right
<bendlas>
but he also said, that there were none for test_os ...
<bendlas>
or the resource module
<LarstiQ>
bendlas: hmm, I'd first check if cpython's testsuite has anything for this
<LarstiQ>
(as bundled with pypy)
<LarstiQ>
bendlas: before going to rpython tests
<LarstiQ>
bendlas: is this urandom specific, or could you test if the rlimit fails altogether?
<bendlas>
it's urandom specific indeed. good catch!
<bendlas>
LarstiQ: `open("..", "r")` fails after `setrlimit` as expected
<bendlas>
`open("/dev/urandom", "r")` and `open("/dev/random", "r")`, specifically, fail as well
rokujyouhitoma has joined #pypy
<bendlas>
just `os.urandom(16)` still works
<LarstiQ>
let's see if I can figure out how that is implemented
<LarstiQ>
there is rlib/rurandom.py
<LarstiQ>
which seems to syscall instead of opening /dev/urandom in some cases?
<LarstiQ>
bendlas: you could try commenting the 'if SYS_getrandom is not None' and 'if n <= 0' in `def urandom(..` and see if that matters
<LarstiQ>
or maybe only make sure the return doesn't happen
rokujyouhitoma has quit [Ping timeout: 268 seconds]
<LarstiQ>
bendlas: as how to run tests, I'll have a look at that
<bendlas>
I'll have to rebuild pypy-c after that change, right?
<LarstiQ>
unless we can test that untranslated, which is the normal workflow
<LarstiQ>
but here my skills as a lurker might not suffice :)
<bendlas>
LarstiQ, fijal: looking at strace, pypy's `os.urandom()` doesn't seem to open `/dev/urandom` at all, but instead uses the getrandom syscall. Is this something our libc is doing? That would explain the success for portable build.
tbodt has joined #pypy
pilne has joined #pypy
<bendlas>
LarstiQ: fijal: it's gotten clear now. pypy uses the getrandom syscall, if it can, which the portable build apparently can't. So the real bug is for the test to expect `/dev/urandom` to be opened when calling `os.urandom()`. I'll patch this out of our build, but you might want to fix it too.