antocuni changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "PyPy: the Gradual Reduction of Magic (tm)"
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<cfbolz>
arigato: so the only place that I can think of that does things wrongly then is __pypy__.intop.int_*. because there we really assume (and test) that it wraps around
<arigato>
ah
<arigato>
I think __pypy__.intop is a strange beast
<arigato>
sure. I'm saying, right now the fact that it wraps around is lost to the JIT
<antocuni>
ok, got it
<arigato>
so if you write code like "x = int_add(y, 5); assert x > 0" and run for y == sys.maxint, it probably hits the assert if interpreted but not if jit-compiled
<cfbolz>
arigato: I would suggest I fix it to use unsigned int operations internally
<cfbolz>
Makes sense?
<cfbolz>
Because making that kind of undefinedness escape to python level is a bit scary to me
<arigato>
fix __pypy__.intop? that's not a fix right now
<cfbolz>
Huh?
<cfbolz>
Why?
<arigato>
it's a fix provided we also implement uint_add in the jit :-)
<cfbolz>
We have it, no?
<cfbolz>
Or it's not supported in the jit at all?
<arigato>
no
<arigato>
the jit really needs a resop called UINT_ADD, and doesn't have it
<cfbolz>
Right
<cfbolz>
OK, annoying
<cfbolz>
arigato: we should grep a bit for the uses of intmask too.
<arigato>
?
<cfbolz>
Well, it can be used for getting wrapping behavior on python ints before translation
<arigato>
ah
<arigato>
right, but these ones are already broken by translation to C
<cfbolz>
Yes
<arigato>
we've had cases
<arigato>
but I see
<cfbolz>
So we should check whether the calls to intmask that work that way are problems
<arigato>
what use case is there for intmask(x) where x is a real 'int' ?
<arigato>
(i.e. a signed number)
marr has quit [Remote host closed the connection]
<cfbolz>
arigato: iirc in the early beginning of rpython we really thought that int add would wrap, and intmask was how you got that behavior before translation
<arigato>
but maybe it should precisely be non-rpython when used with signed numbers
<arigato>
right
<cfbolz>
arigato: the other option would be to define that int addition wraps around
<arigato>
no, that would mean many more guards produced by the jit
<arigato>
basically the intbounds would become mostly pointless
<cfbolz>
So then we should probably fix the docs, and disable intmask(signed)?
<arigato>
yes, let's try
<cfbolz>
Will do
<cfbolz>
And for intops we are a bit unhappy?
<arigato>
or at least intmask(int-sized signed)
<cfbolz>
Yes, sorry
<cfbolz>
That's what I mean, of course
<arigato>
and yes, the next step would be to add UINT_ADD & co to the jit
<cfbolz>
arigato: so far intop is the only use case we can think of though, right?
<arigato>
yes, likely
<arigato>
there are other cases but they are inside some algos, so not seen by the jit
<cfbolz>
So I'll add a doc string to intop at least
<arigato>
I'm more concerned about non-pypy interpreters out there
<cfbolz>
Yes
<cfbolz>
arigato: if they have proper tests where overflow occurs, then changing intmask should show them the problems
<cfbolz>
Big if, of course
mattip has joined #pypy
marky1991 has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
<cfbolz>
arigato: of course rbigint uses intmask in the wrong way
marr has joined #pypy
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 b8f121ce766d /: a branch where to fix again the interaction of vmprof ans stacklets; in particular, after f...
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 2f204b1c432c /rpython/rlib/rvmprof/: move {start,stop}_sampling inside the VMProf API, and start to write a fake class to test t...
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 d5d42f493530 /rpython/rlib/rvmprof/test/: WIP: introduce a pytest fixture which allow us to easily use a global FakeVMProf instead of...
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 b85210ca9c20 /rpython/rlib/rvmprof/: 1) we can't monkey-patch _get_vmprof because it's imported in two places; insead it's easie...
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 ff5fd2e1f430 /rpython/rlib/rvmprof/test/support.py: add a pytest finalizer to check that we called {start,stop}_sampling an even amount of time...
<antocuni>
cfbolz, arigato: do you have any idea on how to write a test for ff5fd2e1f430?
<antocuni>
i.e., to check that if I forget to call start_sampling, the finalizer complains?
<antocuni>
ronny: or maybe you :) ^^^
<cfbolz>
nope, sorry :-(
<antocuni>
ok, np
<antocuni>
I suppose it's not strictly necessary, although it would have been nice to have
marky1991 has quit [Ping timeout: 240 seconds]
<fijal>
cfbolz: I'm against making intmask(signed) explode
<fijal>
It's a useful tool for trying to debug annotation errors
<arigato>
then maybe explode during rtying?
yuyichao_ has quit [Ping timeout: 276 seconds]
<cfbolz>
arigato: explode how?
<arigato>
crash?
<cfbolz>
How is that different?
<arigato>
sorry, I don't understand your question
<cfbolz>
What is 'crash during rtyping' solving?
<cfbolz>
I suppose I don't understand how to use intmask to debug annotation errors in the first place
<arigato>
ah, for fijal, sorry now I get it
<arigato>
I guess he's using intmask to figure out where an unsigned int comes from
<arigato>
not sure though
<arigato>
crashing later makes it usable in the same way if the goal is only to track annotation problems
<cfbolz>
Anyway, trying to make rbigint not use intmask(signed) would already be quite a mess, because it uses intmask in type specialized functions, where only one of the types is Signed
<arigato>
right
<fijal>
Yes that thing
<fijal>
I sometimes use intmask all over the place because I'm not sure where Uint comes from
<fijal>
And functions are specialized
<fijal>
I vastly prefer noop to crash
<fijal>
arigato: I have an idea how to micro benchmark all the Unicode ops
<fijal>
And then we can make a decision, but we need sre
<cfbolz>
arigato: So, then we do what, only update docs and add a warning to __pypy__.intop
<cfbolz>
?
yuyichao_ has joined #pypy
slacky__ has quit [Ping timeout: 240 seconds]
slacky has joined #pypy
<arigato>
guess so
<cfbolz>
arigato: seems eg the tuple hash computation in the objspace relies on wraparound
<cfbolz>
maybe the answer is for me to stop being annoyed and just hope that the C compiler will not just start exploiting this undefinedness
<cfbolz>
(or the jit, for that matter, since there is also a hash version with jit driver)
<arigato>
:-/
<cfbolz>
I guess I'll file an issue, we can then go an ignore that for the next ten years
<cfbolz>
done
Arfrever has joined #pypy
zmt00 has joined #pypy
<fijal>
cfbolz: hahaha :-)
<cfbolz>
sad but true
<cfbolz>
to do it correctly, we would have to be careful everywhere, that's too much work, so we do nothing and are optimistic
leshaste has quit [Quit: Leaving]
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 2907f533041c /rpython/rlib/: move the vmprof API needed by rstacklet from cintf to rvmprof/__init__.py, to integrate bet...
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 d3c13697bffe /: make check_status a real method, so that it can be tested and used also without the fixture
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 85cbd648e7dd /: WIP: add a failing test which shows that we are not taking samples inside callbacks
<kenaan>
antocuni fix-vmprof-stacklet-switch-2 a48521eb6944 /pypy/module/_continuation/test/test_stacklet.py: rename the fixture to app_fakevmprof and use it on all tests: this way, it automatically ch...
drolando has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
yuyichao has joined #pypy
drolando has joined #pypy
yuyichao_ has quit [Ping timeout: 248 seconds]
drolando has quit [Remote host closed the connection]
drolando has joined #pypy
hubert0 has joined #pypy
Arnout_ has joined #pypy
pilne has joined #pypy
<Arnout_>
Hey, this is something for the python/cffi setuptools_ext maintainers. We've just had to change how one of our cffi related modules builds and added cffi_modules to our setup.py. This broke our build. I think I tracked the root of the issue down to setuptools_ext.py, specifically _add_py_module importing build_py (and likely build_ext) from distutils instead of setuptools. This results in any setuptools functionality getting 'lost'
<Arnout_>
_add_c_module just above it already seems to do the right thing