<cfbolz>
njs: given that we support getframe().f_locals I don't see why the proxy solution would be a problem for us
<cfbolz>
It's never going to be particularly fast, of course
rindolf has joined #pypy
<njs>
cfbolz: I guess in theory it's possible that dict(getframe().f_locals) could somehow be less inefficient than getframe().f_locals, but it's definitely not obvious to me how :-)
marvin has quit [Remote host closed the connection]
marvin has joined #pypy
<cfbolz>
njs: hm, I suppose the former could indeed be sort of efficient
<njs>
I assume with the proxy version, it would also depend on what you were doing with the proxy
<cfbolz>
njs: yes
<njs>
'l = locals(); del l' could be pretty cheap! 'locals(["foo"]' can be optimized to just 'foo'. but "{blah}".format(**locals()) could be... worse or better, for all I know
<cfbolz>
njs: as long as the semantics are fine, do you see a reason to not just have locals be very slow?
<njs>
I can certainly think of more important things than whether locals() is slow
<njs>
but since we have to decide between these different ways of doing locals(), it's nice to have some idea about what the consequences will be
<cfbolz>
njs: if you are super concerned about that you could always leave the loophole of deciding on a good semantics for CPython but then calling it implementation dependent
<cfbolz>
I think it's very hard to foresee a priori which of these are hard to optimize
<njs>
I doubt pypy would feel free to pick a different semantics for locals() here, given you historically stuck so close to cpython that you copied the wacky bug that triggered this whole thing :-)
<njs>
but still, that's a useful answer, thanks
<cfbolz>
njs: fair ;-)
<cfbolz>
njs: I think if we could pick a semantic we would probably make locals a real proxy to the local variables of the frame, just like globals
<cfbolz>
Would not be hard to implement in pypy
<cfbolz>
But I would pick that because it's nice and easy to explain and happens to be simple in pypy
<mattip>
pypy2 is now linking to less stuff. If I compare ldd pypy2 to ldd python3, we still have libbz2, librt, libcrypt, libffi, libtinfo, libgcc
<cfbolz>
mattip: nice!
<mattip>
any of those problematic for other c-extensions? they seem very stable (unlike libssl and libcrypto, which we no longer link to)
<mattip>
we could convert libbz2, libcrpt to cffi, but cost-benefit is marginal
Taggnostr has quit [Ping timeout: 252 seconds]
<mattip>
maybe I will take a look at copying filesystemDefaultEncoding and filesystemDefaultEncodeErrors from cpython/Python/pylifecycle.c
<mattip>
and then think about a new release
ronan has quit [Ping timeout: 252 seconds]
ajlawrence has joined #pypy
<njs>
mattip: I think psycopg2 will end up vendoring libcrypt, so that could become an issue
<antocuni>
cfbolz: yes, I started a branch to try to fix the build
<antocuni>
but I don't understand is why it fails now if it used to work
<cfbolz>
antocuni: maybe the default installed packages on travis changed?
<antocuni>
could be
marky1991 has joined #pypy
dayton has quit [Quit: Connection closed for inactivity]
<kenaan>
fijal extradoc e7f187d7eefa /talk/synerise2019/: my talk from today
<fijal>
antocuni: hey
<antocuni>
fijal: hi
<fijal>
so vmprof
<fijal>
I wonder how can we improve the setup
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
<ronan>
fijal: how about adopting some advanced DevOps processes, like not releasing when CI fails? ;-)
<antocuni>
fijal: I fixed the build in a branch, so unless you object I am doing a new release which hopefully will contain wheels
<antocuni>
fijal: but first, I'd like to know how you managed to do a source-only release, because I don't see the tag in git; did you manually upload it?
yaewa has joined #pypy
moei has quit [Ping timeout: 248 seconds]
<ronan>
antocuni: do we have the release process documented anywhere? We seem to have some fancy automation, but it's hard to figure out how it's supposed to work
<antocuni>
yes exactly, I have no idea
<antocuni>
I looked at travis scripts, and it seems that when TRAVIS_TAG is defined, we make wheels and upload them
<antocuni>
so I think that if you create a git tag and push, it will do a new release
<antocuni>
the problem is that each job/platform does it independently. So if things are broken on mac, we still upload the linux wheels
<ronan>
well, I guess it's not a problem as long as you make that the build passes before pushing the tag
<ronan>
hmm, where is it getting the credentials from, though?
<antocuni>
ronan: look at .travis.yml
<antocuni>
the wird "env" section contains encrypted env variables, which are automatically decrypted when you run things on travis
<antocuni>
I GUESS that it contains $PYPI_USERNAME and $PYPI_PASSWORD, which are used by upload-artifacts.sh
<antocuni>
it's a pretty standard procedure
<antocuni>
fijal: I also changed the irc notifications to be here instead of #baroque-dev, it sounds more useful
<antocuni>
ronan: " as long as you make that the build passes before pushing the tag" is not really TDD, eventually someone will forget and/or make a mistake
<antocuni>
I think that the usual procedure is to have separate jobs for building, testing and uploading (the latter being done only on TAGS and if ALL tests succeeded). But since vmprof is quick to build, it is probably find to have a testing phase which builds&tests and a deploy phase which builds&upload
marky1991 has quit [Read error: Connection reset by peer]
<fijal>
antocuni: yes, I did
<fijal>
ronan: yeah good idea. But it's not very useful if our CI tests nothing
<fijal>
(like the released version of vmprof didn't anything at all on OS X othr than hanging)
Masklinn has joined #pypy
<antocuni>
fijal: ok, please don't do it then :)
<fijal>
antocuni: sure, but in the current case I would do it anyway
<fijal>
it was the case of "I'm showing vmprof to clients and it completely refuses to work at all"
<antocuni>
well no, you break tons of deployment if you upload only the source code
<fijal>
but I also unbreak all the OS X
<fijal>
well, I would do the right thing if I knew how
<antocuni>
because then suddenly lots of build could no longer pip install it because they were missing libunwind
<antocuni>
I got tons of complaints about that
<fijal>
I'm unhappy about the whole setup
<antocuni>
yeah sure, I completely agree that vmprof is in a sorry state
<fijal>
why does it require libunwind?
<fijal>
what happens if you don't have it at runtime?
<fijal>
"which part of the world, OS X or linux with no libunwind is a better part of the world to completely break" is not a question we should be asking
<antocuni>
I'm not complaining to you, just pointing out a mistake so that we can avoid to do it in the future
<fijal>
yeah sure
<fijal>
how on earth did it pass tests on OS X?
<fijal>
do tests actually run it?
<antocuni>
fijal: I *THINK* that libunwind is bundled with wheels
<fijal>
is the OS X something old?
<antocuni>
ah I see the problem
<antocuni>
so, linux wheel are compiled inside the manylinux docker images and we run auditwheels on them
<antocuni>
so libunwind is bundled inside the wheels (which is correct)
<antocuni>
but for mac, I don't think we are running auditwheel repair (look at travis/script.sh)
<antocuni>
so the libs are NOT bundled inside the wheel. I claim that the actual bug is this, i.e. that our mac wheels are broken
<fijal>
do we ship wheels for pypy too?
<antocuni>
you could not ship pypy wheels until very recently, because pypy don't support manylinux1
<antocuni>
but now you can in theory build&upload manylinux2010 wheels
<fijal>
I would say step 1 is getting our CI to a point where we have any clue if it works
<fijal>
not running anything on any pypy is a great way to make sure it does not
<antocuni>
I agree
<fijal>
so yeah... as far as I'm concerned I run all the CI I care about
<fijal>
in hindsight
<fijal>
ok, maybe I have some time to spend on improving vmprof like that
<antocuni>
you didn't, mac tests were failing :-P
<antocuni>
(for a dump reason, but still)
<fijal>
yeah I looked at it
<fijal>
it seemed like nonsense to me
<antocuni>
I also agreed with gambit to spend some time on vmprof, but I keep on getting higher priority tasks :(
<fijal>
as in, if it does not run at all, the tests make no sense
<antocuni>
anyway, since we are not confident in our tests, that's a good reason to be super-careful when we do a new release
<antocuni>
but obviously we need to solve that
<fijal>
having nonsense tests is to me one step worse than no tests at all :-)
<fijal>
so we don't ship wheels for pypy, does that mean there is a libunwind dependency?
<fijal>
what do we use libunwind for anyway?
<fijal>
libunwind is not a reliable way to get a traceback
<antocuni>
you are asking tons of questions I can't answer for
<antocuni>
I haven't looked at vmprof for years
<ronan>
fijal: libunwind is used for getting native stack traces
<antocuni>
I was convinced we were using libunwind then at the last sprint you told me we don't :)
<fijal>
ronan: and what do we do with the fact that it corrupts traces in 10% of the cases or so?
<fijal>
we don't use it for python, for sure
<ronan>
antocuni: we only use it on CPython, AFAICT
<antocuni>
I suppose we should investigate whether newer versions work better. Also, if it's not completely reliable, we could probably leave it off by default by still give the user the possibility of turning it on if needed
<antocuni>
you can still get useful information by looking at native frames, IMHO
<fijal>
so what does native=True on pypy mean?
* fijal
looks into the source
panni_ has joined #pypy
<fijal>
my question is whether the ability to see native frames is something that warrants extra complexity of "what do we do with OS X?" "is libunwind installed?" etc.
<fijal>
libunwind does not work on OS X
<fijal>
so it should be not only disabled by default but also just plain out explode if you try to pass it
<fijal>
as far as I can tell, it does something on pypy
<fijal>
it certainly does something, otherwise it would not hang, I would think
<fijal>
maybe "it works and produces reliable tracebacks on pypy" would be a good start
<fijal>
(we are not there in more than one way)
<ronan>
fijal: yes, it does do something on pypy, though it's gated behind a function called init_cpyprof()
<antocuni>
if anybody feels like, when/if it succeeds, could you bump the version to 0.4.15, add a git tag and push? This should result into a new "working" release hopefully
antocuni has quit [Ping timeout: 252 seconds]
marky1991 has joined #pypy
<mattip>
btw, I added a '--only-binary vmprof' option to fix pypy's buildbot usage of vmprof in bc4cf55dc154
Tsundere_cloud has joined #pypy
marky1991 has quit [Ping timeout: 268 seconds]
marky1991 has joined #pypy
Masklinn has quit [Ping timeout: 246 seconds]
Masklinn has joined #pypy
<kenaan>
mattip cffi-libs2 4e4d2be6ff04 /: add cffi implementation of crypt