cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
lritter has quit [Remote host closed the connection]
Rhy0lite has joined #pypy
<kenaan>
arigo py3.6 29268d8eba51 /pypy/interpreter/: Update logic to CPython 3.6.9 for setting f_lineno. Issue #3066 is not resolved; I guess CPython doesn't do the same...
<kenaan>
arigo py3.6 d9f0a8eedf66 /pypy/interpreter/test/apptest_pyframe.py: Skip a test that now crashes just like it does on CPython
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
<kenaan>
arigo py3.6 68e1f1bd6ea6 /pypy/interpreter/: Issue #3066: fix just by not removing dead bytecodes
<arigato>
maybe fixed? at least it should work as well as it does on CPython now, which means it can crash in one situation
<arigato>
but I'm not going to care about that as long as CPython doesn't, I think
<arigato>
or until we get another bug report
<arigato>
it's a problem that was fixed in the past by more obscure logic in the very long function fset_f_lineno()
<arigato>
now I just copied CPython's new version of the logic, which fixes other things but not this
<mattip>
arigato: thanks. I guess we mean it when we say "bug-for-bug compatible"
xcm has quit [Read error: Connection reset by peer]
xcm has joined #pypy
antocuni has quit [Ping timeout: 268 seconds]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
<ronan>
arigato: 68e1f1bd6ea6 is quite a big change, I'm a bit worried about unforeseen consequences
xcm has quit [Remote host closed the connection]
<ronan>
nedbat: any comments on this?
xcm has joined #pypy
antocuni has joined #pypy
<mattip>
"test_trace_onliner_if" fails for me with "python3 pytest.py -D pypy/interpreter/test/apptest_pyframe.py -v"
marky1991 has joined #pypy
<ronan>
mattip: looks like CPython's compiler optimises the bytecode more
YannickJadoul has joined #pypy
<cfbolz>
ronan: would you be less worried if I review the new code
<cfbolz>
?
YannickJadoul has quit [Quit: Leaving]
<ronan>
cfbolz: I'm not too worried about fset_f_lineno(), but rather about the consequences of not removing dead code after a 'return'
<ronan>
e.g. we have to think about what happens if you jump past a RETURN_VALUE
<ronan>
and I think coverage will now report dead code as uncovered
<YannickJadoul>
Which is probably due to me rushing into the problem during the sprints, rather than reading the docs well, but ... if someone could give a hint on where to look; I can't immediately work out what's going on :)
<cfbolz>
YannickJadoul: can you use a pastebin to paste the full error somewhere?
<ronan>
YannickJadoul: RPython is not very well documented :-(
<ronan>
I guess that you're appending to a list that's supposed to have a fixed length
<YannickJadoul>
Sure, I could also create the PR already and add onto it later?
<cfbolz>
(yes, this is one of the most bizarre parts of rpython)
<YannickJadoul>
Hmmm, that indeed seems to solve the error :)
<YannickJadoul>
I'm getting some new one, but that seems unrelated
<cfbolz>
YannickJadoul: bases_w is a resizable list, since you append or extend it. new tuple needs a fixed sized one
<cfbolz>
The slice makes a copy, which converts one to the other?
<YannickJadoul>
cfbolz: Oh, right, that makes more or less sense. And it's not ridiculously expensive to do so, then? There's not need to find another solution?
<cfbolz>
YannickJadoul: no, the cost is fine. In general, class initialization is not on any critical paths
bizulk has joined #pypy
<YannickJadoul>
OK, there are still other annotation errors, it seems. It's maybe easier to push this as a PR, so someone can look at the whole set of changes and see where I went wrong
<cfbolz>
YannickJadoul: if you want, we can continue. What's the next error?
<YannickJadoul>
cfbolz: Sure, OK
<YannickJadoul>
The next error is "Blocked block -- operation cannot succeed"
<fijal>
can you copy paste the whole error?
<fijal>
this means it's the last set of annotation errors, usually
<bizulk>
Hello ! I'am using cffi to wrap a dll. I have an error when using à big struct of around 400 MB, example : typedef struct __align mytest { int table[100000]; } mytest_t;
<fijal>
YannickJadoul: uh, seems maybe like the Fake stuff is not good enough?
<bizulk>
in the python interpreter : mytest = _panda_vcom.ffi.new("mytest_t*") _panda_vcom.ffi.sizeof(mytest) returns 4
<bizulk>
If I reduce the size 100 000 to 100 for example, the allocation is correct
<bizulk>
is there some limitations to cffi to memory allocation ?
<YannickJadoul>
fijal: I've absolutely no clue what that is doing, to be honest. I've only dived into PyPy at the sprints a few weeks ago, and the first part of PEP 560 didn't have these problems
<YannickJadoul>
(Happy to learn, though, if anyone has time)
tsaka__ has quit [Remote host closed the connection]
<fijal>
YannickJadoul: make _update_bases return new_bases_w[:]
<fijal>
(a copy)
tsaka__ has joined #pypy
<arigato>
ronan: so, I'm confused, because both CPython 3.6 and pypy3 trunk without my hack to assemble.py actually pass the test
<arigato>
I guess this hack to assemble.py was needed to pass the test before some other changes I did to fset_f_lineno()
<arigato>
I'll just revert it now
<ronan>
arigato: well, if you copied the logic, pypy3 should behave almost exactly like CPython, so it's not surprising
<kenaan>
arigo py3.6 63d044a9a08a /pypy/interpreter/astcompiler/assemble.py: #3066: revert the change in the code, which was not needed after all
<arigato>
ronan: yes. I'm just confused because I thought it didn't work, and (too quickly) concluded that it was because CPython didn't remove things after RETURN
<arigato>
but it does
<arigato>
I strongly feel that now, both CPython and PyPy pass that test a bit by chance
<arigato>
not too surprized given that I can now build a case where CPython will run bytecodes with an unexpected number of items in the block stack
<ronan>
yes, there are issues with that code
<arigato>
(I guess you can get funny segfault or attacks that way)
<ronan>
I've been wondering about that, but I couldn't get anything nastier than a SystemError
<arigato>
the now-skipped test_f_lineno_set_2() fails with a Py_FatalError
<YannickJadoul>
fijal: Yesss, that seems to make things better :-)
<arigato>
I'm sure it can be tweaked to make more interesting crashes
<YannickJadoul>
Still an error left during "pypy/bin/checkmodule.py __builtin__", but not showing up during full annotation (for now)
<fijal>
YannickJadoul: yes, so maybe we give up - I'm not convinced checkmodule worked for __builtin__
<fijal>
the problem is that checkmodule fakes a whole bunch of stuff and it's imperfect
<fijal>
(and __builtin__ module pokes a lot of things internally)
<bizulk>
well it seem that cdef does not display the correct array size
<fijal>
yes, it definitely explodes
<fijal>
YannickJadoul: probably not worth fixing, was a bad idea to try it in the first place
<YannickJadoul>
fijal: Right, OK, so there's something wrong with that mock-version, then? Then things start making sense :)
<YannickJadoul>
I've got an error during `compile_c` now
<fijal>
well, __builtin__ module is a bit big and poking inside the modules etc.
<fijal>
never had a chance to work
<fijal>
ok, please copy-paste the error? this seems rare
<YannickJadoul>
fijal: Ooooh, right; I assumed it were errors, since in red
<fijal>
warnings
<fijal>
C gets better and better at having warnings
<fijal>
maybe we should fix those, but they're not critical
<YannickJadoul>
fijal: Yeah, no, I should've noticed. Sorry
<YannickJadoul>
Now I just need to fix that last thing, setting `__orig_bases__` only when there's been substitutions
circ-user-KIvj2 has quit [Remote host closed the connection]
circ-user-KIvj2 has joined #pypy
circ-user-KIvj2 has quit [Ping timeout: 240 seconds]
<mental32>
I was wondering, why did pypy choose mercurial and bitbucket over say git and github/gitlab?
<mattip>
mercurial is still the best choice, and gitlab did not exist
<cfbolz>
bizulk: is this a 32bit machine?
<fijal>
mental32: we're unhappy with git and github requires git
<marmoute>
pypy is a good testcase for heptapod by the way. George have been detecting/fixing a couple of bug but trying pypy imports
<fijal>
I'm still excited to try heptapod, even if we have to host it
<arigato>
bizulk: I think that exactly as you asked, "ffi.sizeof(mytest)" is 4 because that's the size of the pointer
<mental32>
mattip: I dont understand what makes mercurial "the best choice", granted I haven't used it much so I'm still in the dark for what its benefits are.
<fijal>
mental32: git does not really have branches, is one thing
<arigato>
bizulk: maybe you're comparing ffi.sizeof(mytest[0]) with ffi.sizeof(mytest)
<fijal>
mental32: we're not too happy with bitbucket (for a while by now), but there weren't that many mercurial hosting options
<fijal>
and I think we have been less unhappy with bitbucket than with git
<mattip>
marmoute: waiting patiently :)
<marmoute>
mattip: we are getting there :-)
<fijal>
marmoute: hey, how are you?
<marmoute>
fijal: too busy, for a change…
<fijal>
hah
<marmoute>
fijal: how are you?
<fijal>
good or bad?
<fijal>
I'm on holiday :)
<arigato>
maybe let's say "git does not have *named* branches at all"
<marmoute>
fijal: mix of good and bad busy.
<mental32>
fijal, arigato: I'm confused as to what else you might be refering to when naming a branch, `git checkout -b <branchname>` seems like it does the job?
<marmoute>
mental32: mercurial's named branch are not just pointers, they are immutable property of the changesets.
<marmoute>
Help to understand older history
<fijal>
marmoute will do a better job at explaining than me :-)
<marmoute>
o:-)
<cfbolz>
Maybe we need to have a 'no version control discussions' policy in this channel again, though I suspect that is hard as long as we don't know where to go
<fijal>
:]
<mental32>
marmoute: ah yes I've read about mercurial changeset history being immutable, but I still dont understand the benefits, is it a feature that makes traversing branch histories simpler?
<marmoute>
mental32: well the point here is not so much about the immutability than the fact each changesets bare branch information.
<marmoute>
mental32: regarding immutability, mercurial is mostly like git. You can rewrite commit, it alter their hashes.
<mental32>
marmoute: so its about changesets being able to backrefrence their branch of origin?
<marmoute>
mental32: to avoid the possible consequence, mercurial has a "phases" concept, were changeset move the "public" phase and immutability is enforced.
<cfbolz>
I recommend this paper about why git is so hard, btw: Purposes, Concepts, Misfits, and a Redesign of Git
<marmoute>
mental32: yes. just image all git commit having a field recording the active branch when committed.
<mental32>
@cfbolz: thanks! I'll give it a read
<mental32>
marmoute: thanks for clarifying mercurial branches, I see their benefits more clearly now :)
<marmoute>
cfbolz: I am trying to keep it informative and civil. Let me know if I fail.
<arigato>
cfbolz: thanks for the paper
tsaka__ has quit [Remote host closed the connection]
<Dejan>
we should all switch to fossil-scm
<Dejan>
0:)
<Dejan>
fossil > (git & mercurial)
tsaka__ has joined #pypy
<marmoute>
I though fossil was in the ground ?
<marmoute>
(cheap joke I conceed)
<cfbolz>
YannickJadoul: thanks a lot for taking this on, btw, welcome to the project :-)
<YannickJadoul>
cfbolz: Thanks :) It's quite exciting, actually; I'm surprised how easy it was to add something without understanding most of what's going on
<YannickJadoul>
But credit to ronan for the amazing mentoring at the sprints ;)
<antocuni>
adding things without understanding what's going on is exactly what we do all the time :)
<ronan>
YannickJadoul: :-)
<YannickJadoul>
Hahaha, fair enough :D I'm just curious where to go from here, if this PEP is finished
<YannickJadoul>
Easy enough to forget or not find the time to do more fun stuff
<cfbolz>
YannickJadoul: a nice followup project might be global __getattr__ support for modules
<YannickJadoul>
Oh, I encountered that one, I think. It's by the same author as PEP 560
<YannickJadoul>
Thanks for that pointer!
<mental32>
cfbolz: PyPy3 supports Python 3.6 but not module level getattr?
<mattip>
mental32: pep 562 is python3.7 and up
<mental32>
mattip: Ah my bad, I didn't realise that
<YannickJadoul>
fijal: Woohoo, fully translated! Thanks a lot! :)
<YannickJadoul>
>>>> import typing
<YannickJadoul>
>>>>
<cfbolz>
YannickJadoul: Yay, congrats
<YannickJadoul>
cfbolz: Let's see; the proof of the coding will be in the nightly tests, I assume? ;)
<mental32>
Shouldn't the Makefile use $(RUNINTERP) when resolving $(URAM)? or shouldn't $(URAM) explicitly use `python2` instead of `python` to account for systems where py3 is the default symlink of `python`?
<mattip>
mental32: no-one really uses the Makefile, perhaps we should remove it or add a test for it. PR welcome.
<mental32>
mattip: What method should I use to build pypy instead of the makefile
<mattip>
just "pypy2 ../../rpython/bin/rpython -Ojit targetpypystandalone.py"
tsaka__ has quit [Remote host closed the connection]
tsaka__ has joined #pypy
<mattip>
everything else is just decoration
<mental32>
mattip: I haven't got pypy2, should I sub it for `pypy` which uses Python 3.6.1? or do I need a pypy build which runs py2?
<mattip>
the cffi_imports target is built as part of running targetpypystandalone.py, so it is redundant
<cfbolz>
mental32: you need a python2 to build any version of pypy
<mattip>
you can run "python2 ../../rpython/bin/rpython -Ojit targetpypystandalone.py", it works but takes longer and may use more memory
<mental32>
cfbolz, mattip: I see, thanks! also should I be in the repository root when I run the command? I see relative paths in the arguments and I'm unsure.
<cfbolz>
mental32: in pypy/goal/
<mental32>
cfbolz: thank you
tsaka__ has quit [Remote host closed the connection]
tsaka__ has joined #pypy
<mental32>
Out of curiosity why do pypy builds take so much time and resources to build?
tsaka__ has quit [Remote host closed the connection]
tsaka__ has joined #pypy
marky1991 has quit [Read error: Connection reset by peer]
marky1991 has joined #pypy
tsaka__ has quit [Remote host closed the connection]
tsaka__ has joined #pypy
<cfbolz>
mental32: it needs to perform type inference, and it automatically generates a JIT compiler
tsaka__ has quit [Remote host closed the connection]
<mental32>
cfbolz: so the build time is wholely dependant on rpython/
tsaka__ has joined #pypy
<cfbolz>
mental32: no? It's also because python itself is a pretty large language
tsaka__ has quit [Remote host closed the connection]
YannickJadoul has quit [Quit: Leaving]
tsaka__ has joined #pypy
tsaka__ has quit [Remote host closed the connection]
tsaka__ has joined #pypy
<fijal>
and we never invented separate compilation
tsaka__ has quit [Remote host closed the connection]
tsaka__ has joined #pypy
tsaka__ has quit [Ping timeout: 265 seconds]
antocuni has quit [Ping timeout: 268 seconds]
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
marky1991 has quit [Ping timeout: 265 seconds]
Rhy0lite has quit [Quit: Leaving]
mattip has quit [Ping timeout: 252 seconds]
mattip has joined #pypy
forgottenone has quit [Remote host closed the connection]