antocuni changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "PyPy: the Gradual Reduction of Magic (tm)"
danieljabailey has quit [Quit: ZNC 1.6.5+deb2build2 - http://znc.in]
<mjacob>
amaury: since you were working on the ast builder... let me know if you do more parser work, so we don't end up doing the same (e.g. now i'm implementing PEP 526)
<amaury>
mjacob: No, I was taking some random test failures.
mokos has joined #pypy
<mjacob>
amaury: that's what we all do ;) well, there are so many failing tests that the chance of two people doing the same is quite low
pie_ has joined #pypy
<mokos>
Hello - does it make the slightest sense, even in purely academic experimentation terms, to create something as demanding as a game engine using pypy ? Not necessarily now but perhaps in some years from now with pypy getting faster and faster ?
astronavt has quit [Ping timeout: 250 seconds]
<pie_>
im a bit confused, does pypy use the same binary for python 2 and 3?
<pie_>
well that may actually be orthogonal for the moment, i get this error when trying to import pyjq: ImportError: No module named _pyjq
<pie_>
im not sure if its a pypy issue or a nixos issue though
<njs>
mokos: lots of popular games are currently written in JITted languages like Javascript, Lua, C#, ...
<njs>
mokos: completely written, not just "the scripting part"
<mokos>
njs: Really ? Examples ?
<njs>
OTOH if you're talking about something like Unreal where the whole point is to push the hardware to the absolute limits, you need absolute latency predictability, etc., then probably not
<pie_>
oh it turns out i can just pass pypy to pypi2nix....
<mokos>
njs: Examples of games completely written in Pypy ?
<njs>
mokos: I don't know of any of those
<pie_>
amaury, thanks for the pointer! o/
<simpson>
pie_: Hi. Figure out the Nix side of things already?
<amaury>
pie_: also IIRC the extension modules are named differently. But "pypy setup.py" should work
<mokos>
njs: You said some game engines work with PyPY
<pie_>
simpson, oh hey. well not quite probably :P
<njs>
mokos: I did not say that, amaury did
<mokos>
Oh sorry
<mokos>
Misread
<pie_>
simpson, do you mean in general or with pypy
<mjacob>
mokos: there are a few problems with real-time applications on pypy (all of these are solvable):
<simpson>
pie_: I mean with PyPy. In the past, pypyPackages has been pretty good to me.
<mjacob>
mokos: one thing is the GC, which is incremental, but in the moment can't guarantee maximum pause times
<mjacob>
mokos: another is the JIT, which itself is not the fastest (e.g. due to PyPy's meta-tracing approach) and has a similar problem as the GC
<pie_>
simpson, ah i didnt see you in nixos so i kinda forgot you use it or something
<mjacob>
mokos: it's a lot easier if there's a specific problem we can look into (e.g. a piece of code where the garbage collection or JIT time is excessively high)
<mjacob>
mokos: improving the GC and JIT to guarentee (soft) real-time constraints is a lot more work... i'd guess around one man month
<mokos>
mjacob: May I pm you ?
<njs>
huh, I would have (completely uninformedly) have guessed way higher than 1 person-month for that
<mjacob>
mokos: sure
marr has quit [Ping timeout: 240 seconds]
<mjacob>
njs: admittedly, since i'm not a GC expert, this was more or less a wild guess
<njs>
my guess is totally worthless, I was more expressing surprise than disagreement :-)
<mokos>
mjacob: So this task not even on the roadmap of pypy ?
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
jcea has quit [Quit: jcea]
<njs>
mokos: scikit-learn basically needs numpy, scipy, cython, and at least numpy and cython should work these days, and I guess scipy is in progress (mattip would know the details)
<njs>
mokos: so it probably does work, or is on its way to working
<mokos>
ok !
citizen_stig has quit [Ping timeout: 276 seconds]
drolando has quit [Remote host closed the connection]
<fijal>
why can we no longer see stdio if the run is not completed from the buildbot?
<arigato>
the error you point to seems to be that "hg pull" was not done
<arigato>
ah no, wait a sec
<arigato>
yes, it's only that cfbolz started this build by putting a revision number; then both hg repos (benchmarks and regular pypy) are updated to that revision
<cfbolz>
The Nightly scheduler named 'nightly-1-00' triggered this build
<arigato>
uh what?
<cfbolz>
My name is just there because I wrote that commit?
<arigato>
aaaah ok sorry
<arigato>
so mattip changes seem to have the effect that the latest revision number is now included in the build properties, and as a result it does "hg up revision" even on the benchmarks repo
<cfbolz>
arigato: damn :-)
dddddd has joined #pypy
Jellyg00se has joined #pypy
slackyy has joined #pypy
amaury has joined #pypy
<fijal>
can we do something about the fact that benchmark run takes 8h?
<fijal>
sorry, 10h
<arigato>
note that this number may be wrong (or maybe not) because I don't know if it includes the time spent waiting for other processes to finish on the machine
<arigato>
I wouldn't be surprized
<arigato>
I think a run translates in parallel with other stuff, and then has to wait for hours before all that other stuff finishes
<fijal>
yeah ok
<fijal>
but it takes 10h of running
<arigato>
?
<arigato>
last one took 9h38 between start to finish, so that number includes all the time spent waiting
<arigato>
(which may or may not be zero)
<arigato>
do you have a hint showing that there was no waiting?
<arigato>
this one might have really started at 6h02 according to the waterfall
<arigato>
ended at 10:27
<arigato>
so seems to run for 4h25
<arigato>
I think that we're seeing these huge numbers just because the 32 and 64 benchmarks are on the same machine, and the 64 starts just after and so will wait for 4h25
<fijal>
ok
<fijal>
I see
<fijal>
then indeed it's not "that bad"
exarkun has quit [Ping timeout: 248 seconds]
exarkun has joined #pypy
oberstet has joined #pypy
dddddd has quit [Remote host closed the connection]
antocuni has quit [Ping timeout: 250 seconds]
amaury has quit [Quit: Konversation terminated!]
amaury has joined #pypy
amaury has quit [Quit: Konversation terminated!]
mattip_away has joined #pypy
<mattip_away>
fijal: fwiw I backed out the change to buildbot that made the hg pull fail
<mattip_away>
updated the buildmaster, but waiting to restart for builds to finish
<fijal>
Cool
<cfbolz>
mattip_away: thanks for your work on the buildbot, it's super important despite being neither fun nor glamorous
<cfbolz>
arigato: do we care about the result of cmp(nan, nan)?
jcea has joined #pypy
mattip_away has quit [Remote host closed the connection]
oberstet has quit [Ping timeout: 248 seconds]
<ronan>
cfbolz: it doesn't seem to make much sense on cpython, so maybe not?
<cfbolz>
ronan: that was my reaction as well
<cfbolz>
Should we document it?
<ronan>
it wouldn't hurt
<marmoute_>
Well, "NaN == Nan → False" is part of the IEEE 754 standard for floating point if I'm not mistaken
<marmoute_>
I would -really- recommand to stick to that standard
<ronan>
marmoute_: yes, but NaN < NaN and Nan > Nan are also False
<cfbolz>
marmoute_: of, so what do you suggest to return from cmp?
<fijal>
but it's not respected in python
<fijal>
not sidely
<fijal>
-1
<fijal>
that's what cpython returns :)
<marmoute_>
fijal: interresting tuple behavior (I can see where this is coming from)
<fijal>
I suppose 1 or 0 is as good
<fijal>
or 42
<marmoute_>
so cpython claim nan is lower than nan when using cmp, but then not deliver when using lower than comparison
<marmoute_>
because cmp cannot express that actually, it assume an order.
<marmoute_>
Interresting.
<ronan>
CPython's behaviour is that cmp(float('nan'), float('nan')) == -1 but nan=float('nan'); cmp(nan, nan) == 0
<marmoute_>
wow
* marmoute_
is happy to hve jumped in that discussion
<cfbolz>
CPython is also platform dependent, according to some white paper I saw
<ronan>
ooh, even nicer!
kirma has quit [Ping timeout: 255 seconds]
vstinner has joined #pypy
<marmoute_>
cfbolz: what is your definion of plateform here ? OS or hardware ?
exarkun has quit [Ping timeout: 248 seconds]
<cfbolz>
I didn't verify this statement
oberstet has joined #pypy
exarkun has joined #pypy
<cfbolz>
ah, I now understand why cmp(nan, nan) == 0 in pypy. it's an effect of "float('nan') is float('nan')"
oberstet has quit [Ping timeout: 268 seconds]
<arigato>
cfbolz: yes
<cfbolz>
ok, I'll document that
<arigato>
there is already some docs about this NaN effect
<arigato>
but please mention cmp too, I suppose
<kenaan>
cfbolz default 96cf224f204a /pypy/: test and document the cmp(nan, nan) == 0 behaviour
adamholmberg has quit [Remote host closed the connection]
<cfbolz>
arigato: yes, just added another sentence
<arigato>
thanks
<cfbolz>
arigato: is there a way to tell objectmodel.r_dict that the hash function that I give is fast, ie that the dict doesn't need to store the hash in each item?
<cfbolz>
I suppose there isn't so far
<arigato>
no
<cfbolz>
arigato: would that be hard?
<cfbolz>
arigato: use case is an rdict that still uses W_Unicode as key, but with a hash and eq that doesn't go through the space
<arigato>
indeed
<arigato>
no, it's probably not hard to add
<cfbolz>
cool, will give it a try then
<arigato>
should also not enable the "paranoia" mode of ridct
<arigato>
...rdict
<cfbolz>
arigato: what's that doing again?
<arigato>
checking for mutations of the dict when doing lookups
<cfbolz>
ah, right
<cfbolz>
arigato: that needs a flag on the dictdef, right?
<arigato>
I think a flag on the r_dict class, to start with, similar to 'force_non_null'
<arigato>
but yes, then the same flag on the DictDef
<cfbolz>
ok
<cfbolz>
I'll give it a try
<cfbolz>
arigato: thank you
<arigato>
thanks!
antocuni has joined #pypy
<kenaan>
rlamy unicode-utf8 09186de461ba /pypy/: Use the same logic for all encoders
<kenaan>
cfbolz default d6474db3c19f /rpython/rlib/debug.py: explain how to do mark_dict_non_null on r_dicts
adamholmberg has quit [Read error: No route to host]
adamholmberg has joined #pypy
mentalita has joined #pypy
<kenaan>
cfbolz rdict-fast-hash bd78548cee25 /rpython/: a branch to support fast hash functions with r_dict get as fast as the first test failing
<kenaan>
cfbolz rdict-fast-hash cba7d424ef73 /rpython/rtyper/lltypesystem/rordereddict.py: small refactoring: rename ENTRIES.hash to .entry_hash (hash is super hard to grep for), also make it take ...
<kenaan>
cfbolz rdict-fast-hash 6a993ce073f5 /rpython/rlib/objectmodel.py: explain what the arguments mean
<kenaan>
cfbolz rdict-fast-hash a903e9b5358b /rpython/: thread the fast_hash info through the various layers into the rordereddict implementation
<cfbolz>
arigato: seems to work. now I go through the code and look for r_dicts that have fast hash functions, I suppose
<arigato>
yes .-)
<cfbolz>
I was joking, but it seems there are a few
<fijal>
arigato: do you feel like fixing bugs in sre or should I have a look?
<arigato>
fijal: I can do that later tonight if you don't come to it first
<fijal>
I feel utterly useless today
<fijal>
so rather unlikely
oberstet has joined #pypy
<kenaan>
cfbolz rdict-fast-hash edf8ee0934a2 /: use fast_hash=True for two r_dicts
<antocuni>
btw: what is the use case for this? I guess dicts whose keys are rpython strings?
<cfbolz>
arigato: works for me
<cfbolz>
antocuni: no, for rpython strings this happens automatically
<cfbolz>
It's for wrapped rpython strings as keys
<epistax>
I'm having an issue with pypy and installing the 'cryptography' module on windows. On plain old python 2.7.12 I can pip install cryptography, but pypy can't seem to build the openssl dependency without intervention. A stack overflow post suggestions needing to set up OpenSSL separately so it can be built. Does it make sense that for pypy this is needed but for python it's self contained?
<arigato>
or more precisely, W_UnicodeObjects in this case
<antocuni>
uhm, and we cannot have a special strategy because you need to keep the character index somewhere?
<arigato>
epistax: that could be the case if cryptography is provided as a prebuilt "wheel" binary for CPython
<cfbolz>
antocuni: yes, and the length
<antocuni>
right
<epistax>
arigato: domo. Can I take advantage of that prebuilt wheel then for pypy or do I need to go through with the separate openssl setup?
<arigato>
no, the wheels contain binary versions of C extension modules, which need to be recompiled
<arigato>
so, sorry
<antocuni>
epistax: generally speaking, you don't find binary wheels compatible with PyPy on pypi.python.org (which is the index used by pip)
<Alex_Gaynor>
epistax: (cryptography maintainer here); the problem is that PyPy doesn't have a sufficiently stable ABI for us to produce wheels for it. For CPython we can get away with 1 wheel per major release per OS.
tazle has quit [Ping timeout: 264 seconds]
<epistax>
antocuni, Alex_Gaynor: Thanks for the infos. I'll import open ssl and give it a shot.
tazle has joined #pypy
<epistax>
I wish it didn't want to use Visual Studio 2008 to do this stuff. We have much newer compilers available! Oh well.
<Rotonen>
epistax: give libressl a spin if you want to use a newer visual studio?
<arigato>
I doubt openssl is the reason?
<epistax>
Nah it's not that. I think it's just pypy
antocuni has quit [Ping timeout: 276 seconds]
<epistax>
It looks like pypy is only available as a VC9 built 32 bit executable on windows for now.
<mattip>
epistax: right, we have not bene able to find a win64 champion. Here is some reading
<Rotonen>
mattip: i actually do quite a chunk of my workload at $WORK doing pywin32 / cpython 3.5 on both 32bit and 64bit - i'll not promise anything, but that kind of a churn hunt seems preliminarily appealing
tazle has joined #pypy
<mattip>
Rotonen: cool. There were various branches to start the work, mostly abandoned
<mattip>
"hg branches --closed |grep win64" seems like the last was win64-stage1, there is also 32ptr-on-64bit
<mattip>
both are _very_ old
<Rotonen>
as a starting point i'll rather build a rickety cardboard castle out of git submodules for the build script and the hack fork of cpython and rather fetch a release source tarball of pypy
<Rotonen>
i have no idea why, but over the years i've never managed to get along with mercurial, and i got along just fine with even RCS and CVS :P
<Rotonen>
so one would take a cpython 3.5 as the starting point?