cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "nothing compares to the timeshifter, my personal polar expedition in software" - pedronis
marky1991 has joined #pypy
marky1991 has quit [Ping timeout: 252 seconds]
lritter has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
Garen has quit [Remote host closed the connection]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 248 seconds]
exarkun has quit [Ping timeout: 264 seconds]
exarkun has joined #pypy
cloudyplain has joined #pypy
_whitelogger has joined #pypy
coreteam has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 260 seconds]
jcea has quit [Quit: jcea]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 240 seconds]
ArneBab has joined #pypy
ArneBab_ has quit [Ping timeout: 260 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
exarkun has quit [Ping timeout: 240 seconds]
exarkun has joined #pypy
lritter_ has joined #pypy
lritter has quit [Ping timeout: 240 seconds]
rokujyouhitoma has joined #pypy
demonimin has quit [Ping timeout: 264 seconds]
demonimin has joined #pypy
rokujyouhitoma has quit [Ping timeout: 248 seconds]
coreteam has quit [Quit: Leaving]
cloudyplain has quit [Remote host closed the connection]
cloudyplain has joined #pypy
cloudyplain has quit [Ping timeout: 252 seconds]
pilne has quit [Quit: Quitting!]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 255 seconds]
cloudyplain has joined #pypy
cloudyplain has quit [Remote host closed the connection]
cloudyplain has joined #pypy
cloudypl_ has joined #pypy
cloudyplain has quit [Ping timeout: 240 seconds]
rokujyouhitoma has joined #pypy
cloudypl_ has quit [Ping timeout: 248 seconds]
rokujyouhitoma has quit [Ping timeout: 240 seconds]
jamesaxl has quit [Read error: Connection reset by peer]
jamesaxl has joined #pypy
exarkun has quit [Ping timeout: 240 seconds]
dw_ has joined #pypy
exarkun has joined #pypy
plan_rich_ has joined #pypy
gclawes has joined #pypy
bogner has joined #pypy
marmoute_ has joined #pypy
holdsworth has joined #pypy
riddle has quit [Disconnected by services]
riddle has joined #pypy
_zware has joined #pypy
lritter_ has quit [*.net *.split]
dw has quit [*.net *.split]
__main__ has quit [*.net *.split]
plan_rich has quit [*.net *.split]
kenaan has quit [*.net *.split]
[0__0] has quit [*.net *.split]
chris| has quit [*.net *.split]
SunDwarf has quit [*.net *.split]
shodan45 has quit [*.net *.split]
holdsworth_ has quit [*.net *.split]
bogner_ has quit [*.net *.split]
gclawes_ has quit [*.net *.split]
Cheery has quit [*.net *.split]
michaelgreene has quit [*.net *.split]
fryguybob has quit [*.net *.split]
zware has quit [*.net *.split]
marmoute has quit [*.net *.split]
shodan45 has joined #pypy
SunDwarf has joined #pypy
chris| has joined #pypy
[0__0] has joined #pypy
fryguybob has joined #pypy
__main__ has joined #pypy
cloudyplain has joined #pypy
Cheery has joined #pypy
danchr has quit [Ping timeout: 240 seconds]
danchr has joined #pypy
michaelgreene has joined #pypy
adamholmberg has joined #pypy
lritter_ has joined #pypy
adamholmberg has quit [Ping timeout: 255 seconds]
Guest97482 has quit [Ping timeout: 240 seconds]
ceridwen has quit [Ping timeout: 246 seconds]
marvin has joined #pypy
marvin is now known as Guest54772
ceridwen has joined #pypy
forgottenone has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
exarkun has quit [Ping timeout: 260 seconds]
exarkun has joined #pypy
cloudyplain has quit [Remote host closed the connection]
cloudyplain has joined #pypy
kenaan has joined #pypy
<kenaan> arigo cffi/cffi 4b0741f26fcd /c/minibuffer.h: Apply the same patch as everyone else
arigato has joined #pypy
<arigato> mattip: regarding the PySlice_IndicesEx issue
<arigato> one generic way to fix all this mess is to no longer use #define PyFoo PyPyFoo
<arigato> but instead produce "static inline x PyFoo(y, z) { return PyPyFoo(y, z); }"
<arigato> or consider it as part of the refactoring we might do one day: declare them as non-exported function pointers with "void (*PyFoo)(y, z);", and fill all function pointers when the module is first imported
realitix has joined #pypy
adamholmberg has joined #pypy
<njs> it's amazing how many stupid problems that PySlice thing has caused
adamholmberg has quit [Ping timeout: 240 seconds]
cloudyplain has quit [Remote host closed the connection]
cloudyplain has joined #pypy
cloudyplain has quit [Ping timeout: 246 seconds]
cloudyplain has joined #pypy
<kenaan> arigo cffi/cffi 23fa387f3e05 /doc/source/whatsnew.rst: Document 4b0741f26fcd
<kenaan> arigo cffi/cffi 32ecdbdba707 /: Update version to 1.11.1
<kenaan> arigo default a3a897f9b96e /: Update to cffi 1.11.1
<kenaan> arigo cffi/cffi a5f702205e81 /c/: "char *" => "const char *" until Python 3.7 is happy
exarkun has quit [Ping timeout: 240 seconds]
exarkun has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 248 seconds]
marr has joined #pypy
arigato has quit [Ping timeout: 240 seconds]
realitix has quit [Ping timeout: 240 seconds]
asmeurer_ has quit [Quit: asmeurer_]
realitix has joined #pypy
oberstet has joined #pypy
adamholmberg has joined #pypy
TheAdversary has quit [Ping timeout: 246 seconds]
cstratak has joined #pypy
cstratak has quit [Remote host closed the connection]
Hasimir has quit [Ping timeout: 240 seconds]
cstratak has joined #pypy
adamholmberg has quit [Ping timeout: 260 seconds]
Hasimir has joined #pypy
TheAdversary has joined #pypy
inhahe_ has quit []
cloudyplain has quit [Ping timeout: 255 seconds]
cloudyplain has joined #pypy
inhahe_ has joined #pypy
the_drow has quit [Ping timeout: 248 seconds]
exarkun has quit [Ping timeout: 252 seconds]
exarkun has joined #pypy
the_drow has joined #pypy
the_drow has quit [Ping timeout: 255 seconds]
the_drow has joined #pypy
arigato has joined #pypy
infernix has quit [Ping timeout: 255 seconds]
yuyichao has quit [Ping timeout: 240 seconds]
<kenaan> arigo default 1495dbdd3cfd /pypy/module/cpyext/api.py: Attempt to hack for https://bugs.python.org/issue29943
_whitelogger has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
cloudyplain has quit [Read error: Connection reset by peer]
ronan has joined #pypy
<kenaan> arigo cffi/cffi[release-1.11] 8adcf995d67c /: hg merge default
ronan has quit [Read error: Connection reset by peer]
exarkun has quit [Ping timeout: 248 seconds]
exarkun has joined #pypy
ronan has joined #pypy
marr has quit [Ping timeout: 240 seconds]
ronan has quit [Read error: Connection reset by peer]
<kenaan> arigo cffi/cffi 989d98a58c95 /c/misc_thread_common.h: More fun with _PyThreadState_Current becoming undefined in 3.7.
<kenaan> arigo cffi/cffi[release-1.11] aadf924dcf23 /c/misc_thread_common.h: hg merge default
<kenaan> arigo cffi/cffi 7005b3b4e54f /doc/source/whatsnew.rst: Mention the fix 989d98a58c95
<kenaan> arigo cffi/cffi[release-1.11] d3e081232b0b /doc/source/whatsnew.rst: hg merge default
arigato has quit [Quit: Leaving]
ronan has joined #pypy
lritter_ has quit [Ping timeout: 240 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 246 seconds]
the_drow has quit [Ping timeout: 246 seconds]
lritter_ has joined #pypy
the_drow has joined #pypy
Rhy0lite has joined #pypy
ronan has quit [Read error: Connection reset by peer]
<kenaan> amauryfa py3.5 826d99189c3b /: Try to fix translation on Windows: skip socket.sendmsg and associated macros.
lritter_ has quit [Ping timeout: 240 seconds]
raynold has quit [Quit: Connection closed for inactivity]
the_drow has quit [Ping timeout: 260 seconds]
marr has joined #pypy
adamholmberg has joined #pypy
the_drow has joined #pypy
ronan has joined #pypy
dmalcolm has quit [Remote host closed the connection]
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
the_drow has quit [Ping timeout: 240 seconds]
adamholmberg has quit [Ping timeout: 246 seconds]
dmalcolm has joined #pypy
ronan has quit [Read error: Connection reset by peer]
ronan has joined #pypy
adamholmberg has joined #pypy
the_drow has joined #pypy
<kenaan> fijal default b67bc051d0c1 /pypy/module/_ssl/interp_ssl.py: (fijal, arigo) change 10M to 10k, we have no clue what that number should be
ronan has quit [Read error: Connection reset by peer]
jcea has joined #pypy
Pimster has joined #pypy
<Pimster> this runs in 4.2 seconds https://paste.gnugen.ch/raw/oxPB, but this runs in 30s https://paste.gnugen.ch/raw/CdAG
<Pimster> pypy 5.8.0, cpython 3.6.2, and cpython 2.7.14 behaves the same as cpython 3
<LarstiQ> Pimster: don't know if pypy3 differs from pypy2 here, but at least doing the elementwise access is slow
forgottenone has quit [Read error: Connection reset by peer]
<Pimster> I know it's supposed to be slow but cypthon is 10x faster so it seems odd
<LarstiQ> Pimster: how about a.dot(b)?
<fijal> Pimster: no, 10x is not odd at all
marky1991 has joined #pypy
<fijal> Pimster: we have to emulate refcounting, raw objects and tons of other crap
<Pimster> LarstiQ: the performance of that is the same in cpython and pypy
<LarstiQ> Pimster: ^^ what fijal said
<fijal> it's actually IMPRESSIVE it's only 10x and works at all
forgottenone has joined #pypy
<LarstiQ> Pimster: keep boundary crossing to a minimum
<fijal> Pimster: we can definitely make it fast, but it seems to be the case like GIL
<Pimster> refcounting what? the loop isn't creating any reference
<fijal> everyone likes to complain about it, but it's not worth even a bit of money to anyone
<fijal> so it's not done
<fijal> Pimster: refcounting to emulate C API from CPython to access numpy
<fijal> if you don't use numpy it'll be like 100x faster than cpy
<Pimster> ok thank you that's why, I wanted to try to see if JIT improved the cpython perfomance but apparently this in not a good usecase
<fijal> if you use numpy, it's using a very complex compatibility layer
<LarstiQ> Pimster: turn the numpy arrays into lists, then do the loop
<fijal> as I said, it's possible to make it very very fast (like 100x)
<LarstiQ> (and maybe use regular floats instead of numpy.float64)
<simpson> Pimster: For the kind of access that you're doing, numpy can be *slower* than boring array.array or lists. Is this example representative of an actual workload that you have?
adamholmberg has quit [Remote host closed the connection]
<Pimster> simpson: no it was just an experiment, I don't loop over np arrays
realitix has quit [Ping timeout: 248 seconds]
<Pimster> indeed with a normal list pypy runs in 285ms and cpython in 1900ms
<Pimster> with dot product in numpy 19 ms
<fijal> Pimster: it would be even faster if you put it in a functions
<fijal> Pimster: also 19ms must exclude allocation time right?
<fijal> second run is 19ms on my computer
<Pimster> fijal: yes without allocation
adamholmberg has joined #pypy
<fijal> (on pypy if you exclude allocation)
realitix has joined #pypy
<Pimster> I get 66ms on your snippet excluding allocation, 200ms if I use a list of float random values instead of 0s
<fijal> so the creation of floats must put Nones or ints there and make it less efficient
<fijal> use array module for example to force it
<fijal> how do you get 66ms?
<fijal> I mean - 66ms might include JIT compilation, run it a few times?
jcea has quit [Quit: jcea]
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/3402 [amaury: force build, py3.5]
<Pimster> maybe there's some degree of lazyness when dot() is on [0.] * 1000000 because if I run dot() on np.zero(1000000).tolist() it's 3x slower (also excluding allocation)
<kenaan> cfbolz regalloc-playground ad2fa98841d9 /: merge default
<kenaan> cfbolz regalloc-playground 81d83bb43b83 /rpython/jit/backend/x86/: adapt to use of constants in the new call hint code
jcea has joined #pypy
<fijal> Pimster: np.zero().tolist() will not create an optimized list
<fijal> or in fact will create a very shitty list that's optimized for usage with CPython C API
<fijal> Pimster: generally speaking, if you want speed don't touch numpy APIs at all
<Pimster> gotcha
<LarstiQ> that can be made more nuanced, but having to emulate the cpython api is a real cost
yuyichao has joined #pypy
<Pimster> well in the end it seems that running np.dot on 2 arrays which are cache hot is just twice as faster as the pypy function on two cache hot lists
<Pimster> I have an ipython notebook if you want to see the results
<pjenvey> fijal: I was just wondering about that SSLContext memory pressure size
<pjenvey> SSLSocket needs one (I'm not sure what size)
<fijal> yeah
<fijal> Pimster: 2x seems about right
infernix has joined #pypy
<pjenvey> it might be larger than 10k with the cert store
<pjenvey> Alex_Gaynor: you must know the averge sizeof every openssl struct by now, what's a good memory pressure size? =]
<Alex_Gaynor> pjenvey: uhh, for what?
<pjenvey> SSLContext (ctx) and _SSLSocket (SSL and maybe an x509 peer cert)
<Alex_Gaynor> x509 certs are big, maybe 1024 bytes
<Alex_Gaynor> shit an SSL* is huge as well
<pjenvey> yea isn't it just > 10k per connection overhead or so
<Alex_Gaynor> a lot of that is in the buffers, which are managed dynamically
<pjenvey> ah yes
<fijal> Alex_Gaynor: I changed 10M to 10k
realitix has quit [Ping timeout: 260 seconds]
_zware is now known as zware
ronan has joined #pypy
ronan has quit [Ping timeout: 252 seconds]
Arfrever has joined #pypy
raynold has joined #pypy
cstratak has quit [Ping timeout: 260 seconds]
asmeurer_ has joined #pypy
<Alex_Gaynor> fijal: 10k seems reasonable
oberstet has quit [Ping timeout: 260 seconds]
Arfrever has quit [Ping timeout: 255 seconds]
<kenaan> fijal memory-accounting 336c50a8852e /rpython/: (arigo, fijal) start a new branch that tries to account better for tracked memory
forgottenone has quit [Read error: Connection reset by peer]
forgottenone has joined #pypy
oberstet has joined #pypy
<bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/3402 [amaury: force build, py3.5]
cstratak has joined #pypy
cstratak has quit [Ping timeout: 248 seconds]
marky1991 has quit [Ping timeout: 252 seconds]
Rhy0lite has quit [Quit: Leaving]
jamesaxl has quit [Read error: Connection reset by peer]
jamesaxl has joined #pypy
<mattip> could someone merge default into py3.5 to get cffi 1.11.1 before the release?
<kenaan> mattip default c1b14bf66a8e /pypy/doc/release-v5.9.0.rst: be more positive about numpy,pandas
<kenaan> mattip release-pypy3.5-5.x 24686d8f9ef3 /: merge py3.5 into branch
<kenaan> mattip default 46f3190ace68 /pypy/doc/release-v5.9.0.rst: update release notes for latest merges
<kenaan> mattip release-pypy2.7-5.x fedadd23d48b /: merge default into release for cffi 1.11.1
<kenaan> mattip py3.5 611fac54ebfd /pypy/module/cpyext/test/test_memoryobject.py: fix for c-89 (visual 9) compilation
<pjenvey> Alex_Gaynor: how does about 6k sound
<pjenvey> for SSL
<Alex_Gaynor> pjenvey: as random as anything else; sure, ship it.
<kenaan> pjenvey default 4302fe3d912e /pypy/module/_ssl/interp_ssl.py: unscientific memory pressure estimate of =~ 6k for SSL*
<kenaan> pjenvey release-pypy2.7-5.x e3ab7b0bd8ac /pypy/module/_ssl/interp_ssl.py: merge default
marky1991 has joined #pypy
Hasimir has quit [Ping timeout: 240 seconds]
<mattip> let's see if nightly tests pass before I start the release cycle
Hasimir has joined #pypy
<nanonyme> mattip, do they usually pass?
<mattip> nanonyme: ^^^
<nanonyme> Meh
marky1991 has quit [Ping timeout: 246 seconds]
<nanonyme> That import lock count changing looks basically like magic to me
<nanonyme> I guess I shouldn't expect to understand when the test code is written in RPython :p
forgottenone has quit [Quit: Konversation terminated!]
<mattip> nanonyme: it crept with the fix for issue 2628, changeset 167b802baf3b
ronan has joined #pypy
<mattip> yet another example of trying to duplicate silly CPython corner cases with magic
<mattip> note that it passes translated, but fails untranslated, running tests untranslated is magic**2
<ronan> mattip: pypy3 nightly can't import pandas :-(
<mattip> ronan: what's missing?
<ronan> mattip: there's some obscure incompatibility in code that runs at import time
* mattip looking at lxml 4, issue 2666
<ronan> TypeError: descriptor '_maybe_convert_freq' requires a 'pandas._libs.period._Period' object but received a 'type'
<ronan> while executing pandas/core/indexes/period.py(222) = "freq = Period._maybe_convert_freq(freq)"
<mattip> maybe some kind of parse error on the name, too many or too few pieces in the type name?
<ronan> no, it looks like it's because we have a bound metaclass method
marky1991 has joined #pypy
<ronan> ah no, it's something about classmethods in Cython cdef classes
<mattip> annoying
<ronan> I'll look at it tomorrow - I'm too zombified to do it today
<mattip> cool. it seems I will try to fix the lxml error which also seems to be a cython/pypy interaction issue
kipras has quit [Ping timeout: 246 seconds]
ronan has quit [Ping timeout: 252 seconds]
kipras has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 248 seconds]
yuyichao_ has joined #pypy
yuyichao has quit [Ping timeout: 248 seconds]
coreteam has joined #pypy
yuyichao has joined #pypy
yuyichao_ has quit [Ping timeout: 248 seconds]
pilne has joined #pypy
cloudyplain has joined #pypy
cloudyplain has quit [Remote host closed the connection]
coreteam has quit [Quit: Leaving]
<kenaan> pjenvey default 943002f74290 /pypy/doc/release-v5.9.0.rst: blurb 4302fe3d912e & b67bc051d0c1
TheAdversary has quit [Ping timeout: 260 seconds]
Hasimir has quit [Ping timeout: 240 seconds]
marky1991 has quit [Ping timeout: 252 seconds]
cloudyplain has joined #pypy
demonimin has quit [Ping timeout: 240 seconds]
demonimin has joined #pypy
cloudyplain has quit [Remote host closed the connection]
cloudyplain has joined #pypy
cloudyplain has quit [Ping timeout: 240 seconds]
demonimin has quit [Ping timeout: 248 seconds]
demonimin has joined #pypy
demonimin has joined #pypy
yuyichao has quit [Ping timeout: 260 seconds]
marky1991 has joined #pypy
marky1991 has quit [Ping timeout: 260 seconds]
yuyichao has joined #pypy