antocuni changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | "PyPy: the Gradual Reduction of Magic (tm)"
danieljabailey has quit [Quit: ZNC 1.6.4+deb1 - http://znc.in]
danieljabailey has joined #pypy
asmeurer_ has joined #pypy
marky1991 has joined #pypy
asmeurer_ has quit [Quit: asmeurer_]
marr has quit [Ping timeout: 240 seconds]
slackyy has quit [Read error: Connection reset by peer]
adamholmberg has joined #pypy
slackyy has joined #pypy
tbodt has joined #pypy
ArneBab has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
ArneBab_ has quit [Ping timeout: 240 seconds]
adamholmberg has quit [Ping timeout: 240 seconds]
pilne has quit [Quit: Quitting!]
marky1991 has quit [Ping timeout: 260 seconds]
ArneBab_ has joined #pypy
ArneBab has quit [Ping timeout: 240 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 260 seconds]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
jamesaxl has joined #pypy
zmt00 has quit [Quit: Leaving]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 260 seconds]
oberstet has quit [Ping timeout: 268 seconds]
the_drow has quit [Ping timeout: 260 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 268 seconds]
asmeurer_ has joined #pypy
asmeurer_ has quit [Client Quit]
the_drow has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 250 seconds]
slackyy has quit [Ping timeout: 250 seconds]
<fijal> XXX I think errors are not handled correctly here
<fijal> date: Wed Nov 14 19:28:11 2007 +0000
<fijal> cfbolz: that's your commit from just over 10 years ago :-)
<cfbolz> of course
<cfbolz> where?
<fijal> in runicode.py
<fijal> I think so too tbh
<fijal> they're surely brittle and untested
<cfbolz> I claim innocence
<fijal> I believe you!
<fijal> cfbolz: do you want to pair on writing hypothesis tests?
<cfbolz> sorry, we're having visitors in like 20 min
<fijal> no worries :)
<fijal> maybe I can drag someone else
jamesaxl has quit [Read error: Connection reset by peer]
jamesaxl has joined #pypy
glyph has left #pypy ["Textual IRC Client: www.textualapp.com"]
<arigato> fijal: I'm draggable, if you want me to
<fijal> arigato: cool!
<fijal> arigato: and yes, I want to, should we try that atom teletype feature btw?
<fijal> or regular screen?
<fijal> let me finish the current incarnation
<arigato> I'm fine with Atom Teletype, but isn't that an OS/X only product?
<fijal> does not seem to be
<fijal> try?
<fijal> it used to be, I think
<fijal> I mean if you exceed ~5min trying to get it to run Im ok giving up :)
marr has joined #pypy
<arigato> sorry, busy with unexpected things, will be back in 10min
<fijal> no worries
<fijal> oh hey, python is adding module.__getattr__, AMAZING
antocuni has joined #pypy
<kenaan_> fijal unicode-utf8 3e5acb0a1e81 /pypy/: start working on more obscure codecs and completely remove hacks that go via UCS4 from unicodehelper. Now unic...
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 268 seconds]
<kenaan_> arigo unicode-utf8 0a3c81c3f67d /rpython/rlib/: Rename this directory to avoid name conflict with "rutf8.py"
<kenaan_> fijal unicode-utf8 e4a80363506c /pypy/interpreter/: in progress
<kenaan_> fijal unicode-utf8 78c8a9571b3d /rpython/rlib/: merge
demonimin has joined #pypy
jcea has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 248 seconds]
oberstet has joined #pypy
raynold has quit [Quit: Connection closed for inactivity]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 264 seconds]
lritter has joined #pypy
<kenaan_> fijal unicode-utf8 4668380f4c79 /pypy/: * Improve ascii/utf8 codecs and unicode escape * Raise instead of looping infinitely when errorhandler returns...
Rhy0lite has joined #pypy
<kenaan_> stian math-improvements e6c9af023bc5 /: Test and fix for int rbinop overflow to long, also add a deeper test for int_floordiv
<kenaan_> stian math-improvements 89a762f37f25 /pypy/objspace/std/test/test_intobject.py: Add test for overflow with regular binops too, now there should be test for all changes to intobject
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 248 seconds]
<kenaan_> stian math-improvements 9291ee92df89 /: Test for int_pow, test+fix for pow ValueError with third argument as 0
<kenaan_> stian math-improvements 6ba5b9334842 /rpython/rlib/rbigint.py: Typo in comment
jcea has quit [Quit: jcea]
<kenaan_> fijal unicode-utf8 d17afc06eedf /pypy/interpreter/: fixes
adamholmberg has joined #pypy
marky1991 has joined #pypy
marky1991 has quit [Ping timeout: 268 seconds]
marky1991 has joined #pypy
the_drow has quit [Ping timeout: 240 seconds]
marky1991 has quit [Ping timeout: 248 seconds]
yuyichao_ has quit [Ping timeout: 240 seconds]
<kenaan_> mattip default d632e9ca79ae /rpython/rlib/rvmprof/test/test_rvmprof.py: call register_code to profile functions, cleanup
mattip_ has joined #pypy
the_drow has joined #pypy
marky1991 has joined #pypy
<mattip_> the linux 32 build slave seems stuck on hg http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-32
mattip_ has left #pypy ["bye"]
yuyichao_ has joined #pypy
<kenaan_> fijal unicode-utf8 d18dd16d58c7 /rpython/rlib/rposix.py: silence a warning on OS X
<kenaan_> fijal default 6c9c3791d06a /rpython/rlib/rposix.py: silence a warning on OS X
<kenaan_> fijal default 21fd35c44d66 /rpython/: merge
<bbot2> Retry: http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-32/builds/2947 [ronan: force build, fix-broken-types]
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-32/builds/2948 [ronan: force build, fix-broken-types]
<bbot2> Retry: http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-32/builds/2948 [ronan: force build, fix-broken-types]
<arigato> seems not to work. I'll try to restart the buildmaster next
bbot2 has quit [Quit: buildmaster reconfigured: bot disconnecting]
bbot2 has joined #pypy
<kenaan_> fijal unicode-utf8 f074b4987d57 /pypy/: fixes until we get to formatting problems
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-32/builds/2949 [ronan: force build, fix-broken-types]
<arigato> mattip: fixed I think
<fijal> arigato: it seems maybe I'm getting somewhere
<arigato> good
<fijal> merge will be a nightmare, but other than that I'm almost done with removing all the references to runicode
<fijal> formatting is a big leftover
mattip_away has joined #pypy
<mattip_away> arigato: thanks. I will look at installing virtualenv as part of the tests a bit later
mattip_away has quit [Remote host closed the connection]
<fijal> space.newunicode(''.join(self.getdata()).decode('latin-1'))
<fijal> how many copies does that do....
<bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-app-level-linux-x86-32/builds/2949 [ronan: force build, fix-broken-types]
<Alex_Gaynor> Does `getdata` allocate stuff, or just return a value?
<Alex_Gaynor> And does it return `list of strings` or `list of chars`?
KronTMX has joined #pypy
<Alex_Gaynor> Assuming it's list of chars, `"".join()` gets you one alloc + copy. `decode('latin-1')` is another alloc + copy.
<fijal> getdata returns a value
<fijal> list of chars, I believe
<Alex_Gaynor> I think it's two allocs + copies then.
dcrosta has quit [Quit: Textual IRC Client: www.textualapp.com]
zmt00 has joined #pypy
the_drow has quit [Ping timeout: 240 seconds]
marky1991 has quit [Ping timeout: 264 seconds]
lritter has quit [Remote host closed the connection]
lazy1 has joined #pypy
<lazy1> I wonder if someone is willing to have a look at my code embedding pypy and see if there are obvious speed improvements that can be done.
<lazy1> The code is at https://github.com/tebeka/nuclio/blob/pypy/pkg/processor/runtime/pypy/nuclio_interface.py and currently I get about 70K req/sec
<fijal> lazy1: that's quite a few, no?
<lazy1> We aim for 100k+
the_drow has joined #pypy
<fijal> C.strdup(val.encode('utf-8'))
<fijal> what is that for?
<fijal> and keep the result alive till the end of the function
<fijal> yes, sure
<fijal> but what you should do is:
<lazy1> IIRC I read it on the cffi docs somewhere. Sharing memory between two runtimes is tricky
<fijal> that seems wrong
<lazy1> Go strings are encoded as utf-8 and from what I saw if I don't use strdup pypy might claim the memory while Go is still using it
<fijal> ffi.new("char*", val.encode("utf8")
<fijal> so x = ffi.new("char*", val.encode("utf8"))
<fijal> yield x
<fijal> and then x survives till the function dies
<lazy1> There are two cases. One is there I call C functions exported from Go and then I can do that
<lazy1> But in the result I can't since it's returned to Go
<fijal> also your dicts have 2 dict-of-indirection
<fijal> but you free the result right?
<lazy1> Yes, in get_result
<lazy1> For some reason freeing it from Go sometimes caused a segfault
marky1991 has joined #pypy
<lazy1> What do you mean by dict-of-indirection?
raynold has joined #pypy
marky1991 has quit [Remote host closed the connection]
<fijal> it's lower_dict -> another dict
<fijal> I have no idea TBH where the time is spent
<lazy1> Oh, that's since it' HTTP headers and they are case insensitive
marky1991 has joined #pypy
<fijal> yeah but you can do that more efficient on lookup
<fijal> I would run it under valgrind or perf (with pypy with debug symbols) and see where the time is spent before actually looking
<lazy1> Are there instructions online somewhere on how to do that?
<fijal> how to do what?
<fijal> a) you compile pypy yourself and b) you run valgrind --tool=callgrind pypy <something>
<fijal> and c) send me the output
<fijal> it's a regular c-level profiler, that's about 30x slower
<lazy1> OK
<fijal> you can try to use perf instead
<lazy1> What about cProfile?
<fijal> lazy1: what I mean is that by using a C-level profiler, you should be able to tell what's taking time. Go? Python? layer in between?
<fijal> cProfile is garbage
<fijal> vmprof is what we recommend (also has problems, but less so than cProfile, maybe)
<antocuni> note that nowadays nightly builds include debug symbols
<antocuni> so no need to recompile pypy
<fijal> ah cool
<lazy1> IIRC compiling pypy was a lot of coffee
the_drow has quit [Ping timeout: 248 seconds]
<fijal> takes ~40min, but indeed, use nightly
<lazy1> fijal: How is vmprof better than cProfile?
<fijal> it's a statistical profiler
<fijal> so comes with virtually no overhead
<fijal> the overhead in cProfile depends a lot from function to function, which means the output cannot be sanely trusted
<antocuni> also, vmprof knows about the JTI
<lazy1> OK, will give it a try
<fijal> antocuni: well, cProfile does too, to an extent
<fijal> lazy1: I wish vmprof was a bit better though, it has somewhat hard to read errors every now and again
<antocuni> I'd say that cProfile gives completely screwed data
<fijal> that too
oberstet has quit [Ping timeout: 248 seconds]
the_drow has joined #pypy
<fijal> arigato: do you know stuff about formatting by any chance?
<fijal> ah never mind found what I was looking for
lazy1 has quit [Ping timeout: 268 seconds]
the_drow has quit [Ping timeout: 240 seconds]
<bbot2> Started: http://buildbot.pypy.org/builders/rpython-linux-x86-32/builds/11 [ronan: force build, fix-broken-types]
jcea has joined #pypy
the_drow has joined #pypy
antocuni has quit [Ping timeout: 252 seconds]
<fijal> pom pom pom
<fijal> formatting looks like it should *almost* work as is
<arigato> :-)
<fijal> (but it's slicing utf8 strings so it's a bit awkward)
marky1991 has quit [Ping timeout: 248 seconds]
oberstet has joined #pypy
<LarstiQ> snow! snow!
<kenaan_> rlamy py3.5 feaba8e9bb0a /pypy/module/_io/: Bug-for-bug compatibility (and performance optimisation) in BufferedReader.readinto1()
tbodt has joined #pypy
<kenaan_> rlamy py3.5 a5d1206f11e4 /lib-python/3/test/test_io.py: Remove comment: this test is not supposed to fail any more
<bbot2> Failure: http://buildbot.pypy.org/builders/rpython-linux-x86-32/builds/11 [ronan: force build, fix-broken-types]
<kenaan_> rlamy py3.5 5c2561dd0c89 /lib-python/3/test/test_pydoc.py: Fix tests to match PyPy behaviour
lazy1 has joined #pypy
lazy1 has quit [Ping timeout: 240 seconds]
<kenaan_> rlamy py3.5 a84f8ceb8740 /lib-python/3/test/test_tracemalloc.py: Skip tracemalloc tests
<fijal> pom pom pom...
slackyy has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
asmeurer has joined #pypy
asmeurer has quit [Quit: asmeurer]
asmeurer__ has joined #pypy
asmeurer__ has quit [Quit: asmeurer__]
tbodt has joined #pypy
Rhy0lite has quit [Quit: Leaving]
marky1991 has joined #pypy
KronTMX has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
marky1991 has quit [Ping timeout: 248 seconds]
antocuni has joined #pypy
<fijal> test/test_dictmultiobject.py::AppTest_DictMultiObject::test_internal_delitem RPython traceback:
<fijal> Fatal RPython error: KeyError
<fijal> Abort trap: 6
<fijal> ...
<arigato> ouch
<arigato> pypy 5.9?
<fijal> no, older one
<fijal> but!
<arigato> I think there was a bug like that, yes
<fijal> test_mutate_while_extend loops infinitely
<fijal> (or at least takes forever)
<fijal> trying to run objspace tests on top of pypy
<arigato> try running on pypy 5.9?
<fijal> I have a more recent build even
<arigato> right, even better
<fijal> arigato: try running objspace/std/test on pypy-c? (without -A)
marky1991 has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
<arigato> I have 38ac9e4520a1, still running
<kenaan_> fijal unicode-utf8 b2f3bd9151c0 /: work on formatting
<fijal> ok
<arigato> no, works fine with this version
<arigato> what version are you using exactly (both of pypy-c and of the pypy source code)?
<fijal> 10e8aaa42286+
<fijal> not very useful :-/
<fijal> b2f3bd9151c0+ (unicode-utf8) tip
<fijal> that's the branch
<arigato> seems my version is slightly older
<fijal> arigato: I *think* the only thing left is charmap encoding
<fijal> (and billions of small things and a merge to py3k)
<arigato> cool, you're definitely getting into things that I don't even know they exist :-)
<arigato> note that the Fatal RPython KeyError you get is documented in the test itself
<arigato> but you only got that with the old PyPy, right? not with 10e8aaa42286+
<fijal> yes
<arigato> ok, and test_mutate_while_extend might just be a bit fragile
marky1991 has quit [Read error: Connection reset by peer]
marky1991 has joined #pypy
pybot has quit [Ping timeout: 252 seconds]
pybot has joined #pypy
jamesaxl has quit [Read error: Connection reset by peer]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
traverseda has quit [Ping timeout: 268 seconds]
traverseda has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
pilne has joined #pypy
asmeurer__ has joined #pypy
antocuni has quit [Ping timeout: 268 seconds]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]