cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | the secret reason for us trying to get PyPy users: to test the JIT well enough that we're somewhat confident about it
systoNK has joined #pypy
systoNK has quit [K-Lined]
marmoute has quit [Ping timeout: 244 seconds]
marmoute has joined #pypy
Kipras_ has quit [Read error: Connection reset by peer]
marmoute has quit [Ping timeout: 252 seconds]
marmoute has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
jcea has quit [Quit: jcea]
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
lritter has quit [Ping timeout: 252 seconds]
adamholmberg has quit [Ping timeout: 252 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 252 seconds]
mmethot_RZ has joined #pypy
mmethot_RZ has quit [Remote host closed the connection]
catern has quit [Excess Flood]
catern has joined #pypy
dotplusJu has joined #pypy
dotplusJu has quit [Remote host closed the connection]
dddddd has quit [Remote host closed the connection]
awkwardpenguin has joined #pypy
speeder39 has joined #pypy
Guest85165 has joined #pypy
Guest85165 has quit [Remote host closed the connection]
redj_ has joined #pypy
redj_ has left #pypy [#pypy]
redj has quit [Disconnected by services]
redj has joined #pypy
redj_ has joined #pypy
danieljabailey has quit [Ping timeout: 252 seconds]
redj has quit [Ping timeout: 252 seconds]
mattip has quit [Ping timeout: 252 seconds]
mattip has joined #pypy
danieljabailey has joined #pypy
redj_ has left #pypy [#pypy]
redj has joined #pypy
nopf has quit [Remote host closed the connection]
bashNinjaeF has joined #pypy
awkwardpenguin has quit [Read error: Connection reset by peer]
bashNinjaeF has quit [Remote host closed the connection]
awkwardpenguin has joined #pypy
awkwardp_ has joined #pypy
awkwardpenguin has quit [Ping timeout: 252 seconds]
awkwardp_ has quit [Remote host closed the connection]
awkwardpenguin has joined #pypy
oberstet has quit [Read error: Connection reset by peer]
oberstet has joined #pypy
McBadders0n has joined #pypy
McBadders0n has quit [Remote host closed the connection]
bauxitBQ has joined #pypy
bauxitBQ has quit [Remote host closed the connection]
smvv has joined #pypy
<smvv>
are there any upcoming PyPy sprints in Amsterdam/the Netherlands? I've sent various performance patches to Ionmonkey (Firefox's optimizing JS JIT) for the past three years (part of a job), and I would like to help with PyPy. I've some ideas how to improve the visualization of IR/JIT-generated code, and I was wondering how fast attribute access is implemented in the jit. In JS it is common to convert dicts (objects) to C structs
<smvv>
when there is a shape guard protecting the optimized code from becoming invalid. The shape guard will check if you have an object with the same properties when calling the function. Are there similar constructs in PyPy and what is the name of an 'object shape' in PyPy?
<mattip>
smvv: If I understand you correctly (shape == structure fields??) then rpython has "immutable fields" and python has "slots"
<mattip>
immutable fields are set at object construction and their contents never* change (except when ...)
<smvv>
example in JS: {a: 1} has a shape that is different from {b: 1} but also different from {b: 1, a: 1}
<smvv>
the shape identifies what type of properties *and in which order* there are stored in memory
<smvv>
the jit uses the shape to project optimized code from becoming invalid. If it knows that a property is always at lets say offset +8, it can generate fast code for loading that value. But it has to guard against objects that have a different shape to avoid getting incorrect property values
<mattip>
hmm. Well, maybe someone else can answer better. But for your first question - there are rumors of a sprint in Feb in Germany
<mattip>
my impression is that since our JIT traces at runtime, it guards against object changes, and the offsets are per-object-type
<mattip>
but maybe someone else can improve that answer
<smvv>
that makes sense but has 'per-object-type' a name? so I can look into the pypy codebase for those parts
speeder39 has quit [Quit: Connection closed for inactivity]
ppfYK has quit [Remote host closed the connection]
realitix has quit [Ping timeout: 268 seconds]
realitix has joined #pypy
antocuni_ has joined #pypy
<LarstiQ>
sprint wise, there is also the option of organizing one :)
marderhNK has joined #pypy
Zaab1t has joined #pypy
marderhNK has quit [Remote host closed the connection]
wleslie has joined #pypy
antocuni_ has quit [Ping timeout: 252 seconds]
wleslie has quit [Quit: ~~~ Crash in JIT!]
rubdos_ is now known as rubdos
manacitbs has joined #pypy
manacitbs has quit [K-Lined]
ronan has quit [Ping timeout: 252 seconds]
<smvv>
LarstiQ: that could be an option indeed. How many people do show up and what is necessary to organize it properly?
<mattip>
smvv: you could ask on the pypy-dev mailing list if there is interest
<mattip>
we would need a space with wifi and convenient sleeping arrangements
<mattip>
I doubt there would be interest in having two in the same month, all of the devs volunteer their pypy time
<mattip>
s/month/season/ even
ronan has joined #pypy
jcea has joined #pypy
Zaab1t has quit [Ping timeout: 252 seconds]
irossyd has joined #pypy
irossyd has quit [K-Lined]
dddddd has joined #pypy
Zaab1t has joined #pypy
jacob22__ has quit [Ping timeout: 245 seconds]
themsay has joined #pypy
<ESphynx>
Why is it that when setup.py builds a cffi module it suddenly gets a .abi3.so extension, instead of the .cpython-35m-x86_64-linux-gnu.so when built directly?
<ESphynx>
and is there a better solution than using: ext = '.so' if get_config_var('EXT_SUFFIX') is None else get_config_var('EXT_SUFFIX'); extra_link_args=['_myothercffimodule' + ext) for inter-modules CFFI dependencies? These direct .so dependencies breaks when setup.py changes the suffix...
Zaab1t has quit [Ping timeout: 252 seconds]
Zaab1t has joined #pypy
<LarstiQ>
ESphynx: what is "directly"?
Zaab1t has quit [Quit: bye bye friends]
Guest21160 has joined #pypy
Guest21160 has quit [Ping timeout: 252 seconds]
antocuni has joined #pypy
adamholmberg has joined #pypy
lritter has joined #pypy
Guest15442 has joined #pypy
Guest15442 has quit [Remote host closed the connection]
Rhy0lite has joined #pypy
xando has joined #pypy
OS-7153_bd has joined #pypy
OS-7153_bd has quit [Remote host closed the connection]
trfl has quit [Remote host closed the connection]
froztbyte has quit [Ping timeout: 252 seconds]
froztbyte has joined #pypy
froztbyte has joined #pypy
froztbyte has quit [Changing host]
demonimin_ has quit [Ping timeout: 252 seconds]
trfl has joined #pypy
demonimin has joined #pypy
demonimin has joined #pypy
arigato has joined #pypy
<arigato>
ESphynx: .abi3.so is the standard for extension modules that are flagged Py_LIMITED_API
<arigato>
means that the extension module will work with any version of python3.x (for x >= minimum)
<arigato>
about 'extra_link_args', that looks like a hack, but I'm not sure what you're trying to do
arigato has quit [Quit: Leaving]
marky1991 has joined #pypy
marky1991 has quit [Read error: Connection reset by peer]
marky1991 has joined #pypy
marky1991_2 has joined #pypy
marky1991 has quit [Ping timeout: 276 seconds]
marky1991_2 has quit [Read error: Connection reset by peer]
marky1991_2 has joined #pypy
marky1991_2 has quit [Read error: Connection reset by peer]
marky1991_2 has joined #pypy
Zaab1t has joined #pypy
KotHuA has joined #pypy
KotHuA has quit [Remote host closed the connection]
xando has quit [Quit: Connection closed for inactivity]
ceridwen has quit [Ping timeout: 240 seconds]
realitix has quit [Ping timeout: 252 seconds]
raynold_ has joined #pypy
Hasimir has quit [Remote host closed the connection]
TheAdversary has quit [Remote host closed the connection]
ceridwen has joined #pypy
ceridwen has quit [Changing host]
ceridwen has joined #pypy
shookees has joined #pypy
raynold_ has quit []
shookees has quit [Remote host closed the connection]
antocuni has quit [Ping timeout: 244 seconds]
daniel2qs has joined #pypy
daniel2qs has quit [Remote host closed the connection]
adamholmberg has quit [Remote host closed the connection]
R0b0t1 has joined #pypy
adamholmberg has joined #pypy
subix5eY has joined #pypy
subix5eY has quit [Remote host closed the connection]
R0b0t1 has quit [Ping timeout: 252 seconds]
adamholmberg has quit [Remote host closed the connection]
Kipras_ has joined #pypy
adamholmberg has joined #pypy
khaledez has joined #pypy
jamesaxl has joined #pypy
bugfixerqZ has joined #pypy
bugfixerqZ has quit [Remote host closed the connection]
<cfbolz>
smvv: if yuo are still around, pypy uses maps, which is exactly like js shapes/hidden classes to represent user-defined objects
<cfbolz>
smvv: also, the sprint is going to be in düsseldorf, which is very convenient to reach from amsterdam
Zaab1t has quit [Quit: bye bye friends]
jacob22__ has joined #pypy
Rhy0lite has quit [Quit: Leaving]
khaledez has quit [Ping timeout: 252 seconds]
khaledez has joined #pypy
khaledez has quit [Client Quit]
khaledez has joined #pypy
smvv has quit [Quit: Connection closed for inactivity]
jcea has quit [Ping timeout: 264 seconds]
marky1991 has joined #pypy
marky1991_2 has quit [Ping timeout: 244 seconds]
marky1991 has quit [Read error: Connection reset by peer]
marky1991 has joined #pypy
mpkeZ has joined #pypy
marky1991_2 has joined #pypy
<ESphynx>
LarstiQ, directly referred to adding the reference to the dependent CFFI module.. Since CFFI doesn't seem to provide any ways to do this without this 'hack'
marky1991 has quit [Ping timeout: 272 seconds]
mpkeZ has quit [Remote host closed the connection]
khaledez has quit [Ping timeout: 244 seconds]
marky1991_2 has quit [Read error: Connection reset by peer]
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 268 seconds]
adamholm_ has joined #pypy
oberstet has quit [Ping timeout: 244 seconds]
adamholm_ has quit [Remote host closed the connection]
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
<ESphynx>
setup.py is renaming the .so to .abi3.so or something...
<ESphynx>
after we expressed dependencies
adamholm_ has joined #pypy
adamholmberg has quit [Read error: Connection reset by peer]
adamholm_ has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 245 seconds]
<njs>
ESphynx: direct C-level dependencies between python extension modules is fraught with difficulties... the two strategies I've seen work well are putting the shared C code in a regular C library and then having the extension modules use it, or if the extension modules have to export a C-level API, doing this by stuffing the function pointers into a Python object that other modules can fetch through Python and then unpack into C pointers
<njs>
(the latter is how e.g. numpy and cython handle this)
<ESphynx>
njs, so we're using CFFI
<ESphynx>
and we have multi-level libraries... dependencies between the diffrent CFFI modules
<ESphynx>
and this is the dependency between the CFFI bindings themselves
<ESphynx>
so there's no way to do that.
<ESphynx>
it works fine except for these shared libraries suddenly being renamed in the middle of this setup.py!
<njs>
all I can tell you is that you're doing things differently from what most people do, which isn't *always* a bad idea but does mean that you're kind of on your own to make it work
<ESphynx>
njs, we've realized this by now. we've been trying to get this working for about a couple year s:P
<ESphynx>
but really this should be well supported by CFFI... is there a better place to get CFFI support?
rubdos has quit [Ping timeout: 260 seconds]
<njs>
not that I know of. Note that I'm not a cffi dev either, just a random kibbitzer.
<njs>
also I don't really know what you're trying to do so it's hard to give good advice :-)
<njs>
arigato or fijal or someone might be able to help moreif you can explain what you're trying to do in more detail. But .abi3.so is a standard thing, it's not just something the cffi devs invented to thwart you :-)
rubdos has joined #pypy
<ESphynx>
njs, the thing is when the library is built, it's e.g. _pyeC.cpython-35m-x86_64-linux-gnu.so
<ESphynx>
but when it makes its way into the egg or wheel that is when it gets the .abi3.so
<ESphynx>
"When the two build scripts are turned into generated files, say _ffi.so and _other_ffi.so, then importing _ffi.so will internally cause _other_ffi.so to be imported"
<ESphynx>
for some reason, this was not the case :S
lirielZU has joined #pypy
<ESphynx>
njs, ok so I tried removing that extra_link, and the issue is that our different CFFI module each links to a C library, and the CFFImodule2's C library, depends not only on CFFImodule1 but also on the CFFImodule1's C library.
<ESphynx>
(But really, there is only one shared library for these modules that includes that C 'library', so I'm still not quite clear
<ESphynx>
Ah, I think the problem is that: " then importing _ffi.so will internally cause _other_ffi.so to be imported." --> this doesn't happen right away so the linker complains before Python actually imports the dependency? because there is a C level dependency?
lirielZU has quit [Ping timeout: 252 seconds]
jamesaxl has quit [Read error: Connection reset by peer]