<nimaje>
sms: a header file in C is normally only a bunch of extern declarations, cffi probably doesn't generate a header, so if you really want a header you write it yourself, but you don't need a header only the declarations
<sms>
nimaje: So in the basic embedding example, how do you call the function do_stuff in C?
<LarstiQ>
plus the cffi embedding approach encourages a very simple api "You can use CFFI to generate C code which exports the API of your choice to any C application that wants to link with this C code. "
<LarstiQ>
sms: the Usage section has an example?
<LarstiQ>
sms: the block that says plugin.h
<LarstiQ>
typedef struct { int x, y; } point_t; extern int do_stuff(point_t *);
<LarstiQ>
for the declartions, and then just call it appropriately
<sms>
Why is it declared with point_t* as an argument?
jamesaxl has quit [Read error: Connection reset by peer]
amaury has quit [Ping timeout: 240 seconds]
jamesaxl has joined #pypy
marr has joined #pypy
vkirilichev has joined #pypy
amaury has joined #pypy
vkirilichev has quit [Ping timeout: 260 seconds]
<LarstiQ>
sms: you mean why -> dereference instead of . in C terms?
<cfbolz>
njs: do you happen to know where the CPython tests for this feature are hiding?
<LarstiQ>
sms: the More reading section points to http://cffi.readthedocs.io/en/latest/using.html#working which mentions "The ctype is usually some constant string describing the C type. It must be a pointer or array type. If it is a pointer, e.g. "int *" or struct foo *, then it allocates the memory for one int or struct foo"
vkirilichev has joined #pypy
<cfbolz>
sms: would you be up to filing a bug so that Armin knows he should add an example?
inhahe_ has quit [Read error: Connection reset by peer]
inhahe_ has joined #pypy
asmeurer_ has joined #pypy
Tiberium has quit [Ping timeout: 260 seconds]
Tiberium has joined #pypy
asmeurer_ has quit [Ping timeout: 246 seconds]
arigato has joined #pypy
antocuni has joined #pypy
Tiberium has quit [Remote host closed the connection]
<fijal>
arigato: hi
<antocuni>
arigato, fijal: is there a 32bit chroot on bencher4?
<arigato>
hi
<arigato>
antocuni: I think not, there is a 64bit Ubuntu chroot I think
<antocuni>
arigato: is it possible to create one? Or, to give me sudo powers so I can do it? :)
<antocuni>
I don't know if you read the logs, I am debugging an RPython AssertionError which shows up only on 32bit, and bencher4 is incredibly fast at translating things :)
vkirilichev has quit [Remote host closed the connection]
<antocuni>
yes, cfbolz suggests that it's because I introduced llop.gc_store_indexed, which is not handled e.g. by writeanalyze.py
<arigato>
but where is that invoked in this small example?
<antocuni>
but it's very weird because my example is very small and doesn't call the new llop at all
<antocuni>
yes exactly
arigo has quit [Quit: Leaving]
<antocuni>
I don't see how the new llop can introduce this bug. To double-check, I translated again at 892ab4160ec6 (just before I introduced llop.gc_store_indexed) and the bug disappeared
<arigato>
I see
<antocuni>
so now I am fixing writeanalyze.py hoping that it will fix the bug, but I don't really understand what's going on
<arigato>
but 892ab4160ec6 can't add the bug, it's only about tests?
<arigato>
you should try to bisect to figure out which exact checkin introduced the problem
<antocuni>
arigato: sure, what I mean is that the bug was introduced *after* 892ab4160ec6
<antocuni>
bisecting is a bit hard because I discovered that 32 bit translation was broken a bit everywhere during the history
<antocuni>
so I have to apply patches from the future to translate
<arigato>
still, no other idea
<antocuni>
(that's why I was craving for a faster machine, since my translations keep failing :))
<antocuni>
ok, let's see what happens with the fixed writeanalyze
<kenaan>
antocuni faster-rstruct-2 85d3ab6fe80b /rpython/translator/backendopt/: fix writeanalyze.py to take into account llop.gc_store_indexed
<antocuni>
arigato: could you please review this ^^^?
<arigato>
I must say I'm skeptical
<arigato>
why would this fix a failure that only shows up on 32bit?
<antocuni>
I am skeptical as well :(
<antocuni>
as I said, I don't really understand what's going on
<antocuni>
anyway, I'm off for now, thanks for the help
<antocuni>
I started a couple of translations, lets see what happens :)
<antocuni>
arigato: fwiw, about my bug: a735e006ad8a seems to work, 3c31e7d36cc9 exhibits the behavior
<antocuni>
now I am translating other three random revisions in the middle of them, although none of this seems related to the bug
<antocuni>
I am wondering whether the bug was already there and it only manifest itself randomly, depending on some detail which might vary between translations
<antocuni>
like the order of jitcodes or things like that. This would at least help to explain why I get it only on 32 bit
<arigato>
all obscure
girish946 has quit [Quit: Leaving]
vkirilichev has joined #pypy
<kenaan>
arigo cpyext-obj-stealing fb0a61fd753d /pypy/: Tweaks and simplifications A few "pointless" operations have been added which I removed again; I'm uns...
t4nk422 has quit [Quit: Page closed]
asmeurer has joined #pypy
vkirilichev has quit [Remote host closed the connection]
asmeurer has quit [Ping timeout: 240 seconds]
arigato has quit [Quit: Leaving]
oberstet has quit [Ping timeout: 246 seconds]
asmeurer has joined #pypy
asmeurer has quit [Ping timeout: 260 seconds]
jamesaxl has quit [Read error: Connection reset by peer]
jamesaxl has joined #pypy
vkirilichev has joined #pypy
adeohluwa has joined #pypy
Tiberium has quit [Remote host closed the connection]
<antocuni>
so, 77c4134d96d9 is good, 3c31e7d36cc9 is bad; testing 196fb3e1e9b3 now, although it still does not make any sense
nimaje1 has joined #pypy
nimaje1 is now known as nimaje
nimaje has quit [Killed (wolfe.freenode.net (Nickname regained by services))]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
jamesaxl has quit [Ping timeout: 240 seconds]
oberstet has quit [Quit: Leaving]
arigato has joined #pypy
mattip has joined #pypy
<mattip>
arigato: thanks for looking at cpyext-obj-stealing, the goal indeed was to make pypy pass this test, especially the "if (tmp != i2) {fail}" part
asmeurer has joined #pypy
asmeurer has quit [Ping timeout: 240 seconds]
vkirilichev has quit [Ping timeout: 258 seconds]
jcea1 has joined #pypy
jcea has quit [Ping timeout: 240 seconds]
jcea1 is now known as jcea
jamesaxl has joined #pypy
asmeurer has joined #pypy
asmeurer has quit [Ping timeout: 240 seconds]
<ronan>
is there actually any point to strbufobject?
<ronan>
I'm tempted to kill it
asmeurer has joined #pypy
* mattip
looking at hg blame for strbufobject to try to figure out why it exists ...
<arigato>
mattip: so, I'm +1 to merge both your cpyext branches
<mattip>
ok, thanks, and thanks for the review/rewrite
<arigato>
ronan: it's an attempt that might be merged to default at some (long) point
<arigato>
this or something similar
<arigato>
more precisely, it attempts to fix the "string += string" complexity difference in some cases, but so far it fails to fix it in all the CPython cases
<ronan>
arigato: well, it seems to have been there since 2010, and never did anything except cost maintenance time
<ronan>
I don't see how it can possibly help with the string += string case
asmeurer has quit [Ping timeout: 255 seconds]
<ronan>
(unless we modify semantics or find a way to mimic the 'refcount == 1' hack)
tilgovi has joined #pypy
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
arigato has quit [Read error: Connection reset by peer]
tbodt has joined #pypy
arigato has joined #pypy
<arigato>
ronan: it works in many cases where CPython does, and I think the remaining cases are documented somewhere but I don't remember them
<ronan>
what are the cases where it works?
<arigato>
the naïve "s += some_string" in a loop
<arigato>
for example
<ronan>
no, that forces the string for every concatenation after the first one
<arigato>
if you're right then strbufobject.py has no point, but I think you're wrong
vkirilichev has joined #pypy
<arigato>
if you show me an example or convince me of strbuf forcing all intermediate strings in the simple example, then yes, it can be killed
<arigato>
the point is that when I wrote it, it didn't
Tiberium has quit [Remote host closed the connection]
<arigato>
you have to do nothing with the intermedate strings
<arigato>
just call "+="
<arigato>
or "s = s + some_string"
<arigato>
the old 's' stays around until the next GC, of course, and the new 's' is longer, but they share the same StringBuffer object
<arigato>
...StringBuilder object
<ronan>
ah, you may be right
<arigato>
anyway, it's probably killable anyway
<arigato>
there's another alternate implementation of strings that we should play with
<arigato>
something having a raw buffer
<arigato>
maybe we could come up with such an implementation that also supports concatenation
<ronan>
well, the case 'for x in xs: s += x' is the one people keep complaining about
<arigato>
or, maybe we keep strbufobject but need to tweak heuristics so that it's invoked less eagerly
<arigato>
yes
<arigato>
but well, the point is that I'm not completely sure they don't have code that is more complicated
<ronan>
if it works for that case, then why don't we enable it?
<arigato>
e.g. for x in xs: if s.endswith('.'): rare_case; s+= x
<arigato>
the answer is complicated, but there are three factors
<arigato>
(1) it doesn't work in all cases where CPython's version works
<arigato>
(2) this kind of "just checking something on 's'" will make it stop working
<arigato>
(3) the impact on JITted code is not completely clear, e.g. how many extra bridges it makes
vkirilichev has quit [Ping timeout: 240 seconds]
<arigato>
so, I think the status has always been "needs more work", to study and fix (3), and to make more of (2) cases still working
<arigato>
mattip: if my latest change to cpyext-obj-stealing broke pandas again, sorry about that
<arigato>
I didn't test
<mattip>
as long as that test passes, all should be good, AFAICT
<arigato>
yes, all our tests pass
<arigato>
but I had to tweak them a bit
<arigato>
or at least, I *did* tweak them because they were relying on obscure details
<arigato>
e.g. storing a string in a list, and then getting the item, might get you a different object because of the list-of-strings strategy
<arigato>
and then the reference count of that different object has no relation with the original object's
<arigato>
I think that's why you added a hack to force the CPyListStrategy in PyList_New()
<mattip>
that was the problem I was trying to fix, and it seems you even tightened this test, maybe more tests are needed
<arigato>
yes, but I removed that change to PyList_New(), which I *think* is fine and all our tests pass
<mattip>
+1
<arigato>
but it's possible that it makes some refcounts behave apparently strangely, again
<arigato>
notably if you store strings or ints or floats, which are not really stored
<arigato>
...at least with PyList_Append()
<arigato>
which doesn't force the CPyListStrategy
<arigato>
with PyList_SET_ITEM, then it would force the CPyListStrategy, so everything should work as expected
<mattip>
PyList_Append is very rarely used
<arigato>
ok
<mattip>
uhh, maybe I am wrong, cython tends to use it :(
<arigato>
maybe PyList_New(n) should really make a CPyListStrategy, but then it would be better if it did so immediately, instead of making a regular list of Nones and converting it...
<ronan>
arigato: so, should we keep strbufobject? It does work, but we're never going to enable it as is
<arigato>
ronan: could we move it to a branch somehow? e.g. create the branch "strbufobject" and then remove it from default
<ronan>
arigato: we could remove it and backout that commit on a branch
<arigato>
right
asmeurer has joined #pypy
<mattip>
arigato: I have moved back to numpy, and am trying to iron out the last ~12 test failures
<arigato>
cool
<mattip>
in hopes that if we can get a green build, they will add us to the CI
<arigato>
+1
<arigato>
for trying it :-)
asmeurer has quit [Ping timeout: 240 seconds]
* ronan
creating branch strbufobject
<mattip>
other than strbufobject, there are some random test failures on default
<ronan>
mattip: we should probably just kill the flaky tests
<mattip>
some of them seem more that just flaky, like the swapped ctypes one