<cfbolz>
Maybe simply propose to add a sentence like 'if at all possible you should use one of those tools instead of writing C extensions by hand'
<fijal>
I would phrase it differently
<fijal>
like "Creating Python extensions using C API is error prone, cumbersome and only works on CPython. We recommend the following third party tools <with more explanation what they do and who uses them>"
<fijal>
I'm happy to try to write it at some stage
<cfbolz>
fijal: please do
<fijal>
cfbolz: also, noone uses SWIG any more. and I have no idea how do you call C with numba? even numeric people don't
<fijal>
8:21 PM <fijal> Apparently cffi stopped working on arm
<fijal>
seems like a broken installation though
Zaab1t has quit [Ping timeout: 272 seconds]
Zaabtop is now known as Zaab1t
<arigato>
yes, Not My Problem if the system's ffi.h does not manage to balance its C preprocessor directives... though that looks rather strange
<reaperhulk>
arigato: python3x.dll will be in windows\system32 on 3.4 and below, not in the virtualenv itself. I'm not sure if virtualenvs allow loading from system32 or not? I filed https://github.com/pypa/virtualenv/issues/1314 against virtualenv for python3.dll at least
<reaperhulk>
I guess it linked against python3x.dll previously so it must work
<arigato>
yes, I doubt that on Windows we could have a situation where \windows\system32 is not on the dll's path
<arigato>
of course, it turns out it's hard to define Py_LIMITED_API in a header on Windows depending on the python version
<arigato>
because we must define Py_LIMITED_API before #including python.h
<arigato>
I think there is no choice but reverting to never defining Py_LIMITED_API on Windows
<arigato>
...no, the problem is really that we can't even #include pyconfig.h on Windows if we plan to define Py_LIMITED_API afterwards
<arigato>
so, we have no way to know the python version
Zaab1t has quit [Quit: bye bye friends]
<kenaan>
arigo cffi/cffi 739f2c59674d /cffi/: On Windows, we still can't use Py_LIMITED_API for bad reasons.
<fijal>
arigato: do you feel like explaining one detail of a regalloc?
<arigato>
that was a hack, it would be better to have a clean solution but it's unclear to me how
<fijal>
when you walk the operations, you can combine the two?
<fijal>
so instead of calling emit_op_int_lt and then emit_op_guard_true
<fijal>
you can call emit_op_guard_true(prev_op)
<fijal>
?
<fijal>
seems quite a bit cleaner to me
<fijal>
I *think* that's how it's done in optimizeopt? or maybe the overflow thing?
<arigato>
int_cmp() is itself a helper used by int_eq/int_ne/int_gt/etc., and there are also other cases than int_cmp. would you like to enumerate them all explicitly in emit_op_guard_true()?
dddddd has joined #pypy
<fijal>
no, but we can call the right thing in emit_op_guard_true
<fijal>
like dispatch_op(prev_op)
<arigato>
what do you do with a int_gt not followed by a guard?
<arigato>
right now it works "automatically"
<fijal>
I want to have something like that
<fijal>
emit_guard_op(prev_op_possibly_none)
<fijal>
which starts with if prev_op_possibly_none: dispatch_op(prev_op)
<fijal>
which will generate the right code, but will avoid storing strange state
<fijal>
it works "automatically" by a hack and strange state
<arigato>
yes, my point is that the "automatically" is important because there are many rare cases here
<arigato>
I don't want to have to write two versions of the code for int_gt, one of which is rarely used
<fijal>
yes, agreed
<fijal>
so well
<fijal>
do you want the aarch64 to retain the current hack or do soemthing cleaner?
<fijal>
let's ignore for now what a cleaner version is, my point is consistenc
<arigato>
I'd say, go ahead if you can find something cleaner that avoids the problem I just described
<arigato>
or even not
<fijal>
ok, then let me try to describe what I have in mind again
<fijal>
no, I claim I just described it, can you ask a specific question what part is not clear?
<arigato>
at this point consistency is not essential and it's probably a good opportunity to experiment
<fijal>
the main difference is that you call the emit_int_lt or whatever from emit_guard_op and not from the dispatch loop, but possibly through a dispatch loop
<fijal>
so you have all the state there and not stashed somewhere
<fijal>
so you don't have hacks like flush_cc
<fijal>
(and the result never gets allocated on the regalloc)
<arigato>
in your solution, the emit_int_lt() would still be called from the dispatch loop if there is no guard_true afterwards, so it must emit a pair CMP/SETL and return a register
<fijal>
yes
<arigato>
so it looks like you need emit_int_lt() calling something more internal that just emits the CMP,
<fijal>
right, so you have a rarely-called version which you are trying to avoid?
<arigato>
and then use that internal thing from emit_guard_true()
<fijal>
yes, that seems saner
<arigato>
yes, agreed
<fijal>
so you either call emit_guard_op or you call emit_comparison_op
<fijal>
and each of them would call the whatever_int_lt?
<fijal>
but one will emit guard code, one will emit the SETL
<fijal>
makes sense?
<arigato>
yes
<fijal>
cool!
<arigato>
well
<fijal>
yes?
<arigato>
I may still be missing something, but I'm not sure
<mattip>
it used to be that os.environ[‘PATH’] was the wndows eqiuvalent to os.environ[‘LD_LIBRARY_PATH’] but it seems both Anaconda and windows want to change that for something more complicated
<kenaan>
stevie_92 cpyext-gc-cycle 09b2440acb51 /: Added support for rawrefcount finalizers in incminimark Added call to tp_finalize in cpyext, if the gc ...
<fijal>
cfbolz: I can write an explanation for a pull request too
<fijal>
notably I think it's worth saying why SWIG or ctypes are a bad idea
<fijal>
anyway, yeah, I like your constructive attitude even though I'm skeptical :)
<fijal>
so don't get discouraged
<arigato>
fijal: do you know how (if at all) to use your arm64 host to test cffi on arm32?
<fijal>
arigato: yes, by creating either a 32bit docker image (probably the best) or 32bit chroot
<kenaan>
cfbolz default 37a9de9e88b0 /pypy/objspace/std/dictmultiobject.py: this should make unicode_eq actually use a direct_call!
<kenaan>
arigo default 1d688af446c2 /pypy/: Refactor PySequence_FAST_XXX to allow notably for specialized tuples (plus it's a clean-up, I think)
<kenaan>
arigo default 7fad2f4ca232 /pypy/module/cpyext/test/test_sequence.py: Fix test
<kenaan>
arigo default 63c291824396 /pypy/module/cpyext/test/test_typeobject.py: Fix test
<fijal>
arigato: so actually we can kinda do better
<fijal>
I wonder if I care
<fijal>
but you have a compare-and-jump instruction
<fijal>
but GCC somehow does not emit it so maybe does not matter
<arigato>
uh, gcc doesn't emit it?
<arigato>
(OK, I compiled without problem on a slow qemu Raspbian I have locally)
<arigato>
fwiw, all tests pass on arm64
<fijal>
maybe it's only CBZ and not other variants?
<arigato>
there's only CBZ and TBZ, yes
<fijal>
anyway, more coffee I think before I progress further
<arigato>
yes, CBZ is used by gcc
<fijal>
arigato: are you ok with using less metaprogramming than usual btw?
<fijal>
than arm or x86
<kenaan>
fijal arm64 5cb0444d17a2 /rpython/jit/backend/aarch64/: add CMP_rr and push forward to next test
<arigato>
sure
beystef has quit [Ping timeout: 258 seconds]
RemoteFo1 has left #pypy [#pypy]
RemoteFox has joined #pypy
beystef has joined #pypy
<arigato>
Alex_Gaynor: please ping me if you're ready to trigger the cffi 1.12.1 release
<arigato>
only changes are a skipped test for some architectures and removing Py_LIMITED_API on windows again
<kenaan>
arigo cffi/cffi[release-1.12] 914388b2b8b8 /: bump version number to 1.12.1
beystef has quit [Ping timeout: 255 seconds]
<kenaan>
arigo default f7ca84265e13 /pypy/module/_cffi_backend/test/_backend_test_c.py: update to cffi/914388b2b8b8
<kenaan>
arigo default e1ca8268d72b /: update to cffi/914388b2b8b8
<petaflot>
hello! I am trying to do bit-flipping in very large integers, _without_ making a copy of that integer (in place). can pypy do that?
<mattip>
petaflot, what is wrong with x ^= 0xffffffff
<petaflot>
mattip: it makes a copy of the integer. when the int is small it's ok, when it's over 100MB in size and you do the operation a few million times, it _is_ a problem
<tos9>
petaflot: like I mentioned in #python, I don't know that it does any copying in PyPy :), it's certainly a possible optimization for PyPy to do in theory, so you could have checked :) but asking here is another way
<petaflot>
tos9: I googled a little and found nothing relevant. so I asked..
<arigato>
petaflot: you're better off using some library dedicated to your use case
<petaflot>
anyhow, since I can get it to work with a bytearray it's not too important
<arigato>
(with pypy it should be written in normal Python; with cpython it should be written in C)
<arigato>
right, operations on a bytearray in plain Python are quite fast with pypy
<cfbolz>
fijal, arigato: eg shouldn't we do this at least:
<cfbolz>
(right now even iteration forces the index storage to be computed)
<arigato>
cfbolz: obviously. but note that cpython2 doesn't have unicode.__iter__
<arigato>
cpython3 does
<cfbolz>
do we care?
<arigato>
I'm pretty sure there is an old bug about it in the tracker
<cfbolz>
arigato: so should I add it to py3 only? or it's one in the list of "doing things a tiny bit differently"?
<arigato>
can't find the bug in the tracker, but then recently I'm suspecting that bitbucket's search is reporting "no item found" quite often
<cfbolz>
yeah :-(
<cfbolz>
ok, I think I'll commit it and wait for complaints
<arigato>
see std/objspace.py: _uni_uses_no_iter()
<arigato>
this change (at least in pypy2) would break that function
<cfbolz>
arigato: that's for subclasses though
<cfbolz>
but yes, I see your point
<arigato>
cfbolz: (1) no? it's called with "isinstance(w_obj, W_UnicodeObject)"; and (2) it breaks the function because that function will always return False
<cfbolz>
arigato: yes, it would break, I agree
<cfbolz>
just saying that it's not really a problem if unicode now has an iter, as long as I fix this function to check for precisely this iter
<arigato>
ah, I see the problem: if you define an app-level subclass of unicode and override __getitem__(), with your change it won't be called any more when iterating
<arigato>
I'm pretty sure we had this problem
<cfbolz>
Right :-(
<cfbolz>
I'm sure I can cheat some way
<arigato>
yes, a simple hack would be to change descroperation, method iter()
<cfbolz>
arigato: right
<cfbolz>
Yes not a bad plan
<arigato>
I thought we had tests for this kind of cases, but I don't find any
<cfbolz>
arigato: will try to add one, while I'm at it
<arigato>
cool
<arigato>
thanks
<fijal>
arigato, cfbolz: calling get item while iterating is at best undefined
<fijal>
I think cpython aims to not call special methods when doing stuff like that
<fijal>
But if you subclass builtin type and override special methods, the results are random
<cfbolz>
fijal: doesn't stop people from relying on it
petaflot has left #pypy [#pypy]
<fijal>
cfbolz: yes, but it also changes between python versions
<fijal>
And it's often different between pypy and cpy
<fijal>
Pypy typically calls them, cpy doesn't
moei has quit [Quit: Leaving...]
xcm is now known as Guest89095
Guest89095 has quit [Killed (hitchcock.freenode.net (Nickname regained by services))]