cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<arigo>
seems that this one is half-broken (but still works)
jacob22 has quit [Read error: Connection reset by peer]
jacob22 has joined #pypy
lritter has joined #pypy
xcm has quit [Ping timeout: 246 seconds]
xcm has joined #pypy
dddddd has joined #pypy
jacob22 has quit [Read error: Connection reset by peer]
jacob22_ has joined #pypy
Rhy0lite has joined #pypy
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
DanielHolth has joined #pypy
<DanielHolth>
Hello
<arigo>
hi!
<arigo>
sorry about my initial replies to cffi issue 344, I was confused
<DanielHolth>
Still having fun with it
<DanielHolth>
Got (most of) an async ASGI server in uWSGI now
<arigo>
:-)
<arigo>
I'm rebuilding a current pypy3, then I'll try again to install everything you mention, it will take a while
<DanielHolth>
It's very funny, the uvicorn server for example has a wsgi mode, at the moment uvicorn wsgi runs at about the same speed as uWSGI ASGI and vice versa.
<DanielHolth>
If 'hello world' is your thing. Need to put up an n-body-problem benchmark and see how the web server handles that.
<DanielHolth>
And it did run perfectly in CPython 3.8 which was a pleasant surprise. Maybe the previous attempt hadn't 'make clean' or had a different cff or was on a Mac?
YannickJadoul has joined #pypy
DanielHolth has quit [Ping timeout: 245 seconds]
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
xcm has quit [Ping timeout: 246 seconds]
xcm has joined #pypy
DanielHolth has joined #pypy
epony has quit [Ping timeout: 258 seconds]
<arigo>
aaaah ok
<arigo>
DanielHolth: I understood what is going on now
xcm has quit [Remote host closed the connection]
<arigo>
virtualenv sets things up so that libpypy3-c.so is a symlink to the real one outside the virtualenv's directory
xcm has joined #pypy
<arigo>
we use the location of libpypy3-c.so to figure out if we're in a virtualenv, and if so, what path to use
<arigo>
in d1382b9d843b (back in 2016) I tried to use the symlink's own location before the location of where it points to
<arigo>
but I reverted it in 42797eb41b78 because dladdr() apparently did not give the symlink's own location
<arigo>
maybe it changed on Linux in the meantime, but nowadays it does give the symlink's location
<arigo>
I have no clue if it does on mac or elsewhere
xcm has quit [Remote host closed the connection]
xcm has joined #pypy
epony has joined #pypy
xcm has quit [Ping timeout: 258 seconds]
xcm has joined #pypy
DanielHolth has quit [Ping timeout: 245 seconds]
DanielHolth has joined #pypy
<DanielHolth>
arigo it links to the libpypy3-c.so from the virtualenv including using rpath
YannickJadoul has quit [Quit: Leaving]
<arigo>
yes
<arigo>
but dladdr() may or may not return the absolute path. it seems it did in 2016 but it no longer does, on linux---or else I messed something up either in 2016 or now
<arigo>
trying now to build a modified version of pypy3
<DanielHolth>
I looked through the pypy init for something I might be able to set to just tell it the env I wanted, in which case I'd prefer to link with the base pypy lib
<DanielHolth>
Didn't see anything in the init code.
<DanielHolth>
sys.executable may be unexpected in this use case too
BPL has quit [Ping timeout: 256 seconds]
BPL has joined #pypy
xcm has quit [Read error: Connection reset by peer]
xcm has joined #pypy
DanielHolth has quit [Ping timeout: 245 seconds]
<arigo>
OK, "fixed" but it has no effect and is bogus because I found out that the virtualenv directory contains no lib-python/lib_pypy directories
<arigo>
instead, why does it work to execute the "pypy3" executable from a virtualenv?
xcm has quit [Remote host closed the connection]
<arigo>
I think site.py is doing something, because running "pypy3 -S" gives us a broken pypy3 that doesn't look inside its own virtualenv
xcm has joined #pypy
<arigo>
ah, site.py reads pyvenv.cfg that sits at the top of the virtualenv directory
<arigo>
and it does locate this pyvenv.cfg from sys.executable only
<arigo>
which indeed is not set in pypy in case you're not running "pypy" but directly linking to libpypy3-c.so
<arigo>
in a similar situation, with CPython 3.6, running "./python -S" gets us another random exception but sets up sys.path correctly
<arigo>
of course, because virtualenv copies the "python3.6" executable in the venv/bin/, whereas for pypy it only makes a symlink
<ofir>
thanks, I read the docs several times but I'm not sure I understand what happens in the in-line vs. out-of-line mode.
<ofir>
One thing I understand is that in the out-of-line mode, the ffi builder script that I provide will *compile* a C code using the host compiler (e.g. gcc)
<ofir>
so it will probably auto-generate a C file and compile it, is that correct?
jvesely has quit [Remote host closed the connection]
<arigo>
there are a confusing number of different modes, but really only two main ones: either you make a build script with ffi.set_source("name", "C source") which, when run, makes and compiles a C code
<arigo>
or you use everything inline in "ABI" mode, without any C code, a bit like using ctypes
<ofir>
the "C source" is not the actual implementation of the library at hand right? it's just a rephrase of the header
<ofir>
at least in the examples they pass: #include "pi.h" as the C source
<ofir>
it looks confusing indeed / redundant, because pi_approx for example is already available via ffibuilder.cdef, so why bother?
jvesely has joined #pypy
xcm is now known as Guest87665
<arigo>
this "C source" is pasted in the middle of the generated C code
xcm has joined #pypy
Guest87665 has quit [Ping timeout: 264 seconds]
<arigo>
so it must be some valid C code that declares everything that is also written in the cdef()
<arigo>
and it can be as simple as '#include "mylib.h"'
<arigo>
but it can also contain more in some cases
<arigo>
like extra C functions that provide some extra wrappers, or even directly useful logic in C
lritter has quit [Quit: Leaving]
<arigo>
the reason for the apparent duplication of pi_approx (in the cdef and in the set_source via #include) is
<arigo>
that the set_source version typically #includes at least one real C header, which can be long and complicated and full of C subtleties
<arigo>
whereas in the cdef(), you put just what you want to use from Python---usually a subset of what is in that header
<ofir>
seems to nail it by applying the preprocessor (through Make) just before build_line.py is called
<ofir>
and from code maintenance perspective it seems cleaner IMHO
<arigo>
yes, applying the preprocessor works in some case, but in other cases it's cleaner to write just what you expect and use the "...;" explicit notation in cdef()
<ofir>
arigo: I could naively start with the preprocessor approach and see how far I get with that.
<ofir>
if I ever need to reiterate the prototypes / primitives from the headers, I will
<DanielHolth>
I'm reviewing it. I see that site.py has already done its thing before any of the embedded python code runs
<ofir>
what do you think?
<arigo>
ofir: sure. the "problem" of the preprocessor is also that different headers (typically large and messy ones) usually need different tweaks even before applying the preprocessor
<DanielHolth>
If I change sys.argv[0] will it set sys.executable
<arigo>
ofir: or after
<arigo>
ofir: but yes, that's doable
<ofir>
alrighty, thanks!
<arigo>
DanielHolth: no, sys.argv[0] and sys.executable are different
<arigo>
but you can set sys.executable to whatever, of course (it won't be used by site.py though, because it's imported early)
<DanielHolth>
I think it's probably correct but it's also easier to do the previous hack of chdir (venv dir) to change virtualenvs, than to link a new copy of the pypy library.
<ofir>
btw, what's the main gain with using cffi as opposed to the Python C API? is it the auto-generated boilerplate and creepy Py_* methods that are spared from the developer?
<arigo>
DanielHolth: sure
<ofir>
I know that also cffi is pypy compatible but I wondered if there are extra gains.
<DanielHolth>
it's fantastic. you just ingest a header and you have a cross-Python extension module.
<DanielHolth>
I've been a fan since before it existed :)
<arigo>
:-)
<ofir>
so I think I got the answer =D
<arigo>
there are a few cases where cffi is not suited, like e.g. if you want to design a new data structure or something that needs the C code to have large amount of references to further Python objects
<arigo>
I'd say it's not really the common case, which is more to write a python wrapper around a C library that does things on its own
<arigo>
another issue (on CPython, not really on PyPy) is that you write more Python code with cffi: e.g. you use cdef()/set_source() to access the raw interface but then you need some extra logic or Python classes around them to make it more suitable for python usage
<arigo>
that can show up as only a performance (or sometimes memory usage) issue, but usually not really
<arigo>
in general, writing more python code than C code is not the issue, it's the solution :-)
<DanielHolth>
hg pull https://foss.heptapod.net/pypy/pypy -r 3f039e602b2 is able to expand to the full revision number but says abort: unknown revision '3f039e602b294087916fcdab94c1e4fbd50adb10'!