cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<fijal>
I find the request a bit strange, you can install C compiler, why can't you install rust compiler?
<tumbleweed>
pip upstream's response to this is that wheels are the solution to everything
<tumbleweed>
(cryptography needing rust, I mean)
<fijal>
not a massive fan of that response, I must say
<cfbolz>
fijal: there's been a massive flame war, seems some people really can't install a rust compiler because the platform is not supported
<cfbolz>
(and they've been entitled assholes about that problem)
<tumbleweed>
and others have been entitled assholes about not having that problem :P
<cfbolz>
maybe so
<tumbleweed>
given rust is still fairly new, there's an extra issue here, that you need a suitably recent rust
<tumbleweed>
which rulses out distro-packaged rust on stable releases for many users
<cfbolz>
right, that part is definitely annoying
<tumbleweed>
so, yeah, hoops for people to jump through
<cfbolz>
tumbleweed: sorry, I know that as a maintainer that really sucks. I am just a bit unhappy with how unconstructive the discussion was basically from comment 1
<tumbleweed>
sure, I was being glib there
<tumbleweed>
but the related threads have also been heavy in popcorn
<cfbolz>
which is not a good thing for the involved, only for the popcorn lovers ;-)
forgottenone has quit [Remote host closed the connection]
<fijal>
cfbolz: what plaforms?
<fijal>
I've seen the flame war
<fijal>
the suitably recent rust is quite problematic, what was the reasoning behind that choice?
ronan has quit [Remote host closed the connection]
<fijal>
cfbolz: it's going to be a serious problem for pypy, I would imagin
<fijal>
that you need a new version of rust compiler
rohan has joined #pypy
rohan has quit [Connection closed]
<cfbolz>
fijal: lol, PyPy is really the last project that can complain about a complicated build process
<fijal>
I'm not saying we should complain really loudly
<fijal>
I'm saying that inability to install half of pypi without getting some version of rust that's newer that's what your distro distributes is going to be a serious adoption problem for us
<fijal>
I don't think I care, to be fair
<fijal>
paying pypy users generally have horrendous gigantic setups anyway and they can do something like that
<cfbolz>
fijal: for 'casual' users I kind of think the answer is 'conda'
<fijal>
ok, I have never really used conda
<cfbolz>
They do a very reasonable job at building binary PyPy things
<fijal>
or docker
<fijal>
maybe we should advertise docker more
<fijal>
either way, we should probably put a story on pypy.org how you are going to get around it
<fijal>
(be it conda or docker or some other solution)
<fijal>
but "pip install cryptography does not work at all unless you compile rust compiler from source" is probably not a good answer we can give to people
<fijal>
nothing stops us I guess.... but it's a bad answer
<fijal>
maybe we should provide cryptography wheels for every release, that would likely solve the problem
<LarstiQ>
which comes back to the stable abi request by reaperhulk, if that's feasible
HepaR has joined #pypy
HepaR has quit [Remote host closed the connection]
CrunchyChewieXv has joined #pypy
ChasedSpade has joined #pypy
ChasedSpade has quit [Remote host closed the connection]
CrunchyChewieXv has quit [Remote host closed the connection]
strugeeIG has joined #pypy
strugeeIG has quit [Remote host closed the connection]
_linker_ has joined #pypy
ninseiNm has joined #pypy
siso_dc has joined #pypy
ninseiNm has quit [Remote host closed the connection]
siso_dc has quit [Remote host closed the connection]
PrettyKittiexc has joined #pypy
PrettyKittiexc has quit [Read error: Connection reset by peer]
SKYWARN has joined #pypy
SKYWARN has quit [Remote host closed the connection]
<mattip>
what if we propose a pypy-stable-abi3 that would not be backward compatible, but would work on any pypy7.3.4+ for python3.7+
<mattip>
anyhow current pypy's cannot import an abi3 c-extension module, so it's not like we would be breaking current users
<mattip>
it just means we would commit to freezing our current macro/function c-api interfaces, with the PyPy_* names
<antocuni>
mattip: the problem is that our ABI tends to break as soon as you touch anything in cpyext
<mattip>
we extend it with new functions, do we break existing ones?
<antocuni>
sometimes we turn functions into macros and/or viceversa?
<antocuni>
well, given the very low cpyext activity these days, it might be a good idea to stabilize the abi
<antocuni>
better than not having cryptography, for sure
<mattip>
we would have to commit to not changing the macros anymore
<mattip>
or at least leave some #define in place when going from macro -> function
<antocuni>
yes, and at worst the day we really need to break the ABI, we will use a newer tag. It would break compatibility with existing wheels, but at least it will happen very rarely, hopefully
<cfbolz>
can we test for this?
<cfbolz>
I mean, can we write a test that would catch the problem
<antocuni>
I don't know if it's useful for us, but it looks promising
muke has joined #pypy
<muke>
I think I've found a limitation with the rffi where it can't see statically defined functions from libraries, even though regular C programs can - is this the case or am I possibly just getting something wrong?
<antocuni>
muke: do you mean functions like "static void foo(void);"?
<muke>
yea
<antocuni>
those functions can be called only from the same .c file where they are defined
<antocuni>
even in C
<muke>
the llvm-c library has a function that's defined with the static keyword but I can call it from a C program just fine
<muke>
but the rffi can't
<antocuni>
I bet it's defined in a .h file?
<muke>
yea
<muke>
i'm including the .h file in the rffi compilation info though of course
<antocuni>
show me the code please
<antocuni>
and the error
<muke>
sure, one moment
<antocuni>
I think that the easiest way to solve is to wrap this function in a non-static one. Look at rpython/rlib/clibffi.py for an example
<muke>
I had actually started this already and was facing a problem with that too, but wanted to confirm I was trying to solve the right problem before bringing that up
<antocuni>
in that file, it defines a function pypy_get_libc_handle by using separate_module_sources + post_include_bits
<muke>
but that example may be what I need for that anyway actually
<antocuni>
so you can probably define your own function which calls the llvm one in the same way
<muke>
here it is with the compilation info defined
<antocuni>
keep in mind that rffi does a lot of dark magic to do 2 different things:
<antocuni>
1. make it possible to call the C functions during untranslated tests. So, in this mode you are calling C function from python through ctypes
<antocuni>
2. after translation, the generated C code will contain calls to the C function directly
<antocuni>
the problem here it's (1). Since the function is static, rffi/ctypes can't find it inside the llvm's .so
<antocuni>
so, my suggestion above should fix it
<muke>
right, I see
<muke>
thanks for the info - I wrote a wrapper but I was struggling to get the shared library detected by the rffi, I'll look into the example though
<muke>
also you said not to use llexternal directly, what should I be using instead?
<antocuni>
no sorry, llexternal is fine
<muke>
ok great, thanks again
<antocuni>
what I meant is that you need an ExternalCompilationInfo, which you have (but didn't paste 😅)
<muke>
Yea, it'd be easier if I had push access so I could just link the branch
<antocuni>
we can give you push access of course
<muke>
I requested a while ago
<antocuni>
ah
* antocuni
looks
<antocuni>
muke: access granted
<muke>
fantastic, thanks
<antocuni>
you're welcome
luizirber has quit [Ping timeout: 265 seconds]
luizirber has joined #pypy
forgottenone has quit [Quit: Konversation terminated!]
<Dejan>
antocuni, is it any better than libabigail's abidiff
luizirber has quit [Ping timeout: 246 seconds]
luizirber has joined #pypy
_whitelogger has joined #pypy
<muke>
antocuni for some reason, including 'separate_module_sources' and 'post_include_bits' into the compilation info makes rffi uanble to see llvm-c altogether..
<antocuni>
muke: as usual, if you show the source code and the error it makes it easier to help you ;)
<antocuni>
(btw, you shouldn't use lines which are so long. In pypy we have a sort-of-implicit limit to ~80 chars/line
<muke>
oh, I wasn't aware, I'll restyle my code
<antocuni>
there is a coding guide IIRC, but I'm not sure if it says anything about code length :)
<antocuni>
it looks like it can't find the header file
<antocuni>
you should use "include_dirs" in ExtenralCompilationInfo instead of passing the flags directly to the compiler
<muke>
yea, not sure why though - if I remove the extra flags that make up the wrapper it works as normal again
<muke>
sure
<antocuni>
uhm
<muke>
oh, that actually fixed it I think
<antocuni>
good :)
<muke>
thanks for pointing that out
<antocuni>
I think that the problem was that under the hood rffi tries to compile a small C file with the code that you put in "separate_module_sources"
<antocuni>
and probably it doesn't pass cflags to that specific compilation (but it DOES respect include_dirs)
<antocuni>
as you can see, rffi is a mess :(
* antocuni
off
<muke>
it's been somewhat of a struggle so far but i am making progress aha
forgottenone has joined #pypy
papangoo[m]1 has joined #pypy
_linker_ has quit [Ping timeout: 272 seconds]
_linker_ has joined #pypy
_linker_ has quit [Remote host closed the connection]