cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<muke>
I'm trying to interact with the llvm-c api through rpython's ffi, it takes and returns a lot of C++ objects that don't seem to actually be defined with concrete/primitive types, instead the llvm-c api itself just defers defining them (i'm honestly not too sure about the details) - I also found a (incomplete) python binding for the llvm-c api which refers to all these types with ctypes using void pointers. Would it be ok to do the same thing in rpython's lltype
<muke>
interface?
<muke>
so, to reiterate that a bit more specifically, for a type like 'LLVMModuleRef', can I specify this as just a void pointer from rpython?
todda7 has quit [Remote host closed the connection]
<mattip>
i don't know if it would be plausible to use the rpython/tool/cparser/cts.py CTypeSpace at all to parse some form of the header files
<mattip>
how would you interface to the llvm-c api from C ?
<muke>
I'm unfamiliar with that script, what's it for?
<muke>
from C you just include and call the functions directly
<mattip>
if you include them there must some concrete "typedef struct" somewhere
<muke>
that's what I thought
<muke>
in the Types.h file for llvm-c though it explains
<muke>
'LLVM uses a polymorphic type hierarchy which C cannot represent, therefore parameters must be passed as base types.'
<muke>
and then it's uses a serise of typedefs to define it's types as ones from, what I think at least, are from the C++ api
<muke>
again I'm not quite sure how this actually works but there aren't any struct definitions anywhere to simply draw from
<muke>
but it seems from the perspective of python, it doesn't really matter as you can refer to them as void pointers and then have the API figure out the details, but I'm just wondering if that would still work under the type inferance rpython requires
<mattip>
how do you use these? If there are interface functions to allocate/free/interact with them, then void* is find
<mattip>
fine
<muke>
that's a good question, I'm still learning the api but from skimming I think there might be yea
<muke>
aha thanks, I have that and many similar tabs open already :p
<mattip>
the problem will be when the llvm and rpython op model diverge
<antocuni>
muke: note that I have little idea of what you are trying to do :), but before diving into the magic world of rpython/rffi/ctypespace, I strongly suggest to play with the llvm-c APIs "by hand" (i.e., write a C program which calls the API you are interested in)
<antocuni>
once you have that, it should be pretty straightforward to turn it into rffi
<muke>
antocuni I'm looking to build a pypy llvm backend for my masters project, so I need (or would at least like) the llvm api through rpython
<muke>
I've messed with using the c api directly but either way the main problem I'm just trying to get my head around is how the types interact
<muke>
why would the llvm backend be special in that case out of curiosity? wouldn't you have to adapt all the backends if you changed something about pypy's IR?
<muke>
sorry if I'm misunderstanding your point
<cfbolz>
mattip says that some of the opcodes are easy to map to llvm, some aren't
<muke>
ah i see
<antocuni>
muke: so you are going to write a LLVM backed for the JIT, right? (as opposed to a rpython backend for the normal translation)
<antocuni>
muke: depending on the details, the easiest solution might be to write your own C api on top of the C++ llvm API
<muke>
that's the goal yes
<cfbolz>
there *is* an existing C api
<muke>
well that already exists as llvm-c
<muke>
it only has partial coverage, but there's another project which extends it i intend to draw from if need be
<muke>
the problem i've been looking at today (and i think solved) has just been how to represent llvm's custom types in rpython's ffi
<antocuni>
cfbolz: yes, I'm just saying that depending on how it's done it might be easier to write our own thin layer on top of it (either on top of llvm-c or llvm-c++). But I'm talking without knowing the details, so it's entirely possible that I'm saying nonsense :)
* cfbolz
hasn't looked either
<muke>
antocuni just out of curiosity are there any specific problems you image another layer solving?
<antocuni>
rffi supports many features of C but not everything
<antocuni>
e.g. if you need to pass/return structs by value, you might have problems
<antocuni>
also, if the C API is large, you might have to declare many C structs and functions in rffi, which is annoying. With a custom API, you might be able to minimize the wrapping by exposing only the few things that you need
<muke>
well that's good to know if i come across something the ffi can't support
<cfbolz>
antocuni: I don't suspect there are many exposed structs
<cfbolz>
I kind of expect almost everything to go via accessor functions
<antocuni>
sure, as I said I'm talking out of the blue :)
<muke>
antocuni I appreciate the input! I'm honestly as blind as you currently aha
<cfbolz>
it's rare for public fields in c++ classes after all
<antocuni>
speaking of the annoyance of rffi wrapping, consider also matti's suggestion of using ctypespace. It's basically a helper which parses C code and spits out rffi structs, so that you can declare your structs/functions in C instead of the ugly rffi DSL
<antocuni>
you can find examples inside cpyext and hpy
<muke>
that's good to know, although unless I've missed it I don't think there are any concrete struct definitions in llvm-c's header files, which I assume is what it would expect?
<muke>
there's a comment explaining what it's doing although I'm not certain what it means
<muke>
but all the typedefs are just one liners like the one at the bottom there, and I think the types they're referring to are defined in the C++ api
<muke>
(that's just a snippet not the full file)
<antocuni>
ah, I think it means that they couldn't precisely map the C++ type hierarchy to C
<antocuni>
so for example suppose that in C++ you have class A and class B(A)
<antocuni>
and you have a function which takes a B*
<antocuni>
I think that for some reason, the C equivalent takes a A* (and note that all B instances are also A instances, so you can cast a B* into a A*)
<antocuni>
but if you call foo() with something which is actually an A* instead of a B*, it complains/explode
<muke>
yea that makes sense
<muke>
there's also a python binding for the llvm-c api which refers to all the types declared in that file with void pointers through ctypes
<muke>
so that's what i was going to do through rffi
<muke>
the api has destructors for each object you create with it so memory managment should be fine
<antocuni>
yeah, I suppose it should work
<muke>
*should*
<muke>
it's a shame the python binding is still a work in progress I suppose
<antocuni>
well, you woulnd't be able to use them anyway