<Waskeend>
Hi everyone. I am trying to wrap a (measurement instrument) driver and from what I read CFFI seems to be the best option for that.
lapinot has quit [Read error: Connection reset by peer]
<Waskeend>
But one of the things that is still unclear to me is: When I compile the CFFI wrapper, do I need to use the same compiler that was used for the original dynamic library?
arigato has joined #pypy
<Waskeend>
e.g. I am fairly certain that the driver I want to wrap was compiled with MSVC. Do I have to use MSVC aswell, or could I use MinGW gcc instead?
<arigato>
on Windows, the situation is a bit complicated
<Waskeend>
(I'd like to avoid having to buy a license for MSVC )
<arigato>
note first that nowadays, you can get MSVC for free
<arigato>
the details on whether it will work or not are: normally yes, but it depends about the precise interface
<arigato>
for example, if the API passes "FILE *", then you need the exact same compiler, because different compilers have different FILE structures
<arigato>
if the API is only passing standard C types, then it should work without problem
<arigato>
(the C built-in types are ok, and the basic Windows HANDLE and stuff are ok)
<Waskeend>
@arigato, hmm, is MSVC also free for commercial use? AFAIK I can only install MSVC by installing the Visual studio IDE, which is not free for enterprise use.
<arigato>
right, maybe not
<Waskeend>
@arigato the only option I found to install MSVC stand-alone was the "Visual C++ Build tools" which is again coupled to a license for Visual Studio. :/
<fniephaus>
hi all, does anyone know of a RPython-based interpreter for the R language?
<Waskeend>
Domo @arigato
<fniephaus>
nevermind, I just found rapydo :)
marr has quit [Ping timeout: 240 seconds]
<cfbolz>
fniephaus: it's pretty dead though
<fniephaus>
cfbolz: saw that on the "Projects Using RPython" page, that's unfortunate..thanks anyway!
<cfbolz>
R is hard
<cfbolz>
arigato: how hard would it be to implement back in time debugging for another rpython language?
nimaje is now known as Guest8579
nimaje1 has joined #pypy
Guest8579 has quit [Killed (card.freenode.net (Nickname regained by services))]
adeohluwa has joined #pypy
oberstet2 has quit [Ping timeout: 245 seconds]
antocuni has quit [Ping timeout: 245 seconds]
vkirilichev has joined #pypy
jacob22_ has joined #pypy
<arigato>
cfbolz: that's relatively easy
<arigato>
see pypy.interpreter.reverse_debugger
<cfbolz>
arigato: cool, I might try for prolog at some point
<cfbolz>
arigato: would be cool to be able to say 'go back to last failure', or 'watch this variable'
oberstet2 has joined #pypy
<arigato>
yes, indeed
<arigato>
note that some of the logic in reverse_debugging.py is an almost-duplicate of the logic in executioncontext.py to detect when we moved to the next line
<arigato>
I reimplemented it so that it has constant-time complexity
<arigato>
I don't want to move that back to executioncontext.py because, probably, some obscure details are different and, probably, someone depends on them
<cfbolz>
arigato: right, I see
<arigato>
I'm not sure I'm missing something, but the only interesting "watch" in prolog would be: take this variable and watch when it becomes bound
<arigato>
you don't even need to evaluate expressions all the time
<arigato>
ah, or yes, you might want to do that to track changes in what the variable is bound to (with backtracking)
<cfbolz>
arigato: yes, it can have other values in other branches
<arigato>
a variable cannot change from "bound to x" to "bound to y" directly though
<arigato>
it has to go via "unbound"
<arigato>
we could write conditions like "X == foo(y)" and have them always be interpreted as "!!(X == foo(y))"
<arigato>
so "X == foo(y)" would trigger a watchpoint when X becomes bound to something else than foo(y)
<arigato>
some natural questions like "who bound X in the past?" don't have natural answers in prolog...
jacob22_ has quit [Quit: Konversation terminated!]
<cfbolz>
arigato: I see
<cfbolz>
arigato: I think I would try to design it using actual prolog bugs
<arigato>
good idea
<cfbolz>
But already bnext would have been great for me in the past ;-)
<arigato>
:-)
<arigato>
hum, above, I meant "X = foo(y)", not "==", but "X == foo(y)" is also interesting and maybe easier to use
<cfbolz>
fniephaus: what do you have planned with rapydo?
<cfbolz>
fniephaus: if you do stuff with it, I'd much rather give you commit access to it and you can take over
arigato has quit [Quit: Leaving]
marr has joined #pypy
gabrielm has joined #pypy
<gabrielm>
Hello, I have a question regarding the StringBuilder in PyPy. I have a script in which I need to concatenate multiple words received from a network socket, and I've used this. Even though it is 3X faster than python, I noticed that it uses 5X more memory. I am wondering why append into a list is consuming so much memory
<cfbolz>
gabrielm: stringbuilder overallocates very aggressively
<gabrielm>
oh, did not know that. Is there another way to do it?
<gabrielm>
more efficient
<cfbolz>
well, it's a time/memory tradeoff
<gabrielm>
i tried with + but it's very slow
<cfbolz>
just use a list?
<gabrielm>
yes, I use it. It's something like L=[] ; while(TRUE): word = readfromsocket(); L.append(word)
<cfbolz>
yes
<gabrielm>
yes, that's the code I'm using right now, with list and append
<gabrielm>
you are saying that this is the best way to do it in pypy?
<cfbolz>
gabrielm: I think stringbuilder is faster, but as you observed takes more memory
inad922 has quit [Quit: Leaving]
<gabrielm>
ok, thx cfbolz!
<cfbolz>
gabrielm: anyway, it's best to just measure
<gabrielm>
I did that :)
<bremner>
would some kind of mixed strategy work, do the low level concats with stringbuilder, then push those larger strings onto a list for further processing?
jamadden has joined #pypy
<gabrielm>
bremner: but StringBuilder already is implemented using a list
<gabrielm>
I don't understand what do you mean
<bremner>
ah, nvm, reading a bit I think the results of stringbuilder are supposed to be immutable.
lapinot has joined #pypy
<cfbolz>
gabrielm: stringbuilder does not use a list
lapinot has quit [Client Quit]
<gabrielm>
cfbolz: i used this in my script: from __pypy__.builders import StringBuilder
<cfbolz>
sure
<gabrielm>
there are other stringbuilders?
<cfbolz>
but that is not implemented as a list
lapinot has joined #pypy
<gabrielm>
I looked in the pypy sources and found this file rstring.py where StringBuilder is implemented using a list
<gabrielm>
rpython/rlib/rstring.py more exactly, class StringBuilder(AbstractStringBuilder):
<cfbolz>
yes, but that is not the implementation used after translation
<cfbolz>
that's just the simulation when running tests
<cfbolz>
the real implementation is more complex, in rpython/rtyper/lltypesystem/rbuilder.py
adeohluwa has quit [Quit: Connection closed for inactivity]
<fniephaus>
cfbloz: no, I don't have anything planned with rapydo (yet), but will let you know if I do
<cfbolz>
ok
mihaid has joined #pypy
pilne has joined #pypy
<mihaid>
hello. I have a question regarding profile guided optimizations. Right now they work, but without the link time optimizations because they used to make the compilation fail for pgo in the early stages of the work. Should I open an issue regarding this? I took another look and I think after the pull request and I think it should be easier to enable
<mihaid>
now.
<cfbolz>
mihaid: yes, if you can get lto to work as well, that would be great
<mihaid>
Then I will open a new issue and submit a PR when its ready.
<mihaid>
cfbolz: should I also include a performance test?
<palecsan>
@fijal, on OSX it's using clang, which is known to be dead slow on LTO :(
<fijal>
should we disable LTO for OS X then?
<palecsan>
the same is happening if you are using clang on linux
<palecsan>
oh, no
<palecsan>
it's slow to build
<palecsan>
but at the end you get a faster binary
<palecsan>
I'd leave it there and hope clang/llvm improves over time
<palecsan>
on my personal laptop I've installed gnugcc with brew to get rid of clang
<palecsan>
I have a question regarding issue #2565 (ctypes) - I found the line that outputs the empty string (return _rawffi.charp2string(resarray.buffer, self._length_) from _ctypes/array.py). If I remove it (together with the wcharp2unicode) the output is the same as CPython
<palecsan>
do you find this fix acceptable?
<palecsan>
is there a reason why you added code to transform the buffer to a string?
<fijal>
palecsan: you're asking very hard questions
<fijal>
I don't think anyone remembers very well by now
<mihaid>
cfbolz: I also have a question regarding the sendmsg / recvmsg. I looked into the methods description in socket.h and I know what should be done, however, I am rookie with rffi / lltype. Are there any examples / documentation you could point me to? The problem is that there are a lot of structures that need to be used and I am not sure how to alloc
<mihaid>
ate memory for them. (lltype malloc works a bit differently than what I am used to)
<palecsan>
fijal: ok, then I'll proceed with the fix
<cfbolz>
mihaid: what kind of memory? Some C structure?
<cfbolz>
fijal: I don't think pgo or lto would be on by default. but we could consider turning them on for release builds
<mihaid>
yes. for these two methods you need a structure that encapsulates one or more messages(iovec --unlike the regular send /recv, these 2 support sending multiple messages). Then on top of that, there is a structure that encapsulate the previous structure, some flags, the address, and some ancillary data (extra data, lets call it). So I am not quite su
<mihaid>
re which flags to set when I allocate memory for these structures. For example, what does immortal = True do? Or nonmovable =True? Should I be worried that the garbage collector will interfere with the structures I created?
<cfbolz>
mihaid: those structs are only needed for the duration of the call, right?
<mihaid>
yes
<cfbolz>
then you are looking for lltype.scoped_alloc
<mihaid>
oh this looks nice! thanks!
<cfbolz>
palecsan: do all the ctypes tests pass with those changes?
<palecsan>
cfbolz: I've ran tests from pypy/module/test_lib_pypy/ctypes_tests/test_array.py and they all pass
<cfbolz>
please run all the tests in there, not just the single file
<cfbolz>
and also ideally the cpython ctypes tests in lib-python/2.7/ctypes/test
<palecsan>
cfbolz: oh, ok
marr123 has joined #pypy
marr is now known as Guest27094
marr123 is now known as marr
Guest27094 has quit [Killed (verne.freenode.net (Nickname regained by services))]
<kenaan>
arigo cffi/cffi[sirtom67/float_complex] 9a8456642302 /c/: Minor style clean-ups. Update the test to conform more closely to the newer version of t...
<kenaan>
arigo cffi/cffi[sirtom67/float_complex] ab6edb3987e7 /c/: Fix test, fix all abort()
<kenaan>
arigo cffi/cffi[sirtom67/float_complex] f0f90cb166e5 /c/: Test explicitly that if we try to call a function in ABI mode with a complex argument or ...
oberstet2 has quit [Ping timeout: 260 seconds]
tbodt has quit [Remote host closed the connection]
pilne has quit [Quit: Quitting!]
realitix has joined #pypy
<realitix>
Hello arigato, I'm not selected currently for Europython :-( On waiting list...
tbodt has joined #pypy
pilne has joined #pypy
<mattip>
numpy did some refactoring of ufunc calling, and the upshot is they re-use a tuple of arguments for the reverse call of a function (i.e.
<arigato>
mattip: time to make it crash explicitly in this case
<mattip>
good idea
vkirilichev has quit [Remote host closed the connection]
vkirilichev has joined #pypy
vkirilichev has quit [Ping timeout: 246 seconds]
<kenaan>
arigo cffi/cffi[sirtom67/float_complex] de5953e587b8 /cffi/vengine_cpy.py: Revert the changes to vengine_cpy: better to say that the long-deprecated verify mode doe...
<kenaan>
arigo cffi/cffi[sirtom67/float_complex] f6e7adb9a3b5 /: Support parsing "float _Complex" and "double _Complex" inside parse_c_type
realitix has quit [Ping timeout: 245 seconds]
jcea has quit [Quit: jcea]
<kenaan>
arigo cffi/cffi[sirtom67/float_complex] 117488cc30fc /: More tweaks, more tests, try harder to avoid including <complex.h> because it is not nece...
cstratak has quit [Quit: Leaving]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
realitix has joined #pypy
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]