kipras`away is now known as kipras
marr has quit [Ping timeout: 268 seconds]
adamholmberg has quit [Remote host closed the connection]
adamholm_ has joined #pypy
adamholm_ has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 272 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
Rhy0lite has quit [Quit: Leaving]
Taggnostr has quit [Read error: Connection reset by peer]
kipras is now known as kipras`away
adamholmberg has quit [Ping timeout: 240 seconds]
yuyichao has quit [Ping timeout: 260 seconds]
Taggnostr has joined #pypy
tav` has joined #pypy
tav has quit [Ping timeout: 255 seconds]
tav` is now known as tav
adamholmberg has joined #pypy
yuyichao has joined #pypy
adamholmberg has quit [Ping timeout: 272 seconds]
vkirilichev has joined #pypy
jcea has quit [Quit: jcea]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 272 seconds]
Taggnostr has quit [Ping timeout: 240 seconds]
ArneBab_ has joined #pypy
pilne has quit [Quit: Quitting!]
ArneBab has quit [Ping timeout: 260 seconds]
lritter has joined #pypy
marky1991 has quit [Ping timeout: 255 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 268 seconds]
marky1991 has joined #pypy
vkirilichev has quit [Ping timeout: 260 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 260 seconds]
vkirilichev has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 260 seconds]
<sthalik> hey
<sthalik> which directory tree part has the internal representation a) for rpython b) for jit?
<sthalik> the data structures and their semantics
jacob22_ has quit [Quit: Konversation terminated!]
vkirilichev has quit [Ping timeout: 240 seconds]
kipras`away is now known as kipras
kipras is now known as kipras`away
jamadden has quit [Quit: Leaving.]
sthalik has quit [Remote host closed the connection]
vkirilichev has joined #pypy
vkirilichev has quit [Remote host closed the connection]
amaury has joined #pypy
amaury has quit [Client Quit]
amaury has joined #pypy
sthalik has joined #pypy
<LarstiQ> sthalik: that question sounds a little off to me
<LarstiQ> sthalik: but check /rpython and /rpython/jit
<LarstiQ> sthalik: stepping back, what is your goal?
<sthalik> LarstiQ, my real question is whether there's intermediate IR from which both jit and rpython C are generated
<LarstiQ> for your llvm experiment?
* LarstiQ doesn't really know the answer
<LarstiQ> I'd probably look at rpython/jit/backend to see what it interfaces with, the hg history for previous non-C backends, and maybe some of the papers
sthalik_ has joined #pypy
<sthalik_> LarstiQ, apologies, can you paste me your response as query or here?
<sthalik_> having the same IR for JIT would make it less regression-prone
<LarstiQ> make what less regression-prone?
<sthalik_> also, the evil idea of making a temporary .so to load
<sthalik_> JIT backends like llvm
sthalik has quit [Ping timeout: 272 seconds]
<sthalik_> wrt regressions -- use LLVM jit-ter for rpython generation, as an additional test case
<sthalik_> also llvm has IPA that actually works when used outside the binutils ld plugin
<sthalik_> oh thanks, that looks really important
<LarstiQ> and its references
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 268 seconds]
GothAlice has joined #pypy
tilgovi has joined #pypy
<GothAlice> https://travis-ci.org/marrow/mongo/jobs/230637362 ← I appear to have a case where Pypy (5.4.1) does not interpret a MutableMapping.register()'d class as a mapping as per CPython 2/3 on the same codebase as seen by PyMongo. (A… complex case.) Commit ref. included in travis job description.
<GothAlice> Temporarily I'll be working around the problem in internal code by referencing the operations attribute directly (the actual filter / update document to PyMongo's eyes), but I'm wanting to avoid the extra dereference on each use in practical use.
<LarstiQ> GothAlice: what about using pypy 5.7?
<GothAlice> LarstiQ: Do you have an example handy of integrating that into Travis testing workflow?
* GothAlice would love to give that a shot.
<LarstiQ> GothAlice: have you tried asking them to update?
<GothAlice> LarstiQ: Is this a known issue, then?
<LarstiQ> GothAlice: I don't know, but 5.4.1 is old
<GothAlice> Okay.
<LarstiQ> maybe it's fixed, maybe not
<GothAlice> Quite so.
<GothAlice> None of that helps, but thank you for trying.
<LarstiQ> just saying that 5.4.1 is not maintained as such, so either the fix will be made to code that is much closer to 5.7.1, or it is already done
<LarstiQ> GothAlice: does this problem appear outside of Travis?
<GothAlice> I've encountered it before.
<GothAlice> Yes.
<LarstiQ> if not, reproducing it is going to be your main problem
<LarstiQ> ok, that's good at least
marky1991 has quit [Ping timeout: 240 seconds]
<GothAlice> The whole ABC thing is kinda wonky on Pypy, in my experience.
realitix has joined #pypy
adamholmberg has joined #pypy
mattip has quit [Ping timeout: 264 seconds]
tilgovi has quit [Ping timeout: 246 seconds]
<GothAlice> Can confirm it's that exact usage of manual ABC registration; avoiding the indirection of the wrapped objects avoids the symptom in follow-up testing (all but one case fixed with three deeper references hard-coded): https://travis-ci.org/marrow/mongo/jobs/230643869
<GothAlice> Subclassing the ABC _does_ work correctly in this particular instance: https://travis-ci.org/marrow/mongo/jobs/230644878 — This does not correct my overall issue, however, as I have classes with incompatible metaclasses that I also wish to participate in the ABC. (With prior Travis failures and work-arounds for the previously encountered incidents of this.)
<GothAlice> I'll… see what I can do to get a later version in there, and will certainly test a newer version locally. (Think I saw an update on brew for that just last week.)
adamholmberg has quit [Ping timeout: 240 seconds]
jacob22_ has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
marr has joined #pypy
adamholmberg has quit [Ping timeout: 245 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 260 seconds]
<njs> GothAlice: here's how I install the latest pypy 3 nightly build on travis for testing, in case it's useful as a reference: https://github.com/python-trio/trio/blob/master/ci/travis.sh#L20
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
<GothAlice> njs: Nice, that's _way_ more elegant than the other things I've been seeing.
<GothAlice> Others involved several dozen lines of .travis.yml. >_<
adamholmberg has joined #pypy
<njs> GothAlice: if I wanted a release build instead I'd probably start with one of squeaky's: https://github.com/squeaky-pl/portable-pypy
<njs> GothAlice: oh yeah, I try to get out of travis.yml asap
adamholmberg has quit [Remote host closed the connection]
<GothAlice> The last one even included YAML merging dict references. Wild.
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
<njs> GothAlice: the .travis.yml that goes with it is here: https://github.com/python-trio/trio/blob/master/.travis.yml
adamholmberg has joined #pypy
<GothAlice> Right, as a custom matrix include.
<njs> I really wish people would stop using yaml entirely, but apparently this is an unpopular opinion
<GothAlice> njs: Thanks! This will really help kickstart https://github.com/marrow/mongo/issues/31 :D
vkirilichev has joined #pypy
<GothAlice> Like any markup, it does serve useful functions. JSON being a valid subset is occasionally handy. (Notably, e-commerce "packages", which bundle discounts applied any number of ways to items or various sub-totals, for price, cost, and reporting, for one or more combination of products with one or more combination of balances with different units of measure… YAML is handy for defining these quickly and easily.)
arigato has joined #pypy
<GothAlice> Way easier than trying to design a UI for such complex nested data, at least, for the short term. ;P
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
<njs> GothAlice: oh sure, there are many things which I acknowledge are useful but that I still avoid if I have any choice in the matter :-)
amaury has quit [Ping timeout: 240 seconds]
adamholmberg has quit [Ping timeout: 255 seconds]
oberstet has joined #pypy
<fijal> hi
<fijal> GothAlice: I didn't read the whole thing
<fijal> but people said pyenv as a "handy way"
<GothAlice> fijal: https://github.com/marrow/mongo/issues/31 is the gist of it.
<fijal> ok
<fijal> GothAlice: so nothing we can fix?
<GothAlice> fijal: I'm trying very hard to not add additional moving parts or complexity. E.g. I used to use Tox. Then Tox started killing my `bird` process (cloud file sync) due to the sheer number (in the millions) of on-disk files being created and deleted regularly, so it was gone. Etc. ;P
<fijal> I'm not even suggesting this is a good solution, I'm just repeating what people told me
<GothAlice> fijal: Once I re-test the issue (still one for the Document class in my system, which has an ABC-incompatible metaclass), if it's still an issue, I'll ping here again.
<GothAlice> I actually think this issue is why I gave up on CPython 2.x Pypy versions; I couldn't work around the ABC weirdness on that version. I'll have to re-enable and re-test that, too.
<fijal> ok
adamholmberg has joined #pypy
antocuni has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
<sthalik_> can a metatraced language's object layout be arbitrarily created?
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
mihaid has joined #pypy
adamholmberg has quit [Remote host closed the connection]
cstratak has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 272 seconds]
<mihaid> Hello, I have a question regarding ctypes segfaulting when the wrong parameter is passed to libc function such as strlen (issue 466 on bitbucket). From what i have gathered, this happens due to a fast path to execution of the function that skips checking arguments. My question is what is the purpose of the fast path ( is the slow path way to slow t
<mihaid> o be always viable)? Also, should there be another parameter check done by the rawffi library? (I am trying to understand whether this is a ctypes issue or rawffi)
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 255 seconds]
marr has quit [Read error: Connection reset by peer]
mattip has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 255 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
<fijal> mihaid: I think ctypes should check
<fijal> out of the blue
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
lritter has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 246 seconds]
<fijal> does anyone know if we have any code that does "recreate me a wrapper with this and that signature"?
<fijal> like, uncluding defaults
<antocuni> fijal: a quick grep for func_defaults doesn't show anything like that
<sthalik_> fijal, can i make an arbitrary data layout with rpython?
<fijal> sthalik_: I'm not sure what's the question, but probably not?
<fijal> e.g. you can't have a list [None, 1, "foo"]
<sthalik_> no, no
<sthalik_> to make values at runtime have a particular in-memory structure layout per datum
<fijal> you have to use other things than regular classes
<fijal> what are you trying to do?
<fijal> like lltype.GcStruct or lltype.Struct
<sthalik_> fijal, i'm reading rpython papers while cross-compiling for win64, and it's interesting, though raises many questions
<fijal> how about we start with compiling on win64 as opposed to cross compiling?
<fijal> but also, generally, we try not to assume certain things unless you're calling system calls
<fijal> in which case we don't pass RPytohn classes down
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
<cfbolz> GothAlice: if you can narrow done the ABC problems to a smallish example, we'd love to take a shot at fixing those problems
adamholmberg has quit [Ping timeout: 240 seconds]
vkirilichev has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
arigato has quit [Read error: Connection reset by peer]
antocuni has quit [Ping timeout: 260 seconds]
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 260 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
stevi3 has joined #pypy
Tiberium has joined #pypy
adamholmberg has quit [Ping timeout: 246 seconds]
<fijal> cfbolz: what is create_if_subclassed?
<fijal> it does not seem to be called anywhere
<cfbolz> seems like a remnant?
oberstet2 has joined #pypy
<cfbolz> fijal: the logic must be somewhere else now though, it's quite common to turn a subclass into just a unicode
<fijal> ok
<fijal> bah, passing space to W_UnicodeObject constructor seems like an awful lot of effort
oberstet has quit [Ping timeout: 240 seconds]
<fijal> btw, somehow recent changes (e.g. flto?) make ld on OS X run for a bit ever
<fijal> 3min so far
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
<fijal> 5
<fijal> ok, I give up
adamholmberg has quit [Remote host closed the connection]
<fijal> yop it's flto
oberstet2 has quit [Ping timeout: 268 seconds]
jcea has joined #pypy
amaury has joined #pypy
Taggnostr has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
<mihaid> I disabledthe fastpath for ctypes on a local fork and issue 466 seems fixed now (no longer segfaults). Is this acceptable?
<fijal> to disable completely the fastpath?
adamholmberg has quit [Remote host closed the connection]
<fijal> ideally we would still use it
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
<fijal> I'm not sure how many people depend on it, since we have cffi, but >0 I would think
adamholmberg has joined #pypy
<mihaid> I understand, but the issue was that fastpath does not check the parameters types, hence the issue of segfault when you pass a boolean to strlen.
<mihaid> The slow path does check just like CPython
<fijal> so how do we make it work?
<fijal> w do need to check the parameter types
<fijal> JIT will optimize it away anyway
amaury has quit [Quit: Konversation terminated!]
adamholm_ has joined #pypy
adamholm_ has quit [Remote host closed the connection]
adamholm_ has joined #pypy
<mihaid> From what I gathered up to now the only difference between the two (slow vs fast) is the time saved by checking the parameter types. I can run some tests, to see exactly how much slower this is.
adamhol__ has joined #pypy
<mihaid> If it is too slow, I'll see then how I optimize this further.
adamhol__ has quit [Remote host closed the connection]
amaury has joined #pypy
adamhol__ has joined #pypy
adamhol__ has quit [Remote host closed the connection]
adamholmberg has quit [Ping timeout: 240 seconds]
<fijal> well, I somehow doubt the explanation is that simple
<fijal> I suspect it has to do with the JIT
adamholmberg has joined #pypy
adamholm_ has quit [Ping timeout: 240 seconds]
<mihaid> would it be ok to add some sort of flag to determine if the code should take the fast path or the slow path?
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
<fijal> and what would it depend on?
adamholmberg has quit [Ping timeout: 268 seconds]
<mihaid> I was thinking this could as parameter flags work right now. Only this one would determine if a parameter needs evaluation or not.
jamadden has joined #pypy
<mihaid> Or for the enitre library, set after importing ctypes
<mihaid> Say, it would work like: import ctypes ; ctypes.checkargumets(True)
<mihaid> By default it would be false
<fijal> no, that wouldn't do
<fijal> we need to make the fast path transparent
<fijal> antocuni might know the details
<fijal> maybe the right answer is to kill the fastpath
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
jacob22_ has quit [Quit: Konversation terminated!]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 260 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholm_ has joined #pypy
adamholm_ has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
antocuni has joined #pypy
Rhy0lite has joined #pypy
Rhy0lite has quit [Changing host]
Rhy0lite has joined #pypy
<kenaan> mattip default e1fb7a07c627 /rpython/translator/platform/windows.py: win32: use jom.exe for parallel builds if available
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
amaury has quit [Ping timeout: 246 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 245 seconds]
<sthalik_> fijal, in rffi_platform, should "void*" be "<* Array of void>" or signed-long-long?
<sthalik_> they get in different classes that way and it makes a boo-boo
<fijal> rffi.VOIDP
amaury has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
<sthalik_> so in case "signed", it gets word size or what?
<sthalik_> that's the place it breaks, but the question is whether you want word size or a typical "int" size
adamholmberg has quit [Ping timeout: 268 seconds]
<LarstiQ> sthalik_: isn't this mentioned in the win64 porting document?
<sthalik_> there's one?
<LarstiQ> maybe you're already into more details than that
<LarstiQ> sthalik_: but if you hadn't seen that yet, take a look
amaury has quit [Ping timeout: 260 seconds]
jacob22_ has joined #pypy
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
<sthalik_> LarstiQ, thanks for the Signed explanation. is it sensible now in the gist for a pull request?
<sthalik_> LarstiQ, that assertion failure for maxint != word size: can it be conditionalized on stage2 build?
<sthalik_> frankly hacking cpython to get a different maxint value in order to translate at all, doesn't make much sense
adamholmberg has quit [Ping timeout: 240 seconds]
<sthalik_> and asm stuff works on offsets to segments etc so it's not like python could generate bad code
<LarstiQ> sthalik_: I'm just a lurker who remembers previous conversations and occasionally knows what link to drop
<sthalik_> LarstiQ, i see :)
<LarstiQ> don't mistake me for someone who knows what they're doing, I don't get my hands dirty enough ;)
<sthalik_> fijal, do you have lltypesystem stage2 import?
<LarstiQ> sthalik_: arigato would be the best person to ask
<LarstiQ> maybe on the mailing list
<sthalik_> yes but fijal told me not to pester Armin!
<LarstiQ> ah ok :)
<sthalik_> years ago :P
<LarstiQ> sthalik_: iirc he wrote http://doc.pypy.org/en/latest/windows.html#what-is-missing-for-a-full-64-bit-translation so asking to clarify details seems imo reasonable
<LarstiQ> but check the authorship
amaury has joined #pypy
<sthalik_> technically one could make a LP64 mingw offshoot but it's crazy
<fijal> sthalik_: i hereby remove the curse ;)
sthalik_ is now known as sthalik
<sthalik> fijal, can we conditionalize or drop this assertion?
<sthalik> if it stands, everyone translating with cpython must hack their copy first
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 260 seconds]
adamholmberg has joined #pypy
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
arigato has joined #pypy
<sthalik> arigato, i'm taking first steps with win64
<sthalik> can we change the assertion about maxint being equal, or if you're doing multiple stage compilation, don't run it on stage1?
<sthalik> i'd only run it on code/runtime generated by pypy
adamholmberg has quit [Ping timeout: 240 seconds]
<sthalik> arigato, yes
<sthalik> i found the first fix in lltypesystem LONG_BITS
<arigato> well, this paragraph says the first thing to do is to hack CPython
<sthalik> arigato, but it's hardly reasonable
<arigato> if you want to discuss changing the plan outlined there, it's fine, but it needs to be discussed first
<sthalik> okay, so first why-nots
<sthalik> 1) each user translating on cpython needing to hack their cpython copy first
<sthalik> 2) making no distinction between build and host environment is bad by itself
<arigato> I know about both 1 and 2
<arigato> it's not going to be easily wished away though
adamholmberg has joined #pypy
<sthalik> 3) making a 64-bit interpreter from a 32-bit multilib is sane
adamholmberg has quit [Remote host closed the connection]
<arigato> the plan being discussed on that page is IMHO the easiest way to go anyway
adamholmberg has joined #pypy
<LarstiQ> sthalik: you only need to get to self-hosting
<arigato> if instead you're attacking the problem of cross-translating a 64-bit version from a 32-bit Python, then it's probably possible too, but I think much harder
adamholmberg has quit [Remote host closed the connection]
<LarstiQ> sthalik: then 1) doesn't matter
amaury has quit [Ping timeout: 240 seconds]
adamholmberg has joined #pypy
<arigato> (for 1., the idea was that as soon as we have a pypy for win64, then it doesn't really matter any more because we can always use the pypy-for-win64 to translate newer versions)
<sthalik> arigato, why not, given that initial checks get the sizes from the C tests?
<sthalik> so "def sizeof" returns the truth with regard to the target environment
<arigato> sthalik: the problem is that we assume a bit left and right that we're not cross-translating, and I can't give you a definite exhaustive list of these palces
<arigato> ...places
<sthalik> there's also a difference in crossing, where the build machine can run the host code, like cygwin|mingw|msvc etc.
<arigato> moreover, likely some places would not crash if they are not fixed; instead they would just silently produce nonsense
<sthalik> arigato, is it before or after the jitter? we could make -O2 and use it to compile a stage2 jitter
<sthalik> there's an msvc flag to make -Werror for that particular warning id
<arigato> the first goal is indeed to get a minimal pypy-win64 with not all modules, just enough to run a complete translation
<arigato> you can't hope to catch all problems at the level of C compilation
<sthalik> but code smell, you know?
<sthalik> it has bad taste to do the maxint step
<arigato> I agree
<arigato> but I think it's more robust
<arigato> despite the smell
<sthalik> for windows there's the .pdb and drmemory also
<arigato> I'm sure
<arigato> but for example:
<arigato> (and it's just an example, by no means the complete set of examples)
<arigato> the value of sys.maxsize is computed as "space.wrap(sys.maxsize)"
<arigato> in RPython
<arigato> that means that if your host Python is 32-bit,
<arigato> it will get the value 2**31-1
<arigato> and write that as C code
<arigato> there is nothing wrong about writing 2147483647 in a C constant of type "long long" or so
<arigato> it's just bogus in this case
<arigato> yes, what's really wrong is the fact that we compute sys.maxsize as "space.wrap(sys.maxsize)" in RPython
<sthalik> what is space.wrap?
<arigato> but too bad, you have to assume that the whole RPython source code is similarly bogus
<arigato> space.wrap(x) just means "make some W_Root subclass with that value in it"
<fijal> antocuni: hi
<antocuni> fijal: hi
<arigato> "PyInt_FromLong()" if you like
<fijal> antocuni: do you know anything about ctypes fast paths?
<antocuni> uh... I suppose I *knew*
<sthalik> arigato, i'm gonna grep for maxsize in addition to LONG, then
<arigato> my point is that it's just an example
<antocuni> but they have been probably refactored a million of times since then
<arigato> sthalik: there are many other less direct ways in which the host interpreter would leak into the translated interpreter
<arigato> and the main problem is:
<fijal> antocuni: so apparently fast path does not check types of arguments
<arigato> there is no automatic way to find them all
<sthalik> arigato, we can use the cygwin interpreter
<antocuni> sthalik: I think that what arigato is trying to say is that "make pypy cross-compilable" and "make pypy working on win64" are two different (and hard) tasks
<sthalik> it's LP64
<antocuni> you can do one or the other, but if you try to do both at the same time, it's going to be a nightmare
<antocuni> fijal: ouch :(
<arigato> sthalik: ah! you mean compiling CPython with cygwin gives a CPython which uses 64 bits for its native integers?
<sthalik> arigato, yes
<arigato> sthalik: cool! then yes, that fixes the first problem
<arigato> well, assuming that the rest of PyPy loads correctly in the cygwin python
marky1991 has joined #pypy
<sthalik> one can also hack a C compiler and headers so it's LP64 but windows types work
Taggnostr has quit [Ping timeout: 272 seconds]
<arigato> that's a complete hack that looks worse than just editing the CPython sources
<arigato> anyway, the CPython64/64 I describe might not be too much work
<sthalik> in cpython sys.maxsize seems read-write
<fijal> antocuni: scroll the history for example
<sthalik> i assigned it a value of one and it still works
<arigato> yes, you can also change sys.maxint and sys.maxsize by manually overwriting it early enough, but at that point it's just as easy to change the CPython sources if you need to change intobject.c anyway
<sthalik> okay
<jkbbwr> Anyone got any ideas
<LarstiQ> plan_rich_: ^^
<fijal> jkbbwr: that was fixed in -dev
<jkbbwr> fijal: is it just pip install vmprof-dev or something? :3
<fijal> jkbbwr: I don't know why the release was not added, please use previous or pip install --dev or --pre or soemthing
<jkbbwr> Sure!
<fijal> I *think* it's pip install --pre
<jkbbwr> Time to run the 2 minute test again
Taggnostr has joined #pypy
<fijal> arigato: hm, I have a commit, but to default instead of a new branch, how do I do the branch?
<arigato> need more context to be able to answer that
<arigato> hg? you have a commit that you didn't push so far?
<fijal> yes
<jkbbwr> I wonder why I am getting 142k read errors and its quite slow http://vmprof.com/#/cb04ffca-b502-42a1-b9dd-77dfa958491d
<arigato> if it's just one commit, just "hg rollback" cancels it
<fijal> arigato: no, rollback does not work
<fijal> (I tried things and it erased rollback info)
<arigato> because..?
<arigato> ok
<fijal> who knows
<fijal> jkbbwr: read errors?
<arigato> then switch to the branch, and do "hg graft"?
<jkbbwr> fijal: Different problem I am musing over myself. Im more interested at how to make this faster :)
<arigato> and when everything looks fine, kill the old commit
<mihaid> fijal & antocuni: sorry to bother you again guys, but I ran some tests on ctypes, and it appears that the slow path is roughly 6X slower than the fast track. Also I tweaked the fast path to just do the type-checking and the result were 6X slower as well (added _convert_args_for_callback in the fast track). I am not sure where to go from here.
<arigato> "hg strip 12345678"
<fijal> (bloc11)[brick:~/src/pypy-str-measure] $ hg graft 8fbed3696bd4
<fijal> skipping ancestor revision 91307:8fbed3696bd4
<fijal> it makes no sense
<arigato> well it's in your ancestry then?
<fijal> arigato: how do I create a branch now?
<arigato> if you want to create a fresh branch, you must start from just before the commit, of course
<arigato> if the goal is to kill that commit in the end
<antocuni> fijal: you can probably also do hg up -r <last-commit-before-yours>, hg branch <name>, hg rebase <your-commit>
<arigato> ah, thanks
<fijal> I wish there was less than 500 non-obvious ways to do all of that
<arigato> "hg branch" then "hg commit -m'my new branch'"
<fijal> yes got it eventally
<fijal> pushing
<kenaan> fijal str-measure 13296622c1b1 /: new branch
<kenaan> fijal str-measure 9923d9d8b6c5 /: a branch to start measuring how strings are used
<antocuni> mihaid: I remember that years ago the ctypes fast path was much much faster than the slowpath
<antocuni> if it's "only" 6x and it's broken, we might consider dropping it
<antocuni> (if you care about performance you should use cffi anyway)
<fijal> I wonder if it has adverse consequences
<jkbbwr> wrk is still giving me tons of bad reads :(
<mihaid> antocuni: Thanks for the answer! I also have a "middle-road" idea, I think. Ctypes allow you to call libc functions without specifying the argtypes (bad code practice, but it is possible). What I thought, is that the fast path should be used only if there are no argtypes specified? It seems less heavy handed than a flag, but again it is just an ide
<mihaid> a to save the fast path as well.
<antocuni> yes, I think it makes sense. But you should also double check that the speedup comes from the fastpath, and not from the fact that we don't check the arguments :)
jcea has quit [Ping timeout: 240 seconds]
jcea has joined #pypy
<jiffe> does python cffi not understand #includes that include other includes?
<jiffe> if I put everything in a single file it works, but separating it out and it doesn't seem to know what the defined structs are
jcea has quit [Remote host closed the connection]
jcea has joined #pypy
<arigato> jiffe: in cdef(), or in set_source()'
<arigato> ?
marky1991 has quit [Ping timeout: 260 seconds]
yuyichao has quit [Ping timeout: 272 seconds]
cstratak has quit [Quit: Leaving]
ronan has quit [Quit: Ex-Chat]
yuyichao has joined #pypy
stevi3 has quit [Quit: Connection closed for inactivity]
<o11c> jiffe: cffi doesn't support #include at all?
<o11c> jiffe: note that set_source's arguments just get passed straight to distutils
<jiffe> yeah set_source seemed to work, it was cdef which didn't like #includes, there was some code in here to remove them as the #includes that were in there before apparently weren't necessary
<jiffe> I reorganized things to make it work
<jiffe> I'm still not really sure how this is working, the cdef content is basically the header file without the includes and then some extern "Python"'s appended to it, but the content of the structs in the definition is dependent on stdint
<o11c> what don't you understand about that?
<o11c> cffi knows about stdint's types directly
<jiffe> ok, that explains it then
<o11c> and in the next release (hopefully) it will add char16_t and char32_t
<o11c> also, it turns out that already-processed file/line directives *are* supported
<o11c> so if you're reading the header from a file, you can just tack one before it and make all the errors refer to that file
<o11c> # 1 "file.h"
cstratak has joined #pypy
mihaid has left #pypy [#pypy]
<jiffe> oh interesting
<arigato> o11c: I think ``#line 1 "file.h"`` may also work, didn't try
realitix has quit [Ping timeout: 240 seconds]
ronan has joined #pypy
marky1991 has joined #pypy
pilne has joined #pypy
<mattip> arigato: if you want, I could take a look at getting stderr ouput to show up somewhere
<mattip> in cffi, issue 316 windows-embedding-show-init-time-errors
rmesta has joined #pypy
rmesta has left #pypy [#pypy]
<mattip> an example to start with that demonstrates the issue would help
antocuni has quit [Ping timeout: 264 seconds]
<kenaan> fijal str-measure 3b8e8de4a414 /pypy/: use a different id
vkirilichev has joined #pypy
amaury has joined #pypy
amaury has quit [Client Quit]
amaury has joined #pypy
mvantellingen has quit [Ping timeout: 240 seconds]
<kenaan> rlamy PyBuffer-backport 4c49b02f7487 /pypy/module/micronumpy/: Update micronumpy
amaury has quit [Ping timeout: 258 seconds]
mvantellingen has joined #pypy
<kenaan> rlamy PyBuffer-backport ed953d9e05ef /pypy/objspace/std/memoryobject.py: Add missing W_MemoryView.copy()
<bbot2> Started: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/4599 [ronan: force build, PyBuffer-backport]
<bbot2> Started: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/5923 [ronan: force build, PyBuffer-backport]
<arigato> mattip: thanks, I could also do it because I have a concrete use case, but:
<arigato> in the cffi repo, demo/extern_python.py
<arigato> call only the first part (until ffi.compile())
<arigato> ah no
<arigato> run it all, but commenting out @ffi.def_extern() def f(): ...
<arigato> it should print an error to stderr
* mattip trying
Rhy0lite has quit [Quit: Leaving]
<mattip> arigato: sorry, I'm probably just adding noise. It prints to the console nicely for me (in cpython)
<mattip> cffi.__version__ == 1.10.0
stevi3 has joined #pypy
<mattip> maybe later
mattip has left #pypy ["bye"]
<arigato> mattip (for the logs): yes, that's the point: when there is a console it works nicely
<arigato> when you are instead in a non-console program, then this goes nowhere
<arigato> and there is no way I can find to start a random program with a console in the first place
<arigato> (though that would be a hack, likely)
* arigato off
arigato has quit [Read error: Connection reset by peer]
arigato has joined #pypy
<arigato> mattip (logs): sorry, some confusion: it is actually demo/embedding*
<arigato> embedding.py and others
<arigato> e.g. if you add a syntax error inside the Python code in the string given to ffibuilder.embedding_init_code()
<arigato> then you get an error report to stderr when running embedding_test.c
<arigato> the point is that if you compile embedding_test.c as a non-console application on Windows, it goes nowhere
<o11c> arigato: cmd /c ?
<o11c> arigato: but yeah, there's a reason that everybody redirects stdout/stderr to files on windows
<jkbbwr> id love to know what im doing wrong to mess up the performance of this.
kipras`away is now known as kipras
<arigato> o11c: doesn't work, at least in this specific example
<arigato> "cmd /c program.exe" remains running, but stderr doesn't go to the console anyway
arigato has quit [Quit: Leaving]
reaperhulk_ has joined #pypy
reaperhulk_ has quit [Quit: ZNC - http://znc.in]
<bbot2> Failure: http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/4599 [ronan: force build, PyBuffer-backport]
reaperhulk_ has joined #pypy
<bbot2> Failure: http://buildbot.pypy.org/builders/own-linux-x86-64/builds/5923 [ronan: force build, PyBuffer-backport]
reaperhulk_ has quit [Remote host closed the connection]
reaperhulk_ has joined #pypy
jamesaxl has quit [Quit: WeeChat 1.7.1]
reaperhulk has quit [Quit: ZNC - http://znc.in]
reaperhulk_ is now known as reaperhulk
reaperhulk has quit [Client Quit]
reaperhulk has joined #pypy
Tiberium has quit [Quit: Leaving]
marr has joined #pypy
Tiberium has joined #pypy
Tiberium has quit [Remote host closed the connection]
ramonvg has joined #pypy
Tiberium has joined #pypy
Tiberium has quit [Remote host closed the connection]
ramonvg has quit [Remote host closed the connection]
cstratak has quit [Ping timeout: 246 seconds]
<jiffe> wrt https://pb.jiffe.com/FXBx1bhqZiebXAs1Z4yk4y any idea why the first one prints test2 and the second one prints test1?
cstratak has joined #pypy
amaury has joined #pypy
cstratak has quit [Ping timeout: 260 seconds]
Taggnostr has quit [Ping timeout: 240 seconds]
nimaje1 has joined #pypy
nimaje1 is now known as nimaje
nimaje is now known as Guest58023
<o11c> jiffe: the first one is UB
<o11c> jiffe: as soon as the object returned by ffi.new goes out of scope, it gets freed
<o11c> jiffe: the docs have a whole section on various ways to avoid this
Taggnostr has joined #pypy
<cwillu_at_work> quibble: /it gets freed/it may get freed/
jcea1 has joined #pypy
jcea has quit [Read error: Connection reset by peer]
jcea1 is now known as jcea
antocuni has joined #pypy
vkirilichev has quit [Remote host closed the connection]
<kenaan> antocuni faster-rstruct-2 a99c31346a82 /rpython/rlib/rstruct/test/test_pack.py: add direct tests for the various pack() functions in rstruct
<kenaan> antocuni faster-rstruct-2 6a63a61cf847 /rpython/rlib/rstruct/: add more tests
<kenaan> antocuni faster-rstruct-2 b6e312499e57 /rpython/rlib/: WIP: break the world, and start to refactor pack_* to build the result inside a MutableStringBuffer ins...
<kenaan> antocuni faster-rstruct-2 addb00e213ad /rpython/rlib/rstruct/standardfmttable.py: fix pack_int for the new interface
<kenaan> antocuni faster-rstruct-2 f3fc7a2efd43 /rpython/rlib/: migrate pack_float to the new buffer-based interface
<kenaan> antocuni faster-rstruct-2 bec494d81ed2 /rpython/rlib/rstruct/: fix pack_bool and pack_string
<kenaan> antocuni faster-rstruct-2 574964807a61 /rpython/rlib/rstruct/: fix pack_pad
<kenaan> antocuni faster-rstruct-2 4c598c7f0b3e /rpython/rlib/rstruct/: fix pack_pascal
<kenaan> antocuni faster-rstruct-2 6d4406afffe5 /rpython/rlib/rstruct/: test&fix native's pack_{float,double}. In the meantime, reduce some code duplication between nativefmtt...
<kenaan> antocuni faster-rstruct-2 b1e1e6b5e8bd /: adapt pypy/module/struct to the new interface for packing, which means to use MutableStringBuffer inste...
<kenaan> antocuni faster-rstruct-2 db941580b490 /rpython/rlib/rstruct/: rpython fixes
<kenaan> antocuni faster-rstruct-2 9142d98549d9 /pypy/objspace/std/marshal_impl.py: fix marshal
vkirilichev has joined #pypy
vkirilichev has quit [Ping timeout: 272 seconds]