<njs>
I spent a while trying to convince armin to do it, but he eventually convinced me that it's not really doable in a useful way :-)
<njs>
("shared JIT caches" seems like it would be great, but remember functions and their namespaces are mutable, so have to be separate objects in the different subinterpreters, so sharing is not at all trivial)
<njs>
I'm pretty dubious about PEP 554 in cpython too actually
Ai9zO5AP has quit [Read error: Connection reset by peer]
Ai9zO5AP has joined #pypy
i9zO5AP has joined #pypy
themsay has joined #pypy
Ai9zO5AP has quit [Ping timeout: 244 seconds]
Ai9zO5AP has joined #pypy
i9zO5AP has quit [Ping timeout: 272 seconds]
themsay has quit [Ping timeout: 240 seconds]
iomari has joined #pypy
moei has joined #pypy
yaewa has quit [Ping timeout: 250 seconds]
samth has quit [Read error: Connection reset by peer]
avakdh has quit [Read error: Connection reset by peer]
samth has joined #pypy
avakdh has joined #pypy
Hodgestar has quit [Remote host closed the connection]
iomari has quit [Read error: Connection reset by peer]
Hodgestar has joined #pypy
themsay has joined #pypy
Zaab1t has joined #pypy
themsay has quit [Ping timeout: 244 seconds]
<arigato>
snow on the ground! (2 centimeters of it)
<arigato>
_aegis_: indeed it's a hard problem
<arigato>
it might not be any easier than the problem of saving JIT code to disk and reloading it the next time
k1nd0f has joined #pypy
k1nd0f has quit [Ping timeout: 258 seconds]
<arigato>
a vague idea about cffi: it has always been slightly strange to write p=ffi.new("mystruct *") to get one mystruct allocated
<arigato>
even though p is really a type "mystruct *", i.e. just a pointer
<arigato>
the vague idea would be to use s=ffi.new("mystruct")
<arigato>
and s is a cdata of the type "mystruct"
<arigato>
you can then write p=ffi.addressof(s) if you need a pointer to it
<arigato>
if I ever implement that, we could then deprecate all ffi.new("T*")
<arigato>
while still not supporting stuff like ffi.new("int")
<arigato>
you'd just need ffi.new("int[1]"), which is kind of sane anyway
<arigato>
(there is no reasonable syntactic way to have ffi.new("int") return a cdata that can be written to)
Zaab1t has quit [Ping timeout: 245 seconds]
iomari has joined #pypy
_whitelogger has joined #pypy
Zaab1t has joined #pypy
dddddd has joined #pypy
<_aegis_>
ok, is subinterpreters without sharing any state at all feasible in pypy?
<_aegis_>
does rpython keep interpreter specific data in the .data segment?
<_aegis_>
s/keep/write to/
<_aegis_>
what's wrong with ffi.new('int').value = 1?
<_aegis_>
it's kinda nice that the current T* system is consistent
<_aegis_>
structs have value assignment the same as ints now anyway
<_aegis_>
s[0] = (1,2,3,4) is a handy initializer in some cases
jcea has joined #pypy
xcm has quit [Read error: Connection reset by peer]
xcm has joined #pypy
Zaab1t has quit [Ping timeout: 250 seconds]
adamholmberg has joined #pypy
danchr_ is now known as danchr
dmalcolm has joined #pypy
<arigato>
data in the .data segment: yes, there is tons of it, but it would be possible to fix that relatively easily. there is little point though
<arigato>
well, or there is, but limited to running things in the same process instead of in different processes, with no size advantage at all
<arigato>
you might as well load libpypy-c.so several times in the same process (using linux-specific hacks based on dlopen() with LOCAL flags)
<arigato>
_aegis_: re ffi.new, I see some confusion (sorry about that)
<arigato>
I used the name 's' above for <cdata 'mystruct'>, by opposition to 'p' which is <cdata 'mystruct *'>
<arigato>
so when you talk about value assignment via "p[0] = ...", it's always via a pointer
<arigato>
finally, my proposal was to make ffi.new more consistent, not less so: ffi.new("T[1]") allocates a T[1], but ffi.new("T*") allocates a T (not a T*)
Rhy0lite has joined #pypy
Zaab1t has joined #pypy
marky1991 has joined #pypy
k1nd0f has joined #pypy
k1nd0f has quit [Ping timeout: 245 seconds]
k1nd0f has joined #pypy
k1nd0f has quit [Ping timeout: 250 seconds]
<_aegis_>
but there's no difference between an array of unknown size and a pointer in C?
<_aegis_>
I think ffi.new('int *') makes sense to me. if I want to pass an int value to a function I use a python int, if I want to pass a pointer to an allocated int I use new('int *')
<_aegis_>
addressof everywhere would be less elegent imo
<_aegis_>
so is the only simple benefit to be gained by sharing two pypy interpreter address spaces shared memory for JIT cache if you have a more abstract JIT serialization?
<_aegis_>
I think that would also have a possible CPU benefit if done right since you could skip a lot of warmup and import work
<_aegis_>
you could argue that there's no difference between this and some kind of AOT compilation but I think there's probably a memory savings in there somewhere if you don't leave everything fully jitted in memory?
<_aegis_>
if cache->asm is much faster than bytecode->asm, you could collect interpreter-specific jit blocks
<_aegis_>
there might also be an icache benefit if the threads share an interpreter loop? I'm not sure how icache works with virtual memory on different architectures
NemeXis has joined #pypy
<_aegis_>
the reason I'm thinking about this is there's no way I'm going to spawn a billion processes but I might be convinced to do a thread pool with a couple of interpreters pinned for latency
<_aegis_>
my main alternative is letting people write luajit kernels that get their own thread if they care about consistency but that makes me sadder
<_aegis_>
(also I'm shipping a cross platform desktop app, so linux specific hacks are no good)
marky1991 has quit [Ping timeout: 268 seconds]
<_aegis_>
hmm a lot of what I want would be satisfied by some kind of sandboxed actor that can't even import
<_aegis_>
(in general I would love better actor model in pypy even if it didn't involve multiple interpreters)
<ronan>
mattip, I'm investigating the html5lib slowdown, the main culprit seems to be html5lib._inputstream.HTMLUnicodeInputStream.charsUntil()
<nimaje>
_aegis_: you can't have a array of unknown size in C, you can have a pointer to the first element or a memory region, where you use pointer arithmetic and pretend it is an array
<ronan>
mattip, I guess re.match() is the problem, because it converts to RPython unicode
<_aegis_>
but at the ffi layer, all arrays are unknown size in C
<_aegis_>
(because they're pointers)
<_aegis_>
so if I make p = int[1] and int *p = malloc(sizeof(int)) and pass them to a function both can be represented as int *
<mdash>
_aegis_: python just not a real good fit for the multiple-sandboxes-in-single-process model sadly :-/
<_aegis_>
I mostly only want sandboxing because I know that solves the GIL sans internal interpreter state
<_aegis_>
(if it can't get a reference to any other memory it doesn't need a lock)
<_aegis_>
which is similar to multi-interpreter, except I don't actually care about having all features of python in the actor
<_aegis_>
at this point I've taken asking about a 3.8 PEP to "wonder if I could do this some other way in pypy without breaking anything"
<_aegis_>
(I'm super ok with breaking cpython compatibility too)
<kanaka>
Any suggestion on how to do the equivalent of float.fromhex() in rpython? E.g. any pure python implementation somebody could point me at?
<_aegis_>
isn't pypy's implementation of it in python?
<_aegis_>
probably a good time for me to check up on PyThreadState_SetAsyncExc
<ronan>
mattip: hmm, 4ef833b2310d needs to be reverted and interp_sre.py fixed properly
<ronan>
kanaka: PyPy's implementation is W_FloatObject.descr_fromhex() in pypy/objspace/std/floatobject.py
<ronan>
the core implementation is mixed with PyPy-specific stuff, but that should be a good base
kipras has joined #pypy
C0nundrum has joined #pypy
<C0nundrum>
Anyone why cqlsh doesn't work properly when installed via pip but works properly when installed via brew even though pip has a higher version?
marky1991 has joined #pypy
<_aegis_>
what do you mean "doesn't work"
<_aegis_>
that's a very abstract notion
<C0nundrum>
When i use the pip version of 5.x i get `Error returned from server while using explicitly set client protocol_version 4` even if i pass --cqlversion. However when i install cassandra via brew which installs cqlsh 5.0.1 it works properly
NemeXis has quit [Remote host closed the connection]
<LarstiQ>
C0nundrum: is this a pypy question?
Taggnostr has quit [Quit: No Ping reply in 180 seconds.]
Taggnostr has joined #pypy
marky1991_2 has joined #pypy
marky1991 has quit [Ping timeout: 258 seconds]
themsay has joined #pypy
marky1991_2 has quit [Ping timeout: 250 seconds]
<kanaka>
ronan: I'm trying to invoke descr_fromhex and trying to figure out the parameters. I think space can by std.objspace.StdObjSpace(), but what is w_cls? float was my guess but that gives "type object 'float' has no attribute 'getclass'".
marky1991_2 has joined #pypy
<ronan>
kanaka: w_cls is a wrapped class object, representing float or a subclass of it. It's only used at the very end, though, to wrap the computed RPython-level float. I guess you should just remove the last 2 lines and 'return sign * value' instead
Rhy0lite has quit [Quit: Leaving]
Zaab1t has quit [Quit: bye bye friends]
C0nundrum has quit [Ping timeout: 256 seconds]
marky1991_2 has quit [Read error: Connection reset by peer]
adamholmberg has quit [Remote host closed the connection]