<Alex_Gaynor>
arigato: do you have a clue about the GC+lare amounts of __del__ bug?
<arigato>
larGe?
<Alex_Gaynor>
yes
<arigato>
not so far, no
<arigato>
though I guess we can find out the reason just by reasoning
<arigato>
something like: say you have N megabytes of objects with __del__ and you run a GC
<arigato>
immediately afterwards, the N megabytes are still there, and the dels will be called soon, and the N megabytes will only disappear at the next GC
<arigato>
at the next GC, there are 2*N megabytes of objects alive
<arigato>
though only N survive
<Alex_Gaynor>
this description seems like it'd leave us with a steady state of 2N memory usage, when really we see unbounded growth
<arigato>
yes, that's also the conclusion I've reached now
<arigato>
something is still unclear, I need to be more awake I guess
<Alex_Gaynor>
Hehe, more coffee :-)
<Alex_Gaynor>
Err, I guess it's the evening for you, not the morning.
<arigato>
yup
<Alex_Gaynor>
Do objects with finalizers go to the nursery, or directly to the older genration?
<arigato>
the problem might be that, the less often we run major GCs, the largest the amount of surviving objects; and the largest the amount of surviving objects, the less often we run the GC
<arigato>
they go directly to the old generation, because they will always survive the next major GC anyway
<arigato>
s/largest/larger
<Alex_Gaynor>
if I run my program with `PYPYLOG=gc:-` I see lots of gc-minor, but no gc-major
<Alex_Gaynor>
also gc-collect-step
<Alex_Gaynor>
it's almost entirely gc-minor though
<arigato>
gc-collect-step is a major GC (one incremental step)
<Alex_Gaynor>
Ok. They seem rare, perhaps they just can't keep up?
<arigato>
yes, as expected, for some reason the major GCs don't run often enough and memory grows (it may be the cause or just a symptom)
<Alex_Gaynor>
Can you think of an easy example class with a light finalizer, I want to verify if it affects them
<arigato>
files, I think
<arigato>
...no
asmeurer_ has joined #pypy
<Alex_Gaynor>
array.array
<arigato>
yes
<Alex_Gaynor>
ok, light finalizers are fine
Ryanar has joined #pypy
asmeurer_ has quit [Ping timeout: 260 seconds]
rokujyouhitoma has joined #pypy
egregius313 has joined #pypy
vkirilichev has quit [Remote host closed the connection]
rokujyouhitoma has quit [Ping timeout: 258 seconds]
egregius313 has quit [Ping timeout: 246 seconds]
<cfbolz>
Alex_Gaynor: I suspect because they don't get resurrected, so they don't make the major GC rarer
jamesaxl has quit [Quit: WeeChat 1.7.1]
<cfbolz>
arigato: I suspects the objects with __del__ that survive another round to get their finalizer called shouldn't count towards one of the limits, or at least shouldn't lead to increased heap sizes
<arigato>
cfbolz: likely, but I'd like to understand exactly why
<cfbolz>
right
<cfbolz>
randomly changing some of the details seems not great
<arigato>
ah, I may get it
<arigato>
say we run one major GC after the memory usage gets to 2x what it was at the previous GC
<arigato>
say there are 1MB of regular long-lived objects around
<arigato>
add 1MB of objects with dels
<arigato>
gc, there are now 2MB of objects
<arigato>
next limit is thus 4MB
<arigato>
at next gc, there are thus 2MB of objects with del
<arigato>
so 3MB of surviving objects
<arigato>
next limit is 6
<arigato>
so there are 3MB of new objects with del
<arigato>
4MB survive
<arigato>
etc.
<Alex_Gaynor>
so we have fibonacci growth :-)
<arigato>
no, I think it's linear
<cfbolz>
arigato: we should be able to write a direct test for that?
<arigato>
the limit grows by 1MB at every major GC
<arigato>
(or no, the limit grows by 2MB, and the size immediately after GC grows by 1MB)
tbodt has joined #pypy
<arigato>
cfbolz: I think so, but /me -> away
arigo has joined #pypy
<Cheery>
I noticed myself worrying about dictionary/object access.
tilgovi has joined #pypy
arigo has quit [Client Quit]
<Cheery>
so I guess it'd be nice time to get introduced to what kind of optimizations can I get on dicts?
<cfbolz>
Cheery: mostly dict strategies
<cfbolz>
but you need to implement those yourself
arigato has quit [Ping timeout: 255 seconds]
ronan has quit [Ping timeout: 276 seconds]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<kenaan>
rlamy py3.5 9068d4152e3f /pypy/module/array/: Make slices of array memoryviews usable as writable buffers if contiguous.
ronan has joined #pypy
arigato has joined #pypy
tilgovi has quit [Ping timeout: 276 seconds]
<arigato>
Cheery: if you're talking about the JIT, there are a few optimizations that you get---but the easiest is to try it out by looking at, and trying out more examples inspired from, jit/metainterp/test/test_dict.py
oberstet has quit [Ping timeout: 255 seconds]
asmeurer_ has joined #pypy
<cfbolz>
arigato: I might try to write a test. would write a direct test? or an lltyped one?
asmeurer_ has quit [Ping timeout: 255 seconds]
<arigato>
probably lltyped, a direct test is messy to write with finalizers
rokujyouhitoma has joined #pypy
<cfbolz>
yes, seems there is no such thing atm
ronan has quit [Quit: Ex-Chat]
ronan has joined #pypy
rokujyouhitoma has quit [Ping timeout: 255 seconds]
tbodt has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<ELFrederich>
arigato: so if I have in my header file "typedef unsigned int tag_t;" can I do "typedef ... tag_t;"? or do I have to put "int" somewhere in there?
<arigato>
ELFrederich: it's explained on that link
<arigato>
see the last paragraph
<ELFrederich>
arigato: now I'm more confused ;-)
<ELFrederich>
first sentence: Currently, it is not supported to find automatically which of the various integer or float types you need at which place.
<ELFrederich>
second sentence: If a type is named, and an integer type, then use typedef int... the_type_name;.
<arigato>
right, I should add "except in this case:" between the two sentences
<ELFrederich>
so at first it says it's not supported to automatically figure out the various integer types... then it says if your type is one of the various integer types you can use int...
<ELFrederich>
arigato: still confused, even if you added "except in this case:"... when would a type not be named?
jamesaxl has joined #pypy
pilne has joined #pypy
asmeurer_ has joined #pypy
black_ant has quit [Ping timeout: 260 seconds]
<kenaan>
rlamy py3.5 f2b370578863 /: Use correct implementation for Decimal.compare_total{,_mag}(). Fixes #2586 and #2587
rokujyouhitoma has joined #pypy
black_ant has joined #pypy
adamholmberg has quit [Ping timeout: 268 seconds]
rokujyouhitoma has quit [Ping timeout: 260 seconds]
<kenaan>
arigo cffi/cffi 0c3756872e68 /doc/source/cdef.rst: Expand this paragraph
black_ant has quit [Ping timeout: 260 seconds]
<arigato>
ELFrederich: a "named" type is one that you name with typedef. An "unnamed" type is, well, all other cases
<arigato>
e.g. the type of an argument, a return type, the type T in "typedef T *foo;" etc.
<arigato>
so, e.g., int[3] or void(*)(int, int)
jamesaxl has quit [Read error: Connection reset by peer]
jamesaxl has joined #pypy
adamholmberg has joined #pypy
ELFrederich has quit [Quit: Page closed]
adamholm_ has joined #pypy
ronan has quit [Ping timeout: 260 seconds]
adamholmberg has quit [Ping timeout: 255 seconds]
Ryanar has quit [Quit: Ryanar]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
rokujyouhitoma has joined #pypy
jcea has joined #pypy
rokujyouhitoma has quit [Ping timeout: 246 seconds]
arigato has quit [Read error: Connection reset by peer]
lritter has quit [Remote host closed the connection]
Ryanar has joined #pypy
kipras`away is now known as kipras
egregius313 has joined #pypy
rokujyouhitoma has joined #pypy
egregius313 has quit [Ping timeout: 246 seconds]
rokujyouhitoma has quit [Ping timeout: 260 seconds]
jcea has quit [Quit: jcea]
rokujyouhitoma has joined #pypy
rokujyouhitoma has quit [Ping timeout: 255 seconds]
tbodt has quit [Read error: Connection reset by peer]
tbodt has joined #pypy
jamesaxl has quit [Quit: WeeChat 1.7.1]
tbodt has quit [Ping timeout: 240 seconds]
adamholm_ has quit [Remote host closed the connection]
adamholmberg has joined #pypy
tbodt has joined #pypy
adamholmberg has quit [Ping timeout: 260 seconds]
marky1991 has quit [Read error: Connection reset by peer]
Ryanar has quit [Quit: Ryanar]
<Cheery>
I guess I found a bug that was caused by bad coding on my part, and an update in PyPy