07:47 <profetes> hi Sean, I know there are no outboxes, so i put the word in quotes, I wanted to indicate that someone might have gotten message from such actor recently. I think GC should free LetterFilter after it run out of scope in https://github.com/plprofetes/advent_of_code/blob/master/2018/aoc5/lib/lib.pony#L591, because there's no explicit reference to it. LetterFilter is conceptually my unit of work.

07:52 <profetes> And "out of scope" I mean when those actors cease to work (indicated logically by promise `job` being fulfilled). All references to that list of 50k actors are inside that LetterFilter. The only remaining thing out of `be go(...)` are promisese: p (to collect results from all the letter filters) and job (that triggers another go() on being accepted https://github.com/plprofetes/advent_of_code/blob/master/2018/aoc5/lib/lib.pony#L

07:52 <profetes> 536)

08:25 <vaninwagen> profetes: what i can see by looking at your code, is that the Units, the LetterFilter and the ReactionWatcher are forming a pretty dense circle (referencing each other)

08:26 <vaninwagen> there is a lot going on in there

08:27 <profetes> true, but disjoint from one another. and they stop sending messages at some point. and hence I expect that those disjoint, "reacted", not active groups can be GCed

08:28 <vaninwagen> i didn't mean this as bad.

08:28 <vaninwagen> true, they should be

08:30 <profetes> I admit I complicated this on purpose :) I wanna test Pony before I can use it on my production - and data structures consisting of actors (like graphs) must be reliable.

08:33 <profetes> I just dont know how to debug that, so i know if it's a bug. If so - I can create more, more minimal case and file an issue. Can I debug GC some how? Can I force GC to free something?

08:34 <vaninwagen> you cannot force it to free something in particular afaik

08:37 <vaninwagen> be aware that each promise is also an actor

08:37 <profetes> okay, maybe can I access reference counters, used by ORCA, from Pony?

08:38 <vaninwagen> this i don't know tbh

08:38 <vaninwagen> it would be nice if we'd have some GC stats available

08:39 <vaninwagen> i think there are some dtrace probes around to help you see sth. slfritchie might know a little more about that

08:39 <profetes> Promises are my suspects for blocking GC. They should not be, because promise gets accepted and actor accepting it can go away - message is delivered to Promise and it will be consumed some time later. It's like acyclic graph. You cannot go back, so GC might be able to figure that out.

08:42 <profetes> There's rarely a situation when actor cooperating with other actors loose all the references to them. Rather - they stop to exchange messages. GCing actors is far more complex that GCing objects :)

08:43 <vaninwagen> indeed

08:48 <vaninwagen> damn, no dtrace for me :(

08:49 <profetes> Microsoft Orleans do that safely - they serialise actors in some DB and sweep actors out of active memory, freeing it. If anyone ever needs that actor again - it's restored from DB. In Pony one would never resurrect GCed actor, so it's *reasonable* to prevent actors from GCing as long as possible.

08:50 <profetes> I'm in an even worse situation - Docker only here ;)

08:50 <vaninwagen> can you ensure, you have no ping-pong messaging around in your actor-circles?

08:54 <profetes> 99.8% since I can pass Result fully through only if all items in a list are stable. And then I do nothing. And list does nothing.

08:54 <vaninwagen> aye

08:55 <profetes> and when i just run part1() program ends normally, after passing Result has succeeded. No delay. If there was something lingering - I presume runtime would prevent program from exiting.

08:57 <profetes> And _final() for actors are called before the very end

09:01 <vaninwagen> the runtime is actually finalizing all actors on shutdown

09:02 <vaninwagen> in https://github.com/ponylang/ponyc/blob/master/src/libponyrt/gc/cycle.c#L674

09:03 <vaninwagen> at least that's how i read it

09:03 <vaninwagen> runtime is not my field at all :/

09:05 <profetes> Thanks for being helpful - look at that, just a few lines down: https://github.com/ponylang/ponyc/blob/master/src/libponyrt/gc/cycle.c#L745

09:07 <profetes> I wanna call that :) now I need a way to call that

09:08 <vaninwagen> you'd need to make that not static and then use ffi

09:08 <profetes> and have pony runtime in debug build

09:14 <profetes> another idea: since CPU is starved there may not be enough time for GC to find the cycles: https://github.com/ponylang/ponyc/blob/e26540ec7021977eb303686adc1ca79be43a6ddf/src/libponyrt/gc/cycle.c#L909

09:14 <profetes> because disjoint part if 50k+ actors is too big. Does that sound probable?

09:16 <vaninwagen> yeah, that could be

09:20 <profetes> but I might be reading that wrong, cycle detector is spawned every 10..1000ms, but it's lifetime is not limited. So it might create too many of them? And stress the runtime even more?

09:23 <profetes> I dont know, I'll wait for someone with more expertise in that matter. I'll try to extract minimal case in the evening for easier debugging.

09:24 <profetes> vaninwagen: thank you for your continuous support!

09:24 <vaninwagen> thanks for your stresstest :)

11:32 <xcombelle> hi

11:51 <vaninwagen> hi. what's up?

12:54 <xcombelle> still like pony actors

13:00 <vaninwagen> yes, good ensemble