<codec_>
the documentation says that apply is a behaviour and as such it should return immediatelly (this is what is written in the tutorial)
dipin has quit [Quit: dipin]
<codec_>
at this point the create of the main actor is done and so in a "traditional" program it would terminate and the remaining threads would be cleaned up by the OS
<SeanTAllen>
pony handles that for you
<SeanTAllen>
when no more work can be done
<SeanTAllen>
the actors will shut down
<codec_>
and if works remain it stays alive?
<SeanTAllen>
what is "it"?
<SeanTAllen>
an actor is kept along so long as
<SeanTAllen>
it has messages in its mailbox or it is able to receive messages
<SeanTAllen>
to receive messages, something has a to have a reference to you
<SeanTAllen>
if nothing does, then the actor can be gc'd
<SeanTAllen>
cycles could exist though
<SeanTAllen>
actor A refers to actor B and vice-versa
<SeanTAllen>
however if nothing refers to either
<SeanTAllen>
and neither is registered for asio events
<codec_>
ok thanks
<SeanTAllen>
then they can be shutdown as being a cycle if neither has any work to do
<codec_>
what are asio events?
<SeanTAllen>
asynchronous io events
<SeanTAllen>
for example, receiving messages over the network
<SeanTAllen>
timers use asio as well
<SeanTAllen>
a tcp connection actor uses asio
<SeanTAllen>
so
<SeanTAllen>
it wont shut down on its own
<SeanTAllen>
you have to explicitly close it at which point it closes down its registered events and can then be gc'd
jemc has joined #ponylang
codec_ has quit [Ping timeout: 260 seconds]
<Candle>
SeanTAllen: So anything in a disconnected cycle and nothing in that cycle can receive events, then it is eligable for GC?
Zalastax has joined #ponylang
bimawa1 has quit [Read error: Connection reset by peer]
bimawa1 has joined #ponylang
endforma1 has joined #ponylang
<SeanTAllen>
Candle: ya
<SeanTAllen>
thats the basic rule Candle
<SeanTAllen>
its a little more nuanced
<SeanTAllen>
and should do "what you expect"
<Candle>
Makes sense.
<Zalastax>
Hi! I'm a masters student at Chalmers looking for a thesis project for this spring. I'm interested in improving the safety of distributed systems via type systems. Pony, CloudHaskell, and http://abs-models.org/ all look like interesting areas and it would be great to know what's going on in the Pony community
<jemc>
I should also let you know that we have a weekly sync call that is open to the public if you ever want to discuss something in realtime or bring something to our attention
<Zalastax>
Pony doesn't have distributed actors yet, am I right?
<Zalastax>
This looks very interesting, especially Blessing's paper. Thank you!
<Zalastax>
Just to be clear about the type system, livelocks are still possible? And there are no proofs or dependent types in the language which could model progress?
Praetonus has quit [Quit: Leaving]
<Zalastax>
I didn't see message passing so it might not be a problem yet.
<Zalastax>
Or rather, I didn't see it in the same sense as in Erlang, using ! and receive
<SeanTAllen>
Zalastax: there's no selective receive in Pony so although it looks like Erlang if you quint right, the semantics are very different
<SeanTAllen>
or put another, its wise to not assume that the erlang way is how it works in pony
<SeanTAllen>
or put another way, i am affirming what you said
<slfritchie>
In Pony source when defining an actor's methods, there's a difference between the keyword `fun` and `be`. The `be` are message sending (by the caller) and receiving (by the ... errr ... receiving/target/destination actor).
<slfritchie>
The Aardvark example in https://tutorial.ponylang.org/types/actors.html doesn't have any `fun` functions, but an actor that calls a `fun` does it synchronously in its execution context (see the "Sequential" in that same web page).
<Zalastax>
Thanks! I need to think about how that affects expressability. ABS is the same way as Pony, while CloudHaskell uses Send and Receive. For Erlang send and receive makes it so that a system can reconfigure itself and decide on its own what the meaning of a message should be. In Pony an actor seems to be a lot more strict as far as I can see
<Zalastax>
Modelling completely dynamic systems like Erlang isn't easy though, so I can see why send and receive are not used in Pony
<SeanTAllen>
send/receive wouldn't work with Pony's type system or rather, you can do it but your lose the advantages of the type system.
<slfritchie>
There isn't a direct analogue to gen_server:call() in Pony. You can fake it sort-of using a Promise, but I haven't actually tried it. (I'm still a new'ish'bie'.) https://stdlib.ponylang.org/promises--index/
<SeanTAllen>
new'ish-bie'. <-- i like that slfritchie
<slfritchie>
I've used Erlang a helluvalot longer than I've used Pony.
<Zalastax>
Yes! I think that's why ABS does it that way as well, to have useful types. To model send / receive well you'd get hellish types and probably need a dependent type checker
<slfritchie>
Specifically about 'direct analogue', I was referring to the send & receive use by gen_server:call().
<slfritchie>
Hmmm, I wonder, if I take very great liberties with the word "selective", then Pony definitely doesn't have selective receive because: 1. you can't do pattern matching on messages so you must handle messages in FIFO order, and 2. your code has no choice of *when* you can receive a message: the current `be` method must return to the runtime before the next message can be handled. (Ditto for actor garbage
<slfritchie>
collection: no GC happens while a Pony behaviour is executing.)
<SeanTAllen>
1 of those 2 things will continue to be true
dipin has joined #ponylang
<SeanTAllen>
eventually that GC behavior will change
<SeanTAllen>
the FIFO order is by design
<SeanTAllen>
the GC one is purely implementation based
<Zalastax>
Hmm, no selective receive seems very limiting. Doesn't the caller and receiver feel rather entangled then? Or would you return promises if you want out-of-order processing?
<SeanTAllen>
FIFO is incredibly useful for reasoning about the syste,
<SeanTAllen>
and results in wonderful causal messaging guarantees
<SeanTAllen>
we've taken advantage of it at Wallaroo Labs
<SeanTAllen>
wouldnt want to give that up
<jemc>
Zalastax: can you clarify "out-of-order processing"?
<Zalastax>
jemc: In Erlang an actor can decide what messages to process in its current state. Unmatched messages are put in an inbox
<Zalastax>
When it goes to the next state it can decide a new pattern to match and will take messages from the inbox
<Zalastax>
As the programmer we only need to think about what messages to process that makes sense now
<jemc>
right, yeah, that's something that would have to be implemented in user space if you wanted that behaviour
<jemc>
personally I like the Pony style a bit better because it forces me to recognize that I can receive any message at any time, and plan around that explicitly
<jemc>
with selective receive it seems very easy to accidently end up neglecting a lot of messages
<Zalastax>
@jemc: It would be great to support both modes. Pony's style is a lot easier to type-check though
<jemc>
also, I suspect that this design choice has something to do with causality guarantees
<Zalastax>
I think so to!
<jemc>
that is, with selective receive, it seems like you're given the opportunity to break causal order of messages
<jemc>
yeah
<Zalastax>
in Erlang messages can come in any order if you go outside a single machine
<Zalastax>
and you should in general think about the program in a completely different way, where you don't rely on the messages coming in a specific order. E.g. by only matching some messages when you are in a certain state
<Zalastax>
Yes, failing to match on some messages is a common problem. You learn to avoid it, but it's still a problem
<Zalastax>
Great chat! Thanks all!
Zalastax has quit [Quit: Page closed]
<SeanTAllen>
you're welcome zalastax
dipin has quit [Quit: dipin]
vaninwagen_ has quit [Ping timeout: 248 seconds]
_andre has quit [Quit: leaving]
codec_ has joined #ponylang
<codec_>
Hi
<SeanTAllen>
Hello codec_
<codec_>
sorry to bother you again, but can you answer a few questions about actors?
<SeanTAllen>
i can try
<SeanTAllen>
ask away
<codec_>
let's say that I want to process an image (basically an Array[U8]) and want to do in with multiple threads to get better performances
<codec_>
how can I do that with Pony?
<SeanTAllen>
thats a hard question to answer
<SeanTAllen>
how would you do it with multiple threads?
<codec_>
you mean with another language?
<SeanTAllen>
yes
<jemc>
probably split the `Array[U8]` into N slices, send each slice to an actor, then have the actors send the results of each slice back to a ResultCollector actor
<jemc>
assuming you can split into slices that can be processed independently
<SeanTAllen>
ya
<SeanTAllen>
thats the part im not sure about
<jemc>
yeah
<SeanTAllen>
i didnt want to assume
<codec_>
I have no real case in mind but for the example we can assume that you can indeed slices the image in smaller part
<codec_>
about perfomance won't we suffers from sending the result back to a ResultCollector?
<SeanTAllen>
im not sure why you say that
<SeanTAllen>
suffer compared to what?
<codec_>
to an implementation in a language with threads
<codec_>
basically in C/C++ or C# I would do it this way
<SeanTAllen>
why do you think the performance would suffer?
<codec_>
ain't each byte of the result would end up being copied ?
<SeanTAllen>
no
<SeanTAllen>
erlang copies messages when sending from one actor to another
<SeanTAllen>
pony has reference capabilities that allow you to safely send data from one actor to another without copying
<codec_>
even for primitive type like u8?
<jemc>
the only time you copy byte buffers in Pony is when you call `clone` on the `Array[U8]`, or one of the other `Array` methods that is documented as creating a copy
<codec_>
I mean isn't faster to copy a byte that sending its adress?
<jemc>
codec_: we're not talking about single bytes here, we're talking about byte arrays
<codec_>
I see, so each actor need to process a chunck big enough to make it worthwhile right?
<SeanTAllen>
machine words like U8 are different than objects like Array[U8]
<SeanTAllen>
make what worthwhile codec_ ? the message send?
<codec_>
yeah
<SeanTAllen>
well
<SeanTAllen>
sure
<SeanTAllen>
but
<SeanTAllen>
in c++, have you already started your threads?
<SeanTAllen>
if not, that is more expensive than a pony message send
<codec_>
because I could create an actor for each pixel but it would be probably less eficient than even a single threaded version
<SeanTAllen>
maybe
<SeanTAllen>
this is all very theoretical
<codec_>
well usually I would use a pool
<SeanTAllen>
ok
<SeanTAllen>
but
<SeanTAllen>
it depends
<SeanTAllen>
is really the answer to almost any question like this
<codec_>
on windows there is a even a nice library that allow to do parallel for
<SeanTAllen>
i can answer things like
<SeanTAllen>
"does pony copy data for message sends"
<SeanTAllen>
thats an answerable question
<codec_>
yes my questions probably look dumb but the change of paradigm is not easy for me
<SeanTAllen>
i cant speculate on theoretical performance of pony message passing vs a nice windows library i know noting about
<SeanTAllen>
i can talk about how pony does things though
<SeanTAllen>
happy to as well
<codec_>
no that is not what I asked
<SeanTAllen>
so
<SeanTAllen>
pony messages sends are quite fast
<codec_>
I just wanted to know how you would have done it
<SeanTAllen>
there's an overhead for gc messages when sending objects from 1 to another
<SeanTAllen>
well
<SeanTAllen>
this is how i approach such problems
<SeanTAllen>
i would write the single actor version
<SeanTAllen>
see how it performs
<SeanTAllen>
if its good enough, i would stop
<SeanTAllen>
if i needed more performance, i would try doing what jemc said with the slices
<SeanTAllen>
but performance there is going to be variable
<codec_>
surely about that
<SeanTAllen>
i wouldn't want to have more actors than there are pony scheduler threads
<codec_>
how the merge of result in the collector works?
<SeanTAllen>
then i'd test that
<SeanTAllen>
i'd probably use a Promise for that codec_
<SeanTAllen>
promises get run in an actor
<SeanTAllen>
but
<SeanTAllen>
i dont really do tasks like this a lot
<SeanTAllen>
so i dont have a good answer for you, i'd end up playing around with a few ideas
<codec_>
because it is not idiomatic in pony?
<SeanTAllen>
no
<SeanTAllen>
because its not something that i do
<codec_>
fair enough :)
<SeanTAllen>
most of my work in pony is on Wallaroo
<SeanTAllen>
and on the Pony runtime (in C)
<SeanTAllen>
i do know
<SeanTAllen>
that in any language if i was doing that
<SeanTAllen>
i would need to know what order the segments that come back should be in
<SeanTAllen>
so i could reassemble them in the correct oder
<codec_>
in pony it's true but in a more common language I would give the full output buffer to each thread
<SeanTAllen>
and what? put a lock around it?
<codec_>
and I would assume (pray ?) that it will write only the values that it is supposed to compute
<SeanTAllen>
how do you know where to write into the buffer?
<codec_>
by given each a index for example
<SeanTAllen>
ok so
<SeanTAllen>
i guess i can know from the size of my incoming slice, or where it starts where to write into a buffer
<SeanTAllen>
i suspect a lot of problems wont have that characteristics
<SeanTAllen>
my general feeling on things from lots of programming experience and many segfaults is that i shouldn't share data between threads
<codec_>
in general I agree
<SeanTAllen>
one of the things i love about pony is that i have a compiler that enforces all the things i learned through various painful experiences
<jemc>
codec_: I see what you're saying now - yeah, pony won't let you do that because you're not supposed to have to pray about how the code shouldn't misbehave with shared memory :)
<codec_>
but this kind of operations is short lived so I did it many and never had any problem with that
<jemc>
if you needed to do this, I'd suggest using FFI, and wrapping that in a nice safe object
<SeanTAllen>
when i wrote c++, i had a series of rules for myself for handling memory and concurrency and usually i got them right, but sometimes i messed up, and it was never pretty
<jemc>
with the object being designed to restrict the memory that each reference could access
<codec_>
yeah but in this case its more or less the same thing that running a loop in parallel
<codec_>
so obviously there is not a lot of room to have concurrency bug in this example
<codec_>
anyway I realize that I have quite a lot to learn about actors
<jemc>
if I were approaching this problem myself, I'd treat the images as a `Array[Array[U8]]` from the beginning and to the end
<jemc>
so that I could slice and dice without having to copy byte buffers
<codec_>
interesting
<jemc>
which is a rather common zero-copy pattern (see iovec from C)
<jemc>
sometimes called "scatter/gather"
<jemc>
and supported by `readv` and `writev` at your input and output stages
<codec_>
so if I follow you you will give each actor a piece of the "external" array so they can write their result in it?
<jemc>
well, I was thinking that each actor would return `Array[U8]` and its index, then your result collector actor would stitch them back into the right order in an `Array[Array[U8]]`
<jemc>
or alternatively, if you wanted better composability, each actor could return `Array[Array[U8]]` and those would get concatenated in the right order by the result collector
<codec_>
does it imply copying arrays when stitiching them?
<SeanTAllen>
it depends.
<SeanTAllen>
in this case, no
<codec_>
really and it would be a continuous array in memory?
<jemc>
you're copying the pointers in the top-level array to stitch them - not the byte buffer pointers in the inner array
<jemc>
and if you're following SeanTAllen's advice about not having more "parallelism pool" actors than scheduler threads, this will be just a few pointers being copied
<codec_>
thanks
<codec_>
so can you give me some pointers (no pun intented) about actors?
<codec_>
the tutorial does not speak much about it. There is a bit more in the pattern book but not so much too
<codec_>
(I mean pointers to learn more about actors)
<jemc>
it's a bit hard to find good links from other languages because the actor model implementation and semantics from different languages usually varies a bit, and the patterns/idioms vary with them
<codec_>
well I guess I will have to learn by trying then
<jemc>
I think you're right that we need more pony-specific resources on that, but it just hasn't happened yet because many people come to pony already knowing a fair bit about actors
<jemc>
I'll try to think about it a little bit as I'm preparing for the talk I'm giving at a conference in a few weeks
<jemc>
and try to come up with some examples and concepts that can be used in the talk, but also get integrated back into the pony patterns book
<codec_>
cool
jemc has quit [Read error: Connection reset by peer]
jemc has joined #ponylang
<endforma1>
SeanTAllen: I noticed you flagged 'finite recursive type aliases' for sync discussion. I'm curious, did anything come of it?
<SeanTAllen>
sylvan is supposed to be writing up what he thinks someone would need to know to implement it endforma1
<SeanTAllen>
but sylvan has been in the process of moving and his life is kind of chaotic the last couple months
<codec_>
thanks again for the answers
<endforma1>
Ah, understandable.
<SeanTAllen>
unrelated but i just discovered i have my network buffers set to high to make it easy to do backpressure testing for sockets on my machine as is
endforma1 has quit [Quit: WeeChat 1.9.1]
endformationage has joined #ponylang
<SeanTAllen>
or i need to send alot more data all at once
<SeanTAllen>
my hope endformationage is that sylvan adds enough info that some brave person takes it on
<endformationage>
Haha, nice. I decided to make use of your lib starter kit, and the issue came up. Did you bring it up due to your msgpack work?
samuell has quit [Quit: Leaving]
<SeanTAllen>
the issue came up because of the lib starter kit?
<SeanTAllen>
I'd like to see a JSON library that is nice to use and that ticket is nec in my mind
<endformationage>
No, I was wondering if, with regard to your msgpack library, the issue of recursive types came up.
<endformationage>
I decided to attempt a neo4j bolt driver, which is supposedly similar to msgpack. I was defining it's list type and realized I couldn't do it recursively. Basically the same issue..
<SeanTAllen>
endformationage: my API is too low level for it to matter at this point
<SeanTAllen>
but I suspect it will eventually
<SeanTAllen>
jemc: i appear to have the last piece of runtime backpressure working
<SeanTAllen>
there's a primitive called Backpressure that allows user code to trigger and release it as well
<SeanTAllen>
so if you have a tcp connection that gets throttled you can trigger it in your throttled notify method
<jemc>
nifty!
<SeanTAllen>
there will be an example app with directions on how to use on unix* to show it working
<SeanTAllen>
quite spiffy it is
<jemc>
nagging capabilities question though - should you require some form of auth reference to be able to apply backpressure?
<SeanTAllen>
well
<SeanTAllen>
you can only apply backpressure to yourself
<slfritchie>
I've finished a set of patches that would trace 100% of all actor messages and 100% of thread messages. I'm wondering how ugly someone else thinks it is. ^_^ Should I just open a PR, or show it to Sean so he can holler at me first, or start an RFC, or .. ?
<SeanTAllen>
open a PR slfritchie !
<SeanTAllen>
lets see what you got
<slfritchie>
Hokay
<SeanTAllen>
that's FN awesome btw slfritchie
<jemc>
SeanTAllen: the right, "yourself" as an actor, but that could be a completely different library if you're calling out to someone else's synchronous code
<slfritchie>
I'm abusing CPP macros a tiny bit, so don't be heaping the laurels quite yet.
<jemc>
anyway, just something to think about as we go forward
<SeanTAllen>
jemc: i dont follow
<jemc>
slfritchie: exciting!
user10032 has joined #ponylang
<jemc>
SeanTAllen: let me think it over a bit more and maybe we can talk about it at next sync
<SeanTAllen>
ok
<jemc>
my mind is too scattered by dau-job stuff atm
<jemc>
s/dau/day/
<SeanTAllen>
this is the backpressure primitive btw
<SeanTAllen>
if there's a refactoring, we need to do perf test at Wallaroo Labs to make sure we dont see regressions.
<SeanTAllen>
although glancing at this
<SeanTAllen>
it doesnt look like the refactoring i thought you meant
<SeanTAllen>
this doesnt look like there would be a perf hit
<SeanTAllen>
slfritchie: next up with the backpressure branch is adding those probes you asked for
<dipin>
slfritchie: i'd suggest making two different copies of the ponyint_messageq_pop/push functions that take the appropriate arguments and remove the callter type branch and the macros altogether.. the message sends are in the hot path and will be called very often and the extra branching will have negative effects in terms of pipeline flushes for incorrect branch predictor guesses..