Praetonus has quit [Remote host closed the connection]
jemc has joined #ponylang
acarrico has quit [Ping timeout: 260 seconds]
acarrico has joined #ponylang
jemc has quit [Ping timeout: 240 seconds]
jemc has joined #ponylang
jemc has quit [Quit: WeeChat 1.4]
jemc has joined #ponylang
dtzWill has quit [Ping timeout: 240 seconds]
dtzWill has joined #ponylang
endformationage has joined #ponylang
oats has joined #ponylang
<oats>
hello, I've recently started the pony tutorial and I'm intrigued by the language :)
<oats>
I have a small question, I've been thinking about writing an irc bot lately, and I'm wondering if pony's unique features would be a good fit for that task.
<jemc>
oats: welcome!
<jemc>
yes, I think writing an IRC bot in pony would be a great project to get started with
<jemc>
should be a nice way to get acquainted with how TCP relates to actors in Pony
lonelypony has joined #ponylang
<lonelypony>
hello
lonelypony has left #ponylang [#ponylang]
jemc has quit [Ping timeout: 248 seconds]
vaninwagen has joined #ponylang
<oats>
pony's syntax seems kind of inconsistent to me
<oats>
you have to use "=>" in a function definition
<oats>
but not in a class, actor, or trait definition
<oats>
it seems like indentation is parsed to make blocks of code
<oats>
but 'end' is also used in control structures
madgoat has joined #ponylang
madgoat has left #ponylang [#ponylang]
<SeanTAllen>
pony is not indentation sensitive oats
<SeanTAllen>
pony's approach to syntax is to have as much/as little is needed to make parsing unambigious
<oats>
SeanTAllen› so I guess that when defining multiple classes/actors/whatever in a row, the next definition marks the end of the last one?
<SeanTAllen>
you don't need => to unambigously parse class, actor or trait so, its not part of the syntax
<SeanTAllen>
oats: correct
<oats>
SeanTAllen› so you can have functions defined inside of functions, then?
<SeanTAllen>
named functions inside of named functions?
<SeanTAllen>
or anonymous functions inside of named functions?
<SeanTAllen>
vaninwagen: did you see my comments on the mailing list?
<SeanTAllen>
re: performance
<SeanTAllen>
also that is a really bad benchmark. running a benchmark on the same computer as the program you are benchmarking and not isolating cpus is just not a real good idea
<vaninwagen>
SeanTAllen yep, saw them. profiling gave some hints at ``buffered.reader.line()`` to be problematic - maybe the error part you wrote about (lots of ``pony_throw`` in the perf report)
<SeanTAllen>
nothing else is going to matter unless all the `errors` are removed
<vaninwagen>
but my issue is more about a severe bug in how non-keepalive connections are handled
<vaninwagen>
i would see performance issues as another track
<SeanTAllen>
you have a lot of performance related comments in your issue, thus my raising the issue
<vaninwagen>
SeanTAllen yeah, i actually wanted to start out understanding the http/tcp architecture and see if i can help improving performance.. but then found this issue
<SeanTAllen>
so suggestion for anyone who reads this
<SeanTAllen>
if you want to write a high performance http server in pony
<SeanTAllen>
start from scratch
<SeanTAllen>
don't try to "fix" one that wasn't designed to do high perf
<SeanTAllen>
talk with me or sylvan or someone from sendence about what you should be looking out for to write high perf code
<SeanTAllen>
then add a little bit of feature at a time
<SeanTAllen>
have a good benchmark that isn't problematic like running wrk on a laptop and keep adding features
<jemc>
SeanTAllen: have you ever had occasion to try adding backpressure to `TCPListener` (to delay accepting new connections, creating new `TCPConnection` actors)?
<SeanTAllen>
jemc: no because that isn't really an issue for us, but could make sense for something folks might want to do
<jemc>
I'm wondering if that might be at play here in this situation, when using `Connection: close`
<SeanTAllen>
i spent a couple hours when i was at qcon looking at the HTTP server and its not written with performance in mind. i think folks trying to make it fast will end up getting frustrated.
<SeanTAllen>
jemc: could be
<SeanTAllen>
i'd have to go looking at it some more
<SeanTAllen>
fyi to anyone doing benchmarking with`wrk` it suffers from coordination omission and its results aren't to be trusted
<jemc>
vaninwagen: it might be interesting to add a counter to `HTTPServer`, to check what the max number that `_sessions` reaches is - that is, add a line like `if _max_sessions < _sessions.size() then _max_sessions = _sessions.size() end`, then at the end print the counter with `Debug`
<vaninwagen>
SeanTAllen thx for your input.
<SeanTAllen>
vaninwagen: i just dont want folks to get really frustrated trying to find a magic perf fix for the http server
<SeanTAllen>
what you've found is definitely interesting. when i have some time, i'll eventually look into it
<vaninwagen>
jemc will do. afaik the number of sessions should be bound by the limit arg given to HTTPServer
<SeanTAllen>
i've written 4 or 5 http servers in my life, i'll admit that i don't want to write another
<jemc>
vaninwagen: if it turns out that the `_max_sessions` counter reaches a very high percentage of the total requests, then backpressure *might* be beneficial
<jemc>
I can try to whip up a quick branch that has backpressure for `TCPListener` if you wanted to try it
<vaninwagen>
jemc interestingly removing the code that kept track of sessions didnt change a thing, still > 99% socket read errors on client side
<jemc>
I don't think that tracking the sessions in the set will have any effect on whether they get GC'd - as long as the `TCPConnection` is still "active" (has an open I/O fd listener), it won't be GC'd
<jemc>
however, employing backpressure could help regulate the max number that get allocated in the first place
<jemc>
it looks like the only reason why it's tracking the sessions is so that they can be disposed of if you call `HTTPServer.dispose`
<vaninwagen>
SeanTAllen there are for sure still some HTTP2 servers left for you ;)
<SeanTAllen>
ugh
<SeanTAllen>
i hope not
<SeanTAllen>
its one of my most disliked protocols
<SeanTAllen>
such a pain
<SeanTAllen>
so looking at socket handling
<jemc>
vaninwagen: actually - I don't need to implement a TCPListener.mute or unmute
<SeanTAllen>
if the "from" that TCPConnection uses is the default "" then SO_REUSEADDR isn't used
<jemc>
TCPListener has a "limit" param you can give on creation that sets the max number of simultaneous connections to accept
<SeanTAllen>
so that could have an impact under a really heavy open close scenario
<SeanTAllen>
in socket.c
<SeanTAllen>
os_socket_connect on line 613
<jemc>
and HTTPServer allows you to pass the limit in its constructor too
<SeanTAllen>
616 and 626 are the relevant lines
<jemc>
vaninwagen: so in your example, please try using the `limit` parameter of `HTTPServer.create` - experiment with different values and see if you get a better result
<jemc>
the default is no limit
<jemc>
SeanTAllen: ah SO_REUSEADDR is a good call
<SeanTAllen>
i think it could come into play given that wrk and the http server are running on the same machine and we are opening and closing connections really quickly
<SeanTAllen>
hmmm, that is only on outgoing though what i pointed out and these are all incoming from listen
jemc has quit [Ping timeout: 260 seconds]
<vaninwagen>
jemc i tried with 0 (no limit), 100 and 1000 - didn't make a difference (in terms of socket read errors)
<vaninwagen>
jemc printing the max sessions just gave 13 (a bit more than were then 10 kept open by wrk)
<vaninwagen>
SeanTAllen sry to bother you with that stuff :/
<SeanTAllen>
vaninwagen: its fine as long as i dont have to write an HTTP server
<SeanTAllen>
you aren't bothering me
<vaninwagen>
:)
<vaninwagen>
SeanTAllen from my side you are free to write what ever you want
<SeanTAllen>
So, if someone was to write a high performance http server, TCPConnection's handling of incoming packets is not optimal if you are going for high throughput
<SeanTAllen>
TCPConnection can be really good for framed protocols
<SeanTAllen>
but without a framed protocol you are using `expect(0)` which is going to dump every tiny little packet received into _notify.received which is going to result in more memory allocations unless everything arrives in one large chunk (unlikely)
<SeanTAllen>
what you would want to do is have the TCPConnection allocate a nice large buffer for incoming data that you can examine for "ok now cut me off a chunk" and then it handed off to handler code
<SeanTAllen>
in c, you can shave off chunks of the buffer akin to how we do "trim" and then free that memory later
<SeanTAllen>
with the pony gc, you can trim off chunks of the buffer but the buffer has a whole won't be freed by gc until all its memory is freed. which for a simple http server is fine. maybe not so much for an application server where the application could hold on to that memory for a long period of time
<vaninwagen>
yeah i saw in tcp_connection that for every packet pony will allocate a new array that it passes on to its notify
<SeanTAllen>
if you look in `pending_reads` in TCPConnection, once you get to understand how it works, you'll see that for non-framed protocols, it favor latency over throughput, whatever the OS delivers will be immediately passed along and new memory allocated
<SeanTAllen>
vaninwagen: unless you have a framed protocol and are using expect
vaninwagen_ has joined #ponylang
jemc has joined #ponylang
<vaninwagen_>
woah, we are far down the rabbit hole here
<vaninwagen_>
:)
jemc has quit [Client Quit]
jemc has joined #ponylang
vaninwagen has quit [Ping timeout: 240 seconds]
vaninwagen_ is now known as vaninwagen
<jemc>
vaninwagen: I don't know if you've used wireshark before, but it might be worth looking at what the TCP exchange looks like for these socket read errors
<vaninwagen>
i did looong time ago. good point
<jemc>
also, it would be interesting to see if using wrk2 as suggested by SeanTAllen made a difference
<SeanTAllen>
it shouldnt for the problems people are seeing so far
<SeanTAllen>
wrk is fine for throughput if you are arent sharing a machine
<SeanTAllen>
its latency numbers arent to be trusted however
<SeanTAllen>
jemc: i opened a PR to fix the trim_in_place error in Array as well
<SeanTAllen>
which i wouldn't have thought of doing if vaninwagen hadn't gotten me thinking about trimming arrays for performance
<vaninwagen>
jemc from my first tests with wrk2 it only reports as many requests as i have a limit set on the HTTPServer
<vaninwagen>
btw i did those tests from another machine
<SeanTAllen>
^5 to that
<vaninwagen>
yay
<vaninwagen>
i will try to gather more data for my issue with wrk2 and wireshark and will attach that to the github issue
<SeanTAllen>
awesome
<jemc>
SeanTAllen: yeah I saw your PR and merged it a while ago ;)
samuell has quit [Quit: Hejdå]
<jemc>
err.. whoops, I just noticed you meant another PR for Array
<jemc>
merged
<jemc>
thank you for those important fixes!
<SeanTAllen>
its too damn hot and muggy to go outside