<SeanTAllen>
3 out of 4 numbers right is still pretty good
<rkallos>
I read it today because $WORK is considering rewriting an old, hand-written, barely-distributed & not-scalable C service, and one of the possibilites was to hook up replicas to raft/etcd, and I'm not convinced we should pay the cost of linearizability
<rkallos>
And I'm also evangelizing Pony at $WORK because I think it could fit really well. We have a lot of Erlang, a good bit of C, and very little Rust, and we're not thrilled about the state of async in Rust.
<SeanTAllen>
I'd love to hear more about rust and async
<rkallos>
Unfortunately, I'm not really the person to elaborate more on that :-P I didn't write the Rust code, but I've looked at tokio & co. and I found it very difficult to understand. Coming from Erlang, Pony makes much more sense to me.
PrsPrsBK has quit [Quit: PrsPrsBK]
acarrico has joined #ponylang
acarrico has quit [Ping timeout: 272 seconds]
endformationage has quit [Quit: WeeChat 1.9.1]
kbobyrev[m] has quit [Ping timeout: 250 seconds]
aturley has quit [Read error: Connection reset by peer]
aturley has joined #ponylang
<vaninwagen>
rkallos i am gonna have some time this week and gonna finish up some simple benchmarks for the https://github.com/ponylang/http project, i am super happy to see if the node parser beats the current pony http parser
<vaninwagen>
If it does, i will send you some pony stickers
<Candle>
ponylang/http was removed, partly because it reportedly uses errors as control flow, though I can't find where. Does anyone else know where?
<vaninwagen>
Ah, yeah, it was removed from the stdlib, yes
<vaninwagen>
Got it, sorry, me tired
* vaninwagen
needs more coffee (as always)
<Candle>
While that section can raise errors, I don't believe it does as a matter course; the parser'll be in the _ExpectResponse state; so should be just about to read a new response, where it's expecting something like "HTTP 1.1 200 OK\r\n".
pzel has quit [Ping timeout: 252 seconds]
<Candle>
Urg, Is it the case where the Reader doesn't have all the data in it's buffer yet? That's where it can raise errors.
<vaninwagen>
Exactly
<vaninwagen>
It raises an error to show it needs more data
<Candle>
That's... kinda annoying.
<vaninwagen>
What i also dont like is that reader.line() always allocates new strings while it could use the data in the buffer
<vaninwagen>
That might be a problem at boundaries of two tcp payloads, but it should be easy to write a wrapper class around two arrays that acts like one array
<Candle>
That might work until you need to have 1/2 an input array as part of your String.
<Candle>
In that case, it may be worth copying the chopped array.
<vaninwagen>
True, but we could also model http request and response in terms of those wrappers
<Candle>
Yes. the whole blob as a collection of arrays, and essentially offsets into it to show where the element boundaries are.
<Candle>
Once you use the Reader, you are probably into erros-as-control-flow though.
<vaninwagen>
Yeah, it is super nice for binary protocols, but for text based ones i think something else is in order
<Candle>
When you know how much you want to read ahead of time (e.g. reading a U128) you can use 'size()' beforehand to remove the chance of errors; but reading arbitrary length bits is problematic.
<vaninwagen>
yep, thats where framed protocols really shine
<Candle>
I wonder. If you're certain that you are in a single-threaded section, and you do `let u: U128 = if reader.u128_available() then reader.u128()? end` you can be certain that the error cannot happen; but that snippet won't compile as `u`'s type would have to be `(U128 | None)` and you have add `... else 0 ...`.
<Candle>
The same sort of thing applies to Map.contains("something") / Map.apply("something").
pzel has joined #ponylang
<vaninwagen>
The good thing with pony, unless a behaviour call is involved, you are always in a single threaded Environment
<Candle>
Exactly. And the compiler knows that you are operating on a data structure that no other thread can modify.
<vaninwagen>
That is one of the things i love about pony
<vaninwagen>
Dealing with optionals might be a way out
<Candle>
The problem with the compiler compiling `let u: U128 = if reader.u128_available() then reader.u128() end` without errors is knowing the relationships between the functions.
<vaninwagen>
But we still would have to handle the else case, no?
<Candle>
True.
<vaninwagen>
And the union with None also needs a boxed U128 - so it is no longer only a machine word
<vaninwagen>
Which is also a cost to pay
<Candle>
Is `let u: U128 = if reader.u128_available() then reader.u128() else 0 end` better than `let u: U128 = if reader.u128_available() then try reader.u128() else 0 end else 0 end`
<Candle>
But then you still need to describe the complex relationships between functions to determine if an error can be raised (what happens if you call `reader.u128_available()` then `reader.u64()` twice? That's probably safe, right?)
<Candle>
does that match still require more than a machine word?
<vaninwagen>
I think so, because it needs the type descriptor at runtime
<Candle>
Hrm, food for thought. I should probably go back to trying to figure out what the hell is going on with the networking here.
pzel has quit [Ping timeout: 244 seconds]
rkallos has quit [Ping timeout: 252 seconds]
pzel has joined #ponylang
acarrico has joined #ponylang
bougyman is now known as BOOgyman
BOOgyman is now known as B00gyman
brightwell has joined #ponylang
TwoNotes has joined #ponylang
<TwoNotes>
buffered Reader append expects a val argument. I have a box. Solution?
rkallos has joined #ponylang
<SeanTAllen>
question 1 would be: why do you have a box?
<SeanTAllen>
if its a box that means its val or ref
<SeanTAllen>
so, you need to treat like a ref and copy it for buffered reader.
brightwell has left #ponylang ["Konversation terminated!"]
endformationage has joined #ponylang
<TwoNotes>
I was under the impression that box let you write funs that would accept two different caps
<TwoNotes>
Reader.append is going to copy it anyway
<SeanTAllen>
When you say "i have a box", what does that mean?
<TwoNotes>
I have my own 'append' fun at a higherlevel of abstraction and it takes a 'box'
<TwoNotes>
Back to the drawing board. The original data is in a memory-mapped file and can not be modified. It makes sense that is represented as 'val' data, correct?
<TwoNotes>
But it cannot be shared between processes.
<SeanTAllen>
correct
jemc has joined #ponylang
<TwoNotes>
The current mapping of the file is only guaranteed to be consistent within the DB 'transaction'. It can go away after that. So I am copying it out into a 'buffered' object
<TwoNotes>
Ah, need a 'clone' in there
<jemc>
Candle, vaninwagen: yeah, I saw you found my old rope library - I never really got it working like I wanted it to.
<jemc>
Candle, vaninwagen: offhand, I'm not sure how feasible it is, but probably the best thing to do for performance of the HTTP library is to try to use `expect` with the TCP socket
<jemc>
Candle, vaninwagen: I think HTTP headers are probably going to cause problems for that plan, though...
<SeanTAllen>
http and expect can be very difficult
<SeanTAllen>
you cant use it for headers as jemc notes
<jemc>
I just took a brief look at the major HTTP parsers in Rust - they all seem to expect a contiguous byte array... :(
<jemc>
to get something like the Rope library to work well, it would need to be compatible with all/most of the places where you would usually use a String
<jemc>
which means either that it would need to be part of the standard library, or most places that use a String would use a String-like interface instead
<jemc>
and that level of indirection would have its own performance cost...
<jemc>
(though still probably paling in comparison to the byte buffer copying cost)
<jemc>
so, coming back to `expect`, here's a wild thought: what if `TCPConnection.expect` had another optional `pre: Array[U8] box` argument, that let you provide some extra "pre" bytes to copy into the start of the `_read_buf`. the size of the `pre` would be added to the effective expected bytes number and also se an offset into which the socket read would start populating bytes.
<SeanTAllen>
so @jemc something like that but also very not like that is already in the wallaroo tcp code
<SeanTAllen>
i completely redid how expect works
<SeanTAllen>
but its never made it into ponyc
<jemc>
so the workflow would basically be, you parse the headers in the normal bumbling way over an arbitrary number of byte buffer boundaries, then you end up with some number of extra bytes leftover, which would be the first few bytes of your expected count
<jemc>
ah, interesting
<SeanTAllen>
because there are edge cases like switching from using expect to doing expect(0) that arent handled
<SeanTAllen>
it has a large buffer
<SeanTAllen>
it fills it up as we go
<SeanTAllen>
expect is used to slice off of it
<SeanTAllen>
perf improvements are rather large
<SeanTAllen>
until sadly gc comes into play
<SeanTAllen>
we'd need to replace orca to get the really big wins there
<SeanTAllen>
the message passing overhead of ORCA hurts a lot there
<SeanTAllen>
which is why looking at non-message-passing GCs is a thing
<malthe>
SeanTAllen: btw i'm trying to rewrite ponylang-mode to use smie; status is that I have a reasonably good setup with tests running via cask and ert-runner and now playing around with grammar and rules.
<SeanTAllen>
nice!
<SeanTAllen>
do you want access to the repo?
<malthe>
for tests, I have files with good indentation, then I remove leading whitespace from all lines and try to indent-region - and compare.
<malthe>
not quite there yet, but I'll let you know when there's something to show.
<SeanTAllen>
ok
<malthe>
smie is pretty hard to grok but I think I am getting the hang of it
<malthe>
what I actually wanted to do is to port my postgresql driver from typescript (ts-postgres) to pony, but without emacs working 100%, it's an impossible task.
bezalel has joined #ponylang
<vaninwagen>
jemc i like those wild thoughts
bezalel has quit [Quit: Konversation terminated!]
TwoNotes has joined #ponylang
<TwoNotes>
Can 'addressof' be used on a class object the same as a struct, when passed to an FFI?