Description of Changes

Today, for each client connection, we process requests one at a time, and we wait for each request to finish before processing the next one. The way this works is that each websocket reader waits for a notification (via a one shot channel) from the JS worker that it is done processing the request. This notification is quite costly because it always wakes the receiving task.

To avoid this costly wake, this patch adds an intermediate relay actor. The relay reads from its message queue periodically and forwards the replies on to the correct websocket reader task. This trades latency for increased throughput.

I would classify this as a hack, and so I'm not sure I would recommend merging this. A potentially better solution would be to remove the reply altogether. However if we want to preserve execution order per connection, I'm not sure if this is possible today, because I'm not sure if we have a single serialization point for all requests. But I think it is ultimately the direction we want to go.

API and ABI breaking changes

None

Expected complexity level and risk

1

Testing

Manual