It appeared that one bottleneck was the poll re-registration of the TCP
socket each time a message was sent or received. Switching from edge mode
to level mode and registering just once had significant impact on the
latency benchmark and gave a slight improvement on the throughput one. It
is still way behind on this one though. And as a side effect I can now use
my laptop has a central heating/hair drier while running the latency
benchmark, thanks to that spinning background thread.
*Average latency (µs)*
| Msg Size | Roundtrips | Nanomsg | Scaproust |
| 512 | 50000 | 19 | 19 |
| 1024 | 10000 | 21 | 17 |
| 8192 | 10000 | 23 | 21 |
| 102400 | 2000 | 56 | 38 |
| 524288 | 500 | 323 | 132 |
| 1048576 | 100 | 794 | 489 |
*Average throughput (Mb/s)*
| Msg Size | Msg Count | Nanomsg | Scaproust |
| 512 | 1000000 | 3091 | 425 |
| 1024 | 500000 | 5511 | 822 |
| 8192 | 50000 | 13865 | 4843 |
| 131072 | 10000 | 19694 | 20840 |
| 524288 | 2000 | 16215 | 26298 |
| 1048576 | 1000 | 12501 | 10927 |
Regards,
Benoît
On Thu, 15 Dec 2016 at 22:07 Benoit Labaere <benoit.labaere@xxxxxxxxx>
wrote:
ALLOCATION:
I had initially planned to use or implement some kind of message pooling
internally, but I could also make it explicit and allow the user to define
it's own pooling strategy.
THREADING MODEL:
I chose non-blocking I/O not to make sure I could get the best performance
possible from day 1 but because having all I/O done in one thread means
most of the code is single-threaded. Which is quite a comfortable situation.
PRE FETCHING:
What I meant by "prefetch of incoming message" was rather to have some
protocols eagerly read and store one message on the background thread so
that when the user requests a read, the message is already available.
I have not yet taken care of processor cache effects, hash maps are all
over the place for the moment for example.
There is surely a lot of things to improve in this aspect.
Thanks for the suggestions,
Benoît
On Thu, 15 Dec 2016 at 08:42 Matthew Hall <mhall@xxxxxxxxxxxxxxx> wrote:
On Wed, Dec 14, 2016 at 04:47:07PM +0000, Benoit Labaere wrote:
There is a per message overhead for several reasons:
- allocation on each send, because the socket takes ownership of the
message when sending.
For a long time I have been wishing for a way to free the messages via
callback handler. Because I usually prefer to create the messages in
special
pooled memory, send them, then mark them free from the pool upon successful
TX. It's basically zero alloc cost but not available in nanomsg or zeromq
yet.
- there is a channel between the user thread and the I/O thread, it is
crossed once by the sent message and once by the 'return code'.
Many people think non-blocking is best. But in my performance studies I
usually find that a very large number (like 4-6 per physical core) of
simple
blocking user threads is faster as it prevents this problem right here, and
the queueing overhead prevents you from managing to totally saturate.
- there is no prefetch of incoming messages.
Yes, you have to try to fit messages into cacheline multiples and proper
alignment then prefetch first 1-3 lines after RX completes before pushing
back
to user code.
Good luck,
Matthew.
