[nanomsg] Re: nn_recv with padding
- From: Garrett D'Amore <garrett@xxxxxxxxxx>
- To: nanomsg@xxxxxxxxxxxxx, Martin Sustrik <sustrik@xxxxxxxxxx>
- Date: Mon, 12 May 2014 22:23:24 -0700
On May 12, 2014 at 10:06:37 PM, Martin Sustrik (sustrik@xxxxxxxxxx) wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 13/05/14 02:03, Drew Crawford wrote:
> It had not occurred to me how inproc is implemented. It’s obvious
> in retrospect that no zero-copy implementation can be done in that
> case, but it wasn’t obvious before I had considered what the
> implementation is.
>
> However, I think it is still possible to support custom allocation
> sizes for TCP/UDP transport, which is where sending messages is the
> slowest and performance matters most. Depending on how serious you
> want to be about supporting the same socket API across different
> transports the implementation could either fall back to memcpy in
> the inproc case or else return an error and force the programmer do
> it.
Yes. But that means breaking the zero-copy promise, which is the thing
you wanted to achieve in the first place. Imagine the documentation
for the socket option:
NN_ALLOC_MORE_MEM: Causes nanomsg to allocate additional bytes per
message. Setting this option to a positive value may cause copies of
the message to be done within the stack, invisible to the user.
That being said, I feel there's some logical fallacy in the whole
problem statement. Either you have large messages (>100kB) in which
case adding 16 padding bytes to the message on the sending side is
going to have no performance impact. Or, possibly, you have small
messages (<100kB) where the whole copying thing is going to happen in
CPU cache without accessing physical memory and zero-copy is thus
going to have negligible performance impact.
Are people using such large (>100kB) messages with nanomsg? I’ve always
assumed that zero copy was a false savings but maybe because I figure that the
largest useful message I’d ever want send is at most a few pages (10k or so).
That said, mangos has an optimization where the header space is allocated in
such a way that attempts to avoid having to realloc or move a message when we
want to add additional header content. But I’m using go byte slices for it.
Sadly, go doesn’t have a nice way for me to preallocate “headroom” like we do
with NIC packet buffers in the kernel.
- Garrett
Other related posts:
