[nanomsg] Re: why memcpy data to be sent?
- From: Jack Dunaway <jack@xxxxxxxxxxxxxxxx>
- To: nanomsg@xxxxxxxxxxxxx
- Date: Mon, 27 Oct 2014 14:08:15 -0500
Steve, I kinda hear what you're saying here.
Likewise, I am writing a binding for an execution system with its own
sandboxed memory manager (LabVIEW), meaning for us "zero copy" with
nn_allocmsg() is best case "only one more copy".
One proposal is to add a new flag to nn_sendmsg() in the spirit of the
"SQLITE_TRANSIENT" flag when inserting data into a sqlite database (
https://www.sqlite.org/capi3ref.html#sqlite3_bind_blob), iff the binding is
willing to guarantee the burden of ensuring that memory location remains
immutable for the duration of the call.
Related, I've considered upgrading the nanomsg allocation manager (
https://github.com/nanomsg/nanomsg/blob/master/src/utils/alloc.c) to be
pluggable via an instance-wide setup function nn_alloc_setup(), in the
spirit of
https://jansson.readthedocs.org/en/2.6/apiref.html#custom-memory-allocation
This could enable nn_recvmsg() the ability to call into the execution
system's allocation manager for zero-copy receives.
Best regards,
Jack R. Dunaway | Wirebird Labs LLC
On Mon, Oct 27, 2014 at 1:13 PM, Steve Vinoski <vinoski@xxxxxxxx> wrote:
>
>
> On Mon, Oct 27, 2014 at 12:27 PM, Dirkjan Ochtman <dirkjan@xxxxxxxxxx>
> wrote:
>
>> On Mon, Oct 27, 2014 at 4:54 PM, Steve Vinoski <vinoski@xxxxxxxx> wrote:
>> > The reason I'm interested is that in the Erlang nanomsg driver, Erlang
>> > naturally sends I/O vecs to the enm:send() function (i.e., the I/O
>> vector is
>> > essentially a built-in type in Erlang), and so passing them along
>> directly
>> > to nanomsg should mean very high efficiency and copy avoidance. So, it
>> would
>> > be very handy if the copying was either removed or if the caller could
>> pass
>> > a flag to say "I guarantee these data blocks will not be freed, please
>> don't
>> > copy them."
>>
>> Theoretically we have http://nanomsg.org/v0.4/nn_allocmsg.3.html for
>> this, right?
>>
>
> Theoretically, yes, but this doesn't help at all when using nanomsg
> underneath other systems that do their own memory allocation. For my
> specific case of sending messages, the Erlang VM allocates the memory for
> the data as part of its per-lightweight-process GC system, and the Erlang
> VM passes pointers to those blocks to my driver which then passes them
> along to nanomsg.
>
> Maybe I'm missing something, but in looking at the nanomsg code some more
> I couldn't see any compelling reason for the copy other than the fact that
> the send functions called after the copy is made don't do the equivalent of
> writev and instead take just a single block of data.
>
> --steve
>
Other related posts:
