[nanomsg] Re: header behavior for ipc vs inproc
- From: Drew Crawford <drew@xxxxxxxxxxxxxxxxxx>
- To: nanomsg@xxxxxxxxxxxxx
- Date: Mon, 14 Jul 2014 05:37:33 -0500
Actually solving the problem in the C codebase is somewhat difficult for
performance reasons.
As a stopgap measure, it would be useful to have some way to query the
transport a given protocol will use, or at least a way to infer if a socket is
going to be operating with NN_PIPE_PARSED or not.
Such a test would allow an implementation of an NN_CHECK_NOT_PIPE_PARSED
directive in device.h, which would be able to at least throw a configuration
error at the user if he configures a device in such a way that header
information cannot be determined.
As it stands the user experience for this situation is very poor.
Drew
On Jul 13, 2014, at 7:56 PM, Garrett D'Amore <garrett@xxxxxxxxxx> wrote:
> Yep. This was one area that I thought wasn't as clean as it could be. I
> took a different approach with mangos.
>
> Sent from my iPhone
>
> On Jul 13, 2014, at 5:05 PM, Drew Crawford <drew@xxxxxxxxxxxxxxxxxx> wrote:
>
>> I noticed today that an nn_device moves different data depending on whether
>> the underlying transport for one of its sockets is ipc or inproc.
>> Considering this setup:
>>
>> (REQ->[XREQ]->inproc->XREP)->DEVICE -> (another socket)
>>
>> That is to say, a req socket connected over inproc to a raw rep socket on a
>> device (the device having another socket)
>>
>> In this configuration, the data the device sees (as determined by
>> nn_device_rewritemsg) is exactly the data that the user sends. Whereas in
>> a different configuration:
>>
>> (REQ->[XREQ]->ipc->XREP)->DEVICE -> (another socket)
>>
>> That is to say, the earlier configuration with ipc instead of inproc.
>>
>> In this ipc configuration, the data the device sees contains 4 extra bytes.
>> These bytes happen to be the request ID from the REQ socket.
>>
>> Well, the presence of the request ID is not very surprising, since normally
>> it is stripped by a REP socket, which does not occur before the device in
>> the dataflow. What is surprising however is that it is only present with
>> certain transports! It is strange to me that the bytes sent would vary on a
>> transport basis.
>>
>> I have traced this behavior all over the codebase, and after tracing through
>> around 10 files involved in the whole network stack what seems to be
>> happening is that in the inproc case, the nn_msg is marked NN_PIPE_PARSED.
>>
>>> /* Specifies that received message is already split into header and body.
>>> This flag is used only by inproc transport to avoid merging and
>>> re-splitting
>>> the messages passed with a single process. */
>>
>> NN_PIPE_PARSED is handled differently by some protocols than if it isn’t
>> NN_PIPE_PARSED.
>>
>> This is all well and good, but it does have two nonintuitive implications:
>>
>> nn_device sees different data for inproc vs non-inproc
>> In the inproc case, any header data that is trimmed by the socket (like
>> channel ID, request ID, etc.) is effectively passed by magic as far as the
>> nn_device is concerned. So no device can reliably use such header
>> information for any purpose, like routing or prioritization, unless it
>> swears off inproc completely.
>>
>> I think having studied this behavior in some detail now I may be able to
>> generate some tolerable workaround for my situation. (We’ll see.) But I
>> wanted to document this somewhere so that it’s less of a stumbling block for
>> the next person.
>>
>> This is also a good sample case of sockets / transports / devices not being
>> as pluggable as one might expect. One might imagine for example that one
>> could read the REQ sourcecode and write a prioritization device for REQREP.
>> And you can, but only for non-inproc transports, and that is really hard to
>> see up front.
>>
>> Drew
>>
>>
Other related posts:
