[nanomsg] Re: accessing control IDs
- From: Achille Roussel <achille.roussel@xxxxxxxxx>
- To: "nanomsg@xxxxxxxxxxxxx" <nanomsg@xxxxxxxxxxxxx>
- Date: Tue, 6 May 2014 20:50:11 -0700
I think REQ/REP being stateless is a great thing, it protects the user from
misusing the tool and solves a single problem. Now it is true that in its
current state nanomsg is missing some sort of stateful protocol, but if you're
trying to use REQ/REP when you need to maintain states between requests coming
from a remote node then you're probably not using the right tool to solve your
problem.
HTTP is stateless but it is showing its limits for today's web, and is being
slowly replaced by SPDY which is a stateful protocol... Maybe nanomsg should
get some sort of hybrid protocol or implement SPDY itself (which is a stateful
req/rep with some sort of pub/sub on top of it).
Achille Roussel
+1 415 490 6339
> On May 6, 2014, at 8:13 PM, Drew Crawford <drew@xxxxxxxxxxxxxxxxxx> wrote:
>
>> Nanomsg is stateless from the point of view of the app. Adding session
>> state would be inappropriate.
>
> I have seen this stated several times in this thread, but I have not seen any
> design justification for this position. Such a position should be defensible.
>
> I can construct a rationale for why there should be a REQ-REP-*like* protocol
> that is stateless. That would clearly be useful. However, in my view it is
> an error to give it the name REQ/REP, because this is confusingly similar to
> the ZeroMQ protocol with the same name. It would be tolerable if nanomsg
> REQ/REP was merely API-incompatible and wire-incompatible with the protocol
> from ZeroMQ; that is to be expected. However when the difference is not just
> in wireformat or argument order but is actually conceptually orthogonal to
> the way ZeroMQ REQ/REP is used (which is stateful) in a way that is hard to
> see up front it is time to invent a new name for a new idea. The connection
> between the two protocols is so flimsy that sharing a name is actively
> misleading.
>
> That aside, you are taking a much stronger position than merely one protocol.
> You are saying “nobody should implement a stateful protocol in nanomsg”.
> Well, why not?
>
> Clearly many applications have stateful networking problems. How should they
> solve these problems?
>
> Should they each implement state atop nanomsg in their own incompatible way?
> This is duplication of effort, and probably leads to buggy code
> Should somebody consolidate those stateful implementations into a library
> that sits atop nanomsg? Now language bindings have to be written for that
> library in addition to nanomsg. This is duplication of effort, and probably
> leads to buggy code
> Should these applications not use nanomsg at all, and instead use ZeroMQ?
> But there are many reasons to prefer nanomsg including use of C, BSD socket
> API, context-free implementation, pluggable transports, routing priorities,
> zero-copy, etc. etc. This option basically tells applications “because you
> have stateful networking, you can enjoy none of these benefits”
> Should these applications fork nanomsg and hack in a stateful alternative?
> This is duplication of effort, and probably leads to buggy code
>
> I think what you imagine is that if you make stateful programs difficult,
> people will rethink their stateful problems into stateless ones. Sometimes
> that is true. But other times, people will rethink their stateful programs
> and decide being stateful is the right decision for their problem. There
> should be some vision to address this case other than “deal with it”.
>
> It is one thing to declare stateful problems out of scope for a protocol. It
> is a different thing to declare them out of scope for a networking library.
> People have stateful problems; and this doesn’t disappear merely because
> stateless protocols are well-behaved or more scalable.
>
> I don’t know exactly what the right solution looks like. Maybe it’s creating
> protocols that are deliberately stateful. Maybe it’s introducing some kind
> of pluggable session layer. But I am absolutely certain that throwing up our
> hands and declaring anything stateful as out of scope would move nanomsg out
> of the “zeromq done right” search space and turn it into a very specialized
> tool for very specialized problems.
>
>> On May 6, 2014, at 8:23 PM, Garrett D'Amore <garrett@xxxxxxxxxx> wrote:
>>
>> Nanomsg is stateless from the point of view of the app. Adding session
>> state would be inappropriate. The pipe is almost certainly insufficient by
>> itself and recognize that it is subject to change as a result of connects or
>> disconnects in the underlying transport.
>>
>> The information I would like to make available would allow for actual strong
>> authentication on those transports that can support it, but admittedly it's
>> only on receipt and it ties to a specific message so at the abstract level
>> is still stateless.
>>
>> Sent from my iPhone
>>
>>> On May 6, 2014, at 4:20 PM, Drew Crawford <drew@xxxxxxxxxxxxxxxxxx> wrote:
>>>
>>> I don’t think putting it in the body is the right solution, or at least not
>>> right for every case.
>>>
>>> For one thing it requires allocating storage for this is in the messages.
>>> It is an interesting question how much storage is required but the naive
>>> implementation would be a 128-bit GUID. For a 4-byte messages this bloats
>>> the network traffic significantly. Particularly when there is a simple
>>> solution with zero overhead—pull the pipe key from nanomsg.
>>>
>>> For a another thing, ZeroMQ clearly and unambiguously supports this model
>>> (via ROUTER). So anyone porting ZeroMQ code is in for a rough time
>>> implementing their own scheme atop nanomsg on all the codebases to get the
>>> same behavior they had before. There are of course legitimate reasons to
>>> be incompatible with ZeroMQ (such as achieving compatibility instead with
>>> BSD sockets) but I think this case is much more harmful than helpful and
>>> does not isolate itself to one particular system or codebase.
>>>
>>>> let’s think about cases where the client sending requests loses
>>>> connection, then reconnects with a different address, or connects from
>>>> multiple endpoints (mobile device, desktop computer, …), if remote
>>>> endpoint information is provided by nanomsg all these endpoints will
>>>> appear as different entities, whereas in your application logic they
>>>> should be considered the same.
>>>
>>>
>>> You should probably not rely on transport-layer guarantees to authenticate
>>> a user. However *given* transport-layer information, it *becomes possible
>>> to implement* many authentication schemes.
>>>
>>> For example (and this is the problem that motivated this discussion)
>>> suppose I have some decision oracle which can determine with complete
>>> certainty whether a particular user intended to send a packet. Then
>>>
>>>> if (!oracle_user_sent_message(user,message)) {
>>>> end_session();
>>>> }
>>>
>>> Now if we know (or can guess) that John sent the message via
>>> transport-layer information, the solution is straightforward. But trying
>>> all the possible users is impractical for a slow oracle. So the
>>> transport-layer information can comprise part of an application-level
>>> authentication scheme to identify which user the oracle should be asked
>>> about. Even for sockets that do not have 1:1 fanout to users, the fanout
>>> may divide the users into enough buckets that asking the oracle about each
>>> member in the bucket becomes practical.
>>>
>>> Now of course we could prepend some session ID to the message rather than
>>> rely on transport-level data, and bloat the message accordingly. However
>>> anybody who gets ahold of the session ID could spoof messages with that
>>> session ID from anywhere on the network. Now these would be rejected by
>>> our perfect oracle, but not before ending the user’s session, comprising a
>>> DDoS attack against the legitimate user. Alternatively, relying on the TCP
>>> information significantly increases the difficulty of the attack, requires
>>> a TCP MITM technique or some other advanced persistent threat capability to
>>> execute. This is a major security and reliability advantage to relying on
>>> TCP data in my situation.
>>>
>>> Drew
>>>
>>>
>>>
>>>
>>>> On May 6, 2014, at 12:13 PM, Achille Roussel <achille.roussel@xxxxxxxxx>
>>>> wrote:
>>>>
>>>> Could you put the state information in the body of your message instead of
>>>> attempting to get it from nanomsg. HTTP is also stateless but websites
>>>> maintain more or less state using cookies, session ids or access tokens…
>>>> maybe you can implement this at the application logic level.
>>>>
>>>> I think it’s a sain design to have your transport protocol separated from
>>>> your application logic, let’s think about cases where the client sending
>>>> requests loses connection, then reconnects with a different address, or
>>>> connects from multiple endpoints (mobile device, desktop computer, …), if
>>>> remote endpoint information is provided by nanomsg all these endpoints
>>>> will appear as different entities, whereas in your application logic they
>>>> should be considered the same.
>>>>
>>>>> On May 6, 2014, at 1:23 AM, Drew Crawford <drew@xxxxxxxxxxxxxxxxxx> wrote:
>>>>>
>>>>>> There are many cases that require state full networking
>>>>>>
>>>>> I’m in such a case. The open question at this point is how to achieve it.
>>>>>
>>>>>
>>>>>> On May 5, 2014, at 10:52 PM, Apostolis Xekoukoulotakis
>>>>>> <xekoukou@xxxxxxxxx> wrote:
>>>>>>
>>>>>> Req rep were designed by default to be stateless, that is why finding
>>>>>> the address of the message has been hidden on purpose.
>>>>>>
>>>>>> There are many cases that require state full networking but state full
>>>>>> is more difficult because it requires that you implement an update
>>>>>> mechanism on the routing information.
>>>>>>
>>>>>>> On May 6, 2014 5:02 AM, "Drew Crawford" <drew@xxxxxxxxxxxxxxxxxx> wrote:
>>>>>>> I have dug a little deeper into this. it appears that in global.c [1]
>>>>>>> msg_controllen is never set. I’m not sure if that’s expected.
>>>>>>>
>>>>>>> The attached patch sets controllen based on the size of the chunk.
>>>>>>> Whether right or wrong, this seems to produce the behavior expected by
>>>>>>> zerotacg and Achille, e.g., control bytes are emitted in the RAW case.
>>>>>>> The 8 bytes are
>>>>>>>
>>>>>>>> d0,4e,c0,00,c1,7f,00,00,
>>>>>>>
>>>>>>>
>>>>>>> Three of which (bytes[3],bytes[4],and bytes[5]) seem to change from
>>>>>>> run-to-run. This is mildly surprising, because the RFC documents the
>>>>>>> control ID at being 32 bits, so one would expect four bytes to change
>>>>>>> from one execution to the next. I’m also unable to account for the
>>>>>>> presence of the remaining bytes. Something may be wrong with my patch,
>>>>>>> or with my understanding of the codebase or RFC.
>>>>>>>
>>>>>>> This is an interesting line of inquiry, but since a solution along this
>>>>>>> line has the limitation of requiring me to implement my own end-to-end
>>>>>>> behaviors on top of a raw socket, I’m wondering if it would be
>>>>>>> desirable to introduce an API for this purpose
>>>>>>>
>>>>>>>> /* Returns an integer that uniquely identifies the immediate sender of
>>>>>>>> the most-recently-received message. Returns an error if no messages
>>>>>>>> have ever been received on the socket */
>>>>>>>
>>>>>>>> int nn_sender(int socket);
>>>>>>>
>>>>>>> Such API could work equally well for raw sockets as full sockets, could
>>>>>>> be implemented for different socket topologies, and does not introduce
>>>>>>> an application-layer dependency on parsing the control header format.
>>>>>>>
>>>>>>>
>>>>>>> [1]
>>>>>>> https://github.com/nanomsg/nanomsg/blob/master/src/core/global.c#L817
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On May 5, 2014, at 4:38 PM, Drew Crawford <drew@xxxxxxxxxxxxxxxxxx>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> I thought about that, however, msg_controllen still returns -1 when
>>>>>>>> using raw sockets, suggesting there are no control information
>>>>>>>> available, as the sample below illustrates. Maybe something is wrong
>>>>>>>> with the code sample?
>>>>>>>>
>>>>>>>> Another problem is that use of raw sockets would require me to roll my
>>>>>>>> own end-to-end behavior which may be undesirable.
>>>>>>>>
>>>>>>>>
>>>>>>>>> int client = nn_socket(AF_SP,NN_REQ);
>>>>>>>>> int server = nn_socket(AF_SP_RAW,NN_REP);
>>>>>>>>> nn_connect(client,"inproc://test");
>>>>>>>>> nn_bind(server,"inproc://test");
>>>>>>>>> nn_send(client,"A",1,0);
>>>>>>>>>
>>>>>>>>> int rc;
>>>>>>>>> void *body;
>>>>>>>>> void *control;
>>>>>>>>> struct nn_iovec iov;
>>>>>>>>> struct nn_msghdr hdr;
>>>>>>>>>
>>>>>>>>> iov.iov_base = &body;
>>>>>>>>> iov.iov_len = NN_MSG;
>>>>>>>>> memset (&hdr, 0, sizeof (hdr));
>>>>>>>>> hdr.msg_iov = &iov;
>>>>>>>>> hdr.msg_iovlen = 1;
>>>>>>>>> hdr.msg_control = &control;
>>>>>>>>> hdr.msg_controllen = NN_MSG;
>>>>>>>>> rc = nn_recvmsg (server, &hdr, 0);
>>>>>>>>> print_array(body,rc,"body”); //contains only A
>>>>>>>>>
>>>>>>>>> printf("msg_iovlen %d\n",hdr.msg_iovlen); // 1
>>>>>>>>> printf("msg_controllen %d\n",hdr.msg_controllen); // -1
>>>>>>>>
>>>>>>>>
>>>>>>>>> On May 5, 2014, at 4:32 PM, Achille Roussel
>>>>>>>>> <achille.roussel@xxxxxxxxx> wrote:
>>>>>>>>>
>>>>>>>>> You have to use AF_SP_RAW sockets to get access to these info in the
>>>>>>>>> control header when receiving a message with nn_recvmsg.
>>>>>>>>>
>>>>>>>>>> On May 5, 2014, at 2:27 PM, Drew Crawford <drew@xxxxxxxxxxxxxxxxxx>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I have a REP socket. I’m trying to identify the channel (sender or
>>>>>>>>>> forwarder) on which some message has arrived to the socket. A
>>>>>>>>>> transport-layer understanding of the sender is not required; any
>>>>>>>>>> identifying value, such as an integer, is sufficient. Consulting
>>>>>>>>>> the REQREP spec suggests that the topmost “channel ID”, one of the
>>>>>>>>>> records in the “backtrace”, is the identifier I’m looking for.
>>>>>>>>>>
>>>>>>>>>> Clearly this identifier is not exposed over the nn_recv interface.
>>>>>>>>>> I had some hopes that it would be accessible in the nn_recvmsg
>>>>>>>>>> interface, possibly as control information, but it seems not to be
>>>>>>>>>> the case:
>>>>>>>>>>
>>>>>>>>>>> int client = nn_socket(AF_SP,NN_REQ);
>>>>>>>>>>> int server = nn_socket(AF_SP,NN_REP);
>>>>>>>>>>> nn_connect(client,"inproc://test");
>>>>>>>>>>> nn_bind(server,"inproc://test");
>>>>>>>>>>> nn_send(client,"A",1,0);
>>>>>>>>>>>
>>>>>>>>>>> int rc;
>>>>>>>>>>> void *body;
>>>>>>>>>>> void *control;
>>>>>>>>>>> struct nn_iovec iov;
>>>>>>>>>>> struct nn_msghdr hdr;
>>>>>>>>>>>
>>>>>>>>>>> iov.iov_base = &body;
>>>>>>>>>>> iov.iov_len = NN_MSG;
>>>>>>>>>>> memset (&hdr, 0, sizeof (hdr));
>>>>>>>>>>> hdr.msg_iov = &iov;
>>>>>>>>>>> hdr.msg_iovlen = 1;
>>>>>>>>>>> hdr.msg_control = &control;
>>>>>>>>>>> hdr.msg_controllen = NN_MSG;
>>>>>>>>>>> rc = nn_recvmsg (server, &hdr, 0);
>>>>>>>>>>> print_array(body,rc,"body”); //contains only A
>>>>>>>>>>>
>>>>>>>>>>> printf("msg_iovlen %d\n",hdr.msg_iovlen); //1
>>>>>>>>>>> printf("msg_controllen %d\n",hdr.msg_controllen); //-1
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I have consulted a previous mailing thread on this topic which
>>>>>>>>>> suggests channel IDs are manipulated in rep.c. Indeed, the
>>>>>>>>>> information I’m looking for seems to be moved around between
>>>>>>>>>> nn_sockbase, nn_msg, nn_rep, and similar structures. However I
>>>>>>>>>> cannot work out a sane way to get those structures from application
>>>>>>>>>> code.
>>>>>>>>>>
>>>>>>>>>> Any suggestions on identifying the sender of a remote message?
>>>>>>>>>>
>>>>>>>>>> Drew
>
Other related posts:
