[nanomsg] Re: A multiple client/worker for NN_PAIR style sockets.
- From: Alex Elsayed <eternaleye@xxxxxxxxx>
- To: nanomsg@xxxxxxxxxxxxx
- Date: Wed, 07 Jan 2015 16:42:45 -0800
Rohit Saboo wrote:
> Hi Alex,
>
> Thanks for helping out. See comments inline.
>
> @Garret, doesn't NN_BUS send the messages to everyone. It can address
> client-to-server communication. However, how will the server address a
> message to a specific client?
>
> Rohit
>
> On Wed, Jan 7, 2015 at 2:28 PM, Alex Elsayed
> <eternaleye@xxxxxxxxx> wrote:
>
>> Honestly, this sounds like a misdesign - the semantics you want are TCP
>> semantics, and you'd basically only be using nanomsg as framing.
>>
>> People use ZMQ that way, but it was explicitly left out of the intended
>> usecases of nanomsg if I remember correctly.
>>
>> It may be best to decompose your protocol into multiple parts. Could you
>> give more information on what _actually_ gets sent back and forth? I'd be
>> willing to help.
>>
>> Nanomsg is designed around "patterns" - higher-level communication
>> primitives - _instead_ of application-level free messages.
>>
>> Here are some that might be useful for your case:
>> Messages that need a response, but only from one peer: REQ/REP
>> Messages that need a response from the set of peers: SURVEY
>> One-way messages to all: PUB/SUB
>> One-way messages to one peer: PUSH/PULL
>>
>
> The "application" is composed of "two" parts -- a set of clients and a
> server. The server will be replicated for load balancing and handling
> failovers. All communication between the clients and servers is expected
> to be authenticated and secure for which I've separate application-level
> logic. The communication between them will fall into one of these
> patterns: * client sends updates to server (at a frequency greater than
> 20Hz), expecting no response from the server;
> * client asks questions to server expecting a response;
I'd model this as a trivial REQ/REP. You can have multiple participants as
the "REP" side of the topology, so your request-level load balancing and
failover is done by nanomsg there (though you still need to do any state
synchronization at the application level, for which a BUS topology among the
workers would be good)
> * server sends updates to all clients;
This I'd model this as PUB/SUB. Now, there can only be one PUB on such a
topology IIRC, but there's a relatively simple way to handle that: using the
BUS among the workers I mentioned previously, have each worker pick a random
number and send it out. Whichever has the highest number "won" the election,
and gets to be the PUB. They then send out a periodic keepalive on the BUS,
and if they go too long without a keepalive the other nodes have a new
election.
> * server sends updates to a specific client; or
This one's a bit funnier - what I'd suggest is to actually have the PUB/SUB
of regular updates be a (generation number, delta) tuple, and the client
keeps track of the last generation it had. That way, it can tell if it
misses an update - and then it can send a "request update" message over the
REQ/REP, and get a full state as a reply. It may want to buffer further
updates from the PUB/SUB while it waits for that response.
To make this work best, clients should connect to the PUB/SUB before the
REQ/REP.
> * server asks questions to a specific client expecting a response.
Now, this one is tricky. Since clients are dynamic - they can come and go,
etc - and at the nanomsg level they aren't distinguished either, there's not
really an application-level way to say "This is for client X."
The granularity of "who a message is to" in nanomsg is pretty much
topology+role. It's to the REP side of a REQ/REP, or the SUB side of a
PUB/SUB, etc, not a specific server. The actual REP message is kind of an
exception - but in order to _implement_ that kind of thing, you have to be
working at the RAW level, and that's below all the tasty nanomsg goodness.
There are a couple options here, and which one you want depends on what your
actual API is at the application level.
If the server's request is small, the client's reply is large, and the
client can tell from the request whether it's the one the message was
intended for, SURVEY might be a good bet. Send the request to all clients,
and only the intended target responds.
>> Also, them being on different networks isn't _that_ tricky - you can use
>> nn_device to create a sort of bridging point.
>>
>>
> Take the following example:
> Client Ci is in network Ni with gateway Gi connecting it to the public
> internet. A server S0 is in network N0 with gateway G0. Now G0's ip is
> well-known, and it can be configured to forward connections on specific
> ports to S0. However, Gi are not in our control and the binary with
> nn_device would have to run on Gi, right?
No, actually. You could think of the nn_device as a load-balancing proxy,
and thus it could run behind G0 and have a port forwarded to it.
One trick I'm fond of is having both sides of the nn_device bind() (as
opposed to connect()), and then both the server (PUB, REP, SURVEY) and
client (SUB, REQ, RESPONDENT) systems would connect() to the appropriate
ports it's listening on.
>> > Hi all,
>> >
>> > I need to create a system where there are several clients that can
>> > communicate simultaneously with a server. For simplicity, we can assume
>> > that the server has a constant number of workers.
>> >
>> > The clients can send messages to the servers and the workers may
>> > respond to these messages and/or send messages by itself possibly
>> > expecting responses. In other words, there's free two way communication
>> > between a client and the server.
>> >
>> > Using NN_PAIR with devices results in the error "transport endpoint
>> > already connected" when the second worker tries to connect to the local
>> > "inproc" socket. Using a solution that creates a pair of
>> > NN_PUSH/NN_PULL sockets also won't work because the client and server
>> > are on different networks and providing a connection endpoint on the
>> > client for the server to connect to will be tricky.
>> >
>> > Is there a way in nanomsg to have free two-way communication between
>> > multiple clients and a single server?
>> >
>> > Thanks,
>> > Rohit
>>
>>
>>
>>
Other related posts:
