[nanomsg] Re: ZMQ_ROUTER like functionality
- From: Paul Colomiets <paul@xxxxxxxxxxxxxx>
- To: "nanomsg@xxxxxxxxxxxxx" <nanomsg@xxxxxxxxxxxxx>
- Date: Wed, 17 Sep 2014 23:48:44 +0300
Hi David,
On Wed, Sep 17, 2014 at 11:00 PM, David Robillard <d@xxxxxxxxxxxx> wrote:
> Well, "scalable" and "handled magically by nanomsg" aren't quite the
> same thing. In this case, the load-balancing and routing to particular
> workers is inherently application specific.
>
> The system is very scalable (partially by virtue of both servers and
> workers being multi-threaded with ZMQ-based inproc load balancing), but
> this particular part of it isn't the sort of thing any network library
> could do on its own.
>
> (The application is a very large database of sorts, fragmented across
> workers, so workers are inherently not interchangeable since one worker
> can not store the entire data set)
>
Yes, I had a similar case. We discussed it here. The problem is that
nobody had implemented this as another nanomsg pattern.
BTW, you can try, if you are proficient with C or Go.
>> So yes, the only solution so far is to make a socket per worker.
>
> Socker per server thread per worker, in this case. Sharing worker
> sockets might be doable, but contention would hurt scalability of the
> server and I doubt this solution would be any good.
>
> If many sockets isn't inherently a performance issue, then that should
> be fine. Memory usage on the server isn't particularly limited.
>
Eh, there is a compile-time socket limit NN_MAX_SOCKETS which is
defined in src/core/global.c, you may need to make it higher (it's 512
by default). Otherwise I believe there is nothing that makes library
slower with the number of sockets. Just be sure you also have high
file descriptor limit ;)
--
Paul
Other related posts:
