[nanomsg] Re: Fanout pattern
- From: Paul Colomiets <paul@xxxxxxxxxxxxxx>
- To: Martin Sustrik <sustrik@xxxxxxxxxx>
- Date: Wed, 16 Jan 2013 22:22:32 +0200
Hi Martin,
On Wed, Jan 16, 2013 at 7:53 AM, Martin Sustrik <sustrik@xxxxxxxxxx> wrote:
> Hm. Thinking about it, PUSH/PULL pattern (distribution of work among
> homogeneous stateless workers) may actually be uniform even in multiple
> push-ers/multiple pull-ers setup. I should relax the restriction.
>
> (Btw, note that the same doesn't apply to fanin pattern.)
>
Ok. Makes sense. But let's now talk about PUB/SUB a bit.
Typical Edge/Origin scenario is following:
1. There are many "origin" nodes which produce some data, let's denote
them A1, A2...
2. There are many "edge" nodes which consume data (they actually
forward data by another protocol, but it's irrelevant for the
discussion), let's denote them B1, B2...
3. Each edge must have each origin's data stream
Now with nanomsg we can connect them by introducing an intermediary C1
which is connected using fanin to A1, A2... to consume data. Also C1
connected to B1, B2... using PUB/SUB pattern. This pattern is clear.
But when C1 is not able to handle the load, we can't do anything. We
must either modify application at A1, A2... to send messages to
multiple fanins, or to modify B1, B2... to receive messages from
multiple sub sockets.
If we don't want to modify application when we scale, we could
implement intermediary C1, so that it has a separate SOURCE per B1,
B2, B3...., and then edge nodes can be connected to intermediaries
using fanin, and all the complexity is hidden inside the intermediary.
Do you have this in mind? Or you have another solution for this
scalability problem?
--
Paul
Other related posts:
