[nanomsg] Re: ZMQ_ROUTER like functionality
- From: Alex Elsayed <eternaleye@xxxxxxxxx>
- To: nanomsg@xxxxxxxxxxxxx
- Date: Wed, 17 Sep 2014 22:40:55 -0700
Paul Colomiets wrote:
> Hi David,
>
> On Thu, Sep 18, 2014 at 12:28 AM, David Robillard <d@xxxxxxxxxxxx> wrote:
>> I am good with C. Since the architecture of nanomsg seems well suited
>> to extension I was considering this as an option.
>>
>> I'm not sure how the sender should/would best indicate the desired
>> destination. There is no suitable ancillary option, though working with
>> that is pretty cryptic anyway.
>
> Well, here is what I proposed a while ago:
>
> //www.freelists.org/post/nanomsg/Trying-to-implement-directory-pattern
>
> The basic idea is that worker subscribe on some subset of topics,
> similarly to how pub-sub works. This concept also works with devices
> (i.e. not only directly connected peers), and can load-balance between
> several workers, if they are subscribed to the same topics. It also
> handles failover nicely: new workers register when appear, with
> nothing needed to be done by user.
>
...huh. Looking at your mails for DIRECTORY, it almost seems like adding
subscriptions/filters (with sender-side suppression for non-matching) to
SURVEY would give pretty much exactly The Right Thing, and sounds like a
useful primitive on its own.
In fact, it could be seen as a simple performance optimization of SURVEY in
the case that respondents only reply to a predictable subset of surveys.
Might be worthwhile to implement it.
As for how the subscriptions would actually go upstream, maybe they start
with a default subscription of zero length (matches anything in a trie, and
doesn't break back-compat), they can add more subscriptions, and when
they're done they (explicitly) unsubscribe from ""?
In the case that they aren't worried about missed surveys, they could also
unsub("") immediately.
The respondent would still need to filter locally, but at very least SURVEY
doesn't have the unidirectionality constraint of pub/sub. That makes pushing
subscriptions upstream much, much easier.
Another decent trick might be making it such that when a message that
matches the local filters arrives, the respondent knows the surveyor doesn't
have its subscriptions, and reprocesses them.
Lastly, it might be good to have both "positive" and "negative"
subscriptions at the protocol level. Mathematically, that's not hard to do -
and it'd give a nice way for respondents to tell the surveyor "I didn't need
to hear that" without reprocessing _all_ of their subscriptions. If they
instead send the broadest single negative selector that a.) would exclude
the message and b.) doesn't conflict with their explicit positive
subscriptions, one message greatly reduces traffic, allowing the respondent
to avoid flooding without flooding the surveyor.
From there, they could "pace out" upstreaming their real subscriptions, to
avoid load explosion.
Heck, maybe implement "positive" and "negative" as "sub" and "unsub" - if
your unsub (resp sub) exactly matches a previous sub (resp unsub), simply
remove the positive (resp negative) trie entry. When walking the trie,
always walk to the leaves, and the last match per endpoint wins.
That would Just Work if they start with a positive sub on "".
Other related posts:
