[nanomsg] Re: patterns for complex applications, pub/sub filtering, tls+tcp;//, and tunneling transports

  • From: "Garrett D'Amore" <garrett@xxxxxxxxxx>
  • To: "nanomsg@xxxxxxxxxxxxx" <nanomsg@xxxxxxxxxxxxx>
  • Date: Wed, 27 Aug 2014 10:05:14 -0700

I'm not sure it reduces the complexity.  It does reduce the amount of
configuration at the TCP layer, but you still have to have that complexity
in the application layer.  (How do you differentiate between PUB/SUB and
REQ/REP, for example?  Or between different REQ/REP "subprotocols"?  You
wind up injecting a new value, which looks unsurprisingly like a port
number, in the "application layer" header.

Plus you then have all the head-of-line blocking challenges, and you wind
up basically forcing everything to use the same reliable delivery stream.
 Which is possibly good for req/rep, but not so good for pub/sub where your
delivery requirements are quite a bit different.

You also ruin any chance of using all that fancy classification stuff that
you have in routers, NICs, and switches, to do any kind of selective
processing.  (Maybe I want to run REQ/REP at a higher priority, but my
Pub/Sub I'm willing to drop on the floor.)

So you're just moving complexity, not reducing, IMO.


On Wed, Aug 27, 2014 at 9:18 AM, Achille Roussel <achille.roussel@xxxxxxxxx>
wrote:

> I've actually thought some sort of TCP muxing would be great, I'm having a
> hard time understand why we limit ourselves to one protocol per endpoint.
>
> The interest I guess is more simplicity and flexibility in a nanomsg-based
> system. Let's say you have a TCP + REQ/REP infra, that's bound to a network
> address and port. Then your product evolves and you want to add a PUB/SUB
> feature to that system... You must use a different port number, so if you
> had load balancers, health checks or firewalls in the middle you have to
> configure them for a new port.
> Also if there are multiple "client" nodes then it can become error-prone
> to deal with multiple port... A configuration mistake is easy to make and
> may be hard to find, with REQ nodes trying to connect to a PUB port for
> example. If there's only one port it reduces the system's complexity.
>
> Achille Roussel
> +1 415 490 6339
>
> > On Aug 27, 2014, at 8:38 AM, Garrett D'Amore <garrett@xxxxxxxxxx> wrote:
> >
> > There is an implementation of tls+tcp but it is in my mangos
> implementation which is written in go. It would be trivial to use the
> device framework with mangos to have a TLS device in front of a traditional
> libnanomsg c program thus giving TLS support to your c program.
> >
> > I had intended to write an OpenSSL based version in libnanomsg as well
> but I've been doing other stuff.
> >
> > Nanomsg is definitely endian safe as is mangos.  I have run both on both
> big and little endian platforms.
> >
> > Pub side filtering is hard because you have to provide reliable delivery
> of the publication requests and reliable reply when the connection fails.
> It's really tricky when you start having repeaters etc in the network.
> >
> > I have to admit that I don't understand the use cases for muxing on a
> single tcp conn.
> >
> > Sent from my iPhone
> >
> >> On Aug 27, 2014, at 2:29 AM, Dirkjan Ochtman <dirkjan@xxxxxxxxxx>
> wrote:
> >>
> >> Here's an attempt at some answers:
> >>
> >>> On Wed, Aug 27, 2014 at 11:00 AM, Roman Puls <puls@xxxxxxxxxxxx>
> wrote:
> >>> II) PUB/SUB Filtering
> >>>
> >>> in the context of I), where some files are needed, pub/sub came into
> my mind
> >>> to push the missing piece of information from a producer into all
> workers.
> >>> Am I right that currently the pub/sub subscription is a host-local
> >>> filtering. or do we have sender-side filtering in nanomsg?
> >>>
> >>> If no pub-side filtering: what are the reasons to do that on the
> receiver
> >>> side only, and are there plans to change that to pub-side filtering?
> >>
> >> That's right. I think we'd like to have sender-side filtering at some
> >> point, but we haven't implemented it yet (it has higher complexity
> >> than receiver-side filtering).
> >>
> >>> III) Transports
> >>>
> >>> I have read a few postings here about creating new transports. The
> first
> >>> stuff I'd be interested in (of course) would be to use a "tcp+tls://"
> >>> socket. Is somebody working on that one? If not, do we have a trivial
> >>> example on how to create new transports, i.e. a loop-back transport?
> >>
> >> I don't think anyone has done work on an implementation yet, but we do
> >> have an RFC for tcp+tls:
> >>
> >>
> https://github.com/nanomsg/nanomsg/blob/master/rfc/sp-tls-mapping-01.txt
> >>
> >> I also don't think we have a trivial example for how to create new
> transports.
> >>
> >>> As a second topic: did somebody ever consider to write a multiplexing
> >>> transport, which would allow you to run i.e. a req/rep and a pub/sub
> over
> >>> the same tcp or tls+tcp connection? Head of line blocking is known,
> but can
> >>> be solved here IMO.
> >>
> >> A tcpmux transport has been discussed on this mailing list.
> >>
> >>> IV) Endianness
> >>>
> >>> I have no information found on that topic, and grepping the sources for
> >>> hton-functions, and the endian keyword did not show me any
> entry-points, so
> >>> here yet another question: is nanomsg endianess aware? I don't mean
> user
> >>> payload, but are the headers swapped in network-byte order, so you can
> >>> connect big-endian with a little-endian system?
> >>
> >> I'm not really sure, but searching for "network byte order" suggests
> >> that it's been thought about at least:
> >>
> >>
> https://github.com/nanomsg/nanomsg/search?utf8=%E2%9C%93&q=network+byte+order&type=Code
> >>
> >> Cheers,
> >>
> >> Dirkjan
> >
>
>

Other related posts:

  1. Home
  2. nanomsg
  3. Archive
  4. 08-2014