[nanomsg] Re: understanding of BUS protocol

  • From: Martin Sustrik <sustrik@xxxxxxxxxx>
  • To: nanomsg@xxxxxxxxxxxxx
  • Date: Thu, 10 Apr 2014 09:33:55 +0200

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Garrett,

On 10/04/14 09:14, Garrett D'Amore wrote:
> I implemented a BUS protocol recently on my SP implementation, and
> I didn’t have nanomsg sources handy.  I went back to look at
> nanomsg and I found some discrepancies.
> 
> So, in *my* implementation, I treat XBUS and BUS identically.  In
> both cases, the following are done:
> 
> * there is exactly one server in the bus (the one that runs
> listen/accept)
> 
> * there may be many clients in the bus.  (i.e. the “bus” is really
> a star topology bus, like ATM for folks who recall such tech)
> 
> * send from a client sends to the server, and the server
> rebroadcasts to all connected clients *except* the one that
> received the message.  it also keeps a copy of the message for
> recv() for itself.
> 
> * send from the server broadcasts to all connected clients.
> 
> * recv at the server rebroadcasts (excluding the originator) *even
> if* the server upper queue is full (meaning there is no requirement
> for the application to recv() at the server to allow messages to
> rebroadcast
> 
> * the messages on the wire are passed around without any extra
> header
> 
> * the messages passed to user (for both protocols) are also
> unmodified — no header is added needed
> 
> I *think* that nanomsg behaves mostly the same.  HOWEVER, I do
> notice that it appears that XBUS is a little different.  In
> particular, it appears that XBUS adds the received pipe information
> as part of the header to the inbound message, and on sends
> optionally uses that to determine who to resend to.  It looks like
> the idea is to give the message to the server, and require the
> application on the server to do the rebroadcast.  This seems kind
> of silly, and fragile, although it helps that I have a reference
> counting system on messages so that I don’t have to copy data to
> rebroadcast.
> 
> The questions thus are:
> 
> a) is my interpretation of the BUS protocol code correct?  (I
> suppose I could write some tests… :-)

BUS protocol is trying to simulate broadcast-style messaging as
commonly used in the financial services industry.

To get an idea, check PGM (RFC 3208).

In short, any message sent by anyone gets delivered to everyone else
in the topology. The protocol is not fully reliable.

It's not a nicest communication pattern out there, but it's used by a
lot of applications, so nanomsg tries to support it.

As for the actual implementation, it's not set in stone and if you
find a better way to implement it, we can change nanomsg as well.

Finally, the tagging of the message is done so that when a nn_device()
gets a message, it will route it forward to everyone, except for the
peer it received it from. If this was not the case, any device in the
topology would result in infinite sending of the same message back and
forth.

Once again, if you have a better idea about implementing devices in
BUS topologies, feel free to propose it.

> b) what are the opinions folks have on the semantics?  I happen to
> think my semantics are more elegant for the application.  But maybe
> I’m missing some point?  I *definitely* think that these semantics
> are better for BUS.  I can imagine that for XBUS, giving this to
> the application *may* be interesting.  But I really really don’t
> like giving pointers to applications as part of the message
> contents — even in the header.  :-)

I think assuming there's only one device in the topology is too limiting.

The typical use case for more complex topologies would by two
datacenters, connected via WAN. In each datacenter the local bus runs
on top of PGM, but there's a device in each datacenter that consumes
the PGM messages and sends them to the other datacenter via TCP. At
the same time it consumes TCP messages from the remote datacenter and
re-boradcast them on the local network.

Does that make sense?

> c) If my API semantics are different from nanomsg’s (but the wire 
> protocol is compatible), is this going to cause grief?

The semantics of the socket are part of the spec (see the req/rep
RFC). That ensures that a topology will behave in a consistent and
predictable manner, even if composed from different implementations of
the protocol (similarly to. say, how TCP specification specifies
semantics of packet re-retransmission).

Martin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJTRkljAAoJENTpVjxCNN9YgLoIAJszCWqELr7vgLuraODZPWux
96GDe5XkVvC89fTwO8RBWnaoi+pHM5hdOAfZK4HxmEm1DXVptpBP91py0DwKDs1S
j1UvJmUCKGRZileJrBOCRl1cGkhRftpI4LGG1iMpluhhwAhBo2wYLEUjzZFFCYrn
nGb8QQCo55j6mvl6+IAn4W7aMbLq5VSeSbkwA0G5lHRbp52Qn7Wl/ZqARHHXZreU
fSo2mZojXz0a7ptdM7GlklmOIHud+XUhFDtIojdj9bd9rWyCMlr67/2TENtWtkMg
MmM5fYbdttnsqufeUyjWIBMfrSUo4eyQLJhM26AAPvBDbvMRtK7/360I7NAMeQE=
=IGxE
-----END PGP SIGNATURE-----

Other related posts: