Background: Today, we permit folks to set a maximum number of devices (via
nn_device, or nng_device, or the mangos device) up to 255. More devices permit
some interesting scale out opportunities, as well as flexibility in traversing
across network transport boundaries. For example, one can use devices to glue
together two legacy nanomsg applications, which doesn’t understand or support
TLS, but use a mangos or NNG router, to forward messages across a TLS or
websocket firewall. (Or with NNG, a ZeroTier network boundary.)
However, doing this means we have to support inserting additional fields in the
message header, allowing the message headers to grow quite a bit. Right now
with 255, that means up to 1KB of extra header (each hop for REQ/REP or
SURVEYOR adds a 32-bit pipe ID in a routing header).
What I’d like to do is limit this quite a bit – in the interest of using a
preallocated message header and reducing branches during the message processing
and eliminating some heap pressure.
If we reduced the number of transitions to say 15 instead of 255, that means
I’d be able to get away with a preallocated header of 64-bytes. (We are
already preallocating 32 bytes for performance, but its still a dynamic
allocation which can fail.) I have yet to see any application which uses more
than a few devices in series. But – I recognize some folks may have very large
or complex networks. Still 15 hops seems rather excessive, even for folks
using scale out. (Imagine a scale out with just 4 layers, and 100 connections
each – that would be able to support 100 million active connections. I don’t
believe anyone is using any of these protocols at this scale currently – but I
could be mistaken. To be honest to support that requires a rather large
investment in hardware, because that 4th tier is going to need something like
1M nng_device instances. More likely is that we see applications with 1,000+
connections each. NNG and mangos both should be able to handle over 10K
connections per instance.)
Even if each of these device instances also uses an extra device to cross
transport domains, we’re still only talking about 8 devices or 8 “hops”.
Clearly a hop count of 255 is ridiculous.
Would anyone object if I limited this (NNG_OPT_MAXTTL) all the way down to 15?
I think there may be performance gains to be had if I did.
- Garrett
