[nanomsg] Re: issue with "received" message size
- From: Garrett D'Amore <garrett@xxxxxxxxxx>
- To: Martin Sustrik <sustrik@xxxxxxxxxx>, nanomsg@xxxxxxxxxxxxx
- Date: Tue, 11 Mar 2014 07:37:10 -0700
On March 10, 2014 at 11:21:24 PM, Martin Sustrik (sustrik@xxxxxxxxxx) wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 10/03/14 22:24, Garrett D'Amore wrote:
>> By the way, I’m calling my package “sp” (as in
>> bitbucket.org/gdamore/sp) rather than any reference to nanomsg.
>> It “seems” that the intent is that “SP” is a reference to the
>> protocols, and “nanomsg” is the specific (reference)
>> implementation thereof. If I’ve misunderstood, please let me
>> know — I can rename the package trivially.
Yes, that's the idea. SP is the name for the set of protocols, nanomsg
is the implementation. That being said, naming a package "sp" sounds
like asking for a namespace collision. Anyway, it's your call.
In Go, you use the full path name to the package to qualify, so its technically
“bitbucket.org/gdamore/sp”
I suck at names. One of the longest lived, and coolest, programs I ever wrote
is still in use at a major enterprise.. it is called “setup”. Which doesn’t
even begin to describe what it does.
> I've also wanted to implement a limit on connections handled by a
> single nanomsg socket, but I've never got so far.
>
>> I think rather than a fixed limit, I might like to see a rate
>> limiter. This would prevent a hard DoS, and give other legitimate
>> connections a chance to get through. :-)
You mean dropping the highest-throughput connections instead of the
last ones to arrive? The question then is how does high throughput
make a connection illegitimate/malevolent?
Indeed, its probably the reverse. DoS attackers are more likely to be able to
attach the SP protocol itself, without any knowledge of the application
protocol underneath. Presumably most server apps would just close a connection
from an attacker that is sending it garbage application layer data.
What, I meant is that rather that creating a fixed limit on the total number of
connections, we would create a limit on the number of new connections per time
interval that we accept. That combined with the approach of tuning the max
receive size, and some sort of security checks at the application layer, would
do a lot to mitigate the risk of attack.
But the more I think about this, the more I think that using transport security
(TLS) is a more robust fix against this problem. That inherently limits the
connections we are going to be able to create. We could even impose a limit of
connections (probably quite small) on the total number of connections using the
same TLS client credentials. I need to think about this a bit more, as I know
not everyone wants to use client authentication with TLS.
>> And quite possibly this should be applied not on a single nanomsg
>> socket, but on something “lower” — like an endpoint. (I’ve not
>> examined your code fully here yet, but I imagine it might be
>> possible to set up multiple “accept()” calls to handle different
>> endpoints and different priorities. I got the sense this is
>> where you were headed with the priority stuff. So we might want
>> to rate limit connections coming from the internet, but not rate
>> limit connections coming from localhost or even our local LAN.)
>> I’ve some thoughts on rate limiting, if you’d like to explore we
>> can discuss later.
That can in fact be solved by receive priotities. Which are at the
moment not exposed by the API -- because we haven't had clear use case
for them.
The idea being that with a lot of messages coming in, those from
higher priority connections are being received by the user first. In
other words, messages from lower priority connections are not passed
to the user, thus TCP pushback is applied effectively resulting in
rate-limiting of incoming data.
The proposal thus: Let's re-introduce receive priority socket option.
Sounds good to me.
>> I’ve no idea what the API for that would like though.
>
>> I’ve also been thinking to implement a few things that will be
>> easier in Go than in C (and I love C — I’m a kernel/driver guy at
>> heart, after all.)
>
>> * tls/ssl transport. This will work exactly like the tcp
>> transport, except that an SSL/TLS handshake shall be performed
>> before the SP headers are exchanged. The details of
>> certificates, verification, etc. will be determined by the
>> application and passed in a crypto.tls.Config object as with
>> other SSL/TLS connections in Go.
Sounds good, although API details may get a bit messy afaict.
They would be in the C API. I think that’s largely because there is no “clean”
way to add SSL/TLS to C applications at present. The OpenSSL and GnuTLS APIs
each have problems here.
The situation for Go is vastly simpler. I could have a socket option to
configure TLS parameters, and assume some defaults if none is provided. This
is because Go gives a nice API for TLS that uses a form of polymorphism so that
applications can treat TLS connections as if they were just TCP. So its an
almost trivial extension to add this to my Go implementation.
One thing that would be nice to include would be a way to obtain information
about the connected peer — particularly if the client cert has been presented.
This then can be used not just for secrecy but for authentication as well. I
need to think about that some more — there are probably some bigger
ramifications here because nanomsg doesn’t currently offer any kind of API
identification of peers.
(I’m thinking that what is really desirable here — and again easier to do in my
new implementation in Go — is an optional extended form of recv that includes
information about the party the message was received from. This could be a
handle to get the peer’s IP address, certificates (if using TLS), or other
kinds of information.)
>> * websocket. ws:// I still need to think about this one — not
>> sure what the exact semantics here should be, but I guess we need
>> an application ID as part of the address.
This actually consists of two parts.
First, there's connection multiplexing: Allow all applications to use
port 80, while dispatching the incoming connections to different SP
sockets. The dispatch should be done based on the URL, I assume.
As for my thoughts on port sharing, have a look here:
http://250bpm.com/blog:18
In the long run we should strive for something like that, but it's
kind of functionality that logically belongs to the kernel and it is
not trivial to implement in the user space.
Sharing incoming “connection” ports, for the initial connect, seems easy
enough. It may be that there is enough need for scalability to justify
avoiding the TCP connect phase for different applications, but I think I’d
prefer to avoid reinventing TCP. ;-)
websocket on the other hand seems vastly simpler. We’d just need to have an
agreement about what the Sec-WebSocket-Protocol values ought to be.
There is also SPDY, which seems to try to address some of the same problems
related to port sharing. Again, I’d hate to reinvent any wheels if other
people with more resources (Google!) are already tackling the problem.
Second, there's the simple part: actual passing of messages via ws.
For both TLS and WS we may also need additional RFCs, although maybe
short ones.
The TLS one would basically be the same (or most likely reference it) as the
TCP one, except that it would indicate that the channel had previously been
established via
>> * sp:// — this is an idea I’ve been thinking about, which is
>> tunneling SP within SP. There might be some weird things we can
>> do to use those to escape firewalls or provide out layer
>> encryption. I’m still noodling through those in my head, but I
>> can imagine some useful cases. I guess the idea here would be to
>> provide a “connection” like semantic built on top of req/rep, and
>> then use that as the underlying transport for whatever other
>> protocols are involved. Sort of like an “SP-specific-VPN”.
Would love to hear more about the use cases.
I need to think about it more, but effectively the ideas I have relate mostly
to attempts to simulate end-to-end security by treating an SP connection as a
byte stream, and then running pseudo-TLS on top of that, and then stuffing the
final application layer SP on top of *that*. Again, I have a bit more thinking
to get this all worked out, but I think tunneling can offer some creative
solutions to some otherwise gnarly problems.
— Garrett
Other related posts:
