[nanomsg] Re: issue with "received" message size
- From: Garrett D'Amore <garrett@xxxxxxxxxx>
- To: nanomsg@xxxxxxxxxxxxx, Martin Sustrik <sustrik@xxxxxxxxxx>
- Date: Mon, 10 Mar 2014 14:24:52 -0700
On March 10, 2014 at 12:51:52 PM, Martin Sustrik (sustrik@xxxxxxxxxx) wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi Garrett,
Yes. I've implemented MAXMSGSIZE socket option back in ZeroMQ, but
it's still lacking in nanomsg. The semantics are: If the advertised
incoming message is larger than MAXMSGSIZE, message is not read and
the connection (for connection-based transports) is closed immediately.
Excellent. I’ve implemented in a similar approach in my pure Go
implementation. Right now its a fixed limit unless the caller supplies a
bigger buffer, but I think a dynamic tunable is better. In particular, I’m not
sure that my “mux” implementation in the protocol layer is going to facilitate
the app providing the receive buffer as easily as I’d like.
By the way, I’m calling my package “sp” (as in bitbucket.org/gdamore/sp)
rather than any reference to nanomsg. It “seems” that the intent is that “SP”
is a reference to the protocols, and “nanomsg” is the specific (reference)
implementation thereof. If I’ve misunderstood, please let me know — I can
rename the package trivially.
I've also wanted to implement a limit on connections handled by a
single nanomsg socket, but I've never got so far.
I think rather than a fixed limit, I might like to see a rate limiter. This
would prevent a hard DoS, and give other legitimate connections a chance to get
through. :-) And quite possibly this should be applied not on a single nanomsg
socket, but on something “lower” — like an endpoint. (I’ve not examined your
code fully here yet, but I imagine it might be possible to set up multiple
“accept()” calls to handle different endpoints and different priorities. I got
the sense this is where you were headed with the priority stuff. So we might
want to rate limit connections coming from the internet, but not rate limit
connections coming from localhost or even our local LAN.) I’ve some thoughts
on rate limiting, if you’d like to explore we can discuss later.
But even if a fixed limit is used, being able to move that “down” to specific
endpoints would be highly useful, IMO.
I’ve no idea what the API for that would like though.
I’ve also been thinking to implement a few things that will be easier in Go
than in C (and I love C — I’m a kernel/driver guy at heart, after all.)
* tls/ssl transport. This will work exactly like the tcp transport,
except that an SSL/TLS handshake shall be performed before the SP headers are
exchanged. The details of certificates, verification, etc. will be determined
by the application and passed in a crypto.tls.Config object as with other
SSL/TLS connections in Go.
* websocket. ws:// I still need to think about this one — not sure
what the exact semantics here should be, but I guess we need an application ID
as part of the address.
* sp:// — this is an idea I’ve been thinking about, which is tunneling
SP within SP. There might be some weird things we can do to use those to
escape firewalls or provide out layer encryption. I’m still noodling through
those in my head, but I can imagine some useful cases. I guess the idea here
would be to provide a “connection” like semantic built on top of req/rep, and
then use that as the underlying transport for whatever other protocols are
involved. Sort of like an “SP-specific-VPN”.
The first two are particularly easy in Go because of the excellent standard
library that Go supplies. The latter would be something more … unique.
- Garrett
Martin
On 10/03/14 17:17, Garrett D'Amore wrote:
> I’ve noticed that the message sizes supported by nanomsg are
> expressed as a 64-bit integer, and I’m thinking with NN_MSG is
> passed to nn_recvmsg(), it could expose the caller to tragic DoS if
> the peer is not trusted. In particular, it seems that a peer could
> ask a system to effectively allocate Terabytes of data for a
> message, without actually holding the message itself.
>
> Actually, reading the code, it seems that even if the caller
> supplies the message size to nn_recvmsg(), the underlying transport
> in stcp.c winds up allocating a message body based on the size
> expressed in the received header. So a sender could ask a peer to
> allocate gigabytes, terabytes, or petabytes(!) of RAM to hold a
> message, that it never actually has to send. This makes a denial
> of service attack also trivial; most systems do not hold up well
> under gigantic memory allocation requests.
>
> This seems tragically bad, potentially. I’d really like to see
> instead that there was a way to limit message allocation size, to
> guard servers from bad peer implementations. What I’m thinking is
> an upper bound on message size, that could be tunable via a socket
> option. (And there could be an option to disable the check, e.g.
> set the limit to 0. But this would be not recommended in
> situations where all participants are not trusted.) A default sane
> limit would probably be around 1M. While a big largish, unless a
> client can arrange to connect and hold open thousands of
> connections to the server, the effective damage is probably
> limited.
>
> A better solution would be to calculate the maximum connection rate
> the server can manage multiplied by the “timeout” for idle
> connections, and divide that into the maximum memory pool we would
> be willing to risk. If process is allowed to grow to say 1GB, and
> can open 1000 connections per second, and the timeout is 10
> seconds, then 100KB is the “safe” limit to guard against a network
> assault.
>
> My guess is most upper layer protocols have smaller message sizes —
> e.g. 64K or smaller — and so they could tune this value down to
> further harden their limit.
>
> The other thing that could be done, is to keep track of
> outstanding connections and utilization, and refuse to accept()
> connections once the total utilized buffer space exceeded a
> configured limit.
>
> It would be even more useful if we had the ability, during
> connection establishment, to exchange maximum message size
> information (perhaps using some of those reserved bits), or even
> exchange “error” state. E.g. I’d like to be able to send to my peer
> that its request to send a message of 10GB is denied because the
> message is too big, or at least during connection establishment be
> able to state things like “Sorry your protocol isn’t supported” or
> “Sorry, I haven’t resources available right now, come back later.”
>
> These things could be useful in providing better error reporting
> back to the user (perhaps via logs or real-time error messages) of
> an agent trying to perform the connection.
>
> -- Garrett D'Amore Sent with Airmail
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQEcBAEBAgAGBQJTHhfLAAoJENTpVjxCNN9Y1tAH/1rlXPcxknxS0mBnr9TiUQjJ
PMNHaT3a3sxo2Lb+Q92DuTrUGiaZy6mg6r1YuVzvYQNeq/ZSSkb9JaTakmlgQhBW
bdxb+I8wIcE29KzxUqmq5AtA1efWRWbMbkXLJw0FY25d3aI5JLfSZz8GUvQgY0XD
tDqpaZ9p4ZZtXTukXw9Vk8MFFkCB/2ig0ZhINYicD+dkwdeOLgISedkD2SYreLRt
11mOcZOLxPY0VcCy5bafZSRG76j8CVWXpM/CZIi3/+pkEzYm9R95ZM5Vh829QWhr
gm9xQOEbwPjiX4h2e1pxyeio9HVP3VDRY4O/OgyQYrO+kGo9CCVF9FelKhoswn8=
=coxc
-----END PGP SIGNATURE-----
Other related posts:
