[nanomsg] Re: Port sharing, tcpmux, web-related roadmap et c.

  • From: Paul Colomiets <paul@xxxxxxxxxxxxxx>
  • To: Martin Sustrik <sustrik@xxxxxxxxxx>
  • Date: Thu, 20 Nov 2014 18:20:13 +0200

Hi Martin,

On Thu, Nov 20, 2014 at 1:22 PM, Martin Sustrik <sustrik@xxxxxxxxxx> wrote:
> Hi Paul,
>
> Thanks for the comments! See my replies inlined.
>
>>> There are no prerequisites for _connecting_ to a tcpmux endpoint. As
>>> for _binding_ to a tcpmux endpoint, you have to have tcpmux daemon
>>> running on the box. You can start the daemon like this:
>>>
>>> tcpmuxd 5555
>>>
>>
>> Which host it listens to?
>
>
> It listens on local TCP port 5555 (all interfaces).
>

Well, what I really mean, is can I specify interface/ip address? I
want that at least for security purposes. (I know I can use firewall,
but I used to limit listening addresses).

>>> However, if we had a default
>>> port, connection strings would look much nicer:
>>>
>>> nn_bind (s, "tcpmux://myservice");
>>
>>
>> 1. It's unclear which host does socket listen to?
>
>
> Default port on the local box. All interfaces.
>

Same here: how would I use different tcpmuxd instance?

>> 2. How it registers with tcpmuxd? What if there are multiple tcpmuxd
>> running?
>
>
> Attempt to start second tcpmuxd fails.
>
> Registration happens over AF_UNIX sockets. The underlying file is
> /tmp/tcpmux-5555.ipc
>
> If there are multiplexers for multiple ports running, they use different
> AF_UNIX address to communicate (replace 5555 by whatever port you are using.
>

Okay. But I believe traditional host-port address should be used. Not
just port number.

>> 3. What happens if multiple processes are trying to bind to socket?
>
>
> You mean bind to the same service name, right? If so, second attempt fails.
> tcpmux transport then retries once in a while.
>

Okay. I then service unregisters when unix socket closed, right?

>>> 2. A similar approach can be used for multiplexing WebSocket
>>> connections. In such implementation, URL can be used as a service
>>> name. Interestingly, the connecting side (client) doesn't even have to
>>> be aware that there is a multiplexer on the other side:
>>>
>>> nn_bind (s, "wsmux://myservice");
>>> nn_connect (s, "ws://www.example.org/myservice");
>>>
>>> Btw, the above looks like a reasonable first step towards integration
>>> of nanomsg with the web. Imagine following set of features:
>>>
>>> a.) Simple JavaScript client.
>>> b.) WebSocket and WebSocket/SSL transports.
>>> c.) Multiplexer, so that no TCP ports except 80 and 443 have to be used.
>>> d.) Possible integration with web servers so that web pages and
>>> nanomsg services can share the same namespace (?)
>>>
>>
>> I believe it still would work for web in general:
>>
>> 1. To keep whole first message in forwarded tcp connection you must
>> read handshake by at most 4 bytes. And websocket handshake is quite
>> long. So your muxer is a subject of DoS attacks easier.
>>
>> 2. The gateway between wild internet and internal service must be DoS
>> protected. Not only nanomsg doesn't provide protection itself, it
>> doesn't even allow you to know which connection is an attacker.
>
>
> Agreed with everything above. But what's the alternative? Any other option
> has to deal with those issues as well, no?
>

Yes. A full gateway aproach, like zerogw does. And see below...

>> 3. It's very expensive to establish many connections to a single
>> server. So I believe one Websocket (from browser) per nanomsg socket
>> is not going to work. Most applications need multiple different
>> services and multiple different patterns to work well. So without
>> tunelling we are going to stuck with bad design decisions, where users
>> would use single socket with single pattern for everything. On the
>> other side if we ever implement tunelling, tcpmux doesn't need to be
>> used.
>
>
> You mean expensive in terms of bytes transferred? If so, fair enough.
>
> However, most arguments against multiple connections don't really stand.
> I've written a blog about that in the past: http://250bpm.com/blog:18
>

Not exactly in terms of bytes transferred. It's ok to open any number
of connections between two services, even if they speak websockets.
But with browsers:

1. The tend to have limits on number of connections (you can have few
tens of services behind a single gateway, and user having 50-100 tabs
is not so uncommon).

2. Websocket connections are usually pinged to keep them closed fast
in case of closing client connection, so traffic is huge

3. Allowing lots of connections per host make server easy DoS attacked
(remember NAT'd clients)

4. Some companies have hundreds of employees behind NAT, so can easily
exhaust number of ports at client-side

>> 4. There are also security checks (for Sec-WebSocket-Origin) that have
>> somehow be configured
>
>
> Dunno. That particular check seems to be pretty weak. AFAICS the client can
> fake any origin it wants. Therefore we can just allow all origins by
> default. If needed, we can add some checks later on.
>

It's not weak. It prevents a javascript on web page
http://malicious.attacker/ to connect websocket to http://gmail.com
with your credentials.

So yes, it's not check against bots. It's check against javascript on
malicious web pages.

>> All in all, It may work for some limited use cases, but would lead to
>> the bad design for the most complex ones.
>
>
> I am wondering about what the alernative would be. Any ideas?
>

The gateway, which does access auth checks and some sort of tunelling.
It's kinda tcpmux on steroids. But, firstly to have some real
authentication and for tunelling, the gateway must read and understand
websocket packets itself

If the gateway does packet-based routing, it may (I would argue it
should) just prefix packet with authentication info and send that
packet over persistent nanomsg connection. I.e. you shouldn't have
nanomsg connection per websocket connection, but a persistent nanomsg
connection between service and gateway.

So it may be an nginx module, or standalone http/websockets server
like zerogw or mongrel, but not a small and transparent one like
tcpmuxd.

-- 
Paul

Other related posts: