[nanomsg] Re: websocket mapping
- From: Garrett D'Amore <garrett@xxxxxxxxxx>
- To: nanomsg@xxxxxxxxxxxxx
- Date: Tue, 3 Feb 2015 08:36:10 -0800
> On Jan 30, 2015, at 9:14 AM, Jack Dunaway <jack@xxxxxxxxxxxxxxxx> wrote:
>
> Hi Garrett, this is exciting news! I'm glad to hear you've worked through the
> problem thoroughly. I acknowledge the problems you bring up, struggled with
> them myself on the C implementation without a solid resolution, and like
> where you're headed with further development on this. I say we figure it out
> and bring Mangos and nanomsg core up to spec with an updated
> sp-websocket-mapping-01.txt
> <https://github.com/nanomsg/nanomsg/blob/master/rfc/sp-websocket-mapping-01.txt>.
That’s the plan. I just want to arrive to consensus before I start doing the
work on nanomsg. I tried to do the separate exchange, and frankly the state
machines in the C nanomsg made this a major PITA. (Whereas in Go it was quite
trivial.) Having actual threads makes life so *vastly* much better. (I’ve
sometimes considered writing a new C version of libnanomsg that used pthreads.)
>
> I'm a proponent of fully encapsulating all nanomsg-specific handshaking into
> the RFC 6455 opening handshake, rather than as a follow-on frame, because ws
> already provides sufficient and desirable framing to achieve all that's
> needed for nanomsg-specific negotiation, including symmetric peer validation
> of protocol/role socket types by both peers.
>
> Currently, the C implementation is deficient by not advertising the "server"
> SP types back to the connecting "client" (see:
> https://github.com/nanomsg/nanomsg/blob/master/src/transports/ws/ws_handshake.c#L1277-L1283
>
> <https://github.com/nanomsg/nanomsg/blob/master/src/transports/ws/ws_handshake.c#L1277-L1283>).
> If it did, the client would then be able to perform peer compatibility check
> using `nn_pipebase_ispeer()`. My suggestion is that the server, in addition
> to acknowledging the client's protocol with the "Sec-WebSocket-Protocol"
> header (an RFC 6455 requirement), also advertise its own protocol with
> "Sec-WebSocket-Protocol-Server" (as per RFC 6455 4.3). This gives the
> initiating socket the ability to fail the connection before the handshake is
> complete.
The RFC gives no detail on the contents of the Sec-WebSocket-Protocol-Server
header. (Likewise for the -Client field.) But more importantly, many
websocket APIs give no ability to access those fields. In fact, the standard
HTML5 API only give access to the Protocol header:
http://dev.w3.org/html5/websockets/ <http://dev.w3.org/html5/websockets/>
So adding extra header fields here is problematic.
>
> Incidentally, the C implementation is already parsing this key
> (https://github.com/nanomsg/nanomsg/blob/master/src/transports/ws/ws_handshake.c#L1127-L1131
>
> <https://github.com/nanomsg/nanomsg/blob/master/src/transports/ws/ws_handshake.c#L1127-L1131>),
> just not using it yet.
>
> Note: Reading RFC 6455, I'm uncertain whether Sec-WebSocket-Protocol-Server
> is a replacement for or to be used in conjunction with repeating
> Sec-WebSocket-Protocol choice requested by the client. Either way, regardless
> the name of the header -- my intention is that the server sends both the
> client's protocol (an RFC 6455 requirement) and a separate header with the
> server's protocol/role (a nanomsg requirement).
So this is the problem — we have mismatched protocol values. The client and
server don’t use the same value.
The way I’ve decided to handle this *relies* on the fact that there is a 1:1
value between a protocol and its valid peer. (REQ & REP for example).
Sometimes they are the same (e.g. PAIR, BUS, etc.)
>
> As for the naming conventions of the protocol/role exchange -- would you see
> any problem in encoding all the information into the "Sec-WebSocket-Protocol"
> header field? In the form
> "nanomsg-<version>-<scalability_protocol>-<endpoint_role>"? For example, a
> REQ socket sends "Sec-WebSocket-Protocol: nanomsg-0-3-0" and a REP socket
> sends "Sec-WebSocket-Protocol: nanomsg-0-3-1". (also note: dropping 'x-'
> prefix as Dirkjan pointed me to:
> https://tools.ietf.org/html/draft-saintandre-xdash-considered-harmful-01
> <https://tools.ietf.org/html/draft-saintandre-xdash-considered-harmful-01>)
Well, it turns out that using numbers works for me, but critically the sever
and client *MUST* send the same value in order to be compatible with existing
WebSockets framework. Sending different values WILL NOT WORK. So I took the
approach that the client asks for the server’s protocol, and the server socket
type is what is used in the exchange. If you’re willing to code the knowledge
of socket types & peer types in the implementation, then this isn’t a big
problem. It can lead to some violation of layering boundaries in the library
implementation, but actually I found this not too bad to do within mangos. The
only real problem is that there is not a good way to support mixing more than
one peer type — e.g. it would be problematic if we ever wanted to make a
protocol type that supported connecting to more than one peer type. There are
no such protocol types at present.
>
> Ideally, these pieces of information would be split into separate headers,
> for clarity and simplicity (no further encoding/decoding required). But as of
> today, Javascript support for all web browsers seems to only support setting
> the "Sec-WebSocket-Protocol" header (even Sec-WebSocket-Extensions is not
> accessible to be set by app code), and this does not appear to be changing
> anytime soon: http://dev.w3.org/html5/websockets/
> <http://dev.w3.org/html5/websockets/>
Exactly my earlier point.
>
> Another reason to fully-encode the handshake in the initial frame is it
> enables nanomsg/ws implementations to be tested with Autobahn TestSuite. A
> follow-on frame would require nanomsg-specific modifications to the test
> suite, or dropping this test suite altogether.
>
> > "I see almost no value in attempting to make nanomsg a general purpose
> > websocket library"
> 100% agree! Though, I see immense value in ensuring nanomsg provides
> idiomatic, simple integration with WebSocket clients in browsers, which
> effectively just means using text instead binary protocols.
Yes. Although some clients now support binary encoding, you can’t rely on broad
support. Probably people building these kinds of systems are just passing JSON
frames around. :-)
>
> Summary -- I'm not diametrically-opposed to the follow-on frame as part of
> negotiation, but feel that it's a complexity that could be simplified using
> stock transport framing to make browser clients simpler and remain compatible
> with industry-best test suite.
Sure.
- Garrett
>
> Best regards,
> Jack R. Dunaway | Wirebird Labs LLC
>
> On Fri, Jan 30, 2015 at 6:36 AM, Garrett D'Amore <garrett@xxxxxxxxxx
> <mailto:garrett@xxxxxxxxxx>> wrote:
> I’ve started (and mostly completed) a websocket mapping of mangos. I started
> hoping to make it compatible with nanomsg’s mapping as specified in the
> websocket nanomsg RFC, but as I got into details, I discovered somethings
> about this RFC that I think are a big mistake.
>
> The biggest problem is that we are relying on the Protocol field to express
> which SP topology is used. This results in at least these problems:
>
> a) The transport needs to know the name of all protocols, which does
> not facilitate expansion and badly breaks the layering abstraction.
>
> b) Some implementations don’t make it easy to pick up on the values
> used by the server and client protocols. For example, the Go websocket
> implementation lets the client specify the protocols it want(s) to use, but
> it cannot see what the advertised protocol on the server side is.
>
> c) Its likely that some implementations are going to have trouble
> when x-nanomsg-req != x-nanomsg-rep — i.e. we cannot have mismatched
> protocols.
>
> I think problem “a” could be easily fixed by using numerics encoded as
> strings, but problems b & c are more severe — I think they come about from a
> basic design philosophy difference — in websocket I think the intention is
> that the ultimate “negotiation” is done on the server side, based on what the
> client claims to desire. Its not really a two-way neg like SP is.
>
> Thinking about this quite a bit, I’ve decided that the best thing would be to
> change the mapping as follows:
>
> 1. The protocol field should just be the value “x-nanomsg” — without any
> reference to the subprotocol.
>
> 2. A message exchange very much like what is is done for TCP should be done
> at when the connection is established. That is each side should send a
> “header”, and then should receive and validate the peer’s header.
>
> 3. Given that some websocket clients really need to deal with text values
> (e.g. older web browsers can only send strings), the content of the header
> should be as follows:
>
> “SP0:<protocol>:0” — these are ASCII characters forming a string.
> The SP0: indicates SP version 0, the <protocol> is a numeric protocol (for
> example PUB would 32, and SUB would be 33) represented in decimal ASCII, and
> the :0 is reserved for future protocol enhancements (like the reserved fields
> in the binary header exchange). So for example, a REQ client would send
> SP0:48:0 and a REP server would send SP0:49:0
>
> This should facilitate working with more websocket implementations, although
> to be clear the header exchange would prevent using this library to talk to
> arbitrary websocket services — peers would only match and peer each other if
> they are running the proper SP peer protocol. I see almost no value in
> attempting to make nanomsg a general purpose websocket library.
>
> Thoughts?
>
> - Garrett
>
>
>
Other related posts:
