[nanomsg] Re: WebSocket Transport design considerations
- From: Jack Dunaway <jack@xxxxxxxxxxxxxxxx>
- To: nanomsg@xxxxxxxxxxxxx
- Date: Thu, 16 Oct 2014 14:35:59 -0500
For clarity, this issue is only for the opening handshake. Once the
handshake is complete, the sockets then talk on a (relatively) fixed-width
header binary protocol. It's just the opening handshake with the issue of
non-fixed width HTTP. (for reference RFC 6455
https://tools.ietf.org/html/rfc6455)
Best regards,
Jack R. Dunaway | Wirebird Labs LLC
On Thu, Oct 16, 2014 at 1:33 PM, Garrett D'Amore <garrett@xxxxxxxxxx> wrote:
> This all sounds... strange.
>
> Maybe because you're trying to handle websockets directly, and so wind up
> basically writing an HTTP handler?
>
> It strikes me that once connected, web sockets act like a TCP stream. Or
> should. I have no idea how Windows and CGI express this, but that would be
> the route I'd have gone. The only "tricky" part is getting this hooked to
> a web server for the serving side. (On the client side, I'd hope this is
> basically abstracted away in a library call that looks like of like a
> connect() call with some extra parameters?)
>
> If you're going to write the web sock handler / handoff directly, I'd
> think you wind up basically doing a non-blocking read, parsing the header
> as it comes in (lines at a time), until the HTTP/1.1 header is complete.
> Once you've finished the handshake, you're in normal TCP transport mode, I
> think, right?
>
>
> On Thu, Oct 16, 2014 at 10:20 AM, Jack Dunaway <jack@xxxxxxxxxxxxxxxx>
> wrote:
>
>> Thank you to Paul and Garrett for weighing in. As an update on progress,
>> adding the ws:// protocol to nanomsg has progressed nicely with no external
>> dependencies. I could use some guidance on one remaining question below,
>> and will then be ready for Autobahn testing and pushing to a public repo.
>>
>> The problem: Since WebSockets send an unknown length HTTP syntax opening
>> handshake, the strategy for receiving this handshake is more complicated
>> than the elegant fixed-width handshake headers of other nanomsg transports
>> (see:
>> https://github.com/nanomsg/nanomsg/blob/master/src/transports/utils/streamhdr.c#L180
>> ).
>>
>> The complication arises since nn_usock_recv() dispatches an async worker
>> whose completion is determined only once all bytes requested have been
>> received. Whereas, since the WebSocket handshake is a termination sequence
>> delimited protocol, we must scan the input stream for the CRLFCRLF termseq
>> as the final token before determining success.
>>
>> Things I've tried (followed by a tl;dr question at the end):
>>
>> 1) On initializing the handshake, starting a nn_usock_recv() operation
>> requesting an arbitrarily large number of bytes, and immediately starting a
>> backoff timer as well. As the backoff timer times out, the buffer to which
>> the underlying socket worker is writing is parsed. The three outcomes of
>> this parse are 1) valid and accepted opening handshake; commence to send
>> response, 2) valid but unacceptable opening handshake; commence to send a
>> failure message then close the connection, and 3) termseq not yet
>> encountered; try again on the next backoff timer timeout. (An implicit
>> fourth mode exists as well, and that is if all requested bytes are
>> received, this will likely result in a failure of the handshake) The first
>> case is the tricky case -- the nn_usock_recv worker continues attempting to
>> receive the arbitrarily-large number of bytes, whereas it should shut down
>> once the opening handshake has been fully received
>> 2) Modifying the nn_usock_recv() interface (
>> https://github.com/nanomsg/nanomsg/blob/master/src/aio/usock.h#L88) to
>> have options of either synchronous or async receive (analogous to
>> NN_DONTWAIT). Which ties in closely to the third investigation...
>> 3) Introducing nn_usock_recv_raw() to the Windows usock, and also making
>> this a public method on the usock object (
>> https://github.com/nanomsg/nanomsg/blob/master/src/aio/usock_posix.inc#L1037).
>> Porting the POSIX method to the Windows side, I got tangled in a web of
>> learning about WSA and completion ports and overlapped I/O and... and
>> decided to consult the mailing list on where to head.
>> 4) Incrementally increasing the socket timeout as the backoff timer so
>> that each time the usock worker times out is the signal for the parse to
>> occur. However, it appears this is not a valid strategy, since socket
>> options cannot be set during the handshake (
>> https://github.com/nanomsg/nanomsg/blob/master/src/aio/usock_posix.inc#L248),
>> and also because on Windows "socket options *SO_RCVTIMEO* and
>> *SO_SNDTIMEO* apply only to blocking sockets" (
>> http://msdn.microsoft.com/en-us/library/windows/desktop/ms741688(v=vs.85).aspx
>> )
>>
>> Things I have investigated but not tried: upgrading the backed worker to
>> complete on a termination sequence rather than on number of bytes received.
>> This is perhaps more performant and more elegant than polling with a
>> backoff timer (or at least, encapsulates this action), yet represents a
>> fundamentally "not so good thing" that could pollute the library with a
>> feature that is only incidentally required for the handshake of one
>> transport. For now, polling with the backoff timer feels a reasonable first
>> goal, and perhaps even the better long-term goal.
>>
>> Conceptually, I feel that #3 above is the most desirable path. In
>> psuedocode, receipt of the opening handshake looks like: 1) Start backoff
>> timer 2) on timeout, call a non-blocking synchronous recv command (no
>> worker), 3) parse, 4) on success, continue handshake; if not enough
>> information has been received increment backoff and goto 1
>>
>> The next best option: 1) Start async recv worker, 2) Start backoff timer,
>> 3) on timeout, parse the buffer to which the async worker is currently
>> writing 4) on success, stop worker and continue handshake; if not enough
>> information has been received, goto 2
>>
>> tl;dr question: Given the current methods available, or suggesting a
>> modification to the usock worker, what is the best way to coordinate
>> polling the recv/parse sequence required here? I would be excited to hear
>> this is a simple problem and I'm missing something obvious!
>>
>> Best regards,
>>
>> Jack R. Dunaway | Wirebird Labs LLC
>>
>> On Fri, Oct 10, 2014 at 8:02 AM, Garrett D'Amore <garrett@xxxxxxxxxx>
>> wrote:
>>
>>> There is an rfc for tcp+tls and mangos conforms to that protocol. I had
>>> started enhancing libnanomsg for this as well but got sidetracked and
>>> didn't get far.
>>>
>>> I would be happy to add websocket support to mangos as well. It should
>>> only take a couple of hours. Go makes this very easy. :-)
>>>
>>> Sent from my iPhone
>>>
>>> > On Oct 10, 2014, at 4:28 AM, Paul Colomiets <paul@xxxxxxxxxxxxxx>
>>> wrote:
>>> >
>>> > Hi Jack,
>>> >
>>> >> On Fri, Oct 10, 2014 at 7:38 AM, Jack Dunaway <jack@xxxxxxxxxxxxxxxx>
>>> wrote:
>>> >> I have begun investigating development of a WebSocket transport for
>>> nanomsg,
>>> >> and I'm curious what design/implementation/brainpower others have put
>>> into
>>> >> this already.
>>> >>
>>> >> The first stage "hack" in this investigation has been retrofitting the
>>> >> existing TCP transport
>>> >> (https://github.com/nanomsg/nanomsg/tree/master/src/transports/tcp)
>>> to
>>> >> dispatch different handshake and send/recv handlers based on transport
>>> >> scheme.
>>> >>
>>> >> Based on a day's worth of hacking, here are a few early-stage design
>>> >> considerations:
>>> >>
>>> >> 1a) It feels heavy-handed to copy-and-paste the entire TCP transport
>>> >> wholesale, just to change a few key areas where WebSockets differ
>>> (e.g., on
>>> >> the surface, only STCP needs non-trivial modification). On the other
>>> hand,
>>> >> the IPC transport very closely resembles TCP, so perhaps there's
>>> merit in
>>> >> this duplication? That said, does it make more sense to duplicate the
>>> TCP
>>> >> protocol as a starting point for the WS protocol, or to extend the
>>> existinga
>>> >> TCP?
>>> >
>>> >
>>> > Probably with current codebase you just have to copy. In fact IPC
>>> > (unix sockets), transport is even more similar to TCP, but they don't
>>> > share codebase. I think it's reasonable trade off.
>>> >
>>> >> 2) There exist several WebSocket protocol parsing libraries in C;
>>> however,
>>> >> I'm not readily finding a sweet spot of License + Quality + RFC 6455
>>> >> coverage. Ideas?
>>> >
>>> > I think it's easy enough to write your own. You can use autobahn as a
>>> > test suite.
>>> >
>>> >> 3) Since the WebSocket protocol does not have a fixed-width header
>>> like the
>>> >> nanomsg TCP wire-level protocol
>>> >> (
>>> https://github.com/nanomsg/nanomsg/blob/master/rfc/sp-tcp-mapping-01.txt#L68
>>> ),
>>> >> this presents a fundamental new challenge buffering and parsing an
>>> incoming
>>> >> stream. Thoughts?
>>> >
>>> > I don't think it's really complex stuff.
>>> >
>>> >> 4) For the ws:// protocol, how should a client negotiate scalability
>>> >> protocol? Via parameters on the URI (e.g., ws://
>>> 127.0.0.1:9876/?sp=NN_SUB)?
>>> >> And, what shall be the value of Sec-WebSocket-Protocol?
>>> >
>>> > I'd say "Sec-WebSocket-Protocol: x-nanomsg-sub"
>>> >
>>> >> 5) Does it make sense to attack this problem of nanomsg<->WebSockets
>>> as a
>>> >> specific application use-case, or as generally desirable transport
>>> >> considered for inclusion into the core nanomsg library?
>>> >
>>> > Don't make a transport if you need websockets for specific application
>>> > usage. Make a "gateway" -- a separate process which does translation,
>>> > using any available http/websocket library.
>>> >
>>> >> 6) wss:// ... ?
>>> >
>>> > Yes, sure. Use libressl :). If you will be doing it, consider also
>>> > looking at tcp+ssl implementation (latter should probably be
>>> > compatible to mangos).
>>> >
>>> >
>>> >
>>> > --
>>> > Paul
>>> >
>>>
>>>
>>
>
Other related posts:
