[nanomsg] Re: WebSocket Transport design considerations
- From: "Garrett D'Amore" <garrett@xxxxxxxxxx>
- To: "nanomsg@xxxxxxxxxxxxx" <nanomsg@xxxxxxxxxxxxx>
- Date: Thu, 16 Oct 2014 11:33:34 -0700
This all sounds... strange.
Maybe because you're trying to handle websockets directly, and so wind up
basically writing an HTTP handler?
It strikes me that once connected, web sockets act like a TCP stream. Or
should. I have no idea how Windows and CGI express this, but that would be
the route I'd have gone. The only "tricky" part is getting this hooked to
a web server for the serving side. (On the client side, I'd hope this is
basically abstracted away in a library call that looks like of like a
connect() call with some extra parameters?)
If you're going to write the web sock handler / handoff directly, I'd think
you wind up basically doing a non-blocking read, parsing the header as it
comes in (lines at a time), until the HTTP/1.1 header is complete. Once
you've finished the handshake, you're in normal TCP transport mode, I
think, right?
On Thu, Oct 16, 2014 at 10:20 AM, Jack Dunaway <jack@xxxxxxxxxxxxxxxx>
wrote:
> Thank you to Paul and Garrett for weighing in. As an update on progress,
> adding the ws:// protocol to nanomsg has progressed nicely with no external
> dependencies. I could use some guidance on one remaining question below,
> and will then be ready for Autobahn testing and pushing to a public repo.
>
> The problem: Since WebSockets send an unknown length HTTP syntax opening
> handshake, the strategy for receiving this handshake is more complicated
> than the elegant fixed-width handshake headers of other nanomsg transports
> (see:
> https://github.com/nanomsg/nanomsg/blob/master/src/transports/utils/streamhdr.c#L180
> ).
>
> The complication arises since nn_usock_recv() dispatches an async worker
> whose completion is determined only once all bytes requested have been
> received. Whereas, since the WebSocket handshake is a termination sequence
> delimited protocol, we must scan the input stream for the CRLFCRLF termseq
> as the final token before determining success.
>
> Things I've tried (followed by a tl;dr question at the end):
>
> 1) On initializing the handshake, starting a nn_usock_recv() operation
> requesting an arbitrarily large number of bytes, and immediately starting a
> backoff timer as well. As the backoff timer times out, the buffer to which
> the underlying socket worker is writing is parsed. The three outcomes of
> this parse are 1) valid and accepted opening handshake; commence to send
> response, 2) valid but unacceptable opening handshake; commence to send a
> failure message then close the connection, and 3) termseq not yet
> encountered; try again on the next backoff timer timeout. (An implicit
> fourth mode exists as well, and that is if all requested bytes are
> received, this will likely result in a failure of the handshake) The first
> case is the tricky case -- the nn_usock_recv worker continues attempting to
> receive the arbitrarily-large number of bytes, whereas it should shut down
> once the opening handshake has been fully received
> 2) Modifying the nn_usock_recv() interface (
> https://github.com/nanomsg/nanomsg/blob/master/src/aio/usock.h#L88) to
> have options of either synchronous or async receive (analogous to
> NN_DONTWAIT). Which ties in closely to the third investigation...
> 3) Introducing nn_usock_recv_raw() to the Windows usock, and also making
> this a public method on the usock object (
> https://github.com/nanomsg/nanomsg/blob/master/src/aio/usock_posix.inc#L1037).
> Porting the POSIX method to the Windows side, I got tangled in a web of
> learning about WSA and completion ports and overlapped I/O and... and
> decided to consult the mailing list on where to head.
> 4) Incrementally increasing the socket timeout as the backoff timer so
> that each time the usock worker times out is the signal for the parse to
> occur. However, it appears this is not a valid strategy, since socket
> options cannot be set during the handshake (
> https://github.com/nanomsg/nanomsg/blob/master/src/aio/usock_posix.inc#L248),
> and also because on Windows "socket options *SO_RCVTIMEO* and
> *SO_SNDTIMEO* apply only to blocking sockets" (
> http://msdn.microsoft.com/en-us/library/windows/desktop/ms741688(v=vs.85).aspx
> )
>
> Things I have investigated but not tried: upgrading the backed worker to
> complete on a termination sequence rather than on number of bytes received.
> This is perhaps more performant and more elegant than polling with a
> backoff timer (or at least, encapsulates this action), yet represents a
> fundamentally "not so good thing" that could pollute the library with a
> feature that is only incidentally required for the handshake of one
> transport. For now, polling with the backoff timer feels a reasonable first
> goal, and perhaps even the better long-term goal.
>
> Conceptually, I feel that #3 above is the most desirable path. In
> psuedocode, receipt of the opening handshake looks like: 1) Start backoff
> timer 2) on timeout, call a non-blocking synchronous recv command (no
> worker), 3) parse, 4) on success, continue handshake; if not enough
> information has been received increment backoff and goto 1
>
> The next best option: 1) Start async recv worker, 2) Start backoff timer,
> 3) on timeout, parse the buffer to which the async worker is currently
> writing 4) on success, stop worker and continue handshake; if not enough
> information has been received, goto 2
>
> tl;dr question: Given the current methods available, or suggesting a
> modification to the usock worker, what is the best way to coordinate
> polling the recv/parse sequence required here? I would be excited to hear
> this is a simple problem and I'm missing something obvious!
>
> Best regards,
>
> Jack R. Dunaway | Wirebird Labs LLC
>
> On Fri, Oct 10, 2014 at 8:02 AM, Garrett D'Amore <garrett@xxxxxxxxxx>
> wrote:
>
>> There is an rfc for tcp+tls and mangos conforms to that protocol. I had
>> started enhancing libnanomsg for this as well but got sidetracked and
>> didn't get far.
>>
>> I would be happy to add websocket support to mangos as well. It should
>> only take a couple of hours. Go makes this very easy. :-)
>>
>> Sent from my iPhone
>>
>> > On Oct 10, 2014, at 4:28 AM, Paul Colomiets <paul@xxxxxxxxxxxxxx>
>> wrote:
>> >
>> > Hi Jack,
>> >
>> >> On Fri, Oct 10, 2014 at 7:38 AM, Jack Dunaway <jack@xxxxxxxxxxxxxxxx>
>> wrote:
>> >> I have begun investigating development of a WebSocket transport for
>> nanomsg,
>> >> and I'm curious what design/implementation/brainpower others have put
>> into
>> >> this already.
>> >>
>> >> The first stage "hack" in this investigation has been retrofitting the
>> >> existing TCP transport
>> >> (https://github.com/nanomsg/nanomsg/tree/master/src/transports/tcp) to
>> >> dispatch different handshake and send/recv handlers based on transport
>> >> scheme.
>> >>
>> >> Based on a day's worth of hacking, here are a few early-stage design
>> >> considerations:
>> >>
>> >> 1a) It feels heavy-handed to copy-and-paste the entire TCP transport
>> >> wholesale, just to change a few key areas where WebSockets differ
>> (e.g., on
>> >> the surface, only STCP needs non-trivial modification). On the other
>> hand,
>> >> the IPC transport very closely resembles TCP, so perhaps there's merit
>> in
>> >> this duplication? That said, does it make more sense to duplicate the
>> TCP
>> >> protocol as a starting point for the WS protocol, or to extend the
>> existinga
>> >> TCP?
>> >
>> >
>> > Probably with current codebase you just have to copy. In fact IPC
>> > (unix sockets), transport is even more similar to TCP, but they don't
>> > share codebase. I think it's reasonable trade off.
>> >
>> >> 2) There exist several WebSocket protocol parsing libraries in C;
>> however,
>> >> I'm not readily finding a sweet spot of License + Quality + RFC 6455
>> >> coverage. Ideas?
>> >
>> > I think it's easy enough to write your own. You can use autobahn as a
>> > test suite.
>> >
>> >> 3) Since the WebSocket protocol does not have a fixed-width header
>> like the
>> >> nanomsg TCP wire-level protocol
>> >> (
>> https://github.com/nanomsg/nanomsg/blob/master/rfc/sp-tcp-mapping-01.txt#L68
>> ),
>> >> this presents a fundamental new challenge buffering and parsing an
>> incoming
>> >> stream. Thoughts?
>> >
>> > I don't think it's really complex stuff.
>> >
>> >> 4) For the ws:// protocol, how should a client negotiate scalability
>> >> protocol? Via parameters on the URI (e.g., ws://
>> 127.0.0.1:9876/?sp=NN_SUB)?
>> >> And, what shall be the value of Sec-WebSocket-Protocol?
>> >
>> > I'd say "Sec-WebSocket-Protocol: x-nanomsg-sub"
>> >
>> >> 5) Does it make sense to attack this problem of nanomsg<->WebSockets
>> as a
>> >> specific application use-case, or as generally desirable transport
>> >> considered for inclusion into the core nanomsg library?
>> >
>> > Don't make a transport if you need websockets for specific application
>> > usage. Make a "gateway" -- a separate process which does translation,
>> > using any available http/websocket library.
>> >
>> >> 6) wss:// ... ?
>> >
>> > Yes, sure. Use libressl :). If you will be doing it, consider also
>> > looking at tcp+ssl implementation (latter should probably be
>> > compatible to mangos).
>> >
>> >
>> >
>> > --
>> > Paul
>> >
>>
>>
>
Other related posts:
