[nanomsg] Re: ws transport
- From: Martin Sustrik <sustrik@xxxxxxxxxx>
- To: nanomsg@xxxxxxxxxxxxx
- Date: Thu, 27 Nov 2014 10:49:01 +0100
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi Jack,
>> "I've made an equivalent of tcp tests for ws and they fail"
>
> Have you tried applying this pull request?
> https://github.com/nanomsg/nanomsg/pull/339 This PR is what enables
> SPs (beyond just PAIR) to work.
Ok, I'll try that.
> As a short-term test of that PR, I tried existing SP tests by
> changing their connection string to ws:// -- as a longer term goal,
> it would be good to upgrade many tests to iterate over all
> available transports. As an example, the REQ/REP currently only
> tests inproc
> (https://github.com/nanomsg/nanomsg/blob/master/tests/reqrep.c#L28),
> but could be upgraded to iterate over inproc/tcp/ipc/ws
>
>> "there's a lot of stuff that's not really needed"
>
> Most notably, content awareness (UTF-8 vs. binary) being exposed
> to user. I agree that for now we can and should only support
> binary payloads. That said, the ability set per-message metadata
> with CMSG headers is one area we ought to continue to push forward.
> nanomsg currently supports pluggability of scalability protocol and
> transport, and it could use growth toward supporting pluggable
> message transformations as well (such as security, compression,
> content awareness...)
You may want to read a proposal by Drew Crawford posted to the list a
week or so ago. He proposes to have a list of transformers attached to
each endpoint.
However, if layering protocols was that easy, the whole area of
networking would be much more advanced today. The problem is that
individual layers in stack do much more then just transform messages.
They do message coallescing, message splitting, produce new messages
even when there in no message from the underlying layer (e.g.
timeouts), attach various properties to messages, tie multiple
instances of the underlying protocol into a single instance of the
protocol above (load balancing, failure tolerance) and so on.
It's not easy to device a plug-in mechanism to account for such broad
set of use cases.
> As for the removing the resource from the URI, and also PING/PONG
> -- those would need to be included as part of the minimal
> implementation.
As already discussed in private, PING/PONG should be implemented.
While we may not send PINGs ourselves, we must be able to handle PINGs
from peers.
As for URIs, that's the long term goal. For now, AFAIU, they are
needed only for testing. If that's the case we can support them as
undocumented socket option for now. However, I am not sure that we can
use standard ws test suite anyway. More on that below.
> One area we've not fully delved into is the Close Handshake, and
> the concept of a nanomsg endpoint notifying the other endpoint(s)
> it is going away. This is one feature of the WebSocket protocol
> that I think could apply generally with all scalability protocols
> and transports in nanomsg -- which again, would rear the
> conversation of content awareness and message type!
What's the use case for generic closing handshakes? Up to now, the
traditional TCP closing handshake seemed to be sufficient.
>> "When sending a message from client, the message body is masked"
>
> Good point. This is a grey area how to proceed, since WebSockets
> defined by RFC 6455 have well-defined 1:1 endpoint roles of Client
> and Server, whereas a nanomsg topology is more general. Maybe this
> means that a nanomsg endpoint has a third type at the transport
> layer, "Peer", which will accept either masked or unmasked
> messages? (thinking aloud; don't know ramifications of this, such
> as whether intermediate reverse proxies might get fussy)
I think there's no real problem here. You just have to take into
account that there are 2 protocols here, layered one on the top of
another: WebSockets and SP.
WebSockets have clear server/client separation and we should adhere to
that.
SP, living on top of ws, somehow blurs the distinction, however,
underlying protocol can (and should) be agnostic about that.
As a side note: The fact that we have both WebSocket and SP protocol
implemented in a single codebase (rather than using an existing
full-featured ws library) makes the boundary between the two protocols
unclear, however, we should strive to be aware of it, at least when
discussing the concepts on the theoretical level.
>> "After initial WebSocket handshake, each peer sends an 8-byte SP
>> header"
>
> While implementing the first draft implementation, I was on the
> fence whether the SP should be communicated via the HTTP handshake,
> or more like TCP as an 8-byte header.
>
> Most considerations equal, I went with putting it in the HTTP
> handshake, in order that the SP might be communicated as
> plain-text. This is to remain discoverable and idiomatic especially
> within the realm of HTTP programming and debugging, and also as
> more metainformation to help steer HTTP intermediaries.
>
> The kink is that SPs need a canonical text representation in
> addition to their numeric representation, meaning
> https://github.com/nanomsg/nanomsg/blob/master/rfc/sp-protocol-ids-01.txt
> reserves not only a number, but the identifying text as well. By
> allowing transports to negotiate either numeric/text representation
> of SP, a transport is then encouraged to remain idiomatic to its
> medium (e.g., a reverse proxy could be deployed as front-end
> defense on top of a nanomsg application, where the proxy is given
> directives to only allow REQ type connections; this defense is lost
> if SP is communicated as the first 8 bytes of a connection. That
> failure mode could be made easy to detect, troubleshoot, and remedy
> by the client: "HTTP 400 PAIR not allowed")
Well, my gut feeling here (but one backed by a decade of protocol
design) is that it's never a good idea to coalesce two adjacent layers
in the network stack, however convenient that may seem at the first
sight. That kind of thing blurs the distinction between layers, over
the time it introduces more and more tight coupling between the
layers, results in hacky code etc.
>> make a minimal implementation of ws transport (see ws2 branch in
>> the repo)
>
> I sincerely appreciate how rigorously you're testing this! Reading
> your TODO comments in the ws2 branch, I can intuit some of your
> considerations, but it looks like we're converging on a good
> implementation.
Some implementation highlights:
1. There's no separate class for the handshake. All the handshaking is
done as part of atcp and ctcp classes.
2. SP headers (the 8 bytes) and bundled with the ws request/reply,
thus there's no need for additional roundtrip during the connection setup.
> To wrap up, here are the areas that I feel need most
> consideration:
>
> 1) Introducing a Close Handshake to nanomsg. Even though it is
> "application level", and certainly not "transport level", it
> widely-applicable enough to consider adding to all transports.
As already asked above, what's the use case?
> 2) Equipping usock to enable sending of messages directly from the
> transport, cooperatively scheduling/queuing them with messages from
> the application. This could enable automatic PONGs for PINGs, and
> better Close Handshake when a blocking operation is already
> currently running.
What exactly are you missing? You can, for example, set up a timer and
send a ping message when it expires.
> 3) Using Autobahn TestSuite as a fuzzing tester. This requires the
> transport be able to set the resource path in the URI on
> connection.
Ok, so here we deal with the problem of coalesced layers.
Imagine you wrote a full-fledged ws library and then used it in
nanomsg to provide ws transport.
You could test the library using standardised ws test suite and the
mapping to sp using nanomsg's test suite.
Unfortunately, we don't have the two layers separated. Thus, there's
no component for ws test suite to test. Testing the full featureset of
ws using nanomsg is impossible in the same way as using ws test suite
cannot be used to fully test TCP (consider TCP OOB data and such).
> 4) Canonicalizing text representation of SPs in
> sp-protocol-ids-01.txt in addition to numeric ID
Yes, if we really want to coalesce the layer, but I still think it's a
bad idea.
Martin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQEcBAEBAgAGBQJUdvONAAoJENTpVjxCNN9YlvUH/0W+liJjqzkciZBk1hN+yfvL
UB2o5D0KmSqx4h7HqfENtPdcgH/WIvm6Zc6fE55WshcPxuFBNrHGqRANw4vl6GWj
VCSYf3V1SEiX5RtQJK7aGOl49ON4P+84wKKh+gAveRcT5bL9x3ODttIO2Htp5unI
l0vEjrzPTT5EWrW00/d4bt0PazVCOwq7NvhG+58paZfqOoVXyiilhM1X4X4qK/NV
u3v49K3AbECwuHOBAzs84gX62nWG8LyUQLZeRHyMYvx77iGTHlF225bDwrb7pjml
g/s2a50mVW0SR2pi0QCAogxxKm/WhxFv3YXRyz+GnUNA5jkrq6sapPbgVKBct90=
=2lkz
-----END PGP SIGNATURE-----
Other related posts:
