[nanomsg] Re: ws transport
- From: David Beckwith <thirdreplicator@xxxxxxxxx>
- To: nanomsg@xxxxxxxxxxxxx
- Date: Sat, 29 Nov 2014 02:43:28 -0800
"would love to [use]" -- I would love to make a chat website that uses
nanomsg on the back end. I have used Node.js's socket.io library and
found it to be really nice.
"totally different things" -
nanomsg is a socket library that provides several common
communication patterns. The following scalability protocols are
currently available: PAIR, BUS, REQREP, PUBSUB, PIPELINE
WebSocket is a protocol providing full-duplex communications
channels over a single TCP connection. The WebSocket protocol was
standardized by the IETF as RFC 6455 in 2011, and the WebSocketAPI in
Web IDL is being standardized by the W3C.
I don't know about you, but those two things don't sound like the same thing.
I guess it comes down to the goal of the project and your taste for
libraries. Personally, I like the "less is more" approach of having a
small minimal set of capabilities that are independent of any other
concerns. Composability is the holy grail of software engineering.
It should be put on a pedestal above everything else. If the goal is
to make a small library of scalability patterns, then I think it has
already succeded. Why expand the scope when we already have something
small, reusable, and independent of any other concerns?
There are already a bunch existing websocket libraries like:
https://github.com/m8rge/cwebsocket
Couldn't you just use one of those and connect it to nanomsg as an
example use case rather than expanding the scope of the project?
The comment that got me to respond to this thread was:
> As a side note: The fact that we have both WebSocket and SP protocol
> implemented in a single codebase (rather than using an existing
> full-featured ws library) makes the boundary between the two protocols
> unclear, however, we should strive to be aware of it, at least when
> discussing the concepts on the theoretical level.
That sounds like a lot of cognitive overhead that I think should be
avoided if possible. Simplicity is another holy grail of software
engineering (specifically, the reading and writing of code part of
software engineering). The less stuff you have to keep in mind while
reading code, the better.
On the other hand, if you want to make a big and massively useful
project like ffmpeg or nginx, then that's okay too. It's up to you
guys.
On Fri, Nov 28, 2014 at 10:29 PM, Jack Dunaway <jack@xxxxxxxxxxxxxxxx> wrote:
>> I would love to using web sockets with nanomsg, but
>> it seems like we're asking for trouble in the long run by muddling two
>> totally different things.
>
> Would you further elaborate upon "would love to using", "asking for
> trouble", "muddling", and "totally different things"? Specific insight will
> help spur and focus development -- thank you!
>
> Best regards,
> -JRD
>
> On Fri, Nov 28, 2014 at 4:21 PM, David Beckwith <thirdreplicator@xxxxxxxxx>
> wrote:
>>
>> What is the advantage and/or rationale of supporting WebSocket in the
>> nanomsg library? It seems to me that it should be in a totally
>> separate library. I would love to using web sockets with nanomsg, but
>> it seems like we're asking for trouble in the long run by muddling two
>> totally different things.
>>
>> On Thu, Nov 27, 2014 at 3:08 PM, Jack Dunaway <jack@xxxxxxxxxxxxxxxx>
>> wrote:
>> >> Thinking about it, can't we at least formally separate the ws library
>> >> (say by putting the files into a dedicated folder) from its sp mapping?
>> >
>> > I tend to support keeping ws:// in mainline, and paring it to fit the
>> > spirit
>> > of nanomsg, rather than separating as a full-fledged library. For
>> > simplicity.
>> >
>> >> Autobahn
>> >
>> > Autobahn is configurable such that we can select only relevant tests via
>> > JSON configuration. I would encourage continuing to use it for nanomsg
>> > ws://
>> > testing by just just removing the UTF-8 validation tests, especially for
>> > its
>> > comprehensive and creative recipes simulating control frames, header
>> > framing, fragmentation, masking, chopping, fuzzing, etc.
>> >
>> > Assuming SP is communicated via HTTP header, rather than 8-bytes
>> > nanomsg-TCP-style, we could still use Autobahn; else, Autobahn TestSuite
>> > would require customization to test nanomsg.
>> >
>> >> Well, my gut feeling here (but one backed by a decade of protocol
>> >> design) is that it's never a good idea to coalesce two adjacent layers
>> >> in the network stack, however convenient that may seem at the first
>> >> sight. That kind of thing blurs the distinction between layers, over
>> >> the time it introduces more and more tight coupling between the
>> >> layers, results in hacky code etc.
>> >
>> > NN_PIPE_PARSED is a case study! (cue Drew :-)
>> >
>> > I agree with you here, full coalescence would yield all manners of
>> > coupling
>> > that hurt usability and extensibility for all SP/transport combinations.
>> > For
>> > clarity, I'm not suggesting or supporting fully coalescing SP with WS --
>> > but
>> > rather, leveraging the protocol's existing mechanisms that are designed
>> > for
>> > that purpose. I guess this means I endorse intentionally overlapping
>> > layers
>> > where it makes sense, where 'coalescing' might not be the right word for
>> > what we're trying to do here.
>> >
>> > Another example of this leverage is the framing of messages. SP messages
>> > sent over TCP from nanomsg defines a header that includes message size.
>> > The
>> > WebSocket protocol does not need to mirror this same header in addition
>> > to
>> > its own, because the current SP hdr is satisfied by the requisite
>> > WebSocket
>> > message header framing. (Though, I prefer the SP protocol of fixed-width
>> > rather than variable-width length!)
>> >
>> > This same design strategy of utilizing the transport's existing protocol
>> > is
>> > what I suggest we apply to the SP swap as part of the WS opening
>> > handshake,
>> > not following it. The transport's existing handshake provides facilities
>> > for
>> > embedding higher-layer protocol metadata that could steer the outcome of
>> > the
>> > connection.
>> >
>> > Again, this is also for the sake of debuggability, discoverability, and
>> > as
>> > another heuristic for intermediate HTTP routing.
>> >
>> > Finally, admittedly, my very first push of the ws:// implementation has
>> > much
>> > more coupling than is healthy, but for two reasons -- 1) ship early!,
>> > and 2)
>> > as monolithic, contained starting point that could then be properly
>> > diced
>> > and pared and reticulated into nanomsg. Said another way, that first
>> > draft
>> > implementation might appear to coalesce the layers, but that's a
>> > byproduct,
>> > not an endorsement, and needs fixing! :-) Conversations like this help
>> > steer
>> > which parts should remain overlapped (like this header swap) and which
>> > parts
>> > should be split/nixed (like L6 UTF-8 validation).
>> >
>> >>> 4) Canonicalizing text representation of SPs in
>> >>> sp-protocol-ids-01.txt in addition to numeric ID
>> >>
>> >> Yes, if we really want to coalesce the layer, but I still think it's a
>> >> bad idea.
>> >
>> > Are you more opposed to framing the SP in the HTTP header altogether, or
>> > more about simply the naming "SP-31" vs "x-nanomsg-rep"? Let's tackle
>> > the
>> > framing first, and then move to the content of that framing.
>> >
>> >>> 1) Introducing a Close Handshake to nanomsg. Even though it is
>> >>> "application level", and certainly not "transport level", it
>> >>> widely-applicable enough to consider adding to all transports.
>> >>
>> >> As already asked above, what's the use case?
>> >
>> > 1) Protocol violation. I like how WebSocket endpoints MUST fail peers if
>> > the
>> > protocol is violated, in order to maintain their own health. A failed
>> > connection in nanomsg could manifest as an errno returned by any
>> > operation
>> > performed on a socket that was failed by the remote.
>> > 2) Stopping automatic reconnect attempts.
>> > 3) Generally applicable for connections not intended to be persistent
>> >
>> > It's possible that I'm overlooking the TCP close handshake's ability to
>> > handle these things.
>> >
>> > Best regards,
>> >
>> > Jack R. Dunaway | Wirebird Labs LLC
>> >
>> > On Thu, Nov 27, 2014 at 4:09 AM, Martin Sustrik <sustrik@xxxxxxxxxx>
>> > wrote:
>> >>
>> >> -----BEGIN PGP SIGNED MESSAGE-----
>> >> Hash: SHA1
>> >>
>> >> On 27/11/14 10:49, Martin Sustrik wrote:
>> >>
>> >> >> 3) Using Autobahn TestSuite as a fuzzing tester. This requires
>> >> >> the transport be able to set the resource path in the URI on
>> >> >> connection.
>> >> >
>> >> > Ok, so here we deal with the problem of coalesced layers.
>> >> >
>> >> > Imagine you wrote a full-fledged ws library and then used it in
>> >> > nanomsg to provide ws transport.
>> >> >
>> >> > You could test the library using standardised ws test suite and
>> >> > the mapping to sp using nanomsg's test suite.
>> >> >
>> >> > Unfortunately, we don't have the two layers separated. Thus,
>> >> > there's no component for ws test suite to test. Testing the full
>> >> > featureset of ws using nanomsg is impossible in the same way as
>> >> > using ws test suite cannot be used to fully test TCP (consider TCP
>> >> > OOB data and such).
>> >>
>> >> Thinking about it, can't we at least formally separate the ws library
>> >> (say by putting the files into a dedicated folder) from its sp mapping?
>> >>
>> >> The API of the lib would deal just with the framing, e.g.
>> >>
>> >> void ws_create_request (void *buf, size_t buflen, conat char *host,
>> >> const char *resource, const char *protocol);
>> >>
>> >> Maybe this "lib" can then be tested with Autobahn?
>> >>
>> >> Thoughts?
>> >> Martin
>> >> -----BEGIN PGP SIGNATURE-----
>> >> Version: GnuPG v1.4.11 (GNU/Linux)
>> >>
>> >> iQEcBAEBAgAGBQJUdvhFAAoJENTpVjxCNN9YCYEH/3eqf98MxwzMnu1FGwK/rTc5
>> >> C1BZZ67pfyTFZcQwGC0CbuARhCJsjW6SNbwfObcc6YuYdyeyAzSAaT0VfsGjuygz
>> >> UR8Br0nJHunSmq6bdb+X0ukE6vUWCNqPhQb9AGYJMWUCJusiggmGDWh5CYK1dVP6
>> >> /xPq5Y2IHtVN4wZMA4+hTjd+8qxQ9dMXcM5nc35y0FBRaJF0QKEp19RRf5ac0q1o
>> >> sZE/KbfXRXXhG+h8bSRujN+NLY8wkvMRUuQoKmCScNNfJgI/YFyXcPAgPyKprQ5R
>> >> OoVdqsQUUC1kwZA1ZVXH49SqMaQXkkkKpbLcNSYSAlpnowDKZxGq1u/39BO7QE4=
>> >> =UnUR
>> >> -----END PGP SIGNATURE-----
>> >>
>> >
>>
>
Other related posts:
