[nanomsg] Re: nanomsg rewrite - new API

  • From: Jan Bramkamp <crest@xxxxxxxxx>
  • To: nanomsg@xxxxxxxxxxxxx
  • Date: Mon, 12 Dec 2016 15:33:06 +0100



On 10/12/2016 08:25, Garrett D'Amore wrote:

I have embarked on the creation of a complete rewrite of nanomsg.  This
rewrite utilizes threads to obtain concurrency.  (In theory “green”
threads or coroutines could be used as well, if the underlying platform
supports the necessary I/O primitives.  I’m not interested in discussion
of these internal details at this point, as I strongly believe that it
is possible to make a very performant library that uses threading for
asynchronous I/O and to engage with greater levels of concurrency.)

In some ways using kernel threads simplifies the concurrency as you have proper threads that can block on everything and aren't restricted to operations which are available as non-blocking. Targeting more than one platform increases the the burden even further because each platform (Linux, *BSD, OSX, Windows) offers a different set of non-blocking operations e.g. FreeBSD offers working POSIX AIO on local file systems, but Linux AIO isn't worth the effort for most applications.

The reason for my message here is to present a revised API for your
consideration.  This API is quite different in many respects from
nanomsg — in particular it presents a more “object oriented” API that
uses “natural” objects rather than attempting to strictly follow POSIX
file descriptor semantics.  (So for example, sockets are actual pointers
to structures rather than integers, and you can obtain information about
the sockets using pointers to opaque structures instead of trying to use
the CMSG API.  There is no attempt to provide a “file descriptor” type
API as a first class citizen, although for those that need to use
select() or poll() there will be a way to obtain notification file
descriptors.  (Under the hood, those descriptors will be lazy allocated,
and not used at all unless someone demands this.  I believe this will
have a dramatic performance benefit for the library, greatly reducing
the total number of system calls required to send and receive messages.

The API will also open up some new richer abilities, like the ability to
introspect into pipes, and to obtain information about where a message
came from, or set different options for different endpoints.  Much of
this is modeled on work I’ve done in mangos, where I’ve been able to add
functionality that folks have requested for nanomsg, but which I cannot
easily add to nanomsg.

Anyway, I’m opening this for discussion on the API.  (Again, let’s not
get bogged down in *how* I’m building this — I’m not interested in the
various eventing frameworks or async I/O engines — I’m doing this using
threads.)

Do you really plan to use one thread per endpoint? This would be a real scalability problem. Do know any up to date documentation how far which kernel can scale in this regard? I doubt Linux or FreeBSD can handle 100k threads as well as they can handle just 100k sockets.

Oh, and just to be clear, the new code will still be written in C (not
C++), and retain the MIT license.

Pleased to hear that. C is a lot more suitable this because its ABI is the lowest common denominator in *nix systems and the MIT license should allow anyone to integrate nanomsg into their application.

This rewrite will facilitate a number of changes with respect to adding
new protocols, and adding new transports.  I look forward to being able
to add ZeroTier, UDP, TLS, and Websocket/HTTPS to this implementation —
something that is *extraordinarily* difficult to do in the existing
implementation.

Utilizing threads also simplifies reusing existing libraries in nanomsg transports.

Without further ado, here’s the header that defines the nng.h API
(nanomsgNG):

[truncated]

The API looks clean and easy to use without constraining applications. Will the objects be thread-safe or does the application have to provide its own locking around the API like in ZeroMQ? The ZeroMQ API without locking makes it very hard to provide sane bindings to some languages.

Other related posts: