Yes, I plan to use a single (or even two) threads per connection endpoint.
Plus one per socket at large. This allows me to use simple blocking
system calls, without having to worry about the vagaries of async I/O.
Modern operating systems (or rather modern thread implementations) can
generally scale to huge numbers of threads. Recall that a thread is really
just a scheduling context with some stack space. To get to 100k+ threads
you’re going to need adequate RAM to hold the stacks (probably several GB
in this case — e.g. 100k 8k stacks is just under a GB). It’s not hard to
build meaningful systems that have vast numbers of threads, especially when
they don’t contend on the same data (this is very important to
scalability). You’re still going to run out of ephemeral TCP ports before
you hit this number.
I don’t necessarily believe the current hype to turn away from threads and
make everything an event loop is “correct” — the problem is that most of
the threaded code that people look at (like say Apache) uses really heavy
weight processes (where context switching requires changing MMU tables
which is really really expensive), or contend on shared resources (which is
the situation where many threads perform worse than one), or where the
threading framework is inefficient. (This used to the situation long ago.
These days thread libraries are *very* efficient.)
Now, all that said, the code is being built with a very open design to
support porting to different platforms, and will be able to accept
solutions that make use of coroutines or green threads. The only thing that
is *required* is that the implementation provide high level blocking calls
for I/O, and thread creation, and a few synchronization primitives.
(mutexes and condition variables).
- Garrett
On Mon, Dec 12, 2016 at 6:33 AM, Jan Bramkamp <crest@xxxxxxxxx> wrote:
On 10/12/2016 08:25, Garrett D'Amore wrote:
I have embarked on the creation of a complete rewrite of nanomsg. This
rewrite utilizes threads to obtain concurrency. (In theory “green”
threads or coroutines could be used as well, if the underlying platform
supports the necessary I/O primitives. I’m not interested in discussion
of these internal details at this point, as I strongly believe that it
is possible to make a very performant library that uses threading for
asynchronous I/O and to engage with greater levels of concurrency.)
In some ways using kernel threads simplifies the concurrency as you have
proper threads that can block on everything and aren't restricted to
operations which are available as non-blocking. Targeting more than one
platform increases the the burden even further because each platform
(Linux, *BSD, OSX, Windows) offers a different set of non-blocking
operations e.g. FreeBSD offers working POSIX AIO on local file systems, but
Linux AIO isn't worth the effort for most applications.
The reason for my message here is to present a revised API for your
consideration. This API is quite different in many respects from
nanomsg — in particular it presents a more “object oriented” API that
uses “natural” objects rather than attempting to strictly follow POSIX
file descriptor semantics. (So for example, sockets are actual pointers
to structures rather than integers, and you can obtain information about
the sockets using pointers to opaque structures instead of trying to use
the CMSG API. There is no attempt to provide a “file descriptor” type
API as a first class citizen, although for those that need to use
select() or poll() there will be a way to obtain notification file
descriptors. (Under the hood, those descriptors will be lazy allocated,
and not used at all unless someone demands this. I believe this will
have a dramatic performance benefit for the library, greatly reducing
the total number of system calls required to send and receive messages.
The API will also open up some new richer abilities, like the ability to
introspect into pipes, and to obtain information about where a message
came from, or set different options for different endpoints. Much of
this is modeled on work I’ve done in mangos, where I’ve been able to add
functionality that folks have requested for nanomsg, but which I cannot
easily add to nanomsg.
Anyway, I’m opening this for discussion on the API. (Again, let’s not
get bogged down in *how* I’m building this — I’m not interested in the
various eventing frameworks or async I/O engines — I’m doing this using
threads.)
Do you really plan to use one thread per endpoint? This would be a real
scalability problem. Do know any up to date documentation how far which
kernel can scale in this regard? I doubt Linux or FreeBSD can handle 100k
threads as well as they can handle just 100k sockets.
Oh, and just to be clear, the new code will still be written in C (not
C++), and retain the MIT license.
Pleased to hear that. C is a lot more suitable this because its ABI is the
lowest common denominator in *nix systems and the MIT license should allow
anyone to integrate nanomsg into their application.
This rewrite will facilitate a number of changes with respect to adding
new protocols, and adding new transports. I look forward to being able
to add ZeroTier, UDP, TLS, and Websocket/HTTPS to this implementation —
something that is *extraordinarily* difficult to do in the existing
implementation.
Utilizing threads also simplifies reusing existing libraries in nanomsg
transports.
Without further ado, here’s the header that defines the nng.h API
(nanomsgNG):
[truncated]
The API looks clean and easy to use without constraining applications.
Will the objects be thread-safe or does the application have to provide its
own locking around the API like in ZeroMQ? The ZeroMQ API without locking
makes it very hard to provide sane bindings to some languages.
