OK, makes sense. So there will be compile time flags to select the precise
platform we're using, correct?
Thanks,
Cem Karan
-----Original Message-----
From: nanomsg-bounce@xxxxxxxxxxxxx [mailto:nanomsg-bounce@xxxxxxxxxxxxx] On ;
Behalf Of Garrett D'Amore
Sent: Monday, December 12, 2016 2:25 PM
To: nanomsg@xxxxxxxxxxxxx
Subject: [nanomsg] Re: [Non-DoD Source] Re: nanomsg rewrite - new API
All active links contained in this email were disabled. Please verify the
identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address to a
Web browser.
________________________________
I have indeed considered function pointers. I rejected that for several
reasons:
a) platforms are “full ports”, where the functionality is determined at
compile time
b) I do not desire nor intend to expose low level “platform” functionality to
library consumers.
c) Using a port should not require modifying application source code.
d) These functions are going to be called so much that I’d rather not have to
chase a function pointer.
The idea here is that only a small handful of people (those porting nng to a
new platform altogether) need to worry about this. I’m trying
to make it easy for those people, without impacting application consumers.
- Garrett
On Mon, Dec 12, 2016 at 10:23 AM, Karan, Cem F CIV USARMY RDECOM ARL (US)
<cem.f.karan.civ@xxxxxxxx < Caution-
mailto:cem.f.karan.civ@xxxxxxxx ;> > wrote:
OK, I just quickly looked at everything the platform directory; it
appears that you're defining the functions directly, correct? That
would mean you'd have to rewrite the functions directly to change how they
are implemented. Instead of that, have you considered
defining function pointers instead, so that library users can easily pass in
different implementations? E.g.:
typedef int (*nni_mutex_create_t)(nni_mutex_t *mp)
<< Lots more typedefs >>
typedef struct nni_thread_functions_t
{
nni_mutex_create_t nni_mutex_create;
<< Lots more stuff >>
} nni_thread_functions_t;
int void nni_set_threading_model(const nni_thread_functions_t* model)
{
<< Store it somewhere >>
}
It would require some more work under the hood to ensure that there
aren't any live threads when someone tries to call
nni_set_threading_model() (so you don't accidentally destroy active data
structures), but with some work it could be done. You could
then ship the library with an initial set of functions that support the C11
standard.
Thanks,
Cem Karan
> -----Original Message-----
> From: nanomsg-bounce@xxxxxxxxxxxxx <
Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;> [Caution-mailto:nanomsg-
bounce@xxxxxxxxxxxxx < Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;> ] On
Behalf Of Garrett D'Amore
> Sent: Monday, December 12, 2016 12:05 PM
> To: nanomsg@xxxxxxxxxxxxx < Caution-mailto:nanomsg@xxxxxxxxxxxxx ;>
> Subject: [nanomsg] Re: [Non-DoD Source] Re: nanomsg rewrite - new API
>
> All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all
links
> contained within the message prior to copying and pasting the address
to a Web browser.
>
>
> ________________________________
>
>
>
> Yes. there is a platform porting layer, and one of the functions that
the “platform” implements is “nni_thread_create()”. This
creates a
> thread, or coroutine, or whatever. Not every platform has pthreads.
>
> I actually intend to build a version of this that uses C11 threads
and synchronization objects at some point.
>
> On Mon, Dec 12, 2016 at 8:55 AM, Karan, Cem F CIV USARMY RDECOM ARL
(US) <cem.f.karan.civ@xxxxxxxx < Caution-
mailto:cem.f.karan.civ@xxxxxxxx ;> < Caution-
> Caution-mailto:cem.f.karan.civ@xxxxxxxx ;<
Caution-mailto:cem.f.karan.civ@xxxxxxxx ;> > > wrote:
>
>
> Are you going to create your own thread/process API? That is,
something where all your code calls through a small set of
well-
> known POSIX thread-like functions that can be easily replaced by
something else? That will make it much easier to test out
green threads
> vs. regular threads vs. full processes.
>
> Thanks,
> Cem Karan
>
> > -----Original Message-----
> > From: nanomsg-bounce@xxxxxxxxxxxxx <
Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;> < Caution-Caution-
mailto:nanomsg-bounce@xxxxxxxxxxxxx ;<
Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;> >
[Caution-Caution-mailto:nanomsg- ;<
Caution-mailto:nanomsg- ;>
> bounce@xxxxxxxxxxxxx < Caution-mailto:bounce@xxxxxxxxxxxxx ;> <
Caution-Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;<
Caution-mailto:nanomsg-bounce@xxxxxxxxxxxxx ;> > ] On Behalf Of Garrett
D'Amore
> > Sent: Monday, December 12, 2016 11:19 AM
> > To: nanomsg@xxxxxxxxxxxxx <
Caution-mailto:nanomsg@xxxxxxxxxxxxx ;> <
Caution-Caution-mailto:nanomsg@xxxxxxxxxxxxx ;<
Caution-mailto:nanomsg@xxxxxxxxxxxxx ;> >
> > Subject: [Non-DoD Source] [nanomsg] Re: nanomsg rewrite - new
API
> >
> > All active links contained in this email were disabled.
Please verify the identity of the sender, and confirm the authenticity
of all
> links
> > contained within the message prior to copying and pasting the
address to a Web browser.
> >
> >
> > ________________________________
> >
> >
> >
> > Yes, I plan to use a single (or even two) threads per
connection endpoint. Plus one per socket at large. This allows me to
use
> simple
> > blocking system calls, without having to worry about the
vagaries of async I/O.
> >
> > Modern operating systems (or rather modern thread
implementations) can generally scale to huge numbers of threads.
Recall
> that a
> > thread is really just a scheduling context with some stack
space. To get to 100k+ threads you’re going to need adequate
RAM
> to hold the
> > stacks (probably several GB in this case — e.g. 100k 8k
stacks is just under a GB). It’s not hard to build meaningful systems
that
> have vast
> > numbers of threads, especially when they don’t contend on the
same data (this is very important to scalability). You’re still
> going to run
> > out of ephemeral TCP ports before you hit this number.
> >
> > I don’t necessarily believe the current hype to turn away
from threads and make everything an event loop is “correct” —
the
> problem is
> > that most of the threaded code that people look at (like say
Apache) uses really heavy weight processes (where context
> switching requires
> > changing MMU tables which is really really expensive), or
contend on shared resources (which is the situation where many
> threads
> > perform worse than one), or where the threading framework is
inefficient. (This used to the situation long ago. These
days
> thread
> > libraries are *very* efficient.)
> >
> > Now, all that said, the code is being built with a very open
design to support porting to different platforms, and will be
able to
> accept
> > solutions that make use of coroutines or green threads. The
only thing that is *required* is that the implementation
provide
> high level
> > blocking calls for I/O, and thread creation, and a few
synchronization primitives. (mutexes and condition variables).
> >
> > - Garrett
> >
> >
>
> > On Mon, Dec 12, 2016 at 6:33 AM, Jan Bramkamp
<crest@xxxxxxxxx < Caution-mailto:crest@xxxxxxxxx ;> < Caution-
Caution-mailto:crest@xxxxxxxxx ;< Caution-mailto:crest@xxxxxxxxx ;> > <
Caution-Caution-
> Caution-mailto:crest@xxxxxxxxx ;< Caution-mailto:crest@xxxxxxxxx ;> <
Caution-Caution-mailto:crest@xxxxxxxxx ;< Caution-
mailto:crest@xxxxxxxxx ;> > > > wrote:
> >
> >
> >
> >
> > On 10/12/2016 08:25, Garrett D'Amore wrote:
> >
> >
> > I have embarked on the creation of a complete
rewrite of nanomsg. This
> > rewrite utilizes threads to obtain concurrency.
(In theory “green”
> > threads or coroutines could be used as well, if
the underlying platform
> > supports the necessary I/O primitives. I’m not
interested in discussion
> > of these internal details at this point, as I
strongly believe that it
> > is possible to make a very performant library
that uses threading for
> > asynchronous I/O and to engage with greater
levels of concurrency.)
> >
> >
> >
> > In some ways using kernel threads simplifies the
concurrency as you have proper threads that can block on everything
and
> aren't
> > restricted to operations which are available as non-blocking.
Targeting more than one platform increases the the burden
even
> further
> > because each platform (Linux, *BSD, OSX, Windows) offers a
different set of non-blocking operations e.g. FreeBSD offers
> working POSIX
> > AIO on local file systems, but Linux AIO isn't worth the
effort for most applications.
> >
> >
> >
> > The reason for my message here is to present a
revised API for your
> > consideration. This API is quite different in
many respects from
> > nanomsg — in particular it presents a more
“object oriented” API that
> > uses “natural” objects rather than attempting
to strictly follow POSIX
> > file descriptor semantics. (So for example,
sockets are actual pointers
> > to structures rather than integers, and you can
obtain information about
> > the sockets using pointers to opaque structures
instead of trying to use
> > the CMSG API. There is no attempt to provide a
“file descriptor” type
> > API as a first class citizen, although for
those that need to use
> > select() or poll() there will be a way to
obtain notification file
> > descriptors. (Under the hood, those
descriptors will be lazy allocated,
> > and not used at all unless someone demands
this. I believe this will
> > have a dramatic performance benefit for the
library, greatly reducing
> > the total number of system calls required to
send and receive messages.
> >
> > The API will also open up some new richer
abilities, like the ability to
> > introspect into pipes, and to obtain
information about where a message
> > came from, or set different options for
different endpoints. Much of
> > this is modeled on work I’ve done in mangos,
where I’ve been able to add
> > functionality that folks have requested for
nanomsg, but which I cannot
> > easily add to nanomsg.
> >
> > Anyway, I’m opening this for discussion on the
API. (Again, let’s not
> > get bogged down in *how* I’m building this —
I’m not interested in the
> > various eventing frameworks or async I/O
engines — I’m doing this using
> > threads.)
> >
> >
> >
> > Do you really plan to use one thread per endpoint? This
would be a real scalability problem. Do know any up to date
> > documentation how far which kernel can scale in this regard?
I doubt Linux or FreeBSD can handle 100k threads as well as
they
> can handle
> > just 100k sockets.
> >
> >
> >
> > Oh, and just to be clear, the new code will
still be written in C (not
> > C++), and retain the MIT license.
> >
> >
> >
> > Pleased to hear that. C is a lot more suitable this
because its ABI is the lowest common denominator in *nix systems
and the
> MIT
> > license should allow anyone to integrate nanomsg into their
application.
> >
> >
> >
> > This rewrite will facilitate a number of
changes with respect to adding
> > new protocols, and adding new transports. I
look forward to being able
> > to add ZeroTier, UDP, TLS, and Websocket/HTTPS
to this implementation —
> > something that is *extraordinarily* difficult
to do in the existing
> > implementation.
> >
> >
> >
> > Utilizing threads also simplifies reusing existing
libraries in nanomsg transports.
> >
> >
> >
> > Without further ado, here’s the header that
defines the nng.h API
> > (nanomsgNG):
> >
> > [truncated]
> >
> >
> >
> > The API looks clean and easy to use without
constraining applications. Will the objects be thread-safe or does the
application
> have
> > to provide its own locking around the API like in ZeroMQ? The
ZeroMQ API without locking makes it very hard to provide
sane
> bindings to
> > some languages.
> >
> >
> >
>
>
>
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
