Nanomsg has only been around for a short time and is already washed-up???Okay,
try again with nng then but it's hard to make money out of vaporwear.
On Thursday, January 5, 2017 5:39 AM, Garrett D'Amore <garrett@xxxxxxxxxx>
wrote:
It actually typically uses 2 threads per socket, and 2 threads per underlying
connection.
This could be altered to use a co-routine library, and its designed to support
that as part of platform porting. Having said that, I’m *strongly* of the
opinion that given a robust and non-crappy threads implementation, that threads
will perform and scale quite highly. Most of the complaints about thread
scalability come from three areas:
a) Poor application/library design leading to lots of lock contention.
Uncontended mutexes are cheap. Contended ones are not. I’ve designed to
minimize contention.
b) Stack consumption. Generally threads *do* each have their own stack. I’ve
taken care to keep my stacks shallow, so its unlikely that we would ever need
more than a single page per thread. At 4K page sizes, this means that you can
have 1000 threads (around 500 connections) in only 4MB RAM. If you need 1M
connections, you might feel the problem. You’ll run out of TCP ports first
though.
c) Crappy threading implementations. This is largely a thing of the past.
Modern thread libraries are quite performant and scale particularly well.
Conversely, threading leads to inherently better multi-core scalability, giving
huge performance wins on larger systems. And, as a particularly nice bonus,
the logic flow in single threads is *lots* easier to understand.
Anyway, so that’s my thinking. Once I’ve gotten a little further we will be
able to test these theories with actual scalability and performance tests. If
it turns out that I’ve misjudged, it will still be possible to retrofit some
kind of coroutine API.
- Garrett
On Wed, Jan 4, 2017 at 11:31 AM, James Root <jamesroot@xxxxxxxxx> wrote:
Hey Garrett,
On your comment that it nng "uses threads internally", could you expand a
little bit? Is it something like 2-4 threads? 1 thread / socket? etc? Also, had
you considered using any co-routines (like libmill) instead of threads?
Thanks,James Root
On Wed, Jan 4, 2017 at 1:48 PM, Garrett D'Amore <garrett@xxxxxxxxxx> wrote:
Yes, nn_device needs its own thread. It blocks forever, until one of the two
sockets is closed or nn_term is called.
libnng is C.
I don’t think I’m ready to accept new contributions on libnng yet, as I’m still
in the midst of writing it. libnng is basically a rewrite of nanomsg using the
lessons learned in mangos, so its a completely different approach internally
(e.g. uses threads internally, and completely does away with the state machine
craziness in libnanomsg) — it’s also going to support further development
efforts including new patterns and new transports *far* more easily than is
possible with legacy libnanomsg. Finally, it will probably also be quite a bit
faster as I’m reducing quite a few system calls. At any rate, its likely going
to be a couple of weeks before I’m ready to start inviting co-contributors,
though you are welcome to start looking at the code — the API details, while
subject to change, are available in the nng.h header file . The repo is at
github.com/nanomsg/nng
On Wed, Jan 4, 2017 at 10:10 AM, Roberto Fichera <kernel@xxxxxxxxxxxxx> wrote:
On 01/04/2017 06:53 PM, Garrett D'Amore wrote:
You can make a device to route automatically between transports like this …
see nn_device().
Ah! Good! I guess it has to run within a separated thread, right?
Note however that if you are updating extremely frequently, you might find
that you lose messages as the PUB/SUB pattern drops messages when queues get
full. As TCP generally is slower than inproc, its easy to overwhelm it.
nanomsg has some concerns at present around inproc as well, and I’d be somewhat
cautious of using it in production (test it yourself). (TCP and IPC are fine
though.)
This is already taken in account. Distributed application knows they might be
lossy, so no problem.
However, if you are able to, you can use mangos (which is a rewrite of
nanomsg in Go), as it does not suffer any of the same concerns.
Constraint is C or C++ interface.
Very soon I think you’ll also have libnng as an option, which I think won’t
have any of the same considerations either. However, libnng is probably a
couple of weeks away from being ready for use in production apps.
Ok! Timespan could be ok. I could start with libnanomsg and move afterwards to
libnng
Current PUB/SUB in nanomsg is handled by subscriber filtering, so the
publisher knows nothing about it. I have a plan to create a new version of
this pattern that actually does publisher side filtering, but that will also be
several weeks away I think. (And its likely that this new pattern will only be
available in libnng and mangos.)
I could also help you on implementing this, I would need to know better the
inside of the library ;-)
- Garrett
On Wed, Jan 4, 2017 at 9:42 AM, Roberto Fichera <kernel@xxxxxxxxxxxxx> wrote:
Hi There.
I'm evaluating to introduce nanomsg as common communication library for my
multithreaded and distributed applications. I would like to know what is the
best
approach to use for my specific use case.
My application updates thousands of separated data from many different threads.
For every data I would like to notify the changes to external subscribers.
So my idea is to have a PUB nn_socket using a TCP transport for the whole app.
Within the app, I could have a SUB nn_socket using a inproc transport
collecting all
subscribers one for each single data (topic) to PUB. Every inproc PUB data
might
wait a new data in a thread and publish the data as soon as it's available.
Finally the SUB nn_socket collecting all the data published could route the
data up
to the PUB nn_socket for delivering outside the data. What is the best way to
"route" messages between TCP PUB (external) and INPROC SUB (internal)?
If this is ok I would like to allocate inproc publishing threads only for
topics (data) that external
subscribers have subscribed, so having real demand, in order to optimize the
performances.
Then this trigger another question: is there anyway to know the topic
subscribed by a subscriber?
Any suggestion?
Thanks in advance,
Roberto Fichera.
