[nanomsg] Re: Inproc latency and programming challenge!

  • From: David Nyström <david.nystrom@xxxxxxxx>
  • To: <nanomsg@xxxxxxxxxxxxx>
  • Date: Wed, 30 Jan 2013 09:11:14 +0100

On 01/30/2013 06:49 AM, Martin Sustrik wrote:
On 29/01/13 20:47, Paul Colomiets wrote:

1. nn_send(A)
2. lock mutex A

3. nn_send(B)
4. lock mutex B
5. message is written to a shared pipe
6. lock mutex A<-------- deadlock happens here
7. signal A's condition variable
etc.


I think it's possible to do "trylock" instead locking in step 6, in
case lock is contended return some error code (e.g. EBUSY), propagate
it all the way down to nn_sock_send, then unlock the socket and try
again immediately. This will probably improve the benchmark, but may
lead to live-locks in practice. What do you think about this strategy?


This might be a good idea, since trylock will not syscall when contended. syscalls are quite indeterministic.
So this could be a polling alternative.

In theory.

In practice it's dauntingly complex. First, there can be multiple
destinations for the message so you would have to keep *list* of mutexes
to be re-tried.

Even worse, once you unlock the original socket, another nn_send() call
may be executed by the user and you are left with two lingering
messages. Ultimately you have to implement user-space queueing of
messages etc.

Martin



Hmm,

Have you considered supporting a lockless implementation of message queues for inproc/ipc ? (libatomic_ops).

Sorry for jumping in into the thread half-way, but what is the end
goal of the scalability protocol called nanomsg ?
To be a scalability protocol in userspace for SMP OS:es, or be a scalability protocol for inter-chip communication ?

Br,
David

Br,
David


Other related posts: