[nanomsg] Re: threading guidelines, and is traditional unix single threaded mode possible?

  • From: Nico Williams <nico@xxxxxxxxxxxxxxxx>
  • To: nanomsg@xxxxxxxxxxxxx
  • Date: Thu, 26 Sep 2013 14:56:32 -0500

On Wed, Sep 25, 2013 at 11:48:44PM -0700, Jason E. Aten wrote:
> On Wed, Sep 25, 2013 at 1:38 PM, Nico Williams <nico@xxxxxxxxxxxxxxxx>wrote:
> > Yes, this is quite true.  And not only that, you can't be free()ing
> > anything on the child side either.  You need to arrange for the child
> > to be able to reset given any parent-side fork-time state.  Depending
> > on your internal data structure design this might be trivial or it
> > might require taking locks in a pre-fork handler (then dropping them
> > in the parent-side post-fork handler) -- that could be a performance
> > problem even if you never fork because it means having locks taken and
> > released elsewhere just to make this possible.
>
> I don't see the need to take locks on the common path, so I don't see a
> performance downside.  Would it not suffice to have each thread poll fairly

One way or another you'd have to ensure that by the time the child-side
atfork handler runs the data structures are in a stable form, so it can
do whatever the right thing is.  Doing this does not come at zero cost.

> regularly on a global integer flag (the "please-prepare-to-fork flag")? At
> most 2 poll intervals can elapse before every thread has noted the global
> flag and then prepared by taking locks or otherwise pausing. You'd only

That would be one way of doing it.  Kernel-side locks would be involved,
so that that approach is not exactly lock-less :)

> need some kind of barrier synchronization that recognized when all threads
> had arrived and then signaled the thread that wants to do the fork to
> proceed.  This would seem to be pretty fast; the only difficulty being
> actually getting all nanomsg worker threads to poll in the first place.

That's the magic word: synchronization.

I said "might" need locks -- I weaseled out of being real specific as to
design choices because... there are lots of them :)

> > As long as you document what to do around fork() it should be OK.  You
> > could just say: you can't use nanomsg on the child-side of fork()
> > without first exec()ing -- this is the simplest thing to do.
> 
> It seems to me that everybody knows that it will be safe to exec after a
> fork, there's no new information there. That is always safe.  In a FAQ, it
> [...]

I'm saying that "using nanomsg on the child-side of fork() without first
exec'ing is not safe."  That using it after exec is safe goes without
saying (whether one uses CLOEXEC or not, though one should; the danger
of non-CLOEXEC fildes that should have been CLOEXEC are generic, not
specific to any one application/library).

Nico
--

Other related posts:

  1. Home
  2. nanomsg
  3. Archive
  4. 09-2013