[nanomsg] Re: issue with "received" message size

  • From: Garrett D'Amore <garrett@xxxxxxxxxx>
  • To: nanomsg@xxxxxxxxxxxxx, Paul Colomiets <paul@xxxxxxxxxxxxxx>
  • Date: Tue, 11 Mar 2014 13:11:35 -0700

Yes the problem is that typically in topology: 

A -> B 

The request from A to B may hang for a long time, only if B makes no 
progress at all. But if we have priorities, the request from A can be 
hang if there is a higher priority pipe: 

A -> B <= C 

And do you think the average programmer expects some request to be 
executed in a hour from start or in a day? Even idempotent (but not 
stateless) requests may fail in this case. This probably means that 
every request must be timestamped. Which in turn means we would need 
to get timestamping into the core, which is not good IMO. 

This is a general problem with fixed priority schedule — fixed priorities can 
cause lower priorities to starve out completely.  IMO, this is only acceptable 
in an environment where you control *everything*, and can guarantee that fixed 
priorities won’t cause a problem.  (Google for priority inversion to see the 
kinds of bad things that can happen when people try to mis use fixed 

A *vastly* better solution for the real world is to treat priorities more like 
QoS.  In this world you assign a “share” or weighted value, and schedule like 
round-robin, but based upon the weight.  So pipe A has weight 1 and pipe C has 
weight 3.  Then for every three requests you service from C, you should look 
receive a request from A.  Assuming of course that both pipes A and C have a 
continuous stream of requests.

This prevents starvation and guarantees forward progress for all participants.  
I’ve built systems like this in the real world, although they were more like 
work schedulers for local workloads (crypto in my particular case, but also in 
the filesystem in ZFS, and on illumos/Solaris we have a FSS scheduler for 
process scheduling that works the same way), and they’re easy to understand.

        - Garrett


Other related posts: