Re: Accurate timers in rumpkernels on Xen / minios

  • From: Antti Kantee <pooka@xxxxxx>
  • To: rumpkernel-users@xxxxxxxxxxxxx
  • Date: Fri, 27 May 2016 11:13:28 +0000

On 27/05/16 10:57, George Dunlap wrote:

{ "now":2033655690, "mops":60, "max_delta":10014214 }
{ "now":3061713795, "mops":118, "max_delta":10015144 }
{ "now":4089251608, "mops":176, "max_delta":10036716 }
{ "now":5117213014, "mops":234, "max_delta":10056997 }
{ "now":6144605339, "mops":292, "max_delta":10041393 }
{ "now":7171986432, "mops":350, "max_delta":10042572 }
{ "now":8199335304, "mops":408, "max_delta":10017662 }
{ "now":9226821190, "mops":466, "max_delta":10060509 }


Thanks for the quick response.  What do you mean it's not a multiple
of 10ms?  It's almost exactly 10ms + a few tens of microseconds.  The
tens of microseconds is easily the result of the time going through
the kernel + hypervisor sleep+wakeup path.

I assumed "max_delta" was based on a clock reading. If a clock ticks at 10ms, it's increased by 10ms. In other words: clock % 10ms => constant, which is clearly not happening.

Based on your email address, I'm sure you know Xen better than I do, but are
you sure Xen itself can provide time and sleep with such an accuracy? I'd
expect that sort of skew especially with the sleeps in a virtualized
environment.  If you can produce better accuracy with pure MiniOS, then
there's a bug in Rumprun (or in the Rumprun copy of MiniOS in platform/xen).

It certainly should be able to set a timer shorter than 10ms.  I
actually have the same "worker" code set to compile both as a normal
Linux process and a rumprun image (in part for ease of development, in
part so that the benchmark could be used for OSes or KVM as well as
Xen).  If I take the "worker-proc" binary and run it in an
unprivileged Linux domU, I get the following:

{ "Now":757641512629922, "Mops":0, "MaxDelta":0 }
{ "Now":757642625697453, "Mops":160, "MaxDelta":93439 }
{ "Now":757643756428905, "Mops":320, "MaxDelta":104451 }
{ "Now":757644887239972, "Mops":480, "MaxDelta":128225 }
{ "Now":757646023603159, "Mops":640, "MaxDelta":105891 }
{ "Now":757647156308278, "Mops":800, "MaxDelta":164396 }

That's in nanoseconds, so closer to 100 microseconds.  Not sure if the
limiting factor on the granularity is because of Xen or Linux, but it
should certainly be able to deliver better than 10ms.

Ok, seems like it should be possible. I'm not sure what's happening, maybe the sleep is somehow rounded up to HZ in the rump kernel. The easiest way to figure out what sort of underlying sleeps are requested and when the wakeup is happening is to put prints into lib/libbmk_core/sched.c::schedule(). You should be able to make them conditional based on some value in bmk_thread to limit spam. That and/or put a breakpoint there and see how the value changes through the call path, assuming you can get gdbsx to work.

Other related posts:

  1. Home
  2. rumpkernel-users
  3. Archive
  4. 05-2016