[visionegg] Re: Dropped frame detection reliability, frame histogram accuracy

Dear Timothy and Martin,

I am sympathetic to your concerns, and I will attempt to answer them here.

I have spent a lot of time with a photo-detector, oscilloscope, and flickering patterns made with the Vision Egg. The result is that once things were operating smoothly under Windows 2000, I never missed frames.

Furthermore, with constant-velocity moving stimuli, it is very easy to see skipped frames, so the equivalent experiment can be performed simply by watching moving patterns. (The above "things operating smoothing" can be taken literally.)

Although the frame-skip detection code is based on some rather ad-hoc heuristics, I don't see an alternative approach without either additional hardware (a vsync counter) or until OpenGL provides information on framecount or timing of the last frame drawn. Providing such information has been talked about for OpenGL 2, but I don't know how the developers prioritize this now that so much emphasis has shifted to pixel and vertex shaders.

I've interspersed answers to specific questions below.

Scenario 2: frame 1 is drawn some time _before_ raster #1, and frame 2 is
drawn some time _after_ raster #2. So frame 2 misses raster #2 and is
therefore dropped, and frame 1 is drawn twice. This is the case even though
the time interval between frames is the same as in scenario 1.

Is this something to worry about, or am I way off on this?

This is unlikely to go undetected because the timestamp for the return of frame 2's swap() call is after raster 3 and will thus be approximately 2x inter-raster-interval. (Of course this assumes you're synchronizing swap() to vsync.)

- I'm guessing that OpenGL (or something low-level) sends a tick to VisionEgg
on every raster and and/or every bufferswap. How accurate is the arrival of
this tick?

As I mentioned above, unfortunately there is no tick from OpenGL. The timepoint is taken within the Vision Egg's FrameTimer.tick() function.

Does the delay between the actual event and the tick vary
depending on system load?


Yes, it could, but see below.

- Under Windows, VisionEgg uses the Windows timer to timestamp events. What's
the accuracy of the windows timer (referred to as time.clock() in
__init__.py)?

In Windows (at least back in Python 2.1), time.clock() was more accurate than time.time(), which is why there's special treatment for Windows in this regard. In terms of absolute accuracy -- I'm not sure, but it seems much (orders of magnitude) better than 1 msec.

If windows is off doing something at the time that VisionEgg is
requesting a timestamp from Windows, does VisionEgg have to wait around for
the timestamp, which by the time it gets it is innacurate?


In theory, you have a point. In practice, it's good to remember that you (hopefully) have a gigahertz machine and that a few clock cycles here and there are insignificant when investigating most biological processes. (Otherwise, Python and the Vision Egg are the wrong tools anyway.)


Timothy Vickery wrote:

Hi Martin and others,
I also think that this method seems problematic, and I hope these
concerns can be addressed. In your example scenarios, you assume that
the frame time is acquired right after the back buffer is updated. I
always sync the buffer swaps with the vertical refresh, and most
people programming experiments probably do this, as well. I don't
think that OpenGL sends a tick: instead, if buffers are synched, then
the system time is polled right after a buffer swap to get a rough
estimate. At least, that's how it seems to be done in the
Presentation.go() function, as well as the demos that create their own
loops (via the FrameTimer.tick method).
The FrameTime.tick method is called directly after a buffer_swap()
call, which simply wraps the pygame.display.flip() function. When
buffer swaps are synced with the refresh, then this function does not
return (I believe) until a buffer swap has actually occurred. So, if a
frame update takes so long that it misses a frame, the duration
between the prior frame's tick and the current frame's tick should be
about 2.0/refresh_rate. On a typical run without dropped frames, the
duration should be clustered around 1.0/refresh_rate. Indeed, that's
what happens most of the time when I run a simple example. This is an
ok way of testing to make sure frames are dropped, but it isn't great,
and I wish there was a better way. As long as the system clock
function is moderately reliable, then values should be very close to
1.0/refresh_rate. If OpenGL absolutely won't give us better info, then
perhaps the test should be more conservative than 2.0*refresh_rate.
However, something weird happens when I run a more intensive demo, for
example the targetBackground.py demo fullscreen @ 1024x768, 75Hz. If
it ran more or less flawlessly, then value in the frame histogram
should have values clustered right around 13ms (since I have the
buffer swaps synched w/ vert. ref.). If it occasionally missed frames,
then most frames should still cluster around 13 ms, with another
cluster at 26ms. If it always missed a frame, then values should
cluster to 26ms. However, what I get is a cluster of (almost the full
600 frames) values at 17ms. Does anyone know why this occurs? Clearly,
frames are getting dropped, but it doesn't look as I would expect it
to look if this were the case.


I hope you're not confused by the initial GUI screen asking for frame rate -- this does not set it, it only tells Vision Egg what to expect! (This is because there is no easy cross-platform way to set frame rate. It may be possible on a platform-by-platform basis, but I haven't yet felt like inflicting that pain on myself.) So, it sounds to me like you're actually doing 60 Hz. What does your monitor tell you? The average framerate computed by the Vision Egg?

Please let me know if you find something grossly wrong.

Andrew (or anyone else), do you have any idea why this would happen? I
hope we are both worried over nothing.
Thanks,
Tim

On Mon, 16 Aug 2004 02:42:47 -0700, Martin Spacek
<mspacek@xxxxxxxxxxxxxxx> wrote:


Hello,

I'm wondering about the reliability of dropped frame detection and the
accuracy of the frame histogram in VisionEgg. There's a spot in the code
('warn_longest_frame_threshold' in flowcontrol module) which decides that a
frame has been dropped if the length of time to draw that frame to the
framebuffer (as measured by the windows timer) is >= 2x that of the screen
refresh time. I guess this makes sense: if the reported time between the
completion of the drawing of frame 1 and frame 2 to the framebuffer is >= 2x
the refresh time, then either frame 1 or frame 2 must have not been drawn to
the buffer in time. There's a comment added in the code: "set to 2.0 for no
false alarms" which I think means that 2.0 or greater will ensure you get no
false positives.

I'm more worried about the false negatives though. Assume that two
consecutive frames are drawn to buffer with a (1.999 x 1/refresh rate) time
interval between them:

Scenario 1: frame 1 is drawn to buffer immediately _after_ raster #0, and
frame 2 is drawn to buffer immediately _before_ raster #2. Both frames are
drawn in time for their rasters, and therefore both are displayed at the
right time. This is the best case scenario, but it happens rarely.

Scenario 2: frame 1 is drawn some time _before_ raster #1, and frame 2 is
drawn some time _after_ raster #2. So frame 2 misses raster #2 and is
therefore dropped, and frame 1 is drawn twice. This is the case even though
the time interval between frames is the same as in scenario 1.

Is this something to worry about, or am I way off on this?

Other things I'm wondering about:

- I'm guessing that OpenGL (or something low-level) sends a tick to VisionEgg
on every raster and and/or every bufferswap. How accurate is the arrival of
this tick? Does the delay between the actual event and the tick vary
depending on system load?

- Under Windows, VisionEgg uses the Windows timer to timestamp events. What's
the accuracy of the windows timer (referred to as time.clock() in
__init__.py)? If windows is off doing something at the time that VisionEgg is
requesting a timestamp from Windows, does VisionEgg have to wait around for
the timestamp, which by the time it gets it is innacurate?

Cheers,

Martin

======================================
The Vision Egg mailing list
Archives: http://www.freelists.org/archives/visionegg
Website: http://www.visionegg.org/mailinglist.html



======================================
The Vision Egg mailing list
Archives: http://www.freelists.org/archives/visionegg
Website: http://www.visionegg.org/mailinglist.html




--
Andrew Straw, Post-doctoral scholar
California Institute of Technology
Bioengineering, Mailcode 138-78
Pasadena, CA 91101 USA
astraw@xxxxxxxxxxx
+1 626 395 5828


======================================
The Vision Egg mailing list
Archives: http://www.freelists.org/archives/visionegg
Website: http://www.visionegg.org/mailinglist.html

Other related posts: