[visionegg] Re: Displaying raw movies

Hi Andrew,

Thanks, that worked like a charm! With Numeric, it takes about 30sec to
load a 120 MB file into a 3D array. With numarray, it takes only a
couple of seconds to do the same. Both seem to be equally quick in
accessing the contents after the fact though (is this because you've
converted the numarray into a Numeric array, as mentioned in the
comments?). On our setup (512MB RAM, Athlon 700MHz, Radeon 9800 Pro,
Windows 2000) it takes about 0.8 ms to load a 2D frame from the 3D
array, draw it, swap the buffers, and change the bit value on our
digital out card (viewed on an oscilloscope). This is plenty fast enough
for the 5 ms available on a 200Hz display. Unfortunately, if I create
two separate texture stimuli to run two movies simultaneously within the
same viewport, things seem to slow down quite a bit, averaging 5.7 ms
per frame. This is even when the identical movie is fed to both stimuli.
Might there be a speed advantage in setting up two separate viewports
instead?

For anyone's reference, I've attached the current code I have for
playing a raw movie from a file.

On another note, in FrameTimer() in VisionEgg.Core, you might want to
consider replacing the word "frame" with "inter frame interval" or
"IFI":

Mean frame was 5.00 msec (200.01 fps), longest frame was 6.52 msec.

to

Mean IFI was 5.00 msec (200.01 fps), longest IFI was 6.52 msec.

For the longest time, I wondered why the total population of the
histogram was always one less than the total number of displayed frames,
'til I finally clued in :)

Cheers,

Martin Spacek
PhD student, Graduate Program in Neuroscience
Dept. of Ophthalmology and Visual Sciences
University of British Columbia, Vancouver, BC, Canada
604-875-4555 ext. 66282
mspacek@xxxxxxxxxxxxxxx
http://swindale.ecc.ubc.ca

----- Original Message ----- 
From: "Andrew Straw" <astraw@xxxxxxxxxxx>
To: <visionegg@xxxxxxxxxxxxx>; <mspacek@xxxxxxxxxxxxxxx>
Sent: Saturday, September 18, 2004 1:09 PM
Subject: [visionegg] Re: Displaying raw movies


Dear Martin,

I just updated Textures.py to accept numarray data as a texel source.
I've updated CVS, and I attach the new files since SourceForge's CVS
mirrors are a little bit slow.

At some point, it would be good to support numarray more directly,
perhaps in a "both supported" approach like the numerix module of
matplotlib

Please let me know if this does not address your issue -- your task is
an important one to get right.

Finally, I've recently become aware of the (not very well publicized)
buffer() builtin function to Python. I think it and its related C API
might be useful in situations like yours, because they allow raw memory
access without many hoops.  In fact, I should probably re-visit the
Vision Egg source code with this in mind...

Cheers!
Andrew

vsyncsperframe = 1
numgreyvsyncs = 400 # number of grey refreshes to display at start
scale = 2 # magification of checkerboard (this many screen pixels squared per 
movie pixel)

##########################################

import VisionEgg, struct, numarray, os, gatoext, pygame
from VisionEgg.Core import *
from VisionEgg.Textures import *
VisionEgg.start_default_logging(); VisionEgg.watch_exceptions() # is this stuff 
needed?

# Open a file for reading in binary format
buffersize = 122884096 # size on disk: 122884096 bytes, is this more ideal than 
the default?
f = file("movie1.m", 'rb', buffersize)
# Read the first five bytes as a header string
header = f.read(5)
# The remaining fields are 2 byte unsigned integers
typecode = 'H' # 'H' is an unsigned short int, which is 2 bytes on this PC
(width,) = struct.unpack(typecode, f.read(2))
(height,) = struct.unpack(typecode, f.read(2))
(numframes,) = struct.unpack(typecode, f.read(2))

print "header =", header
print "width =", width
print "height =", height
print "numframes =", numframes

# Read in the movie data as 8 bit unsigned integers from the file
movie = numarray.fromfile(f, numarray.UInt8, (numframes,height,width)) # 
indices in Python numarray are (z,y,x)==(hyperdim,row,column)
print "Done reading"
movie = movie[::,::-1,::] # flips the movie vertically for OpenGL's bottom left 
origin
print "Done flipping"
'''
# Alternate code to read in data using much slower Numeric method
data = Numeric.array(f.read())
print "Done reading"
movie = Numeric.reshape(data,(numframes, width, height))
print "Done reshaping"
movie = movie[::,::-1,::]
print "Done flipping"
'''
# Check to see if there are any leftover bytes in the file
stuff = f.read()
if stuff != '':
    print "Error: There are unread bytes in the file. Width, height, or 
numframes is incorrect in the header."
    os.abort()

f.close() # Close the file

gatoext.olInitBoard() # Initialize DT340 digital output board

# Initialize OpenGL graphics screen.
screen = get_default_screen()
screen.parameters.bgcolor = (0.5,0.5,0.5,0.0) # set the background color to 
grey (RGBA).
checkerboard_size=(height,width)
scaled_size = (scale*checkerboard_size[0],scale*checkerboard_size[1])

# Create an instance of the texture class, start with a grey frame
greyframe = numarray.zeros([width,height]) + 127
temp_texture = Texture(greyframe)

checkerboard = TextureStimulus(texture=temp_texture,
                               position=(400,300),
                               anchor="center",
                               mipmaps_enabled=0,
                               size=scaled_size,
                               texture_min_filter=gl.GL_NEAREST,
                               texture_mag_filter=gl.GL_NEAREST
                               )

# Create a Viewport instance
viewport = Viewport(screen=screen, stimuli=[checkerboard])

texture_object = checkerboard.parameters.texture.get_texture_object()

# Display the grey frame for the specified number of vsyncs
greyvsyncnum = 0 # 0 based
quit = 0
pause = 0
grey_frame_timer = FrameTimer()
while greyvsyncnum < numgreyvsyncs:
    for eventi in pygame.event.get(): # for all events in the event queue
        if eventi.type == pygame.locals.KEYDOWN:
            if eventi.key == pygame.locals.K_ESCAPE:
                quit = 1
            if eventi.key == pygame.locals.K_PAUSE:
                pause = int(not pause) # toggle pause
    if quit:
        break
    elif pause:
        continue
    screen.clear()
    viewport.draw()
    swap_buffers()
    grey_frame_timer.tick()
    greyvsyncnum += 1
grey_frame_timer.log_histogram()

# Main loop
vsyncnum = 0 # 0 based
framenum = 0 # 0 based
quit = 0
pause = 0
frame_timer = FrameTimer()
while framenum <= numframes-1: # numframes is 1 based
    for eventi in pygame.event.get(): # for all events in the event queue
        if eventi.type == pygame.locals.KEYDOWN:
            if eventi.key == pygame.locals.K_ESCAPE:
                quit = 1
            if eventi.key == pygame.locals.K_PAUSE:
                pause = int(not pause) # toggle pause
    if quit:
        break
    elif pause:
        continue

    # shouldn't use mod, messes up on the last frame, use another counter 
instead
    if (vsyncnum % vsyncsperframe)==0: # if vsyncnum is a multiple of 
vsyncsperframe
        texture_object.put_sub_image(movie[framenum,0:width,0:height]) # load 
next frame
        framenum += 1

    screen.clear()
    viewport.draw()
    swap_buffers()
    gatoext.olPostValue(framenum)
    frame_timer.tick()

    vsyncnum += 1

gatoext.olPostValue(0) # set all DT340 board lines to low
gatoext.olCloseBoard() # close DT340 board
frame_timer.log_histogram()

Other related posts: