[opendtv] Re: Math of oversampling

At 9:14 PM -0400 4/28/05, Tom Barry wrote:
>I think we've already once had something of this conversation on
>AVS.  I still tend to believe we see images in a blend of two (or
>more) different modes.  One would be some sort of edge and/or
>shape perception which is probably not frequency based.  But the
>other is our perception of texture, and that one likely does rely
>on frequency.  It really seems to me our perception of realistic
>sharpness depends upon both.

It is at times like this that I take a bit of pride in the fact that 
the OpenDTV list keeps going, and going, and going. The discussions 
of the past few days have been first rate - I hope everyone is 
learning as much as I have.

Tom is definitely on the right track with respect to the presence of 
specialized receptors in our foveal vision that are tuned to specific 
types of stimuli. The following is from the SPTE Task Force Report on 
Digital Imaging which I helped author in 1992.

3.2.2 Human Visual Processing
Much of the research in visual science today is focused on the 
processing of data acquired by the image receptors. A variety of 
specialized analyzers in the eye process data from small localized 
regions and accumulate the results into channels which are processed 
by the brain to create an integrated view of the physicals 
environment.

There is evidence that the brain directs the activity of the image 
receptors for processes such as establishing white balance and light 
sensitivity levels. Simple localized analyzers are used to enhance 
the data transmitted back to the brain. Some of these analyzers are 
sensitive to a particular edge orientation; there are sufficient 
analyzers at each location to represent a full set of edge 
orientations. Additional tuned analyzers cover portions of the range 
of human sensitivity for spatial frequency, spatial position, 
temporal frequency direction of motion; and binocular disparity.

The data processed by these analyzers moves to the brain through two 
types of channels; a set of fast responding channels with relatively 
transient responses to stimuli, and a set of slower channels with 
relatively sustained responses to stimuli. Transient channels process 
the output of analyzers that are tuned for low spatial and high 
temporal frequency stimuli. Sustained channels process the output of 
analyzers that are tuned for high spatial and low temporal frequency 
stimuli.

So yes, we do have a variety of image receptors that are tuned for 
the acquisition of various components of an image. One of the 
interesting findings of the research that I studied when writing the 
report is that we LEARN how to see different edge orientations. There 
was a study done with rats where they were raised in an environment 
that was devoid of certain edge orientations - I think there were 
vertical lines in the environment, but no horizontal lines.  When 
these rats were later introduced into an environment with edges in 
all orientations, they could not see the edges that they had NOT 
learned to see; they would run into things because they could not see 
them.

It is important to note that there is a huge difference between the 
perception of still images and moving images. With still images we 
have plenty of time to analyze the image and to perceive fine 
details. With moving images the amount of information can overwhelm 
the human visual system. We tend to filter out the most important 
information and used directed eye movements to acquire high 
resolution views of portions of a high resolution image. This is 
especially true for
images that cover a large portion of our field of view, such as an 
large HDTV display - we cannot sample all of the information in the 
image, and are forced to track motion and acquire high resolution 
views of a portion of the image while ignoring most of the detail in 
other portions of the image. The only problem is that we cannot 
predict what a viewer will look at, so we need to have approximately 
the same level of detail everywhere.

So while it is interesting to study the images that Tom and Jeroen 
have created for us, they only tell us part of the story. It takes 
several hundred milliseconds to acquire a high resolution view of any 
image, as the foveal receptors dart around to acquire a high 
resolution view. Thus in a motion imaging system it is likely that 
several temporal samples are contributing to the perception of 
sharpness. Detail from several frames can add to improve the 
perception of sharpness. One of the early compression systems for 
desktop computers simply threw away about 75% of the image samples. 
But it did this by moving the sampling points around in a four pixel 
region - over a four frame time period all of the sample points in 
the original image were presented. This actually worked quite well, 
enabling the perception of more image detail than simply down 
sampling to a frame 1/4 the size. On the other hand, objects that are 
moving need some motion blur - especially at lower frame rates like 
24P - in order to fool the human visual system into seeing continuous 
motion. So a motion imaging system must deal with many issues as it 
attempts to fool the human visual system into seeing sharp, high 
resolution moving images.

Regards
Craig





 
 
----------------------------------------------------------------------
You can UNSUBSCRIBE from the OpenDTV list in two ways:

- Using the UNSUBSCRIBE command in your user configuration settings at 
FreeLists.org 

- By sending a message to: opendtv-request@xxxxxxxxxxxxx with the word 
unsubscribe in the subject line.

Other related posts: