[opendtv] Re: Math of oversampling
- From: Tom Barry <trbarry@xxxxxxxxxxx>
- To: opendtv@xxxxxxxxxxxxx
- Date: Thu, 28 Apr 2005 21:14:47 -0400
Don Munsil wrote:
> The primary reason frequency response is irrelevant to digital
scaling is
> because we do not evaluate images in frequency space. The eye
measures light
> amplitude directly at randomly scattered sites on the retina.
Tthere is no
> direct measurement of "spacial frequency". This is in contrast
to the ear,
> which is a physical frequency analyzer. Thus for audio,
frequency response
> of resampling filters is essential, and measuring the frequency
response
> tells you how well the filter will perform. For images this is
just not
> true.
Don -
I think we've already once had something of this conversation on
AVS. I still tend to believe we see images in a blend of two (or
more) different modes. One would be some sort of edge and/or
shape perception which is probably not frequency based. But the
other is our perception of texture, and that one likely does rely
on frequency. It really seems to me our perception of realistic
sharpness depends upon both.
And I really can't buy the idea of texture just being a bunch of
edges, even if it could be constructed that way. In perceiving
texture it doesn't seem that at least my own eyes capture info
about every single little edge but really just some sort of
concept of how fine grained an area seems in each dimension, much
as we also think of color as being an attribute independent from
edges. I think texture is a basic visual abstraction.
- Tom
> From: "Jeroen Stessen" <jeroen.stessen@xxxxxxxxxxx>
>
>>You can't judge a filter like that... You should judge the
>>flatness of the passband and the deepness of the stopband,
>>which can only be done in the frequency domain. Even a tiny
>>variation in the time or place domain can give a huge
>>difference in the stopband. A good filter should be designed
>>by looking in the frequency domain in the first place. By just
>>windowing some "perfect" filter in the time domain, especially
>>if the window length is too short, you get inferior filters.
>
>
> I disagree almost completely. I don't want to start a huge argument, but I
> feel pretty strongly about this, having spent an inordinately long time
> researching and implementing an image scaling engine.
>
> Evaluating image scaling filters in the frequency domain is basically
> worthless. I mean that without any hyperbole at all - the frequency response
> of a digital image resizing filter does not predict in any meaningful way
> how well it performs, either aesthetically or as a model of how real-world
> image gathering works.
>
> If we imagine scaling to be a physical simulation of swapping one image
> sensor for another image sensor with more or fewer pixels, it's clear that
> any interpolation or resampling operation is not in any way modeling this.
> Fundamental to any resampling mathematics is the treatment of a pixel as a
> point sample, and pixels are clearly not point samples. (There are a few
> researchers who have looked at iterative methods of constructing models of
> likely distributions of photons across pixel sites, and then "repacking"
> them into new rectangles representing the new virtual CCD sensors, but such
> methods are (so far) computationally expensive and don't necessarily produce
> any more pleasing results than traditional interpolation methods.)
>
> That's not to say that interpolation and resampling strategies are worthless
> for scaling; they work quite well. It's just that we shouldn't imagine that
> they represent some kind of "correct" interpretation of image scaling.
>
> Moreover, the filters with the very best frequency response are absolutely
> terrible for scaling, and the better the frequency response the worse the
> performance. One of the primary artifacts of very long filters is excessive
> ringing, and the longer the filter the further the ringing extends from an
> edge. This ringing is not a real-world image phenomenon. You don't start to
> see little halos around objects as you move closer to them or further away.
>
> The primary reason frequency response is irrelevant to digital scaling is
> because we do not evaluate images in frequency space. The eye measures light
> amplitude directly at randomly scattered sites on the retina. There is no
> direct measurement of "spacial frequency". This is in contrast to the ear,
> which is a physical frequency analyzer. Thus for audio, frequency response
> of resampling filters is essential, and measuring the frequency response
> tells you how well the filter will perform. For images this is just not
> true.
>
> There is a pervasive assumption in image processing (implicit in many of the
> standard texts) that image resampling is essentially identical to audio
> resampling, just in two dimensions instead of one. It's not.
>
> In practice, a good video scaling filter will have a relatively
> decent-looking frequency response, but that's a side effect, not a goal.
> Improving the frequency response will more than likely reduce the final
> image quality, not improve it.
>
> Best,
> Don
>
>
>
> ----------------------------------------------------------------------
> You can UNSUBSCRIBE from the OpenDTV list in two ways:
>
> - Using the UNSUBSCRIBE command in your user configuration settings at
> FreeLists.org
>
> - By sending a message to: opendtv-request@xxxxxxxxxxxxx with the word
> unsubscribe in the subject line.
>
>
----------------------------------------------------------------------
You can UNSUBSCRIBE from the OpenDTV list in two ways:
- Using the UNSUBSCRIBE command in your user configuration settings at
FreeLists.org
- By sending a message to: opendtv-request@xxxxxxxxxxxxx with the word
unsubscribe in the subject line.
Other related posts:
