[opendtv] Re: Math of oversampling
- From: Jeroen Stessen <jeroen.stessen@xxxxxxxxxxx>
- To: opendtv@xxxxxxxxxxxxx
- Date: Fri, 29 Apr 2005 11:01:22 +0200
Hello,
From: "Jeroen Stessen" <jeroen.stessen@xxxxxxxxxxx>
>> You can't judge a filter like that... You should judge the
>> flatness of the passband and the deepness of the stopband,
>> which can only be done in the frequency domain.
Don Munsil wrote:
> I disagree almost completely. I don't want to start a huge argument,
> but I feel pretty strongly about this, having spent an inordinately
> long time researching and implementing an image scaling engine.
Okay. Maybe I should have phrased it differently. I said "judge",
it should not have been interpreted as "design" only.
When I design a filter, I start in the frequency domain, and I
design for a reasonably flat pass band, a decently narrow (but
not too narrow) transition band, and a fairly deep stop band.
Then I go back to the time domain to see what it looks like,
especially if it does not have too much ringing. Typically that
is related to the width of the transition band, so when you know
this then you can also directly judge this, and design for this,
in the frequency domain. So I work in both domains.
> Evaluating image scaling filters in the frequency domain is
> basically worthless. I mean that without any hyperbole at all -
> the frequency response of a digital image resizing filter does
> not predict in any meaningful way how well it performs, either
> aesthetically or as a model of how real-world image gathering works.
You are right about that, but I attach less importance to that.
I like to design for low aliasing, because it is aliasing that
gives all that jitter in moving images, and blinking of moving
textures as a function of the (changing) sampling phase. I can
only optimize for this by going to the frequency domain. And I
try to make a down-sampled image that is BETTER than an image
that has been sampled directly by a lower resolution sensor !
> If we imagine scaling to be a physical simulation of swapping
> one image sensor for another image sensor with more or fewer
> pixels,
That is precisely what I do _not_ want to do, because a sensor
with fewer pixels typically generates more aliasing. That is why
John Watkinson has proposed over-sampling (and down-sampling) in
the first place: to get rid of the sensor's sampling artefacts.
> it's clear that any interpolation or resampling operation is
> not in any way modeling this. Fundamental to any resampling
> mathematics is the treatment of a pixel as a point sample,
> and pixels are clearly not point samples.
Correct, and that is precisely the problem. The spatial aperture
of the filter acts as a pre-filter, but as an anti-aliasing
filter it is pretty bad. By over-sampling this will be less
critical, and then we can use a better filter for down-sampling.
You can still find John's paper(s) on a Microsoft web-site.
> That's not to say that interpolation and resampling strategies
> are worthless for scaling; they work quite well. It's just that
> we shouldn't imagine that they represent some kind of "correct"
> interpretation of image scaling.
I agree. But I hope that they represent something better !
> Moreover, the filters with the very best frequency response
> are absolutely terrible for scaling,
That depends on your definition of "very best". It is common to
believe that brick-wall filters are the best, because they give
the best trade-off between sharpness (widest pass-band) and
aliasing (widest stop-band). But with zero transition band, the
ringing is obviously terrible. I agree with you on that.
When I design a polyphase filter with a finite number of taps
(e.g. 6, 8 or 10), and I don't mean windowing or truncating from
a longer filter, then after I have chosen a pass-band and stop-
band, there is not much freedom left for making a narrower
transition band. It is typically quite wide (e.g. from 0.75
to 1.25 of the Nyquist frequency), and thus the filter will have
modest ringing (e.g. +/- 10% on the step response). That is how
I prefer it too, so don't worry.
After seeing the output images from my scaler, surely you will
agree that the ringing is modest ?
> and the better the frequency response the worse the
> performance.
Sure. Too much emphasis on a narrow transition band will give you
horrible ringing in the time domain. I don't like that either.
> One of the primary artifacts of very long filters is excessive
> ringing,
This is not necessarily true. The length can also be used for
making the pass-band flatter or the stop-band deeper. Even a
short filter can have excessive ringing, and a long filter
(e.g. a real Gaussian) can have no ringing at all. It depends
on your design method, on what it tries to optimize.
> and the longer the filter the further the ringing extends from
> an edge. This ringing is not a real-world image phenomenon.
Absolutely true, and it is certainly one of my requirements.
But to totally eliminate it you would have to design a Gaussian-
type low-pass filter, and then people would complain about lack
of sharpness.
For ringing you also have to consider that it only happens in
the output image when there are edges present in the input image
that will "knock" the filter into ringing. If the input image is
sufficiently bandwidth limited then you can afford (and you
need !) a somewhat sharper filter (a steeper transition band).
> You don't start to see little halos around objects as you
> move closer to them or further away.
That is slightly untrue, but never mind.
> The primary reason frequency response is irrelevant to
> digital scaling is because we do not evaluate images in
> frequency space.
Well, aliasing problems are excellently predicted by analysis in
the frequency domain. I find that aliasing is the dominant problem
why a display is not a transparent window to the world. Aliasing
causes jitter on edges and texture, which moves uncorrelated to
the objects. This extra motion can distract from the real scene
content, it interferes with our perception of reality. With a good
scaler design we can significantly reduce this problem.
> The eye measures light amplitude directly at randomly scattered
> sites on the retina. There is no direct measurement of "spacial
> frequency".
True, but aliasing is a real phenomenon involving fold-back of
high frequencies to much more visible lower frequencies, and
moving at the wrong speed too. We have to prevent that.
> This is in contrast to the ear, which is a physical frequency
> analyzer. Thus for audio, frequency response of resampling
> filters is essential, and measuring the frequency response
> tells you how well the filter will perform. For images this
> is just not true.
Even for audio it is not all as "essential" as we may believe.
E.g. a sharp transition band causes ringing, and combined with a
"perfect" group delay this gives pre-ringing, which is non causal.
This gives the impression of a pre-echo, which is not natural.
But for both disciplines a deep stop-band is important to suppress
aliasing. Do not optimize other non-essential qualities instead.
> There is a pervasive assumption in image processing (implicit
> in many of the standard texts) that image resampling is
> essentially identical to audio resampling, just in two
> dimensions instead of one. It's not.
There are similarities, and there are differences. Dr. Ad van den
Enden, who worked mostly on re-sampling of audio, and I who work
on video, have learned a great deal from each other.
> In practice, a good video scaling filter will have a relatively
> decent-looking frequency response, but that's a side effect,
> not a goal.
It is more a means than a goal, but a very useful means.
> Improving the frequency response will more than likely reduce
> the final image quality, not improve it.
It all depends on your definition of "better" !
Thanks, best regards,
-- Jeroen
+-------------------------------+------------------------------------------+
| From: Jeroen H. Stessen | E-mail: Jeroen.Stessen@xxxxxxxxxxx |
| Building: SFJ-5.22 Eindhoven | Deptmt.: Philips Applied Technologies |
| Phone: ++31.40.2732739 | Visiting & mail address: Glaslaan 2 |
| Mobile: ++31.6.44680021 | NL 5616 LW Eindhoven, the Netherlands |
| Pager: ++31.6.65133818 | Website: http://www.apptech.philips.com/
|
+-------------------------------+------------------------------------------+
----------------------------------------------------------------------
You can UNSUBSCRIBE from the OpenDTV list in two ways:
- Using the UNSUBSCRIBE command in your user configuration settings at
FreeLists.org
- By sending a message to: opendtv-request@xxxxxxxxxxxxx with the word
unsubscribe in the subject line.
Other related posts:
