[opendtv] Re: Precision

At 4:14 PM -0400 5/28/07, Tom Barry wrote:

I'd prefer to think of practical (lossy) compression as removing:

1) Redundancy, as you stated
2) Information we don't care about enough to encode, say very high frequency information, and
Careful here. You are treading on thin ice and some of the 1080P zealots may become upset with your logic here.
Very high frequency information may be important as the screen size increases. On smaller screens these details are too small to see, even if they make it through the emission channel. As we increase the screen size, however, we may need these details to deliver what appears as a sharp picture.
Admittedly, these details are the first thing that gets quantized away when we compress for emission, but this does not make them less important to "some" viewers. A viewer with a 27" screen probably does not even care about HDTV - they just want a clean sharp SDTV picture.
For the vast majority of HDTV owners, 720P is sufficient to deliver sharp pictures, especially when the high frequency information is DELIVERED, not quantized away.
But those who have gone to the expense of building a really BIG SCREEN home theater system with 1080P resolution want the extra detail, and they DON"T want no stinking compression artifacts.
And don't bother trying to tell them that MOST OF THE TIME this extra detail will be left on the "compression room" floor. They bought a 1080P display and they know that compression technology is improving, so don't waste your effort trying to explain how stuff "really" works.
3) Related, but not the same as 2), information we don't know about or don't trust. This is information that was captured, but not reliably due to sampling error, noise, whatever. There is a point of diminishing returns on how many bits we can afford to spend encoding unreliable samples or extra bit depth once these things become lost in the noise. 

Yup. Entropy is a bitch.
As Mark pointed out, however, it is an integral part of the sampling process. Most of the extra information that exists in 1080P versus 720P is in that borderline area where noise starts to have an impact on the sample integrity ( not to mention the fact that most of these details are captured with limited contrast due to MTF considerations - that is, they are already inaccurate, but hopefully just attenuated, not completely wrong.
Cameras are already designed to deal with some of this. The designers know the frequencies at which useful details can be captured, and those above which noise makes the samples highly unreliable. So they design the cameras, and downstream processing gear to smoothly roll off the response in the area where some useful details are captured, and to roll off everything above a certain frequency. Without the benefits of oversampling the frequency response of a 1920 x 1080 camera extends only to about 22-24 MHz before the noise overwhelms high frequency details. Most cameras start to roll of the response just above 20 MHz.
The best way to minimize the impact of noise and sampling dither is to OVERSAMPLE. When we resample to a smaller raster we filter out much of the entropy, improving the precision of all of the higher frequency details that are left. We also improve the contrast, which contributes to the perception of a sharper picture.
And then we need to acknowledge real world encoding practices, and the techniques that are used when the peak bit rate requirements exceed the channel bandwidth. We can:
Let the encoder do the best it can, replacing real image detail with quantization noise and in the extreme blocking artifacts. 

OR
Pre-filter the source to reduce the amount of high frequency detail that is presented to the encoder. This is more insidious that resampling to a lower resolution for emission, as we reduce the information but keep all of the encoding overhead for the higher resolution format. I have heard experts talk about the fact that for the 1080 line formats, up to half of the available bits can be consumed just for the transmission of motion vectors when the encoder is stressed.
Once again, however, it is important to recognize that a small percentage of viewers have 1080P displays that need this extra detail. So we need to at least pretend that this stuff is making it through the emission channel to keep them satisfied. Who cares if the programs breaks up into blocks on occasion and the delivered resolution modulates based on the encoding stress... 

At least we are delivering the very best HDTV possible...

;-(
For 2) and 3) however it may be best to not filter or discard them but instead allow the encoder to opportunistically choose whichever values happen to encode nicest. That's one of the reasons I think capturing at much higher bit depths and allowing encoders to quantize them away seems, in some tests, to work more efficiently than some might predict.
I've never seen such a test. Unfortunately, encoders are no smart enough to encode what looks nicest - even with h.264. They run algorithms that are VERY limited in terms of the decision that can be made. H.264 introduced two features that help a little - one improves the decisions, the other masks the mistakes. The frequency domain transform has the ability to select from a range of choices with respect to the content of a transform block. It can be weighted for increased H or V detail and to deal with several types of gradients. The deblocking filter helps to mask the artifacts when a block is over-quantized.
The reality of how compression works is that there is a compression range where only the highest frequencies are quantized. The actual samples are replaced with correlated noise. As long as we operate in this range the pictures look very good. But there are several issues that cause severe problems.
One is high frequency edge information, which we recently established as being critical to human visual perception. Unfortunately, when an encoding block contains this edge information we typically see blocks with a bunch of coefficients that represent the edge. When we quantize these coefficient we introduce distortions in the edge which are typically seen as ringing or noise around the edge. This is particularly bothersome for text overlays. We simply cannot quantize too much, or the distortions become a major problem - like black pixels adjacent to white pixels - which violates sampling theory, which in turn makes the artifacts easier to detect.
Adding bit depth allows us to limit the range of sampling dither. Adding resolution allows us to localize the impact of the frequency transform. But both of these things increase the overhead that must be dedicated to delivering the compressed pictures - i.e. more encoding blocks, more motion vectors, more bits per sample, etc.
The BEST way to deal with this is to resample to a lower resolution raster. We get more accurate samples - less entropy - and there is less encoding overhead, which translates into higher quality samples delivered to the decoder. 

Regards
Craig


----------------------------------------------------------------------
You can UNSUBSCRIBE from the OpenDTV list in two ways:
- Using the UNSUBSCRIBE command in your user configuration settings at FreeLists.org 
- By sending a message to: opendtv-request@xxxxxxxxxxxxx with the word 
unsubscribe in the subject line.

Other related posts: