[opendtv] Re: 20060117 Mark's (Almost) Monday Memo

  • From: Craig Birkmaier <craig@xxxxxxxxx>
  • To: opendtv@xxxxxxxxxxxxx
  • Date: Sat, 21 Jan 2006 09:19:43 -0500

At 3:01 PM -0500 1/20/06, Manfredi, Albert E wrote:
>Except that I *think* what John really wondered is whether
>there is a Shannon's Law equivalent for codecs. That is, a
>theoretical limit to how much a given image (or audio
>stream) can be compressed. This is independent of Moore's
>Law, other than the presumed requirement of more computing
>power needed as compresssion increases.

Is there a Shannon's Law that tells us how much information a frame 
of any given size can contain?


Shannon can help us predict the highest frequency that can be 
represented (without aliasing) for any given raster size. So in 
theory we could create some diabolical image that totally saturates 
the spectra that can be represented in a single frame.

But this is just for one frame. Shannon tells us noting about the 
next frame, except that it can contain no more information than the 
previous "diabolical" frame. But the next frame can be equally 
challenging, but totally different. From a compression perspective 
this is truly diabolical - that is there is no relationship from one 
frame to the next, thus very little opportunity to take advantage of 
interframe entropy coding.

A good example of this, that is sometimes used to test compression 
algorithms, is NOISE.

Good luck trying to compress this, as it has virtually 100% entropy.

Entropy coding takes advantage of the reality that there is 
"typically" a strong correlation between adjacent picture elements in 
a frame, and a strong correlation between the information in adjacent 
frames. The degree to which this can be used to compress the 
information is influenced by entropy (noise and sampling errors) and 
the correlation between areas of a frame and the information in a 
series of frames (e.g. a GOP).

In other words, there is no correct answer to Bert's question. Any 
limits will change dynamically based on the ability of the 
compression algorithm to deal with specific types of image pathology. 
MPEG-2 falls apart when we do something as simple as dissolving 
between two different images, or an image and black; entropy goes 
through the roof and the encoder cannot compress these frames 
efficiently.  MPEG-2 also falls apart on simple gradients; create a 
gradient from corner to corner of a frame and the DCT is useless. If 
you throw away even a single DCT coefficient in a block you will see 
quantization errors.

As Tom alludes to, any theoretical limit on encoding efficiency must 
take into account the image pathology. I mentioned just a few of the 
more difficult problems in my original response to John's question. 
To achieve the theoretical limit of compression efficiency we must be 
able to predict anything that can possibly be represented within a 
frame, and in groups of frames.

For practical reasons, like live TV, we must be able to do this in 
real time (understanding that non-real time encoding may be able to 
do a better job).

This should make it obvious that we are nowhere close to theoretical 
limits on compression efficiency. If you were to look at this as a 
typical logarithmic function we are still in the "learning curve" - 
the flat part where there are still large opportunities to improve. 
As we approach the inflection point, the gains in efficiency will 
grow smaller and smaller, but it is unlikely that we will every reach 
any theoretical limits that may exist.


You can UNSUBSCRIBE from the OpenDTV list in two ways:

- Using the UNSUBSCRIBE command in your user configuration settings at 

- By sending a message to: opendtv-request@xxxxxxxxxxxxx with the word 
unsubscribe in the subject line.

Other related posts: