[opendtv] Re: 20060117 Mark's (Almost) Monday Memo
- From: Craig Birkmaier <craig@xxxxxxxxx>
- To: opendtv@xxxxxxxxxxxxx
- Date: Sat, 21 Jan 2006 09:19:43 -0500
At 3:01 PM -0500 1/20/06, Manfredi, Albert E wrote:
>Except that I *think* what John really wondered is whether
>there is a Shannon's Law equivalent for codecs. That is, a
>theoretical limit to how much a given image (or audio
>stream) can be compressed. This is independent of Moore's
>Law, other than the presumed requirement of more computing
>power needed as compresssion increases.
Is there a Shannon's Law that tells us how much information a frame
of any given size can contain?
Perhaps.
Shannon can help us predict the highest frequency that can be
represented (without aliasing) for any given raster size. So in
theory we could create some diabolical image that totally saturates
the spectra that can be represented in a single frame.
But this is just for one frame. Shannon tells us noting about the
next frame, except that it can contain no more information than the
previous "diabolical" frame. But the next frame can be equally
challenging, but totally different. From a compression perspective
this is truly diabolical - that is there is no relationship from one
frame to the next, thus very little opportunity to take advantage of
interframe entropy coding.
A good example of this, that is sometimes used to test compression
algorithms, is NOISE.
Good luck trying to compress this, as it has virtually 100% entropy.
Entropy coding takes advantage of the reality that there is
"typically" a strong correlation between adjacent picture elements in
a frame, and a strong correlation between the information in adjacent
frames. The degree to which this can be used to compress the
information is influenced by entropy (noise and sampling errors) and
the correlation between areas of a frame and the information in a
series of frames (e.g. a GOP).
In other words, there is no correct answer to Bert's question. Any
limits will change dynamically based on the ability of the
compression algorithm to deal with specific types of image pathology.
MPEG-2 falls apart when we do something as simple as dissolving
between two different images, or an image and black; entropy goes
through the roof and the encoder cannot compress these frames
efficiently. MPEG-2 also falls apart on simple gradients; create a
gradient from corner to corner of a frame and the DCT is useless. If
you throw away even a single DCT coefficient in a block you will see
quantization errors.
As Tom alludes to, any theoretical limit on encoding efficiency must
take into account the image pathology. I mentioned just a few of the
more difficult problems in my original response to John's question.
To achieve the theoretical limit of compression efficiency we must be
able to predict anything that can possibly be represented within a
frame, and in groups of frames.
For practical reasons, like live TV, we must be able to do this in
real time (understanding that non-real time encoding may be able to
do a better job).
This should make it obvious that we are nowhere close to theoretical
limits on compression efficiency. If you were to look at this as a
typical logarithmic function we are still in the "learning curve" -
the flat part where there are still large opportunities to improve.
As we approach the inflection point, the gains in efficiency will
grow smaller and smaller, but it is unlikely that we will every reach
any theoretical limits that may exist.
Regards
Craig
----------------------------------------------------------------------
You can UNSUBSCRIBE from the OpenDTV list in two ways:
- Using the UNSUBSCRIBE command in your user configuration settings at
FreeLists.org
- By sending a message to: opendtv-request@xxxxxxxxxxxxx with the word
unsubscribe in the subject line.
Other related posts:
