[opendtv] Re: Comparison of H.264 with MPEG-2


Bert seems to be a rather tenacious skeptic.

It's important to understand the the statistical nature of the analysis
of test results such as those of the "verification tests" done by MPEG
for AVC.  It is difficult to tease easily-summarizable precise results
from such a test, since the statistical "noise" in the measurement of
video quality can often swamp out the amount of quality difference being
measured.

For example, in a number of the test cases, the quality of the video was
simply so high that doubling the bit rate of MPEG-2 would not produce a
reliably higher measured quality.  In such a case, one cannot reach a
clear conclusion about relative quality because the statistical error is
swamping out the magnitude of the effect that we are trying to measure.
Such cases should simply be conceptually removed from the pool of tests
on which a judgment should be made.

There were also a number of cases where MPEG was able to reach a
conclusion with confidence that one codec performed better than another,
but could not reliably estimate just how much better it was.  Such is
simply the nature of statistical measurement techniques.

There is a big difference between having some number of cases where one
can't reach a statistically confident conclusion that some margin of
improvement had clearly been obtained versus having evidence that the
margin of improvement had *not* been obtained in those cases.  This is
not to say that you should always confer the "benefit of the doubt" to
AVC, but simply that the interpreter should try to avoid bias in any
direction when interpreting results.

If you consider the cases in which some kind of conclusion could be
reached, to quote from the report, one finds that "The overall results
show that the AVC achieved a coding efficiency improvement of 1.5 times
or greater in 78% (66 out of 85) of the statistically conclusive cases,
out of which 77% (51 out of 66) show improvements of 2 times or
greater."  And there were some SD and HD cases with measured improvement
ratios of 2.7x, 3.3x, and 4x.

Anyhow, as I previously stated, "I will not dispute that AVC/H.264 can
probably achieve more improvement in compression for P and B pictures
than for I pictures.  That's probably true, but that doesn't mean you
must stretch out GOP length to get a major benefit."

The MPEG verification tests, of course, are not the only tests that have
been done or will be done.  As far as I know, the conclusions of most
such tests (when set up with some degree of rigor) have been pretty
consistent.  But we will hopefully continue to learn more and more about
the capabilities of these codecs for many years.

Best Regards,

Gary Sullivan
=20

+> -----Original Message-----
+> From: opendtv-bounce@xxxxxxxxxxxxx=20
+> [mailto:opendtv-bounce@xxxxxxxxxxxxx] On Behalf Of Manfredi, Albert E
+> Sent: Monday, September 26, 2005 4:59 PM
+> To: opendtv@xxxxxxxxxxxxx
+> Subject: [opendtv] Re: Comparison of H.264 with MPEG-2
+>=20
+> Craig Birkmaier wrote:
+>=20
+> > Gary Sullivan wrote:
+> > >
+> > > This notion that the gain in compression effectiveness
+> > > for using the new standard instead of MPEG-2 has
+> > > something to do with stretching out the GOP length is
+> > > annoying.  That is definitely *NOT* what is going on,
+> > > and it is *NOT* how we estimate compression benefits in
+> > > codec design.
+> >
+> > Thanks Gary. It's good to see that you are listening, and
+> > willing to take the time to help educate the OpenDTV
+> > community.
+>=20
+> Indeed it was good of Gary to explain the impact of the
+> reduced size of the separate types of pictures (I, P, B).
+>=20
+> The results according to
+>=20
+> http://www.chiariglione.org/mpeg/working_documents/mpeg-04/av
+> c/avc_vt.zi
+> p
+>=20
+> show that if you limit your comparisons to those scenarios
+> that apply to SDTV or HDTV, improvements of 2x or better
+> were not the majority of cases. For SD examples, 2x or
+> better was achieved in 14 out of 40 tests. For HD, 2x or
+> better was achieved in 10 out of 32 tests.
+>=20
+> So even though the GOP length did not change when making
+> these comparisons, GOP length *would* be a factor if you
+> want to show off consistently large imporvements. And
+> that is evidently why GOP length was an issue at IBC, Craig.
+>=20
+> Which is what Gary Hughes pointed out. I don't think either
+> Gary has said "GOP has nothing to do with it."
+>=20
+> Bert
+>=20
+> =20
+> =20
+> -------------------------------------------------------------
+> ---------
+> You can UNSUBSCRIBE from the OpenDTV list in two ways:
+>=20
+> - Using the UNSUBSCRIBE command in your user configuration=20
+> settings at FreeLists.org=20
+>=20
+> - By sending a message to: opendtv-request@xxxxxxxxxxxxx=20
+> with the word unsubscribe in the subject line.
+>=20
+>=20
 
 
----------------------------------------------------------------------
You can UNSUBSCRIBE from the OpenDTV list in two ways:

- Using the UNSUBSCRIBE command in your user configuration settings at 
FreeLists.org 

- By sending a message to: opendtv-request@xxxxxxxxxxxxx with the word 
unsubscribe in the subject line.

Other related posts: