[opendtv] Lip sync problem resurfaces

  • From: "Manfredi, Albert E" <albert.e.manfredi@xxxxxxxxxx>
  • To: "OpenDTV (E-mail)" <opendtv@xxxxxxxxxxxxx>
  • Date: Mon, 19 Apr 2004 17:03:49 -0400

Synching Lucy's lips with TV sound
By Junko Yoshida , EE Times
April 19, 2004 (2:08 PM EDT)
URL: http://www.eet.com/article/showArticle.jhtml?articleId=3D18901792

PARIS - In movie theaters of yesteryear, it was not uncommon for
the audio and picture to be out of synch, with the silver-screened
actors' mouths moving and sound coming a half-second or more later.
It was amusing until it became annoying, and patrons invariably
yelled out to the projectionist, who restarted the film.

Technology eventually solved that problem, but in a bizarre piece
of irony, the technology has gotten so sophisticated that the
problem has resurfaced.

TV makers have only recently begun to recognize the problem of
audio information being asynchronous with lip movement on
flat-panel TVs, realizing that it could become a bigger issue as
sales of flat-panel displays start to take off on the consumer

Average CRTs do not have audio and video synchronization problems
because the time needed to process video and audio - thousandths
of a second - is about the same, said Stefan Hepp, director of
marketing of consumer audio at Micronas GmbH (Freiburg, Germany).
But when advanced TVs, LCDs and plasma displays add enhanced
picture-improvement functions like deinterlacing, motion-adaptive
conversion and scaling, the result is video processing that takes
more time than audio processing, he said. So audio processing time
must be adjusted to match the video.

Observers estimate that two-thirds of flat panels - except for
low-end versions - will need a lip-synching device.

Some display vendors are said to be unaware of the potential
problem and some simply prefer to deny it. Others, forced to look
more closely, have just started, reluctantly, to acknowledge the
lip-voice lag. That could blow into a bigger issue among consumers,
Hepp cautioned, after "consumer test magazines begin to make
measurements on the audio and video asynchronous problem."

Paul Martin, marketing manager for digital TV systems at Philips
Semiconductors, agreed. Video-processing delays "started to become
a problem on LCD displays recently," he said. While LCD displays
could introduce video delays with 15- to 20-millisecond response
times, memory-based video-processing algorithms, often used in
mid-to high-end displays, could also add visible video delays, he
explained. Combine the two and, "You could have a considerable
delay in video - more than 100 milliseconds in some cases."

While the large high-end PDP or LCD panel markets can certainly
afford an additional chip or high-end SoC with audio delay
functions, the real emerging battleground is the midrange LCD or
PDP market, where the cost adder for a 30- or 40-inch flat panel
is not trivial and where the technological problem (or cost of
the solution) could stall a promising market.

Micronas' Hepp cited "motion-adaptive conversion," "scaling" and
"deinterlacing" as three main sources of video delays built into
TV electronics. He also said that some plasma displays, which
need to put data into internal memory before lighting the panel,
could introduce 20 to 60 milliseconds of delay. Overall
system-level video delay can vary among flat-panel displays,
because system vendors often mix different types of scalers or

The industry does not agree, however, on the threshold at which
audio/video asynchronization becomes a problem. How many
milliseconds will it take a consumer to notice that Lucy's lips
aren't catching up to her words? And how long before the
consumer gets irked?

A test by a German university late last year found viewers
began to notice things were "a little strange" at 20 ms, said
Hepp. A 30-ms delay began to irritate them. Beyond 30 ms, they
said, "I don't want to have this on my TV."

Meanwhile, Philips' Martin believes the delay only becomes
noticeable at 100 ms. While no objective data is available,
Philips, reportedly in consultation with customers, has decided
a 50-ms delay is the threshold. "When the delay is below 50 ms,
nobody will be able to see it," Martin said.

Meanwhile, iSuppli Corp.'s principal analyst, Shyam Nagrani,
cited a far lower number. "It can be a big problem if the
deinterlacing delay is greater than one field, i.e., 16.6 ms,"
Shyam said. "Different deinterlacers use two, three, four or
five fields. Most of the deinterlacers use only two fields. It
means that the delay is little more than one field."

Micronas will introduce two devices, both leveraging on-chip
RAM to delay audio information to keep lip movements in synch.
One is a programmable delay function integrated onto the
company's Multistandard Sound Processor family. The other, a
standalone audio delay chip, integrates a controller and RAM.

Micronas targets the fast-growing, mainstream flat-panel TV
market segment with its Multistandard Sound Processor chip,
called MSP 44/46xyK, featuring the new on-chip delay
functionality. Delay time can be programmed to up to 80 ms
(mono) or 40 ms (stereo), the company said.

If longer delays are needed, Micronas' Hepp said, OEMs can
use a new standalone audio delay IC, called MAD 4868A. It can
function as a companion chip to Micronas' sound processor or
anyone's audio DSP. The configurable delay time in such a
setup is between 680 ms (mono) and 85 ms (eight channels). To
optimize cost, there is no active component on the MAD 4868A,
Hepp said.

Alternatively, OEMs can address lip sync with separate
chips - an external RAM and a RAM controller. "That would
require as many as 40 connecting wires to connect the two.
There are also significant EMC issues," Hepp said. Both chips
are available in samples today with volumes scheduled for
early summer.

While Micronas has taken the on-chip RAM approach to delay
the audio and solve the lip-synching problem, there are
different approaches.

Philips, for one, "fixed the problem," said Martin, via a
system-level design by tweaking video enhancement software
to minimize delay while adding a new feature - which
automatically delays audio to match video - to its high-end
Nexperia Home Entertainment Engine: the PNX8550. The IC,
unveiled last year, is a system-on-chip TV that supports all
digital and analog TV standards; CRT, LCD and plasma
displays; 100-Hz progressive-scan video; high-definition
video decode and display and many picture-enhancing features.
Because audio/video processing is on the same chip, sharing
the same memory, Philips did not have to integrate more
memory for audio delay, Martin said. "Although we haven't
heavily advertised it, lip-synching is one of the features
on PNX8550."

Digital TVs using PNX8550 are expected to reach the consumer
market later this year.

Meanwhile, there's an even simpler solution, said iSuppli's
Shyam. That is "to use two-field deinterlacing where the
video quality is almost as good as three- or four-field
processing." Most customers could not tell the difference
between two- or three-field processing, Shyam said. "So in
the interests of costs and complexity we stuck with
two-field deinterlacing."

Copyright =A9 2003 CMP Media
You can UNSUBSCRIBE from the OpenDTV list in two ways:

- Using the UNSUBSCRIBE command in your user configuration settings at 

- By sending a message to: opendtv-request@xxxxxxxxxxxxx with the word 
unsubscribe in the subject line.

Other related posts: