[opendtv] Re: ATSC and Lip Sync

Mark Schubin wrote:



Cliff Benham wrote:


IT SHOULD JUST WORK. PERIOD.

Would that it were that easy.  Here are a few scenarios:
1. A program on storms is showing lightning strikes. They are probably being shot from pretty far away. Should they deliver true lip sync, which would involve the crack occurring perhaps seconds after the flash, or should they fake it?
If its a BBC production the thunder should come properly after the lightning. If its a US production make the thunder simultaneous with the lightning. #;^)
2. A conscientious broadcaster is airing a remote feed via a frame sync, with a matching audio delay. Eventually, the frame sync's buffer will fill and need to be reset, should the audio follow, causing an audible pop, or should it wait, out of sync, until a pause?
I have a relatively inexpensive digital video converter that has 8 frames of storage in it. I can hold the thing in my hand.
Isn't there a broadcast quality synchronizer with enough storage available to not reset in an hour long program or at least until during a local break?
3. A TD/vision mixer is doing an interesting effect, involving pictures passing through several effects systems, each adding a field of video delay. Should the audio mixer delay the audio to match? If so, what should happen when the TD/vision mixer cuts from the effect to the live camera and suddenly loses the video delay? Should the audio pop, should it be out of sync during the effect, or should there be some transition period, again waiting for a pause?
You don't indicate whether this sequence is being produced as a portion of a program to be edited later or if it is a 'live to air' event.
If it is being recorded for later editing, there are probably ways of concealing the audio transition in post production.
If the entire sequence is live to air with music and singers or a visible conductor, any shot that shows a performer 'out of rhythm' with the music will be very disturbing to the viewer. The best alternative here is to use discretion with the effects and minimize close-ups during the sequence. It will be difficult to smoothly make unnoticeable audio transitions live. Likewise, if the sequence is of dancers and lots of long or moving shots are used it will be more difficult to perceive an 'out of rhythm' condition. 

4.  A wireless camera is being used as one of many in a show, and its
encode-decode processing adds latency. Is it better to delay the other dozen cameras and the audio to match, introducing multiple points of failure (and cost) and adding to the director's reaction time? Is it better to delay the audio only when the wireless camera is used? If so, what about split screens or other effects? And what about getting in and out of the audio delay?
Find a wireless camera with latency ~equal~ to the other cameras or use the wireless camera very judiciously.
To solve a problem created by just one camera by 'taxing' the rest and dealing with the technical problems it causes is not worth what the camera brings to the production.
5. A TV show is being presented in a large movie theater. Should the lip sync be correct for someone in the front row? 
NO.
 In that case, it'll

be a frame out for someone roughly a dozen rows back.


Thats nearer to being correct.
A theater director I once worked for designed his shows for the middle third of the audience which were the high priced seats. The sound reinforcement was delayed to match that distance from the stage.
Also, for film in a theater I've read that the sound is acceptable even if its as much as 3 frames behind the picture, but very detectable if it's only two frames ahead.
6. A news show involves a three-way conversation among people at great distances. Is it better to have perfect lip sync or to avoid adding to the conversational delay?
I don't see good lip sync for each person and conversational delay as being mutually exclusive. The conversational delay is by far Einstein's 
greatest curse to back haul TV over satellite.
I watch news shows face this problem every day. They have tried all kinds of ploys like having the local host always end a question or introduction with the distant guest's name as a cue, but it is a workaround at best and a double hop 2 or 3 second delay, -but in perfect lip sync- at best.
Those are just a few issues that don't lend themselves to a "IT SHOULD JUST WORK. PERIOD" solution.
Then there are many situations that DO lend themselves to a simple solution that is simply not being done: frame-rate conversion without compensating delay, Dolby E or AC-3 coding without compensation for both encode and decode delays, etc. I would love to see those wiped out through training and, perhaps, the expenditure of a little money (audio delay capability is built into many audio consoles these days, and in post it can just be slewed).
One engineer has commented to me that the problem with their satellite decoder audio is that management won't pay the licensing fee to use the 'better' audio coding available to them.
He also believes that embedding the audio at the source would help eliminate lip sync problems down the line.
And then there are consumer decoders -- not just ATSC, but also cable and satellite (I don't have any telco-video home experience, so I can't speak to theirs). They can lose lip sync even when the broadcaster does everything right. I share your frustration, and I applaud your complaint, but I'd like to direct it appropriately.
My greatest complaint about consumer decoders is that when there is a very slight video breakup, there is a corresponding 6 to 7 second period of dead silence until the decoder 'catches up' again. Infuriating when there are two or three in a row.
By all means, send a complaint to the FCC about not requiring frequent PTS synching in receivers. I point out merely that it is not the ATSC's fault.
I misused ATSC in the original post. I should have said 'digital video' instead. 

Cliff


TTFN,
Mark





----------------------------------------------------------------------
You can UNSUBSCRIBE from the OpenDTV list in two ways:
- Using the UNSUBSCRIBE command in your user configuration settings at FreeLists.org 
- By sending a message to: opendtv-request@xxxxxxxxxxxxx with the word 
unsubscribe in the subject line.

Other related posts: