[haiku] Re: GSoC: Writing native interfaces for ported applications
- From: David McPaul <dlmcpaul@xxxxxxxxx>
- To: haiku@xxxxxxxxxxxxx
- Date: Wed, 14 Apr 2010 22:30:27 +1000
On 14 April 2010 07:48, Christopher Humphries <redeye4@xxxxxxxxx> wrote:
>> This is an area we need to talk about. I am currently working on a
>> video mixing node (combines multiple video sources into one) with the
>> aim of handling subtitles and overlays.
>>
>> I was also intending to write a timed text decoder.
>
> Sorry for the late answer, moving apartments took a bit of time.
> Subtitle decoding will be a tricky subject. I would prefer not to send
> redundant video frames, and just send one for each set of subtitles, and for
> DVD's these frames would be bitmaps.
> The format provides a bitmap with start time and duration, and I thought it
> would be possible to send regular video frames with a long latency, or would
> it be better to implement a new kind of "bitmap track"?
I have been reading up on the mp4 timed text format which is what I
was intending to implement. That then blends into other subtitle
formats that come with avi. They are all text with fonts and effects
commands.
In an MP4 file subtitles are a seperate set of tracks 1 per language
where each piece of text is basically a string with a start time and a
duration but there are also other things you can do to the text
(highlight, placement, effects, fonts).
The MP4 extractor just needs to expose the subtitles as a track with a
codec id and then a decoder can handle that track and produce bitmaps
from the text in the right font and placement with a constant
framerate (Only because I am not sure we can handle a variable video
frame rate).
The VideoMixer will not care how the bitmaps are produced, it will
just merge streams together and output at the framerate of the primary
stream.
I expect the DVD subtitle tracks would be similar and just like
MediaPlayer recognises multiple audio tracks and allows selection
between them it will need to also handle multiple subtitle tracks and
allow selection.
> It might even be foreseeable to fuse both text and bitmap based subtitles at
> one point, as I imagine that text based subtitles would have to be provided
> as bitmap at one stage.
Yes text subtitles would be rendered as bitmaps. Bitmap subtitles
might actually be tricker as it would be seen as a seperate video
track.
> After further thought, menu overlays would need a lot more flexibility...
> Do you have any ideas about mouse interaction with overlays?
I am not sure how menus are presented in DVDs. The mediaplayer would
have to detect the mouse clicks etc and know what was hit etc. I
will start reading up on that and see what ideas I have.
> Anyway, I'm just thinking aloud...
Sure.
Hopefully I will have the VideoMixer in a state soon that I can commit it.
--
Cheers
David
Other related posts:
