[haiku-development] Re: BMediaTrack::SeekToFrame/Time()

Hi David,

thank you for your insights!

David McPaul wrote:
> Seeking is one of the areas I have struggled with in developing the
> 3ivx decoder for BeOS.  Just when I think I got it right I find a
> problem.
> 
> Anyway, my current understanding is that the decoder is passed the last 
> keyframe decoded AND the frame that is required.  The next ReadChunk call 
> will return the chunk associated with the keyframe. The decoder then 
> decodes forward until it reaches the frame required. This has always been 
> in a forward direction which is the only thing the decoder can do (well 
> any video decoder that relies on previous frames).  This is how the 3ivx 
> decoder works.  When decoding frames that are not required for display it 
> can shortcut some of the decoding so it is quicker to do.  But video with 
> few keyframes will be a problematic.
> 
> The extractor though is often requested to find a keyframe in a backward 
> direction presumably to set up the call to the decoder.
> 
> I think the decoder can refuse to do anything but seek to keyframes by 
> returning the keyframe instead of the required frame.
> 
> All the above applies to time based seeking as well.
> 
> I originally thought that the media kit did all the work and the decoder 
> just decoded a frame whenever it was called but the decoder needs to also 
> know that it is just seeking forward and does not need to do all the 
> decoding work.
> 
> Of course I could be wrong, my MP4 extractor with seeking added in is 
> pretty broken but I think that is because there is some code somewhere 
> that disagrees with my audio/video sync and tries to adjust it.  But what 
> I have done above seems to be fine for AVI and MOV files under BeOS.

Ok, I think I understand what you mean. However, I don't like the API for 
this very much. I mean with regards to the API that codecs implement. 
Instead of this "implicit logic", there should be setup calls which 
configure certain behavior. And the behavior of the BMediaTrack API should 
be consistent:

1) SeekToFrameTime() will _always_ a) seek to exactly the frame you 
requested, if you do not pass _CLOSEST_BACKWARD/FORWARD flags. Or b) 
_always_ seek to the closest keyframe (forward or backward, whichever one 
is closer). Not depending on the specific codec, but implemented in 
BMediaTrack.

2) The Codec API should get a call, with which it is possible to configure 
the behaviour (like the quality or "no output", skipping colorspace 
conversion and deblocking and so on). I have seen code, which tries to use 
the media_decode_info passed to Decode() to figure out what quality it 
should use. This results in different codecs behaving differently and 
duplicating code. If there is a need to hurry up with decoding in order to 
maintain realtime, then I think the implementation to figure this out 
should be one level up, using a new API to configure the codec for certain 
behavior.

Another idea could be this: If the codec does not get passed a buffer for 
decoding, it should decode, but skip output. Because there is another 
problem that the output buffer is provided by the client code, and this way 
we can avoid a temporary/additional allocation. That could be a fairly 
straight forward convention on top of what I already wrote.

What do you think? How is it done in other APIs, can you share some 
insights?

Best regards,
-Stephan

Other related posts: