I'm working in parallel on the Gif plugin and I think it helps for defining the abstraction level. I would like to simplify things as much as possible as viewed from the player, and hide implementation details.
One thing I think I was viewing differently than in your design, is who interacts with the Buffer. Instead of having the Buffer as a middle man between the Decoder and the Renderer, It could be an implementation detail of the Decoder.
Super high level overview <https://docs.google.com/drawings/d/17_2MmbrbaxTB4905eSycmFM-AAtWkbsXLlANbSYNsd0/edit?hl=en_US> of (how I understand) the two alternatives.
Here is an alternative for decoders. It more resembles an enumerator, with a "Current" property (I admit the existing design is also like an enumerator so it will ease the refactoring :-)).
{
Open(path);
Close();
GetThumbs();
MoveNext();
MoveTo(timestamp);
VideoFrame Current { get; }
}
These would be the basic primitives for decoders, and would be called
directly from the rendering/controller thread. Move* methods would set
the "Current" reference to point to a valid frame.
(not showing Cache management, which is another beast that can't be
completely hidden from player).
Some higher level methods like MovePrev(), MoveFirst(), MoveBy(frames),
can also be implemented in the base class for decoders and mapped on
implementer's MoveTo(timestamp).
Now where the Buffer would fit into this ? It would just help implement MoveNext(). The background thread would start immediately at Open() and fill buffer continuously.
- MoveNext() would be implemented to return super fast:If the frame has been decoded by the background thread and is in the buffer, just update ref to "Current". If not we will call it a drop and keep working with whatever is in "Current".
- MoveTo() would check if the target frame is in the buffer, if not: perform the seek, update "Current", and then return.
It doesn't have to be as fast as MoveNext, no concept of drops.You are very right with keeping some of the previous frames around. So I guess it's not a Queue after all. The behavior will be entirely defined by which frame gets unloaded first when buffer is full.
The buffer may end up being very much like the cache, just more dynamic (and thread safe). I guess the Cache could actually be implemented completely within the Buffer class, it would just contain all the frames from the working zone instead of a sub section around the current cursor. Will have to think more about this, that would simplify the architecture. It's probably independant from where the buffer actually fit in the system.
joan. Le 19/08/2011 07:59, Hugh a écrit :
Would something like this work: Notes: ·location is always a position in the video·render calls buffer for image which gets it from itself and gets required items from decoder·render creates image in the background the when ready swaps them to the visual surface·render holds a list of tools to do changes to the image ·classes can be overloaded (so different decoders or renders can be used) public class Decoder { // Construction, loading public Decoder();public result LoadVideo(stream/location); // Loads video from stream or given locationpublic properties GetProperties(); // Returns a structure with video propertiespublic int SetProperty(name, value); // Set a property// Extractionpublic frame[] GetFrames(start, [optional] end); // Returns frames in the rangepublic frame[] GetThumbs([optional] number = 1, [optional] start offset); // Returns thumb framespublic int FillBuffer(buffer, start, end); // Checks buffer and passes required frames to the buffer PutFrame (one at a time to put frame as async method as they are decoded, not a group like getframes)} public class Buffer { // Construction, loading public Buffer();public LoadDecoder(decoder); // Passes a decoder to usepublic SetSize(ahead, [optional] behind = ahead); // Sets buffer size before and after locationpublic result GetSize(); // Updatingpublic int UpdateBuffer([optional] current location); // Checks buffer and fills up as required using the decoder (returns if update was performed)public bool UpToDate(); // Passes true if buffer is full and to the right locationpublic result GetStatus(); // Passes number of empty frames ahead and behindpublic SetLocation(location); // Sets location in the video without updatepublic result GetLocation();public Clear(); // Empties the buffer// Extractionpublic frame GetFrame(location, [optional] update buffer = 1); // Gets the frame and moves buffer (will also need to get from decoder if missing)public int PutFrame(location); // Lets decoder place a frame into the buffer at location} public class Render { // Construction public Render();public LoadBuffer(); // Passes a buffer to usepublic SetSurface(hwnd); // To screenpublic bool SwapFrame(); // Places rendred frame onto surface (back buffer to screen). Checks frame is ready to be drawn and returns true on sucess// Load next framepublic bool Render(location); // Gets image from buffer and starts drawing onto it (done in background memory then when is finished SwapFrame can move it to the visable surface// Toolspublic id AddTool(tool); // Adds a tool to render onto the frame (like drawing, filter etc)public RemoveTool(id); // Removes a tool... }*From:*kinovea-dev-bounce@xxxxxxxxxxxxx [mailto:kinovea-dev-bounce@xxxxxxxxxxxxx] *On Behalf Of *Joan*Sent:* Thursday, 18 August 2011 2:08 AM *To:* kinovea-dev@xxxxxxxxxxxxx *Subject:* [kinovea-dev] Re: C# 3.0 in Kinovea sources Thinking about it again, this is probably the wrong level of abstraction.The player screen should not have to know all this stuff about buffering, etc. It just wants a frame to render.There is also a need to support more file formats than what FFMpeg can offer, specifically a reader for animated GIFs and a reader for a sequence of individual images. The abstraction thus needs to be a level higher. All file readers must implement common services that other parts will use. (getting thumbnails, moving to next frame, moving to a given spot, etc.)The fact that the FFMpeg-backed reader will use a buffer is probably an implementation detail of this particular reader. The animated GIF reader may on the other hand load the entire content in memory righ away.So, before diving into asynchronous decode, the abstraction layer for file readers must be in place with a clean interface. Ideally the extra readers would be loaded dynamically in a plug-in fashion.I'll probably start a branch for this because it will break everything ! Le 16/08/2011 15:05, Joan a écrit : Regarding threading:Trying to organize thoughts I wrote a small page on the wiki with tentative system.http://www.kinovea.org/wiki/doku.php/codeasyncdecodeIt is a bit more complicated than what I wrote in the last mail because it also account for dropping in the decoding thread. If you have some time please review it and tear it appart :-) Especially scenario 2 which has some blurry zone. When the decoding is not ready at the right time, should we present the frame later on if we don't have anything more recent, or drop it anyway ?Thanks
