[kinovea-dev] Re: C# 3.0 in Kinovea sources
- From: Hugh <greatview@xxxxxxxxx>
- To: <kinovea-dev@xxxxxxxxxxxxx>
- Date: Sun, 21 Aug 2011 19:36:31 +1000
There would be advantages either way. My worry would just be if you end up
double coding things and a complication from too much in the one control.
However there could be performance gains mixed together.
Just the decoder will then be the threading and locking will have to be
mixed through. With separate classes it could be split apart. You want to
be able to take images out of the cache while decoding more into it (don?t
want to have to wait for a whole group of frames to decode before pulling
one out).
From: kinovea-dev-bounce@xxxxxxxxxxxxx
[mailto:kinovea-dev-bounce@xxxxxxxxxxxxx] On Behalf Of Joan
Sent: Friday, 19 August 2011 8:48 PM
To: kinovea-dev@xxxxxxxxxxxxx
Subject: [kinovea-dev] Re: C# 3.0 in Kinovea sources
I'm working in parallel on the Gif plugin and I think it helps for defining
the abstraction level.
I would like to simplify things as much as possible as viewed from the
player, and hide implementation details.
One thing I think I was viewing differently than in your design, is who
interacts with the Buffer.
Instead of having the Buffer as a middle man between the Decoder and the
Renderer, It could be an implementation detail of the Decoder.
Super high level overview
<https://docs.google.com/drawings/d/17_2MmbrbaxTB4905eSycmFM-AAtWkbsXLlANbSY
Nsd0/edit?hl=en_US> of (how I understand) the two alternatives.
Here is an alternative for decoders. It more resembles an enumerator, with a
"Current" property (I admit the existing design is also like an enumerator
so it will ease the refactoring :-)).
{
Open(path);
Close();
GetThumbs();
MoveNext();
MoveTo(timestamp);
VideoFrame Current { get; }
}
These would be the basic primitives for decoders, and would be called
directly from the rendering/controller thread. Move* methods would set the
"Current" reference to point to a valid frame.
(not showing Cache management, which is another beast that can't be
completely hidden from player).
Some higher level methods like MovePrev(), MoveFirst(), MoveBy(frames), can
also be implemented in the base class for decoders and mapped on
implementer's MoveTo(timestamp).
Now where the Buffer would fit into this ? It would just help implement
MoveNext().
The background thread would start immediately at Open() and fill buffer
continuously.
- MoveNext() would be implemented to return super fast:
If the frame has been decoded by the background thread and is in the buffer,
just update ref to "Current".
If not we will call it a drop and keep working with whatever is in
"Current".
- MoveTo() would check if the target frame is in the buffer, if not: perform
the seek, update "Current", and then return.
It doesn't have to be as fast as MoveNext, no concept of drops.
You are very right with keeping some of the previous frames around. So I
guess it's not a Queue after all. The behavior will be entirely defined by
which frame gets unloaded first when buffer is full.
The buffer may end up being very much like the cache, just more dynamic (and
thread safe).
I guess the Cache could actually be implemented completely within the Buffer
class, it would just contain all the frames from the working zone instead of
a sub section around the current cursor. Will have to think more about this,
that would simplify the architecture. It's probably independant from where
the buffer actually fit in the system.
joan.
Le 19/08/2011 07:59, Hugh a écrit :
Would something like this work:
Notes:
location is always a position in the video
render calls buffer for image which gets it from itself and gets required
items from decoder
render creates image in the background the when ready swaps them to the
visual surface
render holds a list of tools to do changes to the image
classes can be overloaded (so different decoders or renders can be used)
public class Decoder
{
// Construction, loading
public Decoder();
public result LoadVideo(stream/location);
// Loads video from stream or given location
public properties GetProperties();
// Returns a structure with video properties
public int SetProperty(name, value);
// Set a property
// Extraction
public frame[] GetFrames(start, [optional] end); // Returns
frames in the range
public frame[] GetThumbs([optional] number = 1, [optional]
start offset); // Returns thumb frames
public int FillBuffer(buffer, start, end);
// Checks buffer and passes required frames to the buffer PutFrame (one at a
time to put frame as async method as they are decoded, not a group like
getframes)
}
public class Buffer
{
// Construction, loading
public Buffer();
public LoadDecoder(decoder);
// Passes a decoder to use
public SetSize(ahead, [optional] behind = ahead); // Sets
buffer size before and after location
public result GetSize();
// Updating
public int UpdateBuffer([optional] current location);
// Checks buffer and fills up as required using the decoder (returns if
update was performed)
public bool UpToDate();
// Passes true if buffer is full and to the right location
public result GetStatus();
// Passes number of empty frames ahead and behind
public SetLocation(location);
// Sets location in the video without update
public result GetLocation();
public Clear();
// Empties the buffer
// Extraction
public frame GetFrame(location, [optional] update buffer =
1); // Gets the frame and moves buffer (will also need to get from
decoder if missing)
public int PutFrame(location);
// Lets decoder place a frame into the buffer at location
}
public class Render
{
// Construction
public Render();
public LoadBuffer();
// Passes a buffer to use
public SetSurface(hwnd);
// To screen
public bool SwapFrame();
// Places rendred frame onto surface (back buffer to screen). Checks frame
is ready to be drawn and returns true on sucess
// Load next frame
public bool Render(location);
// Gets image from buffer and starts drawing onto it (done in background
memory then when is finished SwapFrame can move it to the visable surface
// Tools
public id AddTool(tool);
// Adds a tool to render onto the frame (like drawing, filter etc)
public RemoveTool(id);
// Removes a tool
...
}
From: kinovea-dev-bounce@xxxxxxxxxxxxx
[mailto:kinovea-dev-bounce@xxxxxxxxxxxxx] On Behalf Of Joan
Sent: Thursday, 18 August 2011 2:08 AM
To: kinovea-dev@xxxxxxxxxxxxx
Subject: [kinovea-dev] Re: C# 3.0 in Kinovea sources
Thinking about it again, this is probably the wrong level of abstraction.
The player screen should not have to know all this stuff about buffering,
etc. It just wants a frame to render.
There is also a need to support more file formats than what FFMpeg can
offer, specifically a reader for animated GIFs and a reader for a sequence
of individual images.
The abstraction thus needs to be a level higher. All file readers must
implement common services that other parts will use. (getting thumbnails,
moving to next frame, moving to a given spot, etc.)
The fact that the FFMpeg-backed reader will use a buffer is probably an
implementation detail of this particular reader. The animated GIF reader may
on the other hand load the entire content in memory righ away.
So, before diving into asynchronous decode, the abstraction layer for file
readers must be in place with a clean interface. Ideally the extra readers
would be loaded dynamically in a plug-in fashion.
I'll probably start a branch for this because it will break everything !
Le 16/08/2011 15:05, Joan a écrit :
Regarding threading:
Trying to organize thoughts I wrote a small page on the wiki with tentative
system.
http://www.kinovea.org/wiki/doku.php/codeasyncdecode
It is a bit more complicated than what I wrote in the last mail because it
also account for dropping in the decoding thread.
If you have some time please review it and tear it appart :-) Especially
scenario 2 which has some blurry zone. When the decoding is not ready at the
right time, should we present the frame later on if we don't have anything
more recent, or drop it anyway ?
Thanks
Other related posts:
