[kinovea-dev] Re: Direct2D

Hi,

1. FFmpeg update
Generally I update the ffmpeg during the first experimental version and then leave it until the following official release. 
The last update was thus right after 0.8.7, last year.
(There is also a bit of info in "history.txt" file in \Refs\FFMpeg-r23012\)
It would definitely be beneficial to update to a more recent version, and it is planned for the 0.8.16 version (considering 0.8.15 to be the next stable release).
However I'm a bit concerned with the recent mess (fork initiated by some members with web server admin rights, Ramiro Polla stopping his Windows builds, etc.). From my (limited) experience and knowledge of the project, I put all my confidence in Michael Niedermayer branch to be top notch, and I'd like to use his repository. They have moved versionning to git, I don't know if someone has taken over automated builds for Windows (Could you check ?).
By the way, in addition to bug fixes, WebM, and other stuff, they also have a new library libavfilter that will be interesting for some future functions. (Like image rotation, better deinterlacing, etc.). 

2. Threading
If I'm not mistaken the ffmpeg decoding is already running in its own thread (the thread of the multimedia timer, which is spawned from the UI thread in PlayerScreenUserInterface). 

3. Input buffer
I'm all for an input buffer. I haven't thought much about how to do it though. (When to decode without taking all CPU ? How many frames ? etc.) If you want to dive into this, you're welcome, let me know if anything is unclear. I guess I should start by translating the comments that are still in french. (Interfacing with ffmpeg routines has always been the most laborious part of the code for me…). 

4. Misc
In the months after the next release I would also like to initiate two other refactoring efforts : 1. One concerning the drawing tools (I have already started to work on it) to make it easier to add and manage drawing tools and drawings. 2. The other concerning the decoding, to add a layer of abstraction to the "VideoFile" class, for file types that are not supported by ffmpeg (so we can switch file reader on the fly). This is happening with animated GIF (easy to work with in .NET but ffmpeg only gets the first frame). 

Joan.



Le 28/04/2011 03:59, Hugh a écrit :
Just wondering when the ffmpeg was last updated? Would a newer version bring better results (I noticed VC8 is not in the headers). Additionally would it be easy to get it to run in a separate thread in the playerserver code and create an in/out style memory buffer there where frames are loaded ahead of them being called?
*From:*kinovea-dev-bounce@xxxxxxxxxxxxx [mailto:kinovea-dev-bounce@xxxxxxxxxxxxx] *On Behalf Of *joan@xxxxxxxxxxx 
*Sent:* Saturday, 15 January 2011 12:43 AM
*To:* kinovea-dev@xxxxxxxxxxxxx
*Subject:* [kinovea-dev] Re: Direct2D

I wrote a page on the wiki to try to describe two possible architectures.

http://www.kinovea.org/wiki/doku.php/coderenderingpipeline

Feel free to modify it, add comments, opinion, pros/cons, etc.

joan.

On Thu, 13 Jan 2011 22:18:55 +0100, Joan wrote:

    Hi,

    I have tried the Direct2D / SlimDX code you sent.

    I wasn't too sure as to how exactly use the control, here is what
    I did:
    - Added a Direct2DControl member to PlayerScreenUserInterface.
    - In the PlayerScreenUserInterface constructor I call
    Direct2DControl.SetControl() passing in the center panel (to test
    on the back panel for now)
    - immediately call Direct2DControl.Run() to trigger initializations.
    - At each new frame, (in PlayerScreenUserInterface.ShowNextFrame),
    I call Direct2DControl.DrawVideoBitmap() passing in the decoded image,
    - and Direct2DControl.Update() to actually draw on the panel.

    Let me know if I did anything wrong. Did you also modify
    PlayerScreenUserInterface in this way for your tests ?

    It seems to work pretty well.
    I've tried to instrument and get some figures regarding speed, but
    it proved harder than I thought to really pinpoint the rendering time.
    (Also I've recently changed machine and it's like everything is
    running super fast :-))

    The part taking the most time (about 10 to 20ms each frame on my
    machine) would seem to be DrawVideoBitmap().
    This function involves copying the whole GDI+ bitmap into a
    Direct2D bitmap for later use. This is a step we can probably
    avoid completely by decoding directly into a Direct2D bitmap from
    the FFMpeg code.

    This is maybe what you suggested in the forum thread... In the
    case of pre-extraction to memory, we can also probably keep an
    in-memory array of Direct2D bitmaps instead of an array of GDI+
    bitmaps. I'm sure we'll find a way to avoid this extra copy.

    About your concern to release resources, I don't think we need to
    use a separate form for rendering and react to the OnClose() event
    of it.
    Screens must implement a BeforeClose() method (see AbstractScreen)
    which is triggered when the screen is closed (close button,
    changing screens configuration, etc.)
    We should be able to use it to ask the PlayerScreen and the
    renderer to dispose any outstanding resources.

    We'll probably need to transform and simplify the DirectXInterface
    class.
    Also, I don't know if we need the other classes,
    (Direct3D10Context, etc.) ? If not, we might even merge the two
    for simplicity.

    I haven't looked at the direct2D drawing primitives and how we
    might implement the drawing tools visuals.
    The architecture discussion will probably depend on how these can
    be abstracted between Direct2D and GDI+.

    I will create a topic on the wiki to push some schematics of
    possible architectures so we are clearer about what we discuss.
    Let me know of your experiments.

    Thanks,
    joan.




    Le 10/01/2011 21:34, Joan a écrit :

    Hi!

    I am super sorry for the delay.
    Unfortunately, I haven't had much time to look into this, but I will.
    (I have some issues at work with the company wanting to close the
    local office, this has somewhat prevented proper focus on the
    project, but it's going to resolve one way or the other in the
    next few weeks.)

    Thanks
    joan.


    Le 05/01/2011 17:34, Hugh a écrit :

    Hey Jean,

    Was just wondering how the Direct2D code was working for you?  Did
    you have any luck moving the drawing onto a new form?  Just didn’t
    see any recent changes in the code and was hoping it is working!

    Thanks,

    Hugh

On Thu, 13 Jan 2011 22:18:55 +0100, Joan wrote:

Other related posts: