[gmpi] Re: Item 0: Agenda

  • From: "gogins@xxxxxxxxxxxx" <gogins@xxxxxxxxxxxx>
  • To: gmpi@xxxxxxxxxxxxx
  • Date: Wed, 12 Feb 2003 13:18:33 -0500

I think a better picture of the underlying problems would result from
replacing items 4, 5, and 10 below with one item, "representation of music."

I also suggest that you add an agenda item, "audience for and users of
plugins."

And an item for "signal routing" (just in/out, plugins inside plugins, full
multirate synchronous data flow graph, etc.).

And an item for "plugin state machine."

Original Message:
-----------------
From:  RonKuper@xxxxxxxxxxxx
Date: Wed, 12 Feb 2003 10:35:30 -0500
To: gmpi@xxxxxxxxxxxxx
Subject: [gmpi] Item 0: Agenda


Thanks to Laurent for collating all the discussion on the existing +++ RESET
+++ thread.  I have taken this list, and tried to simplify it down into
items to be discussed among the group.  This means some lists may be shorter
than Laurent presented, some some list items were in fact "answers" to
questions that we have not yet discussed.  I have also incoroporated Chris
Grigg's "North Star" goals.  (Laurent's and Chris' original lists are
appended to the end.)

Again, this is a proposal for the agenda.  The purpose of this thread is to
refine and finalize the agenda.

1.  Cross-platform
What processors, O/S, addressing models and programming languages do we need
to support?

2.  Plugin flavors
Audio and/or MIDI processors, sure, but what about driver plugins,
controllers, visuaulization plugins?

3.  Host interface
In-process or out-of-process?  What kinds of special needs are there in
events to/from a plugin?  Is the host a plugin too?

4.  Audio packaging
What do we prefer for interleave, audio sample data types, allocation,
timestamping?

5.  Time representation
Samples or "ticks" or both/neither?  How are discontinuities expressed?

6.  Parameter representation
What data type for values, for names?  How do we support validation, value
distribution?

7.  Persistence
Whose reponsibility?  How do we deal with endian issues?

8.  Threading
What kind of realtime guarantees?  Does the host provide some kind of
threading environment?

9.  GUI
Can GUI be avoided at this point?  What does the interface boundary to the
GUI need to be?

10.  The role of MIDI
How important is it?  How do we ensure something new is still backwards
compatible?

11.  Component packaging
Is there a "base level" possible that's still cross platform?  How do we
support all target platforms?

12.  Enumeration (creation, discovery, etc.)
Create for use vs. query, what can a plugin do at creation time, RT safety?

13.  Copy protection
How should plugins address DRM streams?  How to sign OpenSource plugins?

14.  Localization
How does the host specify its language?  Logical vs. human-readable names?

15.  DSP state
When and how does a plugin reset?  How to respond to format changes?

17.  Source code
How are headers and host libraries/source licensed?  What kind of sample
code is required?

______________________________________________________________________

Appendix A.  Summary of items from +++RESET+++ posts.

1.  Cross-platform
   a. Different processors
   b. Different operating systems, including Windows, Mac,
      Linux, PDA's, embedded
   c. Write once, compile many
   d. At the level of standard C or STL
   e. cross-platform DSP vs cross-platorm GUI
   f. which languages are supported? do we support bindings
      from others?
   g. ability to resolve accross address spaces or not
   h. define limits of cross-platform specifications

2.  Kinds of plugins we seek to build
   a. Audio processors
   b. MIDI processors
   c. Audio processors with MIDI control
   d. No codecs (no changing sample format)
   e. Kinds of plugins matrix.
   f. Driver plugins. Wraps MIDI and audio APIs for GMPI
      hosts to use intead of directly supporting all APIs
      on a platform. Related to 3f, I think.
   g. controllers - designed to control other knobs (LFP,
      envelope, peak-follower)
   h. Anaylsers, visualisation (Cthugha-like)

3.  Interface to hosts
   a. In-Process and/or Out-of-Process execution model?
   b.1 Future work: define a layer for out of process.
   b.2 Future work: recommend protocol for networked
       communication (several good ones emerging, the
       design + implementation of which are orthogonal
       to our issues, i believe)
   c. Parameters and state changes are all kinds of "events"
   d. Events are sent on a non-blocking FIFO
   d.2 ...or as structs in linked list queues, where events
       are fixed allocated from a host managed pool.
   e. Events are timestamped (or not...<G>)
   f. Hosts can provide sources and sinks for disk and
      soundcard
   f.2 ...or 2e; hosts can use (or provide built-in) Driver
       plugins.
   f.3 Support for source and sinks representing other
       applications (as in ReWire, JACK, DirectConnect etc.)
   g. sources and sinks should certainly be able to be
      plugins. hinted audio ports and controls - to make
      automatic connections easier
   h. property listeners and parameter listeners vs. polling

4.  Audio packaging
   a. Interleaved or multiple mono
   b. One assumed data type for audio, defined at compile
      time per CPU/OS
   c. Buffers allocated by the plugin at defined points in
      object lifetime. This depends a great deal on what
      you mean by "buffers".
   d. Buffers allocated by host, perhaps, rather than plugin
   e. Facilitate streaming to external DSP hardware
   f. A method for the host to specify memory alignment of
      a particular audio buffer, and to guarantee a minimum
      buffer alignment. Same for plug-ins
   g. Are audio buffers timestamped? (rationale: should
      plugins be able to work on anything except "the
      current buffer" ?) AU says yes, most other say no.
   h. denormals (plugin concern or host concern or CPU state
      concern)

5.  Time representation
   a. Audio wants samples, MIDI/music wants ticks,
      measures, beats
   a.1 Define and justify "tick"
   a.2 Define units of measures and beats
   b. Timecode would be nice
   c.1 Plugin has access to a host's sample rate, PPQN and
       tempo map / "conductor track"
   c.2 support for "conductor track" per <something>. many
       experimental composers like this (e.g. one CT per
       track, with different tracks running at different
       tempos/meters)
   d. Host-defined 64 bit integer tick per second value
      required to be constant during run-time. All times
      and sample rates are represented in this tick's units.
   e. handling non-linear transport motion (e.g. host is
      looping) (most (all?) existing APIs don't do this well)
   f. "Pre-buffer markers" or other means of telling HDRs
      and the like about target points for which zero
      latency skipping at any time must be possible. (For
      looping, conditional jumps in interactive compositions
      and the like.)

6.  Parameter representation
   a. Define data types, or data type definition mechanism.
   b. BLOB parameters permitted
   b.2 Reading and writing of BLOB files for plugins running
       in a real-time host.
   b.3 Automation or user/GUI triggered loading of BLOBs.
   b.4 BLOB data should be platform independent, as far as
       possible.
   c. Parameter names in Unicode
   c.2 Symbolic parameter names in US ASCII.
   c.3 Internationalization performed by gettext() style
       system and separate translation files, either host
       or plugin side.
   d. Helpers for display formatting and data entry validation
   e. Create categories to substitute "like with like" parameters
   f. String parameters, hinted as file names, paths, scripts etc.
   g. Transmission of large string blocks, using fixed size
      memory blocks, managed by the host.
   h. Natural param vs normalized, conversion functions
      between both.
   i. value distribution curve (none or enum and/or
      custom function)
   j. min, max, default values
   k. unit type (enum or custom string)
   l. all one type or support for multiple types
      (intgeger, decimal, boolean; union, struct)
   m. read/write/visible/invisible attributes
   n. randomizable
   o. hard ranges or soft ranges


7.  Persistence
   a. Persisted data = a snapshot of all parameter values
   b. BLOB parameters (eg sample sets) require asset management
   c. handle multiple state sets internally in plugin or
      handled by host
   d. presets are static or volatile
   e. settings storage format (single or multiple states,
      endian, defined/opaque/mix)

8.  Threading
   a. Thread management is up to the host
   b. Plugin API should have no O/S dependent synchronization
      primitives
   c. A thread identifier passed by the host in selector
      calls, and required for callbacks.
   d. Real time guarantees or only soft real time?
   e. Worker Calls? (Send off a function to run where the
      host finds appropriate. Host posts return value as
      event.)
   f. host should provide simple threading support - plugin
      can run background tasks, possibly with a priority (1-10?)
   g. Well specified threading for each API functions

9.  GUI
   a. We've got enough work just to create a headless API
   b. Once parameter interface is nailed down, another
      group can do GUI
   c. thread/process (non-)specification for GUI/DSP
   d. Issues related to complex plugins (samplers) if GUI
      is separated from host

10.  The role of MIDI
   a. MIDI should be an event type
   b. SysEx challenge: events with buffers
   c. Support multitimbral software synths
   d. we should leave MIDI out except as to note how to
      make it compatible.

11.  Packaging issues
   a. Base: Static object libraries
   b. O/S specific: dynamic libraries (sample code wrappers)
   c. As new platforms emerge, how to specify the packaging?
   d. How to support all targeted platforms? Or how to
      reduce the amount of support necessary?

12.  Enumeration (Creation, Discovery, etc.)
   a. define and provide enumeration OS independent API.
   b. Simpler is better
   c. plugin discovery (standard location, registration)
   d. different levels of creation (just enough for host to
      inquire about properties plus another level to prepare
      resources for audio processing  [for more efficient
      scanning, and other issues], or only one simply level
      of creation)
   e. defined rules for plugin behavior at each creation
      stage. For instance, no user interface allowed during
      capability queries.
   f. Dealing with operations that are not RT safe.
   g. Factory stuff

13.  Copy protection
   a. Leave it to the plugin, but provide guidelines
   b. Plugins need to be aware of DRM in the streams they
      process
   b.1. DRM aware plugins should not convert a stream to
        DRM just to be safe.
   b.2. DRM aware plugins on a non DRM capable host, should
        allow processing of non DRM streams.
   d. How to sign OpenSource plugins?

14. Localization
   a. A well defined method for the host to specify the
      current language.
   b. Separation of logical and "human readable" names for
      controls, ports and the like. (Related to 6c.)
   c. Up to the plug-in

15. DSP state:
   a. when to reset
      1. and what to reset (different types or resetting?)
   b. handle multiple stream formats or not
      2. single plugin can handle multiple i/o configs
      3. single plugin instance can handle multiple sample
         rates
      4. how does the plugin react to any changes in stream
         format (destroy and reinstantiate, DSP state reset,
         higher level DSP resources constructor/destructor
         [see Creation b. above])

17. License for headers and host libraries/source?
   a. LGPL

______________________________________________________________________

Appendix B.  "North Star" goals

- Look for a model that works well in all the runtime environments we 
care about (list of which is TBD)

- Take into account existing plug-in models, but don't be afraid to 
look farther too

- Look for new niceties to help address important shortcomings in 
existing plug formats (list of which is TBD)
         > User-level functionality improvements
         > Developer-level functionality improvements
         > Developer-level efficiency improvements incl. helping to 
streamline the DSP/processing thread

- As far as possible, don't let any of the new niceties gum up the 
DSP/processing thread

- Where appropriate and possible, leave room for future extensibility

- As far as possible, streamline the plug development process, e.g.:
         > Support universal and very common plug operations with a 
supplied (eventually highly optimized) library
         > Regularize interface and calling conventions throughout the 
API as far as possible
         > Regularize calling conventions across hosts as far as possible
         > Encapsulate platform differences well
         > Leverage existing technologies where useful and appropriate
         > Biz issue: Eschew IP-encumbered technology, e.g. 
proprietary & GPL (...I can see the hate mail coming already...)

- As far as possible, streamline the host development process, e.g.:
         > Specify the host side of the plug-in interface well. 
(Maybe document its obligations and options ala 'design by contract'?)
         > Provide good guidance for host implementors, including at a 
minimum example stub calls to the plug-ins
         > Ideally, deliver source for an example host framework

----------------------------------------------------------------------
Generalized Music Plugin Interface (GMPI) public discussion list
Participation in this list is contingent upon your abiding by the
following rules:  Please stay on topic.  You are responsible for your own
words.  Please respect your fellow subscribers.  Please do not
redistribute anyone else's words without their permission.

Archive: //www.freelists.org/archives/gmpi
Email gmpi-request@xxxxxxxxxxxxx w/ subject "unsubscribe" to unsubscribe


--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .



----------------------------------------------------------------------
Generalized Music Plugin Interface (GMPI) public discussion list
Participation in this list is contingent upon your abiding by the
following rules:  Please stay on topic.  You are responsible for your own
words.  Please respect your fellow subscribers.  Please do not
redistribute anyone else's words without their permission.

Archive: //www.freelists.org/archives/gmpi
Email gmpi-request@xxxxxxxxxxxxx w/ subject "unsubscribe" to unsubscribe

Other related posts: