[gmpi] Re: Reqs 3.9. Time - opening arguments.1

  • From: Tim Hockin <thockin@xxxxxxxxxx>
  • To: gmpi@xxxxxxxxxxxxx
  • Date: Fri, 6 Feb 2004 19:19:43 -0800

Whew, long one.

For the record, I'm trying to make comprehensive notes on ALL the
discussions and sort them into threads for further discussion.  I hope to
have that done by weekend.  I accidentally deleted a bunch of mails before I
had done proper attribution of my notes, so I'll either have to scour the
archive or more likely leave some quotes unattributed.


On Fri, Feb 06, 2004 at 08:35:32PM -0500, Michael Stauffer wrote:
> Regarding GMPI and tempo - like David pointed out earlier, I think it's
> important to distinguish between realtime and offline uses and needs of
> tempo information. To avoid confusion, I'll try to use these terms (or
> something better if someone suggests):

And I don't really see the tempos as different, I see the
application/pluugin as different..read on, and I will try to make myself
clearer.

> "offline tempo map"
> Tempo information stored about a whole song/sequence, or section of a
> sequence, like the traditional tempo map concept we're used to in
> sequencers and midi files. The host itself (assuming it has sequencing
> capaiblities and isn't just a dedicated GMPI host) will have (at least)
> one offline tempo map for its current project/song/sequence.

In a realtime environment this does not exist, right?  At any point, the
user may change the tempo.  And now, if we are to allow tempo-changing from
plugins, a plugin may change the tempo at any point.

To build up a comprehensive tempo map that encompasses an entire project,
you need to know:

1. initial tempo
2. all sequenced/recorded tempo changes
3. all realtime user interactions
4. all plugin-originated tempo changes

Obviously you can't know #3 in a realtime setting, the user can do anything.
In an offline setting, #3 doesn't apply.

Obviously you can't know #4 in a realtime setting, as a plugin can respond
to #3.  In an offline setting, you can FIND OUT #4 (provided all
plugin-origintaed tempo changes are deterministic and not random).  This
involves a full pass of the rendering which records all plugin-origintaed
tempo changes.  An offline map can be built and a second pass of the renderer
can apply that map.

OR look at it more simply:

Any plugin which changes the tempo falls into one of four classes:

1. realtime responses to realtime input
2. needs the full input before generating responses (offline)
3. needs significant input before generating responses (latency)
4. not based on input

If your plugin falls into class #1 or #4, it works equally well in realtime
or offline.  If your plugin falls into #3, it CAN be run "realtime" with
latency compensation.  If your plugin falls into #2, you need extra help.

Are we talking about handling #2 specially?  If not, I posit that there is
no such thing as an offline tempo map.  Tempo is a function of all inputs,
including other plugins.

> "stream tempo" & "stream tempo map"
> For the tempo changes/events during a performance, ie while the transport
> and/or stream time is running. What form this would take hasn't been
> decided (and maybe needn't be for the requirement stage?). This might be
> in the form of a list of tempos passed via vector to the plugins, either
> all at once, or in pieces at the begin of each block of samples as
> suggested previously. It might be individual tempo-related events delived
> JIT, ala midi clocks. Or it might be direct callbacks to the host for
> direct processing as was also suggested previously. Other forms?

Most of those forms can work.  I don't know that we should decide that now,
though the relative merits may be worth exploring.

all at once: unless you know a priori what any other tempo-controlling
plugin will do to the tempo, this is not an option

in pieces: we've already agreed that you know everything that will happen in
a block by the time the block starts, so this works

events: this is isomorphic to "in pieces"

callbacks: this can work, too

> offline tempo maps
> ==================
> I'm not sure why it would be prohibitively difficult for a host to
> implement support for these from a plug. In any(?) host that will show me
> the tempo map, I can make changes to the events in the map, insert new
> tempos, or I can draw a new tempo map, and the host takes care of making
> sense of the changes with respect to the song/project. These actions seem
> generally to be undo-able.

So you're talking about dynamically changing the global tempo map?  I don't
see that as any different from a synamic tempo map.

> Plug-ins might have their own offline tempo-maps, but I would suggest
> that there be a clear hierarchy, along the lines of the
> "upstream/downstream" selection for tempo control made earlier. If a plug
> wanted its offline tempo map to be dominant for all plugs and possibly
> multiple plug graphs, it would submit its offline tempo map to the host.
> Then it would get passed to any downstream plugs without their knowing
> necessarily that it was set by another plug, and they could still
> optionally ignore it or modify it for streaming or offline needs.

This is how I have been coming to think of the issue:

assumption: Tempo changes are delivered JIT to plugins, or the plugin
queries for them.

assumption: Any single plugin is driven by a single music-time master.
There may exist multiple music-time masters, but each plugin has ONE ONLY.

If your plugin wants to be a music-time master (control tempo, meter, etc)
then it must tell the host that fact.  The host is responsible assigning
music-time masters to plugins (possibly by user input).

For example, a host might show a dropdown list for each plugin that has a
TEMPO input.  That dropdown says "Music-Time master:" and has options for
"Builtin (default)" (the hosts's tempo controller) and "Foo TimeTracker"
(some tempo controlling plugin).

If you select the "Builtin", then your plugin will get it's tempo events
from the builtin tempo controller.  If you select "Foo TimeTracker", then
your plugin will get it's tempo events from the tempo controller in the
TimeTracker plugin.

> What I'm envisioning for my own plug-in would be to get audio data from a
> track or track selection in the host, process it for tempo, and pass back
> the tempos (and time-signature), with timestamps that allow the host to
> apply them at the correct point in the song/sequence. One of the

So your plugin requires the full input track BEFORE it generates it's
output?  That sounds to me like an ofline plugin that requires special
assistance.  We've talked about offline plugins that require multiple
passes.  We have not gone into detail on that yet.  Maybe this requires a
pre-process scanning pass?

It would be much more interesting to do real time analysis of an input
stream and adjust the tempo of that, perhaps with small latency.

> I see a complication though if multiple plug-ins want to be sync master
> to the host.  There would have to some explicit selection of multiple
> possible masters, or implicit rules for who's in charge. I imagine a plug

The rule is "let the user decide" or "let the host decide".  The
registration scheme above presents the available choices squarely with the
host/user.

Frankly, registration may be as simple as "I have a TEMPO output" or an
actual registration function.  I've been rethinking some of the ideas behind
output controls, lately.

Tim

----------------------------------------------------------------------
Generalized Music Plugin Interface (GMPI) public discussion list
Participation in this list is contingent upon your abiding by the
following rules:  Please stay on topic.  You are responsible for your own
words.  Please respect your fellow subscribers.  Please do not
redistribute anyone else's words without their permission.

Archive: //www.freelists.org/archives/gmpi
Email gmpi-request@xxxxxxxxxxxxx w/ subject "unsubscribe" to unsubscribe

Other related posts: