[gmpi] Re: Topic 6: Time representation

  • From: Chris Grigg <gmpi-public@xxxxxxxxxxxxxx>
  • To: gmpi@xxxxxxxxxxxxx
  • Date: Thu, 1 May 2003 18:46:24 -0700

David wrote:

Chris Grigg wrote:
[...]
 >>  Well, that's kind of what a sequencer track is anyway, yeah?
 >
 >Yeah, but the event/control output of a plugin is not a sequencer
 >track, is it? (Well, it *could* be, but then we're not talking
 > about a real time system.)

 Interesting point.  I don't know where the idea that it's only OK
 for GMPI to be a real-time system came from.  We have input and
 output queues of timed events, after all.

If it's realistically possible to do, I don't think anyone would mind if GMPI does both, but it makes quite a difference if you use priority queues with random access operations, or plain LIFO queues. The latter are basically a structured alternative to audio rate control streams, while the former have lots of implications.

How event queuing would work is an interesting topic. Though OT for now.


(Personally I kind of like the idea that each plug instance gets one ABSTIME-stamped event queue with each timeslice, sorted in time order, with all event types mixed together. I'm not against the host pre-filtering so that each plug only sees the events intended for it, esp. as this fits with the graph-connections-as-patchcords idea, and esp. when you consider the peak density of all the GUI events, automation events, and MUSTIME events that'll be flying around the graph. Avoids a .lot. of redundant event-rejection.)(Event targets are sounding like a good idea.)


[...]
 >>  >  A side effect of that is that plugins must
 >>  >(obviously) send output events through the host, rather than
 >>  > directly to receivers.
 >>
 >>  I'd assumed it was pretty much going to work that way anyway.
 >>  Hosts would manage event stream routing.  Was there a different
 >>  proposal?
 >
 >Well, it's probably easier to leave it to the host, but it's not
 > the only way to do it. For XAP, we designed a system where every
 > control output of a plugin has a "target specification", in the
 > form of an event queue pointer and a cookie. These are created by
 > the target plugins upon connection of controls, which means that
 > a plugin gets to decide where the events for each input go, as
 > well as how to generate the cookies. (Object LUT index,
 > multidimensional indices or whatever - pick what does the job
 > fast and easy.)

 Cool.  But the host still has to stitch the pins together and
 manage the queues, yeh?

The host has to figure out what to connect where - or rather, let the user decide, one way or another. Next, the host asks the plugin that owns the input for a target spec. Finally, the host tells the source plugin to connect the desired output, passing that target spec as an argument.

Gotcha. So you're considering the message routing mechanism to be something other than the host. Whereas I had been thinking of it as one function of the host. Either way, no real difference to the plug.



The queues, however, are very simple linked lists, and need no real
management.

I like that, per above. You can find your insert position by traversing and watching the ABSTIMEs.



Events are allocated from, and returned to, a host
managed pool.

Really like that. Let's avoid the need for dynamic allocation during a process() call as much as possible, eh?



Each event queue operation is done with a few
instructions of inline code. The only time a plugin calls the host is
if/when the event pool is exhausted.

Again, don't see any significant downside to the plug calling the host -- or a support lib call -- for list mgmt. Just seems cleaner to provide the shared functionality in one place, rather than make every single plug carry that stuff. Also if it's a host call, the implementation can be hidden from the plug, giving the host writer more freedom.



The only time a XAP host would touch events on the way between plugins
is when events from two or more contexts (*) have to be
sorted/merged, to guarantee that events arrive in timestamp order.

(*) Each plugin is at least one context. Each inner loop that
    sends or receives events is one context. Every control has
    a plugin local context id, so hosts know which controls can
    safely be routed directly to the same target without, and
    which need sorting.

Gotcha. Neat.



[...]
 >Why this obsession with time slices? (We all do agree that we
 > should use timestamped events, right?) You can have one curve
 > segment per sample if you like. It doesn't have to be restricted
 > to block boundaries in any way.

 Just thinking about making it easy for novice plug developers...
 since ::process() by definition is about one timeslice at a time,
 and in a typical timeslice you'll be ramping parameters all the way
 across the timeslice, not ending in the middle or doing complex
 shapes.  So thinking about making that a default behavior.

I don't see why this makes anything easier - unless the idea is that your average plugin author shouldn't care about implementing sample accurate timing. Then what's the point in using timestamped events at all...?

As long as you're dealing with DSP algorithms that process one sample
frame at a time in the inner loop, it's very easy to implement sample
accurate event processing. Adding linear control ramping to that is
trivial.
[...]
Considering that *anything* that isn't audio rate control data is just
an approximation that may still need further anti-zipper filtering,
why bother? What problem is it intended to solve, really?

I think we should be thinking about the multiple-timeslice picture. For intra-timeslice interpolation, the host has to take high-level, long-duration parameter changes from the sequencer and break them into timeslices. A typical param change move -- probably encoded in the seq track as a ramp in most cases, sometimes with a curve shape -- has to be -- or can be -- broken down into a series of instantaneous values on the timeslice boundaries. At the plug-in level, in each given timeslice, the plug needs to interp between the param's value at the start of the timeslice and its value at the of the timeslice.


So the idea is to make parameter interpolation across the timeslice as easy as possible for the plug developer, by providing at each timeslice, for each parameter, an initial value, a terminal value, and a curve type; and a function for getting the interpolated value at any sample index in the timeslice.

This would be an available alternative to using timestamped events for parameter changes; it would give the host a way to translate seq track high-level long-duration param changes directly into something the plug can easily work with, without the work of reading event queues. Provided that's how the plug and the host want to handle that parameter. Plug developers who want to work only with events and not use this mechanism would be free to do that.


Adding quadratic or cubic curves isn't much harder, and it can be done
in a way that allows the same receiver code to accept non-ramped
input as well as linear, quadratic and cubic "ramping". We're still
talking about code that's simpler and faster than any serious zipper
noise elimination filter - which is something you'll still need in
many cases, even with cubic curves.

However, as soon as you start specifying multiple choices on the API
level (such as "a plugin MAY support cubic curve input"), it's no
longer possible to implement it this way. You have to call the host,
or some helper library to get the control data translated into
something you can use. In that case, I can only think of two
alternatives:

1) Plugins ask for buffers of audio rate control data.

        2) Plugins ask for control values for specific points
           in time.

The first one seems rather pointless and inneffective to me. Why not
just use audio rate control streams throughout, except possibly for
controls that cannot be ramped at all?

Alternative 2) seems even more pointless. If you're only going to ask
for a value every N samples or so, why not just send linear ramp
events with sufficient density?
[...]
IMHO, support for audio rate control data, as an optional alternative
to events with linear ramps, would be simpler, more useful and more
efficient than supporting non-linear ramps through helper functions.

I don't see what's pointless about that at all, especially in light of the following drawbacks to audio-rate param buffers. Such as: It would offload the calculations to the host, it would consume buffer space, the buffers would have to be managed, and it would cause the host 'terp function to be executed once per every single sample in the timeslice (whereas if accessed from process() via a call, the plug would have the option to just sample the curve every so often and do its own simpler [probably linear] interp between the samples). OTOH, having pre-digested data values would in many cases simplify the per-sample code in process()... although as you say below, in many cases like filters the interp'd param value would still have to be translated into low-level params, i.e. coefficients, on a per-sample basis.


BTW, if it helps, consider another alternative: parameter value events, but the parameter is 'slew rate limited' so it smooths to the new value over a defined time period.


[...]
 >Well, there is one problem: What to do if a plugin doesn't support
 > the shapes that some other plugin, or the host, generates? Do all
 > plugins with control outputs have to support multiple output
 > formats, or is it up to the host to insert some form of converter
 > elements as needed?

 The enums would be defined in the headers for a particular GMPI
 version, so if the host & plugs are the same GMPI version, there's
 no mismatch on the enum values.  I thought that interpolation for a
 given shape could be provided as a host function, so in process()
 you could call something like myVal = gmpiHost::getInterpValue(
 startVal, endVal, samplesInTimeslice, curve, sampIndex).  Something
 like this will be eventually be needed if we ever want to support
 nonlinear curves.

Well, I'm just not convinced that there is any reason to go beyond linear ramps with arbitrary density, without going directly for audio rate control data. What's the gain?

See list above.



Most effects can't ramp the *actual* controls, for performance reason.
(Expensive filter coefficient recalculations and whatnot.) They ramp
the coefficients intstead, in a way that resembles the intended curve
shape. Sounds like going from linear to cubic might be a good idea,
because the calculations - slightly more complicated - still only
have to be done twice per curve section.

Now, for these improved approximations to be possible, the plugins
have to *understand* the curve shape in order to approximate it,
right? So, just asking the host to calculate a few values won't give
you a better approximation than linear ramp events with slightly
higher density. The host can render the *control curve* for you, but
it can't give you the coefficients you need for your internal
approximated curves.

Yeah, of course. It's just better accuracy at similar efficiency, and with better flexibility, if you allow the plug to make its own decisions about when to take samples of the 'real' curve.



> Don't understand what you mean about output
formats, ask again?

Plugins have to be able to *generate* control data as well receive it. This shouldn't be a problem, as you just need to make sure that no plugin generates something that the host can make sense of.

I -know- you mean '...that the host -can't- make sense of...' and I agree, and that can't happen because the curves are predefined in an enum in gmpi.h for each released version of gmpi, and the enum only grows over time. Slowly, I would think.



However, considering the above, I'm not sure there's much point in
generating anything that cannot be interpreted directly by the
receiving plugin.

That gets into automation design questions, kind of OT, but it seems like a plug that generates automation should be using a high-level, long-term parameter change description, not a per-timeslice description. Even though plugs respond to low-level, intra-timeslice param changes. Otherwise plugs would be second-class automation sources.



The big deal with timestamped events; the very reason why anyone would
think of using them for controlling DSP code, is that they're cheaper
than audio rate controls. Doing something that actually makes the
shortcut more expensive than the real thing, seems... well, you get
the idea. :-)

Leaving the sampling to the plug developer is more flexible, simpler to program, and less compute-expensive.



[...]
 >Well, our idea was to send timestamped control events to these
 > "tempo" (for tempo changes or ramps) and "position" (for loops
 > and jumps) controls. Plugins will have to receive these events,
 > and apply tempo to position. It's trivial to do on a
 > sample-by-sample basis (something like tempo += dtempo; position
 > += tempo;)...

If the curve is anything but linear, this will give a wrong answer.

Yes, but then you would be feeding the plugin with garbage. If tempo is a control that supports linear ramping, that's all it is. You may send a new tempo event for each sample frame, if piecewise linear approximation isn't sufficient.

Meanwhile, in the context of sequencers and tempo, it seems that some
would even consider actual linear ramping of the tempo overkill...


 >...and easy enough
 >if you just want to do the calculations once per block and when
 > you actually receive the events. Then Plugin SDK could provide
 > some inlines or macros for the latter, along with something
 > similar to the call you suggest.

 I see.  But unless being able to set each plug's tempo & position
 independently is a design requirement -- something I hadn't thought
 of -- why make tempo & position 'controls' of the plug?

Well, for XAP, it just happened to be the most natural way. It turned out to be dead simple, and it cuts down the number of specialized interfaces. "Everything is a control" - and it's not because we think it sounds cool. It's because we hate APIs that you can't use without having the reference docs handy, even after using it for years.

Right, the clean aspect is nice. Does the host have controls too?



> I've been
 thinking of tempo as a sort of global parameter, a property of
 managed by the host, that you go to the host to get, and that looks
 the same to all plugs in the graph.

Though maybe tempo-as-parameter isn't so well-liked.

Either way, I don't really like the idea of considering tempo as something special enough to be host global.

What parameters would you see as special enough to be host-global? Wouldn't it be cool to be able to have a VSO plug-in that can wobble the tempo, in any application?



I'm not remotely into
experimental music, but I've still found myself working against the
tempo map at times. Some things would be so much easier if you could
just throw in another tempo map that you can adjust as needed,
instead of moving events around.

Can you elaborate?



As an example of a more desperate need for multiple timelines,
consider a multimedia audio engine, like the ones used in games....

Yeah, I'm very much from that world, and very much want GMPI to work there.



...It
has to be able to play multiple sequences independently. If it can't,
you can't crossfade between background songs, you can't play fanfares
over the music, and you can't use sequences for soundscapes and sound
effects. Such limitations would strike me as ridiculous.

So where's the rhythmically dependent plug-in in those cases? Your examples only have independent players being mixed together. No problem. Nothing in GMPI requires all player plug-ins to receive their sync from the host, does it? A MIDI file player could be a plug-in even without receiving MUSTIME from the host. They run on their own internal timebases. Or, even if you did need a rhythmically dependent plug for each player, couldn't you set up two GMPI frames each with a playback seq driving whatever rhythmically dependent plugs it needs, then mix the two audio outputs?


-- Chris

----------------------------------------------------------------------
Generalized Music Plugin Interface (GMPI) public discussion list
Participation in this list is contingent upon your abiding by the
following rules:  Please stay on topic.  You are responsible for your own
words.  Please respect your fellow subscribers.  Please do not
redistribute anyone else's words without their permission.

Archive: //www.freelists.org/archives/gmpi
Email gmpi-request@xxxxxxxxxxxxx w/ subject "unsubscribe" to unsubscribe

Other related posts: