Dynamic parameters

Dynamic parameters (dparams sucessor) for future gstreamer version.

The general objective of the dynamic parameter subsystem is to allow gstreamer elements to expose properties to dynamic control. That is, the output of the element can be altered by control signals. Just imagine the controls for volume on a hifi.

In GStreamer-0.10 GstController has replacxe the GstDParam API.

Comparison of approaches

The general idea is that elements expose the control parameters in some form and that there are objects that generate control signals that comply to this spec.

	dparams	control pads	GstEnvelope/Controller
registration	a GParamSpec + an update method registred under a name with the DParamManager, update method can be mem-write or callback	controls pads with the specs in the form of Caps	not registered
controller	DParam objects that receive control value via property and push that into the element via update method	GstElements that receive control value via property and generate an outputbuffer with control data that is linked to the control pad of the element that is controlled	GstEnvelope/GstController object that receives timestamped control values
pro	light-weight update method	concepts is in line with how gstreamer works precise timing, as controlbuffers are timestamped and scheduled one could write elements that modify control data number pads can differ for each instance (needed for multi-voice)	only few changes to elements preceise timing possible (allows for lower control rate)
contra	concepts is orthogonal to how gstreamer works	heavy load on the scheduler all controllable elements need to become loop-based	how to detect which gobject param may be controlled number of gobject properties is class wide (how to implement multiple voices)

Scenarios

1. Can we have a control rate lower than audio/video rate? (e.g. control changes are only picked up every 100 sample values) ?
   • Elements would need to read from each control input at its given control rate
2. How can one control an element via GUI while sequencer loops a part?
   • GUI would need to pass control value changes to the same GstController the sequencer uses
   • GUI can listen to notify::value of the controller to updated widgets when control value changes

Design

Trigger

Current trigger implementation has the problme, that the gst_object_sync() call needs to be exactly at the trigger time. As elements usually only call this once per buffer, applications need to tweak buffer sizes that make sure the beginn of a buffer hits the trigger.

API

To fix this problem, we could also introduce some new API and use that in base-classes (GstBaseSrc and GstBaseTransform).

/*
 * gets next ts that is in controller queues after @cur_ts
 */
next_ts=gst_controller_suggest_next_sync(cur_ts);

And this could be the loop in the baseclass:

processed=FALSE;
if(gst_object_get_controller(self)!=NULL) {
  next_ts=gst_controller_suggest_next_sync(cur_ts);
  sub_len=next_ts-cur_ts;
  while(GST_CLOCK_TIME_IS_VALID(next_ts) && (sub_len < len) {
    // create subbuffer from cur_ts ... cur_ts+sub_len
    process(sub_buffer);
    cur_ts+=sub_len;
    next_ts=gst_controller_suggest_next_sync(cur_ts);
    sub_len=next_ts-cur_ts;
  }
  if(cur_ts>ts) {
    // create remaining subbuffer from cur_ts ... len
    process(sub_buffer);
    processed=TRUE;
  }
}
if(!processed) {
  process(buffer);
}

Interpolation

Similar to the trigger problem is the case of parameter interpolation (linear/curved) and big buffer sizes. Maybe the API sugested in the previous section could help.

API

If we could tell the GstController API what is the update interval for controlled parameters, we could remember the last gst_object_sync_values() timestamp, we could then also take that into account when calculating the value for gst_controller_suggest_next_sync().

Modulator

In the current design we have interpolation methods. These generate inbetween values:

• none: returns last value
• linear: smoothes between last and next
• ...

They all have no parameters except the timestamp they should produce a value for. The simplicity makes it easy to handle. Apps just set key points and can later read in between values.

Another idea is to allow things like a low frequency sine wave oscillator changing the property. This would be modulation methods This would work, but the TimedValue concept would not make sense here.

1. a sine generator has parameters (frequence, phase)
2. a sine does not follow through key point values
3. an interpolator interpolates from last value to next value, while the modulator cycles between min/max or middle +/- mod-depth

Should the params of the sine (like frq) be modulated too? Can this cause cycles?

API

We could introduce a new API:

gst_controller_set_control_mode (obj, "target-param", GST_MODE_INTERPOLATION, GST_INTERPOLATION_LINEAR);
gst_controller_set_control_mode (obj, "target-param", GST_MODE_MODULATION, GST_MODULATION_SINE);

gst_controller_set_interpolation_mode(obj,param,mode) {
  gst_controller_set_control_mode(obj,param,GST_MODE_INTERPOLATION,mode);
}

Next we need to supply the modulation params. We can either set fix values or bind to gobject params:

gst_controller_set_modulation_param (obj, "target-param", GST_MODULATION_SINE_FRQ , 10);
gst_controller_bind_param (obj, "target-param", GST_MODULATION_SINE_FRQ , "source-param");

Problems

Is a bit ugly that the gst_controller_set() API only works in interpolation mode and other methods only work in modulation mode. Maybe expose the GstControlledParam objects as

GstInterpolatedParam
GstModulatedParam

and move methods.

ControlSources

A better design for the addition of Modulators would to define control-sources as GObjects that generate a control-signal. The control-sources could have any API to control their behaviour, but would implement a GstControlSourcesIFace:

GetControlValue get_control_value;
GetControlValueArray get_control_value_array;

For numerical values (int,uint,long,ulong,float,double,...) it would make sense that the control-source only delivers a control-signal ranging from 0.0 .... 1.0 this will be mapped to the GObject property range (FIXME: who is doing that? The element or the GstController).

Benefits

This would even allow to have e.g. a GstCombinedControlSource which gets multiple control-sources as inputs, multiplies all their control_values for one timestamp and therefore returns all control-shapes overlaid on each other.

Control-sources can be reused (applied to multiple parameters).

New control-source types can be defined and even added localy.

Legacy

We need to deprecate:

GstInterpolateMode
gst_controller_set_interpolation_mode()

This would instantiate a GstVectorControlSource and call gst_controller_set_control_source().

We need to move these to the GstVectorControlSource:

gboolean gst_controller_set (GstController * self, gchar * property_name, GstClockTime timestamp, GValue * value);
gboolean gst_controller_set_from_list (GstController * self, gchar * property_name, GSList * timedvalues);
gboolean gst_controller_unset (GstController * self, gchar * property_name, GstClockTime timestamp);
gboolean gst_controller_unset_all (GstController * self, gchar * property_name);
const GList *gst_controller_get_all (GstController * self, gchar * property_name);

The original methods would be deprecated and the implementation would forward the calls to the GstVectorControlSource implementation if such is used.

All the legacy code should be moved to a gstcontrollerlegacy.{c,h} module.