Contents
Introduction
Until recently, adding a new audio codec to OpenH323 required modifying and recompiling the OpenH323 source code. This requires a complete copy of the OpenH323 source code and a compatible development environment, as well as a detailed knowledge of C++ and H.323 capabilities.
As of version 1.14, OpenH323 can load audio codecs at run-time, either from a DLL (for Windows) or from a library.so file (on Unix platforms). This means that OpenH323 codecs are easier to write and change than with the previous monolithic design, and can be written and developed without knowledge of C++ or requiring a complete OpenH323 development environment. It also allows the distribution of binary-only plugins, as well as removing the need to deal with the complex details of H.323 capabilities.
This document provides an overview of the plugin system, and describes how to use and install plugins. It also provides a description of the programming model to allow development of new plugins.
Using plugins
Installing a codec plugin consists of copying the DLL or library.so file into a directory where it can be found by OpenH323. By default, the stack will search "C:/PWLIB_PLUGINS" (on Windows) or "/usr/local/lib/pwlib" (on Unix) and all sub-directories. The default directory will be different if the --prefix option is passed to configure.
At run-time, the default plugin directory can be changed by setting the value of PWLIBPLUGINDIR environment variable. If set, this variable is assumed to contain a list of directories, seperated by ":" (colon) on Unix, and ";" (semi-colon) on Windows.
All plugin codecs must have a name that ends in "_pwplugin.dll" (Windows) or "_pwplugin.so" (for Unix). This allows the plugin search path directory to be safely pointed at a directory like "/usr/lib" or "C:/WINDOWS/SYSTEM32" without PWLib attempting to load every single shared library while looking for codecs.
Media formats
A media format is a string that names an encoded format, such as "G.711-uLaw-64k" or "GSM-06.10". OpenH323 predefines many media format names, but only media formats that have corresponding functions to encode and decode data are available as codecs.
The plugin codec system automatically creates new codecs (including any needed media formats) when plugins are loaded, and these will be available in addition to any statically linked codecs. By default, the only statically linked codecs are G.711 uLaw and ALaw.
The static function H323PluginCodecManager::GetMediaFormats return a list of all of the codecs that are supported by the current set of plugins and statically linked code without requiring any additional hardware. By default, this will include all of the audio codecs that convert can be converted to PCM-16.
As of 13 July, H323PluginCodecManager::GetMediaFormats will also return any video media formats that are supported by the stack. This will always include H.261 (if video is compiled into the stack) and H.263 (if support has been compiled and all necessary run-time loadable libraries are available).
The function OpalLineInterfaceDevice::GetMediaFormats can be used to obtain a list of the codecs supported by an OpalLineInterfaceDevice or descendant.
Capabilities
A H.323 endpoint advertises the codecs it supports by providing a table of capabilities. Each capability specifies a media format, plus other codec-specific parameters such as frames per packet. In OpenH323, a capability is represented by the H323Capability class, and instances of the H323Capability class can are collected into a list represented by a class named H323Capabilities.
An endpoint uses a H323Capabilities to perform the capability negotiation during call setup, and an application can manipulate endpoint's H323CapabilityList using various member functions on the H323Endpoint.
There are three ways to create instances of an H323Capability. Firstly, the capability class can be instantiated directly, as follows:
H323Capability * cap = new H323_GSM0610Capability;
Secondly, the static function H323Capability::Create can be used to create a H.323 capability that explicitly matches the string:
H323Capability * newCap = H323Capability::Create("G.711-uLaw-64k")
Note that the string must explicitly match the string specified. The return value is NULL if there is no match.
Thirdly, all capabilities that match a wildcard string can be added to a H323CapabilityList using the following function:
H323Capabilities caps;
caps.AddAllCapabilities(0, 0, "711");
For historical reasons, the names used by capabilities are the same as the corresponding codec (or media format) name, but may have a suffix of "{sw}" or "{hw}" added to indicate whether it is purely a software or hardware codec. By default, AddAllCapabilities will include any "{sw}" codecs that match the string provided, but will not match any "{hw}" codecs unless that is included in the original codec specification.
A user program must populate an endpoint's capability table with the list of capabilities that match the abilities of the program and endpoint software. This is normally done as follows:
ep.AddAllCapabilities(0, 0, "*");
ep.AddAllUserInputCapabilities(0, P_MAX_INDEX);
At this time (15 June 004) AddAllCapabilities only works for audio codecs. More work will be done on supporting video codecs as work proceeds on supporting video codec plugins.
Converting user programs to use plugins
There are several problems that are likely to be encountered when compiling user programs to use codec plugins:
- Any user programs that include header files corresponding to the old embedded codecs will fail to compile, as these header files have been removed. The solution is to remove the lines that contain the #include statements, and try recompiling.
- A user program that populates a capability table by creating instances of the embedded codecs will fail to compile, as these classes are no longer available. The solution is to replace the calls to
AddCapability(0, 0, new OldCapabilityClass()) with AddAllCapabilities(0, 0, "*"). See below for more information on how to find specific capabilities and add them to the capability table. - Any classes that descend from the old embedded capability or codec classes will fail to compile. The solution is to copy the header and source code from the old codec classes and combine them into the user codec class.
Note that non-plugin codecs can still be linked into the stack in the same way as previously.
Programming model
A codec plugin interoperates with OpenH323 or OPAL by exporting a set of tables that contain all off the data and functions that are needed to access and use the codecs. These tables provide a model of the codec as one or more methods that convert data from one format to another. As a codec normally converts data in both directions, i.e. encoding from PCM-16 to G.723.1 and decoding from G.723.1 back to PCM-16, most codecs will provide two complementary methods to provide conversion of data. When OPAL or OpenH323 loads a codec plugin, it will identify pairs of complementary methods using the sourceFormat and destFormat fields, and combine them to create a codec.
Most codecs have state or instance information that is required for each execution of the conversion method, and must be preserved across subsequent calls to the same conversion function. To allow this, each conversion function has two additional functions that create and destroy a context that is passed to every invocation of a conversion function. There are no restrictions placed on the number of contexts that can be created (unless the code implementation itself has a limitation, perhaps due to license or performance limitations).
All of the structures and definitions required by this plugin interface are contained in the file opalplugin.h which is provided as part of the OpenH323 source package. A copy of this file can also be downloaded from here.
Exported functions
A plugin exports two functions in order to be accessible as a codec plugin. Note that these are exported as "C" functions - if the plugin is written in C++, then these names must be exported with unmangled functin names.
unsigned int PWLibPlugin_GetAPIVersion()
This function is required by all PWLib plugins.
Return value: the PWLib Plugin API version supported by the plugin. The only version currently supported is "0".
PluginCodec_Definition * OpalCodecPlugin_GetCodecs(unsigned * count, unsigned version)
Returns the tables defining the codec interface and parameters
count: The integer value pointed to by this parameter must be set to the number of elements in the table pointed to by the return value.
version: Specifies the version of the codec plugin interface supported by the OpenH323 or OPAL. The versions defined are:
- 1 = original plugin spec (no wideband)
- 2 = supports wideband (i.e. samplerate paramater can be value other than 8000)
Return value: Pointer to an array of structures that describe the codecs implemented by the plugin. The format of the structures is described below. If NULL, no codecs are defined.
PluginCodec_Definition
The OpalCodecPlugin_GetCodecs function returns a pointer an array of structures that provide the primary interface to a codec conversion method. The definition of this structure is as follows:
struct PluginCodec_Definition {
unsigned int version; // codec structure version
struct PluginCodec_information * info; // license information
unsigned int flags; // various codec flags
const char * descr; // text decription
const char * sourceFormat; // source format
const char * destFormat; // destination format
const void * userData; // user data value
unsigned int sampleRate; // samples per second
unsigned int bitsPerSec; // raw bits per second
unsigned int nsPerFrame; // nanoseconds per frame
unsigned int samplesPerFrame; // samples per frame
unsigned int bytesPerFrame; // max bytes per frame
unsigned int recommendedFramesPerPacket; // recommended number of frames per packet
unsigned int maxFramesPerPacket; // maximum number of frames per packet
unsigned char rtpPayload; // IANA RTP payload code (if defined)
const char * sdpFormat; // SDP format string (or NULL)
void * (*createCodec)(const struct PluginCodec_Definition * codec);
void (*destroyCodec) (const struct PluginCodec_Definition * codec, void * context);
int (*codecFunction) (const struct PluginCodec_Definition * codec, void * context,
const void * from, unsigned * fromLen,
void * to, unsigned * toLen,
unsigned int * flag);
struct PluginCodec_ControlDefn * codecControls;
unsigned char h323CapabilityType; // type of H.323 capability
void * h323CapabilityData; // additional H.323 capability information
};
The meaning of each field is as follows:
| Field | Description | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| version | specifies the version of the structure that is being used. The following values are supported:
1 = original version of plugin specification (wideband not supported) The latest version is always defined by the manifest constant PLUGIN_CODEC_VERSION |
||||||||||||||||
| info | pointer to a structure of type PluginCodec_information that specified license and version information. If NULL, default information will be used | ||||||||||||||||
| flags | various bit flags as follows:
|
||||||||||||||||
| descr | pointer to zero-terminated string with text description of codec. If NULL, the codecDescription field will be used | ||||||||||||||||
| sourceFormat | pointer to zero-terminated string defining source data format. Must not be NULL | ||||||||||||||||
| destFormat | pointer to zero-terminated string defining destination data format. Must not be NULL | ||||||||||||||||
| userData | available for the codec implementor to set to any value. | ||||||||||||||||
| sampleRate | sample rate for the codec. If the plugin interface version is 1, then the only value supported is 8000. If the plugin interface version is 2 or greater, then other values may be returned |
||||||||||||||||
| nsPerFrame | nanoseconds per frame | ||||||||||||||||
| samplesPerFrame | number of samples per codec frame | ||||||||||||||||
| bytesPerFrame | number of whole bytes per codec frame | ||||||||||||||||
| recommendedFramesPerPacket | the recommended number of codec frames per RTP packet | ||||||||||||||||
| maxFramesPerPacket | the maximum number of codec frames per RTP packet | ||||||||||||||||
| rtpPayload | specifies the RTP payload type if flags contains PluginCodec_RTPTypeExplicit | ||||||||||||||||
| sdpFormat | pointer to zero-terminated string defining the SDP media format. If NULL, then no SDP format is defined | ||||||||||||||||
| createCodec | pointer to function to create a codec method context. If NULL, no context will be created for the codec. See below for more information | ||||||||||||||||
| destroyCodec | pointer to function to destroy a codec method context. If NULL, this function is not called. See below for more information | ||||||||||||||||
| codecFunction | pointer to codec method function. Cannot be NULL. See below for more information | ||||||||||||||||
| codecControls | pointer to structure containing additional codec control functions. May be NULL for audio codecs, but video codecs are required to provide specific control functions. See below for more information | ||||||||||||||||
| h323CapabilityType | integer value defining what kind of H.323 capability is used for this codec. If zero, no H.323 capabiity is defined. The possible values are described below | ||||||||||||||||
| h323CapabilityData | Additional H.323 capability information. The exact value of this parameter depends on h323CapabilityType, and is described below |
Note 1 - if set, this flag allows the plugin manager to minimise the number of RTP payload codec using the same code for codecs that share the same value for the sdpformat field. This is especially useful in reducing the number of dynamic payload types reqiured for Speex for 19 to one.
Note 2 - if set, this flag indicates that OpenH323 can request the code to create an interpolated frame to replace a missing audio frame if the encode function is called with the PluginCodec_CoderSilenceFrame option set.
Note 3 - some codecs, notably G.726, require the specification of bits per sample
PluginCodec_information
This structure identifies the author of the codec and the plugin interface, as well as indicating what license is applied. It has the following structure:
struct PluginCodec_information {
time_t timestamp; // codec creation time and date
const char * sourceAuthor; // source code author
const char * sourceVersion; // source code version
const char * sourceEmail; // source code email contact information
const char * sourceURL; // source code web site
const char * sourceCopyright; // source code copyright
const char * sourceLicense; // source code license
unsigned char sourceLicenseCode; // source code license
const char * codecDescription; // codec description
const char * codecAuthor; // codec author
const char * codecVersion; // codec version
const char * codecEmail; // codec email contact information
const char * codecURL; // codec web site
const char * codecCopyright; // codec copyright information
const char * codecLicense; // codec license
unsigned short codecLicenseCode; // codec license code
};
| Field | Description | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| timestamp | unsigned integer giving a timestamp for the plugin in seconds since 00:00:00 1970-01-01 UTC. This value can obtained with the Unix command date -u "+%c = %s" | ||||||||||||||
| sourceAuthor | pointer to zero-terminated string naming the author of the plugin interface code. If NULL, no author is specified | ||||||||||||||
| sourceVersion | pointer to zero-terminated string describing the version of the plugin interface. If NULL, no version is specified | ||||||||||||||
| sourceEmail | pointer to zero-terminated string with contact email address for author of the plugin interface. If NULL, no email address is specified | ||||||||||||||
| sourceURL | pointer to zero-terminated string with URL for author of plugin interface. If NULL, no URL is specified | ||||||||||||||
| sourceCopyright | pointer to zero-terminated string containing copyright string for plugin interface code. If NULL, no copyright is specified | ||||||||||||||
| sourceLicense | pointer to zero-terminated string describing the license of the plugin interace code . If NULL, no license is specified | ||||||||||||||
| sourceLicenseCode | integer value specifying license that applies to the plugin interface code. Values are:
|
||||||||||||||
| codecDescription | pointer to zero-terminated string giving the full name of the codec. Cannot be NULL | ||||||||||||||
| codecAuthor | pointer to zero-terminated string naming the author of the codec implementation. If NULL, no author is specified | ||||||||||||||
| codecVersion | pointer to zero-terminated string describing the version of the codec implementation. If NULL, no version is specified | ||||||||||||||
| codecEmail | pointer to zero-terminated string with contact email address for author of the codec implementation. If NULL, no email address is specified | ||||||||||||||
| codecURL | pointer to zero-terminated string with URL for author of the codec implementation. If NULL, no URL is specified | ||||||||||||||
| codecCopyright | pointer to zero-terminated string containing copyright string for the codec implementation. If NULL, no copyright is specified | ||||||||||||||
| codecLicenseCode | integer value specifying license that applies to the codec implementation. See sourceLicenseCode value above |
Codec methods
A codec provides three functions that can be used by OpenH323 to perform the data conversion operations.
- The createCodec function is used to create a context for a particular instance of a codec. It can also be used to provide any initialisation before the codec is started.
- The destroyCodec function is used to destroy the context created by the createCodec function. It can also be used to perform any cleanup after the codec has been stopped.
- The codecFunction is used to perform the actual conversion of data from one format another.
void * createCodec(const struct PluginCodec_Definition * codec);
Allocates and initialises any instance data required by a codec
codec: pointer to the PluginCodec_Definition that defines the codec being created. This allows the same createCodec function to be used for different codecs.
Return value: pointer to the created instance data
void destroyCodec(const struct PluginCodec_Definition * codec, void * context);
Uninitialises and frees any instance data required by a codec
codec: pointer to the PluginCodec_Definition that defines the codec being created. This allows the same destroyCodec function to be used for different codecs.
context: pointer to the context data to be freed
int codecFunction(const struct PluginCodec_Definition * codec,
void * context,
const void * from, unsigned * fromLen,
void * to, unsigned * toLen,
unsigned int * flag
This function converts data from the source format into the destination format. Audio codecs always convert between raw arrays of 16 bit audio samples and raw encoded audio frames.
Video codecs always convert to and from RTP frames. The payload for an encoded frame is specific to each codec. The format for an unencoded video frame is defined as follows:
PluginCodec_Video_FrameHeader {struct
unsigned int x;
unsigned int y;
unsigned int width;
unsigned int height;
unsigned char data[1];
};The data parameter is a placeholder for the video data in YUV 4:2:0 planar format
codec: pointer to the PluginCodec_Definition that defines the codec being created. This allows the same createCodec function to be used for different codecs.
context: pointer to the codec instance data created by createCodec
from: pointer to input data.
fromLen: pointer to an integer containing the length of the input data. Upon exit, this integer must be set to the number of bytes of input data that has been converted
to: pointer to buffer to contain the output data
toLen: pointer to an integer containing the length of the output buffer data. Upon exit, this integer must be set to the number of bytes of output data that have been created.
flag: pointer to an integer bit mask that indicates the kind of conversion to be performed, and on exit, provides additional information on the conversion. Valid values on entry are
Manifest constant Value Description PluginCodec_CoderSilenceFrame 0x0001 create a silence frame (audio) PluginCodec_CoderForceIFrame 0x0002 force creation of an I-frame (video) The following values can be returned:
Manifest constant Value Description PluginCodec_ReturnCoderLastFrame 0x0001 indicates this output frame is the last for a frame PluginCodec_ReturnCoderIFrame 0x0002 indicates this frame is an I-frame PluginCodec_ReturnCoderRequestIFrame 0x0004 indicates that the video decoder is requesting the transmission of an I-frame from the remote end, probably because it has lost sychronisation with the remote end Return value: If 1, the data was converted successfully. If 0, then some error occurred during the conversion process. Note that a return value of zero will cause the media stream to be closed - this value should only be returned if an unrecoverable error has occurred
h323CapabilityType
The h323CapabilityType field indicates what kind of H.323 capability is required to use the codec in H.323 sessions. The use of this field is deprecated for audio codecs and cannot be used for video codecs (see the "get_codec_options" control for more information)
Depending on which capability is selected, the h323CapabilityData field may be used to specify additional information. See the opalplugin.h file for more information about these additional structures, or below for more information on non-standard capabilities
| Value | Manifest constant | h323CapabilityDataField | Description |
|---|---|---|---|
| 0 | PluginCodec_H323Codec_undefined | Not used (set to NULL) | No H.323 capability defined. Can be used when the codec cannot be used in H.323 sessions |
| 1 | PluginCodec_H323Codec_programmed | Not used (set to NULL) | Not yet implemented |
| 2 | PluginCodec_H323Codec_nonStandard | Pointer to structure of type PluginCodec_H323NonStandardCodecData | Defines a H.323 non-standard capability. See more information below |
| 3 | PluginCodec_H323Codec_generic | Not used (set to NULL) | Not yet implemented |
| 4 | PluginCodec_H323AudioCodec_g711Alaw_64k | Not used (set to NULL) | G.711 Alaw at 64k |
| 5 | PluginCodec_H323AudioCodec_g711Alaw_56k | Not used (set to NULL) | G.711 Alaw at 56k |
| 6 | PluginCodec_H323AudioCodec_g711Ulaw_64k | Not used (set to NULL) | G.711 Ulaw at 64k |
| 7 | PluginCodec_H323AudioCodec_g711Ulaw_56k | Not used (set to NULL) | G.711 Ulaw at 56k |
| 8 | PluginCodec_H323AudioCodec_g722_64k | Not used (set to NULL) | G.722 at 64k |
| 9 | PluginCodec_H323AudioCodec_g722_56k | Not used (set to NULL) | G.722 at 56k |
| 10 | PluginCodec_H323AudioCodec_g722_48k | Not used (set to NULL) | G.722 at 48k |
| 11 | PluginCodec_H323AudioCodec_g7231 | If 0, silence supression is disabled If non-zero, silence supression is enabled |
G.723.1 |
| 12 | PluginCodec_H323AudioCodec_g728 | Not used (set to NULL) | G.728 |
| 13 | PluginCodec_H323AudioCodec_g729 | Not used (set to NULL) | G.729 |
| 14 | PluginCodec_H323AudioCodec_g729AnnexA | Not used (set to NULL) | G.729 Annex A |
| 15 | PluginCodec_H323AudioCodec_is11172 | Not used (set to NULL) | Not yet implemented |
| 16 | PluginCodec_H323AudioCodec_is13818Audio | Not used (set to NULL) | Not yet implemented |
| 17 | PluginCodec_H323AudioCodec_g729wAnnexB | Not used (set to NULL) | G.729 Annex B |
| 18 | PluginCodec_H323AudioCodec_g7231AnnexC | Pointer to structure of type H323AudioG7231AnnexC. If NULL, default parameters are used | G.723.1 with Annex C |
| 19 | PluginCodec_H323AudioCodec_gsmFullRate | Pointer to structure of type PluginCodec_H323AudioGSMData. If NULL, default parameters are used | GSM 06.10 |
| 20 | PluginCodec_H323AudioCodec_gsmHalfRate | Pointer to structure of type PluginCodec_H323AudioGSMData. If NULL, default parameters are used | GSM half rate |
| 20 | PluginCodec_H323AudioCodec_gsmEnhancedFullRate | Pointer to structure of type PluginCodec_H323AudioGSMData. If NULL, default parameters are used | GSM enhanced full rate |
| 21 | PluginCodec_H323AudioCodec_g729Extensions | Not used (set to NULL) | Not yet implemented |
| 22 | PluginCodec_H323VideoCodec_h261 | Not used (set to NULL) | Not yet implemented |
| 23 | PluginCodec_H323VideoCodec_h262 | Not used (set to NULL) | Not yet implemented |
| 24 | PluginCodec_H323VideoCodec_h263 | Not used (set to NULL) | Not yet implemented |
| 255 | PluginCodec_H323Codec_NoH323 | Not used (set to NULL) | Used to indicate codec cannot be used for H.323 |
PluginCodec_H323NonStandardCodecData
The plugin interface provides an interface that allows two methods for plugins to be identified as H.323 non-standard codecs:
- A simple interface for capabilities that contain static data (such as a text string) in the nonStandard data element
- A procedural interface for capabilities that need to analyse the nonStandard data to determine if the capability is compatible
These interfaces are implemented using the PluginCodec_H323NonStandardCodecData structure which is defined as follows:
struct PluginCodec_H323NonStandardCodecData {
const char * objectId;
unsigned char t35CountryCode;
unsigned char t35Extension;
unsigned short manufacturerCode;
const unsigned char * data;
unsigned int dataLength;
int (*capabilityMatchFunction)(struct PluginCodec_H323NonStandardCodecData *);
};
| Field | Description |
|---|---|
| objectId | Pointer to a zero-terminated string that specifies an ASN.1 object ID in ASCII form. If specified, the t35CountryCode, t35Extension and manufacturerCode fields are ignored and the nonStandard capability uses the objectID form |
| t35CountryCode | If objectId is NULL, this field specifies the value of the t35CountryCode field in the nonStandard capability |
| t35Extension | If objectId is NULL, this field specifies the value of the t35Extension field in the nonStandard capability |
| manufacturerCode | If objectId is NULL, this field specifies the value of the manufacturerCode field in the nonStandard capability |
| data | Pointer to data specified in the nonStandard capability |
| dataLength | Length of data pointed to by the data field |
| capabilityMatchFunction | Pointer to a function that can be used to determine if this capability matches an arbitrary nonStandard capability. If not NULL, the capability will be compared by calling this function with a single argument that defines the capability to be compared. If NULL, capabilities will be matched by a simple comparison of the other fields in this structure. |
Codec control functions
A codec uses this field to specify an array of function definitions that can be used to perform operations on the code outside the scope of the normal API. Each entry in the array has the following format:
The array is terminated with an entry of the form { NULL, NULL }
A video codec must implement the following codec controls:
| get_output_data_size | Returns the size of the RTP frame output data buffer (in bytes) passed to the codec function. This ensures that the output data can always be created. |
Linking codecs statically
There are situations in which it is more convenient to include the codecs into the final executable rather than including as seperate DLLs. Some support for statically linking codecs has been included in the plugin API, and for Linux this has been made available for the GSM codec via the "--enable-embeddedgsm" switch to configure. Internally this is implemented via the H323_EMBEDDED_GSM symbol - to find more search for this symbol within the OpenH323 source code.
Change History
| 25 April 2006 | Craig Southeren | Added information on video codecs |
| 7 March 2005 | Craig Southeren | Added information on wideband codecs, frame erasure etc |
| 20 August 2004 | Craig Southeren | Added section on static linking of codecs |
| 13 July 2004 | Craig Southeren | Added new information on plugin file name conventions Added information on H323PluginCodecManager::GetMediaFormats returning video media formats |
| 15 June 2004 | Craig Southeren | Fixed cut and paste error, thanks to Matthias Weber Added extensive new documentation on media formats, codecs and capabilities |
| 6 April 2004 | Craig Southeren | Fixed typo, thanks to Andreas Sikkema |
| 5 April 2004 | Craig Southeren | Initial version released |
| 4 May 2004 | Craig Southeren | Document created |