BlackBerry® 10 Best Practices for Native Full-Duplex VoIP Application Development

 

API Reference: https://developer.blackberry.com/native/reference/core/

 

Opening audio interface:

Use audio_manager_snd_pcm_open_name() to open the audio interface

Example:  audio_manager_snd_pcm_open_name(AUDIO_TYPE_VIDEO_CHAT, &hPlayback, &hAudioman, “voice”, SND_PCM_OPEN_PLAYBACK);

This function combines snd_pcm_open_name() with audio_manager_get_handle() and allows the allocation of a specific audio type PCM channel in one step. The audio manager, or audioman, handles device authorization, routing and contention between applications in the system.

AUDIO_TYPE_VOICE should be used for handset calls and AUDIO_TYPE_VIDEO_CHAT for video calls where the default is a speaker-phone profile. 

 

Basic Configuration

“voice” PCM device is intended for full-duplex VoIP where acoustic echo cancellation and noise suppression are needed.  It has been optimized for 8, 16 and 48 kHz, with a recommended fragment size of 16 ms.  A 20 ms fragment size can also be used at the expense of a higher CPU load.

For lowest CPU load, it is recommended that both capture and playback be configured with a fragment size of 16 ms, and that frags_max be increased to accommodate the mismatch between the VoIP packet size and the frag_size used with “voice”.

As with other AIFs (Audio Interchange Formats), the frag_size requested using snd_pcm_plugin_params() is only a suggestion, the actual frag_size configured will be returned by snd_pcm_plugin_params() or a call to snd_pcm_plugin_setup().

The first capture or playback “voice” AIF configured using snd_pcm_plugin_params() becomes the ‘master’ AIF for sample rate, number of channels and sample rate configuration.  A call to snd_pcm_plugin_params() on any successive AIF will only retrieve these configuration settings. This results in an equivalent configuration for all AIFs connected to “voice” PCM device.

The number of VoIP channels should be used for configuration. The difference between the number of VoIP channels and the actual platform hardware capture and playback channels will be managed internally.

 

Stop Modes

It is important to set stop_mode to SND_PCM_STOP_ROLLOVER in order to avoid unnecessary disruptions in audio flow.  Since in this mode snd_pcm_plugin_write()/snd_pcm_plugin_read() will not fail on an underrun/overrun error condition, snd_pcm_plugin_status() should be used to periodically monitor the underrun and overrun counts. Although some underruns/overruns are expected at audio start/stop and when devices are switching, they should not be seen on a regular basis.  Increasing frags_max can reduce underruns / overruns, but if that doesn't help, it is an indication that the threads in the client app servicing “voice” are being starved.  These threads should be at a higher priority than other threads within the client app.

Note that snd_pcm_plugin_set_disable() should be used to disable PLUGIN_MMAP so that the counts return by snd_pcm_plugin_status() are representative.  Also note that both snd_pcm_plugin_set_disable() and snd_pcm_plugin_set_enable() returns a mask value rather than an error code; it is invalid to check their result against EOK or < 0.

 

Starting Audio

Starting audio on either capture or playback will start the audio flowing in both directions at the hardware level.  If the playback AIF is started first, captured audio at the hardware level will be discarded until the capture AIF is started.  Similarly if the capture AIF is started first, zeroes will be inserted at the hardware level until the playback AIF is started.

For capture, use a start_mode of SND_PCM_START_DATA. Client audio flow will start on first read/select of capture AIF.

For playback, use a start_mode of SND_PCM_START_FULL. Client audio flow will start when enough data has been written to fill all available playback frags (specified by frags_max).b

 

Starting Audio with Go

Alternatively a start_mode of SND_PCM_START_GO can be used with snd_pcm_playback_go()/snd_pcm_capture_go(), i.e.go(), to start audio. Again, starting audio on either capture or playback will start the audio flowing in both directions at the hardware level.

For playback, if SND_PCM_START_GO is used but SND_PCM_STOP_ROLLOVER isn't, at least two frags of audio data must be written to prime the playback frag queue before go() is issued or the playback AIF will start in an underrun condition.

 If configured for SND_PCM_START_GO, any time that the status for an AIF changes to anything other than SND_PCM_STATUS_RUNNING, snd_pcm_plugin_prepare() and go() must be re-issued.  For playback, if SND_PCM_STOP_ROLLOVER isn’t being used, then the playback frag queue must be re-primed as well before the go().

 

Device Switching

“voice” PCM device will automatically switch hardware input and output routes when an external device such as headphone or headset is plugged in.

While audio is flowing, the device switch will result in snd_pcm_plugin_write()/snd_pcm_plugin_read() failing. At that point, if PLUGIN_ROUTING has been enabled using snd_pcm_plugin_set_enable()snd_pcm_plugin_status() will return a status of SND_PCM_STATUS_CHANGE.  If PLUGIN_ROUTING has NOT been enabled , snd_pcm_plugin_status() will return a status of SND_PCM_STATUS_UNDERRUN/SND_PCM_STATUS_OVERRUN.  In either case the client app should call snd_pcm_plugin_prepare() before proceeding.  If the start_mode is SND_PCM_START_GO, go() will also need to be called.

If the start mode is SND_PCM_START_GO and a device switch occurs between snd_pcm_plugin_prepare() and go(), then the go() will fail.  At that point snd_pcm_plugin_status() will return a status of SND_PCM_STATUS_CHANGE (or SND_PCM_STATUS_UNDERRUN/SND_PCM_STATUS_OVERRUN if PLUGIN_ROUTING has not been enabled).  The client app should call snd_pcm_plugin_prepare() and re-issue the go().  For playback, if SND_PCM_STOP_ROLLOVER isn’t being used, then the playback frag queue must be re-primed as well before the go().

 

Handset Speaker-Phone Override

In handset mode (AUDIO_TYPE_VOICE), the client application may want to implement a speaker-phone mode.  audio_manager_set_handle_type() can be used to perform the routing override, and audio_manager_set_handle_routing_conditions() to maintain the routing even if an external device such as a headphone or headset is plugged in:

                if (audio_manager_set_audio_type(hAudioman, AUDIO_TYPE_VOICE, AUDIO_DEVICE_SPEAKER, AUDIO_DEVICE_DEFAULT) == EOK) {

                                audio_manager_set_handle_routing_conditions(hAudioman, SETTINGS_NEVER_RESET);

                }

When speaker-phone is no longer required, use audio_manager_set_handle_type() again to reset the routing:

                audio_manager_set_audio_type(hAudioman, AUDIO_TYPE_VOICE, AUDIO_DEVICE_DEFAULT, AUDIO_DEVICE_DEFAULT);

    

Bluetooth®

When the external device is a Bluetooth headset, all acoustic processing (echo cancellation, noise suppression) is disabled.

    

Orientation (Supported starting in 10.1 SDK)

Most hardware platforms have more than one microphone and / or speakers.  If a client application is orientation aware, snd_pcm_plugin_set_best_fit_voices()can be used to select the best microphones and/or speakers given the current orientation angle and number of VoIP channels:

(It is also best practice to call snd_pcm_plugin_set_best_fit_voices() on the playback path before calling it on the capture path.)

                struct {

                                snd_pcm_chmap_t chmap;

                                int pos[2];

                } map;

                map.chmap.channels = voipChannels;

                if (map.chmap.channels == 2) {

                                map.pos[0] = SND_CHMAP_FL;

                                map.pos[1] = SND_CHMAP_FR;

                } else {

                                map.pos[0] = SND_CHMAP_FC;

                }

                snd_pcm_plugin_set_best_fit_voices(hPlayback, orientationAngle, &map.chmap);

 

// For a video call, whether the camera is facing front or rear is also a consideration for microphone selection:

                struct {

                                snd_pcm_chmap_t chmap;

                                int pos[2];

                } map;

                map.chmap.channels = voipChannels;

                if (map.chmap.channels == 2) {

                                if (camera == CAMERA_UNIT_REAR) {

                                                map.pos[0] = SND_CHMAP_RR;

                                                map.pos[1] = SND_CHMAP_RL;

                                } else { // Front or None

                                                map.pos[0] = SND_CHMAP_FL;

                                                map.pos[1] = SND_CHMAP_FR;

                                }

                } else {

                                if (camera == CAMERA_UNIT_REAR) {

                                                map.pos[0] = SND_CHMAP_RC;

                                } else { // Front or None

                                                map.pos[0] = SND_CHMAP_FC;

                                }

                }

                snd_pcm_plugin_set_best_fit_voices(hCapture, orientationAngle, &map.chmap);