Welcome!

Welcome to the official BlackBerry Support Community Forums.

This is your resource to discuss support topics with your peers, and learn from each other.

inside custom component

Native Development

Reply
New Contributor
RejP
Posts: 6
Registered: ‎03-03-2012
My Device: Playbook
Accepted Solution

Real-time audio processing, is it possible?

[ Edited ]

 

Hi I'm trying to port my existing synthesizer app to Playbook and I can't seem to get the audio streaming correctly.

 

I have created a separate thread for my audio processing that basically contains the example code from the "PlayWav" sample but replacing the "read from file" to instead call my process() function for generating audio signal.

 

There's a few problems:

 

1 - Constant underruns: If I just generate whitenoise with a simple rand() instead of calling my process() function, it runs ok, but as soon as I ask it to do a bit of work by running my synth code, it drops to ~10% signal / 90% underrun stall time. The process function is demanding, but the same code runs just fine at ~5-10% CPU load on an equivalent Android device.

 

I tried the following to raise the priority of the audio thread, which doesn't seem to do much, yet it doesn't return any errors either.

 

static pthread_attr_t s_AudioThreadAttr;

int nPolicy;
sched_param_t sch_prm;

pthread_attr_init(&s_AudioThreadAttr);
pthread_create(&s_AudioThread, &s_AudioThreadAttr, &myAudioThread, NULL);
pthread_getschedparam(s_AudioThread, &nPolicy, &sch_prm);
sch_prm.sched_priority = sched_get_priority_max(SCHED_FIFO);
int nErrCheck = pthread_setschedparam(s_AudioThread, SCHED_FIFO, &sch_prm);

 

2 - Buffer sizes: I'm aiming for low-latency processing. The fragment size returned to the pcm plugin info is fine (~15ms) but then there is the number of fragments returned by the plugin setup, which is really high. Here is the info printed by the example code for my setup:

 

SampleRate = 44100, channels = 2, SampleBits = 16
Format Signed 16-bit Little Endian 
Frag Size 2820 
Total Frags 85 
Rate 44100 
Voices 2

 

If I'm reading that right, that's 85 x (2820 / 2 / sizeof(short)) = 59925 samples for the frag buffers... that's over one second if I'm waiting for a full buffer before playing. I'm not 100% certain what that represents but if that's going to be the delay between presing a key and hearing the sound, then I need to find another way.

 

There are apps out there that seem to play with little latency so maybe I'm looking at the wrong API. 

 

Can anyone help me out please?

 

Please use plain text.
Retired
gbeukeboom
Posts: 2,559
Registered: ‎10-16-2009
My Device: BlackBerry Z10

Re: Real-time audio processing, is it possible?

Hi RejP,

 

I don't see which APIs/libraries are being used to actually play the audio, could you post this code (the snippet from PlayWav that was modified and a sample of your process() function which reproduyces this issue would help)?

 

Regards,

Garett
@garettBeuk
--
Goodbye everybody!
Please use plain text.
New Contributor
RejP
Posts: 6
Registered: ‎03-03-2012
My Device: Playbook

Re: Real-time audio processing, is it possible?

Hi Garett, here's the code in question. It's basically a combination of the PlayWav sample and the FallingBlock sample.

 

I have stubbed out the rendering and replaced the process function with a small loop calling sin() 100x per sample to generate signal and put a little bit of load on the CPU.

 

When I run this and record the output, you can see/hear the underruns, which also show up in the output log.

 

Thanks for taking a look, hopefully it's something obvious I've done wrong.

Please use plain text.
Retired
gbeukeboom
Posts: 2,559
Registered: ‎10-16-2009
My Device: BlackBerry Z10

Re: Real-time audio processing, is it possible?

Hi RejP,

 

I copied your code changes into the PlayWav sample and it started working properly. This leads me to believe that the issue could be related to some of the other changes implemented in the application or possibly the threading used. Could you please try modifying the sample PlayWav sample with just the code required to use your ProcessSynth method and let us know what you find?

 

Cheers,

Garett
@garettBeuk
--
Goodbye everybody!
Please use plain text.
New Contributor
RejP
Posts: 6
Registered: ‎03-03-2012
My Device: Playbook

Re: Real-time audio processing, is it possible?

[ Edited ]

Sorry about the delay in response, I moved over to v2.0.0 of the native tools and got all sorts of issues, I ended up re-installing everything.

 

The PlayWav example in the latest SDK is a bit different from the one I was using in v1.0.0. The samplerate set by the WAV file is lower (11025). so my previous "load test" does indeed play fine. 

 

With the latest set of tools, I can almost get the app to follow in real-time if I set my app's output to 22KHz, mono, but the goal is to have 44KHz stereo.

 

So let me refocus the question: What is the correct way of creating the highest priority thread under TabletOS? I noticed my rendering runs very smooth from the main thread. I just need to find a way to balance the load but mostly I need to make the OS understand that my audio thread needs to run uninterrupted until it blocks (during the pcm_write call()) and everything else is secondary. Is this possible?

 

Edit: I can confirm the following code from the original question has no effect on the audio thread's priority so either I'm doing it wrong or it simply isn't possible to raise prioirty. (the 1st is more likely)

 

static pthread_attr_t s_AudioThreadAttr;

int nPolicy;
sched_param_t sch_prm;

pthread_attr_init(&s_AudioThreadAttr);
pthread_create(&s_AudioThread, &s_AudioThreadAttr, &myAudioThread, NULL);
pthread_getschedparam(s_AudioThread, &nPolicy, &sch_prm);
sch_prm.sched_priority = sched_get_priority_max(SCHED_FIFO);
int nErrCheck = pthread_setschedparam(s_AudioThread, SCHED_FIFO, &sch_prm);

 

Thanks,

Rej

Please use plain text.
New Contributor
RejP
Posts: 6
Registered: ‎03-03-2012
My Device: Playbook

Re: Real-time audio processing, is it possible?

Sorry for wasting your time, the error was indeed on my end. My process function had some timing syncronization code left in it from the last device I ported to (it needed sync code because of terrible latency) which was causing unnecessary stalls.

 

The PlayWav sample code now runs my synth just fine, with no need for thread balancing or priority changes. As for latency, by tweaking the number of min-max frags from (1,-1) in the example to (1,4), I'm getting 15ms latency in 44KHz stereo with no underruns, which is fine for now.

 

Apologies again, and thanks for your help.

Please use plain text.