xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

[nsoni]

Hello, I am using imx8m-plus EVK kit along with 8MIC-RPI-MX8 mic board attached to it. I am trying to read 8-mic data from mic board along and 1ch from alsa loopback sound card to be used as aec reference signal (which is coming from PJSIP voip client app running separately). I am getting periodically xrun issue in the application which is reading this audio samples. It seems clocks are not synchronized between mic board and alsa loopback sound card. Looking for help to debug and fix this issue.

[juangutierrez]

Hi nsoni

I think Ohtokani shared the libdummy.so as part of the patch, would you try by replacing this pre-built libdummy.so on your filesystem (I gues at /usr/lib/).

Make sure you removed or renamed the libdummy.so that comes as part of original filesystem.

Maybe try by searching them with 

find / -name libdummyimpl.so* 

and make sure you are replacing both the actual library libdummyimpl.so.1.0 and the sym link libdummyimpl.so pointing to it

[nsoni]

Hi juangutierrez,

I also tried replacing the libdummyimpl.so & libdummyimpl.so.1.0 at /usr/lib/nxp-afe with the ones generated after applying patch. But I can see same issue. I also tried asound conf with 512 period size and 240 size both, this also does not help.  Attached log of my steps. I open playback and capture in separate shell which is not in the log. 

[nsoni]

Tried to check if this is related to clock synchronization issue between SAI interfaces and found that imx8m-plus has hardware ASRC block from imx8m-plus reference manual. I also inserted the snd-soc-fsl-asrc module found in kernel modules but that did not help much. Does this driver support or updated for imx8m-plus SOC or is there any specific way to enable it once the module is inserted ?

[nsoni]

Hi  juangutierrez &  Ohtokani,

Thanks for update so far. Today I tried different period size and sample rate including the default one (512 samples, 16Khz rate) with NXP AFE application and xrun issue is seen with that as well after ~45 minutes. Here is the summary of experiments. This issue is blocking our VOIP demo kit solution.

==========================
Rate is: 48000[Hz]
Period size is: 240frames
Buffer size is: 960frames
Block size : 5ms
Duration: 47 sec
=========================
Rate is: 16000[Hz]
Period size is: 80frames
Buffer size is: 320frames
Block size : 5ms
Duration: 1min 23sec
=========================
Rate is: 48000[Hz]
Period size is: 768frames
Buffer size is: 3072frames
Block size : 16ms
Duration: 25 min
=========================
Rate is: 16000[Hz]
Period size is: 512frames
Buffer size is: 2048frames
Block size : 32ms
Duration: 45 min
=========================

[Ohtokani_Pinzón]

Hi nsoni

I tried adding some changes from an internal afe repo, and I haven't seen the XRUN error running it for an hour. Here is the patch to apply those changes.

[juangutierrez]

Notice this patch is a porting from a previous version of the AFE where we tested and did not see the issue. Ohtokani created a patch that might contain unrelated changes. So this is just for speeding up things and you can test on your side and verify this also fixes the problem at your side.

If so, then we can try to narrow down to see what the actual fix is.

Would you verify this works on your side, please?

BTW, to apply this patch you might remove the previous patch to add the support for dspc library.

[nsoni]

Hi juangutierrez,

I reset the repo, applied the patch by Ohtokani, rebuilt the afe again. I also made sure asound conf is updated for 240 period size. However, I still see the xrun issue within 2 minutes. Let me know if there is any missing step or you guys are also able to see it.

[juangutierrez]

Hi @nsoni 

Not sure if Ohtokani changed the asound.conf. I think you should keep the default value (I guess 512)

@Ohtokani_Pinzón can you confirm if you changed or not the asound.conf?

Also, which command did you use for your testing?

arecord and which parameters?

[Ohtokani_Pinzón]

Hi nsoni

Here are the binaries and the asound.conf I used, I am using Linux hardknott 5.10.35

Can you share with us the asound.conf and the Linux version you are running?

[juangutierrez]

Hi @Ohtokani_Pinzón 

Can you try with a multiple of 240 instead? Like 480 or 960.

[Ohtokani_Pinzón]

Hi nsoni,

I compiled the nxp-afe applying the patch where you changed the rates and I reproduced the same error message. I'm still trying to figure out the problem.

Somting I noticed is that the period size in asound.conf was still set to 512, so I changed it to 240, but the error is still happening.

[nsoni]

Hi juangutierrez,

Thanks for the update. I changed the AFE source code for setting period size to 240 (instead of 512) and rate to 48K (instead of 16K) to make AFE config same as what we are using in our application. I also enabled the XRUN error message in AudioStream.cpp to print the error when stream state is underrun or overrun. I see that XRUN run issue is very easily reproduced with AFE code as well with this period size and sample rate. I used libdummy only for my testing.  

Let us know if you can also reproduce the issue or not. If yes, Is there any way to overcome it ? Since our voice algo uses 240 period size, we need to set the same in application.

Attached the patch for changing period size, sample rate and add the xrun error message.

Attached the log as well. 

[juangutierrez]

Hi

These are the patches on the latest MM_04.06.04_2112_L5.15.y to support libDSPC on the nxp-afe

Additionally to the patch you should have a third-party directory at the top directory of the nxp-afe

The second patch is for this directory but it is NOT included the binaries. I guess you should got this from your side.

Thanks

juan

[nsoni]

Hi juangutierrez,

Thanks for the update. Yes, I am micfil card to read the data. I was able to run afe for some time and it did not show any xrun error. I will go through its AudioStream code and check I can use it for our application. Our application audio plumbing is slightly different where PJSIP (voip client) sends and receives mono channel to loopback card which is sent/receive by DSPC application engine. PJSIP also connects to server and transmit/receive data to/from far end.  

I am not aware about libdspc support with afe but I will check internally about it. However, if you have access to libdspc code & lib, you can send me as well. Thanks

[juangutierrez]

The AFE also supports dspc I wonder if you might have access already for the AFE wrapper with libdspc support

[juangutierrez]

Hi nsoni

I guess you are using micfil card for reading from the microphones. Is this correct?

Have you tried using the AFE wrapper

https://source.codeaurora.org/external/imx/nxp-afe/tree/TODO.md?h=MM_04.06.02_2108_L5.10.y

You can try the libdummy 

## using dummy library
1. ./afe libdummy &
2. aplay S32LE16000.wav &
3. arecord -d10 -fS32_LE -r16000 -c1 dummy_afe_on.wav

The AFE already includes the alsa loopbacks on the asound.conf, so as long as your application (PJSIP) writes to the default alsa device (like in step 1) that  should be loopbacked for AEC.

Within the SignalProcessor_Dummy.cpp you can add your calls for your AEC algorithm at function SignalProcessor_Dummy::processSignal

https://source.codeaurora.org/external/imx/nxp-afe/tree/src/SignalProcessor/SignalProcessor_Dummy.cp...

[juangutierrez]

As you mentioned the problem with the XRUN might be because not using the same clk. using the AFE wrapper this is not an issue because it uses 2 alsa loopbacks: one for the playback (reference signal) and another one for the clean voice. Since both sides (capture and playback) used alsa loopbacks the clk is the same and drifting is avoid.