i.MX Solutions中的主题 Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

nsoni — Thu, 12 May 2022 06:08:10 GMT

Hello, I am using imx8m-plus EVK kit along with 8MIC-RPI-MX8 mic board attached to it. I am trying to read 8-mic data from mic board along and 1ch from alsa loopback sound card to be used as aec reference signal (which is coming from PJSIP voip client app running separately). I am getting periodically xrun issue in the application which is reading this audio samples. It seems clocks are not synchronized between mic board and alsa loopback sound card. Looking for help to debug and fix this issue.

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

juangutierrez — Thu, 12 May 2022 16:18:44 GMT

Hi nsoni

I guess you are using micfil card for reading from the microphones. Is this correct?

Have you tried using the AFE wrapper

https://source.codeaurora.org/external/imx/nxp-afe/tree/TODO.md?h=MM_04.06.02_2108_L5.10.y

You can try the libdummy

## using dummy library
1. ./afe libdummy &
2. aplay S32LE16000.wav &
3. arecord -d10 -fS32_LE -r16000 -c1 dummy_afe_on.wav

The AFE already includes the alsa loopbacks on the asound.conf, so as long as your application (PJSIP) writes to the default alsa device (like in step 1) that should be loopbacked for AEC.

Within the SignalProcessor_Dummy.cpp you can add your calls for your AEC algorithm at function SignalProcessor_Dummy::processSignal

https://source.codeaurora.org/external/imx/nxp-afe/tree/src/SignalProcessor/SignalProcessor_Dummy.cpp?h=MM_04.06.02_2108_L5.10.y#n96

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

juangutierrez — Thu, 12 May 2022 16:22:06 GMT

As you mentioned the problem with the XRUN might be because not using the same clk. using the AFE wrapper this is not an issue because it uses 2 alsa loopbacks: one for the playback (reference signal) and another one for the clean voice. Since both sides (capture and playback) used alsa loopbacks the clk is the same and drifting is avoid.

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

juangutierrez — Thu, 12 May 2022 18:16:57 GMT

The AFE also supports dspc I wonder if you might have access already for the AFE wrapper with libdspc support

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

nsoni — Fri, 13 May 2022 06:53:56 GMT

Hi juangutierrez,

Thanks for the update. Yes, I am micfil card to read the data. I was able to run afe for some time and it did not show any xrun error. I will go through its AudioStream code and check I can use it for our application. Our application audio plumbing is slightly different where PJSIP (voip client) sends and receives mono channel to loopback card which is sent/receive by DSPC application engine. PJSIP also connects to server and transmit/receive data to/from far end.

I am not aware about libdspc support with afe but I will check internally about it. However, if you have access to libdspc code & lib, you can send me as well. Thanks

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

juangutierrez — Mon, 16 May 2022 23:41:25 GMT

These are the patches on the latest MM_04.06.04_2112_L5.15.y to support libDSPC on the nxp-afe

Additionally to the patch you should have a third-party directory at the top directory of the nxp-afe

The second patch is for this directory but it is NOT included the binaries. I guess you should got this from your side.

Thanks

juan

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

nsoni — Tue, 17 May 2022 07:31:27 GMT

Hi juangutierrez,

Thanks for the update. I changed the AFE source code for setting period size to 240 (instead of 512) and rate to 48K (instead of 16K) to make AFE config same as what we are using in our application. I also enabled the XRUN error message in AudioStream.cpp to print the error when stream state is underrun or overrun. I see that XRUN run issue is very easily reproduced with AFE code as well with this period size and sample rate. I used libdummy only for my testing.

Let us know if you can also reproduce the issue or not. If yes, Is there any way to overcome it ? Since our voice algo uses 240 period size, we need to set the same in application.

Attached the patch for changing period size, sample rate and add the xrun error message.

Attached the log as well.

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

Ohtokani_Pinzón — Wed, 18 May 2022 18:47:08 GMT

Hi nsoni,

I compiled the nxp-afe applying the patch where you changed the rates and I reproduced the same error message. I'm still trying to figure out the problem.

Somting I noticed is that the period size in asound.conf was still set to 512, so I changed it to 240, but the error is still happening.

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

juangutierrez — Wed, 18 May 2022 19:54:18 GMT

Hi @Ohtokani_Pinzón

Can you try with a multiple of 240 instead? Like 480 or 960.

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

nsoni — Wed, 18 May 2022 22:22:06 GMT

Hi juangutierrez & Ohtokani,

Thanks for update so far. Today I tried different period size and sample rate including the default one (512 samples, 16Khz rate) with NXP AFE application and xrun issue is seen with that as well after ~45 minutes. Here is the summary of experiments. This issue is blocking our VOIP demo kit solution.

==========================
Rate is: 48000[Hz]
Period size is: 240frames
Buffer size is: 960frames
Block size : 5ms
Duration: 47 sec
=========================
Rate is: 16000[Hz]
Period size is: 80frames
Buffer size is: 320frames
Block size : 5ms
Duration: 1min 23sec
=========================
Rate is: 48000[Hz]
Period size is: 768frames
Buffer size is: 3072frames
Block size : 16ms
Duration: 25 min
=========================
Rate is: 16000[Hz]
Period size is: 512frames
Buffer size is: 2048frames
Block size : 32ms
Duration: 45 min
=========================

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

nsoni — Sat, 21 May 2022 03:00:48 GMT

Tried to check if this is related to clock synchronization issue between SAI interfaces and found that imx8m-plus has hardware ASRC block from imx8m-plus reference manual. I also inserted the snd-soc-fsl-asrc module found in kernel modules but that did not help much. Does this driver support or updated for imx8m-plus SOC or is there any specific way to enable it once the module is inserted ?

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

Ohtokani_Pinzón — Tue, 24 May 2022 17:58:44 GMT

Hi nsoni

I tried adding some changes from an internal afe repo, and I haven't seen the XRUN error running it for an hour. Here is the patch to apply those changes.

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

juangutierrez — Tue, 24 May 2022 22:26:51 GMT

Notice this patch is a porting from a previous version of the AFE where we tested and did not see the issue. Ohtokani created a patch that might contain unrelated changes. So this is just for speeding up things and you can test on your side and verify this also fixes the problem at your side.

If so, then we can try to narrow down to see what the actual fix is.

Would you verify this works on your side, please?

BTW, to apply this patch you might remove the previous patch to add the support for dspc library.

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

nsoni — Tue, 24 May 2022 22:53:38 GMT

Hi juangutierrez,

I reset the repo, applied the patch by Ohtokani, rebuilt the afe again. I also made sure asound conf is updated for 240 period size. However, I still see the xrun issue within 2 minutes. Let me know if there is any missing step or you guys are also able to see it.

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

juangutierrez — Tue, 24 May 2022 23:38:10 GMT

Hi @nsoni

Not sure if Ohtokani changed the asound.conf. I think you should keep the default value (I guess 512)

@Ohtokani_Pinzón can you confirm if you changed or not the asound.conf?

Also, which command did you use for your testing?

arecord and which parameters?

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

juangutierrez — Tue, 24 May 2022 23:47:29 GMT

Hi nsoni

I think Ohtokani shared the libdummy.so as part of the patch, would you try by replacing this pre-built libdummy.so on your filesystem (I gues at /usr/lib/).

Make sure you removed or renamed the libdummy.so that comes as part of original filesystem.

Maybe try by searching them with

find / -name libdummyimpl.so*

and make sure you are replacing both the actual library libdummyimpl.so.1.0 and the sym link libdummyimpl.so pointing to it

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

nsoni — Wed, 25 May 2022 01:27:17 GMT

Hi juangutierrez,

I also tried replacing the libdummyimpl.so & libdummyimpl.so.1.0 at /usr/lib/nxp-afe with the ones generated after applying patch. But I can see same issue. I also tried asound conf with 512 period size and 240 size both, this also does not help. Attached log of my steps. I open playback and capture in separate shell which is not in the log.

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

Ohtokani_Pinzón — Wed, 25 May 2022 15:34:44 GMT

Hi nsoni

Here are the binaries and the asound.conf I used, I am using Linux hardknott 5.10.35

Can you share with us the asound.conf and the Linux version you are running?

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

juangutierrez — Wed, 25 May 2022 16:21:21 GMT

Hi nsoni

Would you try with the binaries shared by Ohtokani and the asound.conf

Place the asound.conf at /etc/ directory.

We no longer see the xruns with those binaries.

which linux version are you using, do you have any other modification?

Re: xrun issue while reading audio data from 8MIC-RPI-MX8 and alsa loopback sound card

nsoni — Wed, 25 May 2022 17:47:15 GMT

HI juangutierrez,

I compared binaries shared by Ohtokani and libdummyimpl.so/libdummyimpl.so.1.0 are same. I used afe binary from Ohtokani and with that it's running for last 30 minutes without any xrun. I hope XRUN message is enabled in the binary. If so, Can you please share the afe source code with me?

There is slight different in asound conf as well but that is in codec part as below.

From Ohtokani :

pcm.spk
{
type plug
slave.pcm "hw:wm8524audio,0"
}

From nsoni :

pcm.spk
{
type plug
slave.pcm "hw:wm8960audio,0"
}

It seems codec is different on our board. I am using imx8m+plus EVK lpddr4 kit by the way. Anyways, I kept mine only during the experiment.