How to play audio from memory on RT1060 EVKB

[antoniohsu888]

I modify a mp3 file to c code file and compile with MCUXpresso IDE ready.

But I check audio play project example, there is only play from SD card.

There is a define for STREAM_PIPELINE_MEM.

Is there any example code for audio play from memory (or static buffer) which using app streamer ?

(for example: like STREAMER_Create() )

[kerryzhou · Accepted solution]

Hi @antoniohsu888 

   A good news to you, I already make the memory .mp3 data play works OK!

   I didn't use your project directly, I am using the SDK2.11.1 evkmimxrt1060_maestro_record, I changed the lib file, I will send you the private message for the project, as we can't share it in the public side, next time, you can create the case in the nxp.com, that will send email private.

  This is my test result:

Test log is in the attached picture.

project check your private message. If you can't find the project, you also can create the case with the following method and let me know your case number

1. Open below SUPPORT site, click blue "Go to Tickets" in the middle.
http://www.nxp.com/support/support:SUPPORTHOME

2.Then you will be requested to Login, if you have no an account, please first Register with your business email.

3.After login, please "Create New Cases" button in the middle, then you can submit your question.

Best Regards,

Kerry

[antoniohsu888]

Hi, Kerry:

Thanks for your great support.

I create a support case:

• Case Number : 00471829
• Status : New
• Priority : Medium
• Project Name: Smart voice recognition device

As I check the attached file in messages, it looks like my original project maestro_demo ( based onSDK version 2.10.1)

Do you mean the attached file evkmimxrt1060_maestro_record.zip is your test project based on SDK 2.11.1 ?

If it is not, please send me your test project to support case.

(I see , looks like it is your test code, let me check and test.)

thank you.

BR, Antonio.

[kerryzhou]

Hi @antoniohsu888 

  Thanks for your cooperation.

  Already send you the project in the case, check your case email, thanks.

Best Regards,

Kerry

[antoniohsu888]

Hi, Kelly:

I test your attached code base, it works well.

Thank you very much.

BR, Antonio.

[kerryzhou]

Hi @antoniohsu888 ,

   Glad to hear you also make it work.

   If your question is solved, please help me to mark the correct answer, just to close this post.

  To the created case, please also tell me to close it, then help me to fill the feedback as resolved after it is closed, thanks so much.

  Any new issues, welcome to create the new case or the new question post.

Best Regards,

Kerry

[antoniohsu888]

Hi, Kelly:

I select your post with demo video for the solution.

You can close case:00471829.

By the way, do you familiar with MATTER over wifi/thread ?

I will create a new case for MATTER integration issue.

Thank you again.

BR, Antonio.

[kerryzhou]

Hi @antoniohsu888 

  I already close the case, after you receive the feedback email, please help to mark it as resolved, thanks.

  About the MATTER over wifi/thread, just tell us, which NXP chip and wifi chip you are using, we will help you in your new case, thanks.

Best Regards,

Kerry

[kerryzhou]

Hi @antoniohsu888 ,

   Sorry for my later reply!

   I have contact with our maestro expert, and do some modification, but still meet issues.

   Our maestro expert think, it should also related to the maestro lib, they will help to fix the maestro lib, and share with us, but it needs time. So, maybe you still need keep patient, when I get the fixed version, I will let you know.

   Currently, you can use the sdcard, mp3 file play at first.

   Thanks a lot for your understanding.

Best Regards,

Kerry

[antoniohsu888]

Hi, Kerry:

Would you please help to check my mp3 play from memory settings.

It feedback the error, I think there must lack of some settings.

STREAMER CATASTROPHIC 0x000000000000000e NOT FOUND [CCID]Metadata Error
STREAMER ERROR 0x0000000000000010 INTERNAL ERROR STREAMER - Process Pipeline Error! Pipeline Index = 0, error = 252
STREAM_MSG_ERROR

please check 

#if STREAMER_FROM_MEM_ENABLE

I use this function STREAMER_file_Create() to modify to play from memory.

Here I list the main settings: In my code, it disables the EAP_PROC

1.Create a source buffer info.

MEMSRC_SET_BUFFER_T inBufInfo = {0};

voice_inBuf = pvPortMalloc(TEMP26_MP3_SIZE);

memcpy(voice_inBuf, &temp_26_mp3[0], TEMP26_MP3_SIZE); //copy mp3 file to this buffer (the mp3 file has transform to c code file)

2. Set pipeline_type 

params.pipeline_type = STREAM_PIPELINE_MEM;

handle->streamer = streamer_create(&params)

3. set source buffer

prop.prop = PROP_MEMSRC_SET_BUFF;
prop.val = (uintptr_t)&inBufInfo;
ret = streamer_set_property(handle->streamer, prop, true);

4.Set decoder type

prop.prop = PROP_DECODER_DECODER_TYPE;
prop.val = (uintptr_t)DECODER_TYPE_MP3;
ret = streamer_set_property(handle->streamer, prop, true);

5. Other settings for sampling rate for test, but not work.

prop.prop = PROP_AUDIOSRC_SET_SAMPLE_RATE;
prop.val = 40*1000;
streamer_set_property(handle->streamer, prop, true);

prop.prop = PROP_AUDIOSRC_SET_CHUNK_SIZE;
prop.val = 40 ;//AUDIO_PLAY_BUFFER_SIZE_ONE_FRAME / (2 * 2);

streamer_set_property(handle->streamer, prop, true);

prop.prop = PROP_AUDIOSRC_SET_NUM_CHANNELS;
prop.val = 1;

streamer_set_property(handle->streamer, prop, true);

I think there must have other settings for mp3 decoder, but I can't find other reference project for it.

Thanks for your help.

[kerryzhou]

Hi @antoniohsu888 ,

   Thanks for your updated information.

  So you put your convert .mp3 data to C file array temp_26_mp3, right?

  Please also tell me how do you convert the .mp3 to the c file, we need to make sure the convert mp3 data is correct at first, then share me your whole project, you can delete all other code which is important to you, just leave the maestro code which include your mp3 memory streamer data, and the related mp3 convert data, then I will help you to test it on my side.

Waiting for your updated information.

Best Regards,

Kerry

[antoniohsu888]

Hi ,Kerry:

I attach whole project files.

I use shellRecMIC() and play_file() to combined vit and voice play.

in cmd.c  #define VIT_WITH_VOICE_PLAY 1

shellRecMIC() -> play_file("temp_26.mp3", 0);

in app_streamer.c #define STREAMER_FROM_MEM_ENABLE 1

I modify play from memory in STREAMER_file_Create()

The converted mp3 file is in maestro_demo\source\temp_26.h

Covert tool:

I use xxd to convert mp3 to c file.

You can check attached xxd tool, command xxd -i your-sound.mp3 ready-to-use-header.c

reference: https://github.com/baldram/ESP_VS1053_Library/issues/18

Thank you.

[kerryzhou]

Hi @antoniohsu888 ,

   Thanks a lot for your effort.

   Please keep patient, and give me more time, I need to do the code checking and testing on my side. As I just have the MIMXRT1060-EVK board on my hand now, I will test it based on this board, if I meet any issues, I will contact with our maestro expert directly, then give you the feedback.

   Thanks a lot for your understanding, any updated information, I will let you know.

Best Regards,

kerry

[antoniohsu888]

Hi, Kelly:

Sorry forgot to roll back a free memory code for voice_inBuf and voice_outBuf 

In cmd.c ->play_file()

file_error:
PRINTF("[play_file] Cleanup\r\n");
STREAMER_Destroy(&streamerHandle);
if (voice_inBuf != NULL)
{
vPortFree(voice_inBuf);
}
if (voice_outBuf != NULL)
{
vPortFree(voice_outBuf);
}
osa_time_delay(100);

[kerryzhou]

Hi @antoniohsu888 ,

   Thanks for your updated information.

   When you do the roll back a free memory code for voice_inBuf and voice_outBuf, whether the issue still exists the same as previously? Or any test result changed?

   I still need more time, these days, a lot of cases in the testing queue, please keep patient, thanks.

Best Regards,

kerry

[antoniohsu888]

Hi, Kerry:

It is just to release memory allocate buffer, not related to mp3 decoder.

By the way, is there any way I can debug the low level code of maestro audio play?

If you can provide a example project for maestro audio play from memory ,

even for other platform, I think that will be helpful. 

(I also checked the USB audio play, but it is not maestro framework.)

BR, Antonio.

[kerryzhou]

Hi @antoniohsu888 ,

  Sorry for my later reply!

  Your request function is reasonable, I totally understand your project after checking the project details, you want to play some feedback voice after the VIT detection. and the feedback voice is stored in the memory, so you need the example about play from memory. I also test your project, the same result as you.

  So, you are right, we need to find a memory play example, as I also don't have the directly example, but I already send email internally to our maestro expert, wish they can help us to provide one, then we can add it to the application project.

  Please keep patient, any updated information, I will let you know.

  Thanks a lot for your understanding.

Best Regards,

Kerry

[antoniohsu888]

Hi, Kerry:

Got it, thanks for your help.

BR, Antonio.

[kerryzhou]

Hi @antoniohsu888 ,

   Could you please tell me which SDK version you are using? Seems you are not using the newest SDK:

  SDK_2_11_1_EVK-MIMXRT1060

  So, please tell me which SDK version you are using? Expert also give me a new lib, but I can't test it directly in your SDK, seems version is not matched, the new lib is for SDK2.11.1, I send you the private message, you also can check the new lib.

Best Regards,

Kerry

[antoniohsu888]

Hi, Kerry:

I use previous version, SDK_2_10_1_MIMXRT1060-EVK.

Because the latest version 2.11 has no maestro_demo project ,

the VIT record and audio play is separated project and hard to integrate with VIT and audio play.

Let me check how to use the new library.

By the way, do you have  maestro_demo project  for SDK 2.11.1 version ?

Thank you very much.

BR, Antonio.

[kerryzhou]

Hi @antoniohsu888 ,

   SDK2.11.1 also has the maestro code:

yes, the code which I share with you is based on the above project.

Do you receive my project in a private message?

If not, please create the case, and let me know, then I will send you the project directly. That project I already add the play_back code, and from .mp4, you can know it works with your mp3 data. I also use VIT, just after the detection finished, then play the memory mp3 data.

Wish it helps you!

Best Regards,

Kerry