How to add an extra language to sln_local2_iot_local_demo Application of MIMXRT106S MCU

[Merry]

Hi,

I need to add a wake word and a command set in the Russian language to the existing demo application. I  added the commands and wake word using the DSpotter tool, put dsmt file in the respective folder, and added necessary changes in other files. When I compile, there is no error occurs, .Flashing was done and after that, I can't get the serial terminal.

I think there is something I left behind. Can you help me in figuring out this?

[kerryzhou]

Hi @Merry ，

    Thanks for your interest in the NXP MIMXRT product, I would like to provide service for you.

    Which detailed document you are referring to add the new language? Except for the DSPortter tool, the code also needs to modify several points.

  But you mentioned, your build has no error, but after download, no serial terminal.

  So, seems your flash or code have issues.

   When you download the code, which method you are using? debugger or MSD?

   If you are using MSD, you need to change the flash address, consider bank A or B.

   But if you are using the debugger, you need to download it to bank A.

  I think you can download it and test it again, just make sure your downloading is OK.

  If you still have issues, just kindly let me know.

Best Regards,

KERRY

[Merry]

Hi @kerryzhou  

There is no proper document for me to get the details regarding the new language. AN13212 document helps me in adding new commands and know little more about DSpotter Tool. Currently, I am adding the changes to the Russian language by analyzing how other languages have been added to the code.

I am using the Debugger method to download the code. I am downloading the application to bank A. Still, the serial terminal is not working . I doubted if I miss anything in the code.

Since my project folder is more than 25Mb size, I am attaching local_voice and source folder which are the main areas where I made changes. Pls kindly check.

[kerryzhou]

Hi @Merry ,

  Please check the private message, I send you a private pdf which tell you how to add the new language, please don't share it externally.

  Then you can try it again. 

  If you still have questions about it, please kindly let me know.

Best Regards,

Kerry

[Merry]

Hi @kerryzhou 

I tried to add the Russian language along with the English language removing other languages in the SDK. I made the changes accordingly with the document shared 'Add New language.pdf' except adding the playback audio files. (Here, adding playback audio files is not necessary for our use case)

Here, my English language contains wake word, custom voice demo named gun, led demo, dialogic demos in the same order. The Russian language contains wake word and gun demo. Here, I didn't add a playback audio file for the gun demo and for the Russian language. When I compile it, there were no errors.

I will attach the output after downloading this code. The solid purple led state occurs soon after flashing. Wake word in English and Russian is not detected while speaking. This led state determines audio stream error right? 

Is the algorithm requires playback audio files? Is it required to add audio files for detecting wake word and voice commands

[kerryzhou · Accepted solution]

Hi @Merry ,

    Sorry for my later reply!

    Please don't worry. In fact, it is not difficult.

   If you totally follow the pdf which I send you privately, it should easy to create it.

    As you know, the SDK already support 4 languages: English, Chinese, German, French.

   If you have issues, I think, you totally can modify one language to Russian, Eg, you can modify French to your Russian language, then use your DSMT generate code in folder:

sln_local2_iot_local_demo\local_voice\oob_demo_fr

In the console, you need select the languge to French:

mutilingual fr

  This will help you to test your Russian language wakeup and voice detect, after it works, then you can use the old method, add the new language.

   Please try it again.

   If you still have issues, you can provide your wakeup and voice detect, when I have time, I may help you to check it.

    But, From 1.29 to 2.6 is our Chinese Spring Festival, I am on the vocation, will back on 2.7.

    Thanks a lot for your understanding.

     BTW, if you can't wait so long time, as I know, you also can send email to : local-commands@nxp.com , our solution team will also help you.

Best Regards,

Kerry

[Merry]

Hi @kerryzhou  

Could you pls check on this.

Thanks 

Merry

[Merry]

Thanks for the document.Will keep it private.

I have doubts regarding adding audio files (in .dat format )in the Russian language. Can we add other audio formats like mp3?

[kerryzhou]

Hi @Merry 

   What's the audio file you want to add? Do you mean play back?

   If this one, you can refer to my doc:

https://community.nxp.com/t5/i-MX-RT-Knowledge-Base/RT106L-S-voice-control-system-based-on-the-Baidu...

 chapter 2.3 playback audio data prepare and burn.

  For the mp3, you can convert it to the wave, then use the script convert it to bin, and burn it to the playback area.

Best Regards,

Kerry

[Merry]

Hi @kerryzhou  

I mean playback audio files.Thanks for the doc.

Baidu Smart Cloud's online TTS function and Audioacity Audio tool helps to get a 48K16bit wav file right?

Here ,the document mainly focuses on getting chinese audio files.So ,what is the case of other languages.Can we get same with these platforms or is there any other platform for other languages.

After saving audios in the filesystem ,rest of the information is given in 2.3 section of the document.So ,I have doubted how to get our 48k16bit wave file('demo name' , 'can i help you' , 'ok') for the russian language. 

[kerryzhou]

Hi @Merry ,

  Baidu Smart Cloud's online TTS is just an example, you also can find some other TTS, which can support the other language.

 you can use Text-To-Speech (TTS) service from Amazon, Microsoft Azure, etc. Or someone can record own voice.

  About the frequency change, you can use the ffmpeg to convert it.

  You can refer to my doc:

https://community.nxp.com/t5/i-MX-RT-Knowledge-Base/RT10xx-SAI-basic-and-SDCard-wave-file-play/ta-p/...

 chapter 2.3 Audio file convert

Wish it helps you!

Best Regards,

Kerry

[Merry]

Hi  @kerryzhou  ,

Thanks for the information. I will check on this.