RNN on FRDM_K64 to denoise

cancel
Showing results for 
Search instead for 
Did you mean: 

RNN on FRDM_K64 to denoise

No ratings

RNN on FRDM_K64 to denoise

RNN on FRDM_K64 to denoise

1 Introduction

Ordinary MCUs are limited by resources, and it is difficult to do some complex deep learning. But although it is difficult, it can still be done.

Last time we used CNN for handwritten recognition. This time ,we use RNN for audio noise reduction. This audio noise reduction uses fixed-point noise reduction. MFCC and gain are used as input parameters for training.

MFCC, Mel frequency cepstrum, is often used for audio feature extraction. RNN achieves the effect of noise reduction by adjusting the gain of different frequencies.

      

2 Experiment

2.1 Required tools: frdm-k64, python 3.7, Pip, IAR

 

2.2 Download the source code of deep learning framework,

https://github.com/majianjia/nnom This is a pure C framework that does not  

rely on hardware structure. Transplantation is very convenient

 

 

2.3 we select the example ‘bubble’ to add the Inc, port and Src folders in NNoM to the project, as shown in the figure

nxf56274_0-1615450961299.png

 

                     Figure 1

Open the file ‘port.h’ . The definitation of NNOM_LOG is changed to PRINTF (__ VA_ ARGS__ , Open the ICF file, and change the heap size to 0x5000, define symbol__ size_ heap__ = 0x5000;

Malloc, which is used in this library, allocates memory from here. If it is small, it can't run the network

 

2.4 Transplant fatfs file system to bubble project

nxf56274_1-1615450975431.png

 

Figure 2

 

2.5 In the bubble.c file, add the following header file

       #include "nnom_port.h"

#include "nnom.h"

#include "weights.h"

#include "denoise_weights.h"

#include "mfcc.h"

#include "wav.h"

#include "equalizer_coeff.h"

 

2.6 Find main.c under ‘examples\rnn-denoise’ in the framework, ‘main_arm.c’ is used for stm32. Main.c is provided for windows running routines. This code is used to open the audio file, then run the network, and finally generate noise reduction. Use this code to facilitate experiments. The APIs for file operations in this file need to be manually changed to MCU APIs. I added a progress bar display. You can refer the attachment.

 

2.7 This example also uses DSP, so DSP support needs to be turned on, as shown in the figure

nxf56274_2-1615450986521.png

 

                                                        Figure 3

2.8 Add macro

nxf56274_3-1615450994191.png

 

                                          Figure 4

 

2.9 After the compilation is passed, download the program. The tested wav needs to be named ‘sample.wav’, and the mcu will reduce noise for it. Finnally mcu will

generate ‘filtered_sample.wav’. Plug the SD card into the computer to listen to the noise-reduced audio. This is serial message.

nxf56274_4-1615451004815.png

 

 

Figure 5

This is the audio analyzed by the audio analysis software. The upper part is the noise reduced, and the lower part is the original sound. You can see that a lot of noise has been suppressed.

nxf56274_5-1615451012178.png

 

                                                 Figure 6

3 training

Through the above steps, we have achieved audio noise reduction, so how is this data trained? You need to install tensorflow, and keras. How to train is written in the README_CN.md file. This tensorflow needs to install the gpu version, and needs to install cuda. Please check the installation yourself.

3.1 Download the voice data https://github.com/microsoft/MS-SNSD and put it under the MS-SNSD folder

3.2 Use pip3 to install the tools in requirements.txt

3.3 Run ‘noisyspeech_synthesizer.py’ directly, this will report an error. You need to change the 15-line float to int type.

3.4 Run the script gen_dataset.py to generate mfcc and gain

3.5 Run main.py, this will generate the noise file of the sample, which can be used for testing and also generated ‘weights.h,denoise_weights.h,equalizer_coeff.h’

PS: Put the code in directory 'boards/frdmk64f/demo_app'. 'sample.zip' has the noise wave.

Attachments
Version history
Revision #:
3 of 3
Last update:
‎03-11-2021 01:30 AM
Updated by: