We already have application note AN13336 about running TFLM micro_speech example on Tensilica HiFi4 DSP on i.MX 8M Plus EVK.
In this document, TFLM micro_speech example will be upgraded to support i.MX DSP remote processors driver in newer Linux kernel, patch to support i.MX8QXP/i.MX8QM MEK will be provided, steps about how to download and install toolchain, how to build and run the example will be listed.
Hardware: i.MX8MP EVK, i.MX8QXP MEK, i.MX8QM MEK.
Software: Linux 5.15.52, TensorFlow Lite Micro
Login https://tensilicatools.com/download/i-mx-8-hifi-4-sdk/, download GETTING STARTED GUIDE, then follow the GUIDE to get a license key for i.MX8MP SDK.
There are two configuration packages:
DSP_configuration_for_Linux_(NEWLIB)_i.MX8MP_RI_2020_4.zip
hifi4_mscale_mpxclib_prod_linux.tgz
The target binary built with xclib is smaller than the one built with newlib.
In this example, we will use NEWLIB package DSP_configuration_for_Linux_(NEWLIB)_i.MX8MP_RI_2020_4.zip
After installed, Xtensa tool package can be found at installation folder:
xtensa/XtDevTools/downloads/RI-2020.4/tools/XtensaTools_RI_2020_4_linux.tgz
mkdir -p ./imx-audio-toolchain/Xtensa_Tool/tools
mkdir -p ./imx-audio-toolchain/Xtensa_Tool/builds
cd imx-audio-toolchain/Xtensa_Tool
tar zxf XtensaTools_RI_2020_4_linux.tgz -C tools/
Unzip DSP_configuration_for_Linux_(NEWLIB)_i.MX8MP_RI_2020_4.zip, you will get hifi4_mscale_v1_0_2_prod_linux.tgz
cd imx-audio-toolchain/Xtensa_Tool/
tar zxf hifi4_mscale_v1_0_2_prod_linux.tgz -C builds/
cd imx-audio-toolchain/Xtensa_Tool
./builds/RI-2020.4-linux/hifi4_mscale_v1_0_2_prod/install --xtensa-tools ./tools/RI-2020.4-linux/XtensaTools --registry ./tools/RI-2020.4-linux/XtensaTools/config
Create ~/.flexlmrc, in the file, set LM_LICENSE_FILE environment variable to the absolute path of your license key folder.
LM_LICENSE_FILE=<absolute path of your license key folder>
#!/bin/bash
export TOOLCHAIN_FOLDER=RI-2020.4-linux
export TOOLCHAIN_PATH=$PWD
export PATH=$TOOLCHAIN_PATH/Xtensa_Tool/tools/$TOOLCHAIN_FOLDER/XtensaTools/bin:$PATH
export XTENSA_CORE=hifi4_mscale_v1_0_2_prod
export XTENSA_BASE=$PWD/Xtensa_Tool/
export XTENSA_TOOLS_VERSION=RI-2020.4-linux
export XTENSA_SYSTEM=$PWD/Xtensa_Tool/tools/RI-2020.4-linux/XtensaTools/config
export PLATF=imx8m
git clone https://github.com/pnikam-cad/tensorflow.git
cd tensorflow
git checkout hifi5_hifi4_nnlib_new_versions
0001-i.MX-8M-Plus-HiFi4-TFLM-Enablement-changes-required-.patch
0002-resolve-download-error-and-upgrade-toolchain-to-2022.patch
0003-upgrade-to-support-remoteproc-in-linux-5.10.72.patch
In imx-audio-toolchain, run setup script setup_env.sh
In tensorflow folder, build micro_speech_mock demo with command
make -f tensorflow/lite/micro/tools/make/Makefile TARGET=xtensa_hifi TARGET_ARCH=hifi4 XTENSA_USE_LIBC=true micro_speech_mock
The compiled binary is tensorflow/lite/micro/tools/make/gen/xtensa_hifi_hifi4/bin/micro_speech_mock
Boot i.MX8MP with imx8mp-evk-dsp.dtb, copy micro_speech_mock to /lib/firmware/imx/dsp/, rename it to hifi4.bin, then load and run the example with command
echo start >/sys/class/remoteproc/remoteproc0/state
On UART4 output(the fourth terminal in J23 DEBUG port), you can get such log
If you want to use new sample data, please refer to AN13336 about how to generate and convert.
Login https://tensilicatools.com/download/i-mx8-quadxplus-hifi-4-sdk/, download GETTING STARTED GUIDE, then follow the GUIDE to get a license key for i.MX8QXP SDK.
There are two configuration packages:
hifi4_nxp_v4_3_1_prod_linux.tgz
hifi4_nxp_v4_3_1_prod_XC_linux_redist.tgz
In this example, we will use XCLIB package: hifi4_nxp_v4_3_1_prod_XC_linux_redist.tgz
After installed, Xtensa tool package can be found at installation folder:
xtensa/XtDevTools/downloads/RI-2020.4/tools/XtensaTools_RI_2020_4_linux.tgz
mkdir -p ./imx-audio-toolchain/Xtensa_Tool/tools
mkdir -p ./imx-audio-toolchain/Xtensa_Tool/builds
cd imx-audio-toolchain/Xtensa_Tool
tar zxf XtensaTools_RI_2020_4_linux.tgz -C tools/
cd imx-audio-toolchain/Xtensa_Tool/
tar zxf hifi4_nxp_v4_3_1_prod_XC_linux_redist.tgz -C builds/
cd imx-audio-toolchain/Xtensa_Tool
./builds/RI-2020.4-linux/hifi4_nxp_v4_3_1_prod_XC/install --xtensa-tools ./tools/RI-2020.4-linux/XtensaTools --registry ./tools/RI-2020.4-linux/XtensaTools/config
Create ~/.flexlmrc, in the file, set LM_LICENSE_FILE environment variable to the absolute path of your license key folder.
LM_LICENSE_FILE=<absolute path of your license key folder>
#!/bin/bash
export TOOLCHAIN_FOLDER=RI-2020.4-linux
export TOOLCHAIN_PATH=$PWD
export PATH=$TOOLCHAIN_PATH/Xtensa_Tool/tools/$TOOLCHAIN_FOLDER/XtensaTools/bin:$PATH
export XTENSA_CORE=hifi4_nxp_v4_3_1_prod_XC
export XTENSA_BASE=$PWD/Xtensa_Tool/
export XTENSA_TOOLS_VERSION=RI-2020.4-linux
export XTENSA_SYSTEM=$PWD/Xtensa_Tool/tools/RI-2020.4-linux/XtensaTools/config
export PLATF=imx8
git clone https://github.com/pnikam-cad/tensorflow.git
cd tensorflow
git checkout hifi5_hifi4_nnlib_new_versions
0001-i.MX-8M-Plus-HiFi4-TFLM-Enablement-changes-required-.patch
0002-resolve-download-error-and-upgrade-toolchain-to-2022.patch
0003-upgrade-to-support-remoteproc-in-linux-5.10.72.patch
In imx-audio-toolchain, run setup script setup_env.sh
In tensorflow folder, build micro_speech_mock demo with command
make -f tensorflow/lite/micro/tools/make/Makefile TARGET=xtensa_hifi TARGET_ARCH=hifi4 micro_speech_mock
The compiled binary is tensorflow/lite/micro/tools/make/gen/xtensa_hifi_hifi4/bin/micro_speech_mock
0001-linux-enable-uart2-for-8qxp-hifi4.patch
Boot i.MX8QXP MEK board, copy micro_speech_mock to /lib/firmware/imx/dsp/, rename it to hifi4.bin, then load and run the example with command
echo start >/sys/class/remoteproc/remoteproc1/state
On base board RS232 port J37, you can get such log
Follow above step 4 ~ step 6.3 to download and setup toolchain, apply TFLM patch and compile Micro Speech example.
In above i.MX8QXP example, micro speech example print log through UART2.
In i.MX8QM MEK Linux demo image imx-image-full-imx8qmmek.wic, M4 is enabled and UART2 is occupied by M4.
To resolve this conflict, we will not enable M4. The steps to run Linux image without M4 are shown below.
1. Apply patch 0001-linux-enable-uart2-for-8qm-hifi4.patch to enable UART2 for HiFi4
2. Build Linux dtb, we will boot i.MX8QM MEK board with imx8qm-mek.dtb.
3. Assume you boot 8QM from SD card, program uboot image without M4 to SD card
sudo dd if=imx-boot-imx8qmmek-sd.bin-flash of=/dev/sdX
Boot i.MX8QM MEK board with your new imx8qm-mek.dtb, copy micro_speech_mock to /lib/firmware/imx/dsp/, rename it to hifi4.bin, then load and run the example with command
echo start >/sys/class/remoteproc/remoteproc2/state
On base board RS232 port J37, you can get such log
Run lsmod, check if imx_dsp_rproc module is loaded. If not, rebuild Linux kernel and modular and update to your board.
Check if you have applied Linux patch to enable UART2 for HiFi4, and bootup your board with the dtb file.
https://www.nxp.com/webapp/Download?colCode=AN13336
XPLOR8QM8QP-UG_Rev0.pdf in https://tensilicatools.com/download/i-mx8-quadxplus-hifi-4-sdk/
ここにコメントを追加するには、ご登録いただく必要があります。 ご登録済みの場合は、ログインしてください。 ご登録がまだの場合は、ご登録後にログインしてください。