MCX N947 TFLM Modelrunner Inference of Neutron Converted Models

[Laurens26]

I want to benchmark a bunch of tflite models (<500KB in size) on my FRDM MCXN947 (CPU vs. NPU) via the TFLM modelrunner using HTTP.

The problem: CPU inference does not work for all models and NPU inference not at all.

hardware:
- FRDM-MCXN947 development board (connected to my windows machine via USB and Ethernet)

software and tool setup:
- host: windows 11 machine
- eIQ Neutron SDK: eiq-neutron-sdk-windows-3.0.1
- MCUXpresso SDK: SDK_26_03_00_FRDM-MCXN947
- Project from SDK: frdmmcxn947_tflm_modelrunner_cm33_core0

ML models for CPU inference:
- ad01_int8.tflite
- kws_ref_model.tflite
- pretrainedResnet_quant.tflite
- vww_96_int8.tflite

ML models neutron converted with Neutron Converter (part of eIQ Neutron SDK) for NPU inference:
eiq-neutron-sdk-windows-3.0.1\bin
neutron-converter --target mcxn94x --input ad01_int8.tflite --output ad01_int8_converted_3_0_1.tflite

- ad01_int8_converted_3_0_1.tflite
- kws_ref_model_converted_3_0_1.tflite
- pretrainedResnet_quant_converted_3_0_1.tflite
- vww_96_int8_converted_3_0_1.tflite

Steps performed
1. Update eIQ Projects with the Latest eIQ Neutron SDK Libraries
Copy from:
eiq-neutron-sdk-windows-3.0.1\target\mcxn94x\common\include\NeutronErrors.h
eiq-neutron-sdk-windows-3.0.1\target\mcxn94x\driver\include\NeutronDriver.h
eiq-neutron-sdk-windows-3.0.1\target\mcxn94x\mcxn\libNeutronDriver.a
eiq-neutron-sdk-windows-3.0.1\target\mcxn94x\mcxn\libNeutronFirmware.a

to:
frdmmcxn947_tflm_modelrunner_cm33_core0\include\NeutronErrors.h
frdmmcxn947_tflm_modelrunner_cm33_core0\driver_include\NeutronDriver.h
frdmmcxn947_tflm_modelrunner_cm33_core0\mcxn\libNeutronDriver.a
frdmmcxn947_tflm_modelrunner_cm33_core0\mcxn\libNeutronFirmware.a

2. Build and flash the project "frdmmcxn947_tflm_modelrunner_cm33_core0" with vscode

3. Run inference via HTTP using the flashed TFLM Modelrunner
Command:
curl -X PUT http://192.168.178.75:10818/v1 -F "block_content=@C:\<mypath>\ad01_int8.tflite"

Output via Serial Monitor:
....
run ms: 6.167000

Command:
curl -X PUT http://192.168.178.75:10818/v1 -F "block_content=@C:\<mypath>\vww_96_int8.tflite"

Output via Serial Monitor:
....
run ms: 182.042000

For any of the other models it doesnt work

Output: 
a) No inference after flashing:
FLASH 0x104000 Erased, Program at 104000: 16384 bytes ==
FLASH 0x108000 Erased, Program at 108000: 16384 bytes ==
FLASH 0x10c000 Erased, Program at 10c000: 16384 bytes ==
FLASH 0x110000 Erased, Program at 110000: 16384 bytes ==
FLASH 0x114000 Erased, Program at 114000: 16384 bytes ==
FLASH 0x118000 Erased, Program at 118000: 16384 bytes ==
FLASH 0x11c000 Erased, Program at 11c000: 16384 bytes ==
FLASH 0x120000 Erased, Program at 120000: 16384 bytes ==
FLASH 0x124000 Erased, Program at 124000: 16384 bytes ==
FLASH 0x128000 Erased, Program at 128000: 16384 bytes ==
FLASH 0x12c000 Erased, Program at 12c000: 16384 bytes ==
FLASH 0x130000 Erased, Program at 130000: 16384 bytes ==
FLASH 0x134000 Erased, Program at 134000: 16384 bytes ==
FLASH 0x138000 Erased, Program at 138000: 544 bytes ==

b) no flashing process at all

c) Error HardFault_Handler

Line 343: ldr r0,=HardFault_Handler
in 
SDK_26_03_00_FRDM-MCXN947\mcuxsdk\devices\MCX\MCXN\MCXN947\gcc\startup_MCXN947_cm33_core0.S
stopped by debugger


Any hints are highly appreciated!

[Harry_Zhang]

Hi @Laurens26 

I think the problem is more likely a mix of version alignment, rather than a pure “model size” issue.

I think you can refer to this link.

How to Update eIQ Projects with the Latest eIQ Neutron SDK Libraries

BR

Harry