I want to benchmark a bunch of tflite models (<500KB in size) on my FRDM MCXN947 (CPU vs. NPU) via the TFLM modelrunner using HTTP.
The problem: CPU inference does not work for all models and NPU inference not at all.
hardware:
- FRDM-MCXN947 development board (connected to my windows machine via USB and Ethernet)
software and tool setup:
- host: windows 11 machine
- eIQ Neutron SDK: eiq-neutron-sdk-windows-3.0.1
- MCUXpresso SDK: SDK_26_03_00_FRDM-MCXN947
- Project from SDK: frdmmcxn947_tflm_modelrunner_cm33_core0
ML models for CPU inference:
- ad01_int8.tflite
- kws_ref_model.tflite
- pretrainedResnet_quant.tflite
- vww_96_int8.tflite
ML models neutron converted with Neutron Converter (part of eIQ Neutron SDK) for NPU inference:
eiq-neutron-sdk-windows-3.0.1\bin
neutron-converter --target mcxn94x --input ad01_int8.tflite --output ad01_int8_converted_3_0_1.tflite
- ad01_int8_converted_3_0_1.tflite
- kws_ref_model_converted_3_0_1.tflite
- pretrainedResnet_quant_converted_3_0_1.tflite
- vww_96_int8_converted_3_0_1.tflite
Steps performed
1. Update eIQ Projects with the Latest eIQ Neutron SDK Libraries
Copy from:
eiq-neutron-sdk-windows-3.0.1\target\mcxn94x\common\include\NeutronErrors.h
eiq-neutron-sdk-windows-3.0.1\target\mcxn94x\driver\include\NeutronDriver.h
eiq-neutron-sdk-windows-3.0.1\target\mcxn94x\mcxn\libNeutronDriver.a
eiq-neutron-sdk-windows-3.0.1\target\mcxn94x\mcxn\libNeutronFirmware.a
to:
frdmmcxn947_tflm_modelrunner_cm33_core0\include\NeutronErrors.h
frdmmcxn947_tflm_modelrunner_cm33_core0\driver_include\NeutronDriver.h
frdmmcxn947_tflm_modelrunner_cm33_core0\mcxn\libNeutronDriver.a
frdmmcxn947_tflm_modelrunner_cm33_core0\mcxn\libNeutronFirmware.a
2. Build and flash the project "frdmmcxn947_tflm_modelrunner_cm33_core0" with vscode
3. Run inference via HTTP using the flashed TFLM Modelrunner
Command:
curl -X PUT http://192.168.178.75:10818/v1 -F "block_content=@C:\<mypath>\ad01_int8.tflite"
Output via Serial Monitor:
....
run ms: 6.167000
Command:
curl -X PUT http://192.168.178.75:10818/v1 -F "block_content=@C:\<mypath>\vww_96_int8.tflite"
Output via Serial Monitor:
....
run ms: 182.042000
For any of the other models it doesnt work
Output:
a) No inference after flashing:
FLASH 0x104000 Erased, Program at 104000: 16384 bytes ==
FLASH 0x108000 Erased, Program at 108000: 16384 bytes ==
FLASH 0x10c000 Erased, Program at 10c000: 16384 bytes ==
FLASH 0x110000 Erased, Program at 110000: 16384 bytes ==
FLASH 0x114000 Erased, Program at 114000: 16384 bytes ==
FLASH 0x118000 Erased, Program at 118000: 16384 bytes ==
FLASH 0x11c000 Erased, Program at 11c000: 16384 bytes ==
FLASH 0x120000 Erased, Program at 120000: 16384 bytes ==
FLASH 0x124000 Erased, Program at 124000: 16384 bytes ==
FLASH 0x128000 Erased, Program at 128000: 16384 bytes ==
FLASH 0x12c000 Erased, Program at 12c000: 16384 bytes ==
FLASH 0x130000 Erased, Program at 130000: 16384 bytes ==
FLASH 0x134000 Erased, Program at 134000: 16384 bytes ==
FLASH 0x138000 Erased, Program at 138000: 544 bytes ==
b) no flashing process at all
c) Error HardFault_Handler
Line 343: ldr r0,=HardFault_Handler
in
SDK_26_03_00_FRDM-MCXN947\mcuxsdk\devices\MCX\MCXN\MCXN947\gcc\startup_MCXN947_cm33_core0.S
stopped by debugger
Any hints are highly appreciated!
