Yolov8n timeout on imx95 NPU

[LucM]

Hello, 

I am trying to test if I can run a yolov8n model exported from ultralytics on the IMX95. I saw that the "yolov8n_pcq_640.tflite" is part of the supported models supported models for the Neutron software in the eIQ Toolkit User Guide.

Follwing instructions in the user guide here are my steps to reproduce the error:

1. exported ultralytics model onnx with channel last format
2. quantized onnx model using eIQ Toolkit
   -> python -m onnx2quant yolov8n.onnx -c 'images;yolo_data_640_640_3' -o yolov8n_quant.onnx
   
   
3. converted to tflite using eIQ Toolkit
   -> python -m onnx2tflite yolov8n_quant.onnx -o yolov8n_quant.tflite
   
   
4. converted to target imx95 using MCU_SDK_25.06.00+Linux_6.12.20_2.0.0/neutron-converter
   -> ./neutron-converter --input yolov8n_quant.tflite --target imx95
   With shortened output logs:
   Conversion statistics:
   Number of operators after import = 270
   Number of operators after optimize = 337
   Number of operators converted = 305
   Number of operators NOT converted = 32
   Number of operators after extract = 34
   Number of Neutron graphs = 2
   Number of operators NOT converted = 32
   Operator conversion ratio = 305 / 337 = 0.905044
   
   
   
   
5. finally I run the model on the i.MX95 replacing the example model
   -> /usr/bin/tensorflow-lite-2.19.0/examples# ./label_image -m yolov8n_base_quant_converted.tflite --external_delegate_path=/lib/libneutron_delegate.so -i grace_hopper.bmp -l labels.txt
   With shortened output logs:
   

   INFO: Loaded model yolov8n_quant_converted.tflite
   INFO: resolved reporter
   INFO: EXTERNAL delegate created.
   INFO: NeutronDelegate delegate: 2 nodes delegated out of 34 nodes with 2 partitions.

   INFO: Neutron delegate version: v1.0.0-be8bf399, zerocp enabled.
   INFO: Neutron delegate version: v1.0.0-be8bf399, zerocp enabled.
   INFO: Applied EXTERNAL delegate.
   INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
   Inference poll timeout
   Error: component='Neutron Driver', category='timeout', code=1501
   ERROR: /usr/src/debug/tensorflow-lite-neutron-delegate/2.19.0/neutron_delegate.cc:246 neutronRC != ENONE (384331 != 0)
   ERROR: Node number 34 (NeutronDelegate) failed to invoke.
   ERROR: Failed to invoke tflite!
   
   

What am I missing? Is the original yolov8n model not adequate? The program will probably not run to the end since I replaced a classifier model with a detection one but the invoke should still work and provide inference time. I have attached the successive model outputs of each step.

I was able to run the original example succesfully:

root@imx95-19x19-lpddr5-evk:/usr/bin/tensorflow-lite-2.19.0/examples# ./label_image -m mobilenet_v1_1.0_224_int8_converted.tflite --external_delegate_path=/lib/libneutron_delegate.so -i grace_hopper.bmp -l labels.txt
INFO: Loaded model mobilenet_v1_1.0_224_int8_converted.tflite
INFO: resolved reporter
INFO: EXTERNAL delegate created.
INFO: NeutronDelegate delegate: 1 nodes delegated out of 4 nodes with 1 partitions.

INFO: Neutron delegate version: v1.0.0-be8bf399, zerocp enabled.
INFO: Applied EXTERNAL delegate.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: invoked
INFO: average time: 1.403 ms
INFO: 0.756863: 653 military uniform
INFO: 0.145098: 907 Windsor tie
INFO: 0.0156863: 458 bow tie
INFO: 0.0117647: 466 bulletproof vest
INFO: 0.00784314: 835 suit

Thank you in advance for your help,

Sincerely,

Luc

[LucM]

I was able to fix it by updating the tool kit version to the latest available (eIQ Toolkit 1.17.0.110)

[Bio_TICFSL]

Good thanks for letting us know.

Regards