Yolov5 tflite deployment on NPU - IMX8MP evk

[abhishek_ml]

We have trained yolov5 with VisDrone dataset in PyTorch. To run the model on IMX8MP-evk, it need to be converted to 8-bit(int8/uint8). Used eIQ Portal to quantize and convert to tflite format. Steps in conversion are as follows: best.pt -> best.onxx -> best.tflite (Input:uint8, model(filter:int8/bias:int32), Output: float32, screenshot attached: tflite-model.JPG). Per channel quantization is used.

Benchmarking: ./benchmark_model --graph=/path2model/best_uint8_int8_float.tflite --external_delegate_path=/usr/lib/libvx_delegate.so --enable_op_profiling=true --max_delegated_partitions=10000 (screenshot attached: benchmark_1.JPG, benchmark_2.JPG, benchmark_3.JPG)

Object Detection:
gstreamer pipeline is used to test above quantized model on webcam. Executing the pipeline on CPU/NPU results in error (screenshot attached: error_pipeline.JPG).

Converting to float32_float32_float32.tflite didn't work as well.
Tensor format should be NCHW for TF-Lite, is the input and output shape correct ?

Need help in creating a working pipeline, C++ code would be best

[abhishek_ml]

Thanks, after changing the input shape it works fine.

How can we find performance on NPU or calculate detection time per frame ? Executing gst-launch with [ GST_DEBUG="GST_TRACER:7" GST_TRACERS="framerate"] indicates FPS of 6 to 9 which is for the complete pipeline not per frame. Is that a correct indication or is there a different way to calculate detection time per frame ?

[sams4]

Did you checked this notes : i.MX8MP NPU debug and fine tune application note - NXP Community .

Let me know for more details on this topic.

Thanks !!!

sams4

[Bio_TICFSL · Accepted solution]

Hello,

The default input model should be filled in with the shape from the original model, take a look at the eIQ link for more information on quantization.

https://community.nxp.com/t5/eIQ-Machine-Learning-Software/tkb-p/eiq@tkb

Regards