Running Yolov8 on I.mx 93

JesseW — Sun, 24 Mar 2024 22:08:55 GMT

I am trying to run a Yolov8 model for detection and segmentation on the NPU of the I.mx 93. I have converted the Torch yolov8 model to an 8 int tflite model and then ran the following code:

import tflite_runtime.interpreter as tflite
import cv2
import numpy as np

ext_delegate = tflite.load_delegate('/usr/lib/libethosu_delegate.so')
interpreter = tflite.Interpreter(model_path='yolov8_int8.tflite', experimental_delegates=[ext_delegate]) interpreter.allocate_tensors()

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
image = cv2.imread('example_image.JPG')
image = cv2.resize(image, (input_details[0]['shape'][2], input_details[0]['shape'][1]))
image = np.expand_dims(image, axis=0) image = image.astype(np.float32)

interpreter.set_tensor(input_details[0]['index'], image)
interpreter.invoke()

This is my output:

sh-5.2$ sudo python3 minimum.py INFO: Ethosu delegate: device_name set to /dev/ethosu0. INFO: Ethosu delegate: cache_file_path set to . INFO: Ethosu delegate: timeout set to 60000000000. INFO: Ethosu delegate: enable_cycle_counter set to 0. INFO: Ethosu delegate: enable_profiling set to 0. INFO: Ethosu delegate: profiling_buffer_size set to 2048. INFO: Ethosu delegate: pmu_event0 set to 0. INFO: Ethosu delegate: pmu_event1 set to 0. INFO: Ethosu delegate: pmu_event2 set to 0. INFO: Ethosu delegate: pmu_event3 set to 0. INFO: EthosuDelegate: 0 nodes delegated out of 292 nodes with 0 partitions. INFO: Created TensorFlow Lite XNNPACK delegate for CPU. sh-5.2$

The python version used was Python 3.11.5. None of the nodes are delegated to the NPU. Does anyone have any information why this might be the case?

Re: Running Yolov8 on I.mx 93

brian14 — Wed, 27 Mar 2024 18:23:48 GMT

Hi @JesseW,

Thank you for contacting NXP Support.

Please have a look to the i.MX Machine Learning User's Guide section 7.2.2 Ethos-U software architecture. It seems that you are not converting your model to execute properly in the i.MX93 NPU.

i.MX Machine Learning User's Guide (nxp.com)

I hope this information will be helpful.

Have a great day!

Re: Running Yolov8 on I.mx 93

JesseW — Thu, 28 Mar 2024 13:11:06 GMT

Thank you Brian, it seems like almost all of the model is not convertible. Do you know if there is anyway to get around this? I attached my output below. This is the script I ran to obtain the quantised yolov8 tflite model:

model = YOLO('best.pt')

model.export(format='tflite',int8=True)

topic Re: Running Yolov8 on I.mx 93 in 8-bit Microcontrollers

Running Yolov8 on I.mx 93

Re: Running Yolov8 on I.mx 93

Re: Running Yolov8 on I.mx 93