I am trying to run a Yolov8 model for detection and segmentation on the NPU of the I.mx 93. I have converted the Torch yolov8 model to an 8 int tflite model and then ran the following code:
import tflite_runtime.interpreter as tflite
import cv2
import numpy as np
ext_delegate = tflite.load_delegate('/usr/lib/libethosu_delegate.so')
interpreter = tflite.Interpreter(model_path='yolov8_int8.tflite', experimental_delegates=[ext_delegate]) interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
image = cv2.imread('example_image.JPG')
image = cv2.resize(image, (input_details[0]['shape'][2], input_details[0]['shape'][1]))
image = np.expand_dims(image, axis=0) image = image.astype(np.float32)
interpreter.set_tensor(input_details[0]['index'], image)
interpreter.invoke()
This is my output:
sh-5.2$ sudo python3 minimum.py INFO: Ethosu delegate: device_name set to /dev/ethosu0. INFO: Ethosu delegate: cache_file_path set to . INFO: Ethosu delegate: timeout set to 60000000000. INFO: Ethosu delegate: enable_cycle_counter set to 0. INFO: Ethosu delegate: enable_profiling set to 0. INFO: Ethosu delegate: profiling_buffer_size set to 2048. INFO: Ethosu delegate: pmu_event0 set to 0. INFO: Ethosu delegate: pmu_event1 set to 0. INFO: Ethosu delegate: pmu_event2 set to 0. INFO: Ethosu delegate: pmu_event3 set to 0. INFO: EthosuDelegate: 0 nodes delegated out of 292 nodes with 0 partitions. INFO: Created TensorFlow Lite XNNPACK delegate for CPU. sh-5.2$
The python version used was Python 3.11.5. None of the nodes are delegated to the NPU. Does anyone have any information why this might be the case?
Thank you Brian, it seems like almost all of the model is not convertible. Do you know if there is anyway to get around this? I attached my output below. This is the script I ran to obtain the quantised yolov8 tflite model:
Hi @JesseW,
Thank you for contacting NXP Support.
Please have a look to the i.MX Machine Learning User's Guide section 7.2.2 Ethos-U software architecture. It seems that you are not converting your model to execute properly in the i.MX93 NPU.
i.MX Machine Learning User's Guide (nxp.com)
I hope this information will be helpful.
Have a great day!