yolov11n,yolov8n,yolov5nu model not getting any output after running on i.MX95 NPU [i.MX95 NPU] YOLOv5n/v8n/v11n Neutron-converted Models Run but Return No Detections (Zero Output) Issue Description I am evaluating YOLO object detection models on the i.MX95 NPU using the Neutron converter. While the INT8 quantized TFLite models run successfully and detect objects on the Cortex-A55 CPU, the compiled neutron.tflite versions yield zero detections (empty/no output) when offloaded to the NPU, despite executing inference without crashing. Environment & Hardware Setup Hardware: i.MX95 19x19 LPDDR5 EVK (A1 Revision) OS/Kernel: Linux 6.12.34-lts-next-gbe78e49cb433 #1 SMP PREEMPT (aarch64) NXP Toolchain: MCU-SDK v25.09.00 + Linux 6.12.34_2.1.0 Models Tested: YOLOv5nu, YOLOv8n, YOLOv11n (Ultralytics) Workflow Steps & Commands Used 1. Quantization (Ultralytics Export) Models were exported to INT8 full integer quantization with a 320x320 resolution: yolo export model=yolov8n.pt format=tflite int8=True imgsz=320# (Repeated identically for yolov11n.pt and yolov5nu.pt)
Status: Works perfectly on CPU. yolovXn_full_integer_quant.tflite detects objects correctly on the A55 cores. 2. Neutron Compilation The TFLite models were compiled for the i.MX95 NPU using the Neutron converter from MCU_SDK_25.09.00+Linux_6.12.34_2.1.0: ./neutron-converter --input yolov8n_full_integer_quant.tflite --target imx95 --output yolov8n_full_integer_quant_neutron.tflite Status: Fails to detect objects on NPU. The compiled model loads and runs inference without throwing syntax or execution errors, but output tensors return zero detections for the exact same test images. Observed Symptoms & Suspected Root Causes Operator Fallbacks: Did the converter fall back to CPU for specific YOLO layers (like custom Anchors, SiLU/Swin activations, or Non-Max Suppression)? Quantization Scaling/Asymmetry: YOLO models exported via Ultralytics often use asymmetric quantization or have specific output tensor scaling that the Neutron NPU driver might misinterpret. Output Tensor Formatting: The inference runs, which suggests the input pipeline is fine, but the output bounding boxes/scores are either blank or completely garbage values. Questions for NXP Experts Are there known limitations or mandatory optimization flags needed in the neutron-converter specifically for Ultralytics YOLO architectures? Should the NMS (Non-Max Suppression) layer be stripped out before passing the TFLite model to the Neutron converter? Does the i.MX95 Neutron SDK require symmetric quantization (per_channel=True or False) to parse the output layers properly? Any guidance, reference scripts, or working YOLO deployment notes for the i.MX95 NPU would be highly appreciated. Re: yolov11n,yolov8n,yolov5nu model not getting any output after running on i.MX95 NPU Hi Alejandro, Thank you for your response. I believe there may be a misunderstanding regarding my hardware platform. My issue is not related to the i.MX91. I am using the following platform: Board: i.MX95 19x19 LPDDR5 EVK (IMX95LPD5EVK-19CM, A1 Revision) Board Quick Start Guide: https://www.nxp.com/docs/en/quick-reference-guide/IMX95LPD5EVK-19CM.pdf Neutron SDK: MCU_SDK_25.09.00+Linux_6.12.34_2.1.0 Kernel: Linux 6.12.34-lts-next-gbe78e49cb433 #1 SMP PREEMPT (aarch64) The guide you shared appears to be for the i.MX91, whereas my question is specifically about YOLO deployment on the i.MX95 Neutron NPU. The original INT8 TFLite models (YOLOv5nu, YOLOv8n, and YOLOv11n) run correctly on the Cortex-A55 CPU and produce valid detections. However, after compiling the same models using the Neutron converter included in MCU_SDK_25.09.00+Linux_6.12.34_2.1.0, inference executes successfully on the NPU without any runtime errors, but the output tensors contain no valid detections. For easier investigation, I have already attached the following files to my original post: * Original INT8 quantized TFLite models. * Neutron-converted TFLite models for YOLOv8n and YOLOv11n. * A Python inference script that can be used to reproduce the issue. Since these are the original pretrained Ultralytics models converted to TFLite, you can use the standard COCO class names directly with the provided script. It should allow you to reproduce the behavior on your i.MX95 platform without requiring any additional modifications. I would appreciate it if you could reproduce the issue using the attached files and let me know whether this is a known limitation or issue with the current Neutron SDK for the i.MX95. Thank you. Re: yolov11n,yolov8n,yolov5nu model not getting any output after running on i.MX95 NPU Hi @vijayranaACL, Thank you for contacting NXP Support. Please refer to this guide. Since you are using the i.MX91 A1 silicon revision, it is possible that some features or functionality may not operate correctly, as A1 is an early silicon revision intended primarily for evaluation and development purposes. For this reason, we recommend using the i.MX91 B0 silicon revision for your testing and validation activities. The guide was developed and validated using the B0 silicon version, so the documented behavior and results are based on that revision. If possible, please confirm which silicon revision you are using and whether you have access to a B0 device for comparison. Best regards, Alejandro Garcia Re: yolov11n,yolov8n,yolov5nu model not getting any output after running on i.MX95 NPU Hi Alejandro, Thank you for the clarification regarding i.MX 95 (not i.MX 91) and for the recommendation to use --optimization-level OOpt. We tried to follow your suggested command on our current setup, but --optimization-level is not available in the neutron-converter paired with our board BSP. Our current environment Component Version Board IMX95LPD5EVK-19 BSP LF6.12.34_2.1.0 (Linux 6.12.34-lts-next) Neutron delegate on board v1.0.0-be8bf399 Host converter eIQ Toolkit 1.17 → neutron-converter 2.1.3+0Xaf140cf5 Converter BSP tag MCU_SDK_25.09.00+Linux_6.12.34_2.1.0 On this converter, neutron-converter --help does not list --optimization-level. Command we actually run neutron-converter \ --input yolov8n_full_integer_quant.tflite \ --output yolov8n_neutron.tflite \ --target imx95 For debugging we also use: neutron-converter \ --input yolov8n_full_integer_quant.tflite \ --output yolov8n_neutron.tflite \ --target imx95 \ --verbose Flags available on our converter (2.1.3) Key options from --help: --input, --output, --target --merge-neutron-graphs --convert-inputs-uint8-to-int8, --convert-outputs-uint8-to-int8 --dump-statistics, --dump-graphs, --verbose --include-between-input-tensors, --exclude-between-input-tensors --show-targets, --show-kernel-kinds --optimization-level is not present on this build. Test results so far Model NPU behaviour YOLOv8n Neutron (our conversion) Invoke OK, but 0 detections YOLOv8n headless backbone NPU output constant (~1.13) Headless CPU backbone + CPU head Detections OK — pipeline logic is correct Questions Is --optimization-level OOpt only supported in a newer neutron-converter than 2.1.3? What is the recommended conversion command for LF6.12.34 / imx95 when OOpt is not available? Re: yolov11n,yolov8n,yolov5nu model not getting any output after running on i.MX95 NPU HI @vijayranaACL, Sorry, that was a typo on my side.
I was referring to the i.MX95, since the i.MX91 does not include an NPU. I recommend trying the following command to convert the model:
.\neutron-converter.exe `
--input " .tflite" `
--target imx95 `
--output " .tflite" `
--optimization-level OOpt
According to the Neutron SDK documentation, it is important to note that the converter is not deterministic for Neutron-S targets such as the i.MX95. The conversion process relies on multithreaded constrained programming solvers, which means that different executions of the converter on the same model may produce slightly different results, particularly regarding TCM memory allocation and generated microcode. Because multiple optimal solutions may exist, different solver threads can converge on different valid solutions during each conversion. Although these solutions may differ internally, they are all considered correct and optimized by the converter. In most cases, these differences should not significantly affect functionality or performance. If you observe variations in behavior, performance, or accuracy, I recommend converting the model multiple times and comparing the results. There are methods available to force deterministic behavior, but they typically increase conversion time considerably and are generally not recommended unless strictly required. Please let me know the outcome of your tests. Best regards, Alejandro Garcia Re: yolov11n,yolov8n,yolov5nu model not getting any output after running on i.MX95 NPU Hi @vijayranaACL, I have tested your code on an i.MX95 EVK with B0 silicon, and it appears to be working correctly on our side without any issues. Using your person_detect.py application, the model loads successfully, the Neutron delegate is initialized correctly, and the application performs inference as expected. During testing, I observed stable object detection and a sustained performance of approximately 13–14 FPS. The logs also confirm that the Neutron delegate is active and that the model is executing properly with NPU acceleration. Based on these results, I recommend moving to the B0 silicon revision. The A0 and A1 silicon versions were released primarily for evaluation and beta testing purposes, and they do not have the same level of software support and validation as B0. Several features and fixes were introduced after the early revisions, which may explain the behavior you are seeing. The relevant portion of my test log is shown below:
root@imx95evk:~# python3 person_detect.py
Opening camera /dev/video52 ...
Trying camera backend: V4L2 /dev/video52
Camera opened via V4L2 /dev/video52
Loading model and NPU delegate ...
Loaded Neutron delegate: /usr/lib/libneutron_delegate.so
/usr/lib/python3.13/site-packages/tflite_runtime/interpreter.py:457: UserWarning: Warning: tf.lite.Interpreter is deprecated and is scheduled for deletion in
TF 2.20. Please use the LiteRT interpreter from the ai_edge_litert package.
See the [migration guide](https://ai.google.dev/edge/litert/migration)
for details.
warnings.warn(_INTERPRETER_DELETION_WARNING)
INFO: NeutronDelegate delegate: 1 nodes delegated out of 33 nodes with 1 partitions.
INFO: Neutron delegate version: v1.0.0-d98743a7, zerocp enabled.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Model input: shape=[ 1 640 640 3] dtype= quant=(0.003921568859368563, -128)
Model output[0]: shape=[ 1 84 8400] dtype= quant=(0.003906319383531809, -128)
Using model input size: 640x640
Re-opening camera after model load ...
Trying camera backend: V4L2 /dev/video52
Camera opened via V4L2 /dev/video52
Person detection running. Press Ctrl+C to stop.
First frame: 640x480
Output tensor shape: (1, 84, 8400)
frame=22 person conf=0.61 box=[143,8,496,476] fps=10.7
--- fps=11.5 detections=0 ---
frame=39 person conf=0.58 box=[138,16,496,473] fps=12.1
frame=50 person conf=0.50 box=[138,12,496,472] fps=12.5
frame=60 person conf=0.58 box=[138,13,496,476] fps=12.8
--- fps=12.8 detections=1 ---
frame=84 person conf=0.61 box=[143,10,496,475] fps=13.3
frame=85 person conf=0.54 box=[138,10,496,475] fps=13.3
frame=86 person conf=0.61 box=[138,13,496,476] fps=13.3
frame=88 person conf=0.54 box=[138,12,496,472] fps=13.3
--- fps=13.3 detections=0 ---
--- fps=13.6 detections=0 ---
frame=136 person conf=0.65 box=[138,13,496,476] fps=13.7
frame=139 person conf=0.58 box=[138,12,496,472] fps=13.7
frame=140 person conf=0.61 box=[136,10,498,475] fps=13.8
frame=141 person conf=0.61 box=[138,13,496,476] fps=13.8
frame=145 person conf=0.50 box=[138,13,496,476] fps=13.8
frame=146 person conf=0.65 box=[138,13,496,476] fps=13.8
frame=148 person conf=0.54 box=[138,16,496,473] fps=13.8
frame=150 person conf=0.50 box=[131,12,498,472] fps=13.8
--- fps=13.8 detections=1 ---
frame=155 person conf=0.50 box=[152,32,497,472] fps=13.8
frame=156 person conf=0.71 box=[180,37,495,422] fps=13.8
frame=157 person conf=0.54 box=[182,38,497,411] fps=13.8
frame=158 person conf=0.65 box=[156,37,493,472] fps=13.8
frame=159 person conf=0.68 box=[158,42,496,472] fps=13.8
frame=160 person conf=0.61 box=[155,46,495,468] fps=13.8
frame=162 person conf=0.54 box=[156,46,493,463] fps=13.8
frame=163 person conf=0.71 box=[171,43,493,466] fps=13.8
frame=164 person conf=0.54 box=[186,41,363,353] fps=13.8
frame=165 person conf=0.61 box=[187,42,492,452] fps=13.8
frame=166 person conf=0.58 box=[190,41,495,393] fps=13.8
frame=167 person conf=0.61 box=[190,42,495,432] fps=13.8
frame=168 person conf=0.61 box=[195,38,495,426] fps=13.8
frame=169 person conf=0.50 box=[190,36,495,423] fps=13.8
frame=170 person conf=0.58 box=[198,37,496,397] fps=13.8
--- fps=13.9 detections=0 ---
frame=202 person conf=0.50 box=[145,28,495,456] fps=13.9
--- fps=14.0 detections=0 ---
^CStopped.
root@imx95evk:~#
Since the same application and model work correctly on B0 silicon, my recommendation is to repeat the test using a B0 device before continuing with further debugging, as the issue may be related to the silicon revision rather than the application itself.
Best Regards, Chavira Re: yolov11n,yolov8n,yolov5nu model not getting any output after running on i.MX95 NPU Hi Alejandro, Could you please confirm that the code and models we discussed are compatible with the FRDM-IMX95 15x15 platform? vijayranaACL_1-1784537726170.png with bsp 6.12.49_2.2.0 vijayranaACL_2-1784537799992.png
記事全体を表示
