Hi there,
I have exported full quantized int8 YOLOv8n Object Detection model from ultralytics. I have converted it to run on NPU using neutron converter from latest eIQ Toolkit version 1.17 and I have tried to execute it on i.MX95 hardware.
I have tried both converted and non-converted model with NPU delegate but seems like only the Neutron Graph which is present on converted model is going to execute on NPU.
When I compare the raw outputs from both model, converted model is giving multiple false positives with more than 95% score. The same script I am using for both converted and non-converted model but having issue with converted model only. I have validated with multiple approaches, getting same everytime.
When I have inspect both the models with netron app, I have found major architectural changes in converted model.
Here, some points I want to ask is,
1. Is the lastest object detection architectures like YOLOv8 and YOLOv11 is supported on Neutron Converter with eIQ Toolkit version 1.17? If yes then what are steps you are following please let me know.
2. Do you have tested the YOLOv8 and YOLOv11 with NPU on i.MX95? If yes then can you please send the model for verification and Post-Processing steps as well.
3. If we want to execute the operations apart from Neutron Graph on Neutron NPU in i.MX95 then what is the process?
4. If we want to execute the models mentioed above on GPU in i.MX95 then what is the process?
I have gone through the Machine Learning User Guide as well but haven't found related details. If you want any details from my side to debug this then feel free to ask.
Thanks,
Vatsal
Hello,
The issue you're experiencing with false positives from the Neutron-converted YOLOv8n model is a known challenge. Current support for YOLOv8 models on the i.MX95 NPU is still being optimized, and architectural changes during conversion can affect performance.
Regarding your specific questions:
1. YOLOv8 architecture support in eIQ Toolkit 1.17 is still being improved. The recommended conversion workflow is:
- Ensure proper int8 quantization when exporting your model
- Use the latest eIQ Toolkit (v1.17) for conversion with neutron-converter
- Command: `./neutron-converter --input [your-model].tflite --target imx95 --use-python-prototype`
2. While YOLOv8 is supported, it may not be fully optimized yet. YOLOv5 has shown better compatibility with current NPU implementations. The NXP team is actively working on improving YOLOv8/YOLOv11 support.
3. To execute operations beyond Neutron Graph on the NPU, you need to analyze your model with the eIQ or netron tool to identify which operators are successfully converted to run on the NPU (shown as neutronop content). Operators not converted will run on CPU.
4. For GPU execution on i.MX95, you would use the TensorFlow Lite GPU delegate instead of the NPU delegate. This requires modifying your inference code to use the GPU delegate.
You can verify model operation distribution by examining the converted model in netron - any operators with "neutronop" content are executed on NPU, while others remain on CPU.
Regards
We have performed the training again with YOLOv5s with our custom dataset and try to do inference as you were claiming that YOLOv5 is giving good results with neutron converter in previous chat. The issue is still there with YOLOv5 as well. I am attaching the results as well for the reference. Where we tried with converted and non-converted int8 tflite YOLOv5s model.
Now, If you are claiming that YOLOv5 is giving good results then why don't you share the validated model with us. Please share all steps you are following from export, quantization to convert and what are the steps you are following for post-processing as well. So that we can validate your model and replicate the steps you have followed at our end.
Please share you benchmarking and profiling data with i.MX95 as well if you can share.
Hello @Bio_TICFSL!
I've noticed NXP team suggest to use the --use-python-prototype flag, however my neutron converter does not recognize it. Is it supported on Windows, or it is only compatible with the Linux eIQ release?
That is also a question for me. I have done with Linux eIQ and it's not working me as well. So I have skipped it and try to convert the model with passing the target arg only.
Hi @Bio_TICFSL ,
Since it's long time, I am still waiting for the reply from your side. We are stuck here, and we have some urgency as well. Kindly request you to please reply as soon as possible.
Regards,
Vatsal
Issue Description
While the INT8 quantized TFLite models run successfully and detect objects on the Cortex-A55 CPU, the compiled neutron.tflite versions yield zero detections (empty/no output) when offloaded to the NPU, despite executing inference without crashing.
Environment & Hardware Setup
- Hardware: i.MX95 19x19 LPDDR5 EVK (A1 Revision)
- OS/Kernel: Linux 6.12.34-lts-next-gbe78e49cb433 #1 SMP PREEMPT (aarch64)
- NXP Toolchain: MCU-SDK v25.09.00 + Linux 6.12.34_2.1.0
- Models Tested: YOLOv5nu, YOLOv8n, YOLOv11n (Ultralytics)
Workflow Steps & Commands Used
1. Quantization (Ultralytics Export)
yolo export model=yolov8n.pt format=tflite int8=True imgsz=320# (Repeated identically for yolov11n.pt and yolov5nu.pt)
- Status: Works perfectly on CPU. yolovXn_full_integer_quant.tflite detects objects correctly on the A55 cores.
2. Neutron Compilation
./neutron-converter --input yolov8n_full_integer_quant.tflite --target imx95 --output yolov8n_full_integer_quant_neutron.tflite- Status: Fails to detect objects on NPU. The compiled model loads and runs inference without throwing syntax or execution errors, but output tensors return zero detections for the exact same test images.
Observed Symptoms & Suspected Root Causes
- Operator Fallbacks: Did the converter fall back to CPU for specific YOLO layers (like custom Anchors, SiLU/Swin activations, or Non-Max Suppression)?
- Quantization Scaling/Asymmetry: YOLO models exported via Ultralytics often use asymmetric quantization or have specific output tensor scaling that the Neutron NPU driver might misinterpret.
- Output Tensor Formatting: The inference runs, which suggests the input pipeline is fine, but the output bounding boxes/scores are either blank or completely garbage values.
Questions for NXP Experts
- Are there known limitations or mandatory optimization flags needed in the neutron-converter specifically for Ultralytics YOLO architectures?
- Should the NMS (Non-Max Suppression) layer be stripped out before passing the TFLite model to the Neutron converter?
- Does the i.MX95 Neutron SDK require symmetric quantization (per_channel=True or False) to parse the output layers properly?