Neutron Converted YOLOv8n model is not giving proper output in i.MX95

[vatsal]

Hi there,

I have exported full quantized int8 YOLOv8n Object Detection model from ultralytics. I have converted it to run on NPU using neutron converter from latest eIQ Toolkit version 1.17 and I have tried to execute it on i.MX95 hardware. 

I have tried both converted and non-converted model with NPU delegate but seems like only the Neutron Graph which is present on converted model is going to execute on NPU.

When I compare the raw outputs from both model, converted model is giving multiple false positives with more than 95% score. The same script I am using for both converted and non-converted model but having issue with converted model only. I have validated with multiple approaches, getting same everytime.

When I have inspect both the models with netron app, I have found major architectural changes in converted model.

Here, some points I want to ask is,

1. Is the lastest object detection architectures like YOLOv8 and YOLOv11 is supported on Neutron Converter with eIQ Toolkit version 1.17? If yes then what are steps you are following please let me know.

2. Do you have tested the YOLOv8 and YOLOv11 with NPU on i.MX95? If yes then can you please send the model for verification and Post-Processing steps as well.

3. If we want to execute the operations apart from Neutron Graph on Neutron NPU in i.MX95 then what is the process?

4. If we want to execute the models mentioed above on GPU in i.MX95 then what is the process?

I have gone through the Machine Learning User Guide as well but haven't found related details. If you want any details from my side to debug this then feel free to ask.

Thanks,
Vatsal

[vatsal]

We have performed the training again with YOLOv5s with our custom dataset and try to do inference as you were claiming that YOLOv5 is giving good results with neutron converter in previous chat. The issue is still there with YOLOv5 as well. I am attaching the results as well for the reference. Where we tried with converted and non-converted int8 tflite YOLOv5s model. 

Now, If you are claiming that YOLOv5 is giving good results then why don't you share the validated model with us. Please share all steps you are following from export, quantization to convert and what are the steps you are following for post-processing as well. So that we can validate your model and replicate the steps you have followed at our end.

Please share you benchmarking and profiling data with i.MX95 as well if you can share. 

[Bio_TICFSL]

Hello,

The issue you're experiencing with false positives from the Neutron-converted YOLOv8n model is a known challenge. Current support for YOLOv8 models on the i.MX95 NPU is still being optimized, and architectural changes during conversion can affect performance.

Regarding your specific questions:

1. YOLOv8 architecture support in eIQ Toolkit 1.17 is still being improved. The recommended conversion workflow is:
- Ensure proper int8 quantization when exporting your model
- Use the latest eIQ Toolkit (v1.17) for conversion with neutron-converter
- Command: `./neutron-converter --input [your-model].tflite --target imx95 --use-python-prototype`

2. While YOLOv8 is supported, it may not be fully optimized yet. YOLOv5 has shown better compatibility with current NPU implementations. The NXP team is actively working on improving YOLOv8/YOLOv11 support.

3. To execute operations beyond Neutron Graph on the NPU, you need to analyze your model with the eIQ or netron tool to identify which operators are successfully converted to run on the NPU (shown as neutronop content). Operators not converted will run on CPU.

4. For GPU execution on i.MX95, you would use the TensorFlow Lite GPU delegate instead of the NPU delegate. This requires modifying your inference code to use the GPU delegate.

You can verify model operation distribution by examining the converted model in netron - any operators with "neutronop" content are executed on NPU, while others remain on CPU.

Regards

[manuelcrisologo]

Hello @Bio_TICFSL!

I've noticed NXP team suggest to use the --use-python-prototype flag, however my neutron converter does not recognize it. Is it supported on Windows, or it is only compatible with the Linux eIQ release?