Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Hello NXP Support Team, We are evaluating object detection on the FRDM i.MX95 platform using the Neutron SDK v3.1.3 and are unable to generate an NPU-compatible model. The Neutron converter successfully loads the model, but reports that 0 operators are mapped to the Neutron NPU. Environment Target Board: FRDM i.MX95 Neutron SDK: 3.1.3 Ultralytics: Tested with both YOLO11 and YOLOv8 eIQ Toolkit: Used for ONNX → TFLite conversion Model: Custom single-class peg detector Training Command $ yolo detect train \ model=yolov11n.pt \ data=/visual_inspect_yolo/dataset/dataset.yaml \ imgsz=640 \ epochs=100 \ batch=16 \ project=models \ name=peg_detector_v8 Export Command $ yolo export \ model=models/peg_detector_v84/weights/best.pt \ format=tflite \ int8=True \ data=/visual_inspect_yolo/dataset/dataset.yaml We also tested an alternative workflow: Export PyTorch → ONNX Convert ONNX → INT8 TFLite using the NXP eIQ Toolkit Both workflows produced the same result when compiled with the Neutron SDK. Neutron Compilation ~/Downloads/eiq-neutron-sdk-linux-3.1.3/bin/neutron-converter \ --target imx95 \ --input best_int8.tflite \ --output my_model_int8_npu.tflite Converter Output The converter reports: Operators after import: 341 Operators after optimization: 367 Operators converted: 0 Operator conversion ratio: 0 / 367 Number of Neutron graphs: 0 Warnings: WARNING: None of the operators from the graph was mapped to Neutron. WARNING: The converted model is the same as the input model because no operators were mapped to Neutron. WARNING: Graph has FLOAT operators which are NOT supported! This can result in low conversion ratio. Additional Information We observed the same behavior with: YOLO11 YOLOv8 Direct Ultralytics TFLite export ONNX → eIQ Toolkit → INT8 TFLite All generated TFLite models result in 0 operators being mapped by the Neutron compiler. Questions Are YOLOv8 or YOLO11 object detection models officially supported by the Neutron compiler for the i.MX95? Is there a recommended export pipeline for YOLO models targeting the i.MX95 NPU? Are there any known limitations with the current Neutron SDK (v3.1.3) regarding YOLO detection heads? Does NXP provide a reference YOLOv8/YOLO11 model that successfully compiles for the i.MX95 NPU? Is there any additional compiler option or preprocessing step required to enable operator mapping? We would appreciate any guidance, recommended workflows, or reference models that are known to work with the i.MX95 Neutron NPU. Thank you. Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Thank you for your response. I would like to inquire if there is a standard procedure available for training, exporting, and deploying models on the IMX95 board. As we currently have the ARA2, we are looking to fully utilize its capabilities and customize our models. We have upcoming demos for NXP Tech Days, and your assistance in this matter would be greatly appreciated. Thank you for your help. Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Tried the yolo8m model from eIQ model zoo on imx95 board with LF 2026 Q2 release image. kernel version is 6.18.20 using neutron SDK 3.1.2. it works.
You can try it firstly by:
wget https://huggingface.co/EdgeFirst/yolov8-det/resolve/main/imx95/yolov8n-det-int8-smart.imx95.tflite
root@imx95evk:/usr/bin/tensorflow-lite-2.19.0/examples# ./benchmark_model --graph=yolov8n-det-int8-smart.imx95.tflite --external_delegate_path=/usr/lib/libneutron_delegate.so
more info you can refer the README eiq-model-zoo/tasks/vision/object-detection/yolov8 at main · NXP/eiq-model-zoo
What's more, you can attached model and details log of convert/complier.
Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Based on the converter log, the first issue to resolve is that the generated TFLite model still contains FLOAT operators:
WARNING: Graph has FLOAT operators which are NOT supported!
For i.MX95 Neutron, the input to neutron-converter must be a TFLite model whose operators and quantization format are compatible with the Neutron compiler. In particular, the i.MX95 Neutron flow expects quantized TFLite and symmetric int8 weights. If the model still contains FLOAT operators/tensors after the Ultralytics export or ONNX-to-TFLite conversion, the converter may be unable to create any Neutron-compatible subgraph, which is consistent with the reported result:
Operators converted: 0
Number of Neutron graphs: 0
YOLOv8 has been evaluated on i.MX95 in some flows, but full end-to-end YOLOv8/YOLO11 offload should not be assumed for arbitrary Ultralytics exports. Depending on the exported TFLite graph, only part of the model may be converted to NeutronGraph and unsupported operators will remain on CPU. Therefore, the recommended next step is to inspect/profile the generated TFLite model and confirm:
the graph is fully quantized,
there are no FLOAT operators,
weights are symmetric int8,
input/output tensor types are compatible, or converted with the Neutron converter uint8-to-int8 options if applicable,
YOLO post-processing such as decode/NMS is kept outside the NPU graph unless the exact operators are confirmed supported by the SDK.
Please also ensure that the neutron-converter version and the Neutron runtime/firmware/delegate on the board are from the same compatible SDK/BSP release.
As a recommended flow, please try the NXP/eIQ conversion path:
PyTorch -> ONNX with static input shape -> NXP/eIQ quantization with representative calibration data -> quantized TFLite -> neutron-converter --target imx95
If the model has uint8 input/output tensors, please also test:
--convert-inputs-uint8-to-int8
--convert-outputs-uint8-to-int8
If the conversion still reports 0 mapped operators after removing FLOAT operators, please share:
- the complete neutron-converter log with verbose/profiling output if available,
- the TFLite operator list,
- tensor data types and quantization parameters,
- the exact BSP/runtime Neutron delegate/firmware versions on the FRDM i.MX95 board,
- whether the YOLO detection head includes NMS or other post-processing inside the TFLite graph. Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Hi i am running ubuntu 24.04, but eiq_toolkit is avaialble for only 20.04.03. how can i use eiqToolkit and Quantization Using eIQ Toolkit Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Recommended End-to-End Workflow
Model Training (PC)
Train using your preferred framework:
Ultralytics YOLOv8
PyTorch
TensorFlow
ONNX-native workflows
For object detection, NXP already provides YOLO reference recipes in the eIQ Model Zoo, including YOLOv8 object detection models. [github.com], [github.com]
Example:
Shell
yolo detect train \
model=yolov8n.pt \
data=dataset.yaml \
imgsz=640 \
epochs=100
`
Export to ONNX
NXP generally recommends using ONNX as the interchange format before quantization and deployment.
yolo export \
model=best.pt \
format=onnx
The Neutron enablement presentations explicitly describe a flow based on:
Plain Text
PyTorch
↓
ONNX
↓
Quantization
↓
TFLite
↓
Neutron Converter
rather than directly targeting deployment from training artifacts.
Quantization Using eIQ Toolkit
The Neutron workflow documentation recommends using the eIQ Toolkit quantization utilities:
python -m onnx2quant \
model.onnx \
-o model_quant.onnx \
-c input::
``
followed by:
python -m onnx2tflite \
model_quant.onnx \
-o model_int8.tflite
Show more lines
This flow is explicitly documented in the i.MX95 Neutron enablement material.
Compile for i.MX95 Neutron NPU
neutron-converter \
--target imx95 \
--input model_int8.tflite \
--output model_neutron.tflite
The Neutron converter creates Neutron-specific graph partitions that can be offloaded to the NPU.
Validate Conversion Ratio
A successful NPU deployment should report something similar to:
Number of operators converted > 0
Number of Neutron graphs > 0
If you see:
Operators converted: 0
Number of Neutron graphs: 0
then the model is not being accelerated by the NPU.
Your current issue falls into this category.
Deploy on FRDM-i.MX95
Run using TensorFlow Lite with the Neutron delegate:
./benchmark_model \
--graph=model_neutron.tflite \
--external_delegate_path=/usr/lib/libneutron_delegate.so
``
or
./label_image \
--external_delegate_path=/usr/lib/libneutron_delegate.so
The i.MX Machine Learning User Guide identifies the Neutron Delegate as the acceleration mechanism for i.MX95 TensorFlow Lite models. Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU In my test I did not train or export the model myself. I used a pre-generated YOLOv8 model from the eIQ Model Zoo and verified that it runs on the i.MX95 platform.
The only command I actually used was:
./benchmark_model \
--graph=yolov8n-det-int8-smart.imx95.tflite \
--external_delegate_path=/usr/lib/libneutron_delegate.so
``
with the model:
wget https://huggingface.co/EdgeFirst/yolov8-det/resolve/main/imx95/yolov8n-det-int8-smart.imx95.tflite
For custom models, the recommended NXP flow is:
PyTorch
↓
ONNX (static input shape)
↓
eIQ Toolkit ONNX2Quant
↓
eIQ Toolkit ONNX2TFLite
↓
Quantized TFLite
↓
neutron-converter --target imx95
Since your model reports:
Plain Text
Operators converted: 0
Number of Neutron graphs: 0
WARNING: Graph has FLOAT operators which are NOT supported!
I suspect your generated TFLite graph is structurally different from the eIQ Model Zoo reference model. The first thing I would recommend is comparing the two models for:
Input/output tensor type (INT8 vs UINT8)
Presence of FLOAT operators
Decode/NMS layers inside the graph
Operator list reported by Netron / TFLite analyzer
Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Can you please tell me how did you convert yolov8m_full_integer_quant.tflite to be able to run on the imx95 NPU? Step followed and environment setup data(HOST).. would greatly help us. Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Since eIQ Toolkit was validated on Ubuntu 20.04, the safest approach is:
Docker
Run a Ubuntu 20.04 container on your Ubuntu 24.04 host:
docker run -it --name eiq \
ubuntu:20.04 /bin/bash
Then install the required dependencies and eIQ Toolkit inside the container. Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Unable to preserve confidence output when converting custom YOLOv8 ONNX model using eIQ Toolkit (onnx2quant) Overview Hi NXP Team, I'm trying to deploy a custom YOLOv8 single-class object detection model on the FRDM i.MX95 using the eIQ Toolkit. The complete conversion pipeline runs successfully, but after onnx2quant, the confidence output becomes all zeros while the bounding box outputs remain valid. Environment - Ubuntu 24.04 - Python 3.10 - eIQ ONNX2TFLite 0.9.0 - ONNX Runtime 1.21.1 - TensorFlow 2.21 - neutron-converter 3.1.3 - Target: FRDM i.MX95 (tflite_runtime 2.19 + Neutron delegate) Conversion Pipeline 1. Train yolo detect train model=yolov8n.pt data=dataset.yaml imgsz=640 epochs=50 2. Export ONNX yolo export model=best.pt format=onnx opset=13 3. Verify ONNX Input : (1,3,640,640) Output: (1,5,8400) ONNX Runtime inference: Confidence Channel Max = 0.773 4. Generate calibration dataset Shape : (1,3,640,640) dtype : float32 Range : 0.0 - 1.0 5. Quantize onnx2quant best.onnx -c "images;calibration/images" -o best_quant.onnx Also tested: onnx2quant best.onnx -u Both produce the same result. 6. Verify Quantized ONNX Output : (1,5,8400) Bounding box channels remain valid. Confidence: Min = 0 Max = 0 Mean = 0 Decoded detections = 0 7. Convert to TFLite onnx2tflite best_quant.onnx -o best.tflite 8. Compile for Neutron neutron-converter --target imx95 --input best.tflite --output best_neutron.tflite Compilation succeeds. Operator conversion: 278 / 325 (85.5%) Investigation Performed Verified: • PyTorch model works • ONNX export works • ONNX Runtime inference works • Calibration dataset is correct • Real and random calibration produce identical results • TFLite reproduces the Quantized ONNX output • Neutron reproduces the TFLite output The issue first appears after: ONNX ↓ onnx2quant ↓ Quantized ONNX (confidence becomes zero) Additional Observation NXP reference model: Input : (1,640,640,3) INT8 Output: (1,84,8400) INT8 My converted model: Input : (1,3,640,640) FLOAT32 Output: (1,5,8400) FLOAT32 Is there a recommended export or quantization workflow for custom YOLOv8 models that preserves the confidence output? Could this be a limitation or bug in onnx2quant for models with a (1,5,8400) output? Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Discussing with the AE team. Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Has end-to-end been evaluated for the Ara240? The datasheet mentions two vector cores that can execute post-processing ops such as sigmoid and NMS. Could the compiler map NMS ops to the vector cores? Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Sorry for the delay. I am trying to reproduce the conversion workflow.
One question for now, why the converted model's data type is FLOAT32? Have you tried to convert to INT8? The Neutron NPU requires the INT8 type as input data. I met the similar error on other models conversion and the root cause is the data type. Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU The confidence output is lost due to a fundamental limitation of full INT8 quantization ( inference_output_type=tf.int8 ) applied to YOLOv8's output tensor.
YOLOv8 packs bounding box coordinates and confidence scores into a single output tensor of shape (1, 5, 8400) . The bbox values have a large dynamic range (~640 pixels), while the confidence scores are in the range of ~0 to 1. When the entire output tensor shares a single quantization scale, that scale is dominated by the large bbox values (~640), leaving only a fraction of one integer level to represent the entire confidence range (~1). As a result, all confidence values are effectively rounded to zero after INT8 quantization.
Recommended Solution Instead of going through onnx2quant , export INT8 TFLite directly from your trained .pt model using Ultralytics, then feed it into neutron-converter :
# Export INT8 TFLite directly (calibration uses your training dataset)
yolo export model=best.pt \
format=litert \
imgsz=640 \
quantize=8 \
data=dataset.yaml \
fraction=0.1
# Compile for Neutron (unchanged)
neutron-converter --target imx95 --input best_int8.tflite --output best_neutron.tflite
For the input and output data type, please ensure they are np.int8:
interp = tf.lite.Interpreter(model_path=TFLITE_INT8)
interp.allocate_tensors()
inp_d = interp.get_input_details()[0]
out_ds = interp.get_output_details()
inp_scale, inp_zp = inp_d["quantization"]
out_d = out_ds[0]
out_scale, out_zp = out_d["quantization"]
print(f" Input dtype={inp_d['dtype']} shape={inp_d['shape'].tolist()}"
f" quant=(scale={inp_scale:.6f}, zp={inp_zp})")
print(f" Output dtype={out_d['dtype']} shape={out_d['shape'].tolist()}"
f" quant=(scale={out_scale:.6f}, zp={out_zp})")# Determine input format from shape
in_shape = inp_d["shape"].tolist() # [1,3,640,640] or [1,640,640,3]
if in_shape[1] == 3:
# NCHW
src=img_nchw
else:
# NHWC
src=img_nhwcif inp_d["dtype"] == np.int8:
src_int8 = np.clip(np.round(src / inp_scale + inp_zp), -128, 127).astype(np.int8)
interp.set_tensor(inp_d["index"], src_int8)
else:
interp.set_tensor(inp_d["index"], src.astype(np.float32))interp.invoke()
raw_out = interp.get_tensor(out_d["index"]) # may be int8 or float32if out_d["dtype"] == np.int8:
dq_out = (raw_out.astype(np.float32) - out_zp) * out_scale
else:
dq_out = raw_out.astype(np.float32)dq_out = dq_out[0] # (5, 8400) normalized# Rescale bbox back to pixel coords for display
BBOX_SCALE = 640.0
tfl_bbox = dq_out[:4] * BBOX_SCALE # (4, 8400)
tfl_conf = dq_out[4] # (8400,)
Re: Unable to compile YOLOv8/YOLO11 TFLite models for i.MX95 Neutron NPU Hi Tried the commands you shared... But neutron-converter is failing to convert the model... please find the log attached for your reference
記事全体を表示
