Environment:
- Host: Windows11, eIQ Toolkit v1.17
- Target SOC: NXP i.MX95 19X19 board
- Model: pruned MNIST CNN, input shape [1,1,28,28]
- ONNX opset: 12, pure FP32, no pre‑inserted QDQ nodes.
Problem description:
I try to run PTQ quantization via eiq‑converter‑onnx2tflite plugin.
Both CLI and GUI Model Tool finish conversion without any error / warning log.
But the output tflite model is still full float32.
All weights, biases and activation tensors have scale=0, zero_point=0. It means PTQ calibration never actually ran.
What I have tried:
As a prerequisite, I have got an .onnx file from python which opset is 12.
Then I want to quantization via eIQ. After quantization, to do the neutron-converter via eIQ.
there are 3 ways to do this, I think.
and after quantization, we can check the quantization is done or not via to see the dtype ,scale and zp.
- I do this via GUI.
| Step | Tool | Output Folder | output | |
| Step1 | 01_train_mnist_cnn.py | 01_train_mnist_cnn_out | mnist_cnn.pth | |
| Step2 | 02_prune_test.py | 02_prune_test_out | prune_after.pth | |
| Step3 | 03_export_onnx.py | 03_export_onnx_out | handwrite_cnn_pruned.onnx | |
| Step4 | GUI ONNX | 04_eIQ_quantization | handwrite_cnn_pruned_quantizationed.onnx | |
| Step5 | GUI ONNX2TFLITE | 05_eIQ_quqntizedonnx2tflite | eIQ_quqntizedonnx2tflite.tflite | there is a question about this output,all scale and zp is 0, that is to say , there is no quantization. I donot know why. |
| Step6 | GUI converter neutron | 06_eIQ_neutron | eIQ_quqntizedonnx2tflite_converted.tflite |
- I do this via command line by qty+2tflite in one step.
eiq-converter.exe --plugin eiq-converter-onnx2tflite --source D:\01_Work\AI_Project\mnist_new_demo\03_export_onnx_out\handwrite_cnn_pruned.onnx --dest D:\01_Work\AI_Project\07_CMD_OUTPUT\mnist_int8.tflite --input-shape "1,1,28,28" --precision int8_ptq --calibration-dataset D:\01_Work\AI_Project\mnist_calib_imgs --quantize-input true
eiq-converter.exe --plugin eiq-converter-onnx2tflite --source D:\01_Work\AI_Project\mnist_new_demo\03_export_onnx_out\handwrite_cnn_pruned.onnx --dest D:\01_Work\AI_Project\07_CMD_OUTPUT\mnist_int8.tflite --input-shape "1,1,28,28" --precision int8_ptq --calibration-dataset D:\01_Work\AI_Project\mnist_calib_imgs --preprocessing "div:255.0"
I have tried this 2 command, both of them can be [success],but all scale and zp is 0, that is to say , there is no quantization. I donot know why.
- I do this via command line by quan+2tflite in two step.
python -m onnx2quant -c "input;D:\01_Work\AI_Project\mnist_new_demo\09_mnist_calib_npy4d" -o D:\01_Work\AI_Project\09_cmd_onnx_qunitiation\model_qdq.onnx D:\01_Work\AI_Project\mnist_new_demo\03_export_onnx_out\handwrite_cnn_pruned.onnx
eiq-converter.exe --plugin eiq-converter-onnx2tflite --source D:\01_Work\AI_Project\09_cmd_onnx_qunitiation\model_qdq.onnx --dest D:\01_Work\AI_Project\09_cmd_onnx_qunitiation\mnist_int8_final.tflite --input-shape "1,1,28,28"
I have tried them, then I got a .tflite which's scale and zp is not all 0.
Then I use this mnist_int8_final.tflite to the GUI eiq-converter-neutron. Then I have got a .tflite file smoothly.
But when I run it on the iMax95,
we can see that it did not run on the NPU.
I think the question is my .tflite file.
So I tried to use the command line to get the the right .tflite file, there is still ERROR: Internal error!
Summary of observations:
1. Using eiq‑converter‑onnx2tflite (GUI or CLI) for native PTQ finishes without errors, but produces full float32 models. Quantization appears to be silently skipped.
2. When I use third‑party tool `onnx2quant` to generate QDQ‑inserted ONNX first, I can get a TFLite with non‑zero scale and zero‑point.
However this externally‑quantized model cannot be fully processed by neutron‑converter, it throws `ERROR: Internal error!` during graph optimization, and fails to generate a working NPU‑compatible model.
I am still learning this toolchain. It may be that I have misused parameters or followed an incorrect workflow.
Could anyone who has successfully run a quantized MNIST‑like CNN on i.MX95 share the verified workflow?
I would really appreciate any hints on how to correctly perform PTQ and produce a valid TFLite model that can be compiled and executed on the NPU.
Thank you very much.