eIQ Toolkit v1.17 PTQ silently skipped, output tflite remains full float32 without error message Environment: - Host: Windows11, eIQ Toolkit v1.17 - Target SOC: NXP i.MX95 19X19 board - Model: pruned MNIST CNN, input shape [1,1,28,28] - ONNX opset: 12, pure FP32, no pre‑inserted QDQ nodes. Problem description: I try to run PTQ quantization via eiq‑converter‑onnx2tflite plugin. Both CLI and GUI Model Tool finish conversion without any error / warning log. But the output tflite model is still full float32. All weights, biases and activation tensors have scale=0, zero_point=0. It means PTQ calibration never actually ran. What I have tried: As a prerequisite, I have got an .onnx file from python which opset is 12. Then I want to quantization via eIQ. After quantization, to do the neutron-converter via eIQ. there are 3 ways to do this, I think. and after quantization, we can check the quantization is done or not via to see the dtype ,scale and zp. I do this via GUI. Step Tool Output Folder output Step1 01_train_mnist_cnn.py 01_train_mnist_cnn_out mnist_cnn.pth Step2 02_prune_test.py 02_prune_test_out prune_after.pth Step3 03_export_onnx.py 03_export_onnx_out handwrite_cnn_pruned.onnx Step4 GUI ONNX 04_eIQ_quantization handwrite_cnn_pruned_quantizationed.onnx Step5 GUI ONNX2TFLITE 05_eIQ_quqntizedonnx2tflite eIQ_quqntizedonnx2tflite.tflite there is a question about this output,all scale and zp is 0, that is to say , there is no quantization. I donot know why. Step6 GUI converter neutron 06_eIQ_neutron eIQ_quqntizedonnx2tflite_converted.tflite I do this via command line by qty+2tflite in one step. eiq-converter.exe --plugin eiq-converter-onnx2tflite --source D:\01_Work\AI_Project\mnist_new_demo\03_export_onnx_out\handwrite_cnn_pruned.onnx --dest D:\01_Work\AI_Project\07_CMD_OUTPUT\mnist_int8.tflite --input-shape "1,1,28,28" --precision int8_ptq --calibration-dataset D:\01_Work\AI_Project\mnist_calib_imgs --quantize-input true eiq-converter.exe --plugin eiq-converter-onnx2tflite --source D:\01_Work\AI_Project\mnist_new_demo\03_export_onnx_out\handwrite_cnn_pruned.onnx --dest D:\01_Work\AI_Project\07_CMD_OUTPUT\mnist_int8.tflite --input-shape "1,1,28,28" --precision int8_ptq --calibration-dataset D:\01_Work\AI_Project\mnist_calib_imgs --preprocessing "div:255.0" I have tried this 2 command, both of them can be [success],but all scale and zp is 0, that is to say , there is no quantization. I donot know why. I do this via command line by quan+2tflite in two step. python -m onnx2quant -c "input;D:\01_Work\AI_Project\mnist_new_demo\09_mnist_calib_npy4d" -o D:\01_Work\AI_Project\09_cmd_onnx_qunitiation\model_qdq.onnx D:\01_Work\AI_Project\mnist_new_demo\03_export_onnx_out\handwrite_cnn_pruned.onnx eiq-converter.exe --plugin eiq-converter-onnx2tflite --source D:\01_Work\AI_Project\09_cmd_onnx_qunitiation\model_qdq.onnx --dest D:\01_Work\AI_Project\09_cmd_onnx_qunitiation\mnist_int8_final.tflite --input-shape "1,1,28,28" I have tried them, then I got a .tflite which's scale and zp is not all 0. Then I use this mnist_int8_final.tflite to the GUI eiq-converter-neutron. Then I have got a .tflite file smoothly. But when I run it on the iMax95, we can see that it did not run on the NPU. I think the question is my .tflite file. So I tried to use the command line to get the the right .tflite file, there is still ERROR: Internal error! Summary of observations: 1. Using eiq‑converter‑onnx2tflite (GUI or CLI) for native PTQ finishes without errors, but produces full float32 models. Quantization appears to be silently skipped. 2. When I use third‑party tool `onnx2quant` to generate QDQ‑inserted ONNX first, I can get a TFLite with non‑zero scale and zero‑point. However this externally‑quantized model cannot be fully processed by neutron‑converter, it throws `ERROR: Internal error!` during graph optimization, and fails to generate a working NPU‑compatible model. I am still learning this toolchain. It may be that I have misused parameters or followed an incorrect workflow. Could anyone who has successfully run a quantized MNIST‑like CNN on i.MX95 share the verified workflow? I would really appreciate any hints on how to correctly perform PTQ and produce a valid TFLite model that can be compiled and executed on the NPU. Thank you very much. EIQ-TOOLKIT
記事全体を表示
