Issue with Fully Integer Quantization - DataStore Not Supported

hamedtaheri — Wed, 05 Jun 2024 21:32:39 GMT

Dear IMX Support Team,

I am encountering an issue when attempting to quantize my model using the eIQ converter (windows version). Despite providing 100 images in the imgs folder for calibration, the converter appears to ignore them and instead uses a white noise dataset. The error message states that datastore datasets are not supported. Below is the command I used and the corresponding output:

C:\nxp\eIQ_Toolkit_v1.11.4>eiq-converter --plugin eiq-converter-tflite --source workspace\models\deeplab_v3\best_model.h5 --dest workspace\models\deeplab_v3\best_model.tflite --default_shape 1,256,256,3 --quantize --quantize_format uint8 --quant_normalization signed --samples imgs eiq-converter 0.0.0 Using Converter Plugin : eiq-converter-tflite We do not support datastore datasets currently Using white noise dataset generation 2024-06-05 16:20:32.735326: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: SSE SSE2 SSE3 SSE4.1 SSE4.2 AVX AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. C:\nxp\eIQ_Toolkit_v1.11.4\python\Lib\site-packages\tensorflow\lite\python\convert.py:947: UserWarning: Statistics for quantized inputs were expected, but not specified; continuing anyway. warnings.warn( 2024-06-05 16:20:35.228063: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:378] Ignored output_format. 2024-06-05 16:20:35.228221: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:381] Ignored drop_control_dependency. 2024-06-05 16:20:35.229380: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: C:\Users\Hamed\AppData\Local\Temp\tmpnh78xhxx 2024-06-05 16:20:35.234016: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2024-06-05 16:20:35.234215: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: C:\Users\Hamed\AppData\Local\Temp\tmpnh78xhxx 2024-06-05 16:20:35.242521: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:382] MLIR V1 optimization pass is not enabled 2024-06-05 16:20:35.244744: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2024-06-05 16:20:35.302547: I tensorflow/cc/saved_model/loader.cc:217] Running initialization op on SavedModel bundle at path: C:\Users\Hamed\AppData\Local\Temp\tmpnh78xhxx 2024-06-05 16:20:35.327293: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 97907 microseconds. 2024-06-05 16:20:35.357246: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable. 2024-06-05 16:20:35.429021: I tensorflow/compiler/mlir/lite/flatbuffer_export.cc:2245] Estimated count of arithmetic ops: 4.502 G ops, equivalently 2.251 G MACs fully_quantize: 0, inference_type: 6, input_inference_type: FLOAT32, output_inference_type: FLOAT32 2024-06-05 16:20:36.773066: I tensorflow/compiler/mlir/lite/flatbuffer_export.cc:2245] Estimated count of arithmetic ops: 4.502 G ops, equivalently 2.251 G MACs SUCCESS: Converted Source: workspace\models\deeplab_v3\best_model.h5 Destination: workspace\models\deeplab_v3\best_model.tflite Source: workspace\models\deeplab_v3\best_model.h5 Destination: workspace\models\deeplab_v3\best_model.tflite

Given that datastore datasets are not supported, how can I perform fully integer quantization using my calibration data? Is there an alternative method or workaround to ensure that my provided images are utilized for calibration?

Thanks

Re: Issue with Fully Integer Quantization - DataStore Not Supported

Zhiming_Liu — Thu, 06 Jun 2024 05:58:05 GMT

Hi @hamedtaheri

You can use MODEL TOOL gui to finish Quantizatio with dataset in eIQ portal.

Re: Issue with Fully Integer Quantization - DataStore Not Supported

hamedtaheri — Thu, 06 Jun 2024 15:43:34 GMT

Hi @Zhiming_Liu

Thank you for the information. I successfully converted my file.

The only issue is that my model, which works perfectly with the .h5 format, does not segment anything when I convert it to a fully integer tflite.

I tried converting it to FP16, but surprisingly, on my IMX 8mPlus, the inference time increased with FP16 instead of decreasing.

Could you please let me know if the IMX 8mPlus supports FP16?

Thanks,

Re: Issue with Fully Integer Quantization - DataStore Not Supported

Zhiming_Liu — Fri, 07 Jun 2024 00:38:44 GMT

Hi @hamedtaheri

Could you please let me know if the IMX 8mPlus supports FP16?

-->Support, but int8/unit8 quantization have best performance.

Best Regards

Zhiming

i.MX ProcessorsのトピックRe: Issue with Fully Integer Quantization - DataStore Not Supported

Issue with Fully Integer Quantization - DataStore Not Supported

Re: Issue with Fully Integer Quantization - DataStore Not Supported

Re: Issue with Fully Integer Quantization - DataStore Not Supported

Re: Issue with Fully Integer Quantization - DataStore Not Supported