tflite + XNNPACK delegate for inference on quantized network not working

edouard_charvet — Wed, 29 Mar 2023 14:35:32 GMT

Hello,

Running on i.mx8QM MEK board, with Yocto Linux LF5.15.71_2.2.0

Following the I.MX Machine Learning User Guide §3.7 to benchmark inference of TFLite using XNNPACK delegate with the example model mobilenet_v1_1.0_224_quant.tflite, I see that XNNPACK delegate is not being used for the inference (most likely because this is a quantized network - as it works fine with fp32 models).

Browsing the web a little bit, it sounds like XNNPACK supports quantized networks since Sept 2021 (cf https://blog.tensorflow.org/2021/09/faster-quantized-inference-with-xnnpack.html).

Is it possible and if so how can I get the XNNPACK delegate to work with quantized networks ? I guess I need to change something somewhere in the yocto linux build scripts... but what and where ?

Thanks

Re: tflite + XNNPACK delegate for inference on quantized network not working

Bio_TICFSL — Thu, 30 Mar 2023 14:23:01 GMT

Hello,

You have to check the deepview https://www.nxp.com/design/software/development-software/eiq-ml-development-environment/eiq-inference-with-deepviewrt:EIQ-INFERENCE-DEEPVIEWRT

Since not all images quantized works it have to be int8 delegate but the fp32 do not.

Regards

i.MX Processors中的主题 Re: tflite + XNNPACK delegate for inference on quantized network not working

tflite + XNNPACK delegate for inference on quantized network not working

Re: tflite + XNNPACK delegate for inference on quantized network not working