how to know the imx8m plus NPU acceleration is enable already?

[fjpmbb_abc]

@xiaodong_zhang 

Hi xiadodong

    I am using the imx8m plus for myself segmentation. BSP is 5.4.70. I want to enable the NPU for acceleration.

    1. When i use the "interpreter->UseNNAPI(accelaration);",imx8m plus told me "ERROR: Interpreter::UseNNAPI is not supported. Use Interpreter::ModifyGraphWithDelegate instead." Detailed information please check the attachment "npu_acceleration_1.c"
2. Then i modified my code as the error said. using the "interpreter->modifygraphwithdelegate()". the MPU said "NN-API applied sucessfully".But "Failed to invoke interpreter". Detailed information please check the attachment "npu_acceleration_2.c"
3. I modified again by using the "delegates_.emplace("NNAPI", tflite::CreateNNAPIDelegate());", this time MPU no NN API acceleration error report, but it seems like the accelaration does't work，as i found that "Elapsed time" is the same as no acceleration scenario.Detailed information please check the attachment "npu_acceleration_3.c"

So,
1. how to know the imx8m plus NPU acceleration is enable already?
2. how to do that can make the NPU acceleration enable?

Thank you.

[xiaodong_zhang]

@fjpmbb_abc 

Can you tell me your development environment? Could you refer to example label_image in tensorflow lite? By default, tensorlfow lite in i.MX Yocto supports NPU by NNAPI when build tflite with -DTFLITE_ENABLE_NNAPI=on.

[fjpmbb_abc]

@xiaodong_zhang 

My development environment is:

1.PC ubuntu 18.04

2.Board imx8m plus evk BSP 5.4.70

I will refer to the NXP offical tensorflow lite  demo code and compile with -DTFLITE_ENABLE_NNAPI=on. Thanks.

[xiaodong_zhang]

@fjpmbb_abc 

If you refer to example label_image (https://source.codeaurora.org/external/imx/tensorflow-imx/tree/tensorflow/lite/examples/label_image?... ), you can get NPU to accelerate inference.

[fjpmbb_abc]

@xiaodong_zhang 

  i check the NXP BSP yocto v5.4.70 tensorflow source code,

/5.4.70/imx8mpevk-build-xwayland/tmp/work/aarch64-poky-linux/tensorflow-lite/2.3.1-r0/git/tensorflow

there is no code use the marco about "TFLITE_ENABLE_NNAPI", but the labelimage could accelerate when input the "a" to enable the acceleration. Why?

Thanks.

[xiaodong_zhang]

@fjpmbb_abc 

./tensorflow/lite/CMakeLists.txt use Macro TFLITE_ENABLE_NNAPI

https://source.codeaurora.org/external/imx/tensorflow-imx/tree/tensorflow/lite/CMakeLists.txt?h=imx_...

[fjpmbb_abc]

@xiaodong_zhang 

Hi Xiaodong

    I refer to the code you mentioned for NPU acceleration, it seems like the NPU is enable, but the program tell me the log as below:

    Operator RESIZE_BILINEAR (v1) refused by NNAPI delegate: Operator refused due performance reasons.

accelaration: true

INFO: Created TensorFlow Lite delegate for NNAPI.
WARNING: Operator RESIZE_BILINEAR (v1) refused by NNAPI delegate: Operator refused due performance reasons.
WARNING: Operator RESIZE_BILINEAR (v1) refused by NNAPI delegate: Operator refused due performance reasons.
WARNING: Operator RESIZE_BILINEAR (v1) refused by NNAPI delegate: Operator refused due performance reasons.
Applied delegate.

tensors size: 185
nodes size: 72
inputs: 1
outputs: 1
input 0 name: sub_7 , type: 1
output 0 name: ResizeBilinear_3 , type: 1
Wanted height: 257
Wanted width: 257
Wanted channels: 3

so NPU refuse acceleration as  Operator RESIZE_BILINEAR (v1) performance reasons?

How to kill the warning?

Thanks.

[xiaodong_zhang]

@fjpmbb_abc 

  Please check if your tflite model is quantized to int8. NPU in i.MX8MP only support int8/16.

[fjpmbb_abc]

@xiaodong_zhang 

   i am using deeplabv3_257_mv_gpu.tflite download from tensoflow lite web page and targeted for mobile devices. I believe it is already quantized and optimized.

  BTW, if i want to check whether the model file is quantized? how to do it?