IMX8QXP Machine learning demos

[darsh_dev]

Hi Support Team,

We are performing the ML demo : Image classification example

root@imx8qxpmek:/usr/bin/tensorflow-lite-2.2.0/examples# ./label_image -m mobile                                                                             net_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt
Loaded model mobilenet_v1_1.0_224_quant.tflite
resolved reporter
invoked
average time: 101.284 ms
0.780392: 653 military uniform
0.105882: 907 Windsor tie
0.0156863: 458 bow tie
0.0117647: 466 bulletproof vest
0.00784314: 835 suit

root@imx8qxpmek:/usr/bin/tensorflow-lite-2.2.0/examples# ./label_image -m mobile                                                                             net_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt -a 1
Loaded model mobilenet_v1_1.0_224_quant.tflite
resolved reporter
INFO: Created TensorFlow Lite delegate for NNAPI.
Applied NNAPI delegate.
W [query_hardware_caps:66]Unsupported evis version
invoked
average time: 98.281 ms
0.784314: 653 military uniform
0.105882: 907 Windsor tie
0.0156863: 458 bow tie
0.00784314: 466 bulletproof vest
0.00392157: 835 suit
root@imx8qxpmek:/usr/bin/tensorflow-lite-2.2.0/examples#

and see the Error:

INFO: Created TensorFlow Lite delegate for NNAPI.
Applied NNAPI delegate.
W [query_hardware_caps:66]Unsupported evis version

 and Avarage time is same with and without GPU.

Can anyone guide me, what could be the issue? Did I miss any package or configuration to add?

Thanks,

[_Alex_]

Has anybody solved this problem.  I am running into the same errors of unsupported evis version. This is on IMX8 Plus with LF5.10.52-2.1.0. 

/usr/bin/tensorflow-lite-2.5.0/examples# USE_GPU_INFERENCE=0 ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hop
INFO: Loaded model mobilenet_v1_1.0_224_quant.tflite
INFO: resolved reporter
INFO: Created TensorFlow Lite delegate for NNAPI.
INFO: Use NNAPI acceleration.
INFO: Applied NNAPI delegate.
W [query_hardware_caps:66]Unsupported evis version
INFO: invoked
INFO: average time: 162.426 ms
INFO: 0.784314: 653 military uniform
INFO: 0.105882: 907 Windsor tie
INFO: 0.0156863: 458 bow tie
INFO: 0.00784314: 466 bulletproof vest
INFO: 0.00392157: 835 suit

@joanxie 

[jalleyne]

Thanks for the response @josephzhou1 but I haven't seen anything as yet.  Where can I get this information?

Jochen

@joanxie

Hi,

I'm having this same problem on a iMX8 quad using  Linux version 5.10.72-2.2.0.  Can you please send me the benchmark data as well as the tools that I can use to generate the data myself?

Thanks,

Jochen

[josephzhou1]

ok

[jalleyne]

Hi,

I'm having this same problem on a iMX8 quad using  Linux version 5.10.72-2.2.0.  Can you please send me the benchmark data as well as the tools that I can use to generate the data myself?

Thanks,

Jochen

[kandarp_rastey]

+1 @joanxie.

- I'm facing the same warning on the IMX8Mquad when running the tflite's lable_image example with the "-a 1" args.

root@imx8mqevk:/usr/bin/tensorflow-lite-2.3.1/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -t 1 -i grace_hopper.bmp -l labels.txt -a 1
Loaded model mobilenet_v1_1.0_224_quant.tflite
resolved reporter
INFO: Created TensorFlow Lite delegate for NNAPI.
Applied NNAPI delegate.
W [query_hardware_caps:66]Unsupported evis version
invoked
average time: 104.07 ms
0.784314: 653 military uniform
0.105882: 907 Windsor tie
0.0156863: 458 bow tie
0.00784314: 466 bulletproof vest
0.00392157: 835 suit

 - The NXP documentation states IMX8Mquad has the presence of a GPU, which implies running the tflite's lable_image example with the "-g 1" args should delegate GPU for inference but it isn't the case here. Can someone please shead some light running inference on the GPU directly?

root@imx8mqevk:/usr/bin/tensorflow-lite-2.3.1/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -t 1 -i grace_hopper.bmp -l labels.txt -g 1
Loaded model mobilenet_v1_1.0_224_quant.tflite
resolved reporter
GPU acceleration is unsupported on this platform.
invoked
average time: 169.118 ms
0.780392: 653 military uniform
0.105882: 907 Windsor tie
0.0156863: 458 bow tie
0.0117647: 466 bulletproof vest
0.00784314: 835 suit

[kandarp_rastey]

+1 @joanxie

-Even I'm finding the same warning on IMX8Mquad running with "-a 1" args.

root@imx8mqevk:/usr/bin/tensorflow-lite-2.3.1/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -t 1 -i grace_hopper.bmp -l labels.txt -a 1
Loaded model mobilenet_v1_1.0_224_quant.tflite
resolved reporter
INFO: Created TensorFlow Lite delegate for NNAPI.
Applied NNAPI delegate.
W [query_hardware_caps:66]Unsupported evis version
invoked
average time: 104.07 ms
0.784314: 653 military uniform
0.105882: 907 Windsor tie
0.0156863: 458 bow tie
0.00784314: 466 bulletproof vest
0.00392157: 835 suit

- The NXP documentation states that the IMX8Mquad has a gpu. which implies running the same tensorflow lite's 'lable_image' example with the "-g 1" args should delegate a GPU for the inference but that isn't the case. Can someone explain the case here?

root@imx8mqevk:/usr/bin/tensorflow-lite-2.3.1/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -t 1 -i grace_hopper.bmp -l labels.txt -g 1
Loaded model mobilenet_v1_1.0_224_quant.tflite
resolved reporter
GPU acceleration is unsupported on this platform.
invoked
average time: 169.21 ms
0.780392: 653 military uniform
0.105882: 907 Windsor tie
0.0156863: 458 bow tie
0.0117647: 466 bulletproof vest
0.00784314: 835 suit

[stanly-Lin]

@joanxie 

Dear Sir:

             I encounter the same problem when I try to benchmark performance of CPU and GPU. According to your last post, it seems there's some  issue with the demo. 

             In short, how can we assigned GPU to do eIQ applications? Seems options " -a 1 " or "USE_GPU_INFERENCE=0" is not work!

BRs

[joanxie]

what bsp version do you use? and did you test imx8qxp B0 or C0 board? did you change or build any source code for this?

[darsh_dev]

@joanxie,

We are working on L5.4.47_2.0.0 and referring the "i.MX Machine Learning User's Guide, Rev. L5.4.47_2.2.0, 30 September 2020" doc. 

Our silicon revision is B0 and Doesn't changed any code/build related to ML. Mostly we did the changes related to u-boot and Kernel (Board specific interface only I2C, GPIO, USB, CSI, Codec, etc...)

[darsh_dev]

@joanxie,

One quick question, Is there any limitation for B0 and/or C0 revision specific to ML demos (specific to armNN, GPU, Tensorflow ...) ?

Can you please share the C0 and B0 revision difference from software perspective?

[joanxie]

for 5.4.47,did you built images for your B0 board? did you build "imx-image-full" image? for difference between B0 and C0, pls refer to the document as below:

"https://www.nxp.com/docs/en/application-note/AN12770.pdf"

one also can refer to the AN:

"https://www.nxp.com/docs/en/application-note/AN12867.pdf"

[darsh_dev]

@joanxie,

Yes, We are using B0- imx8qxpmek machine, find it at meta-freescale/conf/machine. and also used the imx-image-full. Not build issue found. Most of the inteferface are also working fine as expected (HDMI, Audio, CSI, I2C, GPIO and all)

This link gives error (Page Not Found) "https://www.nxp.com/docs/en/application-note/AN12770.pdf"

We would like to perform the ML related custom demo on this board. So, before that, we would like to perform the Arm NN, Tensorflow-Lite, OpenCV example from IMXMLUG doc.

FYI only

root@imx8qxpmek:/unit_tests/GPU# ./gpuinfo.sh 
---- Running < gpuinfo.sh > test ----
GPU Info
gpu      : 0
model    : 7000
revision : 6214
product  : 70002
eco      :    0
VIDEO MEMORY:
  POOL SYSTEM:
    Free :     267730584 B
    Used :        704872 B
    MinFree :  267730584 B
    MaxUsed :     704872 B
    Total :    268435456 B
  POOL VIRTUAL:
    Used :             0 B
    MaxUsed :          0 B
CMA memory info
cat: /sys/kernel/debug/gc/allocators/cma/cmausage: No such file or directory
VidMem Usage (Process 582: weston):
                          Current          Maximum            Total
All-Types                  655360           655360           655360
Index                           0                0                0
Vertex                          0                0                0
Texture                         0                0                0
RenderTarget                    0                0                0
Depth                           0                0                0
Bitmap                          0                0                0
TileStatus                      0                0                0
Image                           0                0                0
Mask                            0                0                0
Scissor                         0                0                0
HZ                              0                0                0
ICache                          0                0                0
TxDesc                          0                0                0
Fence                           0                0                0
TFBHeader                       0                0                0
Command                    655360           655360           655360
All-Pools                  655360           655360           655360
Default                         0                0                0
Local                           0                0                0
Internal                        0                0                0
External                        0                0                0
Unified                         0                0                0
System                     655360           655360           655360
Sram                            0                0                0
Virtual                         0                0                0
User                            0                0                0
Insram                          0                0                0
Exsram                          0                0                0
AllocNonPaged                   0                0                0
AllocContiguous            655360           655360           655360
MapUserMemory                   0                0                0
MapMemory               268435456        268435456        268435456
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Idle percentage:0.000.000.000.00%
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
----  Test < gpuinfo.sh > ended  ----
root@imx8qxpmek:/unit_tests/GPU# 

[joanxie]

try to remove " in the link of anpplication note, try the link

https://www.nxp.com/docs/en/application-note/AN12770.pdf

[darsh_dev]

@joanxieThanks.

Can you please share the update on the Image classification example?

computation on CPU:

root@imx8qxpmek:/usr/bin/tensorflow-lite-2.2.0/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt

>> average time: 101.284 ms

computation on the GPU/NPU hardware accelerator, add the -a 1 command line argument

root@imx8qxpmek:/usr/bin/tensorflow-lite-2.2.0/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt -a 1

INFO: Created TensorFlow Lite delegate for NNAPI.
Applied NNAPI delegate.
W [query_hardware_caps:66]Unsupported evis version
invoked
>> average time: 98.281 ms

Average time is almost same in both case.

[darsh_dev]

Further update,

I'm trying with Running benchmark applications.

root@imx8qxpmek:/usr/bin/tensorflow-lite-2.2.0/examples# ./benchmark_model --graph=efficientnet_lite0_int8_2.tflite --use_nnapi=true --enable_op_profiling=true

From the logs, computation graph was executed on CPU only Not used GPU Hardware.

GPU info:

root@imx8qxpmek:/unit_tests/GPU# ./gpuinfo.sh

---- Running < gpuinfo.sh > test ----
GPU Info
gpu      : 0
model    : 7000
revision : 6214
product  : 70002
eco      :    0

VIDEO MEMORY:
  POOL SYSTEM:
    Free :     267730584 B
    Used :        704872 B
    MinFree :  253147632 B
    MaxUsed :   15287824 B
    Total :    268435456 B
  POOL VIRTUAL:
    Used :             0 B
    MaxUsed :          0 B
CMA memory info
cat: /sys/kernel/debug/gc/allocators/cma/cmausage: No such file or directory
VidMem Usage (Process 415: weston):
                          Current          Maximum            Total
All-Types                  655360           655360           655360
Index                           0                0                0
Vertex                          0                0                0
Texture                         0                0                0
RenderTarget                    0                0                0
Depth                           0                0                0
Bitmap                          0                0                0
TileStatus                      0                0                0
Image                           0                0                0
Mask                            0                0                0
Scissor                         0                0                0
HZ                              0                0                0
ICache                          0                0                0
TxDesc                          0                0                0
Fence                           0                0                0
TFBHeader                       0                0                0
Command                    655360           655360           655360

All-Pools                  655360           655360           655360
Default                         0                0                0
Local                           0                0                0
Internal                        0                0                0
External                        0                0                0
Unified                         0                0                0
System                     655360           655360           655360
Sram                            0                0                0
Virtual                         0                0                0
User                            0                0                0
Insram                          0                0                0
Exsram                          0                0                0

AllocNonPaged                   0                0                0
AllocContiguous            655360           655360           655360
MapUserMemory                   0                0                0
MapMemory               268435456        268435456        268435456

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Idle percentage:0.000.000.000.00%
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

----  Test < gpuinfo.sh > ended  ----

[joanxie]

as the gpu performance data I sent to you, which is from the gpu test team, you can refer to that

[darsh_dev]

@joanxie You have shared the GPU performance result is not matched with the IMXMLUG guide. Which result I can consider the benchmark for the IMX8QXP and IMX8Mquad?

I also checked with IMX8MQuad platform. Observed the weired behaviour
With CPU average time is 52ms and with GPU "-a 1" option average time is 102ms

root@imx8mqevk:/usr/bin/tensorflow-lite-2.2.0/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt
Loaded model mobilenet_v1_1.0_224_quant.tflite
resolved reporter
invoked
average time: 52.605 ms
0.780392: 653 military uniform
0.105882: 907 Windsor tie
0.0156863: 458 bow tie
0.0117647: 466 bulletproof vest
0.00784314: 835 suit
-----------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------

root@imx8mqevk:/usr/bin/tensorflow-lite-2.2.0/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt -a 1
Loaded model mobilenet_v1_1.0_224_quant.tflite
resolved reporter
INFO: Created TensorFlow Lite delegate for NNAPI.
Applied NNAPI delegate.
W [query_hardware_caps:66]Unsupported evis version
invoked
average time: 104.467 ms
0.784314: 653 military uniform
0.105882: 907 Windsor tie
0.0156863: 458 bow tie
0.00784314: 466 bulletproof vest
0.00392157: 835 suit

[lmurillo]

@darsh_dev were you able to figure out what could be the problem here? I am having exactly the same weird behavior in the IMX8Mquad.

@joanxie any suggestions?

Thanks!

[joanxie]

yes, you can consider the data I sent to you as benchmark of  imx8qxp, which is from local testing team, it seems this demo has some issues, the testing team doesn't use this command