topic Re: IMX8QXP Machine learning demos in i.MX Processors

IMX8QXP Machine learning demos

darsh_dev — Mon, 26 Oct 2020 12:19:17 GMT

Hi Support Team,

We are performing the ML demo : Image classification example

root@imx8qxpmek:/usr/bin/tensorflow-lite-2.2.0/examples# ./label_image -m mobile net_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt Loaded model mobilenet_v1_1.0_224_quant.tflite resolved reporter invoked average time: 101.284 ms 0.780392: 653 military uniform 0.105882: 907 Windsor tie 0.0156863: 458 bow tie 0.0117647: 466 bulletproof vest 0.00784314: 835 suit root@imx8qxpmek:/usr/bin/tensorflow-lite-2.2.0/examples# ./label_image -m mobile net_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt -a 1 Loaded model mobilenet_v1_1.0_224_quant.tflite resolved reporter INFO: Created TensorFlow Lite delegate for NNAPI. Applied NNAPI delegate. W [query_hardware_caps:66]Unsupported evis version invoked average time: 98.281 ms 0.784314: 653 military uniform 0.105882: 907 Windsor tie 0.0156863: 458 bow tie 0.00784314: 466 bulletproof vest 0.00392157: 835 suit root@imx8qxpmek:/usr/bin/tensorflow-lite-2.2.0/examples#

and see the Error:

INFO: Created TensorFlow Lite delegate for NNAPI. Applied NNAPI delegate. W [query_hardware_caps:66]Unsupported evis version

and Avarage time is same with and without GPU.

Can anyone guide me, what could be the issue? Did I miss any package or configuration to add?

Thanks,

Re: IMX8QXP Machine learning demos

joanxie — Tue, 27 Oct 2020 08:41:39 GMT

what bsp version do you use? and did you test imx8qxp B0 or C0 board? did you change or build any source code for this?

Re: IMX8QXP Machine learning demos

darsh_dev — Tue, 27 Oct 2020 08:51:11 GMT

@joanxie,

We are working on L5.4.47_2.0.0 and referring the "i.MX Machine Learning User's Guide, Rev. L5.4.47_2.2.0, 30 September 2020" doc.

Our silicon revision is B0 and Doesn't changed any code/build related to ML. Mostly we did the changes related to u-boot and Kernel (Board specific interface only I2C, GPIO, USB, CSI, Codec, etc...)

Re: IMX8QXP Machine learning demos

darsh_dev — Tue, 27 Oct 2020 10:41:00 GMT

@joanxie,

One quick question, Is there any limitation for B0 and/or C0 revision specific to ML demos (specific to armNN, GPU, Tensorflow ...) ?

Can you please share the C0 and B0 revision difference from software perspective?

Re: IMX8QXP Machine learning demos

joanxie — Wed, 28 Oct 2020 04:09:27 GMT

for 5.4.47,did you built images for your B0 board? did you build "imx-image-full" image? for difference between B0 and C0, pls refer to the document as below:

"https://www.nxp.com/docs/en/application-note/AN12770.pdf"

one also can refer to the AN:

"https://www.nxp.com/docs/en/application-note/AN12867.pdf"

Re: IMX8QXP Machine learning demos

darsh_dev — Wed, 28 Oct 2020 06:31:19 GMT

@joanxie,

Yes, We are using B0- imx8qxpmek machine, find it at meta-freescale/conf/machine. and also used the imx-image-full. Not build issue found. Most of the inteferface are also working fine as expected (HDMI, Audio, CSI, I2C, GPIO and all)

This link gives error (Page Not Found) "https://www.nxp.com/docs/en/application-note/AN12770.pdf"

We would like to perform the ML related custom demo on this board. So, before that, we would like to perform the Arm NN, Tensorflow-Lite, OpenCV example from IMXMLUG doc.

FYI only

root@imx8qxpmek:/unit_tests/GPU# ./gpuinfo.sh ---- Running < gpuinfo.sh > test ---- GPU Info gpu : 0 model : 7000 revision : 6214 product : 70002 eco : 0 VIDEO MEMORY: POOL SYSTEM: Free : 267730584 B Used : 704872 B MinFree : 267730584 B MaxUsed : 704872 B Total : 268435456 B POOL VIRTUAL: Used : 0 B MaxUsed : 0 B CMA memory info cat: /sys/kernel/debug/gc/allocators/cma/cmausage: No such file or directory VidMem Usage (Process 582: weston): Current Maximum Total All-Types 655360 655360 655360 Index 0 0 0 Vertex 0 0 0 Texture 0 0 0 RenderTarget 0 0 0 Depth 0 0 0 Bitmap 0 0 0 TileStatus 0 0 0 Image 0 0 0 Mask 0 0 0 Scissor 0 0 0 HZ 0 0 0 ICache 0 0 0 TxDesc 0 0 0 Fence 0 0 0 TFBHeader 0 0 0 Command 655360 655360 655360 All-Pools 655360 655360 655360 Default 0 0 0 Local 0 0 0 Internal 0 0 0 External 0 0 0 Unified 0 0 0 System 655360 655360 655360 Sram 0 0 0 Virtual 0 0 0 User 0 0 0 Insram 0 0 0 Exsram 0 0 0 AllocNonPaged 0 0 0 AllocContiguous 655360 655360 655360 MapUserMemory 0 0 0 MapMemory 268435456 268435456 268435456 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Idle percentage:0.000.000.000.00% >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---- Test < gpuinfo.sh > ended ---- root@imx8qxpmek:/unit_tests/GPU#

Re: IMX8QXP Machine learning demos

joanxie — Wed, 28 Oct 2020 08:43:15 GMT

try to remove " in the link of anpplication note, try the link

https://www.nxp.com/docs/en/application-note/AN12770.pdf

Re: IMX8QXP Machine learning demos

darsh_dev — Wed, 28 Oct 2020 09:27:27 GMT

@joanxieThanks.

Can you please share the update on the Image classification example?

computation on CPU:

root@imx8qxpmek:/usr/bin/tensorflow-lite-2.2.0/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt >> average time: 101.284 ms

computation on the GPU/NPU hardware accelerator, add the -a 1 command line argument

root@imx8qxpmek:/usr/bin/tensorflow-lite-2.2.0/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt -a 1 INFO: Created TensorFlow Lite delegate for NNAPI. Applied NNAPI delegate. W [query_hardware_caps:66]Unsupported evis version invoked >> average time: 98.281 ms

Average time is almost same in both case.

Re: IMX8QXP Machine learning demos

darsh_dev — Thu, 05 Nov 2020 07:01:58 GMT

Further update,

I'm trying with Running benchmark applications.

root@imx8qxpmek:/usr/bin/tensorflow-lite-2.2.0/examples# ./benchmark_model --graph=efficientnet_lite0_int8_2.tflite --use_nnapi=true --enable_op_profiling=true

From the logs, computation graph was executed on CPU only Not used GPU Hardware.

GPU info:

root@imx8qxpmek:/unit_tests/GPU# ./gpuinfo.sh ---- Running < gpuinfo.sh > test ---- GPU Info gpu : 0 model : 7000 revision : 6214 product : 70002 eco : 0 VIDEO MEMORY: POOL SYSTEM: Free : 267730584 B Used : 704872 B MinFree : 253147632 B MaxUsed : 15287824 B Total : 268435456 B POOL VIRTUAL: Used : 0 B MaxUsed : 0 B CMA memory info cat: /sys/kernel/debug/gc/allocators/cma/cmausage: No such file or directory VidMem Usage (Process 415: weston): Current Maximum Total All-Types 655360 655360 655360 Index 0 0 0 Vertex 0 0 0 Texture 0 0 0 RenderTarget 0 0 0 Depth 0 0 0 Bitmap 0 0 0 TileStatus 0 0 0 Image 0 0 0 Mask 0 0 0 Scissor 0 0 0 HZ 0 0 0 ICache 0 0 0 TxDesc 0 0 0 Fence 0 0 0 TFBHeader 0 0 0 Command 655360 655360 655360 All-Pools 655360 655360 655360 Default 0 0 0 Local 0 0 0 Internal 0 0 0 External 0 0 0 Unified 0 0 0 System 655360 655360 655360 Sram 0 0 0 Virtual 0 0 0 User 0 0 0 Insram 0 0 0 Exsram 0 0 0 AllocNonPaged 0 0 0 AllocContiguous 655360 655360 655360 MapUserMemory 0 0 0 MapMemory 268435456 268435456 268435456 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Idle percentage:0.000.000.000.00% >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---- Test < gpuinfo.sh > ended ----

Re: IMX8QXP Machine learning demos

joanxie — Fri, 06 Nov 2020 03:30:23 GMT

as the gpu performance data I sent to you, which is from the gpu test team, you can refer to that

Re: IMX8QXP Machine learning demos

darsh_dev — Fri, 06 Nov 2020 09:30:50 GMT

@joanxie You have shared the GPU performance result is not matched with the IMXMLUG guide. Which result I can consider the benchmark for the IMX8QXP and IMX8Mquad?

I also checked with IMX8MQuad platform. Observed the weired behaviour
With CPU average time is 52ms and with GPU "-a 1" option average time is 102ms

root@imx8mqevk:/usr/bin/tensorflow-lite-2.2.0/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt
Loaded model mobilenet_v1_1.0_224_quant.tflite
resolved reporter
invoked
average time: 52.605 ms
0.780392: 653 military uniform
0.105882: 907 Windsor tie
0.0156863: 458 bow tie
0.0117647: 466 bulletproof vest
0.00784314: 835 suit
-----------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------

root@imx8mqevk:/usr/bin/tensorflow-lite-2.2.0/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt -a 1
Loaded model mobilenet_v1_1.0_224_quant.tflite
resolved reporter
INFO: Created TensorFlow Lite delegate for NNAPI.
Applied NNAPI delegate.
W [query_hardware_caps:66]Unsupported evis version
invoked
average time: 104.467 ms
0.784314: 653 military uniform
0.105882: 907 Windsor tie
0.0156863: 458 bow tie
0.00784314: 466 bulletproof vest
0.00392157: 835 suit

Re: IMX8QXP Machine learning demos

joanxie — Fri, 20 Nov 2020 08:22:36 GMT

yes, you can consider the data I sent to you as benchmark of imx8qxp, which is from local testing team, it seems this demo has some issues, the testing team doesn't use this command

Re: IMX8QXP Machine learning demos

lmurillo — Wed, 10 Mar 2021 02:51:28 GMT

@darsh_dev were you able to figure out what could be the problem here? I am having exactly the same weird behavior in the IMX8Mquad.

@joanxie any suggestions?

Thanks!

Re: IMX8QXP Machine learning demos

stanly-Lin — Wed, 06 Oct 2021 06:03:30 GMT

@joanxie

Dear Sir:

I encounter the same problem when I try to benchmark performance of CPU and GPU. According to your last post, it seems there's some issue with the demo.

In short, how can we assigned GPU to do eIQ applications? Seems options " -a 1 " or "USE_GPU_INFERENCE=0" is not work!

BRs

Re: IMX8QXP Machine learning demos

kandarp_rastey — Mon, 13 Dec 2021 07:03:31 GMT

+1 @joanxie

-Even I'm finding the same warning on IMX8Mquad running with "-a 1" args.

root@imx8mqevk:/usr/bin/tensorflow-lite-2.3.1/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -t 1 -i grace_hopper.bmp -l labels.txt -a 1 Loaded model mobilenet_v1_1.0_224_quant.tflite resolved reporter INFO: Created TensorFlow Lite delegate for NNAPI. Applied NNAPI delegate. W [query_hardware_caps:66]Unsupported evis version invoked average time: 104.07 ms 0.784314: 653 military uniform 0.105882: 907 Windsor tie 0.0156863: 458 bow tie 0.00784314: 466 bulletproof vest 0.00392157: 835 suit

- The NXP documentation states that the IMX8Mquad has a gpu. which implies running the same tensorflow lite's 'lable_image' example with the "-g 1" args should delegate a GPU for the inference but that isn't the case. Can someone explain the case here?

root@imx8mqevk:/usr/bin/tensorflow-lite-2.3.1/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -t 1 -i grace_hopper.bmp -l labels.txt -g 1 Loaded model mobilenet_v1_1.0_224_quant.tflite resolved reporter GPU acceleration is unsupported on this platform. invoked average time: 169.21 ms 0.780392: 653 military uniform 0.105882: 907 Windsor tie 0.0156863: 458 bow tie 0.0117647: 466 bulletproof vest 0.00784314: 835 suit

Re: IMX8QXP Machine learning demos

kandarp_rastey — Mon, 13 Dec 2021 07:22:36 GMT

+1 @joanxie.

- I'm facing the same warning on the IMX8Mquad when running the tflite's lable_image example with the "-a 1" args.

root@imx8mqevk:/usr/bin/tensorflow-lite-2.3.1/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -t 1 -i grace_hopper.bmp -l labels.txt -a 1
Loaded model mobilenet_v1_1.0_224_quant.tflite
resolved reporter
INFO: Created TensorFlow Lite delegate for NNAPI.
Applied NNAPI delegate.
W [query_hardware_caps:66]Unsupported evis version
invoked
average time: 104.07 ms
0.784314: 653 military uniform
0.105882: 907 Windsor tie
0.0156863: 458 bow tie
0.00784314: 466 bulletproof vest
0.00392157: 835 suit

- The NXP documentation states IMX8Mquad has the presence of a GPU, which implies running the tflite's lable_image example with the "-g 1" args should delegate GPU for inference but it isn't the case here. Can someone please shead some light running inference on the GPU directly?

root@imx8mqevk:/usr/bin/tensorflow-lite-2.3.1/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -t 1 -i grace_hopper.bmp -l labels.txt -g 1
Loaded model mobilenet_v1_1.0_224_quant.tflite
resolved reporter
GPU acceleration is unsupported on this platform.
invoked
average time: 169.118 ms
0.780392: 653 military uniform
0.105882: 907 Windsor tie
0.0156863: 458 bow tie
0.0117647: 466 bulletproof vest
0.00784314: 835 suit

Re: IMX8QXP Machine learning demos

jalleyne — Tue, 08 Feb 2022 01:36:15 GMT

Hi,

I'm having this same problem on a iMX8 quad using Linux version 5.10.72-2.2.0. Can you please send me the benchmark data as well as the tools that I can use to generate the data myself?

Thanks,

Jochen

Re: IMX8QXP Machine learning demos

josephzhou1 — Tue, 08 Feb 2022 02:36:42 GMT

Re: IMX8QXP Machine learning demos

jalleyne — Wed, 09 Feb 2022 15:38:27 GMT

Thanks for the response @josephzhou1 but I haven't seen anything as yet. Where can I get this information?

Jochen

@joanxie

Hi,

I'm having this same problem on a iMX8 quad using Linux version 5.10.72-2.2.0. Can you please send me the benchmark data as well as the tools that I can use to generate the data myself?

Thanks,

Jochen

Re: IMX8QXP Machine learning demos

_Alex_ — Thu, 07 Jul 2022 15:51:04 GMT

Has anybody solved this problem. I am running into the same errors of unsupported evis version. This is on IMX8 Plus with LF5.10.52-2.1.0.

/usr/bin/tensorflow-lite-2.5.0/examples# USE_GPU_INFERENCE=0 ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hop
INFO: Loaded model mobilenet_v1_1.0_224_quant.tflite
INFO: resolved reporter
INFO: Created TensorFlow Lite delegate for NNAPI.
INFO: Use NNAPI acceleration.
INFO: Applied NNAPI delegate.
W [query_hardware_caps:66]Unsupported evis version
INFO: invoked
INFO: average time: 162.426 ms
INFO: 0.784314: 653 military uniform
INFO: 0.105882: 907 Windsor tie
INFO: 0.0156863: 458 bow tie
INFO: 0.00784314: 466 bulletproof vest
INFO: 0.00392157: 835 suit

@joanxie