topic Re: i.MX8M+ NPU poor performance than CPU in i.MX Processors

i.MX8M+ NPU poor performance than CPU

tiffanywang — Tue, 30 Mar 2021 08:09:22 GMT

SW: imx-yocto-5.4.70-2.3.0

HW: i.MX8M Plus

We built an image segmentation model using onnx.

The average inference time is 2.77394s on CPU per frame. The performance drops to 21.6163s if VSI NPU EP is used instead.

Attached the files:

- main.cpp: inference on CPU

- main_npu.cpp: inference on NPU

- video.mp4: test video

- model.onnx: image segmentation model

joanxie — Wed, 31 Mar 2021 07:38:53 GMT

what's your test steps? how did you enable NPU, pls send the steps to me, let me reproduce this on my imx8mp board

tiffanywang — Fri, 09 Apr 2021 06:13:30 GMT

Just build the code and run it. It reads video file and performs segmentation.

As for enabling NPU, it is to add VSI NPU EP which list in IMX-MACHINE-LEARNING-UG.pdf. You can refer to my main_npu.cpp.

joanxie — Sat, 10 Apr 2021 04:19:40 GMT

do you mean NPU performance is worse than CPU? it seems that vsi npu calls imx8mp gpu, I have a table about gpu and npu:

tiffanywang — Thu, 22 Apr 2021 00:58:56 GMT

You can reproduce it to see the issue.

tiffanywang — Thu, 22 Apr 2021 01:02:56 GMT

You can reproduce it to see the issue.

joanxie — Thu, 22 Apr 2021 02:58:12 GMT

send me the detailed your test steps to reproduce this

JK-IMX — Fri, 25 Feb 2022 08:00:22 GMT

Dears

Can you tell me how was going on?

The same issue is raised by my cusotmer.

Should they build the code by enabling the execution provider before the cpu/gpu/npu test respectivly?

Regards,

Jack

hansmoleman007 — Wed, 11 Jan 2023 20:15:39 GMT