i.MX8M+ NPU poor performance than CPU

取消
显示结果 
显示  仅  | 搜索替代 
您的意思是: 

i.MX8M+ NPU poor performance than CPU

1,623 次查看
tiffanywang
Contributor II

SW: imx-yocto-5.4.70-2.3.0

HW: i.MX8M Plus

We built an image segmentation model using onnx.

The average inference time is 2.77394s on CPU per frame. The performance drops to 21.6163s if VSI NPU EP is used instead.

Attached the files:

- main.cpp: inference on CPU

- main_npu.cpp: inference on NPU

- video.mp4: test video

- model.onnx: image segmentation model

标签 (1)
标记 (2)
8 回复数

721 次查看
hansmoleman007
Contributor I
0 项奖励

1,210 次查看
JACK_CHA
Contributor IV

Dears

Can you tell me how was going on?

The same issue is raised by my cusotmer. 

Should they build the code by enabling the execution provider before the cpu/gpu/npu test respectivly?

Regards, 

Jack

0 项奖励

1,582 次查看
joanxie
NXP TechSupport
NXP TechSupport

what's your test steps? how did you enable NPU, pls send the steps to me, let me reproduce this on my imx8mp board

0 项奖励

1,552 次查看
tiffanywang
Contributor II

Just build the code and run it. It reads video file and performs segmentation.

As for enabling NPU, it is to add VSI NPU EP which list in IMX-MACHINE-LEARNING-UG.pdf. You can refer to my main_npu.cpp.

 

0 项奖励

1,537 次查看
joanxie
NXP TechSupport
NXP TechSupport

do you mean NPU performance is worse than CPU? it seems that vsi npu calls imx8mp gpu, I have a table about gpu and npu:

 

joanxie_0-1618028373689.png

 

 

0 项奖励

1,476 次查看
tiffanywang
Contributor II

You can reproduce it to see the issue.

0 项奖励

1,480 次查看
tiffanywang
Contributor II
You can reproduce it to see the issue.
0 项奖励

1,466 次查看
joanxie
NXP TechSupport
NXP TechSupport

send me the detailed your test steps to reproduce this

0 项奖励