i.MX8M+ NPU poor performance than CPU

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

i.MX8M+ NPU poor performance than CPU

1,664 Views
tiffanywang
Contributor II

SW: imx-yocto-5.4.70-2.3.0

HW: i.MX8M Plus

We built an image segmentation model using onnx.

The average inference time is 2.77394s on CPU per frame. The performance drops to 21.6163s if VSI NPU EP is used instead.

Attached the files:

- main.cpp: inference on CPU

- main_npu.cpp: inference on NPU

- video.mp4: test video

- model.onnx: image segmentation model

Labels (1)
Tags (2)
8 Replies

762 Views
hansmoleman007
Contributor I
0 Kudos

1,251 Views
JACK_CHA
Contributor IV

Dears

Can you tell me how was going on?

The same issue is raised by my cusotmer. 

Should they build the code by enabling the execution provider before the cpu/gpu/npu test respectivly?

Regards, 

Jack

0 Kudos

1,623 Views
joanxie
NXP TechSupport
NXP TechSupport

what's your test steps? how did you enable NPU, pls send the steps to me, let me reproduce this on my imx8mp board

0 Kudos

1,593 Views
tiffanywang
Contributor II

Just build the code and run it. It reads video file and performs segmentation.

As for enabling NPU, it is to add VSI NPU EP which list in IMX-MACHINE-LEARNING-UG.pdf. You can refer to my main_npu.cpp.

 

0 Kudos

1,578 Views
joanxie
NXP TechSupport
NXP TechSupport

do you mean NPU performance is worse than CPU? it seems that vsi npu calls imx8mp gpu, I have a table about gpu and npu:

 

joanxie_0-1618028373689.png

 

 

0 Kudos

1,517 Views
tiffanywang
Contributor II

You can reproduce it to see the issue.

0 Kudos

1,521 Views
tiffanywang
Contributor II
You can reproduce it to see the issue.
0 Kudos

1,507 Views
joanxie
NXP TechSupport
NXP TechSupport

send me the detailed your test steps to reproduce this

0 Kudos