i.MX8M+ NPU poor performance than CPU

cancel
Showing results for 
Search instead for 
Did you mean: 

i.MX8M+ NPU poor performance than CPU

239 Views
tiffanywang
Contributor II

SW: imx-yocto-5.4.70-2.3.0

HW: i.MX8M Plus

We built an image segmentation model using onnx.

The average inference time is 2.77394s on CPU per frame. The performance drops to 21.6163s if VSI NPU EP is used instead.

Attached the files:

- main.cpp: inference on CPU

- main_npu.cpp: inference on NPU

- video.mp4: test video

- model.onnx: image segmentation model

Labels (1)
Tags (2)
0 Kudos
6 Replies

198 Views
joanxie
NXP TechSupport
NXP TechSupport

what's your test steps? how did you enable NPU, pls send the steps to me, let me reproduce this on my imx8mp board

0 Kudos

168 Views
tiffanywang
Contributor II

Just build the code and run it. It reads video file and performs segmentation.

As for enabling NPU, it is to add VSI NPU EP which list in IMX-MACHINE-LEARNING-UG.pdf. You can refer to my main_npu.cpp.

 

0 Kudos

153 Views
joanxie
NXP TechSupport
NXP TechSupport

do you mean NPU performance is worse than CPU? it seems that vsi npu calls imx8mp gpu, I have a table about gpu and npu:

 

joanxie_0-1618028373689.png

 

 

0 Kudos

92 Views
tiffanywang
Contributor II

You can reproduce it to see the issue.

0 Kudos

96 Views
tiffanywang
Contributor II
You can reproduce it to see the issue.
0 Kudos

82 Views
joanxie
NXP TechSupport
NXP TechSupport

send me the detailed your test steps to reproduce this

0 Kudos