HW: i.MX8M Plus
We built an image segmentation model using onnx.
The average inference time is 2.77394s on CPU per frame. The performance drops to 21.6163s if VSI NPU EP is used instead.
Attached the files:
- main.cpp: inference on CPU
- main_npu.cpp: inference on NPU
- video.mp4: test video
- model.onnx: image segmentation model
Can you tell me how was going on?
The same issue is raised by my cusotmer.
Should they build the code by enabling the execution provider before the cpu/gpu/npu test respectivly?
Just build the code and run it. It reads video file and performs segmentation.
As for enabling NPU, it is to add VSI NPU EP which list in IMX-MACHINE-LEARNING-UG.pdf. You can refer to my main_npu.cpp.