[AN14104] How to improve the performance of the stereo matching algo on iMX8M Plus ?

[khang_letruong]

Dear Community,

Recently it appeared the application note AN14104 - i.MX 8M Plus Stereo Vision that draws much of our attention as we have been developing an industrial stereo camera based on the aforementioned platform since its beginning. One of our ideas was doing some  on-board processing such as on-device depth mapping but we did not really succeed due to the poor performance of the algo running on CPU only. And this also impacted our business a little bit as the depth map computation is a basic feature of our targeted applications.Our conclusion was that the iMX8M Plus was more suitable for a smart camera with certain AI deep learning features together with the real-time and distributed inference with multiple video inputs thanks to its NPU rather than a performant stereo camera.

By looking at the above application note, it looks like that there's still be similar result in chapter 7:

The image processing and algorithm runs on the CPU implying that it is a CPU intensive example. This example can result in 1 to 2 disparity maps per second.

I wonder how one could improve this performance for being useful in reality as we did attempt with the existing OpenVX Stereo demo or even ask for the professional service for the possibility to use the hardware acceleration such as GPU for heavily computational tasks but the answer seemed to be that the feature is not supported.

Thanks in advance and best regards,

Khang

[Zhiming_Liu]

Hi @khang_letruong 

stereo match function in OpenCV with GPU support needs NXP consume a lot of labor to support. This is one point that we can't support it from business side. It's very hard for NXP to support GPU feature in like CUDA. We need add new gpu layer in OpenCV, but there is not enough business or customers need this. 

[khang_letruong]

Hi @Zhiming_Liu ,

Thanks for clarifying the reason. I could also guess that there hasn't been enough business interest on this subject. I just tried to find some practical use-case for the performance of 1 to 2 disparity maps per second in reality.

By the way, the idea of the stereo matching function in OpenCV with GPU support comes from the fact that it was the case of the OpenVX Stereo Demo using the GPU of a certain iMX8 (possibly iMX8QM) with the EVIS instruction. Meanwhile, there's EVIS2 instruction that is supported by iMX8MP and that why I thought it could be more or less straight-forward to have a similar demo with better performance on iMX8MP.

Best Regards,

Khang

[Zhiming_Liu]

Hi @khang_letruong 

It seems that OpenCV has supported the OpenVX. There is one test case about OpenVX: modules/imgproc/test/test_canny.cpp

The test code use setUseOpenVX(true) to enable OpenVX in OpenCV. You can test it with stereo match function. I didn't verify it until now.

Best Regards

Zhiming

[khang_letruong]

Hi @Zhiming_Liu,

Thanks for the info. But it might not be hardware accelerated when running on the selected platform (which is the iMX8M Plus). Do you think so ?

Regards,

Khang

[Zhiming_Liu]

Hi @khang_letruong 

But it might not be hardware accelerated when running on the selected platform (which is the iMX8M Plus). Do you think so ?

-->This depends on OpenCV framework design, the OpenVX hal layer in OpenCV  is using OpenVX-1.x C API. I think the hardware acceleration will work on i.MX8MP.

https://github.com/opencv/opencv/tree/4.x/3rdparty/openvx

Best Regards

Zhiming