i.mx8mm opencv gpu acceleration

erhany — Tue, 08 Mar 2022 06:24:00 GMT

We are using NXP official BSP 5.4.47_2.0.0 where we have gstreamer and opencv 4. We are using xwayland distribution with weston. Writing the app with QT.

We have an RTSP stream where we want to show it on screen after applying some modifications.

If we play the stream with gst-launch everything runs smoothly. But when we want to use opencv (with gstreamer enabled) we cannot show the video smoothly on screen.

When using gstlaunch with: "rtspsrc latency=0 location=rtsp://192.168.1.3:8554/sample caps=\"application/x-rtp\" ! rtph264depay! h264parse ! queue ! vpudec ! videoconvert ! waylandsink sync=false" everything is fine.

In opencv we use the same pipeline like below:

"rtspsrc buffer-mode=0 do-retransmission=false latency=0 location=rtsp://192.168.1.3:8554/sample caps=\"application/x-rtp\" ! rtph264depay! h264parse ! queue ! vpudec ! videoconvert ! appsink sync=false"

But in this case we can only process 2-3 frames per second even we only capture the mat and send to a QOpenGLWidget. Nothing changes event we don't show anything on screen. The cpu usage is high so probably gpu acceleration is not active.

Gstreamer is active in opencv. We also tried to compile opencv with WITH_OPENGL=on parameter but nothing has changed.

What is the best way to use opencv in i.mx8mm? Here is a small code snippet showing what we have done:

cv::VideoCapture cam1{}; cam1.open(STREAM1,cv::CAP_GSTREAMER) if(cap->read(frame) && !frame.empty()){ ui->glWidget->setQImage(QImage(frame.data, frame.cols, frame.rows, frame.step, QImage::Format_RGB888)); }

Sample source can be found in https://tinyurl.com/2p8wacy6

Re: i.mx8mm opencv gpu acceleration

Zhiming_Liu — Thu, 10 Mar 2022 02:05:02 GMT

Do you try producer and consumer model in your code? This can improve fps issue.

Re: i.mx8mm opencv gpu acceleration

erhany — Mon, 14 Mar 2022 11:00:44 GMT

This does not help. In the following link an NXP employee has told that "You can try to use gstreamer, we don't import hardware acceleration in opencv ". Is it true?

https://community.nxp.com/t5/i-MX-Processors/i-MX-8-opencv-videocapture-very-slow-video-file/td-p/1258675

Re: i.mx8mm opencv gpu acceleration

mf_ — Mon, 18 Mar 2024 11:44:24 GMT

@Zhiming_Liu, any updates on how to use OpenCV on the GPU? (For this use case or another like using the cv::BackgroundSubtractorMOG2 algorithm on the GPU)

topic Re: i.mx8mm opencv gpu acceleration in i.MX Processors

i.mx8mm opencv gpu acceleration

Re: i.mx8mm opencv gpu acceleration

Re: i.mx8mm opencv gpu acceleration

Re: i.mx8mm opencv gpu acceleration