i.mx8mm opencv gpu acceleration

[erhany]

We are using NXP official BSP 5.4.47_2.0.0 where we have gstreamer and opencv 4. We are using xwayland distribution with weston. Writing the app with QT.

We have an RTSP stream where we want to show it on screen after applying some modifications.

If we play the stream with gst-launch everything runs smoothly. But when we want to use opencv (with gstreamer enabled) we cannot show the video smoothly on screen.

When using gstlaunch with: "rtspsrc latency=0 location=rtsp://192.168.1.3:8554/sample caps=\"application/x-rtp\" ! rtph264depay! h264parse ! queue ! vpudec ! videoconvert ! waylandsink sync=false" everything is fine.

In opencv we use the same pipeline like below:

"rtspsrc buffer-mode=0 do-retransmission=false latency=0 location=rtsp://192.168.1.3:8554/sample caps=\"application/x-rtp\" ! rtph264depay! h264parse ! queue ! vpudec ! videoconvert ! appsink sync=false"

But in this case we can only process 2-3 frames per second even we only capture the mat and send to a QOpenGLWidget. Nothing changes event we don't show anything on screen. The cpu usage is high so probably gpu acceleration is not active. 

Gstreamer is active in opencv. We also tried to compile opencv with WITH_OPENGL=on parameter but nothing has changed. 

What is the best way to use opencv in i.mx8mm? Here is a small code snippet showing what we have done:

cv::VideoCapture cam1{};

cam1.open(STREAM1,cv::CAP_GSTREAMER)

if(cap->read(frame) && !frame.empty()){

    ui->glWidget->setQImage(QImage(frame.data, frame.cols, frame.rows, frame.step, QImage::Format_RGB888));

}

Sample source can be found in https://tinyurl.com/2p8wacy6

[Zhiming_Liu]

Do you try producer and consumer model in your code? This can improve fps issue.

[erhany]

This does not help. In the following link an NXP employee has told that "You can try to use gstreamer, we don't import hardware acceleration in opencv ". Is it true?

https://community.nxp.com/t5/i-MX-Processors/i-MX-8-opencv-videocapture-very-slow-video-file/td-p/12...

[mf_]

@Zhiming_Liu, any updates on how to use OpenCV on the GPU? (For this use case or another like using the cv::BackgroundSubtractorMOG2 algorithm on the GPU)