OpenCV and GStreamer on iMX8QM MEK

[sebadavalle]

Hi I need help capturing MP4 videos using openCV and GSTREAMER.

Environment:
Board: i.MX8 Quad Max MEK
Linux: NXP i.MX Release Distro 5.4-zeus imx8qmmek 

This is the way I'm playing back MP4 the videos:

gst-launch-1.0 filesrc location=/home/LH.mp4 ! video/quicktime ! aiurdemux ! queue ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! queue ! waylandsink 

So based on that I built the following pipeline for VideoCapture function:

cv::VideoCapture LHcap("filesrc location=/home/LH.mp4 ! video/quicktime ! aiurdemux ! queue ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! queue ! appsink", cv::CAP_GSTREAMER)

But, the frames captured has only 2 channel and all OpenCV conversion functions fails.

cv::Mat img;
LHcap >> img;
cout <<  img.channels() << endl;

So i tried another pipeline

filesrc location=/home/LH.mp4 ! decodebin ! videorate ! video/x-raw,framerate=1/1 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink

using this pipeline, the captured images have 3 channels and the type is well recognized (CV_8UC3), BUT, if I save one of the frames, I can see the image is broken

(See attached)

Does anybody knows how to capture mp4 videos using opencv and GSREAMER?

Thanks

Seba.-

[sebadavalle · Accepted solution]

I was able to make it work..

const std::string pipeline = "filesrc location=/home/videos/RH-yuv420-slow.mp4 ! video/quicktime ! aiurdemux ! queue ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! capsfilter caps=\"video/x-raw,format=UYVY,width=1392,height=512\" ! videoconvert ! queue ! appsink";

cv::VideoCapture cap(pipeline, cv::CAP_GSTREAMER);

The key part is: 

! capsfilter caps=\"video/x-raw,format=UYVY,width=1392,height=512\" ! videoconvert

were you are specifying the capture format.

On the other had, for USB camera capture... you need to specify the  device ID (you can check the device ID using v4l2-ctl --list-devices ) and the preferred Capture API backends to use; in this case cv::CAP_GSTREAMER.

const int camID = 3;
cv::VideoCapture CAMcap(camID, cv::CAP_GSTREAMER);