Gstreamer imxvideoconvert_g2d capfilter NV12 fail

thepnimit — Thu, 02 Sep 2021 07:14:37 GMT

Hi NXP,

With linux version 5.10 on IMX8QXP I have problem with gstreamer about capfilter

the refference from thread https://community.nxp.com/t5/i-MX-Processors/How-to-convert-simultaneously-two-video-streams-from-YUY2-to/m-p/1200108

When I need to use 2 camera device for streaming I found the same problem at v4l2convert

gst-launch-1.0 -v -e v4l2src device=/dev/video3 \
! "image/jpeg, width=3264, height=2448" \
! v4l2jpegdec ! imxvideoconvert_g2d \
! capsfilter caps="video/x-raw, width=320, height=240, pixel-aspect-ratio=1/1" \
! v4l2convert ! v4l2h264enc ! h264parse \
! fakesink

gst-launch-1.0 -v -e v4l2src device=/dev/video5 \
! "image/jpeg, width=3264, height=2448" \
! v4l2jpegdec ! imxvideoconvert_g2d \
! capsfilter caps="video/x-raw, width=320, height=240, pixel-aspect-ratio=1/1" \
! v4l2convert ! v4l2h264enc ! h264parse \
! fakesink

The ISI channel cann't control and show error log from camera number2

mxc-isi 58100000.isi: mxc_isi_m2m_open: ISI channel[0] is busy

I try to fix this problem with doesn't use v4l2convert.

On top command log generate format auto

camera(JPEG)->v4l2jpegdec(NV12)->imxvideoconvert_g2d(YUY2)->v4l2conver(NV12)->v4l2h264

If I set capfilter imxvideoconvert_g2d to use NV12, that's mean I don't need to use v4l2convert right?

from gst-inspect-1.0 imxvideoconvert_g2d I get log

Factory Details:
Rank primary + 1 (257)
Long-name IMX g2d Video Converter
Klass Filter/Converter/Video
Description Video CSC/Resize/Rotate.
Author Multimedia Team <shmmmw@freescale.com>

Plugin Details:
Name imxvideoconvert
Description IMX Video Convert Plugins
Filename /usr/lib/gstreamer-1.0/libgstimxvideoconvert.so
Version 4.6.1
License LGPL
Source module imx-gst1.0-plugin
Binary package Freescle Gstreamer Multimedia Plugins
Origin URL http://www.freescale.com

GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstBaseTransform
+----GstVideoFilter
+----imxvideoconvert_g2d

Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR, (string)UYVY, (string)YUY2, (string)I420, (string)NV12_10LE, (string)YV12, (string)NV16, (string)NV21, (string)NV12 }
video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR, (string)UYVY, (string)YUY2, (string)I420, (string)NV12_10LE, (string)YV12, (string)NV16, (string)NV21, (string)NV12 }

SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR, (string)UYVY, (string)YUY2, (string)NV12 }
video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR, (string)UYVY, (string)YUY2, (string)NV12 }

imxvideoconvert_g2d has capfilter support NV12 but when I try command

gst-launch-1.0 -v -e v4l2src device=/dev/video3 \
! "image/jpeg, width=3264, height=2448" \
! v4l2jpegdec ! imxvideoconvert_g2d \
! capsfilter caps="video/x-raw, format=NV12, width=320, height=240, pixel-aspect-ratio=1/1" \
! fakesink

I found error

====== V4L2DEC: 1.18.0 build on Jun 3 2021 08:59:40. ======
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = image/jpeg, width=(int)3264, height=(int)2448, framerate=(fraction)15/1, pixel-aspect-ratio=(fraction)1/1, parsed=(boolean)true, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = image/jpeg, width=(int)3264, height=(int)2448, framerate=(fraction)15/1, pixel-aspect-ratio=(fraction)1/1, parsed=(boolean)true, interlace-mode=(string)progressive
/GstPipeline:pipeline0/v4l2jpegdec:v4l2jpegdec0.GstPad:sink: caps = image/jpeg, width=(int)3264, height=(int)2448, framerate=(fraction)15/1, pixel-aspect-ratio=(fraction)1/1, parsed=(boolean)true, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = image/jpeg, width=(int)3264, height=(int)2448, framerate=(fraction)15/1, pixel-aspect-ratio=(fraction)1/1, parsed=(boolean)true, interlace-mode=(string)progressive
/GstPipeline:pipeline0/v4l2jpegdec:v4l2jpegdec0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)3264, height=(int)2448, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt2020, framerate=(fraction)15/1
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:01.303332298
Setting pipeline to NULL ...
Freeing pipeline ...

If I change other format it can work (e.g. RGBx, BGRx, UYVY, YUYV)

Do you have any suggestion for fix this problem?

i.MX ProcessorsのトピックGstreamer imxvideoconvert_g2d capfilter NV12 fail

Gstreamer imxvideoconvert_g2d capfilter NV12 fail