Internal data stream error while using imxvideoconvert_g2d

[vinay_hasyagar]

Hi,

Processor: IMX8MP

I am using a gstreamer pipeline in which I am decoding h.264 encoded frames and passing it to v4l2 based sink. Below is the working pipeline.
gst-launch-1.0 rtspsrc latency=0 buffer-mode=1 drop-on-latency=true location=rtsp://10.16.102.70:1111/stream ! rtph264depay ! h264parse ! vpudec disable-reorder=true ! videoconvert ! video/x-raw,format=RGBx ! v4l2sink device=/dev/video3

The v4l2 sink accepts the frames only in RGBx format. The decoder vpudec which I am using is a hardware based decoder. It does not output data in RGBx. Below is the format in which it decodes the data
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { (string)NV12, (string)I420, (string)YV12, (string)Y42B, (string)NV16, (string)Y444, (string)NV24, (string)NV12_10LE }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]

I am using videoconvert element to convert the frame to RGBx format. But the problem with this pipeline is, the performance is very poor since videoconvert is a software based converter

So I came up with a new gstreamer pipeline in which I am using hardware based converter.
gst-launch-1.0 rtspsrc latency=0 buffer-mode=1 drop-on-latency=true location=rtsp://10.16.102.70:1111/stream ! rtph264depay ! h264parse ! vpudec disable-reorder=true ! imxvideoconvert_g2d ! video/x-raw,format=RGBx ! v4l2sink device=/dev/video3

Above pipeline is not working. It is throwing Error: Internal data stream error

If I enable debug mode then its is coming as
0:00:03.480937459 1178 0xffff6c01d8c0 WARN basetransform gstbasetransform.c:1370:gst_base_transform_setcaps:<imxvideoconvert_g2d0> transform could not transform video/x-raw, format=(string)NV12, width=(int)1024, height=(int)768, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:3:5:1, framerate=(fraction)30/1 in anything we support

The source and sink pads of imxvideoconvert_g2d is as below

Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR, (string)I420, (string)NV12, (string)UYVY, (string)YUY2, (string)YVYU, (string)YV12, (string)NV16, (string)NV21 }
video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR, (string)I420, (string)NV12, (string)UYVY, (string)YUY2, (string)YVYU, (string)YV12, (string)NV16, (string)NV21 }

SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR }
video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR }

It is almost similar to videoconvert element. Can anyone please help me to find out why it is throwing internal data stream error? How can I debug this issue?

Thanks in advance,

[joanxie]

refer to your debug information, it seems imxvideoconvert_g2d couldn't transform this data, how about using imxvideoconvert_ocl? if still failed, I will test it on my board tomorrow

[vinay_hasyagar]

Hi @joanxie 

Thank you for your reply. There is no any plugin called imxvideoconvert_ocl. Is it supported in case of iMx8 M Plus? 

My doubt is, both videoconvert and imxvideoconvert_g2d have almost same src and sink pads. The RGB format which I am converting is supported by both the plugins but still imxvideoconvert_g2d is failed to do the conversion. videoconvert is poor in terms of performance.

Thanks

[joanxie]

it shouldn't be the format issue, I tested on my board with the pipeline as below successfully

gst-launch-1.0 -v videotestsrc ! video/x-raw,format=NV12 ! queue ! imxvideoconvert_g2d ! video/x-raw,format=RGBx ! filesink location=file1

and I confirmed this should be v4l2sink issue, maybe you can test other sink plugin again

[vinay_hasyagar]

@joanxie Thank you for your confirmation. Would you suggest any method to debug at sink's side? We can't use any other sink in our case. We have to use the sink which we are using currently since it has few customization which is required for our project.

Thanks

[joanxie]

what display do you need for /dev/video3?