Internal data stream error while using imxvideoconvert_g2d

vinay_hasyagar — Sun, 10 Dec 2023 11:27:02 GMT

Hi,

Processor: IMX8MP

I am using a gstreamer pipeline in which I am decoding h.264 encoded frames and passing it to v4l2 based sink. Below is the working pipeline.
gst-launch-1.0 rtspsrc latency=0 buffer-mode=1 drop-on-latency=true location=rtsp://10.16.102.70:1111/stream ! rtph264depay ! h264parse ! vpudec disable-reorder=true ! videoconvert ! video/x-raw,format=RGBx ! v4l2sink device=/dev/video3

The v4l2 sink accepts the frames only in RGBx format. The decoder vpudec which I am using is a hardware based decoder. It does not output data in RGBx. Below is the format in which it decodes the data
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { (string)NV12, (string)I420, (string)YV12, (string)Y42B, (string)NV16, (string)Y444, (string)NV24, (string)NV12_10LE }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]

I am using videoconvert element to convert the frame to RGBx format. But the problem with this pipeline is, the performance is very poor since videoconvert is a software based converter

So I came up with a new gstreamer pipeline in which I am using hardware based converter.
gst-launch-1.0 rtspsrc latency=0 buffer-mode=1 drop-on-latency=true location=rtsp://10.16.102.70:1111/stream ! rtph264depay ! h264parse ! vpudec disable-reorder=true ! imxvideoconvert_g2d ! video/x-raw,format=RGBx ! v4l2sink device=/dev/video3

Above pipeline is not working. It is throwing Error: Internal data stream error

If I enable debug mode then its is coming as
0:00:03.480937459 1178 0xffff6c01d8c0 WARN basetransform gstbasetransform.c:1370:gst_base_transform_setcaps:<imxvideoconvert_g2d0> transform could not transform video/x-raw, format=(string)NV12, width=(int)1024, height=(int)768, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:3:5:1, framerate=(fraction)30/1 in anything we support

The source and sink pads of imxvideoconvert_g2d is as below

Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR, (string)I420, (string)NV12, (string)UYVY, (string)YUY2, (string)YVYU, (string)YV12, (string)NV16, (string)NV21 }
video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR, (string)I420, (string)NV12, (string)UYVY, (string)YUY2, (string)YVYU, (string)YV12, (string)NV16, (string)NV21 }

SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR }
video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition)
format: { (string)RGB16, (string)RGBx, (string)RGBA, (string)BGRA, (string)BGRx, (string)BGR16, (string)ARGB, (string)ABGR, (string)xRGB, (string)xBGR }

It is almost similar to videoconvert element. Can anyone please help me to find out why it is throwing internal data stream error? How can I debug this issue?

Thanks in advance,

Re: Internal data stream error while using imxvideoconvert_g2d

joanxie — Mon, 11 Dec 2023 03:48:32 GMT

refer to your debug information, it seems imxvideoconvert_g2d couldn't transform this data, how about using imxvideoconvert_ocl? if still failed, I will test it on my board tomorrow

Re: Internal data stream error while using imxvideoconvert_g2d

vinay_hasyagar — Mon, 11 Dec 2023 04:20:34 GMT

Hi @joanxie

Thank you for your reply. There is no any plugin called imxvideoconvert_ocl. Is it supported in case of iMx8 M Plus?

My doubt is, both videoconvert and imxvideoconvert_g2d have almost same src and sink pads. The RGB format which I am converting is supported by both the plugins but still imxvideoconvert_g2d is failed to do the conversion. videoconvert is poor in terms of performance.

Thanks

Re: Internal data stream error while using imxvideoconvert_g2d

joanxie — Tue, 12 Dec 2023 03:40:35 GMT

it shouldn't be the format issue, I tested on my board with the pipeline as below successfully

gst-launch-1.0 -v videotestsrc ! video/x-raw,format=NV12 ! queue ! imxvideoconvert_g2d ! video/x-raw,format=RGBx ! filesink location=file1

and I confirmed this should be v4l2sink issue, maybe you can test other sink plugin again

Re: Internal data stream error while using imxvideoconvert_g2d

vinay_hasyagar — Wed, 13 Dec 2023 16:55:34 GMT

@joanxie Thank you for your confirmation. Would you suggest any method to debug at sink's side? We can't use any other sink in our case. We have to use the sink which we are using currently since it has few customization which is required for our project.

Thanks

Re: Internal data stream error while using imxvideoconvert_g2d

joanxie — Thu, 14 Dec 2023 01:58:17 GMT

what display do you need for /dev/video3?

topic Re: Internal data stream error while using imxvideoconvert_g2d in i.MX Processors

Internal data stream error while using imxvideoconvert_g2d

Re: Internal data stream error while using imxvideoconvert_g2d

Re: Internal data stream error while using imxvideoconvert_g2d

Re: Internal data stream error while using imxvideoconvert_g2d

Re: Internal data stream error while using imxvideoconvert_g2d

Re: Internal data stream error while using imxvideoconvert_g2d