gstreamer-imx imxv4l2src to imxipuvideotransform problem

[xkentr]

Hi,

I am experimenting with the gstreamer-imx (gstreamer-1.0) plugins for i.mx6. I have encountered an issue when connecting imxv4l2src to imxipuvideotransform.

First off - I had to make a slight modification to imxv4l2src to support my camera and driver which is outputting UYVY instead of I420. This in itself does not seem to be introducing my problem since I can make my pipeline work as long as I do not use imxipuvideotransform.

The following (simplified) pipeline works fine:

/usr/bin/gst-launch-1.0 --gst-debug-level=3 --gst-debug=imxv4l2src:9 -e \

imxv4l2src capture-mode=0 device=/dev/video0 ! \

videoconvert ! video/x-raw,format=I420,width=1280,height=720 ! fakesink

However, I would like to hardware-accelerate the conversion from UYVY to I420. So I replaced videoconvert with imxipuvideotransform.

/usr/bin/gst-launch-1.0 --gst-debug-level=3 --gst-debug=imxv4l2src:9 -e \

imxv4l2src capture-mode=0 device=/dev/video0 ! \

imxipuvideotransform ! video/x-raw,format=I420,width=1280,height=720 ! fakesink

I see an internal data flow error and 'reason: not-negotiated (-4)' when using the ipu plugin, but not when using videoconvert.

Any ideas on how to debug this error?

0:00:00.103482116  7303   0xe5b1a0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:309:gst_imx_v4l2src_get_caps:<imxv4l2src0> get caps filter (NULL)

0:00:00.103716457  7303   0xe5b1a0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:319:gst_imx_v4l2src_get_caps:<imxv4l2src0> get caps video/x-raw, format=(string)UYVY, width=(int)[ 16, 2147483647 ], height=(int)[ 16, 2147483647 ], framerate=(fraction)[ 0/1, 100/1 ], pixel-aspect-ratio=(fraction)[ 0/1, 100/1 ]

0:00:00.104312141  7303   0xe5b1a0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:309:gst_imx_v4l2src_get_caps:<imxv4l2src0> get caps filter (NULL)

0:00:00.104395144  7303   0xe5b1a0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:319:gst_imx_v4l2src_get_caps:<imxv4l2src0> get caps video/x-raw, format=(string)UYVY, width=(int)[ 16, 2147483647 ], height=(int)[ 16, 2147483647 ], framerate=(fraction)[ 0/1, 100/1 ], pixel-aspect-ratio=(fraction)[ 0/1, 100/1 ]

0:00:00.105356506  7303   0xe5b1a0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:309:gst_imx_v4l2src_get_caps:<imxv4l2src0> get caps filter (NULL)

0:00:00.105443175  7303   0xe5b1a0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:319:gst_imx_v4l2src_get_caps:<imxv4l2src0> get caps video/x-raw, format=(string)UYVY, width=(int)[ 16, 2147483647 ], height=(int)[ 16, 2147483647 ], framerate=(fraction)[ 0/1, 100/1 ], pixel-aspect-ratio=(fraction)[ 0/1, 100/1 ]

0:00:00.105865521  7303   0xe5b1a0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:309:gst_imx_v4l2src_get_caps:<imxv4l2src0> get caps filter (NULL)

0:00:00.105943857  7303   0xe5b1a0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:319:gst_imx_v4l2src_get_caps:<imxv4l2src0> get caps video/x-raw, format=(string)UYVY, width=(int)[ 16, 2147483647 ], height=(int)[ 16, 2147483647 ], framerate=(fraction)[ 0/1, 100/1 ], pixel-aspect-ratio=(fraction)[ 0/1, 100/1 ]

0:00:00.106376204  7303   0xe5b1a0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:309:gst_imx_v4l2src_get_caps:<imxv4l2src0> get caps filter (NULL)

0:00:00.106480207  7303   0xe5b1a0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:319:gst_imx_v4l2src_get_caps:<imxv4l2src0> get caps video/x-raw, format=(string)UYVY, width=(int)[ 16, 2147483647 ], height=(int)[ 16, 2147483647 ], framerate=(fraction)[ 0/1, 100/1 ], pixel-aspect-ratio=(fraction)[ 0/1, 100/1 ]

Setting pipeline to PAUSED ...

0:00:00.110399658  7303   0xe5b1a0 LOG               imxv4l2src ../src/v4l2src/v4l2src.c:164:gst_imx_v4l2src_start:<imxv4l2src0> start

0:00:00.110578663  7303   0xe5b1a0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:125:gst_imx_v4l2src_capture_setup:<imxv4l2src0> capture mode 0: 1280x720

(gst-launch-1.0:7303): GStreamer-CRITICAL **: gst_debug_log_valist: assertion 'category != NULL' failed

0:00:02.154435252  7303   0xe5b1a0 DEBUG             imxv4l2src ../src/v4l2src/v4l2src.c:180:gst_imx_v4l2src_start:<imxv4l2src0> width = 1280

0:00:02.154604591  7303   0xe5b1a0 DEBUG             imxv4l2src ../src/v4l2src/v4l2src.c:181:gst_imx_v4l2src_start:<imxv4l2src0> height = 720

0:00:02.154718928  7303   0xe5b1a0 DEBUG             imxv4l2src ../src/v4l2src/v4l2src.c:182:gst_imx_v4l2src_start:<imxv4l2src0> sizeimage = 1843200

0:00:02.154821597  7303   0xe5b1a0 DEBUG             imxv4l2src ../src/v4l2src/v4l2src.c:183:gst_imx_v4l2src_start:<imxv4l2src0> pixelformat = 1498831189

Pipeline is live and does not need PREROLL ...

0:00:02.157292005  7303   0xe672f0 FIXME                default /home/kryhorchuk/git/mn2/build/tmp/work/cortexa9hf-vfp-neon-sensity-linux-gnueabi/gstreamer1.0/1.4.5-r0/gstreamer-1.4.5/gst/gstutils.c:3643:gst_pad_create_stream_id_internal:<imxv4l2src0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id

0:00:02.159603408  7303   0xe672f0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:299:gst_imx_v4l2src_negotiate:<imxv4l2src0> negotiated caps video/x-raw, format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1

0:00:02.160265761  7303   0xe672f0 INFO              imxv4l2src ../src/v4l2src/v4l2src.c:328:gst_imx_v4l2src_set_caps:<imxv4l2src0> set caps video/x-raw, format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1

Setting pipeline to PLAYING ...

New clock: GstSystemClock

0:00:02.166468948  7303   0xe672f0 WARN                GST_PADS /home/kryhorchuk/git/mn2/build/tmp/work/cortexa9hf-vfp-neon-sensity-linux-gnueabi/gstreamer1.0/1.4.5-r0/gstreamer-1.4.5/gst/gstpad.c:3739:gst_pad_peer_query:<imxv4l2src0:src> could not send sticky events

0:00:02.351330521  7303   0xe672f0 LOG               imxv4l2src ../src/v4l2src/v4l2src.c:269:gst_imx_v4l2src_fill:<imxv4l2src0> fill

0:00:02.355904325  7303   0xe672f0 WARN                 basesrc /home/kryhorchuk/git/mn2/build/tmp/work/cortexa9hf-vfp-neon-sensity-linux-gnueabi/gstreamer1.0/1.4.5-r0/gstreamer-1.4.5/libs/gst/base/gstbasesrc.c:2933:gst_base_src_loop:<imxv4l2src0> error: Internal data flow error.

0:00:02.356141999  7303   0xe672f0 WARN                 basesrc /home/kryhorchuk/git/mn2/build/tmp/work/cortexa9hf-vfp-neon-sensity-linux-gnueabi/gstreamer1.0/1.4.5-r0/gstreamer-1.4.5/libs/gst/base/gstbasesrc.c:2933:gst_base_src_loop:<imxv4l2src0> error: streaming task paused, reason not-negotiated (-4)

ERROR: from element /GstPipeline:pipeline0/GstImxV4l2Src:imxv4l2src0: Internal data flow error.

Additional debug info:

/home/kryhorchuk/git/mn2/build/tmp/work/cortexa9hf-vfp-neon-sensity-linux-gnueabi/gstreamer1.0/1.4.5-r0/gstreamer-1.4.5/libs/gst/base/gstbasesrc.c(2933): gst_base_src_loop (): /GstPipeline:pipeline0/GstImxV4l2Src:imxv4l2src0:

streaming task paused, reason not-negotiated (-4)

EOS on shutdown enabled -- waiting for EOS after Error

Waiting for EOS...

gstreamer-imx imxv4l2src to imxipuvideotransform problem

[xkentr · Accepted solution]

Answered my own question.

In the current release of the plugins (0.10.1) there is an issue with the caps reported by imxv4l2src and what is required for imxipuvideotransform. imxv4l2src does not report 'interlace-mode' when imxipuvideotransform is expecting it. I don't know why this shows up as an error when PLAYING instead of when the pipeline is parsed (I am not a gstreamer expert!).

I fixed my problem by patching the sources for imxv4l2src. You need to add 'interlace-mode' in gst_imx_v4l2src_negotiate and gst_imx_v4l2src_get_caps. In the case of my camera, the interlace-mode is 'progressive'.

Cheers,

K.

[CarlosCasillas]

Hi Kent,

Thanks for sharing your solution with the Community!!!

Best regards!
/Carlos