i.MX Processors中的主题 Re: i.MX8QXP: unable to link v4l2 decoder to encoder in the same gstreamer pipeline

i.MX8QXP: unable to link v4l2 decoder to encoder in the same gstreamer pipeline

gabrielvalcazar — Wed, 09 Dec 2020 14:42:00 GMT

Good afternoon,

I'm currently testing the multimedia capabilities of an i.MX8QXP-based system running a custom Yocto image based on the 5.4.47_2.2.0 BSP. In one of my tests, I tried queuing the output of the v4l2 JPEG decoder (I assume it's the element I want to use if I'm interested in testing the 8X's VPU) into the v4l2 encoder. Here is a simplified example of what I want to test:

gst-launch-1.0 v4l2src device=/dev/video3 ! v4l2jpegdec ! v4l2h264enc ! fakesink

The command fails with the following error:

ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error. Additional debug info: ../../../../git/libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: streaming stopped, reason not-negotiated (-4) Execution ended after 0:00:01.074351570 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ...

In another example, I'm able to use the v4l2 video decoder and encoder separately, but not together. These two commands work separately:

gst-launch-1.0 -e videotestsrc ! v4l2h264enc ! h264parse ! mp4mux ! filesink location=test.mp4 gst-launch-1.0 -v filesrc location=test.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! testsink

But trying to use both elements in one pipeline fails:

gst-launch-1.0 -v filesrc location=test.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! v4l2h264enc ! testsink

What is the reason for this? Is there a missing element in the pipeline, or is this a known limitation (eiher for gstreamer or for the i.MX8QXP's VPU)?

Thanks in advance,

Gabriel

i.MX8QXP: unable to link v4l2 decoder to encoder in the same gstreamer pipeline

joanxie — Tue, 15 Dec 2020 08:08:05 GMT

how about trying the command as below:

gst-launch-1.0 -v filesrc location=test.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! v4l2convert ! v4l2h264enc ! testsink

Re: i.MX8QXP: unable to link v4l2 decoder to encoder in the same gstreamer pipeline

gabrielvalcazar — Wed, 16 Dec 2020 15:17:00 GMT

Hi joanxie,

Using the command you suggested, gst-launch-1.0 no longer fails with an error and I see kernel messages that suggest that both the decoder and the encoder are being used.

However, when I try stopping the command with Ctrl+C, an error appears and I'm unable to terminate the command properly, so I'm stuck until I reset the target. Here's the output:

~# gst-launch-1.0 -e filesrc location=test.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! v4l2convert ! v4l2h264enc ! h264parse ! mp4mux ! filesink location="result.mp4" Setting pipeline to PAUSED ... Pipeline is PREROLLING ... [ 246.781463] [VPU Decoder] firmware version is 1.6.4 Redistribute latency... [ 246.807488] bypass csc [ 246.809924] input fmt YUYV [ 246.812685] output fmt NV12 Redistribute latency... ^Chandling interrupt. Interrupt: Stopping pipeline ... ERROR: pipeline doesn't want to preroll. Setting pipeline to NULL ...

I feel like this pipeline is close to a proper solution, but it's still isn't working as expected.

Re: i.MX8QXP: unable to link v4l2 decoder to encoder in the same gstreamer pipeline

songhongliang — Thu, 17 Dec 2020 09:48:14 GMT

HI,

gst-launch-1.0 imxcompositor_g2d v4l2src device=/dev/video8 ! 'video/x-raw,framerate=25/1,width=1280,height=720' ! autovideosink

I think the width=1280,height=720 is very important，You need to check the available supported resolutions of the video

Re: i.MX8QXP: unable to link v4l2 decoder to encoder in the same gstreamer pipeline

gabrielvalcazar — Wed, 17 Mar 2021 10:04:40 GMT

Hi again,

I've returned to investigating this issue after some time. I'm currently using the v5.4.70_2.3.1 BSP and I'm still running into the same problem.

My final goal is to be able to transcode a JPEG camera feed into an H.264 format with the least possible amount of CPU overhead. As reference, I'm using the example in section 9.1 of the i.MX 8 GStreamer user guide, which is supposed to accomplish what I'm looking for:

gst-launch-1.0 v4l2src device=/dev/video3 ! jpegparse ! v4l2jpegdec ! queue ! videoconvert ! v4l2h264enc ! h264parse ! matroskamux ! filesink location=out.mkv

However, this pipeline doesn't work on my i.MX8QXP device.

The following pipeline does seem to go through, but no video stream is stored in the out.mkv file and the gst-launch-1.0 app blocks when stopping the stream:

gst-launch-1.0 v4l2src device=/dev/video3 ! jpegparse ! v4l2jpegdec ! imxvideoconvert_g2d ! v4l2convert ! v4l2h264enc ! h264parse ! matroskamux ! filesink location=out.mkv Setting pipeline to PAUSED ... [ 49.299976] uvcvideo: Failed to query (GET_DEF) UVC control 12 on unit 2: -32 (exp. 4). Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock [ 49.926396] [VPU Decoder] firmware version is 1.7.0 Redistribute latency... [ 49.960932] bypass csc [ 49.963307] input fmt YUYV [ 49.966092] output fmt NV12 ^Chandling interrupt. Interrupt: Stopping pipeline ... Execution ended after 0:00:03.492342625 Setting pipeline to PAUSED ... Setting pipeline to READY ... ^C^C^C^C^C

I've also tried piping the stream to the waylandsink, but I'm unable to see anything on screen. I've also tried the suggestion of hardcoding a 1280x720 resolution in the pipeline, but that doesn't work.

Is there a pipeline that will allow me to transcode a JPEG feed to H.264 while avoiding CPU overhead (e.g. using the VPU when possible or zero-copy methods like the ones suggested in the GStreamer user guide)?

Re: i.MX8QXP: unable to link v4l2 decoder to encoder in the same gstreamer pipeline

banhu — Sat, 03 Jul 2021 09:30:19 GMT

Hi ,

The same issue on iMX8QM with 5.4.70_2.3.1. It is fine to preview a USB camera.

gst-launch-1.0 v4l2src device=/dev/video2 ! 'image/jpeg,width=1280,height=720,framerate=30/1' ! v4l2jpegdec ! waylandsink sync=false

But it just fails when encoding v4l2jpegdec's src stream into H.264 by v4l2jpegdec. The v4l2jpegdec's src format(video/x-raw, NV12) meets v4l2jpegdec sink format requirement.

gst-launch-1.0 v4l2src device=/dev/video2 ! 'image/jpeg,width=1280,height=720,framerate=30/1' ! v4l2jpegdec ! video/x-raw, format=NV12, width=1280, height=720, framerate=30/1 ! v4l2h264enc ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.647766125
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Re: i.MX8QXP: unable to link v4l2 decoder to encoder in the same gstreamer pipeline

Hongtao — Mon, 05 Jul 2021 07:09:28 GMT

I have the same problem as you.

In addition, problems can occur when using both v4l2dec and imxcompositor.

V4l2src This route shows that stopping will be blocked

gst-launch-1.0 imxcompositor_g2d name=comp sink_1::xpos=0 sink_1::ypos=0 ! waylandsink v4l2src device=/dev/video4 ! video/x-h264,width=1280,height=720,framerate=30/1 ! v4l2h264dec ! comp.sink_0 videotestsrc ! comp.sink_1 gst-launch-1.0 imxcompositor_g2d name=comp sink_1::xpos=0 sink_1::ypos=0 ! waylandsink v4l2src device=/dev/video2 ! image/jpeg,width=1280,height=720,framerate=30/1 ! v4l2jpegdec ! comp.sink_0 videotestsrc ! comp.sink_1

It works by decoding with jpegdec software, but it is extremely unstable and shows Carton frequently

gst-launch-1.0 imxcompositor_g2d name=comp sink_1::xpos=0 sink_1::ypos=0 ! waylandsink v4l2src device=/dev/video2 ! image/jpeg,width=1280,height=720,framerate=30/1 ! jpegdec ! comp.sink_0 videotestsrc ! comp.sink_1

Re: i.MX8QXP: unable to link v4l2 decoder to encoder in the same gstreamer pipeline

toshinariagata — Wed, 17 Nov 2021 05:46:12 GMT

Hi,

I have the same problem.
Has there been any progress?