[iMX8MPlus] How to mix two video sources into one with Gstreamer ?

cancel
Showing results for 
Search instead for 
Did you mean: 

[iMX8MPlus] How to mix two video sources into one with Gstreamer ?

Jump to solution
1,527 Views
khang_letruong
Senior Contributor III

Dear Community,

Gstreamer has the notion of Composition for outputting multiple video displays but I need to do the opposite way:  I would like to mix/combine two H264-encoded videos (from /dev/video0 and /dev/video1 for example) from iMX8M Plus. Is it possible with Gstreamer, please ?

Best regards,
Khang

 

0 Kudos
1 Solution
501 Views
dianapredescu
NXP Employee
NXP Employee

Khang,

For 1.b. case: can you try the following command on your side and let me know if the performance is increased?

root@imx8mpevk:~# export DISPLAY=:0
root@imx8mpevk:~# gst-launch-1.0 -vvv imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 \
sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 \
! queue ! video/x-raw, width=1440, height=2160 ! fpsdisplaysink video-sink="ximagesink" text-overlay=false sync=false \
v4l2src device=/dev/video2 ! video/x-raw, width=1440, height=1080, framerate=15/1 ! comp.sink_0 \
v4l2src device=/dev/video3 ! video/x-raw, width=1440, height=1080, framerate=15/1 ! comp.sink_1

On my side it looks better than previous pipeline. (I'm using 2 Basler cameras)

Best regards,

Diana

View solution in original post

42 Replies
532 Views
dianapredescu
NXP Employee
NXP Employee

Dear @khang_letruong,

Please let me look into the Linux image of BSP-L5.10.72 first, because I'm not really familiar with the desktop image PoC. In the image I already have on the board I see:

root@imx8mpevk:~# uname -a
Linux imx8mpevk 5.10.72-lts-5.10.y+gb83453f6c6af #1 SMP PREEMPT Fri Jul 29 15:18:00 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
root@imx8mpevk:~# gst-inspect-1.0 | grep v4l2
video4linux2: v4l2vp9dec: V4L2 VP9 Decoder
video4linux2: v4l2vp8dec: V4L2 VP8 Decoder
video4linux2: v4l2h265dec: V4L2 H265 Decoder
video4linux2: v4l2h264dec: V4L2 H264 Decoder
video4linux2: v4l2h265enc: V4L2 H.265 Encoder
video4linux2: v4l2h264enc: V4L2 H.264 Encoder
video4linux2: v4l2convert: V4L2 Video Converter
video4linux2: v4l2deviceprovider (GstDeviceProviderFactory)
video4linux2: v4l2radio: Radio (video4linux2) Tuner
video4linux2: v4l2sink: Video (video4linux2) Sink
video4linux2: v4l2src: Video (video4linux2) Source

Let me check if I've done something special in this image. 

Best regards,

Diana

516 Views
khang_letruong
Senior Contributor III

Hi @dianapredescu ,

Just let you know that the V4L2 elements are not available in my firmware image due to the disabling of vpu_v4l2 in my custom dts (as I wanted to have only /dev/video0, with vpu_v4l2 enabled , there's also /dev/video1) :

&vpu_v4l2 {
	status = "disabled";
};

Again, thanks very much for your support.

Best Regards,
Khang

 

489 Views
khang_letruong
Senior Contributor III

Dear @dianapredescu,

I am sill facing the following error when running below command (with custom resolution 1440x1080) :

root@imx8mpevk:~# gst-launch-1.0 -vvv imxcompositor_g2d name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 ! queue ! videoconvert ! v4l2h264enc output-io-mode=dmabuf-import ! rtph264pay config-interval=1 pt=96 ! udpsink host=192.168.1.8 port=5000 v4l2src device=/dev/video2 io-mode=dmabuf ! imxvideoconvert_g2d ! "video/x-raw, width=1440, height=1080, framerate=5/1" ! queue ! comp.sink_0 v4l2src device=/dev/video3 io-mode=dmabuf ! imxvideoconvert_g2d ! "video/x-raw, width=1440, height=1080, framerate=5/1" ! queue ! comp.sink_1 
Setting pipeline to PAUSED ...

====== V4L2ENC: 1.18.5 build on Nov  9 2021 07:44:26. ======
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/imxvideoconvert_g2d:imxvideoconvert_g2d0.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)RGBx
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)RGBx
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)RGBx
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)RGBx
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)RGBx
/GstPipeline:pipeline0/imxcompositor_g2d:comp.GstImxCompositorPad:sink_0: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)RGBx
/GstPipeline:pipeline0/imxvideoconvert_g2d:imxvideoconvert_g2d0.GstPad:sink: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstV4l2Src:v4l2src1.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/imxvideoconvert_g2d:imxvideoconvert_g2d1.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)RGBx
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)RGBx
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:sink: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)RGBx
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)RGBx
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)RGBx
/GstPipeline:pipeline0/imxcompositor_g2d:comp.GstImxCompositorPad:sink_1: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, format=(string)RGBx
/GstPipeline:pipeline0/imxvideoconvert_g2d:imxvideoconvert_g2d1.GstPad:sink: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)5/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/imxcompositor_g2d:comp.GstAggregatorPad:src: caps = video/x-raw, format=(string)RGBx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)5/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)RGBx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)5/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)RGBx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)5/1
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src1: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src1:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:01.294399495
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstQueue:queue2: Internal data stream error.
Additional debug info:
../git/plugins/elements/gstqueue.c(990): gst_queue_handle_sink_event (): /GstPipeline:pipeline0/GstQueue:queue2:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

 

Do you know how to proceed the investigation?

Best Regards,
K.

0 Kudos
475 Views
dianapredescu
NXP Employee
NXP Employee

Hi @khang_letruong ,

Let us try finding the root cause of the failing pipeline.

First of all, can you try run the same pipeline with GST_DEBUG=5 in front of it, and send us the logs? It might provide us more information than "Internal data stream error".

Also, just to isolate the problem, have you tried to run the pipeline with videotestsrc instead of v4l2src?

This is working on my side:

root@imx8mpevk:~# gst-launch-1.0 imxcompositor_g2d name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 ! queue ! videoconvert ! v4l2h264enc ! rtph264pay config-interval=1 pt=96 ! udpsink host=169.254.235.127 port=5000 videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_0 videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_1
[ 3235.341194] enter isp_mi_stop
Setting pipeline to PAUSED ...

====== V4L2ENC: 1.20.3 build on Sep 15 2022 02:27:06. ======
Pipeline is PREROLLING ...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:17.920952125
Setting pipeline to NULL ...
Freeing pipeline ...

Best regards,

Diana

0 Kudos
471 Views
khang_letruong
Senior Contributor III

Dear @dianapredescu.

Running the pipeline with videotestsrc does NOT work :

 

root@imx8mpevk:~# gst-launch-1.0 imxcompositor_g2d name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 ! queue ! videoconvert ! v4l2h264enc ! rtph264pay config-interval=1 pt=96 ! udpsink host=192.168.1.8 port=5000 videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_0 videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_1 
Setting pipeline to PAUSED ...

====== V4L2ENC: 1.18.5 build on Nov  9 2021 07:44:26. ======
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc1: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc1:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
ERROR: from element /GstPipeline:pipeline0/GstQueue:queue2: Internal data stream error.
Additional debug info:
../git/plugins/elements/gstqueue.c(990): gst_queue_handle_sink_event (): /GstPipeline:pipeline0/GstQueue:queue2:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
ERROR: from element /GstPipeline:pipeline0/GstQueue:queue1: Internal data stream error.
Additional debug info:
../git/plugins/elements/gstqueue.c(990): gst_queue_handle_sink_event (): /GstPipeline:pipeline0/GstQueue:queue1:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Freeing pipeline ...

 

Note that 192.168.1.8 is the IP of my testing PC in the LAN network where iMX8MP-EVK is connected to with IP 192.168.1.11  as well.

Please file the log of above command with GST_DEBUG=5 in attachment.

Btw for single camera (/dev/video2 or /dev/video3)m all below commands work:

 

####  Display  ####
# gst-launch-1.0 -v v4l2src device=/dev/video2 ! "video/x-raw,format=YUY2,width=1440,height=1080" ! queue ! imxvideoconvert_g2d ! waylandsink
# gst-launch-1.0 -v v4l2src device=/dev/video2 ! waylandsink

#### Streaming ####

### RAW UDP
## Server
# gst-launch-1.0 -v v4l2src device=/dev/video2 ! imxvideoconvert_g2d ! "video/x-raw, width=1440, height=1080" ! rtpvrawpay pt=96 timestamp-offset=0 ! queue max-size-buffers=0 ! udpsink host=192.168.1.8
## Client
# gst-launch-1.0 -v udpsrc caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)RGBA, depth=(string)8, width=(string)1440, height=(string)1080, colorimetry=(string)SMPTE240M, payload=(int)96, timestamp-offset=(uint)0, ssrc=(uint)3525922485, seqnum-offset=(uint)12886, a-framerate=(string)60" ! rtpjitterbuffer latency=1 ! queue max-size-buffers=0 ! rtpvrawdepay ! videoconvert ! autovideosink sync=false

### COMPRESSED UDP
## Server
# gst-launch-1.0 -v v4l2src device=/dev/video2 ! imxvideoconvert_g2d ! "video/x-raw, width=1440, height=1080, framerate=30/1" ! vpuenc_h264 ! rtph264pay pt=96 ! rtpstreampay ! udpsink host=192.168.1.8
## Client
# gst-launch-1.0 udpsrc caps = "application/x-rtp-stream, encoding-name=H264" ! rtpstreamdepay ! rtph264depay ! decodebin ! videoconvert ! autovideosink
#  gst-launch-1.0 udpsrc caps = "application/x-rtp-stream, encoding-name=H264" ! rtpstreamdepay ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink

### RAW TCP
## Server
# gst-launch-1.0 -v v4l2src device=/dev/video2 ! imxvideoconvert_g2d ! "video/x-raw, width=1440, height=1080" ! rtpvrawpay pt=96 timestamp-offset=0 ! queue max-size-buffers=0 ! gdppay ! tcpserversink host=192.168.1.11 port=5000 blocksize=512000 sync=false
## Client
# gst-launch-1.0 tcpclientsrc host=192.168.1.11 port=5000 ! gdpdepay ! queue max-size-buffers=0 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)RGBA, depth=(string)8, width=(string)1440, height=(string)1080, colorimetry=(string)SMPTE240M, payload=(int)96, framerate=60/1" ! rtpjitterbuffer latency=100 ! rtpvrawdepay ! decodebin ! videoconvert ! autovideosink sync=false

### COMPRESSED TCP
## Server
# gst-launch-1.0 -v v4l2src device=/dev/video2 ! imxvideoconvert_g2d ! "video/x-raw, width=1440, height=1080, framerate=60/1" ! vpuenc_h264 ! rtph264pay pt=96 ! rtpstreampay ! tcpserversink host=192.168.1.11 port=5000 blocksize=512000 sync=false
## Client
# gst-launch-1.0 tcpclientsrc host=192.168.1.11 port=5000 ! "application/x-rtp-stream, encoding-name=H264" ! rtpstreamdepay ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink

 

 

or for dual cameras :

 

# gst-launch-1.0 -v v4l2src device=/dev/video3 ! imxvideoconvert_g2d ! "video/x-raw, width=1440, height=1080, framerate=30/1" ! vpuenc_h264 ! rtph264pay pt=96 ! rtpstreampay ! tcpserversink host=192.168.1.11 port=5001 blocksize=512000 sync=false & \
gst-launch-1.0 -v v4l2src device=/dev/video2 ! imxvideoconvert_g2d ! "video/x-raw, width=1440, height=1080, framerate=30/1" ! vpuenc_h264 ! rtph264pay pt=96 ! rtpstreampay ! tcpserversink host=192.168.1.11 port=5000 blocksize=512000 sync=false

 

 

 Best Regards,
K

0 Kudos
455 Views
khang_letruong
Senior Contributor III

Dear @dianapredescu,

Have you had time to take a look at my previous post, please ?

Thanks in advance,
K

0 Kudos
448 Views
dianapredescu
NXP Employee
NXP Employee

Hi @khang_letruong,

We had some national days off and I just returned to office. I'll continue looking into your issue and come back when I'll get to a conclusion. Sorry for the delay.

Best regards,

Diana

0 Kudos
446 Views
dianapredescu
NXP Employee
NXP Employee

Dear @khang_letruong, I reproduced the errors you received on 5.10.72 BSP.

After trying to isolate the issue, I'm suspecting it is an H264 encoding issue.  I've tried the same pipeline on 2 BSPs: 5.10.72(your use case) & 5.15.71(currently in field trial for general release). I've splitted the pipeline in 3 steps:

  • video composition of two 1440x1080 streams and displaying it on a HDMI display. 
  • video composition of two 1440x1080 streams, H264 encoding/decoding and displaying it on a HDMI display
  • video composition of two 1440x1080 streams, H264 encoding and streaming over network.

BSP 5.10.72

  • the first step works as expected. 
gst-launch-1.0 imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 \
sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 \
! queue ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=2160 ! glimagesink render-rectangle="<0,0,1440,2160>" \
videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_0 \
videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_1
  • the second step fails with the error you received
root@imx8mpevk:~# gst-launch-1.0 imxcompositor_g2d name=comp \
> sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 \
> sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 \
> ! queue ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=2160, framerate=5/1 ! v4l2h264enc ! queue ! v4l2h264dec ! glimagesink render-rectangle="<0,0,1440,2160>" \
> videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080, framerate=5/1 ! queue ! comp.sink_0 \
> videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080, framerate=5/1 ! queue ! comp.sink_1
Setting pipeline to PAUSED ...

====== V4L2DEC: 1.18.5 build on Nov 9 2021 07:44:26. ======

====== V4L2ENC: 1.18.5 build on Nov 9 2021 07:44:26. ======
Pipeline is PREROLLING ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
ERROR: from element /GstPipeline:pipeline0/GstQueue:queue2: Internal data stream error.
Additional debug info:
../git/plugins/elements/gstqueue.c(990): gst_queue_handle_sink_event (): /GstPipeline:pipeline0/GstQueue:queue2:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc1: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc1:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
ERROR: from element /GstPipeline:pipeline0/GstQueue:queue3: Internal data stream error.
Additional debug info:
../git/plugins/elements/gstqueue.c(990): gst_queue_handle_sink_event (): /GstPipeline:pipeline0/GstQueue:queue3:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
  • the third step is no longer relevant since the encoding doesn't work.

BSP 5.15.71

  • all the steps works as expected. 

Now I will check what's the difference between these versions (V4L2ENC: 1.18.5 - for 5.10.72 & V4L2ENC: 1.20.3 - 5.15.71) and if there's something we can port back to 5.10.72. Still under investigations.

Best regards,

Diana

0 Kudos
406 Views
khang_letruong
Senior Contributor III

Hi @dianapredescu,

There seems to be a new release of 5.10.72 BSP : Linux 5.10.72_2.2.3. Is it worth to test it?

Btw, there's no 5.15.71 BSP in official releases of NXP. Latest one seems to be 5.15.52(_2.1.0​). Where did come the aforementioned BSP, please ?

Best Regards,
K

0 Kudos
403 Views
dianapredescu
NXP Employee
NXP Employee

Hi @khang_letruong,

Regarding the latest BSP, you are right. At the moment you can find on NXP site up to LF5.15.52_2.1.0. The official release of the newest BSP (LF5.15.71-2.2.0) will be available externally starting from December 16.

Now, regarding our issue, I don't believe we can see any improvement in 5.10.72_2.2.3. Looking into linux-imx source code, the tags for 5.10.72_2.2.0 and 5.10.72_2.2.3 are identical. However, starting from 5.15.5_1.0.0 the source code of vpu v4l2 drivers were modified. Just to double check, I tried on my side to apply the changes from here (https://github.com/nxp-imx/linux-imx/commits/lf-5.15.5-1.0.0/drivers/mxc/hantro_v4l2) on my 5.10.72_2.2.0 linux kernel. With those changes I was able to play the pipeline on 5.10.72 BSP.

Can you also test it on your side and let me know the outcome? 

Best regards,

Diana

0 Kudos
400 Views
khang_letruong
Senior Contributor III

Hi @dianapredescu ,

Thank you for your sharing. I will double-check by applying the change of vpu v4l2 drivers to 5.10.72_2.2.0 and confirm with you when done. Hope that the patching will be trivial (no significant conflict).

Best Regards,

Khang

0 Kudos
388 Views
khang_letruong
Senior Contributor III
0 Kudos
350 Views
khang_letruong
Senior Contributor III

Hi @dianapredescu,

Applying the patches I mentioned in previous comment, I was able to test following cases with BSP-5.10.72 and they seemed to work:

1. video composition of two 1440x1080 streams and displaying it on a HDMI display. 

a. From video test sources :

DISPLAY=:0 gst-launch-1.0 imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 \
sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 \
! queue ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=2160 ! glimagesink render-rectangle="<0,0,1440,2160>" \
videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_0 \
videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_1

b. From camera sensor sources :

DISPLAY=:0 gst-launch-1.0 imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 \
sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 \
! queue ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=2160 ! glimagesink render-rectangle="<0,0,1440,2160>" \
v4l2src device=/dev/video2 ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_0 \
v4l2src device=/dev/video3 ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_1

2. video composition of two 1440x1080 streams, H264 encoding/decoding and displaying it on a HDMI display

a. From video test sources :

DISPLAY=:0 gst-launch-1.0 imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 \
sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 \
! queue ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=2160, framerate=5/1 ! v4l2h264enc ! queue ! v4l2h264dec ! glimagesink render-rectangle="<0,0,1440,2160>" \
videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080, framerate=5/1 ! queue ! comp.sink_0 \
videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080, framerate=5/1 ! queue ! comp.sink_1

b. From camera sensor sources :

DISPLAY=:0 gst-launch-1.0 imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 \
sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 \
! queue ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=2160, framerate=5/1 ! v4l2h264enc ! queue ! v4l2h264dec ! glimagesink render-rectangle="<0,0,1440,2160>" \
v4l2src device=/dev/video2  ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080, framerate=15/1 ! queue ! comp.sink_0 \
v4l2src device=/dev/video3  ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080, framerate=15/1 ! queue ! comp.sink_1

3. video composition of two 1440x1080 streams, H264 encoding and streaming over network.

a. From video test sources :

gst-launch-1.0 imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 \
sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 \
! queue ! videoconvert ! v4l2h264enc ! rtph264pay config-interval=1 pt=96 ! udpsink host=192.168.1.8 port=5000 \
videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_0 \
videotestsrc ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_1

b. From camera sensor sources :

gst-launch-1.0 -vvv imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 sink_1::xpos=0 \
sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 ! queue ! videoconvert ! v4l2h264enc output-io-mode=dmabuf-import ! rtph264pay config-interval=1 pt=96 ! udpsink host=192.168.1.8 port=5000 \
v4l2src device=/dev/video2 io-mode=dmabuf ! imxvideoconvert_g2d ! "video/x-raw, width=1440, height=1080, framerate=15/1" ! queue ! comp.sink_0 \
v4l2src device=/dev/video3 io-mode=dmabuf ! imxvideoconvert_g2d ! "video/x-raw, width=1440, height=1080, framerate=15/1" ! queue ! comp.sink_1 

 

However, in the case 1.b), there seems to be following issue of performance :

root@imx8mpevk:~# DISPLAY=:0 gst-launch-1.0 imxcompositor_g2d name=comp \
> sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 \
> sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 \
> ! queue ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=2160 ! glimagesink render-rectangle="<0,0,1440,2160>" \
> v4l2src device=/dev/video2 ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_0 \
> v4l2src device=/dev/video3 ! imxvideoconvert_g2d ! video/x-raw, width=1440, height=1080 ! queue ! comp.sink_1
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped.
Additional debug info:
../git/libs/gst/base/gstbasesink.c(3136): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:19.281492055
Setting pipeline to NULL ...
Total showed frames (22), playing for (0:00:19.281454548), fps (1.141).
Freeing pipeline ...

 

And it seems that there's only stream of 1st camera to be showed on the display for 1) & 2).

Also, could you share the gstreamer setting or VLC setting to visualize the stream on the PC side over the network, please ?

Best Regards,
Khang

0 Kudos
333 Views
dianapredescu
NXP Employee
NXP Employee

Hi @khang_letruong,

Indeed, I'm seeing the same performance issue when running the first pipeline with camera input. Will try to see if we can find the bottleneck and if there's a way we can improve it. However, running 1a. and 2.a. test cases, I can see both streams on the display. Just to double check is your display resolution 4K? Can it fit the 1440x2160 window?

In regards with the second question, I've followed the steps here (https://developer.toradex.com/linux-bsp/application-development/multimedia/audiovideo-over-rtp-with-...) and created a .sdp file for vlc as in the attached link.

Best regards,

Diana

0 Kudos
322 Views
khang_letruong
Senior Contributor III

Dear @dianapredescu ,

Indeed, I'm seeing the same performance issue when running the first pipeline with camera input. 

Did it also happen with BSP-5.15.71 that you tested ?

Best Regards,
Khang

0 Kudos
309 Views
dianapredescu
NXP Employee
NXP Employee

Dear @khang_letruong,

Yes, same behavior with the latest BSP. Let me look again in this issue. Maybe we can find a solution for that as well.

Please note my answers will be delayed (starting from tomorrow we will be out of office due to winter holidays - will be back on 4th of January)

Best regards,

Diana

0 Kudos
502 Views
dianapredescu
NXP Employee
NXP Employee

Khang,

For 1.b. case: can you try the following command on your side and let me know if the performance is increased?

root@imx8mpevk:~# export DISPLAY=:0
root@imx8mpevk:~# gst-launch-1.0 -vvv imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 \
sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 \
! queue ! video/x-raw, width=1440, height=2160 ! fpsdisplaysink video-sink="ximagesink" text-overlay=false sync=false \
v4l2src device=/dev/video2 ! video/x-raw, width=1440, height=1080, framerate=15/1 ! comp.sink_0 \
v4l2src device=/dev/video3 ! video/x-raw, width=1440, height=1080, framerate=15/1 ! comp.sink_1

On my side it looks better than previous pipeline. (I'm using 2 Basler cameras)

Best regards,

Diana

299 Views
khang_letruong
Senior Contributor III

Hi @dianapredescu,

Just update to you that your last instruction works like a charm : 

root@imx8mpevk:~# gst-launch-1.0 -vvv imxcompositor_g2d name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 ! queue ! video/x-raw, width=1440, height=2160 ! fpsdisplaysink video-sink="ximagesink" text-overlay=false sync=false v4l2src device=/dev/video2 ! video/x-raw, width=1440, height=1080, framerate=35/1 ! comp.sink_0 v4l2src device=/dev/video3 ! video/x-raw, width=1440, height=1080, framerate=35/1 ! comp.sink_1
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstXImageSink:ximagesink0: sync = false
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)35/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)35/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/imxcompositor_g2d:comp.GstImxCompositorPad:sink_0: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)35/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)35/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstV4l2Src:v4l2src1.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)35/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)35/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/imxcompositor_g2d:comp.GstImxCompositorPad:sink_1: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)35/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:sink: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)35/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/imxcompositor_g2d:comp.GstAggregatorPad:src: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)35/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)35/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)35/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)35/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)35/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstXImageSink:ximagesink0.GstPad:sink: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)35/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)35/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)35/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstXImageSink:ximagesink0: sync = false
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 12, dropped: 0, current: 22.60, average: 22.60
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 27, dropped: 0, current: 29.03, average: 25.77
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 42, dropped: 0, current: 29.34, average: 26.94
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 57, dropped: 0, current: 28.72, average: 27.39
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 75, dropped: 0, current: 35.49, average: 28.97
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 95, dropped: 0, current: 39.96, average: 30.75
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 115, dropped: 0, current: 39.38, average: 31.97
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 136, dropped: 0, current: 40.03, average: 33.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 156, dropped: 0, current: 39.95, average: 33.75
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 177, dropped: 0, current: 40.46, average: 34.43
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 198, dropped: 0, current: 40.35, average: 34.97
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 218, dropped: 0, current: 39.87, average: 35.37
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 238, dropped: 0, current: 39.84, average: 35.71
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 258, dropped: 0, current: 39.97, average: 36.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 278, dropped: 0, current: 38.88, average: 36.20
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 297, dropped: 0, current: 37.82, average: 36.30
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 317, dropped: 0, current: 39.61, average: 36.49
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 337, dropped: 0, current: 38.82, average: 36.62
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 357, dropped: 0, current: 38.71, average: 36.73
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 376, dropped: 0, current: 37.24, average: 36.76
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 395, dropped: 0, current: 36.90, average: 36.76
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 414, dropped: 0, current: 36.18, average: 36.74
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 432, dropped: 0, current: 34.96, average: 36.66
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 450, dropped: 0, current: 35.03, average: 36.59
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 468, dropped: 0, current: 34.98, average: 36.53
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 486, dropped: 0, current: 34.73, average: 36.46
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 504, dropped: 0, current: 35.25, average: 36.41
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 522, dropped: 0, current: 35.16, average: 36.37
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 540, dropped: 0, current: 34.92, average: 36.32
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 558, dropped: 0, current: 34.89, average: 36.27
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 575, dropped: 0, current: 33.52, average: 36.18
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 594, dropped: 0, current: 36.51, average: 36.19
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 612, dropped: 0, current: 35.10, average: 36.16
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 630, dropped: 0, current: 34.93, average: 36.12
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 648, dropped: 0, current: 34.62, average: 36.08
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 666, dropped: 0, current: 35.43, average: 36.06
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 684, dropped: 0, current: 35.00, average: 36.03
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 702, dropped: 0, current: 35.09, average: 36.01
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 720, dropped: 0, current: 34.90, average: 35.98
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 738, dropped: 0, current: 35.04, average: 35.96
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 756, dropped: 0, current: 34.64, average: 35.92
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 774, dropped: 0, current: 35.49, average: 35.91
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 792, dropped: 0, current: 34.59, average: 35.88
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 810, dropped: 0, current: 35.40, average: 35.87
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 828, dropped: 0, current: 34.97, average: 35.85
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 846, dropped: 0, current: 34.74, average: 35.83
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 864, dropped: 0, current: 35.34, average: 35.82
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 882, dropped: 0, current: 35.05, average: 35.80
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 900, dropped: 0, current: 34.79, average: 35.78
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 918, dropped: 0, current: 34.68, average: 35.76
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 936, dropped: 0, current: 35.47, average: 35.75
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 954, dropped: 0, current: 35.11, average: 35.74
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 972, dropped: 0, current: 33.76, average: 35.70
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 991, dropped: 0, current: 36.16, average: 35.71
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1009, dropped: 0, current: 34.72, average: 35.69
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1027, dropped: 0, current: 34.82, average: 35.68
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1045, dropped: 0, current: 35.58, average: 35.67
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1063, dropped: 0, current: 34.97, average: 35.66
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1081, dropped: 0, current: 35.04, average: 35.65
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1099, dropped: 0, current: 34.99, average: 35.64
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1117, dropped: 0, current: 34.97, average: 35.63
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1135, dropped: 0, current: 34.99, average: 35.62
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1153, dropped: 0, current: 35.00, average: 35.61
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1171, dropped: 0, current: 35.01, average: 35.60
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1189, dropped: 0, current: 34.62, average: 35.58

As you can see, I could reach around 35fps smoothly. But 60fps (which is the limitation of sensor at full resolution) does not seem good :

root@imx8mpevk:~# gst-launch-1.0 -vvv imxcompositor_g2d name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1440 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1440 sink_1::height=1080 ! queue ! video/x-raw, width=1440, height=2160 ! fpsdisplaysink video-sink="ximagesink" text-overlay=false sync=false v4l2src device=/dev/video2 ! video/x-raw, width=1440, height=1080, framerate=60/1 ! comp.sink_0 v4l2src device=/dev/video3 ! video/x-raw, width=1440, height=1080, framerate=60/1 ! comp.sink_1
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstXImageSink:ximagesink0: sync = false
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/imxcompositor_g2d:comp.GstImxCompositorPad:sink_0: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstV4l2Src:v4l2src1.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:src: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/imxcompositor_g2d:comp.GstImxCompositorPad:sink_1: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:sink: caps = video/x-raw, width=(int)1440, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/imxcompositor_g2d:comp.GstAggregatorPad:src: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstXImageSink:ximagesink0.GstPad:sink: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)BGRx, width=(int)1440, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstXImageSink:ximagesink0: sync = false
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 12, dropped: 0, current: 22.68, average: 22.68
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 27, dropped: 0, current: 29.48, average: 26.01
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 42, dropped: 0, current: 28.46, average: 26.84
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 57, dropped: 0, current: 28.30, average: 27.21
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 76, dropped: 0, current: 36.65, average: 29.08
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 96, dropped: 0, current: 39.99, average: 30.83
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 117, dropped: 0, current: 40.04, average: 32.16
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 137, dropped: 0, current: 39.94, average: 33.10
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 158, dropped: 0, current: 40.24, average: 33.90
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 179, dropped: 0, current: 40.18, average: 34.53
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 200, dropped: 0, current: 39.91, average: 35.03
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 220, dropped: 0, current: 39.92, average: 35.42
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 240, dropped: 0, current: 39.90, average: 35.76
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 261, dropped: 0, current: 40.27, average: 36.08
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 282, dropped: 0, current: 39.89, average: 36.34
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 302, dropped: 0, current: 39.78, average: 36.55
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 323, dropped: 0, current: 40.38, average: 36.78
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 343, dropped: 0, current: 39.37, average: 36.92
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 363, dropped: 0, current: 39.96, average: 37.08
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 384, dropped: 0, current: 40.03, average: 37.23
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 404, dropped: 0, current: 39.81, average: 37.35
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 424, dropped: 0, current: 39.65, average: 37.45
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 444, dropped: 0, current: 39.55, average: 37.54
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 464, dropped: 0, current: 39.82, average: 37.63
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 484, dropped: 0, current: 39.78, average: 37.72
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 505, dropped: 0, current: 40.53, average: 37.82
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 526, dropped: 0, current: 40.05, average: 37.91
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 546, dropped: 0, current: 39.84, average: 37.98
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 567, dropped: 0, current: 40.11, average: 38.05
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 588, dropped: 0, current: 39.91, average: 38.11
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 609, dropped: 0, current: 40.11, average: 38.18
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 629, dropped: 0, current: 39.65, average: 38.22
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 649, dropped: 0, current: 39.93, average: 38.28
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 669, dropped: 0, current: 39.65, average: 38.31

Anyway, 30fps with full resolution meets my expectation.  I will continue with the streaming over network.

Thank you very much!

Khang

305 Views
khang_letruong
Senior Contributor III

Hi @dianapredescu,

Thanks for sharing the info. Happy holidays and wish you all the best!

Best Regards,
Khang

391 Views
khang_letruong
Senior Contributor III

Dear @dianapredescu,

I would like to know if you have the patchset related to vpu v4l2 btw the 2 BSPs to share because git format-patch generates huge number of patches and not all of them are relevant to the issue.

Best Regards,
Khang

0 Kudos
441 Views
khang_letruong
Senior Contributor III

Hi @dianapredescu,

Thanks very much for your effort to reproduce the issue as well to test the gst commands in newer BSP.

I am looking forward to hearing from your findings.

Best Regards,
Khang

0 Kudos