imx8 plus GStreamer capture dual cameras problem

[Creative]

good day
Our iMX8 PLUS platform
We have two cameras daA3840-30mc
Trying to get images from two cameras simultaneously using GStreame
Enter the following into the console:

gst-launch-1.0 -v imxcompositor_g2d name = com sink_1 :: xpos = 0 sink_1 :: ypos = 0 sink_1 :: width = 1920 sink_1 :: height = 1080 sink_1 :: keep-ratio = true sink_0 :: xpos = 1920 sink_0 :: ypos = 0 sink_0 :: width = 1920 sink_0 :: height = 1080 sink_0 :: keep-ratio = true! waylandsink sync = false v4l2src device = / dev / video0 io-mode = 4! com.sink_0 v4l2src device = / dev / video1 io-mode = 4! com.sink_1

We see two images, but the delay is large, 40 seconds between frames.
What to do?

[Creative]

gst-launch-1.0 -v imxcompositor_g2d name=com sink_1::xpos=0 sink_1::ypos=0 sink_1::width=1920 sink_1::height=1080 sink_1::keep-ratio=true sink_0::xpos=1920 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_0::keep-ratio=true! waylandsink sync=false v4l2src device=/dev/video0 io-mode=4 ! com.sink_0 v4l2src device=/dev/video1 io-mode=4 ! com.sink_1

[Bio_TICFSL]

Hello Creative,

For dual camera you can use this:

$ gst-launch-1.0 -v imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=640 sink_0::height=480 \
sink_1::xpos=0 sink_1::ypos=480 sink_1::width=640 sink_1::height=480 \
! video/x-raw,format=RGB16 ! waylandsink \
v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480 ! comp.sink_0 \
v4l2src device=/dev/video1 ! video/x-raw,width=640,height=480 ! comp.sink_1 \
17

Regards

[Creative]

Thank you for your answer, we tried this at 640x480, and the speed was about 15 frames per second, but we need an image size of 1920x1080, with the same number of frames, how to do this?

[witekewert-b463]

With BSP 5.10.35_2.0.0​ I was able to get 40 fps, with not much visible delay. fdtfile is set to imx8mp-evk-dual-basler.dts. The image size for ISP capture is set in /opt/imx8-isp/bin/start_isp.sh, this BSP version sets it automatically based on how many basler cameras are set up in the devicetree. In older versions, start_isp.sh had to be modified for 1080p use case.

root@imx8mpevk:~# gst-launch-1.0 -v imxcompositor_g2d name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=1920 sink_0::height=1080 sink_1::xpos=0 sink_1::ypos=1080 sink_1::width=1920 sink_1::height=1080 ! video/x-raw,format=RGB16 ! waylandsink window-width=1920 window-height=1080 v4l2src device=/dev/video2 ! video/x-raw,width=1920,height=1080 ! comp.sink_0 v4l2src device=/dev/video3 ! video/x-raw,width=1920,height=1080 ! comp.sink_1
[ 114.739991] enter isp_mi_stop
[ 114.772055] enter isp_mi_stop
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[ 114.863752] enter isp_mi_stop
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/imxcompositor_g2d:comp.GstImxCompositorPad:sink_0: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstV4l2Src:v4l2src1.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2[ 115.071060] enter isp_mi_stop
, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/imxcompositor_g2d:comp.GstImxCompositorPad:sink_1: caps = video/x-raw, width=(int)1920, height=(i[ 115.093169] enter isp_mi_stop
nt)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
[ 115.578948] enter isp_s_comp
[ 115.581861] enter isp_s_comp
[ 115.584795] enter isp_s_comp
[ 115.604107] enter wdr3_hw_init
[ 115.607180] wdr3 res: 1920 1080
[ 115.611035] enter isp_mi_start
[ 115.629615] enter isp_s_comp
[ 115.632532] enter isp_s_comp
[ 115.635430] enter isp_s_comp
[ 115.652254] enter wdr3_hw_init
[ 115.655317] wdr3 res: 1920 1080
[ 115.659218] enter isp_mi_start
/GstPipeline:pipeline0/imxcompositor_g2d:comp.GstAggregatorPad:src: caps = video/x-raw, format=(string)RGB16, width=(int)1920, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)RGB16, width=(int)1920, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstWaylandSink:waylandsink0.GstPad:sink: caps = video/x-raw, format=(string)RGB16, width=(int)1920, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)RGB16, width=(int)1920, height=(int)2160, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)60/1
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:17.553721000
Setting pipeline to NULL ...
Total showed frames (715), playing for (0:00:17.553709625), fps (40.732).
[ 132.549881] enter isp_mi_stop
[ 132.552920] enter isp_mi_stop
[ 132.674090] enter isp_mi_stop
[ 132.712784] enter isp_mi_stop
Freeing pipeline ...
root@imx8mpevk:~#

Displaying the two streams seems to be a limitation when using gst-launch. If I scale the preview down 2x, the frame rate increases:

gst-launch-1.0 -v imxcompositor_g2d name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=960 sink_0::height=540 sink_1::xpos=0 sink_1::ypos=540 sink_1::width=960 sink_1::height=540 ! video/x-raw,format=RGB16 ! waylandsink window-width=960 window-height=1080 v4l2src device=/dev/video2 ! video/x-raw,width=1920,height=1080 ! comp.sink_0 v4l2src device=/dev/video3 ! video/x-raw,width=1920,height=1080 ! comp.sink_1

Total showed frames (385), playing for (0:00:07.281218625), fps (52.876).