imx6: display several camera with gstreamer

[vaudoitlaurent]

Hi, 

on a imx6dp board, we can plug up to 4 analogic camera;

Im6 is connected through mipi csi2 to isl79987 video decoder.

we have applied the nxp patch to allow up to 4 channel (bsp allow only 2).

With the mxc_v4l2_tvin nxp application (provided with the patch), we are able to get 4 camera display at the same time.

Now we would like to get it work with gstreamer.

gst-launch imxv4l2src device=/dev/video ! imxv4l2sink overlay-left=0 overlay-top=0 overlay-width=960 overlay-height=360 works well

but as far as i relaunch same command on device1 (with lef/top value updated), , i get some  IPU Warning - IPU_INT_STAT_5 = 0x00000002 logs, and first image disappear, and nothing more on the screen.

So i was wondering if the imx gstreamer plugins allows several cpature/display usage?

Another question, i would like to set Contrast/saturation... through the v4l2 ioctl system.

gst plugin v4l2src allow it, but it seems it is not the case for the imxv4l2src. Am i right?

Thanks in advance for your help

Regards

Laurent

[vaudoitlaurent]

Hi,

we allready have the patch installed, and it works well.

i'm able to launch the mxc_tvin application for each camera, and they all are displayed.

problem is only with gstreamer trying to get several device. (gstreamer with one video device works perfectly)

Laurent

[joanxie]

give me your gstreamer commands you use and the log file, tvin unit test is for deinterlacing, so did you try to add deinterlacing by IPU?

[vaudoitlaurent]

Hi,

i'm not a specialist of gstreamer, i've tested the command:

gst-launch-1.0 -e imxv4l2src device=/dev/video0 ! imxv4l2sink overlay-left=0 overlay-top=0 overlay-width=960 overlay-height=360 &
gst-launch-1.0 -e imxv4l2src device=/dev/video1 ! imxv4l2sink overlay-left=960 overlay-top=0 overlay-width=960 overlay-height=360 &

after first command i get this log:

mxc_mipi_csi2 21dc000.mipi: mipi_csi2_reset: mipi_lane_bps = 432 Mbps
[ 46.494474] mxc_mipi_csi2 21dc000.mipi: mipi_csi2_reset: value = 0xc.
====== IMXV4L2SRC: 4.5.1 build on Jan 1 1970 00:00:00. ======
====== IMXV4L2SINK: 4.5.1 build on Jan 1 1970 00:00:00. ======
Setting pipeline to PAUSED ...
display(/dev/fb0) resolution is (1920x720).
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[ 46.792188]
[ 46.792188] In csi_enc_setup cam->is_mipi_cam:1 cam->v2f.fmt.pix.field:0
v4l2sink need allocate 3 buffers.

after second command, i get this:

mxc_mipi_csi2 21dc000.mipi: mipi_csi2_reset: mipi_lane_bps = 432 Mbps
[ 90.580914] mxc_mipi_csi2 21dc000.mipi: mipi_csi2_reset: value = 0xc.
====== IMXV4L2SRC: 4.5.1 build on Jan 1 1970 00:00:00. ======
====== IMXV4L2SINK: 4.5.1 build on Jan 1 1970 00:00:00. ======
Setting pipeline to PAUSED ...
display(/dev/fb0) resolution is (1920x720).
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[ 90.841594]
[ 90.841594] In csi_enc_setup cam->is_mipi_cam:1 cam->v2f.fmt.pix.field:0
v4l2sink need allocate 3 buffers.
ERROR: from element /GstPipeline:pipeline0/GstImxV4l2Src:imxv4l2src0: Internal data stream error.
Additional debug info:
../../../../git/libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstImxV4l2Src:imxv4l2src0:
streaming stopped, reason error (-5)
EOS on shutdown enabled -- waiting for EOS after Error
Waiting for EOS...
Got EOS from element "pipeline0".
EOS received - stopping pipeline...
Execution ended after 0:00:00.150385000
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Total showed frames (0), playing for (0:00:00.150723000), fps (0.000).
Freeing pipeline ...
[ 91.150052] imx-ipuv3 2800000.ipu: IPU Warning - IPU_INT_STAT_5 = 0x00000002
[ 92.702781] imx-ipuv3 2800000.ipu: IPU Warning - IPU_INT_STAT_5 = 0x00000002
[ 93.062126] imx-ipuv3 2800000.ipu: IPU Warning - IPU_INT_STAT_5 = 0x00000002
[ 94.539621] imx-ipuv3 2800000.ipu: IPU Warning - IPU_INT_STAT_5 = 0x00000002
[ 96.256706] imx-ipuv3 2800000.ipu: IPU Warning - IPU_INT_STAT_5 = 0x00000002

I've also tried in a same command, 

gst-launch-1.0 -e imxv4l2src device=/dev/video0 ! imxv4l2sink overlay-left=0 overlay-top=0 overlay-width=960 overlay-height=360 \
imxv4l2src device=/dev/video1 ! imxv4l2sink overlay-left=960 overlay-top=0 overlay-width=960 overlay-height=360

i have only device1 displayed (on the position of device 0)

and i get this log:

28.406061] mxc_mipi_csi2 21dc000.mipi: mipi_csi2_reset: mipi_lane_bps = 432 Mbps
[ 28.413634] mxc_mipi_csi2 21dc000.mipi: mipi_csi2_reset: value = 0xc.
[ 28.826480] mxc_mipi_csi2 21dc000.mipi: mipi_csi2_reset: mipi_lane_bps = 432 Mbps
[ 28.834015] mxc_mipi_csi2 21dc000.mipi: mipi_csi2_reset: value = 0xc.
====== IMXV4L2SRC: 4.5.1 build on Jan 1 1970 00:00:00. ======
====== IMXV4L2SINK: 4.5.1 build on Jan 1 1970 00:00:00. ======
====== IMXV4L2SRC: 4.5.1 build on Jan 1 1970 00:00:00. ======
[ 28.979106] alloc_contig_range: [22280, 222ff) PFNs busy====

Setting pipeline to PAUSED ...
display(/dev/fb0) resolution is (1920x720).
display(/dev/fb0) resolution is (1920x720).
Pipeli[ 28.994283] alloc_contig_range: [22280, 222ff) PFNs busy
ne is live and does not need PRER[ 29.003737] alloc_contig_range: [22380, 223ff) PFNs busy
OLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[ 29.018195] alloc_contig_range: [22280, 222ff) PFNs busy
[ 29.024937] alloc_contig_range: [22380, 223ff) PFNs busy
[ 29.030674] alloc_contig_range: [22280, 222ff) PFNs busy
[ 29.037279] alloc_contig_range: [22380, 223ff) PFNs busy
[ 29.048542] alloc_contig_range: [22280, 222ff) PFNs busy
[ 29.056713] alloc_contig_range: [22380, 223ff) PFNs busy
[ 29.062370] alloc_contig_range: [22280, 222ff) PFNs busy
[ 29.125220]
[ 29.125220] In csi_enc_setup cam->is_mipi_cam:1 cam->v2f.fmt.pix.field:0
[ 29.146302]
[ 29.146302] In csi_enc_setup cam->is_mipi_cam:1 cam->v2f.fmt.pix.field:0
v4l2sink need allocate 3 buffers.
v4l2sink need allocate 3 buffers.
[ 29.334872] mxc_v4l2_output v4l2_out: video output already run

" so did you try to add deinterlacing by IPU?"

no, how can i tst this?

thanks for your help

Laurent

[vaudoitlaurent]

setting some GSTREAMER DEBUG log, i can see whan i launch the second pipelin on the second device

the following log

0:00:00.707456000 418 0xcbcbb0 ERROR imxv4l2sink gstimxv4l2sink.c:788:gst_imx_v4l2sink_show_frame:<imxv4l2sink0> active pool(0x75d071a8) failed.
ERROR: from element /GstPipeline:pipeline0/GstImxV4l2Src:imxv4l2src0: Internal data stream error.

by the way if i use waylandsink in both pipelin instead of imxv4l2src, it seems working -i have no error and both camera are displayed 

Laurent

[joanxie]

is it your typo? you want to say waylandsink instead of imxv4l2sink, right?  I suggest that you use waylandsink, overlaysink or glimagesink

i.MX 8 family:
• waylandsink: video sink based on Wayland interfaces
i.MX 8M:
• kmssink: video sink based on the DCSS KMS driver, only for the second display in
dual-display case
i.MX 6 Family:
• overlaysink : G2D-based video sink plugin
• imxv4l2sink: V4L2-based video sink plugin
i.MX with GPU3D:
• glimagesink: video sink based on EGL

[vaudoitlaurent]

I've seen also that if i launch the nxp test app with g2d option, it works for several cam, if i rmeove the g2d option, i get the same behaviour as with gstreamer.

i've seen there is a plugin imxg2dsink existing, but i do not have it on my board.

Is this plugin could help, and how to build and deploy it?

[joanxie]

you can use imxvideoconvert_g2d, for more detailed information, pls refer to the chapter 7.3.15 Video conversion of enclosed file

[vaudoitlaurent]

Thanks, 

i start try it.

it seems better, but i'm not able to get the image displaying at the correct position.

it seems it creates a whole screen surface, and display the camera in  a little surface inside this one.

I have to integrate this in an embedded wizard HMI.

[vaudoitlaurent]

Hi,

with your help, i'm now able to display my 4 camera on the same screen.

for this i use the follwing pipeline:

gst-launch-1.0 \
imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=320 sink_0::height=240 \
sink_1::xpos=320 sink_1::ypos=0 sink_1::width=320 sink_1::height=240 \
sink_2::xpos=0 sink_2::ypos=240 sink_2::width=320 sink_2::height=240 \
sink_3::xpos=320 sink_3::ypos=240 sink_3::width=320 sink_3::height=240 ! \
overlaysink overlay-left=640 overlay-top=120 overlay-width=640 overlay-height=480 \
imxv4l2src device=/dev/video0 ! comp.sink_0 \
imxv4l2src device=/dev/video1 ! comp.sink_1 \
imxv4l2src device=/dev/video2 ! comp.sink_2 \
imxv4l2src device=/dev/video3 ! comp.sink_3

Now i would like to crops the image, i was able to do it with the imxv4l2sink plugin,

but now i don't see how to do this with imxcompositor/overlaysink.

How can i do this?

Regards

Laurent

[diwakar2599]

the same command i used for two camera .but it shows me syntax ,please tell me how to clear this

[joanxie]

I'm not sure what you mean, it seems this is new topic, if you mean resize, you also can refer to the user guide, try to fin "Composite three videos into one with CSC, resize, rotate, alpha, z-order, and keep aspect ratio"

[vaudoitlaurent]

Hi,

i agree this can be a new toppic.

by cropping i mean the option we can use in imxv4l2sink, to crop the image

(for example remove some line at the bottom, which can be poor quality coming from analogic camera)

for me resize is not the same thing as we keep the whole image.

Here i want to be able to remove a part of the image.

I've been able to do it by pathing some code in the imxcompositor code, but i'm not sure this is the correct solution

Laurent

[vaudoitlaurent]

Hi again,

i just forgot to mention that we are based on fsl bsp release, thud or warrior version,

with a 4.19.35 kernel

Regards

Laurent

[joanxie]

for surround view solution, I suggest that you can refer to the document as below:

"https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/iMX6DQ-ISL79985-79987-MIPI-CSI2-CVBS-cam..."