topic Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...? in i.MX Processors

is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

nkkang — Wed, 21 Aug 2013 04:45:00 GMT

Hello ..

i'm using MCIMX6Q-SD board

my mission is that two camera working together (display or encoding )

give some information about not working simultaneously

is this hw problem or driver problem or gstreamer ??

any body has solution this problem

[]my environments

- MCIMX6Q-SD board

- ubuntu image (ubuntu_oneriric_4.0.0)

- linux kernel version 3.0.35

- gstreamer

- camera module (ov5640 / ov5642)

[]running (before modprobe ov5640_camera_mipi / modprobe ov5642_camera / modprobe mxc_v4l2_capture)

1. gst-launch -v --gst-debug=*:2, mfw_v4lsrc device=/dev/video0 \
! mfw_isink axis-top=10 axis-left=400 disp-width=640 disp-height=480

2. gst-launch -v --gst-debug=*:2, mfw_v4lsrc device=/dev/video1 \
! mfw_isink axis-top=700 axis-left=400 disp-width=640 disp-height=480

--> not working together but one by one is working

1 was running and then 2 is running error message

Setting pipeline to PAUSED ...
0:00:00.957726667 6962    0x17050 ERROR             mfw_v4lsrc mfw_gst_v4lsrc.c:523:mfw_gst_v4lsrc_capture_setup: >>V4L_SRC: VIDIOC_S_INPUT failed
0:00:00.957896000 6962    0x17050 ERROR             mfw_v4lsrc mfw_gst_v4lsrc.c:1012:mfw_gst_v4lsrc_start: v4lsrc:error in opening the device
0:00:00.957991667 6962    0x17050 WARN                 basesrc gstbasesrc.c:2996:gst_base_src_activate_push:<mfwgstv4lsrc0> Failed to start in push mode
0:00:00.958117000 6962    0x17050 WARN                GST_PADS gstpad.c:734:gst_pad_set_active:<mfwgstv4lsrc0:src> Failed to activate pad
ERROR: Pipeline doesn't want to pause.
Setting pipeline to NULL ...
Freeing pipeline ...
[--->FINALIZE isink

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

LeonardoSandova — Thu, 22 Aug 2013 16:43:49 GMT

It is a known issue. LiuYing, is there already a patch to fix this?

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

nkkang — Thu, 22 Aug 2013 23:42:52 GMT

Mr. LeonardoSandovalGonzalez

Efforts have been made, I could not find the patch and version information (Ying Liu)

please give me the detail information about this problem

Thank you.

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

LeonardoSandova — Fri, 23 Aug 2013 16:19:44 GMT

HI,

we found the issue internally and Liu was the one fixing it. Lets wait Ying's input.

Leo

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

LiuYing — Mon, 26 Aug 2013 05:42:18 GMT

If I remember correctly, it is okay to capture 2 cameras' output with two different IPU capture channels, while the issue is to use IPU IC task to render the captured frames to display(one preview out of two has artifacts). I haven't resolved the issue yet.

To use GPU to render the captured frames to display is probably an alternative. Need comments from GPU guys.

If encoding is required only(no preview), I assume the kernel supports this as long as on limitation from VPU part.

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

LeonardoSandova — Mon, 26 Aug 2013 15:14:55 GMT

Thanks Liu. For the GPU work-around, could you construct an example pipeline?

Leo

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

nkkang — Thu, 29 Aug 2013 07:38:37 GMT

Mr.

thanks you reply...

- please give some information about setting IPU for two cameras input ( patch or changing point and so on)....

- is it possible that two camera encoding simultaneously ( not display ) ??

thank you...

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

LeonardoSandova — Thu, 29 Aug 2013 17:16:01 GMT

Two cameras capturing and data being encoded is possible. Just replace video sinks with file sinks. That should work without any patch.

Leo

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

nkkang — Fri, 30 Aug 2013 01:39:25 GMT

thank you reply

i'm already trying these thing... but not working simultaneously...

this is my scripts and error message .. give me your mention about this matter...

my scripts is correct ?

[environments ]

- MCIMX6Q-SD board

- ubuntu image (ubuntu_oneriric_4.0.0) and LTIB

- linux kernel version 3.0.35

- gstreamer

- camera module (ov5640 / ov5642)

[run scripts ]

      gst-launch -v --gst-debug=*:2, mfw_v4lsrc device=/dev/video0 \
      capture-mode=0 \
      ! queue ! vpuenc codec=0 \
      ! matroskamux ! filesink location=output0.mkv sync=false

      gst-launch -v --gst-debug=*:2, mfw_v4lsrc device=/dev/video1 \
     capture-mode=0 \
     ! queue ! vpuenc codec=0 \
     ! matroskamux ! filesink location=output1.mkv sync=false

1 and 2 runing at the same time. not working one of them .. and the other will be end

[error message]

i think that mfw_v4lsrc is problem from IPU....

.......

0:00:07.940555001 6784 0x124788 WARN	mfw_v4lsrc mfw_gst_v4lsrc.c:1210:mfw_gst_v4lsrc_buffer_new: no buffer available in pool
0:00:07.940808668 6784 0x124788 WARN	basesrc gstbasesrc.c:2582:gst_base_src_loop:<mfwgstv4lsrc0> error: Internal data flow error.
0:00:07.940954335 6784 0x124788 WARN	basesrc gstbasesrc.c:2582:gst_base_src_loop:<mfwgstv4lsrc0> error: streaming task paused, reason error (-5)

ERROR: from element /GstPipeline:pipeline0/MFWGstV4LSrc:mfwgstv4lsrc0: Internal data flow error.

Additional debug info:

gstbasesrc.c(2582): gst_base_src_loop (): /GstPipeline:pipeline0/MFWGstV4LSrc:mfwgstv4lsrc0:

streaming task paused, reason error (-5)

Execution ended after 6332854334 ns.

Setting pipeline to PAUSED ...

.......

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

LeonardoSandova — Fri, 30 Aug 2013 01:49:45 GMT

Pipelines look fine. BTW, Could you check the default mfw_v4lsrc's input property? If 1 is not the default, set it to 1 on the pipeline...

Leo

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

nkkang — Fri, 30 Aug 2013 05:01:13 GMT

you advice is ....... input=1 is correct ??

it is not working. and faling into the same situation ...

anyway.. if you give me the working pipeline on the SABRE 6QSD board, it will be very helped

[run scripts ]

      gst-launch -v --gst-debug=*:2, mfw_v4lsrc input=1 device=/dev/video0 \
      capture-mode=0 \
      ! queue ! vpuenc codec=0 \
      ! matroskamux ! filesink location=output0.mkv sync=false

      gst-launch -v --gst-debug=*:2, mfw_v4lsrc input=1 device=/dev/video1 \
     capture-mode=0 \
     ! queue ! vpuenc codec=0 \
     ! matroskamux ! filesink location=output1.mkv sync=false

thanks Mr.

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

LeonardoSandova — Fri, 30 Aug 2013 17:59:13 GMT

NK, you are right, two cameras running at the same time have problems. Could you please try this patch:

diff --git a/arch/arm/mach-mx6/board-mx6q_sabresd.c b/arch/arm/mach-mx6/board-mx6q_sabresd.c

index b5472df..c668065 100644

--- a/arch/arm/mach-mx6/board-mx6q_sabresd.c

+++ b/arch/arm/mach-mx6/board-mx6q_sabresd.c

@@ -1418,7 +1418,7 @@ static struct fsl_mxc_capture_platform_data capture_data[] = {

         .is_mipi = 0,

     }, {

         .csi = 1,

-        .ipu = 0,

+        .ipu = 1,

         .mclk_source = 0,

         .is_mipi = 1,

     },

@@ -1656,9 +1656,9 @@ static void __init fixup_mxc_board(struct machine_desc *desc, struct tag *tags,

}

static struct mipi_csi2_platform_data mipi_csi2_pdata = {

-    .ipu_id     = 0,

+    .ipu_id     = 1,

     .csi_id = 1,

-    .v_channel = 0,

+    .v_channel = 3,

     .lanes = 2,

     .dphy_clk = "mipi_pllref_clk",

     .pixel_clk = "emi_clk",

diff --git a/drivers/media/video/mxc/capture/ov5640_mipi.c b/drivers/media/video/mxc/capture/ov5640_mipi.c

index a373e5b..594d734 100644

--- a/drivers/media/video/mxc/capture/ov5640_mipi.c

+++ b/drivers/media/video/mxc/capture/ov5640_mipi.c

@@ -993,7 +993,7 @@ bool binning_on(void)

static void ov5640_set_virtual_channel(int channel)

{

     u8 channel_id;

-

+    channel=3;

     ov5640_read_reg(0x4814, &channel_id);

     channel_id &= ~(3 << 6);

     ov5640_write_reg(0x4814, channel_id | (channel << 6));

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

nkkang — Fri, 06 Sep 2013 02:48:57 GMT

Thanks your reply

but you patch has some problem. need your help

1. it is not working for recording at the same time and faling into the same situation ...

2. it is working for display at the same time ... but image is not good (mipi camera) ... change color between R and G ... and display is divided ...

thanks

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

LeonardoSandova — Fri, 06 Sep 2013 17:40:45 GMT

NK, can you execute your test on latest release?

https://community.freescale.com/docs/DOC-95560?et=watches.email.document

Leo

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

edison_fernande — Mon, 07 Oct 2013 21:17:45 GMT

Hi,

I'm able to enable both cameras at the same time by using the following patch:

Index: kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/ipu_csi_enc.c
===================================================================
--- kernel.orig/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/ipu_csi_enc.c 2013-10-07 14:42:58.927083104 -0600
+++ kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/ipu_csi_enc.c 2013-10-07 14:44:19.031082649 -0600
@@ -56,6 +56,24 @@
}
/*!
+ * Get CSI dma channel
+ * @param csi       csi number
+ *
+ * @return DMA channel
+ */
+static ipu_channel_t csi_get_dma_channel(int csi)
+{
+ switch(csi){
+ case 0:
+ return CSI_MEM0;
+ case 1:
+ return CSI_MEM1;
+ default:
+ return CSI_MEM0;
+ }
+}
+
+/*!
* CSI ENC enable channel setup function
*
* @param cam       struct cam_data * mxc capture instance
@@ -74,6 +92,8 @@
int csi_id;
#endif
+ uint32_t chan = csi_get_dma_channel(cam->csi);
+
CAMERA_TRACE("In csi_enc_setup\n");
if (!cam) {
printk(KERN_ERR "cam private is NULL\n");
@@ -162,13 +182,13 @@
}
#endif
- err = ipu_init_channel(cam->ipu, CSI_MEM, &params);
+ err = ipu_init_channel(cam->ipu, chan, &params);
if (err != 0) {
printk(KERN_ERR "ipu_init_channel %d\n", err);
return err;
}
- err = ipu_init_channel_buffer(cam->ipu, CSI_MEM, IPU_OUTPUT_BUFFER,
+ err = ipu_init_channel_buffer(cam->ipu, chan, IPU_OUTPUT_BUFFER,
       pixel_fmt, cam->v2f.fmt.pix.width,
       cam->v2f.fmt.pix.height,
       cam->v2f.fmt.pix.bytesperline,
@@ -180,7 +200,7 @@
printk(KERN_ERR "CSI_MEM output buffer\n");
return err;
}
- err = ipu_enable_channel(cam->ipu, CSI_MEM);
+ err = ipu_enable_channel(cam->ipu, chan);
if (err < 0) {
printk(KERN_ERR "ipu_enable_channel CSI_MEM\n");
return err;
@@ -197,18 +217,20 @@
*
* @return status
*/
-static int csi_enc_eba_update(struct ipu_soc *ipu, dma_addr_t eba, int *buffer_num)
+static int csi_enc_eba_update(unsigned int csi, struct ipu_soc *ipu, dma_addr_t eba, int *buffer_num)
{
int err = 0;
+ uint32_t chan = csi_get_dma_channel(csi);
+
pr_debug("eba %x\n", eba);
- err = ipu_update_channel_buffer(ipu, CSI_MEM, IPU_OUTPUT_BUFFER,
+ err = ipu_update_channel_buffer(ipu, chan, IPU_OUTPUT_BUFFER,
*buffer_num, eba);
if (err != 0) {
- ipu_clear_buffer_ready(ipu, CSI_MEM, IPU_OUTPUT_BUFFER,
+ ipu_clear_buffer_ready(ipu, chan, IPU_OUTPUT_BUFFER,
        *buffer_num);
- err = ipu_update_channel_buffer(ipu, CSI_MEM, IPU_OUTPUT_BUFFER,
+ err = ipu_update_channel_buffer(ipu, chan, IPU_OUTPUT_BUFFER,
*buffer_num, eba);
if (err != 0) {
pr_err("ERROR: v4l2 capture: fail to update "
@@ -217,7 +239,7 @@
}
}
- ipu_select_buffer(ipu, CSI_MEM, IPU_OUTPUT_BUFFER, *buffer_num);
+ ipu_select_buffer(ipu, chan, IPU_OUTPUT_BUFFER, *buffer_num);
*buffer_num = (*buffer_num == 0) ? 1 : 0;
@@ -234,6 +256,7 @@
{
cam_data *cam = (cam_data *) private;
int err = 0;
+ uint32_t irq;
CAMERA_TRACE("IPU:In csi_enc_enabling_tasks\n");
cam->dummy_frame.vaddress = dma_alloc_coherent(0,
@@ -250,8 +273,10 @@
     PAGE_ALIGN(cam->v2f.fmt.pix.sizeimage);
cam->dummy_frame.buffer.m.offset = cam->dummy_frame.paddress;
- ipu_clear_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF);
- err = ipu_request_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF,
+ irq = IPU_IRQ_CSI0_OUT_EOF + cam->csi;
+
+ ipu_clear_irq(cam->ipu, irq);
+ err = ipu_request_irq(cam->ipu, irq,
       csi_enc_callback, 0, "Mxc Camera", cam);
if (err != 0) {
printk(KERN_ERR "Error registering rot irq\n");
@@ -283,9 +308,11 @@
int csi_id;
#endif
- err = ipu_disable_channel(cam->ipu, CSI_MEM, true);
+ uint32_t chan = csi_get_dma_channel(cam->csi);
+
+ err = ipu_disable_channel(cam->ipu, chan, true);
- ipu_uninit_channel(cam->ipu, CSI_MEM);
+ ipu_uninit_channel(cam->ipu, chan);
if (cam->dummy_frame.vaddress != 0) {
dma_free_coherent(0, cam->dummy_frame.buffer.length,
@@ -341,7 +368,8 @@
/* free csi eof irq firstly.
* when disable csi, wait for idmac eof.
* it requests eof irq again */
- ipu_free_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF, cam);
+
+ ipu_free_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF + cam->csi, cam);
return ipu_disable_csi(cam->ipu, cam->csi);
}
Index: kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.c
===================================================================
--- kernel.orig/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.c 2013-10-07 14:42:58.939083104 -0600
+++ kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.c 2013-10-07 14:43:15.471083010 -0600
@@ -429,7 +429,7 @@
list_del(cam->ready_q.next);
list_add_tail(&frame->queue, &cam->working_q);
frame->ipu_buf_num = cam->ping_pong_csi;
- err = cam->enc_update_eba(cam->ipu, frame->buffer.m.offset,
+ err = cam->enc_update_eba(cam-> csi, cam->ipu, frame->buffer.m.offset,
   &cam->ping_pong_csi);
frame =
@@ -437,7 +437,7 @@
list_del(cam->ready_q.next);
list_add_tail(&frame->queue, &cam->working_q);
frame->ipu_buf_num = cam->ping_pong_csi;
- err |= cam->enc_update_eba(cam->ipu, frame->buffer.m.offset,
+ err |= cam->enc_update_eba(cam-> csi, cam->ipu, frame->buffer.m.offset,
    &cam->ping_pong_csi);
spin_unlock_irqrestore(&cam->queue_int_lock, lock_flags);
} else {
@@ -2615,7 +2615,7 @@
struct mxc_v4l_frame,
queue);
if (cam->enc_update_eba)
- if (cam->enc_update_eba(cam->ipu, ready_frame->buffer.m.offset,
+ if (cam->enc_update_eba(cam->csi, cam->ipu, ready_frame->buffer.m.offset,
&cam->ping_pong_csi) == 0) {
list_del(cam->ready_q.next);
list_add_tail(&ready_frame->queue,
@@ -2625,7 +2625,7 @@
} else {
if (cam->enc_update_eba)
cam->enc_update_eba(
- cam->ipu, cam->dummy_frame.buffer.m.offset,
+ cam->csi, cam->ipu, cam->dummy_frame.buffer.m.offset,
&cam->ping_pong_csi);
}
Index: kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.h
===================================================================
--- kernel.orig/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.h 2013-10-07 14:42:58.963083104 -0600
+++ kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.h 2013-10-07 14:43:15.471083010 -0600
@@ -178,7 +178,7 @@
struct v4l2_rect crop_defrect;
struct v4l2_rect crop_current;
- int (*enc_update_eba) (struct ipu_soc *ipu, dma_addr_t eba, int *bufferNum);
+ int (*enc_update_eba) (unsigned int csi, struct ipu_soc *ipu, dma_addr_t eba, int *bufferNum);
int (*enc_enable) (void *private);
int (*enc_disable) (void *private);
int (*enc_enable_csi) (void *private);

but when i'm displaying at 480p i get some interference in both channels:

The output looks better when using a 720p resolution:

I'm using kernel 3.0.35-4.0.0.

BTW: i also tried with the patch suggested by Leonardo with same results as NK Kang.

Any help will be appreciated!!

Regards,

Edison

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

rogerio_silva — Tue, 09 Sep 2014 12:46:06 GMT

Hi,

You can try the patch attached on the following page:

Displaying two cameras at the same time on i.MX6Q

Rgds

Rogerio

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

edison_fernande — Sat, 08 Nov 2014 00:14:20 GMT

Hi Rogerio,

Thanks for your help, I will give it a try.

Regards,

Edison

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

edison_fernande — Wed, 10 Dec 2014 23:21:14 GMT

Hi Rogerio,

I finally had the time to give a try to your patch and sadly I'm getting the same results as applying the patch I suggested in previous posts. I basically get some artifacts in the video when I'm capturing from both cameras.

I tested with kernel 3.0.35-4.1.0 and kernel 3.10.17-1.0.0_ga in a Sabre-SDP Qaud Core with the same results. I also tried with gstreamer and with the mxc_v4l2_overlay.out application and both produced the same results:

This is the command I'm using:

./mxc_v4l2_overlay.out -t 200 -ow 640 -oh 480 -di /dev/video0 & ./mxc_v4l2_overlay.out -t 200 -fg -ot 200 -ol 200 -ow 640 -oh 480 -di /dev/video1

Here is a picture of what I'm seeing:

It is not a very good image but at least you can see the artifacts (those horizontal stripes in both videos).

Any help will be appreciated.

Regards,

Edison

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

rogerio_silva — Thu, 11 Dec 2014 14:48:41 GMT

Hi Edison,

Is this problem happening only when you use both cameras at the same time?

If you display only one camera, does the problem happens?

Rgds

Rogerio

Re: is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

edison_fernande — Thu, 11 Dec 2014 16:46:22 GMT

Hi Rogerio,

I tried sending one capture to a fakesink and the other to the display with the same results.

If I only capture from one camera at a time the video is fine, the problem only appears when I'm capturing from both cameras at the same time.

Regards,

Edison