is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

is it possible that two cameras encoding or display simultaneously in SARBRA-SDP board ...?

4,012 Views
nkkang
Contributor II

Hello ..

i'm using MCIMX6Q-SD board

my mission is that two camera working together (display or encoding )

give some information about not working simultaneously

is this hw problem or driver problem or gstreamer ??

any body has solution this problem

[]my environments

   - MCIMX6Q-SD board

   - ubuntu image (ubuntu_oneriric_4.0.0)

   - linux kernel version 3.0.35

   - gstreamer

   - camera module (ov5640 / ov5642)

[]running (before  modprobe ov5640_camera_mipi / modprobe ov5642_camera /  modprobe mxc_v4l2_capture)

     1. gst-launch -v --gst-debug=*:2, mfw_v4lsrc device=/dev/video0 \
          ! mfw_isink  axis-top=10 axis-left=400 disp-width=640 disp-height=480

     2. gst-launch -v --gst-debug=*:2, mfw_v4lsrc device=/dev/video1 \
          ! mfw_isink  axis-top=700 axis-left=400 disp-width=640 disp-height=480

--> not working together  but one by one is working

1 was running and then 2 is running error message

Setting pipeline to PAUSED ...
0:00:00.957726667  6962    0x17050 ERROR             mfw_v4lsrc mfw_gst_v4lsrc.c:523:mfw_gst_v4lsrc_capture_setup: >>V4L_SRC: VIDIOC_S_INPUT failed
0:00:00.957896000  6962    0x17050 ERROR             mfw_v4lsrc mfw_gst_v4lsrc.c:1012:mfw_gst_v4lsrc_start: v4lsrc:error in opening the device
0:00:00.957991667  6962    0x17050 WARN                 basesrc gstbasesrc.c:2996:gst_base_src_activate_push:<mfwgstv4lsrc0> Failed to start in push mode
0:00:00.958117000  6962    0x17050 WARN                GST_PADS gstpad.c:734:gst_pad_set_active:<mfwgstv4lsrc0:src> Failed to activate pad
ERROR: Pipeline doesn't want to pause.
Setting pipeline to NULL ...
Freeing pipeline ...
[--->FINALIZE isink

Labels (2)
22 Replies

1,889 Views
LeonardoSandova
Specialist I

It is a known issue. LiuYing, is there already a  patch to fix this?

0 Kudos

1,889 Views
LiuYing
NXP Employee
NXP Employee

If I remember correctly, it is okay to capture 2 cameras' output with two different IPU capture channels, while the issue is to use IPU IC task to render the captured frames to display(one preview out of two has artifacts). I haven't resolved the issue yet.

To use GPU to render the captured frames to display is probably an alternative. Need comments from GPU guys.

If encoding is required only(no preview), I assume the kernel supports this as long as on limitation from VPU part.

0 Kudos

1,889 Views
nkkang
Contributor II

Mr.

thanks you reply...

- please give some information about setting IPU for two cameras input ( patch or changing point and so on)....

- is it possible that two camera encoding simultaneously ( not display ) ??

 

thank you...

0 Kudos

1,889 Views
LeonardoSandova
Specialist I

Two cameras capturing and data being encoded is possible. Just replace video sinks with file sinks. That should work without any patch.

Leo

0 Kudos

1,888 Views
nkkang
Contributor II

thank you reply

i'm already trying these thing... but not working simultaneously...

this is my scripts and error message .. give me your mention about this matter...

my scripts is correct ?

[environments ]

   - MCIMX6Q-SD board

   - ubuntu image (ubuntu_oneriric_4.0.0) and LTIB

   - linux kernel version 3.0.35

   - gstreamer

   - camera module (ov5640 / ov5642)

[run scripts ]

  1.

      gst-launch -v --gst-debug=*:2, mfw_v4lsrc device=/dev/video0 \
      capture-mode=0 \
      ! queue ! vpuenc codec=0 \
      ! matroskamux ! filesink location=output0.mkv sync=false

  2.

      gst-launch -v --gst-debug=*:2, mfw_v4lsrc device=/dev/video1 \
     capture-mode=0 \
     ! queue ! vpuenc codec=0 \
     ! matroskamux ! filesink location=output1.mkv sync=false

1 and 2 runing at the same time. not working one of them .. and the other will be end

[error message]

i think that mfw_v4lsrc is problem from IPU....

.......

  

0:00:07.940555001  6784   0x124788 WARN          mfw_v4lsrc mfw_gst_v4lsrc.c:1210:mfw_gst_v4lsrc_buffer_new: no buffer available in pool
0:00:07.940808668  6784   0x124788 WARN             basesrc gstbasesrc.c:2582:gst_base_src_loop:<mfwgstv4lsrc0> error: Internal data flow error.
0:00:07.940954335  6784   0x124788 WARN             basesrc gstbasesrc.c:2582:gst_base_src_loop:<mfwgstv4lsrc0> error: streaming task paused, reason error (-5)

ERROR: from element /GstPipeline:pipeline0/MFWGstV4LSrc:mfwgstv4lsrc0: Internal data flow error.

Additional debug info:

gstbasesrc.c(2582): gst_base_src_loop (): /GstPipeline:pipeline0/MFWGstV4LSrc:mfwgstv4lsrc0:

streaming task paused, reason error (-5)

Execution ended after 6332854334 ns.

Setting pipeline to PAUSED ...

.......

0 Kudos

1,889 Views
LeonardoSandova
Specialist I

Pipelines look fine. BTW, Could you  check the default mfw_v4lsrc's input property? If 1 is not the default, set it to 1 on the pipeline...

Leo

0 Kudos

1,889 Views
nkkang
Contributor II

you advice is .......  input=1 is correct ??

it is not working. and faling into the same situation ...

anyway.. if you give me the working pipeline on the SABRE 6QSD board, it will be very helped

[run scripts ]

  1.

      gst-launch -v --gst-debug=*:2, mfw_v4lsrc input=1 device=/dev/video0 \
      capture-mode=0 \
      ! queue ! vpuenc codec=0 \
      ! matroskamux ! filesink location=output0.mkv sync=false

  2.

      gst-launch -v --gst-debug=*:2, mfw_v4lsrc input=1 device=/dev/video1 \
     capture-mode=0 \
     ! queue ! vpuenc codec=0 \
     ! matroskamux ! filesink location=output1.mkv sync=false

thanks Mr.

NK

0 Kudos

1,889 Views
LeonardoSandova
Specialist I

NK, you are right, two cameras running at the same time have problems. Could you please try this patch:

diff --git a/arch/arm/mach-mx6/board-mx6q_sabresd.c b/arch/arm/mach-mx6/board-mx6q_sabresd.c

index b5472df..c668065 100644

--- a/arch/arm/mach-mx6/board-mx6q_sabresd.c

+++ b/arch/arm/mach-mx6/board-mx6q_sabresd.c

@@ -1418,7 +1418,7 @@ static struct fsl_mxc_capture_platform_data capture_data[] = {

         .is_mipi = 0,

     }, {

         .csi = 1,

-        .ipu = 0,

+        .ipu = 1,

         .mclk_source = 0,

         .is_mipi = 1,

     },

@@ -1656,9 +1656,9 @@ static void __init fixup_mxc_board(struct machine_desc *desc, struct tag *tags,

}

static struct mipi_csi2_platform_data mipi_csi2_pdata = {

-    .ipu_id     = 0,

+    .ipu_id     = 1,

     .csi_id = 1,

-    .v_channel = 0,

+    .v_channel = 3,

     .lanes = 2,

     .dphy_clk = "mipi_pllref_clk",

     .pixel_clk = "emi_clk",

diff --git a/drivers/media/video/mxc/capture/ov5640_mipi.c b/drivers/media/video/mxc/capture/ov5640_mipi.c

index a373e5b..594d734 100644

--- a/drivers/media/video/mxc/capture/ov5640_mipi.c

+++ b/drivers/media/video/mxc/capture/ov5640_mipi.c

@@ -993,7 +993,7 @@ bool binning_on(void)

static void ov5640_set_virtual_channel(int channel)

{

     u8 channel_id;

-

+    channel=3;

     ov5640_read_reg(0x4814, &channel_id);

     channel_id &= ~(3 << 6);

     ov5640_write_reg(0x4814, channel_id | (channel << 6));

0 Kudos

1,889 Views
nkkang
Contributor II

Thanks your reply

but you patch has some problem. need your help

1. it is not working for recording  at the same time  and faling into the same situation ...

2. it is working for display at the same time ... but image is not good (mipi camera) ... change color between R and G ... and display is divided ...

cam_test.png

thanks

NK

0 Kudos

1,889 Views
edison_fernande
Contributor III

Hi,

I'm able to enable both cameras at the same time by using the following patch:

Index: kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/ipu_csi_enc.c

===================================================================

--- kernel.orig/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/ipu_csi_enc.c 2013-10-07 14:42:58.927083104 -0600

+++ kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/ipu_csi_enc.c 2013-10-07 14:44:19.031082649 -0600

@@ -56,6 +56,24 @@

}

/*!

+ * Get CSI dma channel

+ * @param csi       csi number

+ *

+ * @return  DMA channel

+ */

+static ipu_channel_t csi_get_dma_channel(int csi)

+{

+ switch(csi){

+ case 0:

+ return CSI_MEM0;

+ case 1:

+ return CSI_MEM1;

+ default:

+ return CSI_MEM0;

+ }

+}

+

+/*!

  * CSI ENC enable channel setup function

  *

  * @param cam       struct cam_data * mxc capture instance

@@ -74,6 +92,8 @@

  int csi_id;

#endif

+ uint32_t chan = csi_get_dma_channel(cam->csi);

+

  CAMERA_TRACE("In csi_enc_setup\n");

  if (!cam) {

  printk(KERN_ERR "cam private is NULL\n");

@@ -162,13 +182,13 @@

  }

#endif

- err = ipu_init_channel(cam->ipu, CSI_MEM, &params);

+ err = ipu_init_channel(cam->ipu, chan, &params);

  if (err != 0) {

  printk(KERN_ERR "ipu_init_channel %d\n", err);

  return err;

  }

- err = ipu_init_channel_buffer(cam->ipu, CSI_MEM, IPU_OUTPUT_BUFFER,

+ err = ipu_init_channel_buffer(cam->ipu, chan, IPU_OUTPUT_BUFFER,

       pixel_fmt, cam->v2f.fmt.pix.width,

       cam->v2f.fmt.pix.height,

       cam->v2f.fmt.pix.bytesperline,

@@ -180,7 +200,7 @@

  printk(KERN_ERR "CSI_MEM output buffer\n");

  return err;

  }

- err = ipu_enable_channel(cam->ipu, CSI_MEM);

+ err = ipu_enable_channel(cam->ipu, chan);

  if (err < 0) {

  printk(KERN_ERR "ipu_enable_channel CSI_MEM\n");

  return err;

@@ -197,18 +217,20 @@

  *

  * @return  status

  */

-static int csi_enc_eba_update(struct ipu_soc *ipu, dma_addr_t eba, int *buffer_num)

+static int csi_enc_eba_update(unsigned int csi, struct ipu_soc *ipu, dma_addr_t eba, int *buffer_num)

{

  int err = 0;

+ uint32_t chan = csi_get_dma_channel(csi);

+

  pr_debug("eba %x\n", eba);

- err = ipu_update_channel_buffer(ipu, CSI_MEM, IPU_OUTPUT_BUFFER,

+ err = ipu_update_channel_buffer(ipu, chan, IPU_OUTPUT_BUFFER,

  *buffer_num, eba);

  if (err != 0) {

- ipu_clear_buffer_ready(ipu, CSI_MEM, IPU_OUTPUT_BUFFER,

+ ipu_clear_buffer_ready(ipu, chan, IPU_OUTPUT_BUFFER,

        *buffer_num);

- err = ipu_update_channel_buffer(ipu, CSI_MEM, IPU_OUTPUT_BUFFER,

+ err = ipu_update_channel_buffer(ipu, chan, IPU_OUTPUT_BUFFER,

  *buffer_num, eba);

  if (err != 0) {

  pr_err("ERROR: v4l2 capture: fail to update "

@@ -217,7 +239,7 @@

  }

  }

- ipu_select_buffer(ipu, CSI_MEM, IPU_OUTPUT_BUFFER, *buffer_num);

+ ipu_select_buffer(ipu, chan, IPU_OUTPUT_BUFFER, *buffer_num);

  *buffer_num = (*buffer_num == 0) ? 1 : 0;

@@ -234,6 +256,7 @@

{

  cam_data *cam = (cam_data *) private;

  int err = 0;

+ uint32_t irq;

  CAMERA_TRACE("IPU:In csi_enc_enabling_tasks\n");

  cam->dummy_frame.vaddress = dma_alloc_coherent(0,

@@ -250,8 +273,10 @@

     PAGE_ALIGN(cam->v2f.fmt.pix.sizeimage);

  cam->dummy_frame.buffer.m.offset = cam->dummy_frame.paddress;

- ipu_clear_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF);

- err = ipu_request_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF,

+ irq = IPU_IRQ_CSI0_OUT_EOF + cam->csi;

+

+ ipu_clear_irq(cam->ipu, irq);

+ err = ipu_request_irq(cam->ipu, irq,

       csi_enc_callback, 0, "Mxc Camera", cam);

  if (err != 0) {

  printk(KERN_ERR "Error registering rot irq\n");

@@ -283,9 +308,11 @@

  int csi_id;

#endif

- err = ipu_disable_channel(cam->ipu, CSI_MEM, true);

+ uint32_t chan = csi_get_dma_channel(cam->csi);

+

+ err = ipu_disable_channel(cam->ipu, chan, true);

- ipu_uninit_channel(cam->ipu, CSI_MEM);

+ ipu_uninit_channel(cam->ipu, chan);

  if (cam->dummy_frame.vaddress != 0) {

  dma_free_coherent(0, cam->dummy_frame.buffer.length,

@@ -341,7 +368,8 @@

  /* free csi eof irq firstly.

  * when disable csi, wait for idmac eof.

  * it requests eof irq again */

- ipu_free_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF, cam);

+

+ ipu_free_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF + cam->csi, cam);

  return ipu_disable_csi(cam->ipu, cam->csi);

}

Index: kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.c

===================================================================

--- kernel.orig/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.c 2013-10-07 14:42:58.939083104 -0600

+++ kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.c 2013-10-07 14:43:15.471083010 -0600

@@ -429,7 +429,7 @@

  list_del(cam->ready_q.next);

  list_add_tail(&frame->queue, &cam->working_q);

  frame->ipu_buf_num = cam->ping_pong_csi;

- err = cam->enc_update_eba(cam->ipu, frame->buffer.m.offset,

+ err = cam->enc_update_eba(cam-> csi, cam->ipu, frame->buffer.m.offset,

   &cam->ping_pong_csi);

  frame =

@@ -437,7 +437,7 @@

  list_del(cam->ready_q.next);

  list_add_tail(&frame->queue, &cam->working_q);

  frame->ipu_buf_num = cam->ping_pong_csi;

- err |= cam->enc_update_eba(cam->ipu, frame->buffer.m.offset,

+ err |= cam->enc_update_eba(cam-> csi, cam->ipu, frame->buffer.m.offset,

    &cam->ping_pong_csi);

  spin_unlock_irqrestore(&cam->queue_int_lock, lock_flags);

  } else {

@@ -2615,7 +2615,7 @@

  struct mxc_v4l_frame,

  queue);

  if (cam->enc_update_eba)

- if (cam->enc_update_eba(cam->ipu, ready_frame->buffer.m.offset,

+ if (cam->enc_update_eba(cam->csi, cam->ipu, ready_frame->buffer.m.offset,

  &cam->ping_pong_csi) == 0) {

  list_del(cam->ready_q.next);

  list_add_tail(&ready_frame->queue,

@@ -2625,7 +2625,7 @@

  } else {

  if (cam->enc_update_eba)

  cam->enc_update_eba(

- cam->ipu, cam->dummy_frame.buffer.m.offset,

+ cam->csi, cam->ipu, cam->dummy_frame.buffer.m.offset,

  &cam->ping_pong_csi);

  }

Index: kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.h

===================================================================

--- kernel.orig/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.h 2013-10-07 14:42:58.963083104 -0600

+++ kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.h 2013-10-07 14:43:15.471083010 -0600

@@ -178,7 +178,7 @@

  struct v4l2_rect crop_defrect;

  struct v4l2_rect crop_current;

- int (*enc_update_eba) (struct ipu_soc *ipu, dma_addr_t eba, int *bufferNum);

+ int (*enc_update_eba) (unsigned int csi, struct ipu_soc *ipu, dma_addr_t eba, int *bufferNum);

  int (*enc_enable) (void *private);

  int (*enc_disable) (void *private);

  int (*enc_enable_csi) (void *private);

but when i'm displaying at 480p i get some interference in both channels:

6658_6658.jpg480p.JPG.jpg

The output looks better when using a 720p resolution:

6662_6662.jpg720p.JPG.jpg

I'm using kernel 3.0.35-4.0.0.

BTW: i also tried with the patch suggested by Leonardo with same results as NK Kang.

Any help will be appreciated!!

Regards,

Edison

0 Kudos

1,889 Views
rogerio_silva
NXP Employee
NXP Employee

Hi,

You can try the patch attached on the following page:

Displaying two cameras at the same time on i.MX6Q

Rgds

Rogerio

0 Kudos

1,889 Views
edison_fernande
Contributor III

Hi Rogerio,

I finally had the time to give a try to your patch and sadly I'm getting the same results as applying the patch I suggested in previous posts. I basically get some artifacts in the video when I'm capturing from both cameras.

I tested with kernel 3.0.35-4.1.0 and kernel 3.10.17-1.0.0_ga in a Sabre-SDP Qaud Core with the same results. I also tried with gstreamer and with the mxc_v4l2_overlay.out application and both produced the same results:

This is the command I'm using:

./mxc_v4l2_overlay.out -t 200 -ow 640 -oh 480 -di /dev/video0 & ./mxc_v4l2_overlay.out -t 200 -fg -ot 200 -ol 200 -ow 640 -oh 480 -di /dev/video1

Here is a picture of what I'm seeing:

DualCapture.jpg

It is not a very good image but at least you can see the artifacts (those horizontal stripes in both videos).

Any help will be appreciated.

Regards,

Edison

0 Kudos

1,889 Views
rogerio_silva
NXP Employee
NXP Employee

Hi Edison,

Is this problem happening only when you use both cameras at the same time?

If you display only one camera, does the problem happens?

Rgds

Rogerio

0 Kudos

1,889 Views
edison_fernande
Contributor III

Hi Rogerio,

I tried sending one capture to a fakesink and the other to the display with the same results.

If I only capture from one camera at a time the video is fine, the problem only appears when I'm capturing from both cameras at the same time.

Regards,

Edison

0 Kudos

1,889 Views
rogerio_silva
NXP Employee
NXP Employee

I'll try to reproduce it.

0 Kudos

1,889 Views
rogerio_silva
NXP Employee
NXP Employee

Hi Edison,

I reproduced the problem on my board and I haven't found a possible solution.

A patch was officially applied to work with i.MX6SX (that doesn't have IPU):

http://git.freescale.com/git/cgit.cgi/imx/linux-2.6-imx.git/commit/?h=imx_3.10.31_1.1.0_beta2&id=755...

Maybe it can help you to find a solution for i.MX6DQ.

0 Kudos

1,889 Views
edison_fernande
Contributor III

Hi Rogerio,

Thanks for the update. I'll take a look at the patch and I'll let you know if I can find a solution.

Regards,

Edison

1,889 Views
edison_fernande
Contributor III

Hi Rogerio,

Thanks for your help, I will give it a try.

Regards,

Edison

0 Kudos

1,889 Views
LeonardoSandova
Specialist I

NK, can you execute your test on latest release?

https://community.freescale.com/docs/DOC-95560?et=watches.email.document

Leo

0 Kudos

1,889 Views
LeonardoSandova
Specialist I

Thanks Liu. For the GPU work-around, could you construct an example pipeline?

Leo

0 Kudos