i.MX ProcessorsのトピックRe: i.mx6：two camera work at the same time

i.mx6：two camera work at the same time

GaoJianzhong — Thu, 13 Jun 2013 10:53:34 GMT

Hi,

I am using i.mx6q SD board to get two camera(ov5640 and ov3640) work at the same time.

It is no need to display the image on display, i just want to get the camera datas in memory and

save them to local files.

To confirm is it support two camera at the same time, I do the follow things:

1. Put the OV5640 camera image on display with mxc_v4l2_overlay.

2. then capture the OV3640 data to local file with use mxc_vpu_test

But i got following error when i do second step:

imx-ipuv3.0: handler already installed on irq 0

So i checked the drivers,and found that there is only CSI_MEM(which is CSI_MEM0),

I think that if want use two cameras at the same time,

It is need to add a CSI_MEM1 to let second camera data use CSI_MEM1 channel to memory.

Am i right? and is there any ideas to save two camera data to memory at same time?

Thank you.

Re: i.mx6：two camera work at the same time

LeonardoSandova — Thu, 13 Jun 2013 15:35:04 GMT

Hi Gao,

Unfortunately, the system can not handle two captures and two video sinks simultaneously. But in your case is different, you want to just capture + [encode] + save, right? before digging into the code, try to use GStreamer, which can be seen as a wrapper to codecs. Try these pipelines :

gst-launch mfw_v4lsrc device="/dev/video0" ! filesink location=camera0.yuv &

gst-launch mfw_v4lsrc device="/dev/video1" ! filesink location=camera1.yuv

In case you need to stored encoded data, just add vpuenc between the src and the sink. BTW, before running these, make sure you have the devices nodes (/dev/video[0|1]) and the modules loaded (modprobe ...).

Leo

Re: i.mx6：two camera work at the same time

GaoJianzhong — Fri, 14 Jun 2013 03:43:28 GMT

Hi Leo,

Thanks for your help.

I tried the way you said. But the error is the same:

imx-ipuv3 imx-ipuv3.0: handler already installed on irq 0

Error registering rot irq

It seems that BSP only support the CSI_MEM0,

Now I added a driver use CSI_MEM1, and it is worked now.

And another question,

I want to get following worked:

Can gst-launch do this?

Thank you.

Re: i.mx6：two camera work at the same time

LeonardoSandova — Fri, 14 Jun 2013 15:29:03 GMT

yes.

gst-launch videotestsrc ! tee name=t ! queue ! vpuenc codec=avc ! filesink location=file.h264 t. ! queue ! vpuenc codec=mjpg ! filesink location=file.mjpg

Re: i.mx6：two camera work at the same time

linuslin — Wed, 07 Aug 2013 04:14:41 GMT

Hi, Gao,

Would you mind sharing the vendor or source of your camera module?

Thank you

Linus

Re: i.mx6：two camera work at the same time

edison_fernande — Wed, 28 Aug 2013 14:44:44 GMT

Hi Gao,

I'm using a Sabre SDP board and i want to use both cameras at the same time and i realized that you were able to make them work and i wonder if you could guide me a little bit through the steps you followed to have the 2 cameras working at the same time.

Regards,

Edison

Re: i.mx6：two camera work at the same time

GaoJianzhong — Thu, 29 Aug 2013 01:37:31 GMT

You just need to copy the "drivers/media/video/mxc/capture/ipu_csi_enc.c", name it to ipu_csi_enc1.c.

And modify the ipu_csi_enc.c 's csi to use csi0,

modify the ipu_csi_enc1.c 's csi to use csi1.

And add some processes of ipu_csi_enc1.c and Makefile.

In tha application, you can use ioctl "VIDIOC_S_INPUT" with 1 & 2 to control which camera you use.

I attached the file i modified, you can referenc them.

It may be not all the files needs to modify, i 'm not sure whi !ch file i modified.

Re: i.mx6：two camera work at the same time

GaoJianzhong — Thu, 29 Aug 2013 01:41:26 GMT

How can i attach the source code to you ?

Re: i.mx6：two camera work at the same time

edison_fernande — Fri, 30 Aug 2013 23:07:09 GMT

Hi Gao,

I'm now able to use both cameras (i modified the ipu_csi_enc.c so the driver uses a different DMA channel for each camera) but when i'm displaying both channels one of them has some interference. I would like to compare what i'm doing with what you did. I don't know if you can attach source code here but if not my email is edison.fernandez@ridgerun.com.

Thanks,

Edison

Re: i.mx6：two camera work at the same time

edison_fernande — Mon, 07 Oct 2013 21:13:09 GMT

Hi,

I'm able to enable both cameras at the same time by using the following patch:

Index: kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/ipu_csi_enc.c

===================================================================

--- kernel.orig/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/ipu_csi_enc.c 2013-10-07 14:42:58.927083104 -0600

+++ kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/ipu_csi_enc.c 2013-10-07 14:44:19.031082649 -0600

@@ -56,6 +56,24 @@

}

/*!

+ * Get CSI dma channel

+ * @param csi       csi number

+ *

+ * @return DMA channel

+ */

+static ipu_channel_t csi_get_dma_channel(int csi)

+{

+ switch(csi){

+ case 0:

+ return CSI_MEM0;

+ case 1:

+ return CSI_MEM1;

+ default:

+ return CSI_MEM0;

+ }

+}

+

+/*!

* CSI ENC enable channel setup function

*

* @param cam       struct cam_data * mxc capture instance

@@ -74,6 +92,8 @@

int csi_id;

#endif

+ uint32_t chan = csi_get_dma_channel(cam->csi);

+

CAMERA_TRACE("In csi_enc_setup\n");

if (!cam) {

printk(KERN_ERR "cam private is NULL\n");

@@ -162,13 +182,13 @@

}

#endif

- err = ipu_init_channel(cam->ipu, CSI_MEM, &params);

+ err = ipu_init_channel(cam->ipu, chan, &params);

if (err != 0) {

printk(KERN_ERR "ipu_init_channel %d\n", err);

return err;

}

- err = ipu_init_channel_buffer(cam->ipu, CSI_MEM, IPU_OUTPUT_BUFFER,

+ err = ipu_init_channel_buffer(cam->ipu, chan, IPU_OUTPUT_BUFFER,

       pixel_fmt, cam->v2f.fmt.pix.width,

       cam->v2f.fmt.pix.height,

       cam->v2f.fmt.pix.bytesperline,

@@ -180,7 +200,7 @@

printk(KERN_ERR "CSI_MEM output buffer\n");

return err;

}

- err = ipu_enable_channel(cam->ipu, CSI_MEM);

+ err = ipu_enable_channel(cam->ipu, chan);

if (err < 0) {

printk(KERN_ERR "ipu_enable_channel CSI_MEM\n");

return err;

@@ -197,18 +217,20 @@

*

* @return status

*/

-static int csi_enc_eba_update(struct ipu_soc *ipu, dma_addr_t eba, int *buffer_num)

+static int csi_enc_eba_update(unsigned int csi, struct ipu_soc *ipu, dma_addr_t eba, int *buffer_num)

{

int err = 0;

+ uint32_t chan = csi_get_dma_channel(csi);

+

pr_debug("eba %x\n", eba);

- err = ipu_update_channel_buffer(ipu, CSI_MEM, IPU_OUTPUT_BUFFER,

+ err = ipu_update_channel_buffer(ipu, chan, IPU_OUTPUT_BUFFER,

*buffer_num, eba);

if (err != 0) {

- ipu_clear_buffer_ready(ipu, CSI_MEM, IPU_OUTPUT_BUFFER,

+ ipu_clear_buffer_ready(ipu, chan, IPU_OUTPUT_BUFFER,

        *buffer_num);

- err = ipu_update_channel_buffer(ipu, CSI_MEM, IPU_OUTPUT_BUFFER,

+ err = ipu_update_channel_buffer(ipu, chan, IPU_OUTPUT_BUFFER,

*buffer_num, eba);

if (err != 0) {

pr_err("ERROR: v4l2 capture: fail to update "

@@ -217,7 +239,7 @@

}

}

- ipu_select_buffer(ipu, CSI_MEM, IPU_OUTPUT_BUFFER, *buffer_num);

+ ipu_select_buffer(ipu, chan, IPU_OUTPUT_BUFFER, *buffer_num);

*buffer_num = (*buffer_num == 0) ? 1 : 0;

@@ -234,6 +256,7 @@

{

cam_data *cam = (cam_data *) private;

int err = 0;

+ uint32_t irq;

CAMERA_TRACE("IPU:In csi_enc_enabling_tasks\n");

cam->dummy_frame.vaddress = dma_alloc_coherent(0,

@@ -250,8 +273,10 @@

     PAGE_ALIGN(cam->v2f.fmt.pix.sizeimage);

cam->dummy_frame.buffer.m.offset = cam->dummy_frame.paddress;

- ipu_clear_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF);

- err = ipu_request_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF,

+ irq = IPU_IRQ_CSI0_OUT_EOF + cam->csi;

+

+ ipu_clear_irq(cam->ipu, irq);

+ err = ipu_request_irq(cam->ipu, irq,

       csi_enc_callback, 0, "Mxc Camera", cam);

if (err != 0) {

printk(KERN_ERR "Error registering rot irq\n");

@@ -283,9 +308,11 @@

int csi_id;

#endif

- err = ipu_disable_channel(cam->ipu, CSI_MEM, true);

+ uint32_t chan = csi_get_dma_channel(cam->csi);

+

+ err = ipu_disable_channel(cam->ipu, chan, true);

- ipu_uninit_channel(cam->ipu, CSI_MEM);

+ ipu_uninit_channel(cam->ipu, chan);

if (cam->dummy_frame.vaddress != 0) {

dma_free_coherent(0, cam->dummy_frame.buffer.length,

@@ -341,7 +368,8 @@

/* free csi eof irq firstly.

* when disable csi, wait for idmac eof.

* it requests eof irq again */

- ipu_free_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF, cam);

+

+ ipu_free_irq(cam->ipu, IPU_IRQ_CSI0_OUT_EOF + cam->csi, cam);

return ipu_disable_csi(cam->ipu, cam->csi);

}

Index: kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.c

===================================================================

--- kernel.orig/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.c 2013-10-07 14:42:58.939083104 -0600

+++ kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.c 2013-10-07 14:43:15.471083010 -0600

@@ -429,7 +429,7 @@

list_del(cam->ready_q.next);

list_add_tail(&frame->queue, &cam->working_q);

frame->ipu_buf_num = cam->ping_pong_csi;

- err = cam->enc_update_eba(cam->ipu, frame->buffer.m.offset,

+ err = cam->enc_update_eba(cam-> csi, cam->ipu, frame->buffer.m.offset,

   &cam->ping_pong_csi);

frame =

@@ -437,7 +437,7 @@

list_del(cam->ready_q.next);

list_add_tail(&frame->queue, &cam->working_q);

frame->ipu_buf_num = cam->ping_pong_csi;

- err |= cam->enc_update_eba(cam->ipu, frame->buffer.m.offset,

+ err |= cam->enc_update_eba(cam-> csi, cam->ipu, frame->buffer.m.offset,

    &cam->ping_pong_csi);

spin_unlock_irqrestore(&cam->queue_int_lock, lock_flags);

} else {

@@ -2615,7 +2615,7 @@

struct mxc_v4l_frame,

queue);

if (cam->enc_update_eba)

- if (cam->enc_update_eba(cam->ipu, ready_frame->buffer.m.offset,

+ if (cam->enc_update_eba(cam->csi, cam->ipu, ready_frame->buffer.m.offset,

&cam->ping_pong_csi) == 0) {

list_del(cam->ready_q.next);

list_add_tail(&ready_frame->queue,

@@ -2625,7 +2625,7 @@

} else {

if (cam->enc_update_eba)

cam->enc_update_eba(

- cam->ipu, cam->dummy_frame.buffer.m.offset,

+ cam->csi, cam->ipu, cam->dummy_frame.buffer.m.offset,

&cam->ping_pong_csi);

}

Index: kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.h

===================================================================

--- kernel.orig/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.h 2013-10-07 14:42:58.963083104 -0600

+++ kernel/linux-3.0.35-4.0.0/drivers/media/video/mxc/capture/mxc_v4l2_capture.h 2013-10-07 14:43:15.471083010 -0600

@@ -178,7 +178,7 @@

struct v4l2_rect crop_defrect;

struct v4l2_rect crop_current;

- int (*enc_update_eba) (struct ipu_soc *ipu, dma_addr_t eba, int *bufferNum);

+ int (*enc_update_eba) (unsigned int csi, struct ipu_soc *ipu, dma_addr_t eba, int *bufferNum);

int (*enc_enable) (void *private);

int (*enc_disable) (void *private);

int (*enc_enable_csi) (void *private);

but when i'm displaying at 480p i get some interference in both channels:

The output looks better when using a 720p resolution:

I'm using kernel 3.0.35-4.0.0.

Any help will be appreciated!!

Regards,

Edison

Re: i.mx6：two camera work at the same time

GaoJianzhong — Tue, 08 Oct 2013 00:52:51 GMT

Sorry to later.

I have the same result.

Is it an analog reason.

Re: i.mx6：two camera work at the same time

LeonardoSandova — Tue, 08 Oct 2013 15:13:10 GMT

GaoJianzhong please mark this thread as unanswered. karinavalencia we need to escalate this issue, several customer are asking for a fix.

Leo

Re: i.mx6：two camera work at the same time

GaoJianzhong — Wed, 09 Oct 2013 00:00:45 GMT

I can't access the url by Karina Valencia Aguilar,

https://community.freescale.com/thread/313747?prevContainerType=14&prevContainerID=2004

Re: i.mx6：two camera work at the same time

karina_valencia — Wed, 09 Oct 2013 13:37:31 GMT

As soon as we have an update engineer will post the comments here.

Re: i.mx6：two camera work at the same time

Raana — Fri, 11 Oct 2013 11:57:00 GMT

Hi edison.fernandez,

I was able to get the OV5640 MIPI and OV7725 working individually on our custom i.Mx6Q board. But the problem is, both a camera not working at the same time on different display (LCD and HDMI).

I got some IPU related issues.

At that time I tried your patch. But I got some other issue like, imx-ipuv3 imx-ipuv3.0: handler already installed on irq 0

What can i do further ?

Thanks in Advance.

Best Regards,

J.P.Raja

Re: i.mx6：two camera work at the same time

edison_fernande — Fri, 11 Oct 2013 14:06:07 GMT

Hi Raana,

If you see at the ipu_csi_enc.c you'll see that for both cameras the driver is using the same irq and DMA channel and for that reason it is complaining when you try to use both cameras at the same time. What i did is assign the irq and channel depending on the camera so you don't have a resource conflict.

You can take a look at the patch a posted above. It is for kernel version 3.0.35-4.0.0, if you are using a different version then you'll need to modify it but it should work.

Edison

Re: i.mx6：two camera work at the same time

rogerio_silva — Thu, 17 Oct 2013 18:34:30 GMT

Hi,

Unfortunately with current drivers it's not possible to process on ipu 2 signals coming from 2 different CSIs (although the processor is capable to do it).

You can use at the same time 2 mxc_v4l2_capture.out instances to caputure. One using csi->prp_enc->mem and other using csi->mem:

./mxc_v4l2_capture.out -i 1 -d /dev/video1 test.aaa & ./mxc_v4l2_capture.out -d /dev/video0 test.bbb

This way you will record the 2 video files at the same time.

Rgds

Rogerio

Re: i.mx6：two camera work at the same time

edison_fernande — Mon, 21 Oct 2013 18:31:02 GMT

Hi Rogerio,

I know this thread is closed but i'm still having the same results even with your recommendation. I used gstreamer to capture video from both cameras at the same time (using a Sabre-SDP), thhis is the pipeline i used:

gst-launch -e mfw_v4lsrc input=0 ! 'video/x-raw-yuv,width=176,height=144' ! vpuenc codec=6 ! matroskamux ! filesink location=test0.mkv mfw_v4lsrc device=/dev/video1 input=1 ! vpuenc codec=6 ! matroskamux ! filesink location=test1.mkv

As you can see in the attached videos, test0.mkv has an annoying interference. This is exactly the same results i get by using the patch i mentioned previously. Is this an analog issue or is a conflict with DMA channels?. I don't think it has to be with the memory because with the method that you suggested both cameras are using totally different channels and by applying the patch the DMA channel is assigned depending on the CSI.

Any advice wold be appreciated!!

Regards,

Edison

Re: i.mx6：two camera work at the same time

karina_valencia — Mon, 21 Oct 2013 18:45:51 GMT

rogeriopimentel can you continue with the follow up on this case?

Re: i.mx6：two camera work at the same time

makislivadas — Mon, 21 Oct 2013 20:06:42 GMT

I experienced same issue with my board after modifying the driver as per similar thread below:

OV5640 and OV5642 cameras on iMX6

I have now upgraded my sources to L3.0.35_4.1.0, which is the latest kernel and drivers.

Clearly Freescale has added more changes to the drivers. Unfortunately ipu_csi_enc.c (driver) is now worse off.

gestreamer seems to use this driver (the way I use it) to capture video from the cameras and before modifying it I could capture with the previous release L3.0.35_4.0.0 with one camera at the time. In current release once one capture is enabled the next time I try to capture I get te irq error mentioned here even if I stop the previous capture session. Clearly some sort of regression.

However, when the capture software uses ipu_prp_enc.c driver, the behaviour is OK. Both cameras can be used but not at the same time of course. It seems that the only way we can get 2 cameras at the same time without having to modified the release software is by using ipu_csi_enc.c (driver) for one camera and ipu_prp_enc.c (driver) for the other. Is there a way to do this with gstreamer?

I have not managed to get the cameras to work yet with the latest kernel, but the display artifacts when 2 camera are used are an issue. I found that if I swap the two cameras around the artifacts are less prominent. Is the fact that on the SabreSD board there is a MIPI and a CSI camera used which do not have the same interface that causes the artifacts?