topic Re: i.MX8M EVK MIPI CSI Camera Frame Rate in i.MX Processors

i.MX8M EVK MIPI CSI Camera Frame Rate

felixradensky — Mon, 28 May 2018 11:49:08 GMT

Hello,

We have purchased NXP MIPI-CSI camera MINISASTOCSI kit and followed the instructions in this application note https://www.nxp.com/docs/en/application-note/AN12187.pdf to test the camera on NXP i.MX8M EVK rev B3 running L4.9.51-ga Linux BSP release. Both camera connectors were tested. The camera output was sent to HDMI display. The frame rate we've received is around 7fps, even in lowest resolution. What should be done to get the maximum frame rates advertised by the application note above (e.g. 30fps at 1080p).

Thanks.

Felix.

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

lily_zhang — Mon, 28 May 2018 12:04:48 GMT

Hello, Felix:

Please check 2 changes:

- Please apply for tuning patches sent by email for eMMC test..

- Please enlarge GST_V4L2_MIN_BUFFERS in the following file from 2 to 7

/gstreamer1.0-plugins-good/1.12.2-r0/git/sys/v4l2/gstv4l2object.h :

#define GST_V4L2_MIN_BUFFERS 7

I will check whether more changes are required tomorrow.

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

felixradensky — Mon, 28 May 2018 17:17:38 GMT

Hello Lily,

With eMMC patches applied and number of buffers increased we get 18fps at 1080p

Felix

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

lily_zhang — Tue, 29 May 2018 01:44:35 GMT

Hello, Felix:

We will check further and feedback to you.

gogoer‌, FYI.

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

haidong_zheng — Tue, 29 May 2018 05:25:20 GMT

hi Felix:

Please check FPS by below command:

gst-launch-1.0 v4l2src ! video/x-raw,width=1920,height=1080 ! fpsdisplaysink

And please enable dma-coherent in fsl-imx8mq.dtsi for csix_bridge:

csi1_bridge: csi1_bridge@30a90000 {
compatible = "fsl,imx8mq-csi", "fsl,imx6s-csi";
reg = <0x0 0x30a90000 0x0 0x10000>;
interrupts = <GIC_SPI 42 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&clk IMX8MQ_CLK_DUMMY>,
<&clk IMX8MQ_CLK_CSI1_ROOT>,
<&clk IMX8MQ_CLK_DUMMY>;
clock-names = "disp-axi", "csi_mclk", "disp_dcic";
dma-coherent;
status = "disabled";
};

csi2_bridge: csi2_bridge@30b80000 {
compatible = "fsl,imx8mq-csi", "fsl,imx6s-csi";
reg = <0x0 0x30b80000 0x0 0x10000>;
interrupts = <GIC_SPI 43 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&clk IMX8MQ_CLK_DUMMY>,
<&clk IMX8MQ_CLK_CSI2_ROOT>,
<&clk IMX8MQ_CLK_DUMMY>;
clock-names = "disp-axi", "csi_mclk", "disp_dcic";
dma-coherent;
status = "disabled";
};

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

oferausterlitz — Tue, 29 May 2018 07:45:43 GMT

Hi Tom,

After adding dma-coherent property we get about 30fps at 1080p, which is great. We've also tested maximum resolution, 2592x1944 and received the following error:

0:00:00.643195654 510 0xff16ca0 ERROR v4l2bufferpool gstv4l2bufferpool.c:854:gst_v4l2_buffer_pool_start:<v4l2src0:pool:src> we received 6 buffer from device '/dev/video0', we want at least 7

With DMA coherency enabled, is it necessary to increase GST_V4L2_MIN_BUFFERS to 7 ? Can this parameter be modified at run-time instead of compile-time ?

Thanks a lot.

Felix.

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

haidong_zheng — Tue, 29 May 2018 08:27:09 GMT

hi Felix:

You can decrease GST_V4L2_MIN_BUFFERS or increase MAX_VIDEO_MEM in drivers/media/platform/mxc/capture/mx6s_capture.c to enable camera resolution 2592x1944 preview work, but this resolution performance is not good, that is about 8 FPS. I will check this.

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

haidong_zheng — Tue, 29 May 2018 09:37:05 GMT

hi Felix:

Please use new 2592x1944 setting, the MAX FPS supported by OV5640 is 15FPS.

static struct reg_value ov5640_setting_15fps_QSXGA_2592_1944[] = {
{0x3008, 0x42, 0, 0},
{0x4202, 0x0f, 0, 0}, /* stream off the sensor */
{0x3820, 0x40, 0, 0}, {0x3821, 0x06, 0, 0}, /*disable flip*/
{0x3035, 0x21, 0, 0}, {0x3036, 0xA8, 0, 0}, {0x3c07, 0x08, 0, 0},
{0x3c09, 0x1c, 0, 0}, {0x3c0a, 0x9c, 0, 0}, {0x3c0b, 0x40, 0, 0},
{0x3820, 0x40, 0, 0}, {0x3821, 0x06, 0, 0}, {0x3814, 0x11, 0, 0},
{0x3815, 0x11, 0, 0}, {0x3800, 0x00, 0, 0}, {0x3801, 0x00, 0, 0},
{0x3802, 0x00, 0, 0}, {0x3803, 0x00, 0, 0}, {0x3804, 0x0a, 0, 0},
{0x3805, 0x3f, 0, 0}, {0x3806, 0x07, 0, 0}, {0x3807, 0x9f, 0, 0},
{0x3808, 0x0a, 0, 0}, {0x3809, 0x20, 0, 0}, {0x380a, 0x07, 0, 0},
{0x380b, 0x98, 0, 0}, {0x380c, 0x0b, 0, 0}, {0x380d, 0x1c, 0, 0},
{0x380e, 0x07, 0, 0}, {0x380f, 0xb0, 0, 0}, {0x3810, 0x00, 0, 0},
{0x3811, 0x10, 0, 0}, {0x3812, 0x00, 0, 0}, {0x3813, 0x04, 0, 0},
{0x3618, 0x04, 0, 0}, {0x3612, 0x2b, 0, 0}, {0x3708, 0x21, 0, 0},
{0x3709, 0x12, 0, 0}, {0x370c, 0x00, 0, 0}, {0x3a02, 0x03, 0, 0},
{0x3a03, 0xd8, 0, 0}, {0x3a08, 0x01, 0, 0}, {0x3a09, 0x27, 0, 0},
{0x3a0a, 0x00, 0, 0}, {0x3a0b, 0xf6, 0, 0}, {0x3a0e, 0x03, 0, 0},
{0x3a0d, 0x04, 0, 0}, {0x3a14, 0x03, 0, 0}, {0x3a15, 0xd8, 0, 0},
{0x4001, 0x02, 0, 0}, {0x4004, 0x06, 0, 0}, {0x4713, 0x02, 0, 0},
{0x4407, 0x0c, 0, 0}, {0x460b, 0x35, 0, 0}, {0x460c, 0x22, 0, 0},
{0x3824, 0x01, 0, 0}, {0x5001, 0x83, 0, 70}, {0x3008, 0x02, 0, 0},
{0x4202, 0x00, 0, 0}, /* stream on the sensor */
};

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

oferausterlitz — Tue, 29 May 2018 12:18:07 GMT

Hi Tom,

With new 2592x1944 settings and GST_V4L2_MIN_BUFFERS=6 the capture at maximum resolution works at 15fps. All other resolutions work at 30fps.

Thanks a lot for your help.

Felix.

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

Bio_TICFSL — Wed, 30 May 2018 13:31:07 GMT

Hi Felix,

Did it work for you as well?

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

felixradensky — Wed, 30 May 2018 14:41:47 GMT

Hi,

Yes it works for me. It was me posting as Ofer Austerlitz.

Thanks.

Felix.

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

ryandeville2 — Fri, 08 Jun 2018 20:58:01 GMT

Hello xiaoli.zhang,

How vital are the tuning patches to the eMMC for streaming to disk (or rather NVM)? We can get 30fps 1080p through two cameras to /tmp, but not to SD/eMMC.

Ryan

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

chunfengren — Thu, 14 Jun 2018 23:47:08 GMT

I meet a similiar problem:

We have purchased NXP MIPI-CSI Ov5640 camera,
and we do camera test on NXP i.MX8M EVK, The camera output was sent to HDMI display.
when we preview and take video on ov5640, and take video of moving objects, video is very blur.

from kernel logs to see , there is many RX fifo overflow, it make some captured Frames by camera lost.

[ 867.956960] mx6s-csi 30a90000.csi1_bridge: mx6s_csi_irq_handler Rx fifo overf
low
[ 870.956451] mx6s-csi 30a90000.csi1_bridge: mx6s_csi_irq_handler Rx fifo overf
low
[ 874.273032] mx6s-csi 30a90000.csi1_bridge: mx6s_csi_irq_handler Rx fifo overf
low

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

haidong_zheng — Fri, 15 Jun 2018 13:28:25 GMT

"rx fifo over flow" is another topic.

If you can, please carefully apply another two packages patches.

One package(tuning.7z) include ATF, u-Boot, Kernel patches for DDR.

Another package(Camera.tar.bz2) include CSI driver patches.

Two packages must be applied on GA release for this problem.

If you still meet the "rx fifo over flow" after those packages merged, please tell us how to repeat it and show us the log info.

Thanks!

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

chunfengren — Tue, 10 Jul 2018 05:48:32 GMT

HI, Tom zheng

thanks for your patch you provide, it work very well now.

if use 640*480 resolution to take video of moving object, recorded video is very clear and the video quality is good.

if use 1280*720(or 1920*1024) to take video of moving object, video is very blur and the video quality isn't good.

is it be affected by no vpu in i.MX8M EVK?

How do I solve it?

thanks a lot

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

haidong_zheng — Wed, 11 Jul 2018 01:50:35 GMT

Yes, i.MX8M does not have HW video encode. so, video encode is done by SW. If high FPS is needed in high resolution. Some encode simply and optimize are needed done by customer.

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

chunfengren — Thu, 12 Jul 2018 02:06:07 GMT

HI,Tom zheng

Thanks for your reply.

Do you have similar image encode code patch to support high resolution recording?

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

haidong_zheng — Thu, 12 Jul 2018 02:50:24 GMT

no ,thanks!

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

james_kim — Tue, 25 Feb 2020 15:11:29 GMT

Thanks Tom,

It really helped me.

James

Re: i.MX8M EVK MIPI CSI Camera Frame Rate

jobusch — Thu, 09 Apr 2020 11:00:57 GMT

Hello Tom,

thank you for providing these patches. We have a similar problem. We get FIFO_RX_overflow error too. But only when we have high cpu usage at the same time. So thats why your "tuning" patches seem perfect. But they do not fit my version. I am running 4.14.78. On which GA version are the patches made for?