i.mx8M Mini OV5640 fps limitation

cancel
Showing results for 
Search instead for 
Did you mean: 

i.mx8M Mini OV5640 fps limitation

Jump to solution
2,835 Views
raptor8
Contributor II

Hello everyone,

I bought the imx8mmevk with the i.MX8M Mini SoC with the MINISASTOCSI accessory, flashed Linux (L4.14.78_1.0.0_ga_images_MX8MMEVK) and managed to stream from the OV5640 with gstreamer successfully.

But I was expecting to be able to use this sensor at 60fps as the datasheet of the sensor specify and the video encoders VP8/H264 are capable of outputting at least 720p60.

pastedImage_5.png

I have done a quick search for the cause of this but I see that's it's using 2 mipi_csi lanes

root@imx8mmevk:~# dmesg | grep "mipi"
[    1.257708] adv7511 1-003d: Probe failed. Remote port '/mipi_dsi@32E10000' disabled
[    1.549094] mxc_mipi-csi 32e30000.mipi_csi: 32e30000.mipi_csi supply mipi-phy not found, using dummy regulator
[    1.559458] mxc_mipi-csi 32e30000.mipi_csi: mipi csi v4l2 device registered
[    1.566436] CSI: Registered sensor subdevice: mxc_mipi-csi.0
[    1.572121] mxc_mipi-csi 32e30000.mipi_csi: lanes: 2, hs_settle: 13, clk_settle: 2, wclk: 1, freq: 333000000
[    1.582387] ov5640_mipi 2-003c: No sensor reset pin available
[    1.588182] ov5640_mipi 2-003c: 2-003c supply DOVDD not found, using dummy regulator
[    1.595981] ov5640_mipi 2-003c: 2-003c supply DVDD not found, using dummy regulator
[    1.603677] ov5640_mipi 2-003c: 2-003c supply AVDD not found, using dummy regulator
[    2.786869] mxc_mipi-csi 32e30000.mipi_csi: Registered sensor subdevice: ov5640_mipi 2-003c
[    2.797202] ov5640_mipi 2-003c: Camera is found

And the capabilities are tied to 30 fps max with V4L2

root@imx8mmevk:~# v4l2-ctl -D -d /dev/video0 --list-formats-ext
Driver Info (not using libv4l2):
        Driver name   : mx6s-csi
        Card type     : i.MX6S_CSI
        Bus info      : platform:32e20000.csi1_bridge
        Driver version: 4.14.78
        Capabilities  : 0x84200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04200001
                Video Capture

                Streaming
                Extended Pix Format
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'YUYV'
        Name        : YUYV 4:2:2
                Size: Discrete 640x480
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 720x480
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2592x1944
                        Interval: Discrete 0.067s (15.000 fps)
                Size: Discrete 0x0

Now I would like to know what is possible to do to get to 60fps with this sensor and from where the limitation is coming.

Is it from some registers in the driver? The clock speed used? Do I need to recompile something with other parameters?

I was also planning on using other sensors in the future, on the MIPI I/F or any other serial or parallel I/F. Are there others drivers included in the default linux image? How is it difficult to implement a new sensor?

I see many sensors listed on the official linux github that can interest me.

linux/drivers/media/i2c at master · torvalds/linux · GitHub 

Thank you for your time.

1 Solution
1,235 Views
raptor8
Contributor II

I'm updating my question now as I'm rebuilding the Linux kernel now...
The driver is locked to 30 fps in multiple places like here:
https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/i2c/ov5640.c?h=imx_4.14.78_1...
https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_...

And many other references to the framerate here

./drivers/media/i2c/ov5640.c
./drivers/media/platform/mxc/capture/ov5640_v2.c
./drivers/media/platform/mxc/capture/ov5640.c
./drivers/media/platform/mxc/capture/ov5640_mipi.c
./drivers/media/platform/mxc/capture/ov5640_mipi_v2.c
./drivers/media/platform/imx8/ov5640_mipi_v3.c
./drivers/media/platform/imx8/ov5640_v3.c

And the second part of my question was adding a new camera sensor, I suggest taking a look at the i.MX_BSP_Porting_Guide at chapter 13.2:

"13.2 Adding support for a new CMOS camera sensor"

Which explains how to add a driver to the make menuconfig to compile it.

View solution in original post

5 Replies
1,235 Views
andreasgustafss
Contributor III

The ov5640 have a driver upstream that supports all camera modes, including 720p 60fps.

I would try to get the upstream driver to work on the kernel you're using.

It can be found here:

linux/ov5640.c at master · torvalds/linux · GitHub 

Both drivers utilize the v4l2-subdev system so a drop in replacement should be possible.

Good luck!

0 Kudos
1,236 Views
raptor8
Contributor II

I'm updating my question now as I'm rebuilding the Linux kernel now...
The driver is locked to 30 fps in multiple places like here:
https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/i2c/ov5640.c?h=imx_4.14.78_1...
https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/media/platform/mxc/capture/ov5640_...

And many other references to the framerate here

./drivers/media/i2c/ov5640.c
./drivers/media/platform/mxc/capture/ov5640_v2.c
./drivers/media/platform/mxc/capture/ov5640.c
./drivers/media/platform/mxc/capture/ov5640_mipi.c
./drivers/media/platform/mxc/capture/ov5640_mipi_v2.c
./drivers/media/platform/imx8/ov5640_mipi_v3.c
./drivers/media/platform/imx8/ov5640_v3.c

And the second part of my question was adding a new camera sensor, I suggest taking a look at the i.MX_BSP_Porting_Guide at chapter 13.2:

"13.2 Adding support for a new CMOS camera sensor"

Which explains how to add a driver to the make menuconfig to compile it.

View solution in original post

1,235 Views
michael_glembot
Contributor III

Could you please provice the i.MX_BSP_Porting_Guide? I only found https://www.nxp.com/docs/en/user-guide/i.MX_BSP_Porting_Guide_Linux.pdf , which does not support imx8mm. The registers are different.

0 Kudos
1,235 Views
raptor8
Contributor II

You should be able to download it from here: https://www.nxp.com/webapp/Download?colCode=imx-yocto-L4.14.98_2.0.0_ga 
It's available in the i.MX Software | NXP  under Documentation > Linux
(also if you can't login onto nxp website like me, try with another browser)

1,235 Views
diegoadrian
NXP Employee
NXP Employee

Hello,

Unfortunately, the maximum fps that the board supports is 30fps. Probably you can make support 60 fps by changing the gstreamer v4l2src plug-in. However, I cannot guarantee anything, since the fps output depends on many things. I apologize if this could be a problem for you.

Best regards,

Diego.