Recommended MIPI-CSI camera for i.MX93

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Recommended MIPI-CSI camera for i.MX93

2,485 Views
csenatore
Contributor II

I would be interested to know if NXP has any recommendations for MIPI-CSI cameras for the iMX93 platform. As I see it in the BSP examples for the iMX93, NXP only uses a parallel camera interface so far.

Is there already a sensor driver in Linux for a specific camera that is compatible with the ISI and MIPI-CSI2 drivers?

Can NXP list a certain number of CSI cameras that can be easily integrated on an iMX93 platform?

 

 

Labels (2)
Tags (1)
0 Kudos
Reply
5 Replies

2,469 Views
Chavira
NXP TechSupport
NXP TechSupport

Hi @csenatore!

 

I our iMX93 EVK board we have already set up the AP1302 camera.

 

 

0 Kudos
Reply

2,312 Views
csenatore
Contributor II

Hi @Chavira ,

 

thx for your answer :).

We bought the RPI-CAM-KIT from NXP and are currently trying to get the camera up and running.

I am using the documentation ‘UM11933: RPi-CAM-MIPI Board User Manual’ as a guide.

Our Linux kernel is based on the lf-6.6.36_2.1.0 release version for testing, but I am currently trying to use the new lf.6.1.55_2.2.2 release as well.


I have made these changes in the Devicetree, which I have taken from the UserGuide.

 

/ {
	reg_dvdd_sel: regulator-dvdd_sel {
		compatible = "regulator-fixed";
		regulator-name = "DVDD_SEL";
		gpio = <&adp5585gpio_isp 0 GPIO_ACTIVE_HIGH>;
		enable-active-high;
		startup-delay-us = <2000>;
	};

	reg_dvdd_1v2: regulator-dvdd {
		compatible = "regulator-fixed";
		regulator-name = "DVDD_1V2";
		gpio = <&adp5585gpio_isp 6 GPIO_ACTIVE_HIGH>;
		regulator-min-microvolt = <1200000>;
		regulator-max-microvolt = <1200000>;
		enable-active-high;
		vin-supply = <&reg_dvdd_sel>;
	};

	reg_vdd_3v3: regulator-vdd {
		compatible = "regulator-fixed";
		regulator-name = "VDD_3V3";
		gpio = <&adp5585gpio_isp 5 GPIO_ACTIVE_HIGH>;
		regulator-min-microvolt = <3300000>;
		regulator-max-microvolt = <3300000>;
		startup-delay-us = <4000>;
		enable-active-high;
	};

	reg_vddio_1v8: regulator-vddo {
		compatible = "regulator-fixed";
		regulator-name = "VDDIO_1V8";
		gpio = <&adp5585gpio_isp 9 GPIO_ACTIVE_HIGH>;
		regulator-min-microvolt = <1800000>;
		regulator-max-microvolt = <1800000>;
		startup-delay-us = <4000>;
		enable-active-high;
		vin-supply = <&reg_vdd_3v3>;
	};

	reg_vaa_sel: regulator-vaa_sel {
		compatible = "regulator-fixed";
		regulator-name = "VAA_SEL";
		gpio = <&adp5585gpio_isp 1 GPIO_ACTIVE_HIGH>;
		enable-active-high;
	};

	reg_avdd_2v8: regulator-avdd {
		compatible = "regulator-fixed";
		regulator-name = "AVDD_2V8";
		gpio = <&adp5585gpio_isp 7 GPIO_ACTIVE_HIGH>;
		regulator-min-microvolt = <2800000>;
		regulator-max-microvolt = <2800000>;
		enable-active-high;
		vin-supply = <&reg_vaa_sel>;
	};
};


&lpi2c2 {
	clock-frequency = <400000>;
	adp5585_isp: mfd-isp@34 {
		compatible = "adi,adp5585";
		reg = <0x34>;
		status = "okay";

		adp5585gpio_isp: gpio-isp {
			compatible = "adp5585-gpio";
			gpio-controller;
			#gpio-cells = <2>;
		};

		adp5585pwm_isp: pwm-isp {
			compatible = "adp5585-pwm";
			#pwm-cells = <3>;
		};
	};

	ap1302: ap1302_mipi@3c {
		compatible = "onsemi,ap1302";
		reg = <0x3c>;
		reset-gpios = <&expander 14 GPIO_ACTIVE_LOW>;
		isp_en-gpios  = <&adp5585gpio_isp 2 GPIO_ACTIVE_HIGH>;
		DVDD-supply   = <&reg_dvdd_1v2>;
		VDDIO-supply  = <&reg_vddio_1v8>;
		AVDD-supply   = <&reg_avdd_2v8>;
		status = "okay";

		port {
			ar1302_mipi_ep: endpoint {
				remote-endpoint = <&mipi_csi_ep>;
			};
		};
	};
};

&cameradev {
	status = "okay";
};

&isi_0 {
	status = "okay";

	cap_device {
		status = "okay";
	};
};

&mipi_csi {
	status = "okay";

	port {
		mipi_csi_ep: endpoint {
			remote-endpoint = <&ar1302_mipi_ep>;
			data-lanes = <2>;
			cfg-clk-range = <28>;
			hs-clk-range = <0x2b>;
			bus-type = <4>;
		};
	};
};

 

The firmware for the camera was also stored under /lib/firmware/imx/camera/ap1302.fw according to the User Guide

In Linux I can see the following messages with dmesg:

 

dmesg | grep "ap1302_mipi 1-003c:"
[    2.628511] ap1302_mipi 1-003c: AP1302 Chip ID is 0x265
[    2.632533] ap1302_mipi 1-003c: Direct firmware load for imx/camera/ap1302.fw failed with error -2
[    2.636416] ap1302_mipi 1-003c: AP1302 is found
[    2.640192] ap1302_mipi 1-003c: Falling back to sysfs fallback for: imx/camera/ap1302.fw
[    8.900490] ap1302_mipi 1-003c: Load firmware successfully.
dmesg | grep mx8-img-md
[    7.283909] mx8-img-md: Registered mxc_isi.0.capture as /dev/video0
[    7.284027] mx8-img-md: Registered sensor subdevice: ap1302_mipi 1-003c (1)
[    7.284035] mx8-img-md: created link [mxc_isi.0] => [mxc_isi.0.capture]
[    7.284041] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0]
[    7.284048] mx8-img-md: created link [ap1302_mipi 1-003c] => [mxc-mipi-csi2.0]

 

It looks to me like the drivers for the CSI pipeline are all successful. It also creates /dev/video0 and /dev/media0.

The following v4l2-ctl commands also work:

 

v4l2-ctl --list-devices
FSL Capture Media Device (platform:42800000.bus:camera):
        /dev/media0

mxc-isi-cap_v1 (platform:4ae40000.isi:cap_devic):
        /dev/video0
v4l2-ctl --list-formats
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'RGBP' (16-bit RGB 5-6-5)
        [1]: 'RGB3' (24-bit RGB 8-8-8)
        [2]: 'BGR3' (24-bit BGR 8-8-8)
        [3]: 'YUYV' (YUYV 4:2:2)
        [4]: 'YUV4' (32-bit A/XYUV 8-8-8-8)
        [5]: 'NV12' (Y/UV 4:2:0)
        [6]: 'NM12' (Y/UV 4:2:0 (N-C))
        [7]: 'YM24' (Planar YUV 4:4:4 (N-C))
        [8]: 'XR24' (32-bit BGRX 8-8-8-8)
        [9]: 'AR24' (32-bit BGRA 8-8-8-8)
v4l2-ctl -d0 --list-framesizes YUYV
ioctl: VIDIOC_ENUM_FRAMESIZES
        Size: Discrete 1920x1080
        Size: Discrete 1280x800
        Size: Discrete 1280x720
        Size: Discrete 640x480

 

BUT reading data from the camera does not work.

 

v4l2-ctl -d0 --set-fmt-video=width=1280,height=800,pixelformat=YUYV --stream-mmap=4 --stream-count=10 --stream-to=test.yuv
                VIDIOC_STREAMON returned -1 (Invalid argument)

 

The output of dmesg is here:

 

[ 7007.161929] mxc_isi.0: set remote fmt fail!

 

The examples with gst-launch do not work either. The error message in dmesg is identical.

 

This is actually not the first time I have seen this error message, and I had already mentioned it in the context of another camera, see:
https://community.nxp.com/t5/i-MX-Processors/Adding-support-for-Camera-sensor-IMX708/m-p/2030536/hig...

other users have also encountered the problem with the imx93, see:
https://community.nxp.com/t5/i-MX-Processors/Camera-Driver-Guide-for-i-MX-93-EVK/m-p/2038532

So I have the following questions:

- Does the NXP BSP currently support the RPI-CAM for the iMX93?

- Can NXP validate the camera with the latest BSP version for the imx93 evalkit to ensure the drivers work?

 

I have also noticed that the DSI driver assigned in linux-imx for the imx93 has not received any commits.

The following compatible is referenced in imx93.dtsi:
https://github.com/nxp-imx/linux-imx/blob/lf-6.6.52-2.2.0/arch/arm64/boot/dts/freescale/imx93.dtsi#L...

Accordingly, this is the driver used:
https://github.com/nxp-imx/linux-imx/blob/lf-6.6.52-2.2.0/drivers/staging/media/imx/dwc-mipi-csi2.c

- Will there be a further development of the driver for the iMX93 so that the staging area can be left?

I also came across a second dwc driver in Linux, which can apparently also be used for the iMX93, but is not used?
https://github.com/nxp-imx/linux-imx/blob/lf-6.6.52-2.2.0/drivers/media/platform/nxp/dwc-mipi-csi2.c

- Will the DWC driver currently in use on the iMX95 also be in use on the iMX93 in the future?

-How can NXP support me with the RPI-CAM and the BSP code to get the camera running?

0 Kudos
Reply

2,301 Views
Chavira
NXP TechSupport
NXP TechSupport

Hi @csenatore!

To use the camera is not needed any changes in device tree, the default device tree is working with the camera.

 

My outputs are the next:

root@imx93evk:~# uname -r
6.6.52-lts-next-ge0f9e2afd4cf
root@imx93evk:~# media-ctl -p
Media controller API version 6.6.52

Media device information
------------------------
driver          mxc-md
model           FSL Capture Media Device
serial
bus info        platform:42800000.bus:camera
hw revision     0x0
driver version  6.6.52

Device topology
- entity 1: mxc_isi.0 (16 pads, 2 links, 0 routes)
            type V4L2 subdev subtype Unknown flags 0
        pad0: Sink
                <- "mxc-mipi-csi2.0":4 [ENABLED]
        pad1: Sink
        pad2: Sink
        pad3: Sink
        pad4: Sink
        pad5: Sink
        pad6: Sink
        pad7: Sink
        pad8: Sink
        pad9: Sink
        pad10: Sink
        pad11: Sink
        pad12: Source
                -> "mxc_isi.0.capture":0 [ENABLED]
        pad13: Source
        pad14: Source
        pad15: Sink

- entity 18: mxc_isi.0.capture (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video0
        pad0: Sink
                <- "mxc_isi.0":12 [ENABLED]

- entity 22: mxc-mipi-csi2.0 (8 pads, 2 links)
             type Node subtype V4L flags 0
             device node name /dev/v4l-subdev0
        pad0: Sink
                <- "ap1302_mipi 2-003c":0 [ENABLED,IMMUTABLE]
        pad1: Sink
        pad2: Sink
        pad3: Sink
        pad4: Source
                -> "mxc_isi.0":0 [ENABLED]
        pad5: Source
        pad6: Source
        pad7: Source

- entity 31: ap1302_mipi 2-003c (1 pad, 1 link, 0 routes)
             type V4L2 subdev subtype Unknown flags 0
        pad0: Source
                -> "mxc-mipi-csi2.0":0 [ENABLED,IMMUTABLE]

root@imx93evk:~# v4l2-ctl --list-devices
FSL Capture Media Device (platform:42800000.bus:camera):
        /dev/media0

mxc-isi-cap_v1 (platform:4ae40000.isi:cap_devic):
        /dev/video0

root@imx93evk:~# v4l2-ctl -d0 --list-formats
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'RGBP' (16-bit RGB 5-6-5)
        [1]: 'RGB3' (24-bit RGB 8-8-8)
        [2]: 'BGR3' (24-bit BGR 8-8-8)
        [3]: 'YUYV' (YUYV 4:2:2)
        [4]: 'YUV4' (32-bit A/XYUV 8-8-8-8)
        [5]: 'NV12' (Y/UV 4:2:0)
        [6]: 'NM12' (Y/UV 4:2:0 (N-C))
        [7]: 'YM24' (Planar YUV 4:4:4 (N-C))
        [8]: 'XR24' (32-bit BGRX 8-8-8-8)
        [9]: 'AR24' (32-bit BGRA 8-8-8-8)
root@imx93evk:~# v4l2-ctl -d0 --list-framesizes YUYV
ioctl: VIDIOC_ENUM_FRAMESIZES
        Size: Discrete 1920x1080
        Size: Discrete 1280x800
        Size: Discrete 1280x720
        Size: Discrete 640x480
root@imx93evk:~# v4l2-ctl -d0 --set-fmt-video=width=1280,height=800,pixelformat=YUYV --stream-mmap=4 --stream-count=10 --stream-to=test.yuv
<<<<<<<<<<
root@imx93evk:~# gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-raw,format=YUY2,width=1280,height=800" ! queue ! waylandsink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)800, framerate=(fraction)120/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)800, framerate=(fraction)120/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)800, framerate=(fraction)120/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)800, framerate=(fraction)120/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)800, framerate=(fraction)120/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstWaylandSink:waylandsink0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)800, framerate=(fraction)120/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
Redistribute latency...
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:01:13.582441035
Setting pipeline to NULL ...
Total showed frames (4609), playing for (0:01:13.582263743), fps (62.637).
Freeing pipeline ...
root@imx93evk:~#

 

Chavira_1-1739284555050.png

 

Chavira_2-1739284575525.png

 

 

 

0 Kudos
Reply

2,277 Views
csenatore
Contributor II

Hey @Chavira ,

Thank you very much for your quick and very detailed reply.

Your answer helped me a lot.

 

To use the camera is not needed any changes in device tree, the default device tree is working with the camera.

I try to run the NXP RPI-CAM on our custom board.
The entries I mentioned above were the additions for our DTS.
Perhaps I should have mentioned that directly :D.

For further tests on our baseboard, I built the latest version of linux-imx (lf-6.6.52-2.2.0) again, but the problem is still the same.
I also use imx_v8_defconfig to build the kernel.

With a small hack in the driver imx8_isi_cap.c I was able to start the camera successfully.
I removed the following line:

https://github.com/nxp-imx/linux-imx/blob/lf-6.6.52-2.2.0/drivers/staging/media/imx/imx8-isi-cap.c#L...
Here, I ignore the return value in the IOCTRL VIDIOC_STREAMON and let the function continue to run.

Is it possible that the line ‘src_fmt.format.code = MEDIA_BUS_FMT_UYVY8_1X16;’ is incompatible for the dwc driver?

The format is not defined:
https://github.com/nxp-imx/linux-imx/blob/lf-6.6.52-2.2.0/drivers/staging/media/imx/dwc-mipi-csi2.c#...

I know that my change in imx8-isi-cap.c is not exactly ideal, but it is functional.
Can NXP tell me whether the drivers will be developed further?

Will imx8-isi-cap.c and dwc-mipi-csi2.c leave the driver/staging area?

0 Kudos
Reply

2,250 Views
Chavira
NXP TechSupport
NXP TechSupport

Hi @csenatore!

It is very difficult for us to debug a custom board by our side since we don't have the hardware with us.

 

By my side I can say that the problem are not the drivers since in the iMX93-evk and FRDM-iMX93 the camera is working without changes.

 

Please contact to your local FAE to get more support for this case.

 

Best Regards!

Chavira

0 Kudos
Reply