I would be interested to know if NXP has any recommendations for MIPI-CSI cameras for the iMX93 platform. As I see it in the BSP examples for the iMX93, NXP only uses a parallel camera interface so far.
Is there already a sensor driver in Linux for a specific camera that is compatible with the ISI and MIPI-CSI2 drivers?
Can NXP list a certain number of CSI cameras that can be easily integrated on an iMX93 platform?
Hi @Chavira ,
thx for your answer :).
We bought the RPI-CAM-KIT from NXP and are currently trying to get the camera up and running.
I am using the documentation ‘UM11933: RPi-CAM-MIPI Board User Manual’ as a guide.
Our Linux kernel is based on the lf-6.6.36_2.1.0 release version for testing, but I am currently trying to use the new lf.6.1.55_2.2.2 release as well.
I have made these changes in the Devicetree, which I have taken from the UserGuide.
/ {
reg_dvdd_sel: regulator-dvdd_sel {
compatible = "regulator-fixed";
regulator-name = "DVDD_SEL";
gpio = <&adp5585gpio_isp 0 GPIO_ACTIVE_HIGH>;
enable-active-high;
startup-delay-us = <2000>;
};
reg_dvdd_1v2: regulator-dvdd {
compatible = "regulator-fixed";
regulator-name = "DVDD_1V2";
gpio = <&adp5585gpio_isp 6 GPIO_ACTIVE_HIGH>;
regulator-min-microvolt = <1200000>;
regulator-max-microvolt = <1200000>;
enable-active-high;
vin-supply = <®_dvdd_sel>;
};
reg_vdd_3v3: regulator-vdd {
compatible = "regulator-fixed";
regulator-name = "VDD_3V3";
gpio = <&adp5585gpio_isp 5 GPIO_ACTIVE_HIGH>;
regulator-min-microvolt = <3300000>;
regulator-max-microvolt = <3300000>;
startup-delay-us = <4000>;
enable-active-high;
};
reg_vddio_1v8: regulator-vddo {
compatible = "regulator-fixed";
regulator-name = "VDDIO_1V8";
gpio = <&adp5585gpio_isp 9 GPIO_ACTIVE_HIGH>;
regulator-min-microvolt = <1800000>;
regulator-max-microvolt = <1800000>;
startup-delay-us = <4000>;
enable-active-high;
vin-supply = <®_vdd_3v3>;
};
reg_vaa_sel: regulator-vaa_sel {
compatible = "regulator-fixed";
regulator-name = "VAA_SEL";
gpio = <&adp5585gpio_isp 1 GPIO_ACTIVE_HIGH>;
enable-active-high;
};
reg_avdd_2v8: regulator-avdd {
compatible = "regulator-fixed";
regulator-name = "AVDD_2V8";
gpio = <&adp5585gpio_isp 7 GPIO_ACTIVE_HIGH>;
regulator-min-microvolt = <2800000>;
regulator-max-microvolt = <2800000>;
enable-active-high;
vin-supply = <®_vaa_sel>;
};
};
&lpi2c2 {
clock-frequency = <400000>;
adp5585_isp: mfd-isp@34 {
compatible = "adi,adp5585";
reg = <0x34>;
status = "okay";
adp5585gpio_isp: gpio-isp {
compatible = "adp5585-gpio";
gpio-controller;
#gpio-cells = <2>;
};
adp5585pwm_isp: pwm-isp {
compatible = "adp5585-pwm";
#pwm-cells = <3>;
};
};
ap1302: ap1302_mipi@3c {
compatible = "onsemi,ap1302";
reg = <0x3c>;
reset-gpios = <&expander 14 GPIO_ACTIVE_LOW>;
isp_en-gpios = <&adp5585gpio_isp 2 GPIO_ACTIVE_HIGH>;
DVDD-supply = <®_dvdd_1v2>;
VDDIO-supply = <®_vddio_1v8>;
AVDD-supply = <®_avdd_2v8>;
status = "okay";
port {
ar1302_mipi_ep: endpoint {
remote-endpoint = <&mipi_csi_ep>;
};
};
};
};
&cameradev {
status = "okay";
};
&isi_0 {
status = "okay";
cap_device {
status = "okay";
};
};
&mipi_csi {
status = "okay";
port {
mipi_csi_ep: endpoint {
remote-endpoint = <&ar1302_mipi_ep>;
data-lanes = <2>;
cfg-clk-range = <28>;
hs-clk-range = <0x2b>;
bus-type = <4>;
};
};
};
The firmware for the camera was also stored under /lib/firmware/imx/camera/ap1302.fw according to the User Guide
In Linux I can see the following messages with dmesg:
dmesg | grep "ap1302_mipi 1-003c:"
[ 2.628511] ap1302_mipi 1-003c: AP1302 Chip ID is 0x265
[ 2.632533] ap1302_mipi 1-003c: Direct firmware load for imx/camera/ap1302.fw failed with error -2
[ 2.636416] ap1302_mipi 1-003c: AP1302 is found
[ 2.640192] ap1302_mipi 1-003c: Falling back to sysfs fallback for: imx/camera/ap1302.fw
[ 8.900490] ap1302_mipi 1-003c: Load firmware successfully.dmesg | grep mx8-img-md
[ 7.283909] mx8-img-md: Registered mxc_isi.0.capture as /dev/video0
[ 7.284027] mx8-img-md: Registered sensor subdevice: ap1302_mipi 1-003c (1)
[ 7.284035] mx8-img-md: created link [mxc_isi.0] => [mxc_isi.0.capture]
[ 7.284041] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0]
[ 7.284048] mx8-img-md: created link [ap1302_mipi 1-003c] => [mxc-mipi-csi2.0]
It looks to me like the drivers for the CSI pipeline are all successful. It also creates /dev/video0 and /dev/media0.
The following v4l2-ctl commands also work:
v4l2-ctl --list-devices
FSL Capture Media Device (platform:42800000.bus:camera):
/dev/media0
mxc-isi-cap_v1 (platform:4ae40000.isi:cap_devic):
/dev/video0v4l2-ctl --list-formats
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture Multiplanar
[0]: 'RGBP' (16-bit RGB 5-6-5)
[1]: 'RGB3' (24-bit RGB 8-8-8)
[2]: 'BGR3' (24-bit BGR 8-8-8)
[3]: 'YUYV' (YUYV 4:2:2)
[4]: 'YUV4' (32-bit A/XYUV 8-8-8-8)
[5]: 'NV12' (Y/UV 4:2:0)
[6]: 'NM12' (Y/UV 4:2:0 (N-C))
[7]: 'YM24' (Planar YUV 4:4:4 (N-C))
[8]: 'XR24' (32-bit BGRX 8-8-8-8)
[9]: 'AR24' (32-bit BGRA 8-8-8-8)v4l2-ctl -d0 --list-framesizes YUYV
ioctl: VIDIOC_ENUM_FRAMESIZES
Size: Discrete 1920x1080
Size: Discrete 1280x800
Size: Discrete 1280x720
Size: Discrete 640x480
BUT reading data from the camera does not work.
v4l2-ctl -d0 --set-fmt-video=width=1280,height=800,pixelformat=YUYV --stream-mmap=4 --stream-count=10 --stream-to=test.yuv
VIDIOC_STREAMON returned -1 (Invalid argument)
The output of dmesg is here:
[ 7007.161929] mxc_isi.0: set remote fmt fail!
The examples with gst-launch do not work either. The error message in dmesg is identical.
This is actually not the first time I have seen this error message, and I had already mentioned it in the context of another camera, see:
https://community.nxp.com/t5/i-MX-Processors/Adding-support-for-Camera-sensor-IMX708/m-p/2030536/hig...
other users have also encountered the problem with the imx93, see:
https://community.nxp.com/t5/i-MX-Processors/Camera-Driver-Guide-for-i-MX-93-EVK/m-p/2038532
So I have the following questions:
- Does the NXP BSP currently support the RPI-CAM for the iMX93?
- Can NXP validate the camera with the latest BSP version for the imx93 evalkit to ensure the drivers work?
I have also noticed that the DSI driver assigned in linux-imx for the imx93 has not received any commits.
The following compatible is referenced in imx93.dtsi:
https://github.com/nxp-imx/linux-imx/blob/lf-6.6.52-2.2.0/arch/arm64/boot/dts/freescale/imx93.dtsi#L...
Accordingly, this is the driver used:
https://github.com/nxp-imx/linux-imx/blob/lf-6.6.52-2.2.0/drivers/staging/media/imx/dwc-mipi-csi2.c
- Will there be a further development of the driver for the iMX93 so that the staging area can be left?
I also came across a second dwc driver in Linux, which can apparently also be used for the iMX93, but is not used?
https://github.com/nxp-imx/linux-imx/blob/lf-6.6.52-2.2.0/drivers/media/platform/nxp/dwc-mipi-csi2.c
- Will the DWC driver currently in use on the iMX95 also be in use on the iMX93 in the future?
-How can NXP support me with the RPI-CAM and the BSP code to get the camera running?
Hi @csenatore!
To use the camera is not needed any changes in device tree, the default device tree is working with the camera.
My outputs are the next:
root@imx93evk:~# uname -r
6.6.52-lts-next-ge0f9e2afd4cf
root@imx93evk:~# media-ctl -p
Media controller API version 6.6.52
Media device information
------------------------
driver mxc-md
model FSL Capture Media Device
serial
bus info platform:42800000.bus:camera
hw revision 0x0
driver version 6.6.52
Device topology
- entity 1: mxc_isi.0 (16 pads, 2 links, 0 routes)
type V4L2 subdev subtype Unknown flags 0
pad0: Sink
<- "mxc-mipi-csi2.0":4 [ENABLED]
pad1: Sink
pad2: Sink
pad3: Sink
pad4: Sink
pad5: Sink
pad6: Sink
pad7: Sink
pad8: Sink
pad9: Sink
pad10: Sink
pad11: Sink
pad12: Source
-> "mxc_isi.0.capture":0 [ENABLED]
pad13: Source
pad14: Source
pad15: Sink
- entity 18: mxc_isi.0.capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "mxc_isi.0":12 [ENABLED]
- entity 22: mxc-mipi-csi2.0 (8 pads, 2 links)
type Node subtype V4L flags 0
device node name /dev/v4l-subdev0
pad0: Sink
<- "ap1302_mipi 2-003c":0 [ENABLED,IMMUTABLE]
pad1: Sink
pad2: Sink
pad3: Sink
pad4: Source
-> "mxc_isi.0":0 [ENABLED]
pad5: Source
pad6: Source
pad7: Source
- entity 31: ap1302_mipi 2-003c (1 pad, 1 link, 0 routes)
type V4L2 subdev subtype Unknown flags 0
pad0: Source
-> "mxc-mipi-csi2.0":0 [ENABLED,IMMUTABLE]
root@imx93evk:~# v4l2-ctl --list-devices
FSL Capture Media Device (platform:42800000.bus:camera):
/dev/media0
mxc-isi-cap_v1 (platform:4ae40000.isi:cap_devic):
/dev/video0
root@imx93evk:~# v4l2-ctl -d0 --list-formats
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture Multiplanar
[0]: 'RGBP' (16-bit RGB 5-6-5)
[1]: 'RGB3' (24-bit RGB 8-8-8)
[2]: 'BGR3' (24-bit BGR 8-8-8)
[3]: 'YUYV' (YUYV 4:2:2)
[4]: 'YUV4' (32-bit A/XYUV 8-8-8-8)
[5]: 'NV12' (Y/UV 4:2:0)
[6]: 'NM12' (Y/UV 4:2:0 (N-C))
[7]: 'YM24' (Planar YUV 4:4:4 (N-C))
[8]: 'XR24' (32-bit BGRX 8-8-8-8)
[9]: 'AR24' (32-bit BGRA 8-8-8-8)
root@imx93evk:~# v4l2-ctl -d0 --list-framesizes YUYV
ioctl: VIDIOC_ENUM_FRAMESIZES
Size: Discrete 1920x1080
Size: Discrete 1280x800
Size: Discrete 1280x720
Size: Discrete 640x480
root@imx93evk:~# v4l2-ctl -d0 --set-fmt-video=width=1280,height=800,pixelformat=YUYV --stream-mmap=4 --stream-count=10 --stream-to=test.yuv
<<<<<<<<<<
root@imx93evk:~# gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-raw,format=YUY2,width=1280,height=800" ! queue ! waylandsink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)800, framerate=(fraction)120/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)800, framerate=(fraction)120/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)800, framerate=(fraction)120/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)800, framerate=(fraction)120/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)800, framerate=(fraction)120/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstWaylandSink:waylandsink0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)800, framerate=(fraction)120/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
Redistribute latency...
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:01:13.582441035
Setting pipeline to NULL ...
Total showed frames (4609), playing for (0:01:13.582263743), fps (62.637).
Freeing pipeline ...
root@imx93evk:~#
Hey @Chavira ,
Thank you very much for your quick and very detailed reply.
Your answer helped me a lot.
To use the camera is not needed any changes in device tree, the default device tree is working with the camera.
I try to run the NXP RPI-CAM on our custom board.
The entries I mentioned above were the additions for our DTS.
Perhaps I should have mentioned that directly :D.
For further tests on our baseboard, I built the latest version of linux-imx (lf-6.6.52-2.2.0) again, but the problem is still the same.
I also use imx_v8_defconfig to build the kernel.
With a small hack in the driver imx8_isi_cap.c I was able to start the camera successfully.
I removed the following line:
https://github.com/nxp-imx/linux-imx/blob/lf-6.6.52-2.2.0/drivers/staging/media/imx/imx8-isi-cap.c#L...
Here, I ignore the return value in the IOCTRL VIDIOC_STREAMON and let the function continue to run.
Is it possible that the line ‘src_fmt.format.code = MEDIA_BUS_FMT_UYVY8_1X16;’ is incompatible for the dwc driver?
The format is not defined:
https://github.com/nxp-imx/linux-imx/blob/lf-6.6.52-2.2.0/drivers/staging/media/imx/dwc-mipi-csi2.c#...
I know that my change in imx8-isi-cap.c is not exactly ideal, but it is functional.
Can NXP tell me whether the drivers will be developed further?
Will imx8-isi-cap.c and dwc-mipi-csi2.c leave the driver/staging area?
Hi @csenatore!
It is very difficult for us to debug a custom board by our side since we don't have the hardware with us.
By my side I can say that the problem are not the drivers since in the iMX93-evk and FRDM-iMX93 the camera is working without changes.
Please contact to your local FAE to get more support for this case.
Best Regards!
Chavira