topic Re: IMX8MP MIPI-CSI2 Problems in custom camera in i.MX Processors

IMX8MP MIPI-CSI2 Problems in custom camera

v_parri — Mon, 22 Feb 2021 08:28:26 GMT

Hi, my name is Victor and I'm in a project where we're trying to send image from a custom 4-lane camera to the IMX8MP (in a imx8mp-evk board) using the MIPI-CSI2 interface but I'm stuck trying to make it work.

I want to modify the ov5640 config and drivers but I'm not sure how many things I have to change. I'm ony interested in make it work sending image, the I2C is not needed at this moment and the camera is conected to the MIPI_CSI_0.

1) Is it enough if I modify the device tree configuration? which parameters do I have to modify? This is the configuration that I'm using:

imx8mp.dtsi:

mipi_csi_0: csi@32e40000 {
compatible = "fsl,imx8mp-mipi-csi", "fsl,imx8mn-mipi-csi";
reg = <0x32e40000 0x10000>;
interrupts = <GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>;
clock-frequency = <500000000>;
clocks = <&clk IMX8MP_CLK_MEDIA_CAM1_PIX>,
<&clk IMX8MP_CLK_MEDIA_AXI>,
<&clk IMX8MP_CLK_MEDIA_APB>;
clock-names = "mipi_clk", "disp_axi", "disp_apb";
assigned-clocks = <&clk IMX8MP_CLK_MEDIA_CAM1_PIX>;
assigned-clock-parents = <&clk IMX8MP_SYS_PLL2_1000M>;
assigned-clock-rates = <500000000>;
bus-width = <4>;
csi-gpr = <&mediamix_gasket0>;
gpr = <&mediamix_blk_ctl>;
no-reset-control;
power-domains = <&mipi_phy1_pd>;
status = "disabled";
};

imx8mp-evk.dts:

&mipi_csi_0 {
#address-cells = <1>;
#size-cells = <0>;
status = "okay";

port@0 {
reg = <0>;
mipi_csi0_ep: endpoint {
remote-endpoint = <&ov5640_mipi_0_ep>;
data-lanes = <4>;
};
};
};

2) Do I have to modify the ov5640 driver too? I've located this driver: ov5640_mipi_v2.c but not sure about it

Just modifying the device tree if I try to build a pipe with gstreamer to read from video0 I'm getting this error:

gst-launch-1.0 -v v4l2src ! "video/x-bayer,format=bggr,width=640,height=480,framerate=30/1" ! bayer2rgb ! videoconvert ! autovideosink sync=false
Setting pipeline to PAUSED ...
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device '/dev/video0' is not a capture device.Additional debug info:
../../../git/sys/v4l2/v4l2_calls.c(677): gst_v4l2_open (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Capabilities: 0x4204000
Setting pipeline to NULL ...
Freeing pipeline ...

Also the configuration read from v412-ctl tool is:

v4l2-ctl --list-devices
mxc-isi-m2m (platform:32e00000.isi:m2m_devic):
/dev/video0

v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture Multiplanar

[0]: 'RGBP' (16-bit RGB 5-6-5)
[1]: 'RGB3' (24-bit RGB 8-8-8)
[2]: 'BGR3' (24-bit BGR 8-8-8)
[3]: 'YUYV' (YUYV 4:2:2)
[4]: 'YUV4' (32-bit A/XYUV 8-8-8-8)
[5]: 'NV12' (Y/CbCr 4:2:0)
[6]: 'YM24' (Planar YUV 4:4:4 (N-C))
[7]: 'XR24' (32-bit BGRX 8-8-8-8)
[8]: 'AR24' (32-bit BGRA 8-8-8-8)

Please any help will be appreciated

Re: IMX8MP MIPI-CSI2 Problems in custom camera

joanxie — Fri, 26 Feb 2021 06:08:50 GMT

I just got my imx8mp board, firstly, pls tell me what bsp version you use? let me reproduce this

Re: IMX8MP MIPI-CSI2 Problems in custom camera

v_parri — Sun, 28 Feb 2021 19:50:40 GMT

Hi, the BSP we're curently using is the 5.4.47.

Any help will be appreciated,

Thanks

Re: IMX8MP MIPI-CSI2 Problems in custom camera

joanxie — Thu, 18 Mar 2021 06:58:29 GMT

which dts file do you use? did you try to use imx8mp-evk-basler-ov5640.dtb?

Re: IMX8MP MIPI-CSI2 Problems in custom camera

Matt_S — Fri, 14 May 2021 08:56:00 GMT

Hi Victor, I have a similar issue. I only have video0, which is on the isi device, not csi, v4l-ctl --all says its driver is mxc-isi-m2m, so despite doing pretty much what you have with imx8mp-evk.dts, the csi device just isnt showing, and neither is yours. The csi device is on 0x32e40000, the isi device on 32e00000, which is what you have in --list-devices.

So the question is, how to get the csi device to appear. dmesg shows:
mxc-mipi-csi2-sam 32e40000.csi: lanes 1, hs_settle: 22, clck_settle: 0, wclk: 1, freq: 5000000000 (my dts changes the values for settle)

So it looks as if the csi device loaded ok, it just isnt creating a videox device.

Any ideas? Its a bit of a mystery.

Re: IMX8MP MIPI-CSI2 Problems in custom camera

MicMoba — Mon, 17 May 2021 05:12:31 GMT

Did you find the reason why csi doesn't create a video device? I just have the same issue. csi is loaded but there is no /dev/video0

Thanks

Re: IMX8MP MIPI-CSI2 Problems in custom camera

Matt_S — Mon, 17 May 2021 05:28:45 GMT

No, I didnt yet. Do you know which driver creates the /dev/video device, I was thinking about working back from there and seeing in the code why it is failig.

FYI dmesg looks normal, no errors, but the device is showing as suspended, so adding a power handler to the dts node perhaps?

Re: IMX8MP MIPI-CSI2 Problems in custom camera

Matt_S — Mon, 17 May 2021 05:30:11 GMT

I just used the evk dts as a base. Should I use the Hassler one to get the camera working?

Re: IMX8MP MIPI-CSI2 Problems in custom camera

MicMoba — Mon, 17 May 2021 05:38:20 GMT

where can you see that the device is suspend? Adding a power handler can be possible

Here is one of the dmesg messages:

mxc_mipi-csi 32e40000.csi: 32e40000.csi supply mipi-phy not found, using dummy regulator

Re: IMX8MP MIPI-CSI2 Problems in custom camera

Matt_S — Mon, 17 May 2021 05:49:04 GMT

Yes, I see exactly the same trace. Not sure what it means though.

I dont remember where I saw it sais suspended, somewhere in the file system. I'll try to find where.

Re: IMX8MP MIPI-CSI2 Problems in custom camera

Matt_S — Mon, 17 May 2021 05:52:08 GMT

From dmesg search the file system for that driver name, 'mxc-mipi-csi2-sam', go to that dir, and follow the hardware address.csi link, in that dir isa power dir, and in there a status. I assume this is the state of the hardware.

Re: IMX8MP MIPI-CSI2 Problems in custom camera

MicMoba — Mon, 17 May 2021 06:09:24 GMT

Matt what kind of driver do you use for mipi_csi_0?
I currently use the mxc_mipi_csi driver out of the /driver/media/platform/mxc/capture folder

&mipi_csi_0 {

compatible = "fsl,imx8mm-mipi-csi";

Re: IMX8MP MIPI-CSI2 Problems in custom camera

Matt_S — Mon, 17 May 2021 07:03:09 GMT

I have the one set as default in the dtsi file, but in my case I am on the imx8mp board, so it is :
dsti looks like this:

mipi_csi_0: csi@32e40000 {

compatible = "fsl,imx8mp-mipi-csi", "fsl, imx8mm-mipi-csi";

...

my dts file doesnt change this, its mipi_csi_0 node just sets it to enabled, and sets a remote enpoint.

Re: IMX8MP MIPI-CSI2 Problems in custom camera

MicMoba — Mon, 17 May 2021 07:31:36 GMT

In my origin imx8mp.dtsi the mipi_csi_0 compatible string is:

compatible = "fsl,imx8mp-mipi-csi", "fsl,imx8mn-mipi-csi";

There is no driver for imx8mp-mipi-csi. And if I use the imx8mn-mipi-csi I got a kernel crash during boot up. That is the reason why I changed to imx8mm-mipi-csi.

Furthermore I think the suspend state of the driver has something to do with the power management of the kernel. When the device is not use the kernel set it into suspend mode for saving power.

Is it possible that there is a link missing between mipi_csi and maybe the isp?

Re: IMX8MP MIPI-CSI2 Problems in custom camera

Matt_S — Mon, 17 May 2021 07:52:11 GMT

I am using the 5.4 kernel, there is an 8mp driver there.

I dont know how this is supposed to fit together, so I cant really comment I am afraid. There could be something missing, there seems to be, but I just dont know. .

Re: IMX8MP MIPI-CSI2 Problems in custom camera

MicMoba — Mon, 17 May 2021 08:17:31 GMT

I also use the 5.4.70 kernel. Where did you find the driver? When I search for fsl,imx8mp-mipi-csi overall, I only find entries in the imx8mp.dtsi file but nowhere in the drivers section.

Re: IMX8MP MIPI-CSI2 Problems in custom camera

MicMoba — Tue, 18 May 2021 16:36:31 GMT

I switch back to the driver imx8mn-mipi-csi (mxc-mipi-csi2-sam) and added to my driver the media entities. And now I can see a video0 device.

dmesg | grep mipi [ 1.888060] mxc-mipi-csi2-sam 32e40000.csi: 32e40000.csi supply mipi-phy not found, using dummy regulator [ 1.902159] mxc-mipi-csi2-sam 32e40000.csi: lanes: 2, hs_settle: 13, clk_settle: 2, wclk: 1, freq: 500000000 [ 2.862258] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0] [ 2.868710] mx8-img-md: created link [s2d13p04 2-0037] => [mxc-mipi-csi2.0] moba@moba-hmi:~$ dmesg | grep video0 [ 2.842503] mx8-img-md: Registered mxc_isi.0.capture as /dev/video0

I hope that I am step closer to my objective now.

Re: IMX8MP MIPI-CSI2 Problems in custom camera

Matt_S — Fri, 11 Jun 2021 10:58:39 GMT

What is 'media entities' and how do you add a driver to it? Interesting to see you have a link created, I think this is important, it seems to be the way the device topography is established

Re: IMX8MP MIPI-CSI2 Problems in custom camera

Matt_S — Fri, 11 Jun 2021 11:03:38 GMT

Did you get any where with this? It is important to get the capture device functioning.

Re: IMX8MP MIPI-CSI2 Problems in custom camera

MicMoba — Mon, 14 Jun 2021 05:39:24 GMT

This is the way I added the media entities:

v4l2_i2c_subdev_init(&sensor->subdev, i2c_client, &sensor_subdev_ops);

sensor->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;

sensor->subdev.internal_ops = &sensor_subdev_internal_ops;

sensor->subdev.entity.function = MEDIA_ENT_F_CAM_SENSOR;

sensor->subdev.entity.ops = &sensor_entity_ops;

sensor->pad[0].flags = MEDIA_PAD_FL_SOURCE;

ret = media_entity_pads_init(&sensor->subdev.entity, 1,sensor->pad);

You can print out the topology with media-ctl tool

media-ctl --print-topology
media-ctl --print-dot > graph.dot
dot -Tpng graph.dot > graph.png