IMX8MP MIPI-CSI2 Problems in custom camera

cancel
Showing results for 
Search instead for 
Did you mean: 

IMX8MP MIPI-CSI2 Problems in custom camera

1,025 Views
v_parri
Contributor I

Hi, my name is Victor and I'm in a project where we're trying to send image from a custom 4-lane camera to the IMX8MP (in a imx8mp-evk board) using the MIPI-CSI2 interface but I'm stuck trying to make it work.

I want to modify the ov5640 config and drivers but I'm not sure how many things I have to change. I'm ony interested in make it work sending image, the I2C is not needed at this moment and the camera is conected to the MIPI_CSI_0.

1) Is it enough if I modify the device tree configuration? which parameters do I have to modify? This is the configuration that I'm using:

imx8mp.dtsi:

mipi_csi_0: csi@32e40000 {
compatible = "fsl,imx8mp-mipi-csi", "fsl,imx8mn-mipi-csi";
reg = <0x32e40000 0x10000>;
interrupts = <GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>;
clock-frequency = <500000000>;
clocks = <&clk IMX8MP_CLK_MEDIA_CAM1_PIX>,
<&clk IMX8MP_CLK_MEDIA_AXI>,
<&clk IMX8MP_CLK_MEDIA_APB>;
clock-names = "mipi_clk", "disp_axi", "disp_apb";
assigned-clocks = <&clk IMX8MP_CLK_MEDIA_CAM1_PIX>;
assigned-clock-parents = <&clk IMX8MP_SYS_PLL2_1000M>;
assigned-clock-rates = <500000000>;
bus-width = <4>;
csi-gpr = <&mediamix_gasket0>;
gpr = <&mediamix_blk_ctl>;
no-reset-control;
power-domains = <&mipi_phy1_pd>;
status = "disabled";
};

imx8mp-evk.dts:

&mipi_csi_0 {
#address-cells = <1>;
#size-cells = <0>;
status = "okay";

port@0 {
reg = <0>;
mipi_csi0_ep: endpoint {
remote-endpoint = <&ov5640_mipi_0_ep>;
data-lanes = <4>;
};
};
};

 

2) Do I have to modify the ov5640 driver too? I've located this driver: ov5640_mipi_v2.c but not sure about it

Just modifying the device tree if I try to build a pipe with gstreamer to read from video0 I'm getting this error:

gst-launch-1.0 -v v4l2src ! "video/x-bayer,format=bggr,width=640,height=480,framerate=30/1" ! bayer2rgb ! videoconvert ! autovideosink sync=false
Setting pipeline to PAUSED ...
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device '/dev/video0' is not a capture device.Additional debug info:
../../../git/sys/v4l2/v4l2_calls.c(677): gst_v4l2_open (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Capabilities: 0x4204000
Setting pipeline to NULL ...
Freeing pipeline ...

Also the configuration read from v412-ctl tool is:

v4l2-ctl --list-devices
mxc-isi-m2m (platform:32e00000.isi:m2m_devic):
/dev/video0

 

v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture Multiplanar

[0]: 'RGBP' (16-bit RGB 5-6-5)
[1]: 'RGB3' (24-bit RGB 8-8-8)
[2]: 'BGR3' (24-bit BGR 8-8-8)
[3]: 'YUYV' (YUYV 4:2:2)
[4]: 'YUV4' (32-bit A/XYUV 8-8-8-8)
[5]: 'NV12' (Y/CbCr 4:2:0)
[6]: 'YM24' (Planar YUV 4:4:4 (N-C))
[7]: 'XR24' (32-bit BGRX 8-8-8-8)
[8]: 'AR24' (32-bit BGRA 8-8-8-8)

 

Please any help will be appreciated

0 Kudos
24 Replies

149 Views
malik_cisse
Contributor II

Hi v_parri,

Could you make the 4-lanes work?

What changes did you make to CSI part in imx8mp.dtsi file?

Thanks, MC

0 Kudos

772 Views
Matt_S
Contributor I

Hi   Victor, I have a similar issue.    I only have video0, which is on the isi device, not csi, v4l-ctl --all  says its driver is mxc-isi-m2m, so despite doing pretty much what you have with  imx8mp-evk.dts, the csi device just isnt showing, and neither is yours.  The csi device is on 0x32e40000, the isi device on 32e00000, which is what you have in --list-devices.   

So the question is, how to get the csi device to appear. dmesg shows:
mxc-mipi-csi2-sam   32e40000.csi: lanes 1, hs_settle: 22, clck_settle:  0, wclk: 1, freq: 5000000000  (my dts changes the values for settle)

So it looks as if the csi device loaded ok, it just isnt creating a  videox device.

Any ideas? Its a bit of a mystery. 

0 Kudos

761 Views
MicMoba
Contributor IV

Did you find the reason why csi doesn't create a video device? I just have the same issue. csi is loaded but there is no /dev/video0

Thanks

0 Kudos

749 Views
Matt_S
Contributor I

No, I didnt yet.  Do  you know which driver creates the /dev/video device, I was thinking about working back from there and seeing in the code why it is failig.

FYI dmesg looks normal, no errors,  but the device is showing as suspended, so adding a power handler to the dts node perhaps?  


0 Kudos

734 Views
MicMoba
Contributor IV

where can you see that the device is suspend? Adding a power handler can be possible

Here is one of the dmesg messages: 

mxc_mipi-csi 32e40000.csi: 32e40000.csi supply mipi-phy not found, using dummy regulator

0 Kudos

720 Views
Matt_S
Contributor I

From dmesg search the file system for that driver name, 'mxc-mipi-csi2-sam',  go to that dir, and follow the hardware address.csi link, in that dir isa power dir, and in there a status.    I assume this is the state of the hardware. 

0 Kudos

723 Views
Matt_S
Contributor I

Yes, I see exactly the same trace.  Not sure what it  means though. 

I dont remember where I saw it sais suspended, somewhere in  the file system. I'll try to find where. 

0 Kudos

711 Views
MicMoba
Contributor IV

Matt what kind of driver do you use for mipi_csi_0?
I currently use the mxc_mipi_csi driver out of the /driver/media/platform/mxc/capture folder

&mipi_csi_0 {
compatible = "fsl,imx8mm-mipi-csi";
0 Kudos

697 Views
Matt_S
Contributor I

I have the one set as default in the dtsi file, but in my case I am on the imx8mp board, so it is :
dsti looks like this:

mipi_csi_0: csi@32e40000 {

    compatible = "fsl,imx8mp-mipi-csi", "fsl, imx8mm-mipi-csi";

...

my dts file doesnt change this, its mipi_csi_0 node just sets it to enabled, and sets a remote enpoint.

0 Kudos

689 Views
MicMoba
Contributor IV

In my origin imx8mp.dtsi the mipi_csi_0 compatible string is:

compatible = "fsl,imx8mp-mipi-csi", "fsl,imx8mn-mipi-csi";
 
There is no driver for imx8mp-mipi-csi. And if I use the imx8mn-mipi-csi I got a kernel crash during boot up. That is the reason why I changed to  imx8mm-mipi-csi.
 
Furthermore I think the suspend state of the driver has something to do with the power management of the kernel. When the device is not use the kernel set it into suspend mode for saving power.
 
Is it possible that there is a link missing between mipi_csi and maybe the isp?
 
0 Kudos

682 Views
Matt_S
Contributor I

I am using the 5.4 kernel, there is an 8mp driver there. 

I dont know how this is supposed to fit together, so I cant really comment I am afraid.  There could be something missing, there seems to be, but I just dont know. .

 

0 Kudos

676 Views
MicMoba
Contributor IV

I also use the 5.4.70 kernel. Where did you find the driver? When I search for fsl,imx8mp-mipi-csi overall, I only find entries in the imx8mp.dtsi file but nowhere in the drivers section.

 

0 Kudos

655 Views
MicMoba
Contributor IV

I switch back to the  driver imx8mn-mipi-csi (mxc-mipi-csi2-sam) and added to my driver the media entities. And now I can see a video0 device.

dmesg | grep mipi
[    1.888060] mxc-mipi-csi2-sam 32e40000.csi: 32e40000.csi supply mipi-phy not found, using dummy regulator
[    1.902159] mxc-mipi-csi2-sam 32e40000.csi: lanes: 2, hs_settle: 13, clk_settle: 2, wclk: 1, freq: 500000000
[    2.862258] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0]
[    2.868710] mx8-img-md: created link [s2d13p04 2-0037] => [mxc-mipi-csi2.0]
moba@moba-hmi:~$ dmesg | grep video0
[    2.842503] mx8-img-md: Registered mxc_isi.0.capture as /dev/video0

I hope that I am step closer to my objective now. 

0 Kudos

475 Views
Matt_S
Contributor I

What is 'media entities' and how do you add a driver to it?  Interesting to see you have a link created, I think this is important, it seems to be the way the device topography is established 

0 Kudos

453 Views
MicMoba
Contributor IV
This is the way I added the media entities:
 
v4l2_i2c_subdev_init(&sensor->subdev, i2c_client, &sensor_subdev_ops);

sensor->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
sensor->subdev.internal_ops = &sensor_subdev_internal_ops;
sensor->subdev.entity.function = MEDIA_ENT_F_CAM_SENSOR;
sensor->subdev.entity.ops = &sensor_entity_ops;

sensor->pad[0].flags = MEDIA_PAD_FL_SOURCE;

ret = media_entity_pads_init(&sensor->subdev.entity, 1,sensor->pad);
 
You can print out the topology with media-ctl tool
 
media-ctl --print-topology
media-ctl --print-dot > graph.dot
dot -Tpng graph.dot > graph.png
 
graph.png
 

441 Views
Matt_S
Contributor I

Sorry, crash was not in probe, but in mxc_isi_m2m_g_fmt_vid_caps.

0 Kudos

447 Views
Matt_S
Contributor I

Thats a big help, thanks.   I saw the use of media to set the topography in a script, but that was on older systems, I guess on later kernels it is done programatically via pads. 

And thanks for the tip on using it to show the topography, thats perfect!

What I have is I get a kennel null pointer dereference in imx8-isi-m2m.c then it is probed.  The fmt member is null. 

This is the device that creates /dev/video0 

Have you seen this?  I am assuming is a configuration issue, and am about to debug it, but if you knew its cause that would be a great help to.

Thx!


0 Kudos

444 Views
MicMoba
Contributor IV

I also had a NULL pointer issue. In the function .set_fmt of the v4l2_subdev_pad_ops the fmt pointer was set to NULL. 

I saw that there was an issue in the function. But meanwhile I changed the code so many times.

But thats exactly the field where I am currently working on. My feeling is the the pads are connected together in the right way, but the negotiation of frame format, colorspace and fps in not correct. 

I can list the available formats but the type is not correct. It always says GRAY (8-bit Greyscale)

v4l2-ctl --device /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
       Type: Video Capture Multiplanar

       [0]: 'GREY' (8-bit Greyscale)
               Size: Discrete 768x576
                       Interval: Discrete 0.040s (25.000 fps)
               Size: Discrete 640x480
                       Interval: Discrete 0.033s (30.000 fps)

 

0 Kudos

426 Views
Matt_S
Contributor I

You have made some progress then,  I am not getting /dev/video (created by imx8-isi-m2m.c, to even have capture abilities when queried. 

And I dont have a media0 device either.   I am starting to think the camera is broken!   Nothing  I have tried, with any of the CONFIGs has made any difference. 

0 Kudos

933 Views
joanxie
NXP TechSupport
NXP TechSupport

which dts file do you use? did you try to use imx8mp-evk-basler-ov5640.dtb?

0 Kudos