i.MX ProcessorsのトピックIMX8MP HDMI input Convert to MIPI-CSI debug issue

IMX8MP HDMI input Convert to MIPI-CSI debug issue

QiangF — Fri, 13 May 2022 12:13:44 GMT

Problem Description：

Use the gst-launch command to get the image data from the HDMI camera and display it on the Ubuntu GUI interface. At present, it is located that the MIPI CSI hardware interrupt cannot be triggered, and the MIPI DPHY state is incorrect. Before MIPI transmits data, the MIPI line does not enter the stop state, resulting in data Unable to send to display. I want to confirm why the data line of MIPI DPHY does not enter the stop state. Is there any good debug method at this stage? If need measure signal, I should focus on which one signals?

Hardware Environment：

The HDMI camera is connected to the imx8mp mipi csi2 interface through the HDMI TO CSI adapter board (LT6911C), as shown below

IMX8MP controls lt6911UXC chip through I2C2 bus

Software Environment：

1. Rootfs Ubuntu 20.04 rootfs, build from nxp yocto project
2. Kernel version 5.4.70, from Feiling BSP
3. DTS LT6911UXC Node Configuration

&i2c2 {

clock-frequency = <100000>;

pinctrl-names = "default";

pinctrl-0 = <&pinctrl_i2c2>;

status = "okay";

lt6911uxc@2b {

compatible = "lontium,lt6911uxc";

reg = <0x2b>;

status = "okay";

// 省略部分无关信息

interrupt-parent = <&gpio5>;

interrupts = <5 IRQ_TYPE_LEVEL_HIGH>;

reset-gpios = <&gpio5 4 GPIO_ACTIVE_LOW>;

plugin-det-gpios = <&gpio2 6 GPIO_ACTIVE_HIGH>;

hpd-ctl-gpios = <&gpio5 5 GPIO_ACTIVE_HIGH>;

camera-module-index = <0>;

port {

hdmi2csi_lt6911_in: endpoint {

data-lanes = <1 2 3 4>;

clock-lanes = <0>;

remote-endpoint = <&hdmi2csi_csi1_ep>;

link-frequencies = /bits/ 64 <297000000>;

};

&mipi_csi_1 {

#address-cells = <1>;

#size-cells = <0>;

status = "okay";

port@1 {

reg = <1>;

hdmi2csi_csi1_ep: endpoint {

remote-endpoint = <&hdmi2csi_lt6911_in>;

data-lanes = <4>;

csis-hs-settle = <13>;

csis-clk-settle = <2>;

csis-wclk;

};

}

Kernel boot initialization log

[ 18.450613] isi-capture 32e02000.isi:cap_device: mxc_isi_subdev_registered

[ 18.450620] isi-capture 32e02000.isi:cap_device: mxc_isi_register_cap_device

[ 18.450967] mx8-img-md: Registered mxc_isi.1.capture as /dev/video0

[ 18.451079] mx8-img-md: Registered sensor subdevice: LT6911UXC_CAM_00 (1)

[ 18.451086] mx8-img-md: created link [mxc_isi.1] => [mxc_isi.1.capture]

[ 18.451091] mx8-img-md: created link [mxc-mipi-csi2.1] => [mxc_isi.1]

[ 18.451095] mx8-img-md: created link [LT6911UXC_CAM_00] => [mxc-mipi-csi2.1]

[ 18.451102] mxc-md 32c00000.bus:camera: mxc_md_create_links

Camera frame rate information can be read back through the i2c interface, as follows

[ 25.052282] LT6911UXC 1-002b: bypass power on for lt6911uxc chip

[ 25.057630] LT6911UXC 1-002b: bypass power on for lt6911uxc chip

[ 30.078769] LT6911UXC_CAM_00: MIPI Byte clk: 80999KHz, MIPI bitrate: 647Mbps, lanes:4

[ 30.095893] LT6911UXC_CAM_00: act:1280x900, total:1692x947, pixclk:148492000, fps:93

[ 30.095900] LT6911UXC_CAM_00: hfp:82, hs:44, hbp:286, vfp:4, vs:5, vbp:38, inerlaced:0

[ 30.095906] LT6911UXC_CAM_00: Format_change: New format: 1280x900p92.67 (1692x947)

[ 30.290737] LT6911UXC_CAM_00: MIPI Byte clk: 80999KHz, MIPI bitrate: 647Mbps, lanes:4

[ 30.308329] LT6911UXC_CAM_00: act:1280x900, total:1692x947, pixclk:148492000, fps:93

[ 30.308337] LT6911UXC_CAM_00: hfp:82, hs:44, hbp:286, vfp:4, vs:5, vbp:38, inerlaced:0

error occurs

Execute the following command on the CLI:

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=4 ! video/x-raw, format=YUY2, width=1280, height=720,framerate=30/1 ! imxvideoconvert_g2d ! queue ! video/x-raw, format=RGB16, width=1024, height=600 ! glimagesink

The output log is as follows, no preview window pops up on the ubuntu GUI：

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";

Pipeline is PREROLLED ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock // blocked there

Through the kernel dmesg CMD, output the mipi csi register:

As shown in the figure above, the register value of CSIS_DPHYSTATUS[20] is 0Xb1, which should be 0Xf1 normally, indicating that all lines enter the stop state. The register description is as follows:

attach LT6911C brief

Re: IMX8MP HDMI input Convert to MIPI-CSI debug issue

khang_letruong — Fri, 13 May 2022 16:47:16 GMT

Hi @QiangF ,

The value of CSIS_DPHYSTATUS, especially the values of Stop State for clock and data lanes are quite instantaneous right after the streaming started and they do not reflect exactly the status of those signals.

Not sure if the firmware programed to your LT6911UXC similar to mine but I remember that the firmware for my LT6911UXC will automatically switch output from 2 data lanes to 4 data lanes for resolution 1080p to 4K. So ensure that the data is present on 4 lanes when you stream with gst-launch (I saw that you streamed width=1280, height=720).

Best Regards,
Khang.

Re: IMX8MP HDMI input Convert to MIPI-CSI debug issue

QiangF — Sat, 14 May 2022 07:32:01 GMT

@khang_letruong

Thanks your reply，

The current situation is that there is data on the MIPI lane, but the cpu cannot see the interrupt trigger of the MIPI-CSI

MIPI clock and camera resolution as below:

124.811438] LT6911UXC_CAM_00: MIPI Byte clk: 80999KHz, MIPI bitrate: 647Mbps, lanes:4
[ 124.827856] LT6911UXC_CAM_00: act:1280x900, total:1692x947, pixclk:148492000, fps:93
[ 124.827863] LT6911UXC_CAM_00: hfp:82, hs:44, hbp:286, vfp:4, vs:5, vbp:38, inerlaced:0

the frame_end bit in the MIPI CSII_INTSRC register is never set

Do you have any debug comments and methods for the next step?

Re: IMX8MP HDMI input Convert to MIPI-CSI debug issue

khang_letruong — Mon, 16 May 2022 11:31:40 GMT

Hi @QiangF ,

Do you have the following information during or when stopping streaming?

[ 106.507262] mxc-mipi-csi2.0: Frame End events: 0 [ 106.514692] mxc-mipi-csi2.0: Frame Start events: 0 [ 106.519483] mxc-mipi-csi2.0: Non-image data after odd frame events: 0 [ 106.525922] mxc-mipi-csi2.0: Non-image data before odd frame events: 0 [ 106.532448] mxc-mipi-csi2.0: Non-image data after even frame events: 0 [ 106.538974] mxc-mipi-csi2.0: Non-image data before even frame events: 0 [ 106.545586] mxc-mipi-csi2.0: Unknown Error events: 0 [ 106.550548] mxc-mipi-csi2.0: CRC Error events: 0 [ 106.555166] mxc-mipi-csi2.0: ECC Error events: 0 [ 106.559782] mxc-mipi-csi2.0: FIFO Overflow Error events: 0 [ 106.565264] mxc-mipi-csi2.0: Lost Frame End Error events: 0 [ 106.570836] mxc-mipi-csi2.0: Lost Frame Start Error events: 0 [ 106.576581] mxc-mipi-csi2.0: SOT Error events: 0

Note that above log was in the case that there's no data from the sensor (lt6911uxc in your case) at all. One of those counters should be different than 0 in case there's data from the sensor.

Regards,

Khang

Re: IMX8MP HDMI input Convert to MIPI-CSI debug issue

QiangF — Mon, 16 May 2022 12:03:58 GMT

Hi, @khang_letruong

Thanks for you reply.

after update the firmware of LT6911C，we can see the interrupt trigger of the MIPI-CSI，But a new error occurs as below:

We check the field descriptions, status = 0x80000 means line3 fails to start transmission, 19~16 bits of the following registers:

The MIPI host reports Start of transmission error, [Lane3,Lane2,Lane1,Lane0]

Re: IMX8MP HDMI input Convert to MIPI-CSI debug issue

khang_letruong — Mon, 16 May 2022 13:55:11 GMT

Hi @QiangF ,

Might you provide more info about the firmware of the LT6911C such as output format on number of lanes for example ?

Regards,
K.

Re: IMX8MP HDMI input Convert to MIPI-CSI debug issue

QiangF — Mon, 16 May 2022 15:24:32 GMT

Hi,

@khang_letruong

the LT6911 Output format as below log:

[124.811438] LT6911UXC_CAM_00: MIPI Byte clk: 80999KHz, MIPI bitrate: 647Mbps, lanes:4
[ 124.827856] LT6911UXC_CAM_00: act:1280x900, total:1692x947, pixclk:148492000, fps:93
[ 124.827863] LT6911UXC_CAM_00: hfp:82, hs:44, hbp:286, vfp:4, vs:5, vbp:38, inerlaced:0

thank you

Re: IMX8MP HDMI input Convert to MIPI-CSI debug issue

khang_letruong — Mon, 16 May 2022 17:24:58 GMT

Hi @QiangF ,

1. By below, do you confirm that you measured with the oscilloscope and saw burst of data (waveform) on the MIPI clock and all data lines?

The current situation is that there is data on the MIPI lane, but the cpu cannot see the interrupt trigger of the MIPI-CSI

2. By below, I would like to know what is the pixel format of the output behind lt6911 (or input of MIPI-CSI2 of iMX8MP) ?

Might you provide more info about the firmware of the LT6911C such as output format on number of lanes for example ?

3. Do you use ISI (&isi_0 node enabled) or ISP (&isp_0 node enabled) in device-tree ?

Regards,
K

Re: IMX8MP HDMI input Convert to MIPI-CSI debug issue

ronjacoby1 — Wed, 26 Jul 2023 21:10:51 GMT

im trying to bring up a similar setup, can you share the lt6911 driver you are using, and device tree if it worked for you ?

Re: IMX8MP HDMI input Convert to MIPI-CSI debug issue

kamalesh — Fri, 07 Feb 2025 07:57:49 GMT

Hi,

We are also doing the same setup here?
Have you got driver and working setup?

Could you please share the file with us?
Thanks.

Regards,
Kamalesh