Integrate ADV7280-M with IMX8X

[eduardo3]

I am trying to integrate a ADV7280-M with our IMX8X board, but so far we have not been successful.

I am pasting here the device tree definitions and what we are getting on Linux.

Basically, there are 2 issues:

mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!

and

mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!

Not sure if the second one is a side effect of the first one.

We have a CVBS camera connected to the AIN1 of the ADV7280-M.

I've seen other posts about is_entity_link_setup but none helped me. Could anyone please review my configuration and advise how I should proceed here?

Attached part of the schematic as well.

&i2c1 {

/* other stuff on this bus... */

  adv7280m: adv7280m@21 {
    compatible = "adi,adv7280-m";
    reg = <0x21>;
    status = "okay";
    virtual-channel;
    pinctrl-names = "default";
    pinctrl-0 = <&pinctrl_cvbs_int>;
    powerdown-gpios = <&gpioexp0 6 GPIO_ACTIVE_LOW>;
    reset-gpios = <&gpioexp0 5 GPIO_ACTIVE_LOW>;
    interrupts = <4 IRQ_TYPE_EDGE_FALLING>;
    interrupt-parent = <&lsio_gpio3>;
    interrupt-names = "adv7280m_intrq";
    port {
      adv7280_ep: endpoint {
        remote-endpoint = <&mipi_csi0_ep>;
      };
    };
  };
};

&mipi_csi_0 {
  status = "okay";
  virtual-channel;
  csi = <0>;
  #address-cells = <1>;
  #size-cells = <0>;
  port@0 {
    reg = <0>;
    mipi_csi0_ep: endpoint {
      remote-endpoint = <&adv7280_ep>;
      data-lanes = <1>;
    };
  };
};

img_subsys: bus@58000000 {
  compatible = "simple-bus";
  #address-cells = <1>;
  #size-cells = <1>;
  ranges = <0x58000000 0x0 0x58000000 0x1000000>;
  /* other stuff... */
  cameradev: camera {
    compatible = "fsl,mxc-md", "simple-bus";
    #address-cells = <1>;
    #size-cells = <1>;
    ranges;
    /* other stuff... */
    mipi_csi_0: csi@58227000 {
      compatible = "fsl,mxc-mipi-csi2";
      reg = <0x58227000 0x1000>, <0x58221000 0x1000>;
      clocks = <&csi0_core_lpcg 0>, <&csi0_esc_lpcg 0>, <&csi0_pxl_lpcg 0>;
      clock-names = "clk_core", "clk_esc", "clk_pxl";
      assigned-clocks = <&csi0_core_lpcg 0>, <&csi0_esc_lpcg 0>;
      assigned-clock-rates = <360000000>, <72000000>;
      power-domains = <&pd IMX_SC_R_CSI_0>, <&pd IMX_SC_R_ISI_CH0>;
      power-domain-names = "pd_csi", "pd_isi_ch0";
      status = "disabled";
    };
  };
};

NXP i.MX Release Distro 5.15-kirkstone imx8qxp ttyLP0

root@imx8qxp-d7:~# uname -a
Linux imx8qxp 5.15.71+g9b8ac3af0403 #1 SMP PREEMPT Mon Apr 3 13:43:17 UTC 2023 aarch64 GNU/Linux

root@imx8qxp-d7:~# dmesg | grep -e adv -e csi -e isi
[ 1.720669] mxc-isi 58100000.isi: mxc_isi.0 registered successfully
[ 1.733934] mxc-isi 58110000.isi: mxc_isi.1 registered successfully
[ 1.741600] mxc-isi 58120000.isi: mxc_isi.2 registered successfully
[ 1.749188] mxc-isi 58130000.isi: mxc_isi.3 registered successfully
[ 2.122965] i2c 1-0021: Fixing up cyclic dependency with 58227000.csi
[ 2.686016] adv7180 1-0021: chip id 0x43 found @ 0x21 (5a810000.i2c)
[ 2.918402] mxc-mipi-csi2 58227000.csi: lanes: 1, name: mxc-mipi-csi2.0
[ 3.259051] isi-m2m 58100000.isi:m2m_device: Register m2m success for ISI.0
[ 7.445837] mx8-img-md: Registered mxc_isi.0.capture as /dev/video3
[ 7.468924] mx8-img-md: Registered mxc_isi.1.capture as /dev/video4
[ 7.481234] mx8-img-md: Registered mxc_isi.2.capture as /dev/video5
[ 7.491578] mx8-img-md: Registered mxc_isi.3.capture as /dev/video6
[ 7.502608] mx8-img-md: Registered sensor subdevice: adv7180 1-0021 (1)
[ 7.502631] mx8-img-md: created link [mxc_isi.0] => [mxc_isi.0.capture]
[ 7.502643] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0]
[ 7.502650] mx8-img-md: created link [mxc_isi.1] => [mxc_isi.1.capture]
[ 7.502657] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.1]
[ 7.502665] mx8-img-md: created link [mxc_isi.2] => [mxc_isi.2.capture]
[ 7.502671] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.2]
[ 7.502679] mx8-img-md: created link [mxc_isi.3] => [mxc_isi.3.capture]
[ 7.502686] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.3]
[ 8.661369] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
[ 8.672359] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
[ 8.679262] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
[ 8.741962] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!

root@imx8qxp-d7:~# v4l2-ctl -d /dev/video3 --list-formats-ext
[ 228.856447] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
ioctl: VIDIOC_ENUM_FMT[ 228.863708] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!

Type: Video Capture Multiplanar

[0]: 'RGBP' (16-bit RGB 5-[ 228.872617] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
6-5)
[1]: 'RGB3' (24-bit RGB 8-8-8)
[ 228.885291] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[2]: 'BGR3' (24-bit BGR 8-8-8)
[ 228.895736] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[3]: 'YUYV' (YUYV 4:2:2)
[ 228.905630] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[4]: 'YUV4' (32-bit A/XYUV 8-8-8-8)
[ 228.915036] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[5]: 'NV12' (Y/CbCr 4:2:0)
[ 228.925473] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[6]: 'NM12' (Y/CbCr 4:2:0 (N-C))
[ 228.934963] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[7]: 'YM24' (Planar YUV 4:4:4 (N-C))
[ 228.945041] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[8]: 'XR24' (32-bit BGRX 8-8-8-8)
[ 228.955466] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[9]: 'AR24' (32-bit BGRA 8-8-8-8)

root@imx8qxp-d7:~# v4l2-ctl --list-devices
[ 252.749448] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
[ 252.756157] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
[ 252.762850] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
[ 252.769574] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
amphion vpu decoder (platform: amphion-vpu):
/dev/video0
/dev/video1
/dev/media0

mxc-isi-cap (platform:58100000.isi:cap_devic):
/dev/video3

mxc-isi-m2m (platform:58100000.isi:m2m_devic):
/dev/video2

mxc-isi-cap (platform:58130000.isi:cap_devic):
/dev/video6

mxc-jpeg codec (platform:58400000.jpegdec):
/dev/video7

mxc-jpeg codec (platform:58450000.jpegenc):
/dev/video8

mxc-isi-cap (platform:58130000.isi:cap_devic):
/dev/video6

mxc-jpeg codec (platform:58400000.jpegdec):
/dev/video7

mxc-jpeg codec (platform:58450000.jpegenc):
/dev/video8

[eduardo3 · Accepted solution]

This is for reference for anyone having the same issue.

After adding debug info to ISI and MIPI-CSI2 drivers, adjusted the device tree to the following:

&i2c1 {     
    adv7280m: adv7280m@21 {      
	    compatible = "adi,adv7280-m";
	    reg = <0x21>;
		status = "okay";
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_cvbs_int>;
		reset-gpios = <&gpioexp0 5 GPIO_ACTIVE_LOW>;
		powerdown-gpios = <&gpioexp0 6 GPIO_ACTIVE_LOW>;
		interrupts = <4 IRQ_TYPE_LEVEL_LOW>;
		interrupt-parent = <&lsio_gpio3>;
		interrupt-names = "adv7280m_intrq";

		port {
			adv7280_ep: endpoint {
				remote-endpoint = <&mipi_csi0_ep>;
				data-lanes = <1>;
				clock-lanes = <0>;
			};
		};
    };
};

&mipi_csi_0 {
	status = "okay";

	#address-cells = <1>;
	#size-cells = <0>;

	port@0 {
		reg = <0>;
		mipi_csi0_ep: endpoint {
			remote-endpoint = <&adv7280_ep>;
			data-lanes = <1>;
		};
	};
};

Adding virtual channels would cause failures. Also, the media/subdev functionality expects a link_setup function.

Added the following to adv7180.c:

static int adv7180_link_setup(struct media_entity *entity,
			      const struct media_pad *local,
			      const struct media_pad *remote, u32 flags)
{
	return 0;
}

static const struct media_entity_operations adv7180_sd_entity_ops = {
	.link_setup = adv7180_link_setup,
};


static int adv7180_probe(struct i2c_client *client,
			 const struct i2c_device_id *id)
{
	/* ... */
	sd->entity.ops = &adv7180_sd_entity_ops;
	ret = media_entity_pads_init(&sd->entity, 1, &state->pad);
	/* ... */
}

With the changes above, I no longer get the "no remote pad found" error, I was able to use an analogue camera with our board.

We had to add further changes to adv7180 to be able to select its input (default is 0), we have 4 cameras connected to the board.

Also, ADV7280A-M has an I2P (interlace to progressive block), so the kernel was not patched for de-interlacing as suggested by support. I don't know if that is the right approach but I had to force the field format to be 'none' instead of 'alternate' so the height of the image is not halved. ISI won't upscale the image and that is the reason I've done that. If anyone has a suggestion, I would like to know it.

v4l2-ctl --list-devices
amphion vpu decoder (platform: amphion-vpu):
        /dev/video2
        /dev/video3
        /dev/media0

mxc-isi-cap (platform:58100000.isi:cap_devic):
        /dev/video4

mxc-isi-m2m (platform:58100000.isi:m2m_devic):
        /dev/video5

mxc-jpeg codec (platform:58400000.jpegdec):
        /dev/video0

mxc-jpeg codec (platform:58450000.jpegenc):
        /dev/video1

FSL Capture Media Device (platform:mxc-md):
        /dev/media1


media-ctl -p -d1
Media controller API version 5.15.71

Media device information
------------------------
driver          mxc-md
model           FSL Capture Media Device
serial          
bus info        
hw revision     0x0
driver version  5.15.71

Device topology
- entity 1: mxc_isi.0 (16 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
        pad0: Sink
                <- "mxc-mipi-csi2.0":4 [ENABLED]
        pad1: Sink
        pad2: Sink
        pad3: Sink
        pad4: Sink
        pad5: Sink
        pad6: Sink
        pad7: Sink
        pad8: Sink
        pad9: Sink
        pad10: Sink
        pad11: Sink
        pad12: Source
                -> "mxc_isi.0.capture":0 [ENABLED]
        pad13: Source
        pad14: Source
        pad15: Sink

- entity 18: mxc_isi.0.capture (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video4
        pad0: Sink
                <- "mxc_isi.0":12 [ENABLED]

- entity 22: mxc-mipi-csi2.0 (8 pads, 2 links)
             type Node subtype V4L flags 0
             device node name /dev/v4l-subdev0
        pad0: Sink
                <- "adv7180 1-0021":0 [ENABLED,IMMUTABLE]
        pad1: Sink
        pad2: Sink
        pad3: Sink
        pad4: Source
                -> "mxc_isi.0":0 [ENABLED]
        pad5: Source
        pad6: Source
        pad7: Source

- entity 31: adv7180 1-0021 (1 pad, 1 link)
             type V4L2 subdev subtype Decoder flags 0
             device node name /dev/v4l-subdev1
        pad0: Source
                [fmt:UYVY8_2X8/720x576@1/25 field:none colorspace:smpte170m]
                -> "mxc-mipi-csi2.0":0 [ENABLED,IMMUTABLE]

[Rita_Wang]

Which bsp are you using?

[eduardo3]

Any suggestions? Thanks.

[eduardo3]

NXP Yocto (Kirkstone), kernel 5.15.71. (5.15.71-2.2.0)

https://github.com/nxp-imx/imx-manifest/blob/imx-linux-kirkstone/imx-5.15.71-2.2.0.xml

[joanxie]

did you use interlaced format? PAL or NTSC?

[eduardo3]

I am seeing that with PAL and NTSC cameras.

[joanxie]

I assume that you set ADV7280 as BT656 mode, right? if yes, you need set ISI to deinterlace  mode, if you use this, pls tell me, let me send the demo for android for your reference

[eduardo3]

Yes, I tried with and without "adv,force-bt656-4" to ADV7280-M device tree node (I am assuming this is the one as it is not mentioned in your reply). I don't see a deinterlace option in imx8-isi-cap.c or imx8-isi-core.c. Is this the thing you are talking about android example?

What I was hoping from NXP support is to get a device tree example how that should be configured (adv7280-m node, isi node, etc).

with adv,force-bt656-4, I still get:

# v4l2-ctl -d /dev/video3 --list-formats-ext
[ 277.363752] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
ioctl: VIDIOC_ENUM_FMT[ 277.370947] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!

Type: Video Capture Multiplanar

[0]: 'RGBP' (16-bit RGB 5-[ 277.379918] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
6-5)
[1]: 'RGB3' (24-bit RGB 8-8-8)
[ 277.392584] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[2]: 'BGR3' (24-bit BGR 8-8-8)
[ 277.403024] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[3]: 'YUYV' (YUYV 4:2:2)
[ 277.412924] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[4]: 'YUV4' (32-bit A/XYUV 8-8-8-8)
[ 277.422329] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[5]: 'NV12' (Y/CbCr 4:2:0)
[ 277.432660] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[6]: 'NM12' (Y/CbCr 4:2:0 (N-C))
[ 277.442242] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[7]: 'YM24' (Planar YUV 4:4:4 (N-C))
[ 277.452313] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[8]: 'XR24' (32-bit BGRX 8-8-8-8)
[ 277.462792] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[9]: 'AR24' (32-bit BGRA 8-8-8-8)

or

# gst-launch-1.0 v4l2src device=/dev/video3 ! videoconvert ! waylandsink
[ 342.759687] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
[ 342.766872] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
[ 342.774009] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
[ 342.781083] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
Setting pipeline to PAUSED ...
[ 342.818678] mxc-mipi-csi2.0: is_entity_link_setup, No remote pad found!
Pipeline is live and does not need PREROLL ...[ 342.826925] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!

Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
[ 342.837713] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
New clock: GstSystemClock[ 342.850107] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!

[ 342.859411] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[ 342.866623] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[ 342.873853] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[ 342.881056] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[ 342.888245] mxc-mipi-csi2.0: mipi_csi2_enum_framesizes, No remote pad found!
[ 342.901905] mxc-mipi-csi2.0: mipi_csi2_g_frame_interval, No remote pad found!
[ 342.945003] mxc-mipi-csi2.0: mipi_csi2_s_power, No remote pad found!
[ 342.951478] mxc_isi.0: Call subdev s_power fail!
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
../git/sys/v4l2/gstv4l2src.c(759): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.119040250
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

[joanxie]

you can refer to the link as below

https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/ISL79987-and-adv7180-de-interlace-driver...