iMX7D : MIPI CSI : ADV7280-M

[dh29]

I have just starting to try and bring up a ADV7280-M video grabber interfaced via MIPI-CSI interface using a iMX7D. I am using kernel v4.9.x (Freescale/NXP).

Upon delving in it looks like there are two possible driver options:

• /drivers/media/platform/mxc/subdev/adv7180.c
• /drivers/media/i2c/adv7180.c

It seems that /drivers/media/platform/mxc/subdev/adv7180.c is provided by NXP so seems the logical way to go, so I have enabled MXC_TVIN_ADV7180 in my kernel options and made sure that VIDEO_ADV7180 option is disabled.

I have put together a device-tree, but I am not sure if I have everything, and I am struggling to understand some of the fields. I have also read that I may need parts of the OV5640 (MIPI camera) driver? At the moment I have not attempted this.

This is the relevant parts of my device tree so far:

adv7280m@21 {
   status = "okay";
   compatible = "adi,adv7280-m";
   reg = <0x20>;
   pwn-gpios = <&tca6424_sb_23 13 GPIO_ACTIVE_HIGH>;
   csi_id =<0>; /* ?? */
   cvbs = <8>; /* Analogue channel: input 8 */
   clocks = <&clks IMX7D_CSI_MCLK_ROOT_CLK>;
   clock-names = "csi_mclk";
   mipi_camera = <1>; /* ?? */
   ipu_id = <0>; /* ?? */

   adv7280m_out: endpoint {
      clock-lanes = <0>;
      data-lanes = <1>;
      remote-endpoint = <&mipi_sensor_ep>;
      };
};

&mipi_csi {
   clock_frequency = <24000000>;
   status = "okay";

   port {
      mipi_sensor_ep: endpoint1 {
      remote-endpoint = <&adv7280m_out>;
      clock-lanes = <0>;
      data-lanes = <1>;
      csis-hs-settle = <13>;
      csis-clk-settle = <2>;
      };

         csi_mipi_ep: endpoint2 {
            remote-endpoint = <&csi_ep>;
         };
   };

&csi1 {
   csi-mux-mipi = <&gpr 0x14 4>;   /* ?? */
   status = "okay";

   port {
      csi_ep: endpoint {
      remote-endpoint = <&csi_mipi_ep>;
   };
};

On boot I see the following:

MIPI CSI2 driver module loaded

mxc_mipi-csi 30750000.mipi-csi: mipi csi v4l2 device registered
CSI: Registered sensor subdevice: mxc_mipi-csi.0
mxc_mipi-csi 30750000.mipi-csi: lanes: 1, hs_settle: 13, clk_settle: 2, wclk: 0, freq: 166000000

But nothing regarding the ADV7280. I have placed some diags in the driver and its not being probed, so I'm missing something (probably obvious!).

Any help would be appreciated.

[dh29]

I have added 'printk' debug lines through the ADV7280M driver code and ioctl_dev_init() [sets up the mipi-csi2 interface] is never being called. Probably explains why the mipi clock is not functional.

A function pointer to ioctl_dev_init() is part of the v4l2 sensor interface referenced from vidioc_int_dev_init_num enumerator, but searching the linux source code I cannot find where it is called from.

ioctl_dev_init() should be called when the 'slave attaches to the master'.

Can anybody explain this slave/master relationship and what should be calling ioctl_dev_init()?

[dh29]

Some progress....

After receiving no replies from NXP on a number of posts for over a week (very disappointing) I dug back through the 1000's of pages of poor/mis-leading NXP's documentation. As part of the "i.MX Linux Reference Manual", "Chapter 19:OmniVision Camera Driver" states that there are two driver source code paths: one for the i.MX6Dual/Quad etc... (media/platform/mxc/capture) and one for the i.MX7D etc.. (media/platform/mxc/subdev).

I have developed my ADV7280M driver for the wrong framework. I should be using the 'subdev' framework.

I have now re-written my driver for the 'subdev' framework and I am having a little more success (using gstreamer I can occasionally get a partially recognizable image), but only if I use v4l2src as my src.

I get no success if I use imxv4l2videosrc as my src, getting an error of 'inappropriate ioctl for device'.

I am still struggling with a number of questions:

• What should I see with lsmod for a working system?
• joanxie mentioned previously to check:

  For MIPI CSI input, the clock mode in IPU_CSI_SENS_CONF must be gated clock mode.

  p->u.bt656.clock_curr = 1,

  For interlaced input, IDMAC 0 should be set to interlaced mode: "params.csi_mem.interlaced = true;"

  For progressive input, IDMAC 0 shoudl be set to progressive mode: "params.csi_mem.interlaced = false;"

  case IPU_CSI_CLK_MODE_GATED_CLK: in file drivers\media\video\mxc\capture\ipu_csi_enc.c, function csi_enc_setup().

  For device type:

  adv7280_data.sen.pix.priv = 1;  /* 1 is used to indicate TV in */"

         But I can't figure out where to set these using the 'subdev' framework ???

• What video encoding format should I be using? The ADV7280M states its 8-bit YCrCb 4:2:2 which I understand is 'equivalent' to 8-bit YUV 4:2:2. But the various parts of the 'subdev' farmework uses different terminology e.g. YUYV, UYVY etc....???

[joanxie]

I got some hints from expert team about adv7280, hope helpful for you

"Some summary:

1. the adv7280 should be enabled after mipi_csi2_reset().

2. For MIPI CSI input, the clock mode in IPU_CSI_SENS_CONF must be gated clock mode.

          p->u.bt656.clock_curr = 1;

3. For interlaced input, IDMAC 0 should be set to interlaced mode: "params.csi_mem.interlaced = true;" For progressive input, IDMAC 0 shoudl be set to progressive mode: "params.csi_mem.interlaced = false;"

          case IPU_CSI_CLK_MODE_GATED_CLK: in file drivers\media\video\mxc\capture\ipu_csi_enc.c, function csi_enc_setup().

4. For device type:

adv7280_data.sen.pix.priv = 1;  /* 1 is used to indicate TV in */"

[joanxie]

one also can refer to the document as below:

https://community.nxp.com/docs/DOC-339925?sr=search&searchId=217d6ec0-f7b8-4000-bcf3-1c1408006485&se... 

[dh29]

I have checked the 'expert' hints and can confirm I have the same:

1. the adv7280 should be enabled after mipi_csi2_reset().

2. For MIPI CSI input, the clock mode in IPU_CSI_SENS_CONF must be gated clock mode.

          p->u.bt656.clock_curr = 1;

3. For interlaced input, IDMAC 0 should be set to interlaced mode: "params.csi_mem.interlaced = true;" For progressive input, IDMAC 0 shoudl be set to progressive mode: "params.csi_mem.interlaced = false;"

          case IPU_CSI_CLK_MODE_GATED_CLK: in file drivers\media\video\mxc\capture\ipu_csi_enc.c, function csi_enc_setup().

4. For device type:

adv7280_data.sen.pix.priv = 1;  /* 1 is used to indicate TV in */"

I was having some trouble building the imx-test v4l2 utilities to help me diagnose further, but I have since resolved this [thanks to imx-test : Compile error "linux/mxcfb.h: No such file or directory" ].

Delving straight in, upon running ./mxc_v4l2_capture.out I get the following error:

---- Running < ./mxc_v4l2_capture.out > test ----

in_width = 640, in_height = 480
out_width = 176, out_height = 144
top = 0, left = 0
VIDIOC_DBG_G_CHIP_IDENT failed.
get format failed

It seems I have something fundamentally wrong.

As I have no previous experience of a mipi / csi camera system on a linux platform, so I am not sure what to look for next. Highlighted boot log:

Linux video capture interface: v2.00
MIPI CSI2 driver module loaded
pxp-v4l2 pxp_v4l2_out: initialized
[ADV7280M] Init 1
[ADV7280M] Probe-3a addr = 0x20
[ADV7280M] Probe-3b name = adv7280
[ADV7280M] Probe-6, csi_tx_addr: 0x50
[ADV7280M] Probe-9, vpp_tx_addr: 0x54
[ADV7280M] Probe-11 Rd i2c_dev:0x20 addr:0x11
adv7280_i2c_probe: device found, rev_id 0x42
mxc_mipi-csi 30750000.mipi-csi: mipi csi v4l2 device registered
CSI: Registered sensor subdevice: mxc_mipi-csi.0
mxc_mipi-csi 30750000.mipi-csi: lanes: 1, hs_settle: 13, clk_settle: 2, wclk: 0, freq: 166000000

As seen above, the ADV7280-M is probed at boot without any errors. I have added some extra diags. I am not sure what else I would expect to see on a working system?

find -name video* gives me:

./sys/devices/soc0/pxp_v4l2_out/video4linux
./sys/devices/soc0/pxp_v4l2_out/video4linux/video0
./sys/devices/soc0/soc/30400000.aips-bus/30710000.csi/video4linux
./sys/devices/soc0/soc/30400000.aips-bus/30710000.csi/video4linux/video1
./sys/class/video4linux
./sys/class/video4linux/video1
./sys/class/video4linux/video0
./sys/module/mxc_vout/parameters/video_nr
./sys/module/videobuf2_core
./sys/module/videobuf_core
./sys/module/videobuf2_v4l2
./sys/module/mxc_pxp_v4l2/parameters/video_nr
./sys/module/mxc_v4l2_capture/parameters/video_nr
./usr/lib/modules/4.9.11-cl-som-imx7-5.0/kernel/drivers/video
./dev/video1
./dev/video0

Any further advice / help would be much appreciated.

[joanxie]

if you have interlaced input, you need to use mxc_v4l2_tvin.out to test

[dh29]

Hi Joan,

Thank you for your interest.

I have been checking out the CSI-MIPI signals with a scope. I can see activity on the CSI-MIPI data lines, but no activity on the CSI-MIPI clock lines. A post on the Analogue Devices forum ( adv7280-M clock problem | EngineerZone ) discusses this issue. Upon isolating the ADV7280M CSI-MIPI clock lines from the iMX7D the ADV7280M CSI-MIPI clock lines suddenly have activity (200 to 300 mV pk-pk signal, oscillating around 200 MHz). The post concludes that one cause of this problem could be that the iMX7D's CSI-MIPI has not been configured correctly.

Any suggestions on what to check to confirm correct set-up of the CSI-MIPI?

[dh29]

Hi Joan,

Many thanks for your support.

I am sure your advice is correct, but I have not reached that detail yet. My original question was more general on how to get started / how others have approached a driver for the ADV7280M.

Having searched around a number of the community posts it seems that there isn't a pre-existing driver for the ADV7280M. It looks like the correct approach is to write my own driver ( see post https://community.nxp.com/message/830310?commentID=830310#comment-830310 ).

Most of the posts are for earlier Kernels (I am using Kernel 4.9.11), but it looks like the general approach of writing your own driver is still the case using ov5640_mipi.c as a starting point ???