FAQs
English (US) 日本語 | Japanese 中文 | Chinese (Simplified)

iMX8QM video capture drivers docs? (intergrating ADV748x)

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 

iMX8QM video capture drivers docs? (intergrating ADV748x)

‎07-27-2022 01:05 AM
895 Views
akochubey
Contributor I

Hello everyone,

We are trying to capture video through MIPI-CSI2 interface on iMX8QM processor. The video comes through Analog Devices ADV7481 chip (PAL/HDMI convertor to CSI-2). We use third-party iMX8 module (https://www.tq-group.com/de/produkte/tq-embedded/arm-architektur/tqma8x/) and a custom "backplane" board. Linux distribution is from tq-group, slightly modified by us (kernel config, device tree changes), 5.10.17 kernel.

The datapath is:

Camera -> ADV7481 -> iMX8QM CSI2 input (ISI 1)

We can see ADV7481 chip on I2C bus, adv748x modules loads and prints chip identification, etc, but /dev/video1 does not appear in Linux. There is some "pad" mismatch during mx8-img-md module load (see log below).

The problem is that we are not sure if we are using the right drivers at all.

There seems to be a lot of variances between iMX processors ("i.MX Linux Reference Manual, Rev. LF5.15.32_2.0.0, 30 June 2022", p.135), which causes variance to used drivers. There are many similar drives in drivers/media/, drivers/staging, etc, which are hardly distinguishable by name (imx... vs imx8...).

Could anyone point us to a document, which would describe video capture modules (and other changes), necessary for integrating CSI camera on iMX8QM processor?

Say, lsmod now shows for us:

imx8_capture
imx8_isi_m2m
imx8_isi_cap
imx8_mipi_csi2
imx8_isi_hw
v4l2_mem2mem
imx8_media_dev

adv748x

v4l2_dv_timings
v4l2_fwnode
vpu_windsor
vpu_malone
videobuf2_dma_contig
videobuf2_vmalloc
videobuf2_memops
videobuf2_v4l2
videobuf2_common

Are these the correct modules at all (those we should use)? What are the responsibilities of those modules (which should create /dev/videoX)?  What is their correct load sequence?

------------------ dmesg ----------------

[ 531.589348] imx8_media_dev: module is from the staging directory, the quality is unknown, you have been warned.
[ 531.608226] mx8-img-md: Registered mxc_isi.0.capture as /dev/video1
[ 531.615246] mx8-img-md: Registered mxc_isi.4.capture as /dev/video2
[ 531.621992] mxc-md bus@58000000:camera: Driver mxc-md uses device fwnode, incorrect match may occur
[ 531.631117] mxc-md bus@58000000:camera: Consider updating driver mxc-md to match on endpoints
[ 531.639752] mx8-img-md: Registered sensor subdevice: adv748x 3-0071 txa (1)
[ 531.646747] mxc-md bus@58000000:camera: Driver mxc-md uses device fwnode, incorrect match may occur
[ 531.655826] mxc-md bus@58000000:camera: Consider updating driver mxc-md to match on endpoints
[ 531.664401] mx8-img-md: Registered sensor subdevice: adv748x 3-0070 txa (2)
[ 531.671401] mx8-img-md: created link [mxc_isi.0] => [mxc_isi.0.capture]
[ 531.678042] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0]
[ 531.684511] mx8-img-md: created link [mxc_isi.4] => [mxc_isi.4.capture]
[ 531.691148] mx8-img-md: created link [mxc-mipi-csi2.1] => [mxc_isi.4]
[ 531.697714] ------------[ cut here ]------------
[ 531.702357] WARNING: CPU: 1 PID: 650 at drivers/media/mc/mc-entity.c:669 media_create_pad_link+0x1b4/0x210 [mc]
[ 531.712445] Modules linked in: imx8_media_dev(C+) fsl_jr_uio caam_jr caamkeyblob_desc caamhash_desc caamalg_desc crypto_engine rng_core authenc libdes crct10dif_ce snd_soc_fsl_asoc_card snd_soc_tlv320aic32x4_spi adv748x gpio_pca953x v4l2_dv_timings imx8_capture(C) imx8_isi_m2m(C) snd_soc_tlv320aic32x4_i2c snd_soc_tlv320aic32x4 imx8_isi_cap(C) imx8_mipi_csi2(C) imx8_isi_hw(C) v4l2_mem2mem v4l2_fwnode vpu_malone vpu_windsor videobuf2_dma_contig videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev caam error mc galcore(O) fuse [last unloaded: imx8_media_dev]
[ 531.763740] CPU: 1 PID: 650 Comm: modprobe Tainted: G WC O 5.10.74-tq+gedbaba3667f6 #1
[ 531.772700] Hardware name: TQ-Systems i.MX8QM TQMa8QM on MBa8x (DT)
[ 531.778971] pstate: 20000005 (nzCv daif -PAN -UAO -TCO BTYPE=--)
[ 531.784986] pc : media_create_pad_link+0x1b4/0x210 [mc]
[ 531.790213] lr : subdev_notifier_complete+0x51c/0x870 [imx8_media_dev]
[ 531.796739] sp : ffff80001370b700
[ 531.800048] x29: ffff80001370b700 x28: ffff0008042dc6e0
[ 531.805364] x27: ffff00080076ecf8 x26: 0000000000000001
[ 531.810681] x25: ffff0008042dc080 x24: 00000000fffffdfd
[ 531.815998] x23: 0000000000000038 x22: 0000000000000000
[ 531.821315] x21: ffff0008042dc4c0 x20: ffff00080076ecf8
[ 531.826632] x19: ffff0008044f5080 x18: 0000000000000030
[ 531.831948] x17: ffff80001370b750 x16: 0000000000000000
[ 531.837265] x15: ffff00080c36da18 x14: ffffffffffffffff
[ 531.842582] x13: ffff800011841730 x12: 0000000000001d82
[ 531.847898] x11: 0000000000000038 x10: 0101010101010101
[ 531.853215] x9 : 00000000fffff000 x8 : 7f7f7f7f7f7f7f7f
[ 531.858532] x7 : ff2f2d316872622c x6 : 0000000000000080
[ 531.863848] x5 : 0000000000000000 x4 : 0000000000000003
[ 531.869166] x3 : 0000000000000000 x2 : ffff0008044f5080
[ 531.874482] x1 : 0000000000000000 x0 : 0000000000000001
[ 531.879800] Call trace:
[ 531.882251] media_create_pad_link+0x1b4/0x210 [mc]
[ 531.887135] subdev_notifier_complete+0x51c/0x870 [imx8_media_dev]
[ 531.893368] v4l2_async_notifier_try_complete.part.0+0x58/0x64 [videodev]
[ 531.900183] __v4l2_async_notifier_register+0x130/0x144 [videodev]

------------------ Kernel config ----------------

CONFIG_MEDIA_SUPPORT=m
CONFIG_VIDEO_DEV=m
CONFIG_DVB_CORE=m
CONFIG_VIDEO_V4L2=m
CONFIG_V4L2_MEM2MEM_DEV=m
CONFIG_V4L2_FWNODE=m
CONFIG_VIDEOBUF2_CORE=m
CONFIG_VIDEOBUF2_V4L2=m
CONFIG_VIDEOBUF2_MEMOPS=m
CONFIG_VIDEOBUF2_DMA_CONTIG=m
CONFIG_VIDEOBUF2_VMALLOC=m
CONFIG_VIDEO_MXC_CAPTURE=m
CONFIG_VIDEO_MX8_CAPTURE=m
CONFIG_VIDEO_MXC_CSI_CAMERA=m
CONFIG_MXC_MIPI_CSI=m
CONFIG_MXC_CAMERA_OV5640_MIPI_V2=m
CONFIG_MXC_CAMERA_OV5647_MIPI=m
CONFIG_IMX8_MIPI_CSI2_YAV=m
CONFIG_VIDEO_IMX290=m
CONFIG_VIDEO_OV5640=m
CONFIG_VIDEO_OV5647=m
CONFIG_VIDEO_OV9281=m
CONFIG_VIDEO_VC_SENSORS_FPGA=m
CONFIG_VIDEO_VC_SENSORS=m
CONFIG_MEDIA_TUNER=m
CONFIG_USB_F_UVC=m
CONFIG_VIDEO_IMX_CAPTURE=m
CONFIG_IMX8_MIPI_CSI2=m
CONFIG_IMX8_MIPI_CSI2_SAM=m
CONFIG_IMX8_ISI_HW=m
CONFIG_IMX8_ISI_CORE=m
CONFIG_IMX8_ISI_CAPTURE=m
CONFIG_IMX8_ISI_M2M=m
CONFIG_GMSL_MAX9286=m
CONFIG_IMX8_MEDIA_DEVICE=m
CONFIG_IMX8_PARALLEL_CSI=m
CONFIG_MXC_HANTRO_V4L2=m
CONFIG_MXC_VPU_MALONE=m
CONFIG_MXC_VPU_WINDSOR=m
# CONFIG_MXC_GPU_VIV is not set

CONFIG_VIDEO_ADV_DEBUG=y

CONFIG_VIDEO_ADV748X=m

 

----------- device tree change ---------

///////////////////////////
// Image Sensor Interface

&isi_0 {
status = "okay";
interface = <2 0 2>; // See AN13430 p3 : CSI0 - VC 0 - Memory

cap_device {
status = "okay";
};

m2m_device {
status = "okay";
};
};

&isi_1 {
status = "disabled";

cap_device {
status = "disabled";
};
};

&isi_2 {
status = "disabled";

cap_device {
status = "disabled";
};
};

&isi_3 {
status = "disabled";

cap_device {
status = "disabled";
};
};

&isi_4 {
status = "disabled";
interface = 0 2>; // See AN13430 p3 : CSI1 - VC 0 - Memory

cap_device {
status = "disabled";
};

m2m_device {
status = "disabled";
};
};

&isi_5 {
status = "disabled";

cap_device {
status = "disabled";
};
};

&isi_6 {
status = "disabled";

cap_device {
status = "disabled";
};
};

&isi_7 {
status = "disabled";

cap_device {
status = "disabled";
};
};

///////////////////////////
// CSI blocks

&mipi_csi_0 {
#address-cells = <1>;
#size-cells = <0>;
/delete-property/virtual-channel;

status = "okay";

port@0 {
reg = <0>;
mipi_csi_0_in: endpoint {
remote-endpoint = <&adv7481_0_txa>;
data-lanes = <1 2 3 4>;
};
};
};

&mipi_csi_1 {
#address-cells = <1>;
#size-cells = <0>;
/delete-property/virtual-channel;

status = "disabled";

port@0 {
reg = <0>;
mipi_csi_1_in: endpoint {
remote-endpoint = <&adv7481_1_txa>;
data-lanes = <1 2 3 4>;
};
};
};

///////////////////////////
// I2C2

&i2c2 {
status = "okay";

#address-cells = <0x1>;
#size-cells = <0x0>;

clock-frequency = <100000>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_lpi2c2>;

tca_expander: tca9534@38 {
compatible = "ti,tca9554";
reg = <0x38>;
gpio-controller;
#gpio-cells = <2>;
};


video-receiver@70 {
status = "okay";
compatible = "adi,adv7481";
// reg = <0x70>;
// reg-names = "main";
reg = <0x70 0x41 0x42 0x43 0x44 0x45 0x46 0x47 0x48 0x49 0x4A 0x4B>;
reg-names = "main", "dpll", "cp", "hdmi", "edid", "repeater", "infoframe", "cbus", "cec", "sdp", "txa", "txb";

#address-cells = <1>;
#size-cells = <0>;

port@a {
reg = <0x0a>;

adv7481_0_txa: endpoint {
clock-lanes = <0>;
data-lanes = <1 2 3 4>;
remote-endpoint = <&mipi_csi_0_in>;
};
};

port@8 {
reg = <8>;

adv74812_0_hdmi: endpoint {
// remote-endpoint = <&hdmi_in>;
};
};
};

video-receiver@71 {
status = "disabled";
compatible = "adi,adv7481";
reg = <0x71 0x51 0x52 0x53 0x54 0x55 0x56 0x57 0x58 0x59 0x5A 0x5B>;
reg-names = "main", "dpll", "cp", "hdmi", "edid", "repeater", "infoframe", "cbus", "cec", "sdp", "txa", "txb";

#address-cells = <1>;
#size-cells = <0>;

port@0 {
reg = <0>;

adv7481_1_pal: endpoint {
// remote-endpoint = <&hdmi_in>;
};
};

port@a {
reg = <10>;

adv7481_1_txa: endpoint {
clock-lanes = <0>;
data-lanes = <1 2 3 4>;
remote-endpoint = <&mipi_csi_1_in>;
};
};
};
};

///////////////////////////
// IO mux

&iomuxc {
pinctrl_lpi2c2: lpi2c2grp {
fsl,pins = <
IMX8QM_GPT1_CLK_DMA_I2C2_SCL 0xcA000020
IMX8QM_GPT1_CAPTURE_DMA_I2C2_SDA 0xcA000020
>;
};
};

Tags (1)
0 Kudos
Reply
0 Replies