iMX8MQ with camera MIPI CSI2 not streaming

[cleitonbueno]

We are using an iMX8MQ and a camera using the ADV7281 Sensor and MIPI CSI2.

The Driver starts and we receive the message:

adv7180 1-0020: chip id 0x43 found @ 0x20 (30a30000.i2c)

Device /dev/video0 is created, I get information via v4l2-ctl:

Driver Info:
Driver name: mx6s-csi
Card type: i.MX6S_CSI
Bus info: platform:30a90000.csi1_bridge
Driver version: 5.4.142
Capabilities: 0x84200001
Video capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps: 0x04200001
Video capture
Streaming
Extended Pix Format
Priority: 0
Video input: 0 (Camera: ok)
VideoStandard = 0x0000b000
NTSC-M/M-JP/M-KR
Video Capture Format:
Width/Height: 720/480
Pixel Format: 'UYVY' (UYVY 4:2:2)
Field: None
Bytes per Line: 0
Size Image : 691200
Colorspace: Default
Transfer Function: Default (maps to Rec. 709)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization: Default (maps to Limited Range)
Flags:
Crop Capability Video Capture:
Bounds: Left 0, Top 0, Width 0, Height 0
Default: Left 0, Top 0, Width 0, Height 0
Pixel Aspect: 1/1
Selection Video Capture: crop, Left 0, Top 0, Width 0, Height 0, Flags:
Selection Video Capture: crop_default, Left 0, Top 0, Width 0, Height 0, Flags:
Selection Video Capture: crop_bounds, Left 0, Top 0, Width 0, Height 0, Flags:
Selection Video Capture: compose, Left 0, Top 0, Width 0, Height 0, Flags:
Selection Video Capture: compose_default, Left 0, Top 0, Width 0, Height 0, Flags:
Selection Video Capture: compose_bounds, Left 0, Top 0, Width 0, Height 0, Flags:
Selection Video Capture: compose_padded, Left 0, Top 0, Width 0, Height 0, Flags:
Selection Video Capture: native_size, Left 0, Top 0, Width 0, Height 0, Flags:

But when trying to use gstreamer for streaming or v4l2 it doesn't stream there is no video!

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,format=UYVY,width=720,height=240' ! videoconvert! autovideosink -v

Output dmesg `echo 0x1f > /sys/class/video4linux/video0/dev_debug`:

[  266.984783] video0: VIDIOC_QUERYCAP: driver=mx6s-csi, card=i.MX6S_CSI, bus=platform:30a90000.csi1_bridge, version=0x0005048e, capabilities=0x84200001, device_caps=0x04200001
[  266.984833] videodev: v4l2_release: video0: release
[  267.138202] videodev: v4l2_open: video0: open (0)
[  267.138222] video0: VIDIOC_QUERYCAP: driver=mx6s-csi, card=i.MX6S_CSI, bus=platform:30a90000.csi1_bridge, version=0x0005048e, capabilities=0x84200001, device_caps=0x04200001
[  267.138274] videodev: v4l2_release: video0: release
[  267.514775] videodev: v4l2_open: video0: open (0)
[  267.514794] video0: VIDIOC_QUERYCAP: driver=mx6s-csi, card=i.MX6S_CSI, bus=platform:30a90000.csi1_bridge, version=0x0005048e, capabilities=0x84200001, device_caps=0x04200001
[  267.514897] video0: VIDIOC_ENUMINPUT: index=0, name=Camera, type=2, audioset=0x0, tuner=0, std=0x00000000, status=0x0, capabilities=0x4
[  267.514998] video0: VIDIOC_ENUMINPUT: error -22: index=1, name=, type=0, audioset=0x0, tuner=0, std=0x00000000, status=0x0, capabilities=0x4
[  267.515018] video0: VIDIOC_ENUMSTD: error -61: index=0, id=0x0, name=, fps=0/0, framelines=0
[  267.515037] video0: VIDIOC_QUERYCTRL: error -25: id=0x80000000, type=0, name=, min/max=0/0, step=0, default=0, flags=0x00000000
[  267.515055] video0: VIDIOC_QUERYCTRL: error -25: id=0x980900, type=0, name=, min/max=0/0, step=0, default=0, flags=0x00000000
[  267.515071] video0: VIDIOC_QUERYCTRL: error -25: id=0x8000000, type=0, name=, min/max=0/0, step=0, default=0, flags=0x00000000
[  267.515088] video0: VIDIOC_SUBSCRIBE_EVENT: error -25: type=0x5, id=0x0, flags=0x0
[  267.515110] video0: VIDIOC_G_STD: error -515: std=0x00000000
[  267.515125] video0: VIDIOC_G_INPUT: value=0
[  267.516570] video0: VIDIOC_ENUM_FMT: index=0, type=vid-cap, flags=0x0, pixelformat=UYVY, description='UYVY 4:2:2'
[  267.516591] video0: VIDIOC_ENUM_FMT: error -22: index=1, type=vid-cap, flags=0x0, pixelformat=\x00\x00\x00\x00, description=''
[  267.516611] video0: VIDIOC_CROPCAP: type=vid-cap, bounds wxh=0x0, x,y=0,0, defrect wxh=0x0, x,y=0,0, pixelaspect 1/1
[  267.516644] video0: VIDIOC_ENUM_FRAMESIZES: error -515: index=0, pixelformat=UYVY, type=0
[  267.516668] video0: VIDIOC_TRY_FMT: type=vid-cap, width=720, height=240, pixelformat=UYVY, field=none, bytesperline=1440, sizeimage=345600, colorspace=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0
[  267.516692] video0: VIDIOC_TRY_FMT: type=vid-cap, width=720, height=240, pixelformat=UYVY, field=none, bytesperline=1440, sizeimage=345600, colorspace=1, flags=0x0, ycbcr_enc=0, quantization=0, xfer_func=0
[  267.517034] video0: VIDIOC_TRY_FMT: type=vid-cap, width=720, height=240, pixelformat=UYVY, field=none, bytesperline=1440, sizeimage=345600, colorspace=1, flags=0x0, ycbcr_enc=2, quantization=2, xfer_func=1
[  267.518273] video0: VIDIOC_S_FMT: type=vid-cap, width=720, height=240, pixelformat=UYVY, field=none, bytesperline=1440, sizeimage=345600, colorspace=1, flags=0x0, ycbcr_enc=2, quantization=2, xfer_func=1
[  267.518315] video0: VIDIOC_G_PARM: error -515: type=vid-cap, capability=0x0, capturemode=0x0, timeperframe=0/0, extendedmode=0, readbuffers=0
[  267.518347] video0: VIDIOC_EXPBUF: error -22: fd=0, type=vid-cap, index=4294967295, plane=4294967295, flags=0x00080002
[  267.518368] video0: VIDIOC_G_CTRL: error -25: id=0x980927, value=0
[  267.518671] video0: VIDIOC_REQBUFS: count=0, type=vid-cap, memory=mmap
[  267.518687] video0: VIDIOC_CREATE_BUFS: error -25: index=0, count=0, memory=mmap, type=vid-cap, width=720, height=240, pixelformat=UYVY, field=none, bytesperline=1440, sizeimage=345600, colorspace=1, flags=0x0, ycbcr_enc=2, quantization=2, xfer_func=1
[  267.518710] video0: VIDIOC_REQBUFS: count=0, type=vid-cap, memory=userptr
[  267.518721] video0: VIDIOC_CREATE_BUFS: error -25: index=0, count=0, memory=userptr, type=vid-cap, width=720, height=240, pixelformat=UYVY, field=none, bytesperline=1440, sizeimage=345600, colorspace=1, flags=0x0, ycbcr_enc=2, quantization=2, xfer_func=1
[  267.518744] video0: VIDIOC_REQBUFS: error -22: count=0, type=vid-cap, memory=dmabuf
[  267.518888] video0: VIDIOC_G_CTRL: error -25: id=0x980927, value=0
[  267.522969] video0: VIDIOC_REQBUFS: count=4, type=vid-cap, memory=mmap
[  267.523021] video0: VIDIOC_QUERYBUF: 00:00:00.00000000 index=0, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0x0, length=345600
[  267.523045] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  267.523057] video0: VIDIOC_QUERYBUF: 00:00:00.00000000 index=1, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0x55000, length=345600
[  267.523075] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  267.523084] video0: VIDIOC_QUERYBUF: 00:00:00.00000000 index=2, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xaa000, length=345600
[  267.523104] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  267.523113] video0: VIDIOC_QUERYBUF: 00:00:00.00000000 index=3, type=vid-cap, request_fd=0, flags=0x00002000, field=any, sequence=0, memory=mmap, bytesused=0, offset/userptr=0xff000, length=345600
[  267.523133] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  267.523191] video0: VIDIOC_EXPBUF: fd=18, type=vid-cap, index=0, plane=0, flags=0x00080002
[  267.523290] video0: VIDIOC_QBUF: 00:00:00.00000000 index=0, type=vid-cap, request_fd=0, flags=0x00002003, field=any, sequence=0, memory=mmap, bytesused=345600, offset/userptr=0x0, length=345600
[  267.523310] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  267.523333] video0: VIDIOC_EXPBUF: fd=19, type=vid-cap, index=1, plane=0, flags=0x00080002
[  267.523361] video0: VIDIOC_QBUF: 00:00:00.00000000 index=1, type=vid-cap, request_fd=0, flags=0x00002003, field=any, sequence=0, memory=mmap, bytesused=345600, offset/userptr=0x55000, length=345600
[  267.523379] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  267.523400] video0: VIDIOC_EXPBUF: fd=25, type=vid-cap, index=2, plane=0, flags=0x00080002
[  267.523427] video0: VIDIOC_QBUF: 00:00:00.00000000 index=2, type=vid-cap, request_fd=0, flags=0x00002003, field=any, sequence=0, memory=mmap, bytesused=345600, offset/userptr=0xaa000, length=345600
[  267.523447] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  267.523467] video0: VIDIOC_EXPBUF: fd=26, type=vid-cap, index=3, plane=0, flags=0x00080002
[  267.523497] video0: VIDIOC_QBUF: 00:00:00.00000000 index=3, type=vid-cap, request_fd=0, flags=0x00002003, field=any, sequence=0, memory=mmap, bytesused=345600, offset/userptr=0xff000, length=345600
[  267.523516] timecode=00:00:00 type=0, flags=0x00000000, frames=0, userbits=0x00000000
[  267.562141] video0: VIDIOC_STREAMON: type=vid-cap
[  267.562217] videodev: v4l2_poll: video0: poll: 00000000
[  273.094967] videodev: v4l2_poll: video0: poll: 00000000
[  273.095170] video0: VIDIOC_STREAMOFF: type=vid-cap
[  273.095503] video0: VIDIOC_REQBUFS: count=0, type=vid-cap, memory=mmap
[  273.111848] videodev: v4l2_release: video0: release

[joanxie]

what format do you capture? NTSC? if you use interlaced video, did you set mipi csi as de-interlaced mode?

[cleitonbueno]

We are using NTSC.

The ADV7281m CI that we use does not have the I2P flag enabled in the driver structure (we confirmed).

The configured device-tree ADV7281 and MIPI_CSI2.

/ {

    reg_1p8v: regulator-1p8v {
        compatible = "regulator-fixed";
        regulator-name = "1P8V";
        regulator-min-microvolt = <1800000>;
        regulator-max-microvolt = <1800000>;
        regulator-always-on;
    };

    reg_3p3v: regulator-3p3v {
        compatible = "regulator-fixed";
        regulator-name = "3P3V";
        regulator-min-microvolt = <3300000>;
        regulator-max-microvolt = <3300000>;
        regulator-always-on;
    };
};

&i2c2 {
    status = "okay";

    adv7281a_mipi1: adv7281-csi1@20 {
        compatible = "adi,adv7281-m";
        pinctrl-names = "default";
        pinctrl-0 = <&pinctrl_adv7281_csi>;
        reg = <0x20>;
        
        reset-gpios = <&gpio3 19 GPIO_ACTIVE_LOW>;
        powerdown-gpios = <&gpio3 21 GPIO_ACTIVE_HIGH>;
       
        ainterrupt-parent = <&gpio3>;
        interrupts = <23 IRQ_TYPE_LEVEL_LOW>;

        clocks = <&clk IMX8MQ_CLK_CLKO2>;
        clock-names = "csi_mclk";
       /* Disabled CLKO2, since DART-MX8M camera expansion board uses
        * its own oscillator. Enable CLK02 if your design requires it
        */
#if 0
        assigned-clocks = <&clk IMX8MQ_CLK_CLKO2>;
        assigned-clock-parents = <&clk IMX8MQ_SYS2_PLL_200M>;
        assigned-clock-rates = <24000000>;
#endif
        
        csi_id = <0>;

        mclk = <24000000>;
        mclk_source = <0>;
        
        AVDD-supply = <&reg_1p8v>;
	DOVDD-supply = <&reg_3p3v>;
	DVDD-supply = <&reg_1p8v>;
	PVDD-supply = <&reg_1p8v>;
        
        status = "okay";
     
        port {
            adv7281_mipi1_ep: endpoint {
                remote-endpoint = <&mipi1_sensor_ep>;
            };
        };

    };

};

&csi1_bridge {
    status = "okay";

    port {
        csi1_ep: endpoint {
             remote-endpoint = <&csi1_mipi_ep>;
         };
    };

};

&mipi_csi_1 {
    #address-cells = <1>;
    #size-cells = <0>;
    status = "okay";

    clock-frequency = <108000000>;

    port {
        mipi1_sensor_ep: endpoint@0 {
			csis-hs-settle = <13>;
			csis-clk-settle = <2>;
			csis-wclk;
			data-lanes = <1>;
                        remote-endpoint = <&adv7281_mipi1_ep>;
        };
       
        csi1_mipi_ep: endpoint@1 {
            remote-endpoint = <&csi1_ep>;
        };
    };  
};

New message in dmesg, I understand that it is not recognizing the format.

[ 61.552370] unknown mbus:0x0
[ 62.596482] unknown mbus:0x0

[joanxie]

let me remind again, current imx8mq doesn't support bt656, if you want to setup the ADV chip, you need set to mipi data, not BT656, otherwise you couldn't get proper data

[cleitonbueno]

I deleted the property here:
https://github.com/varigit/linux-imx/blob/5.4-2.1.x-imx_var01/arch/arm64/boot/dts/freescale/imx8mq-v...

Regarding disabling BT656, okay, but I didn't understand what you meant by "you need set to mipi data". Do you have any examples of this with iMX8MQ?

[joanxie]

I mean current mipi csi of imx8mq couldn't handle BT656 data, only mipi data

[joanxie]

 as I known current bsp should only support PAL progressive mode, if you use NTSC mode, we don't have patch for this, maybe you can refer to the application note as below

https://www.analog.com/en/resources/app-notes/an-1337.html

you also can refer to the link as below

https://community.nxp.com/t5/i-MX-Processors/iMX7D-NTSC-Video-Capture/m-p/840497

this should be useful for you, they use the same csi driver

[cleitonbueno]

I tried to switch from the NTSC standard to PAL, the auto-detect shows PAL, but without success!
https://github.com/varigit/linux-imx/blob/5.4-2.1.x-imx_var01/drivers/media/i2c/adv7180.c#L1349

[joanxie]

pls refer to this patch too, this should be useful for your case

https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/How-to-enable-i-MX6UL-CSI-port-support-B...