Trouble with (TC358748) Parallel to CSI2 video bridge

[shaswath]

Hello,

I am working on a project that requires interfacing an FPGA to our custom i.MX8MP board. I am trying to modify the tc358746 (Parallel -> CSI2) driver to support for our yocto lf-5.15.71 version.

I want the video pipeline (FPGA (YUV4:2:2) -> bridge (YUV4:2:2/4-lane) -> mipi_csi0 -> isi0) to output 1920x1080@30Hz with YUV4:2:2 format for now but I am unable to capture any frames using Gstreamer or v4l2-ctl.

The v4l-subdev1 won't show up in the v4l2 device list

Please find the media-ctl output and the dmesg log for the bridge, Any insights on troubleshooting the issue is greatly appreciated.

root@:~$ media-ctl -p
Media controller API version 5.15.71

Media device information
------------------------
driver          mxc-md
model           FSL Capture Media Device
serial          
bus info        
hw revision     0x0
driver version  5.15.71

Device topology
- entity 1: mxc_isi.0 (16 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
        pad0: Sink
                <- "mxc-mipi-csi2.0":4 [ENABLED]
        pad1: Sink
        pad2: Sink
        pad3: Sink
        pad4: Sink
        pad5: Sink
        pad6: Sink
        pad7: Sink
        pad8: Sink
        pad9: Sink
        pad10: Sink
        pad11: Sink
        pad12: Source
                -> "mxc_isi.0.capture":0 [ENABLED]
        pad13: Source
        pad14: Source
        pad15: Sink

- entity 18: mxc_isi.0.capture (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
        pad0: Sink
                <- "mxc_isi.0":12 [ENABLED]

- entity 22: mxc-mipi-csi2.0 (8 pads, 2 links)
             type Node subtype V4L flags 0
             device node name /dev/v4l-subdev0
        pad0: Sink
                <- "tc358748 2-000e":0 [ENABLED,IMMUTABLE]
        pad1: Sink
        pad2: Sink
        pad3: Sink
        pad4: Source
                -> "mxc_isi.0":0 [ENABLED]
        pad5: Source
        pad6: Source
        pad7: Source

- entity 31: tc358748 2-000e (1 pad, 1 link)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev1
        pad0: Source
                [fmt:UYVY8_2X8/1920x1080@1/30 field:none colorspace:srgb]
                -> "mxc-mipi-csi2.0":0 [ENABLED,IMMUTABLE]

root@:~$ v4l2-ctl --list-devices
 ():
        /dev/v4l-subdev0

mxc-isi-cap (platform:32e00000.isi:cap_devic):
        /dev/video2

FSL Capture Media Device (platform:mxc-md):
        /dev/media0

vsi_v4l2dec (platform:vsi_v4l2dec):
        /dev/video1

vsi_v4l2enc (platform:vsi_v4l2enc):
        /dev/video0

root@:~$ v4l2-ctl --device /dev/video2 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture Multiplanar

    [3]: 'YUYV' (YUYV 4:2:2)
         Size: Discrete 1920x1080
         Interval: Discrete 0.033s (30.000 fps)
         Interval: Discrete 0.033s (30.000 fps)

root@:~$ v4l2-ctl -d /dev/video2 --all
Driver Info:
        Driver name      : mxc-isi-cap
        Card type        : mxc-isi-cap
        Bus info         : platform:32e00000.isi:cap_devic
        Driver version   : 5.15.71
        Capabilities     : 0x84201000
                Video Capture Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04201000
                Video Capture Multiplanar
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : mxc-md
        Model            : FSL Capture Media Device
        Serial           : 
        Bus info         : 
        Media version    : 5.15.71
        Hardware revision: 0x00000000 (0)
        Driver version   : 5.15.71
Interface Info:
        ID               : 0x03000014
        Type             : V4L Video
Entity Info:
        ID               : 0x00000012 (18)
        Name             : mxc_isi.0.capture
        Function         : V4L2 I/O
        Pad 0x01000013   : 0: Sink
          Link 0x02000021: from remote pad 0x100000e of entity 'mxc_isi.0' (Video Pixel Formatter): Data, Enabled
Priority: 2
Format Video Capture Multiplanar:
        Width/Height      : 1920/1080
        Pixel Format      : 'RGBP' (16-bit RGB 5-6-5)
        Field             : None
        Number of planes  : 1
        Flags             : 
        Colorspace        : sRGB
        Transfer Function : Default
        YCbCr/HSV Encoding: ITU-R 601
        Quantization      : Full Range
        Plane 0           :
           Bytes per Line : 3840
           Size Image     : 4147200
Crop Capability Video Output:
        Bounds      : Left 0, Top 0, Width 1920, Height 1080
        Default     : Left 0, Top 0, Width 1920, Height 1080
        Pixel Aspect: 1/1
Selection Video Output: compose, Left 0, Top 0, Width 1920, Height 1080, Flags: 
Selection Video Output: compose_default, Left 0, Top 0, Width 1920, Height 1080, Flags: 
Selection Video Output: compose_bounds, Left 0, Top 0, Width 1920, Height 1080, Flags: 
Streaming Parameters Video Capture:
        Capabilities     : timeperframe
        Frames per second: 30.000 (30/1)
        Read buffers     : 0

User Controls

                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
                alpha_component 0x00980929 (int)    : min=0 max=255 step=1 default=0 value=0

[ 15.547696] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 26.966015] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
.
.
.
[ 26.968164] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 26.968181] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 26.968195] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 26.968210] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 26.971188] mxc_isi.0: Not match format, set default
[ 26.971199] mxc_isi.0: Not match format, set default
[ 27.305025] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.305150] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.305352] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.305392] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.305442] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.305507] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.309455] enter tc358748_s_power = ON
[ 27.309471] tc358748 2-000e: tc358748_enable_hw
[ 27.311682] tc358748 2-000e: tc358748_set_pad_format
[ 27.311696] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.311705] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.316716] bypass csc
[ 27.316724] input fmt YUV4
[ 27.316728] output fmt YUYV
[ 27.667770] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.667876] tc358748 2-000e: tc358748_set_pad_format
[ 27.667882] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.667889] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.672900] bypass csc
[ 27.672909] input fmt YUV4
[ 27.672912] output fmt YUYV
[ 27.672922] enter tc358748_s_power = OFF
[ 27.672929] tc358748 2-000e: tc358748_disable_hw
[ 27.701511] enter tc358748_s_power = ON
[ 27.701530] tc358748 2-000e: tc358748_enable_hw
[ 27.703406] tc358748 2-000e: tc358748_set_pad_format
[ 27.703418] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.703427] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 27.708440] bypass csc
[ 27.708451] input fmt YUV4
[ 27.708456] output fmt YUYV
[ 28.035292] tc358748 2-000e: [tc358748_stream,1152]: Stream enable ...
[ 28.035309] tc358748 2-000e: tc358748_set_pll
[ 28.035313] tc358748 2-000e: tc358748_calc_pll
[ 28.035382] tc358748 2-000e: PLL: refclk 10000000, fbd 88, prd 0, frs 1
[ 28.035388] tc358748 2-000e: PLL: pll_clk: 445000000, CSIClk 222500000, CSIByteClk 55625000
[ 28.035393] tc358748 2-000e: PLL: pclk 74166666 (FPGA(Hardcode=>: 74250000MHz))
[ 28.037869] tc358748 2-000e: tc358748_sw_reset
[ 28.038566] tc358748 2-000e: tc358748_apply_misc_config
[ 28.038902] tc358748 2-000e: DATAFMT: 0x60
[ 28.039301] tc358748 2-000e: CONFCTL[PDATAF]: 0x0
[ 28.039642] tc358748 2-000e: FIFOCTL: 0x20
[ 28.039976] tc358748 2-000e: WORDCNT: 0xf00
[ 28.039984] tc358748 2-000e: tc358748_enable_csi_lanes: 1
[ 28.040454] tc358748 2-000e: CLW_CNTRL: 0 0x140
[ 28.040777] tc358748 2-000e: D0W_CNTRL: 0 0x144
[ 28.041158] tc358748 2-000e: D1W_CNTRL: 0 0x148
[ 28.041538] tc358748 2-000e: D2W_CNTRL: 0 0x14c
[ 28.041899] tc358748 2-000e: D3W_CNTRL: 0 0x150
[ 28.041908] tc358748 2-000e: tc358748_apply_dphy_config
[ 28.041912] tc358748 2-000e: dsiclk_nsk: 4494
[ 28.041916] tc358748 2-000e: ui_nsk: 2247
[ 28.041920] tc358748 2-000e: csibclk_nsk: 17977
[ 28.041924] tc358748 2-000e: phy_delay_nsk: 26965
[ 28.042278] tc358748 2-000e: LINEINITCNT: 0x15ba
[ 28.042661] tc358748 2-000e: LPTXTIMECNT: 0x2
[ 28.043043] tc358748 2-000e: TCLK_HEADERCNT: 0xa03
[ 28.043456] tc358748 2-000e: TCLK_TRAILCNT: 0xffffffff
[ 28.043803] tc358748 2-000e: THS_HEADERCNT: 0xffffee03
[ 28.044156] tc358748 2-000e: TWAKEUP: 0x49e0
[ 28.044535] tc358748 2-000e: TCLK_POSTCNT: 0x7
[ 28.044914] tc358748 2-000e: THS_TRAILCNT: 0x1
[ 28.045298] tc358748 2-000e: HSTXVREGEN: 0x1f
[ 28.045307] tc358748 2-000e: tc358748_enable_csi_module: 1
[ 28.045680] tc358748 2-000e: STARTCNTRL: 0x1
[ 28.046025] tc358748 2-000e: CSI_START: 0x1
[ 28.046407] tc358748 2-000e: CSI_CONFW_REG: 2734719110 (0x518)
[ 32.176687] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 32.176772] tc358748 2-000e: tc358748_get_pad_format code=0x2006, w/h=(1920,1080), colorspace=8, field=1
[ 32.178272] tc358748 2-000e: [tc358748_stream,1152]: Stream disable ...
[ 32.240654] enter tc358748_s_power = OFF
[ 32.240673] tc358748 2-000e: tc358748_disable_hw
[ 60.422080] imx-sdma 30bd0000.dma-controller: firmware found.
[ 60.422080] imx-sdma 30e10000.dma-controller: firmware found.
[ 60.422326] imx-sdma 30bd0000.dma-controller: loaded firmware 4.6
[ 66.531400] cfg80211: failed to load regulatory.db

[joanxie]

could you send the logfile and dts file to me

[shaswath]

Please find the attached files.

[joanxie]

from imx8mp side, I don't find any issue,  the connection set up correctly

[ 7.992257] mx8-img-md: Registered sensor subdevice: tc358748 2-000e (1)
[ 8.002921] mx8-img-md: created link [mxc_isi.0] => [mxc_isi.0.capture]
[ 8.018980] mx8-img-md: created link [mxc-mipi-csi2.0] => [mxc_isi.0]
[ 8.025519] mx8-img-md: created link [tc358748 2-000e] => [mxc-mipi-csi2.0]
[ 8.032634] mxc-md 32c00000.bus:camera: mxc_md_create_links

in fact isi0 is for mipi csi0, after you add port@1 in the mipi csi0, you should get some errors in the current bsp, so do you change any mipi driver? but since you set up correctly, this shouldn't affect your capture, you can check the FPGA side, to check if they set the correct clock or send the correct data, to double review the registers of TC358748, also pls dump the mipi csi and ISI registers to check any error bits

[shaswath]

Please find he register dump.

[joanxie]

for now but I am unable to capture any frames using Gstreamer or v4l2-ctl.

> what kind gstreamer command do you use and what message do you get? could you confirm if your signal from TC358748 is total fine? did you check TC358748 side？

[shaswath]

Hello @joanxie,

Yes the input signals to TC358748 is okay and I see signals coming out of TC358748.

To start capture using Gstreamer:

gst-launch-1.0 v4l2src device=/dev/video2 ! video/x-raw,width=1920,height=1080 ! autovideosink

Another issue is when the ISI node is enabled in the dts the system hangs after few minutes.