TC358743 on i.MX8MM

[dpog]

I'm trying to use the TC358743 HDMI to CSI bridge on our custom board. I'm using the 5.15 BSP and the drivers imx7-mipi-csis and imx7-media-csi from the staging area. (Using the default drivers is of no use, as it fails to create the /dev/media0 device)

When I try to use the /dev/video0 with ffmpeg (ffmpeg -f video4linux2 -i /dev/video0 -vframes 1 -video_size 640x480 test.jpeg), I get an "Inappropriate ioctl for device" error, and the same error when using v4l2-ctl as mentioned here: https://forums.raspberrypi.com/viewtopic.php?t=281972

The output from media-ctl -p shows the following:

media-ctl -p
Media controller API version 5.15.87

Media device information
------------------------
driver imx7-csi
model imx-media
serial
bus info platform:32e20000.csi1_bridge
hw revision 0x0
driver version 5.15.87

Device topology
- entity 1: csi (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:UYVY8_2X8/800x600 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
<- "csis-32e30000.mipi_csi":1 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:UYVY8_2X8/800x600 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
-> "csi capture":0 [ENABLED,IMMUTABLE]

- entity 4: csi capture (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "csi":1 [ENABLED,IMMUTABLE]

- entity 10: csis-32e30000.mipi_csi (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
<- "tc358743 2-000f":0 []
pad1: Source
[fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
-> "csi":0 [ENABLED,IMMUTABLE]

- entity 15: tc358743 2-000f (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:RGB888_1X24/640x480 field:none colorspace:srgb]
[dv.caps:BT.656/1120 min:640x350@13000000 max:1920x1200@165000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
[dv.query:no-link]
[dv.current:BT.656/1120 640x480p59 (800x525) stds:CEA-861,DMT flags:has-cea861-vic]
-> "csis-32e30000.mipi_csi":0 []

Any idea what I am missing?

If needed, I will gladly provide more informations.

[joanxie]

pls share the logfile and the result with command "v4l2-ctl --list-device"

[dpog]

Hi @joanxie,

Did you mean the bootlog/dmesg? I attached it for that case.

The output from v4l2-ctl --list-device:

imx-capture (platform:32e20000.csi1_bridge):
    /dev/video0
    /dev/media0

[joanxie]

could you use gstreamera successfully? I found 5.15 does has v4l2 ioctl issue, the internal team is still working on this, do you mind testing gstreamer to check if you can capture anything from the TC358743 successfully or not, to confirm if your issue is related the bsp bug or your own code issue

[dpog]

I tried gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, width=800, height=600 ! filesink location=vid.raw;
(when using 800x600 the ioctl error disappears)
I then get the error "pipeline start failed with -67", because the link between the mipi_csi and tc358753 isn't active.
When activating it with media-ctl -d /dev/media0 --links "'tc358743 2-000f':0 -> 'csis-32e30000.mipi_csi':0 [1]", the error changes to "pipeline start failed with -32"

[joanxie]

do you mind sharing more detailed registers and logfile? it's hard to find root cause based on your limited information, and by the way you can check the clock mode according to the mipi csi status registers, you also can refer to this link 

Solved: IMX8MP and TC358743 - NXP Community

[dpog]

I'm fairly new to CSI on the iMX8MM, can you please tell me which registers you want informations from? Also, which logfile do you mean?

[joanxie]

logfile is the print information from your uart when you boot up the board, for registers, you can refer to the reference manual, find the mipi csi and csi registers address, then dump them out

[dpog]

I attached a more detailed logfile and the register dump via devmem.

I got the pipeline running by adding a small patch to the tc358743 driver, providing V4L2_CID_PIXEL_RATE control and linking the pads as well as setting the format with

fmt=UYVY8_1X16/640x480
media-ctl --set-v4l2 "'tc358743 2-000f':0[fmt:$fmt field:none]"
media-ctl --set-v4l2 "'csis-32e30000.mipi_csi':0[fmt:$fmt field:none]"
media-ctl --set-v4l2 "'csi':0[fmt:$fmt field:none]"

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, width=640, height=480 ! filesink location=vid.raw
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
[ 67.964628] imx7-csi 32e20000.csi1_bridge: capture_validate_fmt
New clock: GstSystemClock
[ 67.975860] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
[ 67.985862] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver

the resulting vid.raw file stays empty, even thou gstreamer reports the pipeline PLAYING

[dpog]

@joanxie do you have any more input on this topic?

[joanxie]

I'm not sure of your testing gstreamer command is correct or not, I don't have board now, do you have connect any display with your board? try to use waylandsink instead of filesink to check it, and also dump the CSI_CR18 registers to check by yourself according the reference manual

[dpog]

When I use waylandsink, I get:
Setting pipeline to PAUSED ...
Failed to set pipeline to PAUSED.
Setting pipeline to NULL ...
Freeing pipeline ...

and no output.
CSI_CR18 reads 0x3c42d000, so CSI seems to be disabled. When enabling it (devmem2 0x32e20048 w bc42d000), I still see the same behaviour, for filesink (empty file) and waylandsink (failed to pause)

[joanxie]

I found you don't use the nxp csi driver for your imx8mm, since you use your own driver, pls refer to the nxp source code, refer to your register, it seems that your driver is different from nxp source code, this bsp is used by the current evk board

https://github.com/nxp-imx/linux-imx/blob/lf-6.6.y/drivers/media/platform/mxc/capture/mx6s_capture.c

[dpog]

Hi @joanxie,
I switched to the scarthgap release and tried with linux-imx and linux-fslc-imx, with the mx6s_capture driver. The results stay the same, but now I can't use media-ctl anymore, as it is not shipped in the image and no /dev/media0 is created

[joanxie]

I don't know how you change and why you get such result, you can refer to the link I already shared with you before, if you use the third party company SW before, you need contact them to check what they changed based on NXP bsp

https://community.nxp.com/t5/i-MX-Processors/IMX8MP-and-TC358743/m-p/1737433

[dpog]

I checked again, the driver in use is now mxc_mipi_csi (compatible = imx8mm-mipi-csi)
I suspect that the pixelformats might not be compatible, mxc driver only supports YUYV, VYUY and SBGGR, while the tc358743 only supports UYVY and RGB888.

[joanxie]

obviously there are some problems with your driver, if you need add new format, you need add in the mx6s_fmt formats of mx6s_capture.c and mipi_csis_formats of mxc_mipi_csi.c

[dpog]

As an addition: i tried to read the CSI status register, but every time I try to read from 0x32e20048, the board hangs