Integrating iMX8M with SONY FCB Camera

[Manikandan]

Hi,

We have developed a custom iMX8M based board to integrate SONY FCB-EV9500M MIPI camera. we want to receive the video data from SONY FCB-EV9500M MIPI camera so how can i integrate SONY FCB-EV9500M MIPI camera with custom iMX8M based board kindly help us.

here i have attached our camera module datasheet please find it for the reference

[volers]

FCB-EW9500H is SONY’s new generation of 400W high-definition digital integrated camera movement, using 1/1.8 Type STARVIS CMOS Sensor, to achieve 30x enhanced optical zoom, with high definition, high sensitivity, high dynamic wide dynamic and super image stabilization and other functions.

The new FCB-9500 series includes three models: FCB-EV9500M, FCB-EW9500H, FCB-EV9500M.

Sony FCB-EW9500H(FCBEW9500H) 4M-HDMI 30x Optical Zoom Camera Block - Sony 4K UHD camera (sony-hd.com...

[VLopez]

I apologise for bringing back an old post.

I have purchased a FCB-EV9500M for a Jetson TX2 NX, thinking the camera's advertised MIPI interface would make the connection to the Jetson's CSI-2 bridge easy.

I have not been able to find an evaluation kit or cable to connect this camera, how did you manage to connect the camera to the Jetson? Did you have a custom cable or PCB built? Is there something obvious that I am missing?

[khang_letruong]

Hi @VLopez ,

For my case, I used a custom Jetson Xavier /TX2 NX based board which disposes 30-pin KEL connector.

Regards,

K.

[VLopez]

Thanks a lot for the information.

I guess this carrier board is not available for purchase anywhere, right?

We may have to look into getting one custom made as well.

[Manikandan]

Hi @VLopez 

We made a custom carrier board for EV9500M Camera. it is available with us, if you need please let us know

[benarji-buddha08]

I need this carrier board to use  EV9500M Camera.

My number is +91-9474651407.

Please call me or ping me. Will discuss and proceed further

[bpatil]

Hi @Manikandan 

Do you have adapter board for FCB-EV9500M  to XAVIER? Please send details on bpatil@phoenix.tech

[VLopez]

Hi @Manikandan ,

We might be interested, could we continue the conversation by mail? Please write me at victor.lopez@keybotic.com

@khang_letruongfeel free to write me too.

Thanks for the information

[Manikandan]

Hi @VLopez 

Please feel free to contact us by info@oppila.in

Thanks

[Manikandan]

Hi,

Thanks for your reply.  As you told, in our camera we have tc358743 bridge chipset to convert HDMI signals to MIPI-CSI2 signals. and a driver for tc358743 is also available in our imx platform OS  but the driver is completely based on I2C.  and in our camera we don't have I2C, in that case how can we use this I2C based tc358743 driver to get the video frames. please advise me how to approach this driver.

As you mentioned in your reply,

"The good thing is that you will not need to control it via I2C as there's an MCU that handle this communication with the bridge which will simplify your work alot."

if so, how to populate /dev/video*?

[khang_letruong]

Hi @Manikandan ,

You can inspire from below driver but remove any I2C access via i2c_rd8, i2c_rd16, i2c_rd32, i2c_wr8, i2c_wr16, i2c_wr32 or i2c_transfer ... :

https://github.com/boundarydevices/linux/blob/boundary-imx_5.4.x_2.3.0/drivers/media/i2c/tc358743.c

https://github.com/boundarydevices/linux/blob/boundary-imx_5.4.x_2.3.0/arch/arm64/boot/dts/freescale...

The most important thing to populate /dev/video* in the tc358743_probe() function at following lines :

	sd = &state->sd;
	v4l2_i2c_subdev_init(sd, client, &tc358743_ops);
	sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;

and

	sd->dev = &client->dev;
	err = v4l2_async_register_subdev(sd);

More resource :

https://developer.technexion.com/docs/vizionlink-hdmi-use-guide

https://developer.technexion.com/docs/mipi-csi-periph-wb-edm-g

Regards,

K

[Udhay_]

@khang_letruong 

can you please share the details of your codes to "umoorthi@phoenix.tech"?

did you get the data from mipi after removing i2c related to things ?

which board you have used ?

[Manikandan]

hi @khang_letruong 

Thanks for the help. As you suggested i have approached in the same way but instead of using    tc358743 driver i have used ov5645 driver  (because both are similar and going to give us mipi data and in both cases we are not going to use i2c).    

1. Removed i2c related stuff from the driver

    Debug done, as you told  the below  functions passed

v4l2_i2c_subdev_init(sd, client, &tc358743_ops);

v4l2_async_register_subdev(sd);

2. Linked  with DTS file kept only one entry 

3. tried getting frames by using

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=10 --stream-to=aaa.raw

but not getting anything. i'm not getting any error even to proceed

but if i give the same command with different pixelformat (NV12) or resolution ( 1280 x720 )  i'm getting only one frame and stucked there only.

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=NV12 --stream-mmap --stream-count=10 --stream-to=aaa.raw

can you help me how can i proceed further

Here i'm attaching my modified driver 

[khang_letruong]

HI @Manikandan ,

You can :

1. Use an oscilloscope to measure the waveform of MIPI-CSI2 clock lane and data lanes to make sure that there's output from the FCB camera (the tc358743 more precisely).

2. You can enable the MIPI-CSI2 registers' dumping to see if there's any events (including errors such as FIFO overflow, etc).

Regards,

K.

[Manikandan]

Hi @khang_letruong

1). As you suggested we have tested it already with oscilloscope to measure the waveform of MIPI CSI2 and we got the output from the FCB camera

2). Can you suggest me any procedure to enable the MIPI-CSI2 registers dumping

3). I Got the Sony camera driver(EV9500M) from the vendor. but actually the driver is for nVidia platform. in that DTS file they have provided many properties (please check for the attached DTS file for the reference) but here we are providing the below properties only

compatible = "ovti,ov5645_mipi";

reg = <0x3c>;

status = "okay";

pinctrl-names = "default";

pinctrl-0 = <&pinctrl_csi2>;

clocks = <&clk IMX8MQ_CLK_DUMMY>;

clock-names = "csi_mclk"; csi_id = <0>;

pwn-gpios = <&gpio3 8 1>; rst-gpios = <&gpio1 12 0>;

mclk = <20000000>;

mclk_source = <0>;

ae_target = <52>;

Do i need to provide all those properties which is given in the nVidia DTS file ?

[khang_letruong]

Hi @Manikandan ,

You can take a look at the following function in <path-to-linux-imx>/drivers/staging/media/imx/imx8-mipi-csi2-sam.c :

static void dump_csis_regs(struct csi_state *state, const char *label)
{
        struct {
                u32 offset;
                const char * const name;
        } registers[] = {
                { 0x00, "CSIS_VERSION" },
                { 0x04, "CSIS_CMN_CTRL" },
                { 0x08, "CSIS_CLK_CTRL" },
                { 0x10, "CSIS_INTMSK" },
                { 0x14, "CSIS_INTSRC" },
                { 0x20, "CSIS_DPHYSTATUS" },
                { 0x24, "CSIS_DPHYCTRL" },
                { 0x30, "CSIS_DPHYBCTRL_L" },
                { 0x34, "CSIS_DPHYBCTRL_H" },
                { 0x38, "CSIS_DPHYSCTRL_L" },
                { 0x3C, "CSIS_DPHYSCTRL_H" },
                { 0x40, "CSIS_ISPCONFIG_CH0" },
                { 0x50, "CSIS_ISPCONFIG_CH1" },
                { 0x60, "CSIS_ISPCONFIG_CH2" },
                { 0x70, "CSIS_ISPCONFIG_CH3" },
                { 0x44, "CSIS_ISPRESOL_CH0" },
                { 0x54, "CSIS_ISPRESOL_CH1" },
                { 0x64, "CSIS_ISPRESOL_CH2" },
                { 0x74, "CSIS_ISPRESOL_CH3" },
                { 0x48, "CSIS_ISPSYNC_CH0" },
                { 0x58, "CSIS_ISPSYNC_CH1" },
                { 0x68, "CSIS_ISPSYNC_CH2" },
                { 0x78, "CSIS_ISPSYNC_CH3" },
        };
        u32 i;

        v4l2_dbg(2, debug, &state->sd, "--- %s ---\n", label);

        for (i = 0; i < ARRAY_SIZE(registers); i++) {
                u32 cfg = mipi_csis_read(state, registers[i].offset);
                v4l2_dbg(2, debug, &state->sd, "%20s[%x]: 0x%.8x\n", registers[i].name, registers[i].offset, cfg);
        }
}

To enable above code snippet, you can either apply following patch and rebuild the kernel :

$ git diff drivers/staging/media/imx/imx8-mipi-csi2-sam.c
diff --git a/drivers/staging/media/imx/imx8-mipi-csi2-sam.c b/drivers/staging/media/imx/imx8-mipi-csi2-sam.c
index 928c64d3be49..791d538c362d 100644
--- a/drivers/staging/media/imx/imx8-mipi-csi2-sam.c
+++ b/drivers/staging/media/imx/imx8-mipi-csi2-sam.c
@@ -450,7 +450,7 @@ struct csi_state {
        bool hdr;
 };
 
-static int debug;
+static int debug=2;
 module_param(debug, int, 0644);
 MODULE_PARM_DESC(debug, "Debug level (0-2)");

or add following command in u-Boot bootargs : mxc-mipi-csi2-sam.debug=2

Regards,

Khang

[Manikandan]

Hi @khang_letruong 

Actually i'm using Mendel linux OS there the exact file is not available. once if i start yocto build then ill try this way of debugging

can you please answer me the below questions :

1. I Got the Sony camera driver(EV9500M) from the vendor. but actually the driver is for nVidia platform. in that DTS file they have provided many properties (please check for the attached DTS file for the reference) but here we are providing the below properties only

compatible = "ovti,ov5645_mipi";

reg = <0x3c>;

status = "okay";

pinctrl-names = "default";

pinctrl-0 = <&pinctrl_csi2>;

clocks = <&clk IMX8MQ_CLK_DUMMY>;

clock-names = "csi_mclk"; csi_id = <0>;

pwn-gpios = <&gpio3 8 1>; rst-gpios = <&gpio1 12 0>;

mclk = <20000000>;

mclk_source = <0>;

ae_target = <52>;

Do i need to provide all those properties which is given in the nVidia DTS file ?

2.) I have followed the procedure which you have suggested the only difference is instead of TC3587 i have used OV5645 Driver, will it be a problem. why i didn't get any frames. why it got stucked. can you help me out to proceed with this procedure

Thank you Mr.K

[Udhay_]

@Manikandan 

I can't able to get frames also and tell me how can i registered the device using UART?

[khang_letruong]

Hi @Manikandan ,

Do i need to provide all those properties which is given in the nVidia DTS file ?

These properties are platform-dependent, i.e. you only need ones that will be parsed and used by other elements in the pipelines (CSI->ISI for NXP SoC or CSI->VI for Nvidia Jetson). You can grep them in the source of the BSP to see what is used if you are not sure.

I have followed the procedure which you have suggested the only difference is instead of TC3587 i have used OV5645 Driver, will it be a problem. why i didn't get any frames. why it got stucked. can you help me out to proceed with this procedure

Enabling the registers dumping/debugging would provide more info about the issue.

Best regards,

K.

[khang_letruong]

Hi @Manikandan,

Your Sony FCB-EV9500M camera has a Toshiba tc358743 bridge chipset to convert HDMI signals to MIPI-CSI2 signals. Then you will need a driver for that bridge. The good thing is that you will not need to control it via I2C as there's an MCU that handle this communication with the bridge which will simplify your work alot. You will only need to register the driver so that /dev/videoX is populated and read the video data via that /dev/videoX. 

From my experience with this FCB camera, following is the driver of tc358743 for Nvidia Jetson for your reference : https://gist.github.com/olesia-kochergina/c2af863c250c748c3c58dbb7acfe84bf

Regards,
Khang