Hi,
I have 2 cameras connected to an imx8mp processor baseboard.
Connected cameras with i2c1 and i2c2 respectively. and it is connected to MIPI switch NX3DV642. then having a single CSI0 port to use based on GPIO selected.
From the DTS is it possible to enable both the camera node and build the final dtb if its connected to same node &mipi_csi0_ep?
Its giving error to me. Please help me how to do this.
Thanks,
Ritesh Kumar
已解决! 转到解答。
Hi @joanxie ,
Thanks for the help I am able to stream 3 camera in single boot.
I have added below change in mx8-media-dev.
@@ -989,10 +991,19 @@ static int register_sensor_entities(struct mxc_md *mxc_md)
if (!of_device_is_available(node))
continue;
- /* csi2 node have only port */
- port = of_get_next_child(node, NULL);
- if (!port)
- continue;
+ /* csi2 node have 2 port now for csi0 , Allow media dev driver to bind MIPIA
+ or MIPIB based on user camera sensor loaded, made change to get port@2 detail
+ from csi0 to create pipeline for both the camera modules MIPIA and MIPIB */
+ if(!strcmp(camera_media_link_param, "MIPIA&FF") && index == 0){
+ port1 = of_get_next_child(node, NULL);
+ port = of_get_next_child(node, port1);
+ if (!port)
+ continue;
+ }else if(!strcmp(camera_media_link_param, "MIPIB&FF") || index == 1) {
+ port = of_get_next_child(node, NULL);
+ if (!port)
+ continue;
+ }
/* port can have only endpoint */
ep = of_get_next_child(port, NULL);
@@ -1007,6 +1018,8 @@ static int register_sensor_entities(struct mxc_md *mxc_md)
return -EINVAL;
}
+ if (endpoint.base.port == 2)
+ endpoint.base.port = 0;
mxc_md->sensor[index].id = endpoint.base.port;
if (!of_node_cmp(node->name, MIPI_CSI2_OF_NODE_NAME))
@@ -1169,6 +1182,8 @@ static struct platform_driver mxc_md_driver = {
};
module_platform_driver(mxc_md_driver);
+module_param(camera_media_link_param, charp, 0);
+MODULE_PARM_DESC(camera_media_link_param, "MIPIA&FF");
MODULE_AUTHOR("Freescale Semiconductor, Inc.");
MODULE_DESCRIPTION("MXC Media Device driver");
and now at console user need to load and unload the modules of CSI0 and able to use based on their usecase.
Thanks. I am marking this case as fixed.
what error messages do you get and how did you set your dtb files?
>>> Error is like this
DTC arch/arm64/boot/dts/freescale/imx8mp-custom-board.dtb
arch/arm64/boot/dts/freescale/imx8mp-custom-board.dts:606.34-632.11: ERROR (duplicate_label): /soc@0/bus@30800000/i2c@30a40000/ov5640_mipi@3e: Duplicate label 'ov5640_0' on /soc@0/bus@30800000/i2c@30a40000/ov5640_mipi@3e and /soc@0/bus@30800000/i2c@30a30000/ov5640_mipi@3e
arch/arm64/boot/dts/freescale/imx8mp-custom-board.dts:626.52-630.27: ERROR (duplicate_label): /soc@0/bus@30800000/i2c@30a40000/ov5640_mipi@3e/port/endpoint: Duplicate label 'ov5640_mipi_0_ep' on /soc@0/bus@30800000/i2c@30a40000/ov5640_mipi@3e/port/endpoint and /soc@0/bus@30800000/i2c@30a30000/ov5640_mipi@3e/port/endpoint
ERROR: Input tree has errors, aborting (use -f to force output)
make[1]: *** [scripts/Makefile.lib:347: arch/arm64/boot/dts/freescale/imx8mp-custom-board.dtb] Error 2
make: *** [Makefile:1430: freescale/imx8mp-custom-board.dtb] Error 2
and the reason is I have 2 camera at different i2c bus but both used same CSI channel like this:-
&i2c2 {
ov5640_0: ov5640_mipi@3e {
compatible = "ovti,ov5640";
reg = <0x3e>;
- - - - - - - - - - -
port {
ov5640_mipi_0_ep: endpoint {
remote-endpoint = <&mipi_csi0_ep>;
data-lanes = <1 2>;
clock-lanes = <0>;
};
};
};
};
and the other cam is:-
&i2c3 {
ov5640_0: ov5640_mipi@3e {
compatible = "ovti,ov5640";
reg = <0x3e>;
- - - - - - - - - - -
port {
ov5640_mipi_0_ep: endpoint {
remote-endpoint = <&mipi_csi0_ep>;
data-lanes = <1 2>;
clock-lanes = <0>;
};
};
};
};
Both connected to same CSI like this
&mipi_csi_0 {
#address-cells = <1>;
#size-cells = <0>;
status = "okay";
port@0 {
reg = <0>;
mipi_csi0_ep: endpoint {
remote-endpoint = <&ov5640_mipi_0_ep>;
data-lanes = <2>;
csis-hs-settle = <13>;
csis-clk-settle = <2>;
csis-wclk;
};
};
};
So, How I can enable it to make it work I just toggle the MIPI select switch.
could you bring up one camera?
>>> I have tested both cameras working fine but only one at a time.
Thanks, Let me know if you need more info.
Ritesh Kumar
you couldn't define the same endpoint to different port, and how did you connect NX3DV642? where did you define this switch? did you use i2c to control NX3DV642? then NX3DV642 connect to the dual camera? if yes, you should define NX3DV642 in the dts, I don't know how you control dual camera by NX3DV642
Hi @joanxie,
I have tested this just configure the MIPI-select GPIO 1 or 0 to test the camera connected with i2c2 and the camera connected with i2c3. I am not getting how to add the detail of CSI0 in dts to add the ov5640 camera for both.
did you use i2c to control NX3DV642? then NX3DV642 connect to the dual camera? if yes, you should define NX3DV642 in the dts, I don't know how you control dual camera by NX3DV642.
>>> we don't have I2C control as this is not connected to I2C but this is used just to select CSI0 for ov5640 cam for i2c2 or i2c3 for data flow.
I hope Now it will be clear to you. I just want to use MIPIA(i2c2 cam) or MIPIB (i2c3 cam) enabled in dts to use the same CSI later in the driver I just load and unload the camera module to use MIPIA cam or MIPIB Camerra.
Also can you Just let me know if I have 2 camera which is connected to same CSI0 unit, How I can add the device tree detail for that. This will give my answer actually.
Thanks,
Ritesh Kumar
you need to change the dts file like this, for example
&mipi_csi_0 {
#address-cells = <1>;
#size-cells = <0>;
status = "okay";
port@0 {
reg = <0>;
mipi_csi0_ep: endpoint {
remote-endpoint = <&ov5640_ep_0>;
data-lanes = <2>;
csis-hs-settle = <13>;
csis-clk-settle = <2>;
csis-wclk;
};
};
port@1 {
reg = <1>;
mipi_csi1_ep: endpoint {
remote-endpoint = <&ov5640_ep_1>;
data-lanes = <2>;
csis-hs-settle = <13>;
csis-clk-settle = <2>;
csis-wclk;
};
};
};
then set one ov5640 endpoint as ov5640_ep_0, remote-endpoint as mipi_csi0_ep, another one as ov5640_ep_1 and mipi_csi1_ep, then they wouldn't be conflict, you couldn't set one mipi csi to different I2C camera as the same endpoint, hope my explanation is clear now
Hi @joanxie ,
I have tested this and figures out camera which is connected with CSI port@0 that only getting worked. The camera connected with port@1 not worked.
How to make it work. Faileld in csi binding.
Also If i am allowed to use only one at a time then I can't able to make it work with is connected via port@1. Please suggest something.
Thanks,
Ritesh
Hi @joanxie ,
Thanks for the quick response. I get it I will try this and will let you know if it is passed or not.
But in the final binding, I can't probe both drivers at the same time right as isi unit is 2 only in imx8mp and I have 3 cameras on board?
Thanks,
Ritesh Kumar
Hi @joanxie ,
Yes I have connected 3 Cameras.
2 with CSI0
1 with CSI1
2 cam conne connected with CSI0 which I want to switch the sensor module using insmod to use the same CSI0 at a time. Is this possible??
It looks completely doubtful to me as when I added port@0 and port@1 inside mipi_csi_0 dts node then I can see CSI driver only bind the 1st endpoint which is there. Then later on i checked the CSI driver there it look like the current design only adds one endpoint only.
static int mipi_csis_parse_dt(struct platform_device *pdev,
struct csi_state *state)
{
struct device_node *node = pdev->dev.of_node;
state->index = of_alias_get_id(node, "csi");
if (of_property_read_u32(node, "clock-frequency", &state->clk_frequency))
state->clk_frequency = DEFAULT_SCLK_CSIS_FREQ;
if (of_property_read_u32(node, "bus-width", &state->max_num_lanes))
return -EINVAL;
node = of_graph_get_next_endpoint(node, NULL);
if (!node) {
dev_err(&pdev->dev, "No port node\n");
return -EINVAL;
}
/* Get MIPI CSI-2 bus configration from the endpoint node. */
of_property_read_u32(node, "csis-hs-settle", &state->hs_settle);
of_property_read_u32(node, "csis-clk-settle", &state->clk_settle);
of_property_read_u32(node, "data-lanes", &state->num_lanes);
state->wclk_ext = of_property_read_bool(node, "csis-wclk");
of_node_put(node);
return 0;
}
Is it possible to add 2 endpoints in CSI0 so that we can load and unload the camera driver to use 2 cameras that are connected to the same CSI port?
Please let me know about this. This is very urgent to us.
Thanks,
Ritesh
I reproduce this on my board, yes, it seems the mipi csi only connect to the port0, that's why I ask you if you use I2C to connect the switch, if yes, mipi csi can only use one port, but now, maybe you need change the driver to switch the different port in mipi csi, otherwise I don't think this solution can work
Hi @joanxie ,
Thanks for testing at your end. I2c switch does not work in my case as all the camera modules are connected to different I2C buses.
I am seeing the only solution could be adding 2 endpoints after modifying the CSI core driver. If this is able to do then I will load/ unload a module to switch the camera which is connected to CSI0.
So, Can you please let us know if is it possible to add 2 endpoints in mipi csi driver? Can you please check once and let us know?
Thanks,
Ritesh Kumar
sorry for my delay, I need to build the source code and modify the driver to test for you, now, I can boot up the camera on the second port, so I think mipi csi can read the second port, you can find my result, the dts I modify as below, I only have os08a0 camera, so I attach this camera with port@1
1)dts file:
&mipi_csi_0 {
status = "okay";
port@0 {
reg = <0>;
mipi_ep: endpoint {
remote-endpoint = <&ov5640_mipi1_ep>;
data-lanes = <2>;
csis-hs-settle = <10>;
};
};
port@1 {
reg = <1>;
endpoint {
remote-endpoint = <&os08a20_mipi_0_ep>;
data-lanes = <4>;
csis-hs-settle = <16>;
};
};
2) then I add the mipi csi driver to search the second port
node1 = of_graph_get_next_endpoint(node, NULL);
node = of_graph_get_next_endpoint(node, node1);
3) you also need to change the imx8-media-dev.c
/* csi2 node have only port */
// port = of_get_next_child(node, NULL); //joan
port1 = of_get_next_child(node, NULL);//joan
port = of_get_next_child(node, port1);//joan
v4l2_info(&mxc_md->v4l2_dev,
"port name is %s\n", port->full_name);
I add print information, can find the name from logfile as below
4) don't forget change the "else if (mxc_md->mipi_csi2[sensor->id].sd) " to
"else if (mxc_md->mipi_csi2[0].sd) " and change "mipi_csi2 = &mxc_md->mipi_csi2[sensor->id];“ to
"mipi_csi2 = &mxc_md->mipi_csi2[0];"
and don't forget you need reload imx8-media-dev.ko again after you build this driver
the result you can find
[ 8.642136] mx8-img-md: created link [os08a20 1-0036] => [mxc-mipi-csi2.0]
[ 8.642148] mxc-md 32c00000.bus:camera: mxc_md_create_links
koot@imx8mp-lpddr4-evk:~# gst-launch-1.0 v4l2src device=/dev/video ! video/x-raw,width=640,height=480 ! waylandsinkot@imx8mp-lpddr4-evk:~# gst-launch-1.0 v4l2src device=/dev/video2 ! video/x-raw,width=640,height=480 ! waylandsinkt@imx8mp-lpddr4-evk:~# gst-launch-1.0 v4l2src device=/dev/video2
[ 34.424103] enter isp_mi_stop
[ 34.580144] enter isp_mi_stop
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[ 34.962138] enter isp_mi_stop
[ 35.152477] enter isp_s_comp
[ 35.155396] enter isp_s_comp
[ 35.158280] enter isp_s_comp
[ 35.175593] enter wdr3_hw_init
[ 35.178660] wdr3 res: 1920 1080
[ 35.201481] enter isp_set_stream 1
[ 35.216139] enter isp_mi_start
Redistribute latency...
0:00:00.5 / 99:99:99.
0:00:00.6 / 99:99:99.
0:00:00.7 / 99:99:99.
0:00:00.8 / 99:99:99.
hope these are helpful for your customer
Hi @joanxie ,
I have tried this patch but not working. Can you check what else I am missing in this???
ritesh@dell:~/kernel-source$ git diff drivers/staging/media/imx/imx8-media-dev.c drivers/staging/media/imx/imx8-mipi-csi2-sam.c
diff --git a/drivers/staging/media/imx/imx8-media-dev.c b/drivers/staging/media/imx/imx8-media-dev.c
index 0d0355844eab..bb3297e71e75 100644
--- a/drivers/staging/media/imx/imx8-media-dev.c
+++ b/drivers/staging/media/imx/imx8-media-dev.c
@@ -514,8 +514,8 @@ static int mxc_md_create_links(struct mxc_md *mxc_md)
v4l2_info(&mxc_md->v4l2_dev,
"created link [%s] => [%s]\n",
source->name, sink->name);
- } else if (mxc_md->mipi_csi2[sensor->id].sd) {
- mipi_csi2 = &mxc_md->mipi_csi2[sensor->id];
+ } else if (mxc_md->mipi_csi2[0].sd) {
+ mipi_csi2 = &mxc_md->mipi_csi2[0];
source = &sensor->sd->entity;
sink = find_entity_by_name(mxc_md, mipi_csi2->sd_name);
@@ -970,6 +970,7 @@ static int register_sensor_entities(struct mxc_md *mxc_md)
/* Attach sensors linked to MIPI CSI2 / paralle csi / HDMI Rx */
for_each_available_child_of_node(parent, node) {
struct device_node *port;
+ struct device_node *port1;
if (!of_node_cmp(node->name, HDMI_RX_OF_NODE_NAME)) {
mxc_md->sensor[index].fwnode = of_fwnode_handle(node);
@@ -990,7 +991,8 @@ static int register_sensor_entities(struct mxc_md *mxc_md)
continue;
/* csi2 node have only port */
- port = of_get_next_child(node, NULL);
+ port1 = of_get_next_child(node, NULL);
+ port = of_get_next_child(node, port1);
if (!port)
continue;
diff --git a/drivers/staging/media/imx/imx8-mipi-csi2-sam.c b/drivers/staging/media/imx/imx8-mipi-csi2-sam.c
index 1d12365520a6..2d318a4d96ca 100644
--- a/drivers/staging/media/imx/imx8-mipi-csi2-sam.c
+++ b/drivers/staging/media/imx/imx8-mipi-csi2-sam.c
@@ -1486,6 +1486,7 @@ static int mipi_csis_parse_dt(struct platform_device *pdev,
struct csi_state *state)
{
struct device_node *node = pdev->dev.of_node;
+ struct device_node *node1;
state->index = of_alias_get_id(node, "csi");
@@ -1495,7 +1496,8 @@ static int mipi_csis_parse_dt(struct platform_device *pdev,
if (of_property_read_u32(node, "bus-width", &state->max_num_lanes))
return -EINVAL;
- node = of_graph_get_next_endpoint(node, NULL);
+ node1 = of_graph_get_next_endpoint(node, NULL);
+ node = of_graph_get_next_endpoint(node, node1);
if (!node) {
dev_err(&pdev->dev, "No port node\n");
return -EINVAL;
Hi @joanxie ,
Yes, I have updated the imx8-media-dev.ko module driver.
I give more time and dig into the driver I think you added these 2 ports for mipi_csi_0 but you have missed out mipi_csi_1.
Is the camera connected with mipi_csi_1 will work ??
Also I got these errors with your suggested Change.
root@spencer-gen2:~# dmesg | grep csi
[ 1.559969] i2c 1-003e: Fixing up cyclic dependency with 32e40000.csi
[ 1.624601] i2c 2-003e: Fixing up cyclic dependency with 32e40000.csi
[ 1.643013] i2c 3-003e: Fixing up cyclic dependency with 32e50000.csi
[ 2.101757] mxc-mipi-csi2-sam 32e40000.csi: supply mipi-phy not found, using dummy regulator
[ 2.110515] : mipi_csis_imx8mp_phy_reset, No remote pad found!
[ 2.121180] mxc-mipi-csi2-sam 32e40000.csi: lanes: 2, hs_settle: 13, clk_settle: 2, wclk: 1, freq: 500000000
[ 2.132013] mxc-mipi-csi2-sam 32e50000.csi: Unsupported number of data lanes: 0 (max. 4)
[ 2.140139] mxc-mipi-csi2-sam: probe of 32e50000.csi failed with error -22
[ 6.070940] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 6.140568] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 6.268545] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 6.318696] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 6.360637] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 6.399080] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 7.669472] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 7.729950] mxc-md 32c00000.bus:camera: deferring csi device registration
root@spencer-gen2:~# dmesg | grep mxc-mipi-csi2-sam
[ 2.101757] mxc-mipi-csi2-sam 32e40000.csi: supply mipi-phy not found, using dummy regulator
[ 2.121180] mxc-mipi-csi2-sam 32e40000.csi: lanes: 2, hs_settle: 13, clk_settle: 2, wclk: 1, freq: 500000000
[ 2.132013] mxc-mipi-csi2-sam 32e50000.csi: Unsupported number of data lanes: 0 (max. 4)
[ 2.140139] mxc-mipi-csi2-sam: probe of 32e50000.csi failed with error -22
root@spencer-gen2:~# [ 33.892755] dspin_motor1_reset: disabling
[ 33.896791] dspin_motor2_reset: disabling
[ 33.900822] dspin_motor3_reset: disabling
dmesg | grep mxc-md
[ 6.070940] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 6.140568] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 6.268545] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 6.318696] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 6.360637] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 6.399080] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 7.669472] mxc-md 32c00000.bus:camera: deferring csi device registration
[ 7.729950] mxc-md 32c00000.bus:camera: deferring csi device registration
root@spencer-gen2:~# dmesg | grep mx8-img-md
[ 6.044868] mx8-img-md: Registered mxc_isi.0.capture as /dev/video3
[ 6.057471] mx8-img-md: Registered mxc_isi.1.capture as /dev/video4
[ 6.122818] mx8-img-md: Registered mxc_isi.0.capture as /dev/video3
[ 6.129391] mx8-img-md: Registered mxc_isi.1.capture as /dev/video4
[ 6.250874] mx8-img-md: Registered mxc_isi.0.capture as /dev/video3
[ 6.258037] mx8-img-md: Registered mxc_isi.1.capture as /dev/video4
[ 6.297644] mx8-img-md: Registered mxc_isi.0.capture as /dev/video3
[ 6.304742] mx8-img-md: Registered mxc_isi.1.capture as /dev/video4
[ 6.329545] mx8-img-md: Registered mxc_isi.0.capture as /dev/video3
[ 6.352196] mx8-img-md: Registered mxc_isi.1.capture as /dev/video4
[ 6.383578] mx8-img-md: Registered mxc_isi.0.capture as /dev/video3
[ 6.392671] mx8-img-md: Registered mxc_isi.1.capture as /dev/video4
[ 7.648145] mx8-img-md: Registered mxc_isi.0.capture as /dev/video3
[ 7.657907] mx8-img-md: Registered mxc_isi.1.capture as /dev/video4
[ 7.712486] mx8-img-md: Registered mxc_isi.0.capture as /dev/video3
[ 7.723069] mx8-img-md: Registered mxc_isi.1.capture as /dev/video4
This is the problem I observed and I can see the mipi_csi_1 failed ie. not our goal. we want 1 camera from mipi_csi_0 and 1 camera from mipi_csi_1 should work at a time and there must be the option of switching the camera of mipi_csi_0.(whether we need to unload or load the sensor module and imx8-media-dev driver).
[ 2.140139] mxc-mipi-csi2-sam: probe of 32e50000.csi failed with error -22
try to change the port@1 under mipi csi0 to port@2 {reg = <2>;....
add" if (endpoint.base.port ==2)
endpoint.base.port=0; " before mxc_md->sensor[index].id = endpoint.base.port; in the function register_sensor_entities
customer can change "else if (mxc_md->mipi_csi2[0].sd) and mipi_csi2 = &mxc_md->mipi_csi2[0];" back to "else if (mxc_md->mipi_csi2[sensor->id].sd) and mipi_csi2 = &mxc_md->mipi_csi2[sensor->id];", then customer can use mipi csi1, for different port under mipi csi0, you need add some code to decide which port you need to use
Hi @joanxie ,
Thanks for the help I am able to stream 3 camera in single boot.
I have added below change in mx8-media-dev.
@@ -989,10 +991,19 @@ static int register_sensor_entities(struct mxc_md *mxc_md)
if (!of_device_is_available(node))
continue;
- /* csi2 node have only port */
- port = of_get_next_child(node, NULL);
- if (!port)
- continue;
+ /* csi2 node have 2 port now for csi0 , Allow media dev driver to bind MIPIA
+ or MIPIB based on user camera sensor loaded, made change to get port@2 detail
+ from csi0 to create pipeline for both the camera modules MIPIA and MIPIB */
+ if(!strcmp(camera_media_link_param, "MIPIA&FF") && index == 0){
+ port1 = of_get_next_child(node, NULL);
+ port = of_get_next_child(node, port1);
+ if (!port)
+ continue;
+ }else if(!strcmp(camera_media_link_param, "MIPIB&FF") || index == 1) {
+ port = of_get_next_child(node, NULL);
+ if (!port)
+ continue;
+ }
/* port can have only endpoint */
ep = of_get_next_child(port, NULL);
@@ -1007,6 +1018,8 @@ static int register_sensor_entities(struct mxc_md *mxc_md)
return -EINVAL;
}
+ if (endpoint.base.port == 2)
+ endpoint.base.port = 0;
mxc_md->sensor[index].id = endpoint.base.port;
if (!of_node_cmp(node->name, MIPI_CSI2_OF_NODE_NAME))
@@ -1169,6 +1182,8 @@ static struct platform_driver mxc_md_driver = {
};
module_platform_driver(mxc_md_driver);
+module_param(camera_media_link_param, charp, 0);
+MODULE_PARM_DESC(camera_media_link_param, "MIPIA&FF");
MODULE_AUTHOR("Freescale Semiconductor, Inc.");
MODULE_DESCRIPTION("MXC Media Device driver");
and now at console user need to load and unload the modules of CSI0 and able to use based on their usecase.
Thanks. I am marking this case as fixed.