how ov5640 camera works on imx8mp-evk

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

how ov5640 camera works on imx8mp-evk

Jump to solution
2,751 Views
zhangjunsheng
Contributor III

Hi,

     I encounter failure when I try to use ov5640 camera on imx8mp-evk board and need guide and help.

     I confirm ov5640 camera module (bought together with evk board) works on imx8mm-evk board with command: gst-launch-1.0 v4l2src device=/dev/video0 ! autovideosink with image: LF_v5.15.32-2.0.0_images_IMX8MMEVK.zip.

     I only have one camera: ov5640 on hand, so I connect the camera to imx8mp-evk board J12 (CSI1 MIPI). imx8mp-evk board is downloaded with image: LF_v5.15.32-2.0.0_images_IMX8MPEVK.zip.

     System detected the camera, and v4l2 devices looks ok:

$ v4l2-ctl --list-devices
():
/dev/v4l-subdev0

mxc-isi-cap (platform:32e00000.isi:cap_devic):
/dev/video3

mxc-isi-m2m (platform:32e00000.isi:m2m_devic):
/dev/video2

FSL Capture Media Device (platform:mxc-md):
/dev/media0

vsi_v4l2dec (platform:vsi_v4l2dec):
/dev/video1

vsi_v4l2enc (platform:vsi_v4l2enc):
/dev/video0

 

 we can see the capture device presented is /dev/video3 (ov5640 using ISI, it's correct)

 I try to using different tools to use ov5640 camera, looks nothing error but hang here, so I think there is no problem for start_stream IOCTL, but there is no data return to user space, so user space tool waiting there for data.

$ gst-launch-1.0 -e v4l2src device=/dev/video3 ! video/x-raw ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[ 1802.093915] bypass csc
[ 1802.096289] input fmt YUV4
[ 1802.099019] output fmt YUYV

 

$ /unit_tests/V4L2/mx8_v4l2_cap_drm.out -cam 8 -fmt NV12 -ow 1920 -oh 1080 -num 1 -of
init channel[3] save_file_name=3.NV12
init channel[3] v4l2_dev_name=/dev/video3 w/h=(1280,800)
open 3.NV12 success
open /dev/video3 success
planes=2 WxH@fps = 1920x1080@30
bypass csc
input fmt YUV4
output fmt NV12
channel[3] v4l_dev=0x4 start capturing

$ ls -l
-rw-r----- 1 root root     0 Oct  9 20:03 3.NV12

 

$ v4l2-ctl -d /dev/video3 --set-fmt-video=width=1280,height=720,pixelformat='NV12' --stream-mmap --stream-count=5
[  981.256852] bypass csc
[  981.259229] input fmt YUV4
[  981.261954] output fmt NV12

 

 When preview with gst-launch-1.0, did not see any capturing on HDMI. when save camera data with mx8_v4l2_cap_drm.out, the size of saved file is always 0. when getting camera data with v4l2-ctl, no data presented (if there is data, could see something like <<<<<<<<<<<<<<<<<<<<<<<<<< from console).

 I also try to connect the camera to CSI2 and using dtb file: imx8mp-evk-basler-ov5640.dtb. in that case, the ov5640 capturing device is /dev/video2. I also try above three tools and the result is the same.

  Could you help to provide the comment why it happen and how I can make ov5640 camera work on imx8mp-evk board?

 

   Thanks.

Johnson

 

 

 

 

 

Tags (2)
0 Kudos
1 Solution
2,623 Views
joanxie
NXP TechSupport
NXP TechSupport

did you use nxp IMX8MP board and prebuilt images, right? it seems the board detect ov5640 already, and I tested this on my board successfully, I couldn't reproduce this on my side. did you have other camera module to test J13 port?

View solution in original post

0 Kudos
13 Replies
2,277 Views
Suraj_Vagoli
Contributor I

@zhangjunsheng @joanxie We are also facing the below issue on our custom board imx8mp

 

 we can see the capture device presented is /dev/video2 (ov5640 using ISI, it's correct)

 we tried gstreamer tool to use ov5640 camera, looks nothing error but hang here, so I think there is no problem for start_stream IOCTL, but there is no data return to user space, so user space tool waiting there for data.

 

$ gst-launch-1.0 -e v4l2src device=/dev/video2 ! video/x-raw ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[ 1802.093915] bypass csc
[ 1802.096289] input fmt YUV4
[ 1802.099019] output fmt YUYV

 

 

0 Kudos
1,578 Views
araja
Contributor III
Hi Suraj,
I am also facing the same issue with iMX8MP EVK.
Did you find any solution to come out of this hang condition of gstreamer?
Thanks in advance.
0 Kudos
1,568 Views
Suraj_Vagoli
Contributor I

Hi @araja 

In our case we were facing this issue on the custom board which was based on imx8mp. It was pipeline issue and after fixing this we were able to get it working. 

 

We had tested on EVK board and ov5640 sensor was working fine with imx8mp-evk.dtb.

Could you please share your setup and command details? when the issue occurs is there any data available on the data lines and is the clk is proper?

 

I hope the camera is connected on CSI1 port of the EVK board. 

 

 

0 Kudos
1,565 Views
araja
Contributor III

Thanks for the response,

Command details,

gst-launch-1.0 -e v4l2src device=/dev/video2 ! video/x-raw ! autovideosink

Setup, 

IMX8MP EVK board with NXP ov5640 camera. on CSI0 port.

DTB -> imx8mp-evk.dtb

DTS entry,

ov5640_0: ov5640_mipi@3c {
compatible = "ovti,ov5640";
reg = <0x3c>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_csi0_pwn>, <&pinctrl_csi0_rst>, <&pinctrl_csi_mclk>;
clocks = <&clk IMX8MP_CLK_IPP_DO_CLKO2>;
clock-names = "xclk";
assigned-clocks = <&clk IMX8MP_CLK_IPP_DO_CLKO2>;
assigned-clock-parents = <&clk IMX8MP_CLK_24M>;
assigned-clock-rates = <24000000>;
csi_id = <0>;
powerdown-gpios = <&gpio2 11 GPIO_ACTIVE_HIGH>;
reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
mclk = <24000000>;
mclk_source = <0>;
mipi_csi;
status = "okay";

port {
ov5640_mipi_0_ep: endpoint {
remote-endpoint = <&mipi_csi0_ep>;
data-lanes = <1 2 3 4>;
clock-lanes = <0>;
};
};
};

 

&mipi_csi_0 {
#address-cells = <1>;
#size-cells = <0>;
status = "okay";

port@0 {
reg = <0>;
mipi_csi0_ep: endpoint {
remote-endpoint = <&ov5640_mipi_0_ep>;
data-lanes = <1 2 3 4>;
clock-lanes = <0>;
csis-hs-settle = <13>;
csis-clk-settle = <2>;
csis-wclk;
};
};
};

 

I will send the data lines and the CLK details shortly.

>>I hope the camera is connected to the CSI1 port of the EVK board.

I have connected in CSI0 as per the dts above. Correct me if I am doing wrong here.

Also one more thing, In my case, the camera is not enumerated directly with the EVK default config, I had to enable a couple of configs in menuconfig to make it work to this level. Not sure how it works for you with the default config in evk board, can you share the steps that you made the camera work in evk board please (any document )?

0 Kudos
1,493 Views
Suraj_Vagoli
Contributor I

@araja 

Try with default prebuilt images - Image-imx8mpevk.bin and imx8mp-evk.dtb

Rename Image-imx8mpevk.bin  with Image

 

 

Below are the changes from the default evk file

ov5640_0: ov5640_mipi@3c {
compatible = "ovti,ov5640";
reg = <0x3c>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_csi0_pwn>, <&pinctrl_csi0_rst>, <&pinctrl_csi_mclk>;
clocks = <&clk IMX8MP_CLK_IPP_DO_CLKO2>;
clock-names = "xclk";
assigned-clocks = <&clk IMX8MP_CLK_IPP_DO_CLKO2>;
assigned-clock-parents = <&clk IMX8MP_CLK_24M>;
assigned-clock-rates = <24000000>;
csi_id = <0>;
powerdown-gpios = <&gpio2 11 GPIO_ACTIVE_HIGH>;
reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
mclk = <24000000>;
mclk_source = <0>;
mipi_csi;
status = "okay";

port {
ov5640_mipi_0_ep: endpoint {
remote-endpoint = <&mipi_csi0_ep>;
data-lanes = <1 2>;
clock-lanes = <0>;
};
};
};

&mipi_csi_0 {
#address-cells = <1>;
#size-cells = <0>;
status = "okay";

port@0 {
reg = <0>;
mipi_csi0_ep: endpoint {
remote-endpoint = <&ov5640_mipi_0_ep>;
data-lanes = <2>;
csis-hs-settle = <13>;
csis-clk-settle = <2>;
csis-wclk;
};
};
};

 

 

0 Kudos
1,512 Views
zhangjunsheng
Contributor III

Hi @araja

     You can monitor if there is any interrupt triggered by printing interrupt flags in CSI driver to help you to locate issue.

     user level applcation get data from the stream: Camera (MIPI) - SOC MIPI PHY - SOC MIPI CSI - SOC ISI. Any node in the steam has problem, can lead user space has no data recevied and stuck there.

      You can also check you configuration. I remember OV4560 uses two lanes but I see in your dts configuration is 4 lane, I am not sure if it is the cause.

      You can also read the register with tool /units_test/memtool to read back register and understand the value from SOC user manual during running gst-lunch to know the SOC working status, such as SOC MIPI PHY status.

 

Best regards.

Johnson

0 Kudos
1,487 Views
araja
Contributor III

Hi @Suraj_Vagoli  @zhangjunsheng,

Changing the Data lane from 4 to 2 in dts file made the camera work. 

Thank you so much for the support.

0 Kudos
2,251 Views
zhangjunsheng
Contributor III
Hi,

share with you my previous issue is located at the connection of camera and the evk board.

start stream ok, should say camera i2c interface works. you may check the mipi interface, as after stream start, the further ioctl is waiting for image stream data from camera through mipi interface. let's think the condition that there is no data or error data mipi csi received, as the result, csi interface is keeping waiting for the data frame, then application will hang at waiting state.

the issue may locate at other area, depending your actual debugging findings.
0 Kudos
2,655 Views
zhangjunsheng
Contributor III

attach the log of ov5640 working on imx8mp-evk CSI1 (J13)

$ setenv fdtfile imx8mp-evk-basler-ov5640.dtb;boot

$ dmesg | grep ov5640
ov5640 2-003c: supply DOVDD not found, using dummy regulator
ov5640 2-003c: supply AVDD not found, using dummy regulator
ov5640 2-003c: supply DVDD not found, using dummy regulator
mx8-img-md: Registered sensor subdevice: ov5640 2-003c (1)
mx8-img-md: created link [ov5640 2-003c] => [mxc-mipi-csi2.1]
mx8-img-md: Registered sensor subdevice: ov5640 2-003c (1)
mx8-img-md: created link [ov5640 2-003c] => [mxc-mipi-csi2.1]

$ dmesg | grep mx8-img-md
mx8-img-md: Registered mxc_isi.1.capture as /dev/video2
mx8-img-md: Registered sensor subdevice: ov5640 2-003c (1)
mx8-img-md: created link [mxc_isi.1] => [mxc_isi.1.capture]
mx8-img-md: created link [mxc-mipi-csi2.1] => [mxc_isi.1]
mx8-img-md: created link [ov5640 2-003c] => [mxc-mipi-csi2.1]
mx8-img-md: Registered mxc_isi.1.capture as /dev/video2
mx8-img-md: Registered sensor subdevice: ov5640 2-003c (1)
mx8-img-md: created link [mxc_isi.1] => [mxc_isi.1.capture]
mx8-img-md: created link [mxc-mipi-csi2.1] => [mxc_isi.1]
mx8-img-md: created link [ov5640 2-003c] => [mxc-mipi-csi2.1]

$ gst-launch-1.0 -v v4l2src device=/dev/video2 ! video/x-raw ! queue ! waylandsink
enter isp_mi_stop
mxc-mipi-csi2.0: mipi_csis_imx8mp_phy_reset, No remote pad found!

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1
/GstPipeline:pipeline0/GstWaylandSink:waylandsink0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)1:4:5:1

bypass csc
input fmt YUV4
output fmt YUYV

Redistribute latency...
^C handling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:03.168273375
Setting pipeline to NULL ...
Total showed frames (50), playing for (0:00:03.168278750), fps (15.781).
Freeing pipeline ...
0 Kudos
2,624 Views
joanxie
NXP TechSupport
NXP TechSupport

did you use nxp IMX8MP board and prebuilt images, right? it seems the board detect ov5640 already, and I tested this on my board successfully, I couldn't reproduce this on my side. did you have other camera module to test J13 port?

0 Kudos
2,614 Views
zhangjunsheng
Contributor III
Hi joanxie:
o5640 works fine on my side on J13 port. sorry for misleading you. the log I post is only for sharing, it shows gst tools working fine.

Johnson
0 Kudos
2,717 Views
zhangjunsheng
Contributor III

Hi,

OK, thanks for the support.

Today we reconnet the camera with the camera module board, and tested well.

 

Best regards.

Johnson

0 Kudos
2,727 Views
joanxie
NXP TechSupport
NXP TechSupport

I have tested on imx8mp board with 5.15.32 bsp without any issue

joanxie_0-1673590680949.png

 

Tags (1)
0 Kudos