Porting IMX477 for IMX8MP Isp

CanberkOzbey · ‎05-11-2023

Hello All

I am working on porting an IMX477 camera driver to the IMX8MP EDM-g-WB board of technexion.I am following the i.MX 8M Plus Camera and Display Guide, section 2 Camera Sensor Porting Guide and 8MP ISP New Camera Porting Guide.

When i try to use gstreamer command i get the following error.

root@edm-g-imx8mp:~# gst-launch-1.0 -vvv v4l2src device=/dev/video0 num-buffers=1 \! "video/x-bayer,width=1920,height=1080,framerate=(fraction)25/1,format=(string)bggr" \! bayer2rgb ! pngenc ! filesink location=test.png
[ 2872.204035] enter isp_mi_stop
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../git/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.717251750
Setting pipeline to NULL ...
Freeing pipeline ...

When try to use v4l2 to get a photo i get this as output

root@edm-g-imx8mp:~# v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=frame.raw --stream-count=1 [ 3319.179967] ------------[ cut here ]------------
[ 3319.184597] WARNING: CPU: 1 PID: 884 at drivers/media/common/videobuf2/videobuf2-core.c:807 vb2_core_reqbufs+0x138/0x400
[ 3319.195477] Modules linked in: imx477(O) imx8_media_dev(C) vvcam_video(O) vvcam_isp(O) vvcam_dwe(O) algif_skcipher fsl_jr_uio caam_jr caamkeyblob_desc caamhash_desc caamalg_desc crypto_engine rng_core authenc libdes crct10dif_ce snd_soc_imx_hdmi wlan(O) snd_soc_fsl_easrc dw_hdmi_cec snd_soc_fsl_sai snd_soc_fsl_aud2htx flexcan can_dev caam secvio error i2c_gpio [last unloaded: imx8_media_dev]
[ 3319.230214] CPU: 1 PID: 884 Comm: v4l2-ctl Tainted: G WC O 5.15.52+gd6c58d2ff21e #1
[ 3319.238910] Hardware name: TechNexion EDM-G-IMX8MP and WB baseboard (DT)
[ 3319.245608] pstate: 80000005 (Nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 3319.252567] pc : vb2_core_reqbufs+0x138/0x400
[ 3319.256923] lr : vb2_core_reqbufs+0x100/0x400
[ 3319.261278] sp : ffff80001b32ba50
[ 3319.264588] x29: ffff80001b32ba50 x28: ffff000004119400 x27: 0000000000000000
[ 3319.271725] x26: 0000000000000000 x25: ffff80001b32bd28 x24: 0000000000000000
[ 3319.278861] x23: ffff000004119508 x22: 0000000000000001 x21: ffff80001b32bd28
[ 3319.285997] x20: 0000000000000000 x19: ffff0000041194a0 x18: 0000000000000000
[ 3319.293133] x17: 0000000000000000 x16: 0000000000000000 x15: 0000ffffc1f7c328
[ 3319.300268] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
[ 3319.307404] x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000
[ 3319.314540] x8 : ffff000004119548 x7 : 0000000000000000 x6 : 0000000010000000
[ 3319.321676] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000001
[ 3319.328811] x2 : ffff80001b32baac x1 : ffff80001b32baa0 x0 : ffff80001b32baa8
[ 3319.335947] Call trace:
[ 3319.338391] vb2_core_reqbufs+0x138/0x400
[ 3319.342399] vb2_reqbufs+0x5c/0x7c
[ 3319.345800] vidioc_reqbufs+0x144/0x200 [vvcam_video]
[ 3319.350854] v4l_reqbufs+0x50/0x6c
[ 3319.354255] __video_do_ioctl+0x178/0x3dc
[ 3319.358262] video_usercopy+0x368/0x6ec
[ 3319.362096] video_ioctl2+0x1c/0x30
[ 3319.365583] v4l2_ioctl+0x44/0x64
[ 3319.368897] __arm64_sys_ioctl+0xac/0xf0
[ 3319.372821] invoke_syscall+0x48/0x114
[ 3319.376570] el0_svc_common.constprop.0+0xd4/0xfc
[ 3319.381275] do_el0_svc+0x2c/0x94
[ 3319.384588] el0_svc+0x28/0x80
[ 3319.387643] el0t_64_sync_handler+0xa8/0x130
[ 3319.391914] el0t_64_sync+0x1a0/0x1a4
[ 3319.395575] ---[ end trace d7a0f9a20adad406 ]---
VIDIOC_REQBUFS returned -1 (Invalid argument)

My Dmesg output

root@edm-g-imx8mp:~# dmesg |grep imx477
[ 45.532556] enter imx477_probe
[ 45.536796] enter imx477_retrieve_capture_properties
[ 45.541962] imx477 1-001a: supply DOVDD not found, using dummy regulator
[ 45.549007] imx477 1-001a: supply DVDD not found, using dummy regulator
[ 45.555806] imx477 1-001a: supply AVDD not found, using dummy regulator
[ 45.562618] enter imx477_regulator_enable
[ 45.566660] enter imx477_set_clk_rate
[ 45.570386] enter imx477_power_on
[ 45.574056] enter imx477_reset
[ 45.631773] enter imx477_read_reg
[ 45.635629] enter imx477_read_reg
[ 45.639498] imx477_probe camera mipi imx477, is found

My media-ctl output

root@edm-g-imx8mp:~# media-ctl -p
Media controller API version 5.15.52

Media device information
------------------------
driver vvcam-video
model viv_media
serial
bus info
hw revision 0x0
driver version 5.15.52

Device topology
- entity 1: viv_v4l20 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "vvcam-isp.0":0 [ENABLED]

- entity 5: vvcam-isp.0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/v4l-subdev0
pad0: Source
-> "viv_v4l20":0 [ENABLED]

- entity 7: vvcam-dwe.0 (2 pads, 0 link)
type Node subtype V4L flags 0
device node name /dev/v4l-subdev1
pad0: Source
pad1: Sink

My V4L2 output

root@edm-g-imx8mp:~# v4l2-ctl --list-devices
[ 2480.347798] enter isp_mi_stop
():
/dev/v4l-subdev0
/dev/v4l-subdev1

VIV (platform:viv0):
/dev/video0

viv_media (platform:vvcam-video):
/dev/media0

My Dts

/dts-v1/;

#include "imx8mp-edm-g-wb.dts"

&i2c2 {

/delete-node/ov5640_mipi@3c;

imx477_0: imx477@1a {

compatible = "sony,imx477";

reg = <0x1a>;

pinctrl-names = "default";

pinctrl-0 = <&pinctrl_csi0_pwn>, <&pinctrl_csi0_rst>, <&pinctrl_csi_mclk>;

pwn-gpios = <&pca9555_a21 2 GPIO_ACTIVE_HIGH>;

rst-gpios = <&gpio1 8 GPIO_ACTIVE_LOW>;

clocks = <&clk IMX8MP_CLK_IPP_DO_CLKO2>;

clock-names = "csi_mclk";

assigned-clocks = <&clk IMX8MP_CLK_IPP_DO_CLKO2>;

assigned-clock-parents = <&clk IMX8MP_CLK_24M>;

assigned-clock-rates = <24000000>;

mclk = <24000000>;

mclk_source = <0>;

mipi_csi;

csi_id = <0>;

status = "okay";

port {

imx477_0_ep: endpoint {

data-lanes = <1 2>;

clock-lanes = <0>;

max-pixel-frequency = /bits/ 64 <500000000>;

clock-noncontinuous;

remote-endpoint = <&mipi_csi0_ep>;

};

&i2c3 {

clock-frequency = <100000>;

pinctrl-names = "default";

pinctrl-0 = <&pinctrl_i2c3>;

status = "okay";

ov5640_1: ov5640_mipi@3c {

status = "disabled";

};

&cameradev {

status = "okay";

};

&isi_0 {

status = "disabled";

};

&isi_1 {

status = "disabled";

};

&isp_0 {

status = "okay";

};

&dewarp {

status = "okay";

};

&mipi_csi_0 {

status = "okay";

#address-cells = <1>;

#size-cells = <0>;

clock-frequency = <500000000>;

assigned-clock-rates = <500000000>;

port@0 {

reg = <0>;

mipi_csi0_ep: endpoint {

remote-endpoint = <&imx477_0_ep>;

data-lanes = <1 2>;

csis-hs-settle = <13>;

csis-clk-settle = <1>;

csis-wclk;

};

&mipi_csi_1 {

status = "disabled";

};

ISI and VVCAM driver is attached

Thank you,

Canberk

joanxie · ‎05-11-2023

pls refer to the link as below:

"https://community.nxp.com/t5/i-MX-Processors/imx477-sensor-driver-for-i-MX8M-Plus/m-p/1271067"

CanberkOzbey · ‎05-11-2023

Hi Joanxie
Thanks for your reply but i already checked that link and it doesn't resolve my problem.

joanxie · ‎05-20-2023

do you mind sharing the logfile with me? and pls refer to the enclosed application note, pls check if your setup is correct or not