topic Re: IMX8M Mini New MIPI-CSI Camera Driver Integration in i.MX Processors

IMX8M Mini New MIPI-CSI Camera Driver Integration

matthewmader — Wed, 30 Sep 2020 17:51:42 GMT

Hello,

I have a Sony IMX219 camera that I am trying to integrate with the IMX8M Mini. I have modeled my driver on the OV5640 driver provided the linux-imx repo. However, I cannot seem the camera to stream video. I run the following command:

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480 ! autovideosink sync=false

and get the following error:

ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:../../../../git/libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)

I have tried variation on the above command but always get the same output.

When I query the camera via the v4l2-ctl tools I get the following responses:

v4l2-ctl --list-devices

i.MX6S_CSI (platform:32e20000.csi1_bridge):
/dev/video0

v4l2-ctl --list-formats-ext

ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

[0]: 'BA81' (8-bit Bayer BGBG/GRGR)
Size: Discrete 3280x2464
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1640x1232
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)

v4l2-ctl --all

Driver Info:
Driver name : mx6s-csi
Card type : i.MX6S_CSI
Bus info : platform:32e20000.csi1_bridge
Driver version : 5.4.3
Capabilities : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Priority: 0
Video input : 0 (Camera: ok)
Format Unknown (00000000): <= Not sure why this is 'Unknown'
Crop Capability Video Capture:
Bounds : Left 0, Top 0, Width 0, Height 0
Default : Left 0, Top 0, Width 0, Height 0
Pixel Aspect: 1/1
Crop: Left 0, Top 0, Width 0, Height 0
Selection: crop, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: crop_default, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: crop_bounds, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: compose, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: compose_default, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: compose_bounds, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: compose_padded, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: native_size, Left 0, Top 0, Width 0, Height 0, Flags:
Streaming Parameters Video Capture:
Capabilities : timeperframe
Frames per second: 30.000 (30/1)
Read buffers : 0

Below is what my DTS file looks like:

/* IMX219 Camera */
imx219@10 {
status = "okay";
compatible = "sony,imx219";
reg = <0x10>;

port {
imx219_ep: endpoint {
remote-endpoint = <&mipi1_sensor_ep>;
clock-lanes = <0>;
data-lanes = <1 2>;
clock-noncontinuous;
link-frequencies = /bits/ 64 <456000000>;
};
};
};

&csi1_bridge {
fsl,mipi-mode;
status = "okay";
port {
csi1_ep: endpoint {
remote-endpoint = <&csi1_mipi_ep>;
};
};
};

&mipi_csi_1 {
#address-cells = <1>;
#size-cells = <0>;
status = "okay";

port {
mipi1_sensor_ep: endpoint@1 {
remote-endpoint = <&imx219_ep>;
data-lanes = <2>;
csis-hs-settle = <13>; <= Not sure what this field does
csis-clk-settle = <2>; <= Not sure what this field does
csis-wclk; <= Not sure what this field does
};

csi1_mipi_ep: endpoint@2 {
remote-endpoint = <&csi1_ep>;
};
};
};

Re: IMX8M Mini New MIPI-CSI Camera Driver Integration

weidong_sun — Thu, 01 Oct 2020 01:26:09 GMT

Hello matthewmader ,

See below, please!
1. i.MMX8MM MIPI CSI doesn't support capturing RAW10 data for camera by default.
2. i.MX8MM doesn't support ISP, so RAW10 data can't be displayed directly, eLCDIF needs YUV or RGB format to display camera image.
3. About SoftISP we provide from customer
Actually we provide softisp for customer, but the softisp needs OpenCL, and i.MX8MM doesn't support opencl, so softisp can't run on i.MX8MM platfor.
4. About customer's application for RAW data.
---customer should write soft ISP code by herself.
---customer should add these code to mx6s_capture.c to support capturing RAW10 data.(Try it , please!)

.......

+ {
+.name = "RAWRGB10 (SBGGR10)",
+.fourcc = V4L2_PIX_FMT_SBGGR10,
+.pixelformat = V4L2_PIX_FMT_SBGGR10,
+.mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
+.bpp = 1,
+}

case V4L2_PIX_FMT_YUV32:
case V4L2_PIX_FMT_SBGGR8:
+case V4L2_PIX_FMT_SBGGR10:
width = pix->width;
break;

.......

Hope these information is helpful for you.
Have a nice day!
B.R,
weidong

Re: IMX8M Mini New MIPI-CSI Camera Driver Integration

matthewmader — Thu, 01 Oct 2020 10:05:11 GMT

Thank you Weidong for your reply,

However I am not attempting to use the RAW10 format yet. Instead I am using RAW8 which should be natively supported the mx6s_capture driver but I will add your suggestions for when we decided to move to RAW10.

It is good to know that regardless of RAW10 or RAW8 it needs to be converted to RGB for the LCDIF. I was unaware of this restriction.

I am not sure I understand your comment about SoftISP? I am not attempting to use SoftISP.

Thank you for your help.

Re: IMX8M Mini New MIPI-CSI Camera Driver Integration

matthewmader — Thu, 01 Oct 2020 10:26:26 GMT

I am not an expert with gstreamer or v4l2, however with Weidong's comment above about the eLCDIF not natively supporting RAW8/RAW10, I revised my gst-launch-1.0 command to the following and it worked (mostly)!

gst-launch-1.0 -v v4l2src ! "video/x-bayer,format=bggr,width=640,height=480,framerate=30/1" ! bayer2rgb ! videoconvert ! autovideosink sync=false

I am getting a very dark image that is off color, but that is probably some other defect in my driver. However this is enough progress to me to mark this question as closed!

Re: IMX8M Mini New MIPI-CSI Camera Driver Integration

wjboylan — Thu, 08 Jul 2021 21:23:57 GMT

Hello @weidong_sun,

I am also using imx219 with imx8mm mini but using Linux Kernel 5.10.35. I don't see a mx6s_capture.c file to add support for capturing RAW10 data. Attached are all the imx media files that are available in 5.10.35. Which file should I update?

Thank you,

Will

Re: IMX8M Mini New MIPI-CSI Camera Driver Integration

jhpark_vine — Tue, 28 Sep 2021 21:59:56 GMT

Can you share how to integrate MipI-csi camera driver?

1. how to configure mipi-csi dphy clock settings?

2. patches of ov5640 camera driver and mx6s_capture

3. is this work on linux or android ?

Re: IMX8M Mini New MIPI-CSI Camera Driver Integration

Sen_tian1 — Thu, 07 Oct 2021 10:14:12 GMT

Hello

在imx8mm上使用uvc camera跑v4l2官方的sample没有问题，但是使用ov5640的camera就跑不通，会报select timeout的错误。

https://linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/v4l/capture.c.html

v4l2-ctl --all

crop，selection的size都为0，是这个的原因嘛？

有什么建议或可以提供一些解决的方向嘛？

十分感谢。

Re: IMX8M Mini New MIPI-CSI Camera Driver Integration

udhayamoorthi — Sat, 24 Sep 2022 10:44:58 GMT

can you pls share your whole dtsi , mipi-csi,csir driver codes?

Re: IMX8M Mini New MIPI-CSI Camera Driver Integration

udhayamoorthi — Sat, 08 Oct 2022 15:18:09 GMT

Did you take proper image?

Re: IMX8M Mini New MIPI-CSI Camera Driver Integration

Bhavesh_Bhadricha — Thu, 25 Apr 2024 05:30:53 GMT

hi weidong,

As per imx8m mini reference manual, RAW format ,pg. 4019 , CSI2 support RAW10.
kindly let us know the reason for mentioning CSI2 not supporting RAW10.
kindly correct if I mis understood your reply

Re: IMX8M Mini New MIPI-CSI Camera Driver Integration

dhruvinrajpura — Fri, 25 Apr 2025 07:38:06 GMT

Hello @matthewmader

I am using same command but getting error, giving you configurations as well in logs

root@imx93-11x11-lpddr4x-evk:~# gst-launch-1.0 -v v4l2src ! "video/x-bayer,format=bggr,width=640,height=480,framerate=30/1" ! bayer2rgb ! videoconvert ! autovideosink sync=false Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0: Failed to initialize egl: EGL_NOT_INITIALIZED Additional debug info: /usr/src/debug/gstreamer1.0-plugins-base/1.24.7.imx/ext/gl/gstglimagesink.c(1140): _ensure_gl_setup (): /GstGLImageSinkBin:autovideosink0-actual-sink-glimage/GstGLImageSink:sink Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstSystemClock ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error. Additional debug info: /usr/src/debug/gstreamer1.0/1.24.7.imx/libs/gst/base/gstbasesrc.c(3177): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: streaming stopped, reason not-negotiated (-4) Execution ended after 0:00:00.003832912 Setting pipeline to NULL ... Freeing pipeline ... root@imx93-11x11-lpddr4x-evk:~# root@imx93-11x11-lpddr4x-evk:~# media-ctl -p Media controller API version 6.6.23 Media device information ------------------------ driver mxc-md model FSL Capture Media Device serial bus info platform:42800000.bus:camera hw revision 0x0 driver version 6.6.23 Device topology - entity 1: mxc_isi.0 (16 pads, 2 links, 0 routes) type V4L2 subdev subtype Unknown flags 0 pad0: Sink <- "mxc-mipi-csi2.0":4 [ENABLED] pad1: Sink pad2: Sink pad3: Sink pad4: Sink pad5: Sink pad6: Sink pad7: Sink pad8: Sink pad9: Sink pad10: Sink pad11: Sink pad12: Source -> "mxc_isi.0.capture":0 [ENABLED] pad13: Source pad14: Source pad15: Sink - entity 18: mxc_isi.0.capture (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video0 pad0: Sink <- "mxc_isi.0":12 [ENABLED] - entity 22: mxc-mipi-csi2.0 (8 pads, 2 links) type Node subtype V4L flags 0 device node name /dev/v4l-subdev0 pad0: Sink <- "imx219 2-0010":0 [ENABLED,IMMUTABLE] pad1: Sink pad2: Sink pad3: Sink pad4: Source -> "mxc_isi.0":0 [ENABLED] pad5: Source pad6: Source pad7: Source - entity 31: imx219 2-0010 (1 pad, 1 link, 0 routes) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev1 pad0: Source [stream:0 fmt:SRGGB10_1X10/3280x2464 field:none colorspace:raw xfer:none quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(8,8)/3280x2464] -> "mxc-mipi-csi2.0":0 [ENABLED,IMMUTABLE] root@imx93-11x11-lpddr4x-evk:~# root@imx93-11x11-lpddr4x-evk:~# v4l2-ctl --list-devices FSL Capture Media Device (platform:42800000.bus:camera): /dev/media0 mxc-isi-cap_v1 (platform:4ae40000.isi:cap_devic): /dev/video0 root@imx93-11x11-lpddr4x-evk:~#