IMX8M Mini New MIPI-CSI Camera Driver Integration

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

IMX8M Mini New MIPI-CSI Camera Driver Integration

Jump to solution
7,234 Views
matthewmader
Contributor II

Hello,

I have a Sony IMX219 camera that I am trying to integrate with the IMX8M Mini.  I have modeled my driver on the OV5640 driver provided the linux-imx repo.  However, I cannot seem the camera to stream video.  I run the following command: 

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480 ! autovideosink sync=false

and get the following error:

ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:../../../../git/libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)

I have tried variation on the above command but always get the same output.

 

When I query the camera via the v4l2-ctl tools I get the following responses:

v4l2-ctl --list-devices

i.MX6S_CSI (platform:32e20000.csi1_bridge):
/dev/video0

v4l2-ctl --list-formats-ext

ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

[0]: 'BA81' (8-bit Bayer BGBG/GRGR)
Size: Discrete 3280x2464
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1640x1232
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)

v4l2-ctl --all

Driver Info:
Driver name : mx6s-csi
Card type : i.MX6S_CSI
Bus info : platform:32e20000.csi1_bridge
Driver version : 5.4.3
Capabilities : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Priority: 0
Video input : 0 (Camera: ok)
Format Unknown (00000000): <= Not sure why this is 'Unknown'
Crop Capability Video Capture:
Bounds : Left 0, Top 0, Width 0, Height 0
Default : Left 0, Top 0, Width 0, Height 0
Pixel Aspect: 1/1
Crop: Left 0, Top 0, Width 0, Height 0
Selection: crop, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: crop_default, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: crop_bounds, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: compose, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: compose_default, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: compose_bounds, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: compose_padded, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: native_size, Left 0, Top 0, Width 0, Height 0, Flags:
Streaming Parameters Video Capture:
Capabilities : timeperframe
Frames per second: 30.000 (30/1)
Read buffers : 0


Below is what my DTS file looks like:

/* IMX219 Camera */
imx219@10 {
    status = "okay";
    compatible = "sony,imx219";
    reg = <0x10>;

    port {
        imx219_ep: endpoint {
            remote-endpoint = <&mipi1_sensor_ep>;
            clock-lanes = <0>;
            data-lanes = <1 2>;
            clock-noncontinuous;
            link-frequencies = /bits/ 64 <456000000>;
      };
    };
};

&csi1_bridge {
    fsl,mipi-mode;
    status = "okay";
    port {
      csi1_ep: endpoint {
        remote-endpoint = <&csi1_mipi_ep>;
      };
    };
};

&mipi_csi_1 {
    #address-cells = <1>;
    #size-cells = <0>;
    status = "okay";

    port {
      mipi1_sensor_ep: endpoint@1 {
        remote-endpoint = <&imx219_ep>;
        data-lanes = <2>;
        csis-hs-settle = <13>; <= Not sure what this field does
        csis-clk-settle = <2>; <= Not sure what this field does
        csis-wclk; <= Not sure what this field does
      };

      csi1_mipi_ep: endpoint@2 {
        remote-endpoint = <&csi1_ep>;
      };
    };
};

Labels (1)
0 Kudos
1 Solution
7,210 Views
matthewmader
Contributor II

I am not an expert with gstreamer or v4l2, however with Weidong's comment above about the eLCDIF not natively supporting RAW8/RAW10, I revised my gst-launch-1.0 command to the following and it worked (mostly)!

gst-launch-1.0 -v v4l2src ! "video/x-bayer,format=bggr,width=640,height=480,framerate=30/1" ! bayer2rgb ! videoconvert ! autovideosink sync=false

I am getting a very dark image that is off color, but that is probably some other defect in my driver.  However this is enough progress to me to mark this question as closed!

View solution in original post

0 Kudos
8 Replies
3,389 Views
udhayamoorthi
Contributor II

can you pls share your whole dtsi , mipi-csi,csir driver codes?

 

0 Kudos
7,211 Views
matthewmader
Contributor II

I am not an expert with gstreamer or v4l2, however with Weidong's comment above about the eLCDIF not natively supporting RAW8/RAW10, I revised my gst-launch-1.0 command to the following and it worked (mostly)!

gst-launch-1.0 -v v4l2src ! "video/x-bayer,format=bggr,width=640,height=480,framerate=30/1" ! bayer2rgb ! videoconvert ! autovideosink sync=false

I am getting a very dark image that is off color, but that is probably some other defect in my driver.  However this is enough progress to me to mark this question as closed!

0 Kudos
5,424 Views
jhpark_vine
Contributor III

Can you share how to integrate MipI-csi camera driver?

1. how to configure mipi-csi dphy clock settings?

2. patches of ov5640 camera driver and mx6s_capture  

3. is this work on linux or android ?

 

0 Kudos
7,219 Views
weidong_sun
NXP TechSupport
NXP TechSupport

Hello matthewmader ,

See below, please!
1. i.MMX8MM MIPI CSI doesn't support capturing RAW10 data for camera by default.
2. i.MX8MM doesn't support ISP, so RAW10 data can't be displayed directly, eLCDIF needs YUV or RGB format to display camera image.
3. About SoftISP we provide from customer
Actually we provide softisp for customer, but the softisp needs OpenCL, and i.MX8MM doesn't support opencl, so softisp can't run on i.MX8MM platfor.
4. About customer's application for RAW data.
---customer should write soft ISP code by herself.
---customer should add these code to mx6s_capture.c to support capturing RAW10 data.(Try it , please!)

.......

+ {
+.name = "RAWRGB10 (SBGGR10)",
+.fourcc = V4L2_PIX_FMT_SBGGR10,
+.pixelformat = V4L2_PIX_FMT_SBGGR10,
+.mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
+.bpp = 1,
+}


case V4L2_PIX_FMT_YUV32:
case V4L2_PIX_FMT_SBGGR8:
+case V4L2_PIX_FMT_SBGGR10:
width = pix->width;
break;

.......

Hope these information is helpful for you.
Have a nice day!
B.R,
weidong

5,346 Views
Sen_tian1
Contributor II

Hello

在imx8mm上使用uvc camera跑v4l2官方的sample没有问题,但是使用ov5640的camera就跑不通,会报select timeout的错误。

https://linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/v4l/capture.c.html

v4l2-ctl --all

Driver Info:
Driver name : mx6s-csi
Card type : i.MX6S_CSI
Bus info : platform:32e20000.csi1_bridge
Driver version : 5.4.3
Capabilities : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format
Priority: 0
Video input : 0 (Camera: ok)
Format Unknown (00000000): <= Not sure why this is 'Unknown'
Crop Capability Video Capture:
Bounds : Left 0, Top 0, Width 0, Height 0
Default : Left 0, Top 0, Width 0, Height 0
Pixel Aspect: 1/1
Crop: Left 0, Top 0, Width 0, Height 0
Selection: crop, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: crop_default, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: crop_bounds, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: compose, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: compose_default, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: compose_bounds, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: compose_padded, Left 0, Top 0, Width 0, Height 0, Flags:
Selection: native_size, Left 0, Top 0, Width 0, Height 0, Flags:
Streaming Parameters Video Capture:
Capabilities : timeperframe
Frames per second: 30.000 (30/1)
Read buffers : 0

crop,selection的size都为0,是这个的原因嘛?

有什么建议或可以提供一些解决的方向嘛?

十分感谢。

0 Kudos
5,866 Views
wjboylan
Contributor I

Hello @weidong_sun,

I am also using imx219 with imx8mm mini but using Linux Kernel 5.10.35.  I don't see a mx6s_capture.c file to add support for capturing RAW10 data.  Attached are all the imx media files that are available in 5.10.35.  Which file should I update?

Thank you, 

Will 

 

0 Kudos
3,281 Views
udhayamoorthi
Contributor II

Did you take proper image?

0 Kudos
7,212 Views
matthewmader
Contributor II

Thank you Weidong for your reply,

However I am not attempting to use the RAW10 format yet.  Instead I am using RAW8 which should be natively supported the mx6s_capture driver but I will add your suggestions for when we decided to move to RAW10. 

It is good to know that regardless of RAW10 or RAW8 it needs to be converted to RGB for the LCDIF.  I was unaware of this restriction.

I am not sure I understand your comment about SoftISP?  I am not attempting to use SoftISP.

Thank you for your help.

0 Kudos