Support for VC MIPI IMX327C Bayer RAW10 camera on IMX6

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Support for VC MIPI IMX327C Bayer RAW10 camera on IMX6

2,830 Views
valentinryzhenk
Contributor II

Hello.

I am working on camera support for visual components VC MIPI IMX327C for DART-MX6 from Variscite. Camera transmits video in bayer raw10 format 1920 x 1080 60FPS.

I modified the standard ov5640_mipi.c driver and the following kernel files:

  • mxc_v4l2_capture.c:
    static struct v4l2_input mxc_capture_inputs[MXC_V4L2_CAPTURE_NUM_INPUTS] = {
     {
     .index = 0,
     // .name = "CSI IC MEM",
     .name = "CSI MEM",
    ****
    };‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
    static inline int valid_mode(u32 palette)
    {
     return ((palette == V4L2_PIX_FMT_RGB565) ||
    ****
     (palette == V4L2_PIX_FMT_NV12) ||
     (palette == IPU_PIX_FMT_GENERIC_16) ||
     (palette == V4L2_PIX_FMT_SRGGB10P) ||
     (palette == V4L2_PIX_FMT_SRGGB10));
    }‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
    static int mxc_v4l2_s_fmt(cam_data *cam, struct v4l2_format *f)
    {
     f->fmt.pix.pixelformat = V4L2_PIX_FMT_SRGGB10; //V4L2_PIX_FMT_SRGGB10
    ****
    switch (f->fmt.pix.pixelformat) {
    ****
     case V4L2_PIX_FMT_SRGGB10P:
     case V4L2_PIX_FMT_SRGGB10:
     size = f->fmt.pix.width * f->fmt.pix.height * 2;
     bytesperline = f->fmt.pix.width * 2;
     default:
     break;
     }
    ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
  • mxc_mipi_csi2.c:
    int mipi_csi2_reset(struct mipi_csi2_info *info)
    {
    ****
     mipi_csi2_write(info, 0x0000000C, MIPI_CSI2_PHY_TST_CTRL1);//14
    ****
     return 0;
    }
    ‍‍‍‍‍‍‍

F = FH * FW * FPS * BI * DF = 1920 * 1080 * 60 * 1 * 1.35 = 167.9 MHz

MIPI D-PHY clock: (167.9 * 10 (RAW10)) / 4(4 lanes) / 2 = 210 MHz

MIPI_CSI2_PHY_TST_CTRL1 setting = 210 MHz * 2 (DDR mode) = 420 MHz

  • ipu_capture.c:
    int32_t
    ipu_csi_init_interface(struct ipu_soc *ipu, uint16_t width, uint16_t height,
     uint32_t pixel_fmt, ipu_csi_signal_cfg_t cfg_param)
    {
    ****
    switch (pixel_fmt) {
    ****
     case IPU_PIX_FMT_GENERIC:
     case IPU_PIX_FMT_SRGGB10P:
     case IPU_PIX_FMT_GENERIC_16:
     case V4L2_PIX_FMT_SRGGB10:
     cfg_param.data_fmt = CSI_SENS_CONF_DATA_FMT_BAYER;
     break;
    ****
    }‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
  • ipu_csi_enc.c:
    static int csi_enc_setup(cam_data *cam)
    {
    ****
     if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_YUV420)
     pixel_fmt = IPU_PIX_FMT_YUV420P;
    ****
     else if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_SRGGB10P || cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_SRGGB10)
     pixel_fmt = IPU_PIX_FMT_GENERIC_16;
    ****
    }
  • Device tree:
vc_mipi_imx327c: vc_mipi_imx327c@1a {
 compatible = "sony,vc_mipi_imx327c";
 reg = <0x1a>;
 csi_id = <0>;
 clocks = <&clks 200>;
 clock-names = "csi_mclk";
 mclk = <24000000>;
 mclk_source = <0>;
 data-lanes = <1 2 3 4>;
};
&mipi_csi {
 status = "okay";
 ipu_id = <0>;
 csi_id = <0>;
 v_channel = <0>;
 lanes = <4>;
 };

The result is the following picture (shone a flashlight at the camera):

cd4b4658-e371-4963-8e10-3e0011743e8e.jpg

The video is taken by the imxv4l2videosrc plugin processed by gstreamer and then transferred to the bayer2rgb plugin.

If I try to independently convert the raw buffer to a buyer (every 5 bytes in 4 uint16 numbers) before sending it to the bayer2rgb plugin, I get the following picture (shone a flashlight at the camera):

2020-06-30 14-48-10.png

My question is how to get active pixels from the buffer correctly and do I need to do this? Or maybe I'm wrong and pick up the buffer incorrectly?

P.S. Buffer size 1920 * 1080 * 2 = 4147200 bytes.

Labels (5)
0 Kudos
6 Replies

2,602 Views
joanxie
NXP TechSupport
NXP TechSupport

pls refer to the document as below:

i.MX6Q capture raw(bayer) data and debayer  

have you tested by ./cap.out first?

0 Kudos

2,602 Views
valentinryzhenk
Contributor II

I checked ./cap.out and this is what happenedGray_Image_0x0C.jpg

The resulting buffer was converted using opencv.

A frame in memory looks like this:
header - 320 bytes
11 lines (vertical offset) - 26400 bytes (1920*1.25*11)
1080 lines - frame (1920*1080*1.25)

pastedImage_2.png

I cannot figure out how to remove these vertical stripes.

0 Kudos

1,731 Views
dimitarmaznekov
Contributor II

Did you succeed with this taks ?

0 Kudos

2,602 Views
joanxie
NXP TechSupport
NXP TechSupport

have you applied to the patches in the link I sent to you? since you use the third party board and not use ov5640, maybe you couldn't apply these patches directly but you need to  change unit test mxc_v4l2_capture.c fistly

0 Kudos

2,602 Views
valentinryzhenk
Contributor II

I was looking at the patch and manually changing the settings for my camera. As a result, the unit test mxc_v4l2_capture.c captures a frame from the camera. I cannot figure out how to remove these vertical stripes.

0 Kudos

2,602 Views
joanxie
NXP TechSupport
NXP TechSupport

how about use lower resoltuion? any improvement?

0 Kudos