Support for VC MIPI IMX327C Bayer RAW10 camera on IMX6

[valentinryzhenk]

Hello.

I am working on camera support for visual components VC MIPI IMX327C for DART-MX6 from Variscite. Camera transmits video in bayer raw10 format 1920 x 1080 60FPS.

I modified the standard ov5640_mipi.c driver and the following kernel files:

• mxc_v4l2_capture.c:

  static struct v4l2_input mxc_capture_inputs[MXC_V4L2_CAPTURE_NUM_INPUTS] = {
   {
   .index = 0,
   // .name = "CSI IC MEM",
   .name = "CSI MEM",
  ****
  };‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
  static inline int valid_mode(u32 palette)
  {
   return ((palette == V4L2_PIX_FMT_RGB565) ||
  ****
   (palette == V4L2_PIX_FMT_NV12) ||
   (palette == IPU_PIX_FMT_GENERIC_16) ||
   (palette == V4L2_PIX_FMT_SRGGB10P) ||
   (palette == V4L2_PIX_FMT_SRGGB10));
  }‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍
  static int mxc_v4l2_s_fmt(cam_data *cam, struct v4l2_format *f)
  {
   f->fmt.pix.pixelformat = V4L2_PIX_FMT_SRGGB10; //V4L2_PIX_FMT_SRGGB10
  ****
  switch (f->fmt.pix.pixelformat) {
  ****
   case V4L2_PIX_FMT_SRGGB10P:
   case V4L2_PIX_FMT_SRGGB10:
   size = f->fmt.pix.width * f->fmt.pix.height * 2;
   bytesperline = f->fmt.pix.width * 2;
   default:
   break;
   }
  ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

• mxc_mipi_csi2.c:

  int mipi_csi2_reset(struct mipi_csi2_info *info)
  {
  ****
   mipi_csi2_write(info, 0x0000000C, MIPI_CSI2_PHY_TST_CTRL1);//14
  ****
   return 0;
  }
  ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

F = FH * FW * FPS * BI * DF = 1920 * 1080 * 60 * 1 * 1.35 = 167.9 MHz

MIPI D-PHY clock: (167.9 * 10 (RAW10)) / 4(4 lanes) / 2 = 210 MHz

MIPI_CSI2_PHY_TST_CTRL1 setting = 210 MHz * 2 (DDR mode) = 420 MHz

• ipu_capture.c:

  int32_t
  ipu_csi_init_interface(struct ipu_soc *ipu, uint16_t width, uint16_t height,
   uint32_t pixel_fmt, ipu_csi_signal_cfg_t cfg_param)
  {
  ****
  switch (pixel_fmt) {
  ****
   case IPU_PIX_FMT_GENERIC:
   case IPU_PIX_FMT_SRGGB10P:
   case IPU_PIX_FMT_GENERIC_16:
   case V4L2_PIX_FMT_SRGGB10:
   cfg_param.data_fmt = CSI_SENS_CONF_DATA_FMT_BAYER;
   break;
  ****
  }‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

• ipu_csi_enc.c:

  static int csi_enc_setup(cam_data *cam)
  {
  ****
   if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_YUV420)
   pixel_fmt = IPU_PIX_FMT_YUV420P;
  ****
   else if (cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_SRGGB10P || cam->v2f.fmt.pix.pixelformat == V4L2_PIX_FMT_SRGGB10)
   pixel_fmt = IPU_PIX_FMT_GENERIC_16;
  ****
  }‍‍‍‍‍‍‍‍‍‍

• Device tree:

vc_mipi_imx327c: vc_mipi_imx327c@1a {
 compatible = "sony,vc_mipi_imx327c";
 reg = <0x1a>;
 csi_id = <0>;
 clocks = <&clks 200>;
 clock-names = "csi_mclk";
 mclk = <24000000>;
 mclk_source = <0>;
 data-lanes = <1 2 3 4>;
};
&mipi_csi {
 status = "okay";
 ipu_id = <0>;
 csi_id = <0>;
 v_channel = <0>;
 lanes = <4>;
 };
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The result is the following picture (shone a flashlight at the camera):

The video is taken by the imxv4l2videosrc plugin processed by gstreamer and then transferred to the bayer2rgb plugin.

If I try to independently convert the raw buffer to a buyer (every 5 bytes in 4 uint16 numbers) before sending it to the bayer2rgb plugin, I get the following picture (shone a flashlight at the camera):

My question is how to get active pixels from the buffer correctly and do I need to do this? Or maybe I'm wrong and pick up the buffer incorrectly?

P.S. Buffer size 1920 * 1080 * 2 = 4147200 bytes.

[deepakbeniya]

hi @valentinryzhenk 
Did you solved the error?
If yes share the solution please. It will be helpfull for others also.

[joanxie]

pls refer to the document as below:

i.MX6Q capture raw(bayer) data and debayer  

have you tested by ./cap.out first?

[valentinryzhenk]

I checked ./cap.out and this is what happened

The resulting buffer was converted using opencv.

A frame in memory looks like this:
header - 320 bytes
11 lines (vertical offset) - 26400 bytes (1920*1.25*11)
1080 lines - frame (1920*1080*1.25)

I cannot figure out how to remove these vertical stripes.

[dimitarmaznekov]

Did you succeed with this taks ?

[joanxie]

have you applied to the patches in the link I sent to you? since you use the third party board and not use ov5640, maybe you couldn't apply these patches directly but you need to  change unit test mxc_v4l2_capture.c fistly

[valentinryzhenk]

I was looking at the patch and manually changing the settings for my camera. As a result, the unit test mxc_v4l2_capture.c captures a frame from the camera. I cannot figure out how to remove these vertical stripes.

[joanxie]

how about use lower resoltuion? any improvement?