imx8mq csi raw10 rx problem

[martinkepplinge]

I try to capture mipi raw10 data from a sensor (so probably 5 bytes are 4 pixels) in bayer. the mxc-mipi-csi2_yav driver should be no problem afaik.
I apply the following to the mx6s_capture.c csi driver:

--- a/drivers/media/platform/mxc/capture/mx6s_capture.c
+++ b/drivers/media/platform/mxc/capture/mx6s_capture.c
@@ -266,6 +266,12 @@ static struct mx6s_fmt formats[] = {
    .pixelformat = V4L2_PIX_FMT_SBGGR8,
    .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8,
    .bpp = 1,
+   }, {
+   .name = "RAWRGB10 (SBGGR10)",
+   .fourcc = V4L2_PIX_FMT_SBGGR10,
+   .pixelformat = V4L2_PIX_FMT_SBGGR10,
+   .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
+   .bpp = 2,
    }
    };

@@ -601,6 +607,10 @@ static void csi_set_imagpara(struct mx6s_csi_dev *csi,
{
    int imag_para = 0;
    unsigned long cr3 = __raw_readl(csi->regbase + CSI_CSICR3);
+   int rx_count;
+
+   rx_count = (width * height) >> 2;
+   __raw_writel(rx_count, csi->regbase + CSI_CSIRXCNT);

   imag_para = (width << 16) | height;
   __raw_writel(imag_para, csi->regbase + CSI_CSIIMAG_PARA);
@@ -838,6 +848,7 @@ static int mx6s_configure_csi(struct mx6s_csi_dev *csi_dev)
    switch (csi_dev->fmt->pixelformat) {
    case V4L2_PIX_FMT_YUV32:
    case V4L2_PIX_FMT_SBGGR8:
+   case V4L2_PIX_FMT_SBGGR10:
    width = pix->width;
    break;
    case V4L2_PIX_FMT_UYVY:
@@ -857,7 +868,6 @@ static int mx6s_configure_csi(struct mx6s_csi_dev *csi_dev)
    if (csi_dev->csi_mipi_mode == true) {
    cr1 = csi_read(csi_dev, CSI_CSICR1);
    cr1 &= ~BIT_GCLK_MODE;
-   csi_write(csi_dev, cr1, CSI_CSICR1);

    cr18 = csi_read(csi_dev, CSI_CSICR18);
    cr18 &= ~BIT_MIPI_DATA_FORMAT_MASK;
@@ -871,12 +881,17 @@ static int mx6s_configure_csi(struct mx6s_csi_dev *csi_dev)
    case V4L2_PIX_FMT_SBGGR8:
    cr18 |= BIT_MIPI_DATA_FORMAT_RAW8;
    break;
+   case V4L2_PIX_FMT_SBGGR10:
+   cr18 |= BIT_MIPI_DATA_FORMAT_RAW10;
+   cr1 |= BIT_PIXEL_BIT;
+   break;
    default:
    pr_debug(" fmt not supported\n");
    return -EINVAL;
    }

    csi_write(csi_dev, cr18, CSI_CSICR18);
+   csi_write(csi_dev, cr1, CSI_CSICR1);
    }
    return 0;
    }
@@ -2002,7 +2017,7 @@ static const struct mx6s_csi_soc mx6sl_soc = {
    };
    static const struct mx6s_csi_soc mx8mq_soc = {
    .rx_fifo_rst = true,
-    .baseaddr_switch = 0x80030,
+    .baseaddr_switch = 0x80000,
    };

you can see I switched baseaddr switch from "vsync" to "atomically by DMA completed" because I don't get EOF (or the appropriate) interrupts for vsync.

the dts description for csi1 I:

mipi_csi_1: mipi_csi1@30a70000 {
compatible = "fsl,mxc-mipi-csi2_yav";
reg = <0x30a70000 0x1000>; /* MIPI CSI1 Controller base addr */
interrupts = <GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&clk IMX8MQ_CLK_CSI1_CORE>,
<&clk IMX8MQ_CLK_CSI1_ESC>,
<&clk IMX8MQ_CLK_CSI1_PHY_REF>,
<&clk IMX8MQ_CLK_CLKO2>;
clock-names = "clk_core", "clk_esc", "clk_pxl", "clk_clko2";
assigned-clocks = <&clk IMX8MQ_CLK_CSI1_CORE>,
<&clk IMX8MQ_CLK_CSI1_PHY_REF>,
<&clk IMX8MQ_CLK_CSI1_ESC>;
assigned-clock-rates = <266000000>, <150000000>, <66000000>;
assigned-clock-parents = <&clk IMX8MQ_SYS1_PLL_266M>,
<&clk IMX8MQ_SYS2_PLL_1000M>,
<&clk IMX8MQ_SYS1_PLL_800M>;
power-domains = <&pgc_mipi_csi1>;
csis-phy-reset = <&src 0x4c 7>;
phy-gpr = <&iomuxc_gpr 0x88>;
status = "okay";

port {
#address-cells = <1>;
#size-cells = <0>;
mipi1_sensor_ep: endpoint@1 {
reg = <1>;
remote-endpoint = <&camera1_ep>;
data-lanes = <1 2>;
};

csi1_mipi_ep: endpoint@2 {
reg = <2>;
remote-endpoint = <&csi1_ep>;
};
};
};

csi1_bridge: csi1_bridge@30a90000 {
compatible = "fsl,imx8mq-csi", "fsl,imx6s-csi";
reg = <0x30a90000 0x10000>;
interrupts = <GIC_SPI 42 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&clk IMX8MQ_CLK_DUMMY>,
<&clk IMX8MQ_CLK_CSI1_ROOT>,
<&clk IMX8MQ_CLK_DUMMY>;
clock-names = "disp-axi", "csi_mclk", "disp_dcic";
power-domains = <&pgc_mipi_csi1>;
interconnects = <&noc IMX8MQ_ICM_CSI1 &noc IMX8MQ_ICS_DRAM>;
fsl,mipi-mode;
status = "okay";

port{
csi1_ep: endpoint {
remote-endpoint = <&csi1_mipi_ep>;
};
};
};


the sensor driver uses MEDIA_BUS_FMT_SBGGR10_1X10. now CSI says raw10 to be (converted?) and padded to 16bits per pixel, so I used 2bpp in mx6s_capture.c.

When capturing one frame of 640x480 now, the size is indeed 2x640x480 bytes but it's actually 2 frames concatinated, with each one having obviously 1byte per pixel. (Therefore of course the data is not correct too because the sensor sends raw10):

v4l2-ctl -d 0 --set-fmt-video=width=640,height=480,pixelformat=BG10
v4l2-ctl -d 0 --stream-mmap --stream-to=640.raw --stream-count=1

I attach `hexdump -v 640.raw > 640.raw.txt` and the png of it.

What can I do for RAW10 to be correctly processed by mipi and the csi bridge? Is anything missing here? thank you very much in advance!

[joanxie]

+   .name = "RAWRGB10 (SBGGR10)",
+   .fourcc = V4L2_PIX_FMT_SBGGR10,
+   .pixelformat = V4L2_PIX_FMT_SBGGR10,
+   .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10,
+   .bpp = 2,

how about setting bpp=1? and don't forget enable two-8bit-sensor-mode in the dts file

[martinkepplinge]

thank you. that helped. 2bpp is indeed needed and shows a "correct" frame, with two-8bit-mode.

note that we had to fix the hardcoded 640x960 that's set in mx6s_capture to instead reflect the actual mode.

one problem persists though: the color range is not correct. white is grey and black is not fully black. see the attached raw file of 640x480 with a color bars test pattern.

[joanxie]

why did you set baseaddr_switch to 0? which mean set BASEADDR_SWITCH_EN = 0, not care BASEADDR_SWITCH_SEL. The frame buffer only used one and the data is refresh by the embedded DMA. This is not used in the driver; but we have tested that the best way to set 0x80030, how about set to the default value 0x80030?

[martinkepplinge]

absolutely. that's really only been for testing purposes. for correct modes, the default setting is even needed in order to all frames to be correctly in-sync. That's unrelated to a reduced color-space we currently see though.

Thanks a lot for having a look here.

[martinkepplinge]

to follow up, here's what I receive in 1280x720 mode: the second frame isn't complete:

[joanxie]

let me update, your first question is about you set 640x480, but you get 2x640x480, then you fix this issue when you set bpp=1, right? now the new issue is that you couldn't get the completed second frame and correct color when you set 1280x720 frame size? right?

[martinkepplinge]

oh thanks for asking! no, 2bpp is definitely correct for a 10-bit bayer format. it obviously doesn't fit into 1bpp and when looking at the data, it looks like correctly padded to 16bit. your previous answer helped. I didn't know that two-8bit is correct in that case (although it *is* mentioned correctly in the RM now that I know it

the size-issue could be resolved: previously  I called the problem "not completed second frame". with 2bpp it's "not completed frame": "imag_para = (640 << 16) | 960;" was hard-coded and changing that to the actual size (allowing more data) fixes the problem.

for the color-issue I'm actually not even sure anymore: "v4l2-ctl --stream-to-host" and viewing with "qvidcap -p" on the host gives correct colors. Does that "debayer only" (and no other post-processing of the color space)? I'm not even sure yet.

to sum up: This ticket can actually be marked as resolved. I face minor other issues I could open new tickets for.