IMX6D output yuv422 video with vsync and hsync

[azmatliu]

Hi community,

   Can imx6D output YUV422(8 bit) video stream just like a camera? We are trying to use the patch given by Patch to Support BT656 and BT1120 Output For i.MX6 BSP .But it seems to work incorrectly, we can't get vsync or hsync signal in this patch because they are embedded.Could you give me some examples to support our use case?

Best regard,

Azmat

[azmatliu]

Qiang_FSL‌Any idea?

[joanxie]

the patch is for rgb656, for the yuv422 output, the bsp supports this, what's your solution? capture the raw data from camera, then output to yuv422 by VPU and IPU?

[azmatliu]

Thank you for your reply.

     Do you mean we can use the bsp directly?Our bsp version is "L4.1.15_2.1_GA" and we want to output yuv422 8-bit data to TI913A-Q1(serializer).Can we just set u-boot bootargs to"console=ttymxc0,115200 root=/dev/mmcblk2p2 video=mxcfb0:dev=lcd,1280x960M@60,if=UYVY16"?How do we connect the DISP_DATA pins?

The pin we connected:

            fsl,pins = <
                MX6QDL_PAD_DI0_DISP_CLK__IPU1_DI0_DISP_CLK 0x10

                MX6QDL_PAD_DI0_PIN2__IPU1_DI0_PIN02        0x10   /* HSYNC */
                MX6QDL_PAD_DI0_PIN3__IPU1_DI0_PIN03        0x10   /* VSYNC */
                MX6QDL_PAD_DISP0_DAT0__IPU1_DISP0_DATA00  0x10
                MX6QDL_PAD_DISP0_DAT1__IPU1_DISP0_DATA01  0x10
                MX6QDL_PAD_DISP0_DAT2__IPU1_DISP0_DATA02  0x10
                MX6QDL_PAD_DISP0_DAT3__IPU1_DISP0_DATA03  0x10
                MX6QDL_PAD_DISP0_DAT4__IPU1_DISP0_DATA04  0x10
                MX6QDL_PAD_DISP0_DAT5__IPU1_DISP0_DATA05  0x10
                MX6QDL_PAD_DISP0_DAT6__IPU1_DISP0_DATA06  0x10
                MX6QDL_PAD_DISP0_DAT7__IPU1_DISP0_DATA07  0x10
            >;

Something we find:

   mxc_ipuv3_fb.c

      -->mxcfb_option_setup()

We can only set "if=" to YUYV16/UYVY16/YVYU16/VYUY16(16-bit data?)

Give us more information about yuv422 8-bit output,please.

[joanxie]

for the hw connection, pls refer to the chapter 4.12.10.4 IPU Display Interface Signal Mapping of data sheet:

https://www.nxp.com/docs/en/data-sheet/IMX6DQCEC.pdf 

the kernel should support yuv422, you can try the uboot commad as you mentions

[azmatliu]

Thanks for your reply.

   I have tried "UYVY16" but it didn't work.I think "UYVY16" means we need to connect 16 data pins,but there are only 8 data pins in our usage scenario.So we should write a driver for "UYVY8"?Any advice is welcomed!

Azmat

[joanxie]

if you need to connect yuv 16bits, try to connect from DISP_DI0 to DISP_DI15

[azmatliu]

We need to use yuv 8-bit because of the hardware limitation.So how do we patch the BSP to support UVVY 8-bit?I don't think we've got the ability to solve this issue.Can we pay for it?

[qiang_li-mpu_se]

You's better use BT656 mode for 8 bits UYVY.

[azmatliu]

I have tried that patch,but we need external VSYNC and HSYNC  signal in our using case(BT656 mode's timing information is embedded on data line:EAV/SAV).

[qiang_li-mpu_se]

Please check the discussion of " Patch to Support BT656 and BT1120 Output For i.MX6 BSP ", there is examlpe for how to add VSYNC and HSYNC on BT656 output.

[azmatliu]

Hi Qiang Li

   I have read the discussion but can't find the example for how to add VSYNC and HSYNC on BT656 output.T_T~~

Would you send me the example for me,please?Thanks a lot.

Regards

Azmat

[qiang_li-mpu_se]

@@ -3123,11 +3123,11 @@ int32_t ipu_init_sync_panel(struct ipu_soc *ipu, int disp, uint32_t pixel_clk,
      DI_SYNC_NONE,  /* offset resolution */
      0,   /* repeat count */
      DI_SYNC_NONE,  /* CNT_CLR_SEL */
-     0,   /* CNT_POLARITY_GEN_EN */
+     1,   /* CNT_POLARITY_GEN_EN */
      DI_SYNC_NONE,  /* CNT_POLARITY_CLR_SEL */
-     DI_SYNC_NONE,  /* CNT_POLARITY_TRIGGER_SEL */
+     DI_BT656_SYNC_BASECLK,  /* CNT_POLARITY_TRIGGER_SEL */
      0,   /* COUNT UP */
-     0  /* COUNT DOWN */
+     2*80  /* COUNT DOWN */
      );
 
    vsync_cnt = DI_BT656_SYNC_IVSYNC;
@@ -3142,11 +3142,11 @@ int32_t ipu_init_sync_panel(struct ipu_soc *ipu, int disp, uint32_t pixel_clk,
      DI_SYNC_NONE,  /* offset resolution */
      0,   /* repeat count */
      DI_SYNC_NONE,  /* CNT_CLR_SEL */
-     0,   /* CNT_POLARITY_GEN_EN */
+     1,   /* CNT_POLARITY_GEN_EN */
      DI_SYNC_NONE,  /* CNT_POLARITY_CLR_SEL */
-     DI_SYNC_NONE,  /* CNT_POLARITY_TRIGGER_SEL */
+     DI_BT656_SYNC_HSYNC,  /* CNT_POLARITY_TRIGGER_SEL */
      0,   /* COUNT UP */
-     0  /* COUNT DOWN */
+     2*10  /* COUNT DOWN */
      );
 
    /* COUNTER_5: first active line */

[azmatliu]

Hi Qiang Li

   Thanks for your reply.

   I have used the patch "L4.1.15_GA2.1.0_Support-BT656-and-BT1120-output-for-iMX6-ipuv3.patch",then modified driver/mxc/ipuv3/ipu_disp.c as you mentioned.But there is still VSYNC or HSYNC signal output yet...

Regards

Azmat

[qiang_li-mpu_se]

If you had use BT656 progressive mode successfully, with the above modification, HSYNC will be pull high for 80 pixel clocks, and VSYNC will be pull high for 10 lines. This is verified working, but if you are used BT656 interlaced mode, it can't work.

Maybe you can attach your ipu_disp.c file here.

[azmatliu]

The Vsync and Hsync did output successfully now but the timing information seemed abnormal and it couldn't meet our requirement.I set bootarg to:

"video=mxcfb0:dev=bt656,1280x960M@50,if=BT656"

and the information from "fbset":

"mode "1280x960-50"
# D: 83.001 MHz, H: 49.406 kHz, V: 49.854 Hz
geometry 1280 960 1280 2880 16
timings 12048 200 72 24 3 128 4
accel false
rgba 5/11,6/5,5/0,0/0
endmode"

I measured the frequency of the PCLK\VSYNC\HSYNC:

PCLK:83MHz(seems correct)

VSYNC:25Hz(not 50Hz?)

HSYNC:30.6kHz(not 49.406kHz?)

By the way,I want to add a fb_videomode in mxc_bt656if.c just like:

    {
    /* User Define Progressive output */
    "BT656-TEST", 50, 1280, 960, 12048,
    200, 72,
    24, 3,
    128, 4,
    FB_SYNC_HOR_HIGH_ACT | FB_SYNC_VERT_HIGH_ACT,
    FB_VMODE_NONINTERLACED,
    FB_MODE_IS_DETAILED,},

But I don't know how do I calculate the pixel clock.

   h_total = left_margin + right_margin + hsync_len + width * 2

               = 200 + 72 + 128 + 1280 * 2 = 2960

   v_total = (upper_margin + lower_margin + vsync_len + height)*h_total

              = (24 + 3 + 4 + 960)*2960 = 2933360

   But what is the next step to calculate "1280"?Can you give me an example for this?Thanks a lot.

Regards

Azmat

[qiang_li-mpu_se]

Please don't use fbset.

Please study the example BT656 progressive timing parameters from mxc_bt656if.c.

An example for 720*480P

+ /* NTSC Progressive output */
+ "BT656-480P", 60, 720, 480, 18518,
+ 36, 9,
+ 0, 0,
+ 276, 1,
+ FB_SYNC_HOR_HIGH_ACT | FB_SYNC_VERT_HIGH_ACT,
+ FB_VMODE_NONINTERLACED,
+ FB_MODE_IS_DETAILED,},

That means the frame is followed, 276 = (1+67+1) * 4:

For the HSYNC/VSYNC adjusting, please read the discussion in https://community.nxp.com/docs/DOC-94019.

[azmatliu]

Hi Qiang Li:
    I got some probled when I was adjusting the timing of 1280x960P(BT656 8-bit):
1. How do I stop horizontal sync siginal in the period of vertical blanking?
2. For function:
_ipu_dc_setup_bt656_progressive(ipu, u_map, y_map, v_map, 0, BT656_IF_DI_MSB,
        bt656_h_start_width,
        bt656_v_start_width_field0, bt656_v_end_width_field0);

loop_frame_blank_line_times = bt656_v_start_width + bt656_v_end_width - 1;
loop_blank_video_times = (bt656_h_start_width - 8) / 2 - 1;

So if I want to generate correct v_sync signal,"v_sync_width" should be "0 < v_sync_width < bt656_v_start_width + bt656_v_end_width" and the "offset" for first active line should be "bt656_v_start_width".Am I right?
 
             vsync_cnt = DI_BT656_SYNC_IVSYNC;
@@ -3033,11 +3026,11 @@ int32_t ipu_init_sync_panel(struct ipu_s
                     DI_SYNC_NONE,    /* offset resolution */
                     0,        /* repeat count */
                     DI_SYNC_NONE,    /* CNT_CLR_SEL */
-                    0,        /* CNT_POLARITY_GEN_EN */
+                    1,        /* CNT_POLARITY_GEN_EN */
                     DI_SYNC_NONE,    /* CNT_POLARITY_CLR_SEL */
-                    DI_SYNC_NONE,    /* CNT_POLARITY_TRIGGER_SEL */
+                    DI_BT656_SYNC_HSYNC,    /* CNT_POLARITY_TRIGGER_SEL */
                     0,        /* COUNT UP */
-                    0        /* COUNT DOWN */
+                    2*v_sync_width        /* COUNT DOWN */
                     );
 
            /* COUNTER_5: first active line */
            _ipu_di_sync_config(ipu,
                    disp,        /* display */
                    DI_BT656_SYNC_AFIELD,        /* counter */
                    0,        /* run count */
                    DI_BT656_SYNC_HSYNC,    /* run_resolution */
-                    2,        /*    offset, 2 means at least 2 blank lines from VSYNC to data */
+                    bt656_v_start_width,        /*    offset, 2 means at least 2 blank lines from VSYNC to data */
                    DI_BT656_SYNC_HSYNC,    /* offset resolution */
                    1,    /* repeat count */
                    DI_BT656_SYNC_IVSYNC,        /* CNT_CLR_SEL */
                    0,        /* CNT_POLARITY_GEN_EN */
                    DI_SYNC_NONE,    /* CNT_POLARITY_CLR_SEL */
                    DI_SYNC_NONE,    /* CNT_POLARITY_TRIGGER_SEL */
                    0,        /* COUNT UP */
                    0        /* COUNT DOWN */
                    );

[qiang_li-mpu_se · Accepted solution]

COUNTER_2 means the signals on DI_PIN2, and COUNTER_3 means the signals on DI_PIN3.

The COUNT UP, COUNT DOWN, offset can be used to adjust the HSYNC/VSYNC position and width, but you also need check the register bit width, there is limitation for such use case.

[azmatliu]

There are so much helpfull information in the discussion but I've still got some prroblem:
    Though I want to output 960P(1280x960),I am planning to add a config for 720p(1280x720) output as the discussion in https://community.nxp.com/docs/DOC-94019. mentioned first.But I don't how to modify the code for the clock setting because I am using L_4.1.15_2.1.0_GA.

The patch I don't know how to apply:
--- a/drivers/mxc/ipu3/ipu_disp.c
+++ b/drivers/mxc/ipu3/ipu_disp.c
@@ -2600,13 +2600,13 @@ int32_t ipu_init_sync_panel(struct ipu_soc *ipu, int disp, uint32_t pixel_clk,
  } else if ((pixel_fmt == IPU_PIX_FMT_BT656) || (pixel_fmt == IPU_PIX_FMT_BT1120)) {
   pixel_clk = pixel_clk / 10000;
   pixel_clk = pixel_clk * 10000;
-  if (pixel_clk < 74000000) {
-   rounded_pixel_clk = 108000000; /* For PAL and NTSC */
-
-   clk_set_rate(di_parent, 540000000);
-   clk_set_rate(ipu->di_clk[disp], rounded_pixel_clk);
-   clk_set_parent(&ipu->pixel_clk[disp], ipu->di_clk[disp]);
-  } else {
+//  if (pixel_clk < 74000000) {
+//   rounded_pixel_clk = 108000000; /* For PAL and NTSC */
+//
+//   clk_set_rate(di_parent, 540000000);
+//   clk_set_rate(ipu->di_clk[disp], rounded_pixel_clk);
+//   clk_set_parent(&ipu->pixel_clk[disp], ipu->di_clk[disp]);
+//  } else {
    rounded_pixel_clk = pixel_clk * 2;
    rounded_parent_clk = clk_round_rate(di_parent,
       rounded_pixel_clk);
@@ -2622,7 +2622,7 @@ int32_t ipu_init_sync_panel(struct ipu_soc *ipu, int disp, uint32_t pixel_clk,
     clk_round_rate(ipu->di_clk[disp], pixel_clk);
    clk_set_rate(ipu->di_clk[disp], rounded_pixel_clk);
    clk_set_parent(&ipu->pixel_clk[disp], ipu->di_clk[disp]);
-  }
+//  }
  } else {
   /* try ipu clk first*/
   dev_dbg(ipu->dev, "try ipu internal clk\n");

The patch for 720P output test:

                /* COUNTER_1: basic clock */
             _ipu_di_sync_config(ipu,
                     disp,        /* display */
@@ -3018,7 +3037,7 @@ int32_t ipu_init_sync_panel(struct ipu_s
                     DI_SYNC_NONE,    /* CNT_POLARITY_CLR_SEL */
                     DI_BT656_SYNC_BASECLK,    /* CNT_POLARITY_TRIGGER_SEL */
                     0,        /* COUNT UP */
-                    2*80        /* COUNT DOWN */
+                    2 * 80        /* COUNT DOWN */
                     );
 
             vsync_cnt = DI_BT656_SYNC_IVSYNC;
@@ -3037,7 +3056,7 @@ int32_t ipu_init_sync_panel(struct ipu_s
                     DI_SYNC_NONE,    /* CNT_POLARITY_CLR_SEL */
                     DI_BT656_SYNC_HSYNC,    /* CNT_POLARITY_TRIGGER_SEL */
                     0,        /* COUNT UP */
-                    2*10        /* COUNT DOWN */
+                    2 * 10        /* COUNT DOWN */
                     );
 
             /* COUNTER_5: first active line */
Index: linux-imx/drivers/video/fbdev/mxc/mxc_bt656if.c
===================================================================
--- linux-imx.orig/drivers/video/fbdev/mxc/mxc_bt656if.c
+++ linux-imx/drivers/video/fbdev/mxc/mxc_bt656if.c
@@ -81,6 +81,15 @@ static struct fb_videomode bt656if_moded
     FB_SYNC_HOR_HIGH_ACT | FB_SYNC_VERT_HIGH_ACT,
     FB_VMODE_NONINTERLACED,
     FB_MODE_IS_DETAILED,},
+    {
+    /* User Define Progressive output */
+    "BT656-720P", 60, 1280, 720, 20833,
+    25, 5,
+    0, 0,
+    732, 1,
+    0,
+    FB_VMODE_NONINTERLACED,
+    FB_MODE_IS_DETAILED,},
 };
 static int bt656if_modedb_sz = ARRAY_SIZE(bt656if_modedb);
 
Index: linux-imx/arch/arm/mach-imx/clk-imx6q.c
===================================================================
--- linux-imx.orig/arch/arm/mach-imx/clk-imx6q.c
+++ linux-imx/arch/arm/mach-imx/clk-imx6q.c
@@ -796,7 +796,8 @@ static void __init imx6q_clocks_init(str
         imx_clk_set_parent(clk[IMX6QDL_CLK_LDB_DI1_SEL], clk[IMX6QDL_CLK_PLL2_PFD0_352M]);
     }
     /* ipu clock initialization */
-    imx_clk_set_parent(clk[IMX6QDL_CLK_IPU1_DI0_PRE_SEL], clk[IMX6QDL_CLK_PLL3_PFD1_540M]);  /* For CVBS 27MHz clock */
+    //imx_clk_set_parent(clk[IMX6QDL_CLK_IPU1_DI0_PRE_SEL], clk[IMX6QDL_CLK_PLL3_PFD1_540M]);  /* For CVBS 27MHz clock */
+    imx_clk_set_parent(clk[IMX6QDL_CLK_IPU1_DI0_PRE_SEL], clk[IMX6QDL_CLK_PLL5_VIDEO_DIV]);
     imx_clk_set_parent(clk[IMX6QDL_CLK_IPU1_DI1_PRE_SEL], clk[IMX6QDL_CLK_PLL5_VIDEO_DIV]);
     imx_clk_set_parent(clk[IMX6QDL_CLK_IPU2_DI0_PRE_SEL], clk[IMX6QDL_CLK_PLL5_VIDEO_DIV]);
     imx_clk_set_parent(clk[IMX6QDL_CLK_IPU2_DI1_PRE_SEL], clk[IMX6QDL_CLK_PLL5_VIDEO_DIV]);

[azmatliu]

I forgot to add pinctrl in my device tree...

        pinctrl_ipu1_bt656: ipu1grp-3 {
            fsl,pins = <
                MX6QDL_PAD_DI0_DISP_CLK__IPU1_DI0_DISP_CLK 0x10
                MX6QDL_PAD_DI0_PIN2__IPU1_DI0_PIN02        0x10
                MX6QDL_PAD_DI0_PIN3__IPU1_DI0_PIN03        0x10
                MX6QDL_PAD_DISP0_DAT0__IPU1_DISP0_DATA00  0x10
                MX6QDL_PAD_DISP0_DAT1__IPU1_DISP0_DATA01  0x10
                MX6QDL_PAD_DISP0_DAT2__IPU1_DISP0_DATA02  0x10
                MX6QDL_PAD_DISP0_DAT3__IPU1_DISP0_DATA03  0x10
                MX6QDL_PAD_DISP0_DAT4__IPU1_DISP0_DATA04  0x10
                MX6QDL_PAD_DISP0_DAT5__IPU1_DISP0_DATA05  0x10
                MX6QDL_PAD_DISP0_DAT6__IPU1_DISP0_DATA06  0x10
                MX6QDL_PAD_DISP0_DAT7__IPU1_DISP0_DATA07  0x10
            >;
        };

External VSYNC and HSYNC signal did output now.