i.MX6 PAL deinterlacer artifact

[herveschmitt]

We use PAL camera sensors (via a TVP5154 video decoder) on i.MX6 and have installed the Freescale patch described by De-interlace Capture Device + a small capture modification to support PAL format :

diff --git a/drivers/mxc/ipu3/ipu_capture.c b/drivers/mxc/ipu3/ipu_capture.c

index c768d8b..578c19d 100644 (file)

--- a/drivers/mxc/ipu3/ipu_capture.c

+++ b/drivers/mxc/ipu3/ipu_capture.c

@@ -137,7 +137,7 @@ ipu_csi_init_interface(struct ipu_soc *ipu, uint16_t width, uint16_t height,

                ipu_csi_write(ipu, csi, 0x40030, CSI_CCIR_CODE_1);

                ipu_csi_write(ipu, csi, 0xFF0000, CSI_CCIR_CODE_3);

        } else if (cfg_param.clk_mode == IPU_CSI_CLK_MODE_CCIR656_INTERLACED) {

-               if (width == 720 && height == 625) {

+               if (width == 720 && (height == 625 || height == 576)) {

                        /* PAL case */

                        /*

                         * Field0BlankEnd = 0x6, Field0BlankStart = 0x2,

1. Dynamic aspects

It globally works as expected for dynamic aspects: with motion, we clearly see odd and even lines shifts without VDI that are significantly reduced with VDI.

2. Static aspects

Unfortunately, static aspects of images are slightly degraded, as shown by the following snapshots :

Without VDI :

With VDI :

• We can see a blue / green line of 2 pixels thick before the last 2 lines. This is clearly the biggest artifact caused by the VDI.
• It seems aliasing is slightly more visible with VDI (it's not really visible on the above snapshots, but we checked that on other camera).
• Fix images with aliasing sometimes seem to move slightly when VDI is active.

Any idea on how to fix the blue / green line ?

Thanks in advance.

[herveschmitt]

The above problem doesn't appear using the standalone application mxc_v4l2_tvin provided by De-interlace Capture Device, but appears using the gstreamer plugin provided by Freescale named mfw_v4lsrc. By checking every ioctl, we have found the root cause of the problem:

• the VDI works OK using a pixel format V4L2_PIX_FMT_UYVY (as used by mxc_v4l2_tvin)
• the VDI works with the above artifact using a pixel format V4L2_PIX_FMT_YUV420 (as used by mfw_v4lsrc)

It is very easy to reproduce the problem in mxc_v4l2_tvin; just change the line:

int g_fmt = V4L2_PIX_FMT_UYVY;

by:

int g_fmt = V4L2_PIX_FMT_YUV420;

...and you will see the artifact !

Fixing the problem in mfw_v4lsrc is unfortunately not as easy as changing the pixel format, and the next plugin for common streaming usage is vpuenc that only accept YUV420 format as input...

Is there a way to make the VDI support YUV420 format without artifact ?

TIA.

[herveschmitt]

Hi all,

Everyone using a BT656 PAL video input might be interested by the following table describing the compliance of i.MX6D&Q VDI with different modes (and using HDMI output, just in case it could influence results). It was establish thanks to the –f option of the mxc_v4l2_tvin utility, running on Linux 3.0.35 with the deinterlacer patch :

Mode	Verdict
/* RGB formats */
#define V4L2_PIX_FMT_RGB332 v4l2_fourcc('R', 'G', 'B', '1') /* 8 RGB-3-3-2 */	KO
#define V4L2_PIX_FMT_RGB444 v4l2_fourcc('R', '4', '4', '4') /* 16 xxxxrrrr ggggbbbb */	KO
#define V4L2_PIX_FMT_RGB555 v4l2_fourcc('R', 'G', 'B', 'O') /* 16 RGB-5-5-5 */	KO
#define V4L2_PIX_FMT_RGB565 v4l2_fourcc('R', 'G', 'B', 'P') /* 16 RGB-5-6-5 */	KO : no display followed by « DMA Busy » forever
#define V4L2_PIX_FMT_RGB555X v4l2_fourcc('R', 'G', 'B', 'Q') /* 16 RGB-5-5-5 BE */	KO
#define V4L2_PIX_FMT_RGB565X v4l2_fourcc('R', 'G', 'B', 'R') /* 16 RGB-5-6-5 BE */	KO
#define V4L2_PIX_FMT_BGR666 v4l2_fourcc('B', 'G', 'R', 'H') /* 18 BGR-6-6-6 */	KO
#define V4L2_PIX_FMT_BGR24 v4l2_fourcc('B', 'G', 'R', '3') /* 24 BGR-8-8-8 */	OK but image is pink / blue
#define V4L2_PIX_FMT_RGB24 v4l2_fourcc('R', 'G', 'B', '3') /* 24 RGB-8-8-8 */	OK but image is pink / blue
#define V4L2_PIX_FMT_BGR32 v4l2_fourcc('B', 'G', 'R', '4') /* 32 BGR-8-8-8-8 */	OK but image is pink / blue
#define V4L2_PIX_FMT_RGB32 v4l2_fourcc('R', 'G', 'B', '4') /* 32 RGB-8-8-8-8 */	OK but image is pink / blue
/* Grey formats */
#define V4L2_PIX_FMT_GREY v4l2_fourcc('G', 'R', 'E', 'Y') /* 8 Greyscale */	KO
#define V4L2_PIX_FMT_Y4 v4l2_fourcc('Y', '0', '4', ' ') /* 4 Greyscale */	KO
#define V4L2_PIX_FMT_Y6 v4l2_fourcc('Y', '0', '6', ' ') /* 6 Greyscale */	KO
#define V4L2_PIX_FMT_Y10 v4l2_fourcc('Y', '1', '0', ' ') /* 10 Greyscale */	KO
#define V4L2_PIX_FMT_Y12 v4l2_fourcc('Y', '1', '2', ' ') /* 12 Greyscale */	KO
#define V4L2_PIX_FMT_Y16 v4l2_fourcc('Y', '1', '6', ' ') /* 16 Greyscale */	KO
/* Grey bit-packed formats */
#define V4L2_PIX_FMT_Y10BPACK v4l2_fourcc('Y', '1', '0', 'B') /* 10 Greyscale bit-packed */	KO
/* Palette formats */
#define V4L2_PIX_FMT_PAL8 v4l2_fourcc('P', 'A', 'L', '8') /* 8 8-bit palette */	KO
/* Luminance+Chrominance formats */
#define V4L2_PIX_FMT_YVU410 v4l2_fourcc('Y', 'V', 'U', '9') /* 9 YVU 4:1:0 */	KO
#define V4L2_PIX_FMT_YVU420 v4l2_fourcc('Y', 'V', '1', '2') /* 12 YVU 4:2:0 */	KO, and unable to launch anything else after
#define V4L2_PIX_FMT_YUYV v4l2_fourcc('Y', 'U', 'Y', 'V') /* 16 YUV 4:2:2 */	OK
#define V4L2_PIX_FMT_YYUV v4l2_fourcc('Y', 'Y', 'U', 'V') /* 16 YUV 4:2:2 */	KO
#define V4L2_PIX_FMT_YVYU v4l2_fourcc('Y', 'V', 'Y', 'U') /* 16 YVU 4:2:2 */	KO
#define V4L2_PIX_FMT_UYVY v4l2_fourcc('U', 'Y', 'V', 'Y') /* 16 YUV 4:2:2 */	OK
#define V4L2_PIX_FMT_VYUY v4l2_fourcc('V', 'Y', 'U', 'Y') /* 16 YUV 4:2:2 */	KO
#define V4L2_PIX_FMT_YUV422P v4l2_fourcc('4', '2', '2', 'P') /* 16 YVU422 planar */	KO : no display followed by « DMA Busy » forever
#define V4L2_PIX_FMT_YUV411P v4l2_fourcc('4', '1', '1', 'P') /* 16 YVU411 planar */	KO
#define V4L2_PIX_FMT_Y41P v4l2_fourcc('Y', '4', '1', 'P') /* 12 YUV 4:1:1 */	KO
#define V4L2_PIX_FMT_YUV444 v4l2_fourcc('Y', '4', '4', '4') /* 16 xxxxyyyy uuuuvvvv */	KO
#define V4L2_PIX_FMT_YUV555 v4l2_fourcc('Y', 'U', 'V', 'O') /* 16 YUV-5-5-5 */	KO
#define V4L2_PIX_FMT_YUV565 v4l2_fourcc('Y', 'U', 'V', 'P') /* 16 YUV-5-6-5 */	KO
#define V4L2_PIX_FMT_YUV32 v4l2_fourcc('Y', 'U', 'V', '4') /* 32 YUV-8-8-8-8 */	KO
#define V4L2_PIX_FMT_YUV410 v4l2_fourcc('Y', 'U', 'V', '9') /* 9 YUV 4:1:0 */	KO
#define V4L2_PIX_FMT_YUV420 v4l2_fourcc('Y', 'U', '1', '2') /* 12 YUV 4:2:0 */	OK with the blue line artifact
#define V4L2_PIX_FMT_HI240 v4l2_fourcc('H', 'I', '2', '4') /* 8 8-bit color */	KO
#define V4L2_PIX_FMT_HM12 v4l2_fourcc('H', 'M', '1', '2') /* 8 YUV 4:2:0 16x16 macroblocks */	KO
#define V4L2_PIX_FMT_M420 v4l2_fourcc('M', '4', '2', '0') /* 12 YUV 4:2:0 2 lines y, 1 line uv interleaved */	KO
/* two planes -- one Y, one Cr + Cb interleaved */
#define V4L2_PIX_FMT_NV12 v4l2_fourcc('N', 'V', '1', '2') /* 12 Y/CbCr 4:2:0 */	OK
#define V4L2_PIX_FMT_NV21 v4l2_fourcc('N', 'V', '2', '1') /* 12 Y/CrCb 4:2:0 */	KO
#define V4L2_PIX_FMT_NV16 v4l2_fourcc('N', 'V', '1', '6') /* 16 Y/CbCr 4:2:2 */	KO
#define V4L2_PIX_FMT_NV61 v4l2_fourcc('N', 'V', '6', '1') /* 16 Y/CrCb 4:2:2 */	KO
/* two non contiguous planes - one Y, one Cr + Cb interleaved */
#define V4L2_PIX_FMT_NV12M v4l2_fourcc('N', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 */	KO
#define V4L2_PIX_FMT_NV12MT v4l2_fourcc('T', 'M', '1', '2') /* 12 Y/CbCr 4:2:0 64x32 macroblocks */	KO
/* three non contiguous planes - Y, Cb, Cr */
#define V4L2_PIX_FMT_YUV420M v4l2_fourcc('Y', 'M', '1', '2') /* 12 YUV420 planar */	KO
/* Bayer formats - see http://www.siliconimaging.com/RGB%20Bayer.htm */
#define V4L2_PIX_FMT_SBGGR8 v4l2_fourcc('B', 'A', '8', '1') /* 8 BGBG.. GRGR.. */	KO
#define V4L2_PIX_FMT_SGBRG8 v4l2_fourcc('G', 'B', 'R', 'G') /* 8 GBGB.. RGRG.. */	KO
#define V4L2_PIX_FMT_SGRBG8 v4l2_fourcc('G', 'R', 'B', 'G') /* 8 GRGR.. BGBG.. */	KO
#define V4L2_PIX_FMT_SRGGB8 v4l2_fourcc('R', 'G', 'G', 'B') /* 8 RGRG.. GBGB.. */	KO
#define V4L2_PIX_FMT_SBGGR10 v4l2_fourcc('B', 'G', '1', '0') /* 10 BGBG.. GRGR.. */	KO
#define V4L2_PIX_FMT_SGBRG10 v4l2_fourcc('G', 'B', '1', '0') /* 10 GBGB.. RGRG.. */	KO
#define V4L2_PIX_FMT_SGRBG10 v4l2_fourcc('B', 'A', '1', '0') /* 10 GRGR.. BGBG.. */	KO
#define V4L2_PIX_FMT_SRGGB10 v4l2_fourcc('R', 'G', '1', '0') /* 10 RGRG.. GBGB.. */	KO
#define V4L2_PIX_FMT_SBGGR12 v4l2_fourcc('B', 'G', '1', '2') /* 12 BGBG.. GRGR.. */	KO
#define V4L2_PIX_FMT_SGBRG12 v4l2_fourcc('G', 'B', '1', '2') /* 12 GBGB.. RGRG.. */	KO
#define V4L2_PIX_FMT_SGRBG12 v4l2_fourcc('B', 'A', '1', '2') /* 12 GRGR.. BGBG.. */	KO
#define V4L2_PIX_FMT_SRGGB12 v4l2_fourcc('R', 'G', '1', '2') /* 12 RGRG.. GBGB.. */	KO
#define V4L2_PIX_FMT_SGRBG10DPCM8 v4l2_fourcc('B', 'D', '1', '0') /* 10bit raw bayer DPCM compressed to 8 bits */	KO
#define V4L2_PIX_FMT_SBGGR16 v4l2_fourcc('B', 'Y', 'R', '2') /* 16 BGBG.. GRGR.. */	KO

Just a few remarks:

• It seems format conversion is performed – by IC – to support other format than the native one which is YCbCr 4:2:2 of BT656.
• 4 different RGB formats are supported (surprisingly), but are not correctly converted as images seem to be composed of pink and blue. Nevertheless, this issue has nothing to do with VDI because the same color conversion occurs without VDI. So, from a VDI point of vue the behavior is correct (deinterlacing works OK).
• YUV420 is the only format for which VDI works on 99% of the image area, but there is a blue (or green depending on the background) artifact on the low edge. This mode is particularly important because it’s the one used by the Freescale source plugin mfw_v4lsrcvpuenc and the only one accepted by vpuenc.

[herveschmitt]

We have just posted a fix here: De-interlace Capture Device

Feedback is welcome (as we test only in PAL, we may not be aware of bugs affecting NTSC for example).