i.MX6Q+ADV7481: Capturing YUV420, but from script setting up YUV422

willhoie · ‎03-23-2018

I am using an ADV7481 that connects via MIPI to an i.MX6Q (running Linux) over 4 CSI lanes. I am attempting to set up the ADV7481 for HDMI input, YUV 422 8-bit, 4-Lane CSI output. I originally posted this question in the Analog Devices video and Linux forums, but I am still stuck even with their help.

I am using the “:08-08 Free-run MIPI TxA CSI 4-Lane - YUV422 8-Bit, 1280x720p 60Hz:” snippet from the ADV7481ES3C-VER.3.6c.txt file for the ADV7481 required settings in order to generate a test pattern (colorbars) over the MIPI connection to the i.MX6.

I am using the driver developed by Wally Yeh, and discussed here. The driver has been working so far, and I have extended it with the script I just mentioned above. Wally had been using it for 2-Lane capture, but I was able to set it up for 4 lanes, and I have confirmed that in the lab.

I can communicate with the part successfully, and can program the EDID successfully (verified with a GoPro camera hooked up to it), so I know communication is good between the i.MX6 and ADV7481.

However, when I capture the test pattern, it has obvious defects. I’m not sure what it could be, but I am also unsure what the color pattern is actually supposed to look like at this resolution. I have attached a screen grab of the test pattern I am getting. It looks like the color bar pattern is being repeated 8 times, and I don't think that's right.

Looking at the binary data for this raw video capture, the format actually looks like yuv420, not yuv422, which is very strange. The settings are clearly yuv422 going to the ADV7481, so this is the most important issue to resolve at first. Why am I receiving what appears to be yuv420 back? I am using ffplay (from the ffmpeg project) in order to play the raw video capture, and it claims that the color format is also yuv420 when it attempts to discover the format automatically. This software has been used to view other video captures in yuv422 format, so I trust it is not being faked about the format.

Command line to run raw video file:

ffplay.exe -f rawvideo -pixel_format yuv420p -video_size 1280x720 test-color-bars-1280x720-yuv422.yuv

Another developer analyzed the binary data when I ran the same script (except set for solid blue screen instead of color bars), and he also concluded that the format was actually yuv420 instead of yuv422:

“Looks like YUV 4:2:0 (https://en.wikipedia.org/wiki/Chroma_subsampling)

I see 0x0f for the first 0xe1000 bytes(0xe1000 == 921600 == 1280*720)

0xC4 for the next 0x38400 bytes(0x38400 == 230400 == (1280 * 720 / 4))

0x75 for the next 0x38400 bytes

”

I have attached both the color bar raw video file, and the solid blue raw video file in separate zipped attachments.

Here is a link to the post I made in the Analog Devices Video forum, and here is the one I made in the Analog Devices Linux forum.

In summary, I’d like to know what could possibly cause the repeating color bar pattern in the i.MX6, given the setting from that script, and why the received format appears to be YUV 420 instead of YUV 422 like the script is attempting to set up in the ADV7841.

My hunch is that the ADV7481 is set up correctly, but I'm missing something in the processing chain in the i.MX6. Hope that makes sense.

Here is what I have from a 1920x1080p test run, and this has 3 repeating test patterns. The settings I am putting in the ADV7481 are the same except for the video resolution.

Thank you for taking a look!

willhoie · ‎04-25-2018

I was able to get to the bottom of the pixel format issue, and it was between the i.MX gstreamer plugin and the MXC capture driver in the V4L2 stack. When the gstreamer plugin would use the VIDIOC_G_FMT ioctl to get the pixel format, the MXC capture driver would not report back the format from my camera driver, but instead would report back the YUV420 format from it's own internal default settings. There was no way for me to override that.

So, I modified the MXC capture driver so that it would first query the camera sensor pixel format, and if it was different, return the camera pixel format. Otherwise it would return the default as it had in the past (even though I think that is a bug anyway).

But I am still getting the same issue with my color bar test pattern. For 720p, even though my camera is set to send 1280x720 width and height, my gstreamer output is 800x720. So when it is viewed as 1280x720 (like it is supposed to be) it looks like this:

But when viewed at 800x720, it looks right:

Even more strange, is when looking at a 1080p camera capture, you can see that it isn't 3 repeating patterns (like I showed at the end of my original post), but 9!:

(repost the 1080p colorbars):

That's 9 images in one frame. If you look at the 1080p colorbars, you can see now that the smaller frames are stacked on top of one another to make it look seamless, but it really is 9 images.

I have dumped all of the IPU and MIPI CSI-2 registers I can see. Also dumped the CPMEM mega-words in the IDMAC. Everything looks right in the settings. Here is a dmesg dump showing them:

[ 157.783858] In MVC: mxc_v4l2_master_attach
[ 157.783874]    slave.name = adv748x
[ 157.783883]    master.name = mxc_v4l2_cap0
[ 157.783898] End of mxc_v4l2_master_attach: v2f pix widthxheight 288 x 352
[ 157.783909] End of mxc_v4l2_master_attach: crop_bounds widthxheight 1280 x 720
[ 157.783919] End of mxc_v4l2_master_attach: crop_defrect widthxheight 1280 x 720
[ 157.783927] End of mxc_v4l2_master_attach: crop_current widthxheight 1280 x 720
[ 157.783954] adv748x 1-0070: adv7480 is found
[ 184.289911]
[ 184.289911] In MVC: mxc_v4l_open
[ 184.289929]    device name is Mxc Camera
[ 184.289943]    clock_curr=mclk=24000000
[ 184.289953] End of mxc_v4l_open: v2f pix widthxheight 288 x 352
[ 184.289963] End of mxc_v4l_open: crop_bounds widthxheight 1280 x 720
[ 184.289973] End of mxc_v4l_open: crop_defrect widthxheight 1280 x 720
[ 184.289982] End of mxc_v4l_open: crop_current widthxheight 1280 x 720
[ 184.289990] On Open: Input to ipu size is 1280 x 720
[ 184.290024] imx-ipuv3 2400000.ipu: CSI_SENS_CONF = 0x00000A00
[ 184.290036] imx-ipuv3 2400000.ipu: CSI_ACT_FRM_SIZE = 0x02CF04FF
...
[ 185.183725] In MVC: mxc_v4l_do_ioctl 80085617
[ 185.183734]    case VIDIOC_G_STD
[ 185.183742] In mxc_v4l2_g_std
[ 185.183752] In MVC:mxc_v4l_ioctl
[ 185.183761] In MVC: mxc_v4l_do_ioctl 40085618
[ 185.183769]    case VIDIOC_S_STD
[ 185.183777] In mxc_v4l2_s_std 0
[ 185.183786] ERROR: unrecognized std! 0 (PAL=ff, NTSC=b000
[ 185.189195] In MVC:mxc_v4l_ioctl
[ 185.189205] In MVC: mxc_v4l_do_ioctl c0cc5604
[ 185.189214]    case VIDIOC_G_FMT
[ 185.189223] In MVC: mxc_v4l2_g_fmt type=1
[ 185.189232]    type is V4L2_BUF_TYPE_VIDEO_CAPTURE
[ 185.189241]    mismatch in v2f and camera pixel format, using camera
[ 185.189251] End of mxc_v4l2_g_fmt: v2f pix widthxheight 288 x 352
[ 185.189259] End of mxc_v4l2_g_fmt: crop_bounds widthxheight 720 x 625
[ 185.189269] End of mxc_v4l2_g_fmt: crop_defrect widthxheight 1280 x 720
[ 185.189278] End of mxc_v4l2_g_fmt: crop_current widthxheight 720 x 576
[ 185.189287] In MVC:mxc_v4l_ioctl
[ 185.189296] In MVC: mxc_v4l_do_ioctl c02c564a
[ 185.191635] find_adv7480_support_mode : choose width is 1280, height is 720
[ 185.191831] In MVC:mxc_v4l_ioctl
[ 185.191843] In MVC: mxc_v4l_do_ioctl c0045627
[ 185.191853]    case VIDIOC_S_INPUT
[ 185.191861] In MVC:mxc_streamoff
[ 185.191871] In MVC:mxc_v4l_ioctl
[ 185.191881] In MVC: mxc_v4l_do_ioctl c0cc5616
[ 185.191889]    case VIDIOC_S_PARM
[ 185.191896] In mxc_v4l2_s_param
[ 185.191906]    Current capabilities are 1001
[ 185.191915]    Current capturemode is 0 change to 0
[ 185.191923]    Current framerate is 60 change to 60
[ 185.191933] ioctl_s_parm : type is 1
[ 185.203701] find_adv7480_support_mode : choose width is 1280, height is 720
[ 185.203716] adv7480_change_mode_direct: mode is 3
[ 185.933669] adv7480_init_mode in ioctl_s_parm, ret is 0
[ 185.933687]    clock_curr=mclk=24000000
[ 185.933697]    clock_curr=mclk=-1
[ 185.933709]    g_fmt_cap returns widthxheight of input as 1280 x 720
[ 185.933743] imx-ipuv3 2400000.ipu: CSI_SENS_CONF = 0x00000A10
[ 185.933756] imx-ipuv3 2400000.ipu: CSI_ACT_FRM_SIZE = 0x02CF04FF
[ 185.933780] In MVC:mxc_v4l_ioctl
[ 185.933793] In MVC: mxc_v4l_do_ioctl c0cc5605
[ 185.933801]    case VIDIOC_S_FMT
[ 185.933808] In MVC: mxc_v4l2_s_fmt
[ 185.933816]    type=V4L2_BUF_TYPE_VIDEO_CAPTURE
[ 185.933829] End of mxc_v4l2_s_fmt: v2f pix widthxheight 1280 x 720
...
[ 186.348656] In MVC: mxc_v4l_do_ioctl 40045612
[ 186.348664]    case VIDIOC_STREAMON
[ 186.348672] In MVC:mxc_streamon
[ 186.367137] imx-ipuv3 2400000.ipu: init channel = 15
[ 186.367162] imx-ipuv3 2400000.ipu: ipu busfreq high requst.
[ 186.367190] imx-ipuv3 2400000.ipu: initializing idma ch 0 @ b08c0000
[ 186.367218] imx-ipuv3 2400000.ipu: ch 0 word 0 - 00000000 00000000 00000000 E0001800 000B3C9F
[ 186.367234] imx-ipuv3 2400000.ipu: ch 0 word 1 - 071E0000 00E3C000 0147C000 00027FC0 00000000
[ 186.367246] imx-ipuv3 2400000.ipu: PFS 0xa,
[ 186.367257] imx-ipuv3 2400000.ipu: BPP 0x3,
[ 186.367269] imx-ipuv3 2400000.ipu: NPB 0x1f
[ 186.367280] imx-ipuv3 2400000.ipu: FW 1279,
[ 186.367291] imx-ipuv3 2400000.ipu: FH 719,
[ 186.367303] imx-ipuv3 2400000.ipu: EBA0 0x38f00000
[ 186.367314] imx-ipuv3 2400000.ipu: EBA1 0x38f00000
[ 186.367326] imx-ipuv3 2400000.ipu: Stride 2559
[ 186.367336] imx-ipuv3 2400000.ipu: scan_order 0
[ 186.367345] imx-ipuv3 2400000.ipu: uv_stride 0
[ 186.367356] imx-ipuv3 2400000.ipu: u_offset 0x0
[ 186.367367] imx-ipuv3 2400000.ipu: v_offset 0x0
[ 186.367377] imx-ipuv3 2400000.ipu: Width0 0+1,
[ 186.367386] imx-ipuv3 2400000.ipu: Width1 0+1,
[ 186.367397] imx-ipuv3 2400000.ipu: Width2 0+1,
[ 186.373717] imx-ipuv3 2400000.ipu: Width3 0+1,
[ 186.373731] imx-ipuv3 2400000.ipu: Offset0 0,
[ 186.373743] imx-ipuv3 2400000.ipu: Offset1 0,
[ 186.373754] imx-ipuv3 2400000.ipu: Offset2 0,
[ 186.373764] imx-ipuv3 2400000.ipu: Offset3 0
[ 186.373781] eba 38300000
[ 186.373792] eba 38500000

And here is a dump of the IPU registers after I get the first frame:

[ 186.373885] In MVC:mxc_v4l_ioctl
[ 186.373898]    case VIDIOC_DQBUF
[ 186.373907] In MVC:mxc_v4l_dqueue
[ 186.417511] In MVC:camera_callback
[ 186.417531] eba 38700000
[ 186.417573] imx-ipuv3 2400000.ipu: IPU_CONF =        0x10000101
[ 186.417586] imx-ipuv3 2400000.ipu: IDMAC_CONF =        0x0000002F
[ 186.417598] imx-ipuv3 2400000.ipu: IDMAC_CHA_EN1 =        0x00000001
[ 186.417609] imx-ipuv3 2400000.ipu: IDMAC_CHA_EN2 =        0x00000000
[ 186.417619] imx-ipuv3 2400000.ipu: IDMAC_CHA_PRI1 =    0x18800003
[ 186.417629] imx-ipuv3 2400000.ipu: IDMAC_CHA_PRI2 =    0x00000000
[ 186.417638] imx-ipuv3 2400000.ipu: IDMAC_BAND_EN1 =    0x00000000
[ 186.417649] imx-ipuv3 2400000.ipu: IDMAC_BAND_EN2 =    0x00000000
[ 186.417659] imx-ipuv3 2400000.ipu: IPU_CHA_DB_MODE_SEL0 =    0x00000001
[ 186.417669] imx-ipuv3 2400000.ipu: IPU_CHA_DB_MODE_SEL1 =    0x00000000
[ 186.417680] imx-ipuv3 2400000.ipu: IPU_CHA_TRB_MODE_SEL0 =    0x00000000
[ 186.417690] imx-ipuv3 2400000.ipu: IPU_CHA_TRB_MODE_SEL1 =    0x00000000
[ 186.417701] imx-ipuv3 2400000.ipu: DMFC_WR_CHAN =        0x00000090
[ 186.417711] imx-ipuv3 2400000.ipu: DMFC_WR_CHAN_DEF =    0x202020F6
[ 186.417721] imx-ipuv3 2400000.ipu: DMFC_DP_CHAN =        0x00009694
[ 186.417732] imx-ipuv3 2400000.ipu: DMFC_DP_CHAN_DEF =    0x2020F6F6
[ 186.417741] imx-ipuv3 2400000.ipu: DMFC_IC_CTRL =        0x00000002
[ 186.417751] imx-ipuv3 2400000.ipu: IPU_FS_PROC_FLOW1 =    0x00000000
[ 186.417760] imx-ipuv3 2400000.ipu: IPU_FS_PROC_FLOW2 =    0x00000000
[ 186.417770] imx-ipuv3 2400000.ipu: IPU_FS_PROC_FLOW3 =    0x00000000
[ 186.417780] imx-ipuv3 2400000.ipu: IPU_FS_DISP_FLOW1 =    0x00000000
[ 186.417790] imx-ipuv3 2400000.ipu: IPU_VDIC_VDI_FSIZE =    0x00000000
[ 186.417799] imx-ipuv3 2400000.ipu: IPU_VDIC_VDI_C =    0x00000000
[ 186.417809] imx-ipuv3 2400000.ipu: IPU_IC_CONF =        0x00000000
[ 186.417818] imx-ipuv3 2400000.ipu: IPU_IC_PRP_ENC_RSC =        0x20002000
[ 186.417828] imx-ipuv3 2400000.ipu: IPU_IC_PRP_VF_RSC =        0x20002000
[ 186.417837] imx-ipuv3 2400000.ipu: IPU_IC_PP_RSC =        0x20002000
[ 186.417846] imx-ipuv3 2400000.ipu: IC_IDMAC_1 =        0x00000000
[ 186.417855] imx-ipuv3 2400000.ipu: IC_IDMAC_2 =        0x00000000
[ 186.417864] imx-ipuv3 2400000.ipu: IC_IDMAC_3 =        0x00000000
[ 186.417873] imx-ipuv3 2400000.ipu: IC_IDMAC_4 =        0x00000000
[ 186.417883] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_1 =    0x00000001
[ 186.417894] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_2 =    0x00000000
[ 186.417904] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_3 =    0x00000000
[ 186.417914] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_4 =    0x00000000
[ 186.417924] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_5 =    0xBFFEFF2F
[ 186.417934] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_6 =    0x001FFF02
[ 186.417944] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_7 =    0x00000000
[ 186.417953] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_8 =    0x00000000
[ 186.417963] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_9 =    0xDC000001
[ 186.417973] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_10 =    0x777F000F
[ 186.417983] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_11 =    0x00000000
[ 186.417992] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_12 =    0x00000000
[ 186.418002] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_13 =    0x00000000
[ 186.418012] imx-ipuv3 2400000.ipu: IPU_INT_CTRL_14 =    0x00000000
[ 186.418021] imx-ipuv3 2400000.ipu: IPU_INT_STAT_1 =    0x00000000
[ 186.418031] imx-ipuv3 2400000.ipu: IPU_INT_STAT_2 =    0x00000000
[ 186.418041] imx-ipuv3 2400000.ipu: IPU_INT_STAT_3 =    0x00000001
[ 186.418050] imx-ipuv3 2400000.ipu: IPU_INT_STAT_4 =    0x00000000
[ 186.418060] imx-ipuv3 2400000.ipu: IPU_INT_STAT_5 =    0x00000000
[ 186.418069] imx-ipuv3 2400000.ipu: IPU_INT_STAT_6 =    0x00000000
[ 186.418079] imx-ipuv3 2400000.ipu: IPU_INT_STAT_7 =    0x00000000
[ 186.418089] imx-ipuv3 2400000.ipu: IPU_INT_STAT_8 =    0x00000000
[ 186.418098] imx-ipuv3 2400000.ipu: IPU_INT_STAT_9 =    0x00000000
[ 186.418108] imx-ipuv3 2400000.ipu: IPU_INT_STAT_10 =    0x00000000
[ 186.418117] imx-ipuv3 2400000.ipu: IPU_INT_STAT_11 =    0x00000001
[ 186.418127] imx-ipuv3 2400000.ipu: IPU_INT_STAT_12 =    0x00000000
[ 186.418137] imx-ipuv3 2400000.ipu: IPU_INT_STAT_13 =    0x00000001
[ 186.418146] imx-ipuv3 2400000.ipu: IPU_INT_STAT_14 =    0x00000000
[ 186.418156] imx-ipuv3 2400000.ipu: IPU_GPR =        0x00000000
[ 186.418165] imx-ipuv3 2400000.ipu: IPU_CHA_CUR_BUF_0 =    0x00000001
[ 186.418175] imx-ipuv3 2400000.ipu: IPU_CHA_CUR_BUF_1 =    0x00000000
[ 186.418184] imx-ipuv3 2400000.ipu: IPU_CHA_BUF0_RDY_0 =    0x00000001
[ 186.418194] imx-ipuv3 2400000.ipu: IPU_CHA_BUF0_RDY_1 =    0x00000000
[ 186.418203] imx-ipuv3 2400000.ipu: IPU_CHA_BUF1_RDY_0 =    0x00000000
[ 186.418212] imx-ipuv3 2400000.ipu: IPU_CHA_BUF1_RDY_1 =    0x00000000
[ 186.418221] imx-ipuv3 2400000.ipu: IDMAC_CHA_BUSY_1 =    0x00000001
[ 186.418231] imx-ipuv3 2400000.ipu: IDMAC_CHA_BUSY_2 =    0x00000000
[ 186.418241] imx-ipuv3 2400000.ipu: CSI_SENS_CONF =        0x04000A10
[ 186.418250] imx-ipuv3 2400000.ipu: CSI_SENS_FRM_SIZE =        0x02CF04FF
[ 186.418259] imx-ipuv3 2400000.ipu: CSI_ACT_FRM_SIZE =        0x02CF04FF
[ 186.418269] imx-ipuv3 2400000.ipu: CSI_OUT_FRM_CTRL =        0x00000000
[ 186.418278] imx-ipuv3 2400000.ipu: CSI_CCIR_CODE_1 =    0x00000000
[ 186.418287] imx-ipuv3 2400000.ipu: CSI_CCIR_CODE_2 =    0x00000000
[ 186.418296] imx-ipuv3 2400000.ipu: CSI_CCIR_CODE_3 =    0x00000000
[ 186.418305] imx-ipuv3 2400000.ipu: CSI_MIPI_DI =        0xFFFFFF1E
[ 186.418316] imx-ipuv3 2400000.ipu: SMFC_MAP =        0x00000000
[ 186.418325] imx-ipuv3 2400000.ipu: SMFC_BS =        0x00000007
[ 186.434211] In MVC:mxc_v4l_ioctl
[ 186.434230]    case VIDIOC_DQBUF
[ 186.434239] In MVC:mxc_v4l_dqueue
[ 186.477676] In MVC:camera_callback
[ 186.477703] eba 38900000

This happens whether I am using 2-lane MIPI or 4-lane MIPI. Same results either way, 720p has the image width reduced from 1280 to 800, and 1080p gives me 9 reduced frames for each frame. I have no explanation for what could cause this.

Is there something in the CSI2IPU gasket that could cause this issue? I can't locate the driver module that configures the gasket, so I have been wondering about that.

Any help would be appreciated!

igorpadykov · ‎04-26-2018

Hi Will

what gstreamer used in the case, please check last post below for two options:

i.MX6DL not smooth full HD accelerated video play

In general one can try both, for NXP gst-1.0 usage one can look in Linux Guide

included in linux documentation on

i.MX Software|NXP

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

willhoie · ‎05-11-2018

Hi Igor,

Here is a gstreamer run showing the format and a warning I get. This was a run to capture color bars at 720p.

root@SN-62007:~# gst-launch-1.0 --version
gst-launch-1.0 version 1.4.5
GStreamer 1.4.5
Unknown package origin
root@SN-62007:~# gst-launch-1.0 -v -m imxv4l2videosrc ! filesink location=test-1280x720-30fps.uyvy
Setting pipeline to PAUSED ...
[ 278.853747] ERROR: unrecognized std! 0 (PAL=ff, NTSC=b000
[ 279.594177] ERROR: v4l2 capture: unknown ioctrl(9a091b)!
Pipeline is live and does not need PREROLL ...
Got message #3 from element "filesink0" (state-changed): GstMessageStateChanged, old-state=(GstState)GST_STATE_NULL, new-state=(GstState)GST_STATE_READY, pending-state=(GstState)GST_STATE_VOID_PENDING;
Got message #4 from element "imxv4l2videosrc0" (state-changed): GstMessageStateChanged, old-state=(GstState)GST_STATE_NULL, new-state=(GstState)GST_STATE_READY, pending-state=(GstState)GST_STATE_VOID_PENDING;
Got message #5 from element "pipeline0" (state-changed): GstMessageStateChanged, old-state=(GstState)GST_STATE_NULL, new-state=(GstState)GST_STATE_READY, pending-state=(GstState)GST_STATE_PAUSED;
Got message #9 from pad "imxv4l2videosrc0:src" (stream-status): GstMessageStreamStatus, type=(GstStreamStatusType)GST_STREAM_STATUS_TYPE_CREATE, owner=(GstElement)"\(GstImxV4l2VideoSrc\)\ imxv4l2videosrc0", object=(GstTask)"\(GstTask\)\ task0";
Got message #10 from element "imxv4l2videosrc0" (state-changed): GstMessageStateChanged, old-state=(GstState)GST_STATE_READY, new-state=(GstState)GST_STATE_PAUSED, pending-state=(GstState)GST_STATE_VOID_PENDING;
Got message #11 from element "pipeline0" (state-changed): GstMessageStateChanged, old-state=(GstState)GST_STATE_READY, new-state=(GstState)GST_STATE_PAUSED, pending-state=(GstState)GST_STATE_VOID_PENDING;
Setting pipeline to PLAYING ...
Got message #12 from element "pipeline0" (new-clock): GstMessageNewClock, clock=(GstClock)"\(GstSystemClock\)\ GstSystemClock";
New clock: GstSystemClock
Got message #14 from element "imxv4l2videosrc0" (state-changed): GstMessageStateChanged, old-state=(GstState)GST_STATE_PAUSED, new-state=(GstState)GST_STATE_PLAYING, pending-state=(GstState)GST_STATE_VOID_PENDING;
Got message #15 from pad "imxv4l2videosrc0:src" (stream-status): GstMessageStreamStatus, type=(GstStreamStatusType)GST_STREAM_STATUS_TYPE_ENTER, owner=(GstElement)"\(GstImxV4l2VideoSrc\)\ imxv4l2videosrc0", object=(GstTask)"\(GstTask\)\ imxv4l2videosrc0:src";
Got message #16 from element "pipeline0" (stream-start): GstMessageStreamStart, group-id=(uint)0;
/GstPipeline:pipeline0/GstImxV4l2VideoSrc:imxv4l2videosrc0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)UYVY\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1\,\ pixel-aspect-ratio\=\(fraction\)1/"
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)UYVY\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1\,\ pixel-aspect-ratio\=\(fraction\)1/1"
Got message #22 from element "filesink0" (state-changed): GstMessageStateChanged, old-state=(GstState)GST_STATE_READY, new-state=(GstState)GST_STATE_PAUSED, pending-state=(GstState)GST_STATE_VOID_PENDING;
Got message #24 from element "pipeline0" (async-done): GstMessageAsyncDone, running-time=(guint64)18446744073709551615;
Got message #26 from element "filesink0" (state-changed): GstMessageStateChanged, old-state=(GstState)GST_STATE_PAUSED, new-state=(GstState)GST_STATE_PLAYING, pending-state=(GstState)GST_STATE_VOID_PENDING;
Got message #27 from element "pipeline0" (state-changed): GstMessageStateChanged, old-state=(GstState)GST_STATE_PAUSED, new-state=(GstState)GST_STATE_PLAYING, pending-state=(GstState)GST_STATE_VOID_PENDING;
[ 280.775670] imx-ipuv3 2400000.ipu: IPU Warning - IPU_INT_STAT_5 = 0x00000001
^Chandling interrupt.
Got message #28 from element "pipeline0" (application): GstLaunchInterrupt, message=(string)"Pipeline\ interrupted";
Interrupt: Stopping pipeline ...
Execution ended after 0:00:02.816715000
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
[ 282.616289] ERROR: v4l2 capture: unknown ioctrl(9a091b)!
Setting pipeline to NULL ...
Freeing pipeline ...
root@SN-62007:~#

igorpadykov · ‎05-13-2018

Hi Will

imxv4l2videosrc is part of open source implementation gstreamer-imx

supported by community. Issue may be posted on

Issues · Freescale/gstreamer-imx · GitHub

Best regards
igor

willhoie · ‎05-14-2018

Thanks Igor. I did find an issue with reporting and setting the right pixel format in gstreamer-imx, and I'll let them know if it hasn't been fixed already.

But the issue I am having with the video is only happening at 720p and higher resolutions, not all the time. At 480p and 800x600, everything works great. No problems with the resolution, test pattern looks good. It's only at 720p and up that things go wrong. I have scrubbed through imxv4l2videosrc and found some issues, but nothing that explains that problem.

I have been looking at the frames just after they get transferred from the IDMAC, and they are already wrong when they hit main memory. I've been printing out the first handful of bytes in each frame right after the DMA xfer, because those should always be the same given a test pattern. But that is showing that the image is already corrupted, and that's before gstreamer has a chance to put them through the gstreamer pipeline, so I don't really think it's a post-IPU issue. I think it's happening somewhere between the MIPI input and the output of the IDMAC.

willhoie · ‎06-07-2018

Hi Igor. I have confirmed that the MIPI interface is working correctly. I have used a MIPI tester to send good MIPI packets, and bad ones with a bad CRC. The MIPI peripheral correctly determines good and bad packets. The issue is between the MIPI CSI-2 peripheral and the IDMAC.

I have run experiments by changing the MIPI frequency, and I have found that I can get 720p video to work correctly at transfer speeds under 400Mbps over 2 lanes. At four lanes, the frequency must be cut in half (200Mbps) in order to receive 720p video without distortion.

Again, this is looking at frames just after the DMA xfer completes and before the gstreamer pipeline, so gstreamer is eliminated as the problem. The distortion is occurring somewhere in the processing sequence of the IPU.

Again, the distortion I am experiencing is that at 720p, the video frames are downsized from 1280x720 to 800x720. At 1080p, the video frames are downsized to 640x360 and tiled 3x3 into a single 1920x1080 frame. Lower resolutions work fine.

I have looked at the downsizer block, and I wondered if perhaps the DMA was misfiring into the downsizer at the higher data speeds. But I changed all the downsizing and resizing coefficients in IPUx_IC_PRP_ENC_RSC, IPUx_IC_PRP_VF_RSC, and IPUx_IC_PP_RSC but it had no effect. However, the output looks like it is going through a downsizer somehow, but only at MIPI speeds above a certain threshold that depends on the number of data lanes (2 or 4).

igorpadykov · ‎06-08-2018

Hi Will

NXP has special service for porting drivers for customer needs

NXP Professional Services|NXP

Best regards
igor

willhoie · ‎06-09-2018

Thanks Igor. I have also tried the OV5640 camera MIPI driver with the MIPI tester, and it also exhibits the same distortion of video above 400 Mbps.

I don’t think it’s a problem with my camera driver, or the OV5649 driver either. There is something going wrong between the MIPI CSI-2 receiver, in which the MIPI packets are received successfully based on the CRC check, and the IPU DMA channel.

I posted the IPU register dump up thread already, but I don’t see anything wrong with it. Nothing that would explain downsizing above 400 Mbps.

igorpadykov · ‎03-26-2018

Hi Will

one can try to debug it further analyzing memory data captured from sensor,

using for example baremetal sdk :

Github SDK
https://github.com/backenklee/swp-report/tree/master/iMX6_Platform_SDK

and sect.40.5 Payload Data Output Format i.MX6DQ Reference Manual
https://www.nxp.com/docs/en/reference-manual/IMX6DQRM.pdf

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

i.MX6Q+ADV7481: Capturing YUV420, but from script setting up YUV422

i.MX6Q+ADV7481: Capturing YUV420, but from script setting up YUV422

i.MX6Quad