iMX6 camera patch to support CSI->VDI->IC->MEM and CSI->VDI->MEM capture

cancel
Showing results for 
Search instead for 
Did you mean: 

iMX6 camera patch to support CSI->VDI->IC->MEM and CSI->VDI->MEM capture

iMX6 camera patch to support CSI->VDI->IC->MEM and CSI->VDI->MEM capture

The patches are based on iMX6 L3.10.53 and 3.14.52 GA BSP.

 

In default linux BSP, the followed two pathes were supported in kernel driver mxc_v4l2_capture.c:
CSI->IC->MEM
CSI->MEM

 

After appied these patches, it can support the followed path:
CSI->VDI->IC->MEM
CSI->VDI->MEM

In this mode, the VDI de-interlace will be handled on the fly, so the whole system bandwidth will be reduced.

 

Limitations:
1. Since the IC can only output resolution up to 1024*1024, so this is the limation on output.
2. Only VDI motion mode 2 was supported.

 

mxc_v4l2_tvin.zip:
It is the test aplication,
test command for CSI->VDI->IC->MEM ("-i 2" means CSI->VDI->IC->MEM path.):
./mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 800 -oh 480 -i 2 -g2d"

 

test command for CSI->VDI->MEM ("-i 3" means CSI->VDI->MEM path.):
./mxc_v4l2_tvin.out -ol 0 -ot 0 -ow 800 -oh 480 -i 3 -g2d"

 

Tags (3)
Attachments
Comments

Looks very interesting...

Is this patch a super set of your previous de-interlace patch?

https://community.nxp.com/docs/DOC-173003 

i.e. do I just need so use this patch, and forget about the previous example?

Also, the IC has a 1024 raster height limitation, as I understand it

- however, the CSI->VDI->MEM path should be able to support 1080i -> 1080p @ 60Hz, right?

Thanks!

Phil

The 60fps VDI use case is for CSI->SMFC->MEM->VDI->MEM, they are two different use case.

Due to the limitation of output resolution, it can't work for 1080 output.

Qiang Li wrote:

The 60fps VDI use case is for CSI->SMFC->MEM->VDI->MEM, they are two different use case.

Due to the limitation of output resolution, it can't work for 1080 output.

Apologies for not understanding fully, but if I want to do 1080i -> 1080p de-interlace @ 60 fps

- but this 'CSI->VDI->MEM' can support this (1080p / 60) ?

- but the older patch: https://community.nxp.com/message/856215?commentID=856215&et=watches.email.thread#comment-856215 

- does support 1080p /60?

I need to convert 1080i/60 -> 1080p/60 (-->HDMI Output)

- and I am not sure the best way to go

Sorry for late response, the CSI->VDI->MEM and CSI->VDI->IC->MEM path can't be used for 1920x1080 input, when output from VDI and IC, there is resolution limitation, it must be less than 1024*1024. This is hardware limitation.

But you can capture the 1080I input with CSI -> MEM path, then in capture buffer, you will get 30fps 1920x1080 frames, you can send them to HDMI display, and you can also handle them with MEM->VDI->MEM, then send them to HDMI.

Qiang_FSL

         我打了你这个补丁用于去隔行在android6.0(内核4.1.15,打补丁没有冲突)的版本上,用命令:mxc-v4l2-tvin -ol 0 -ot 0 -ow 800 -oh 480 -i 3 -g2d进行测试,但无法抓到Camera数据(我用android的app已经可能正常预览如下图)。

android正常显示.jpg

错误log:

130|root@sabresd_6dq:/ # mxc-v4l2-tvin -ol 0 -ot 0 -ow 800 -oh 480 -i 3 -g2d
[ 85.821151]
[ 85.821151] In MVC: mxc_v4l_open
[ 85.825943] device name is Mxc Camera
[ 85.829894] ==>func = ioctl_g_ifparm ,line = 412
[ 85.834812] ==>func = ioctl_g_fmt_cap ,line = 544
[ 85.839632] End of mxc_v4l_open: v2f pix widthxheight 288 x 352
[ 85.845675] End of mxc_v4l_open: crop_bounds widthxheight 960 x 625
[ 85.852200] End of mxc_v4l_open: crop_defrect widthxheight 960 x 625
[ 85.858672] End of mxc_v4l_open: crop_current widthxheight 960 x 625
[ 85.865162] On Open: Input to ipu size is 960 x 625
[ 85.870207] ==>func = ioctl_s_power ,line = 450
[ 85.876333] In tp2825:ioctl_init
[ 85.880848] tp2825:ioctl_dev_init
[ 85.884360] In MVC:mxc_v4l_ioctl
[ 85.888151] In MVC: mxc_v4l_do_ioctl 80685600
[ 85.892729] case VIDIOC_QUERYCAP
[ 85.897325] In MVC:mxc_v4l_ioctl
[ 85.900573] In MVC: mxc_v4l_do_ioctl c0045627
[ 85.904937] case VIDIOC_S_INPUT
[ 85.908811] In MVC:mxc_streamoff
[ 85.912054] ====>name = CSI VDI MEM
[ 85.915677] ====>vdi_enc_select
[ 86.919626] In MVC:mxc_v4l_ioctl
[ 86.922882] In MVC: mxc_v4l_do_ioctl 80085617
[ 86.927346] case VIDIOC_G_STD
[ 86.930599] In mxc_v4l2_g_std
[ 86.933593] ==>func = ioctl_g_fmt_cap ,line = 544
[ 86.938625] ==>func = tp2825_get_std ,line = 364
[ 86.943283] In MVC:mxc_v4l_ioctl
[ 86.946604] In MVC: mxc_v4l_do_ioctl 40085618
[ 86.950985] case VIDIOC_S_STD
[ 86.954224] In mxc_v4l2_s_std ff
[ 86.957781] Setting standard to PAL ff
[ 86.961834] In MVC:mxc_v4l_ioctl
[ 86.965094] In MVC: mxc_v4l_do_ioctl c02c563a
[ 86.969781] case VIDIOC_CROPCAP
[ 86.973238] In MVC:mxc_v4l_ioctl
[ 86.976527] In MVC: mxc_v4l_do_ioctl c0cc5616
[ 86.980897] case VIDIOC_S_PARM
[ 86.984219] In mxc_v4l2_s_param
[ 86.987415] ==>func = ioctl_g_parm ,line = 469
[ 86.991875] Current capabilities are 0
[ 86.995925] Current capturemode is 0 change to 0
[ 87.000905] Current framerate is 30 change to 0
[ 87.005827] ==>func = ioctl_s_parm ,line = 511
[ 87.010284] ==>func = ioctl_g_ifparm ,line = 412
[ 87.014909] clock_curr=mclk=0
[ 87.018459] ==>func = ioctl_g_fmt_cap ,line = 544
[ 87.023182] g_fmt_cap returns widthxheight of input as 960 x 625
[ 87.029731] In MVC:mxc_v4l_ioctl
[ 87.032978] In MVC: mxc_v4l_do_ioctl c0cc5605
[ 87.037632] case VIDIOC_S_FMT
[ 87.040876] In MVC: mxc_v4l2_s_fmt
[ 87.044285] type=V4L2_BUF_TYPE_VIDEO_CAPTURE
[ 87.048870] End of mxc_v4l2_s_fmt: v2f pix widthxheight 960 x 576
[ 87.054979] End of mxc_v4l2_s_fmt: crop_bounds widthxheight 960 x 625
[ 87.061462] End of mxc_v4l2_s_fmt: crop_defrect widthxheight 960 x 625
[ 87.068048] End of mxc_v4l2_s_fmt: crop_current widthxheight 960 x 576
[ 87.074603] In MVC:mxc_v4l_ioctl
[ 87.077894] In MVC: mxc_v4l_do_ioctl c0cc5604
[ 87.082306] case VIDIOC_G_FMT
[ 87.085578] In MVC: mxc_v4l2_g_fmt type=1
[ 87.089600] type is V4L2_BUF_TYPE_VIDEO_CAPTURE
[ 87.094398] End of mxc_v4l2_g_fmt: v2f pix widthxheight 960 x 576
[ 87.100536] End of mxc_v4l2_g_fmt: crop_bounds widthxheight 960 x 625
[ 87.107033] End of mxc_v4l2_g_fmt: crop_defrect widthxheight 960 x 625
[ 87.113572] End of mxc_v4l2_g_fmt: crop_current widthxheight 960 x 576
g_in_width = 960, g_in_height = 5[ 87.120298] In MVC:mxc_v4l_ioctl
76.
[ 87.126421] In MVC: mxc_v4l_do_ioctl c0145608
[ 87.131258] case VIDIOC_REQBUFS
[ 87.134665] In MVC:mxc_streamoff
fb_fix.id = DISP3 FG.
fb: smem_start = 0x2f700000, smem_len = 0x708000.
fb: frame buffer size = 0x258000 bytes.
fb: g_screen_info.xres = 1024, g_screen_info.yres = 600.
fb: g_display_left = 0.
fb: g_display_top = 0.
fb: g_display_width = 800.
fb: g_display_height = 480.
[ 87.204895] In MVC:mxc_v4l_ioctl
[ 87.208181] In MVC: mxc_v4l_do_ioctl c0445609
[ 87.212550] case VIDIOC_QUERYBUF
[ 87.216073] In MVC:mxc_v4l2_release_bufs
[ 87.220005] In MVC:mxc_v4l2_prepare_bufs
[ 87.223940] In MVC:mxc_v4l_ioctl
[ 87.227236] In MVC: mxc_v4l_do_ioctl c0445609
[ 87.231604] case VIDIOC_QUERYBUF
[ 87.235099] In MVC:mxc_v4l2_release_bufs
[ 87.239056] In MVC:mxc_v4l2_prepare_bufs
[ 87.243038] In MVC:mxc_v4l_ioctl
[ 87.246321] In MVC: mxc_v4l_do_ioctl c0445609
[ 87.251968] case VIDIOC_QUERYBUF
[ 87.255558] In MVC:mxc_v4l2_release_bufs
[ 87.259497] In MVC:mxc_v4l2_prepare_bufs
[ 87.263486] In MVC:mxc_v4l_ioctl
[ 87.268051] In MVC: mxc_v4l_do_ioctl c0445609
[ 87.273459] case VIDIOC_QUERYBUF
[ 87.277378] In MVC:mxc_v4l2_release_bufs
[ 87.281648] In MVC:mxc_v4l2_prepare_bufs
[ 87.285934] In MVC:mxc_v4l_ioctl
[ 87.290447] In MVC: mxc_v4l_do_ioctl c044560f
[ 87.295390] case VIDIOC_QBUF
[ 87.298957] In MVC:mxc_v4l_ioctl
[ 87.303373] In MVC: mxc_v4l_do_ioctl c044560f
[ 87.308147] case VIDIOC_QBUF
[ 87.312488] In MVC:mxc_v4l_ioctl
[ 87.316074] In MVC: mxc_v4l_do_ioctl c044560f
[ 87.320623] case VIDIOC_QBUF
[ 87.323940] In MVC:mxc_v4l_ioctl
[ 87.327458] In MVC: mxc_v4l_do_ioctl c044560f
[ 87.332009] case VIDIOC_QBUF
[ 87.335942] In MVC:mxc_v4l_ioctl
[ 87.339210] In MVC: mxc_v4l_do_ioctl 40045612
[ 87.343584] case VIDIOC_STREAMON
[ 87.347164] In MVC:mxc_streamon
[ 87.350326] IPU:In vdi_enc_enabling_tasks
[ 87.360613] In vdi_enc_setup
start time = 82 s, 650072 us
[ 87.363647] In MVC:mxc_v4l_ioctl
[ 87.369516] In MVC: mxc_v4l_do_ioctl c0445611
[ 87.373883] case VIDIOC_DQBUF
[ 87.377166] In MVC:mxc_v4l_dqueue
[ 97.375336] ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0
VIDIOC_DQBUF failed.
[ 97.381873] In MVC:mxc_v4l_ioctl
[ 97.387275] In MVC: mxc_v4l_do_ioctl 40045613
[ 97.391845] case VIDIOC_STREAMOFF
[ 97.395911] In MVC:mxc_streamoff
[ 97.895675] In MVC:mxc_free_frames
[ 97.936805] In MVC:mxc_v4l_close
[ 97.941207] In MVC:mxc_streamoff
[ 97.944724] ==>func = ioctl_s_power ,line = 450
[ 97.949348] mxc_v4l_close: release resource
[ 97.953775] MVC: In mxc_free_frame_buf
[ 97.957621] In MVC:mxc_free_frames
root@sabresd_6dq:/ #

I used this patch for android6, kernel 4.1.15 and have same problem, can't use path CSI->VDI->MEM. 

can this patch use for 4.1.15 or not?

Hi Nguyen, your resolution is strange, I think you need double check the active data size setting in VDI, if the data size is wrong, VDI can't get enough data from CSI, it will not output.

Hi Qiang,

              Can this patch apply with ISL79985 and ISL79987, L4.1.15_ISL7998x_Surroundview_Patch_20161122.zip? Becasue ISL79985 only support 720x240, the system need more overhead costs for de-interlace. Can this patch help reduce these overheads?

Hi Qiang,

I  run mxc-v4l2-tvin  -ow 1280 -oh 720 -i 3  ,only get one frame and then get follow information :

<4>[   72.528933] imx-ipuv3 2400000.ipu: IPU Warning - IPU_INT_STAT_9 = 0x04000000

VIDIOC_DQBUF failed.

<3>[   82.525936] ERROR: v4l2 capture: mxc_v4l_dqueue timeout enc_counter 0

<3>[   83.087057] imx-ipuv3 2400000.ipu: warning: disable ipu dma channel 5 during its busy stat

.................................................

这个补丁遗漏一部分,没有在 ipu_common.c 文件中的 ipu_enable_channel 函数里添加CSI_VDI_MEM和CSI_VDI_PRP_VF_MEM 对应的enable代码。下面是补丁

From 67d877007ed11c4f897fb539d19f28e9c7505c0a Mon Sep 17 00:00:00 2001
From: stan <klshu36@163.com>
Date: Thu, 8 Feb 2018 15:50:12 +0800
Subject: [PATCH] add-CSI_VDI_MEM-to-ipu_enable_channel

---
drivers/mxc/ipu3/ipu_common.c | 10 ++++++++++
1 file changed, 10 insertions(+)

diff --git a/drivers/mxc/ipu3/ipu_common.c b/drivers/mxc/ipu3/ipu_common.c
index b856a35..46c34d2 100644
--- a/drivers/mxc/ipu3/ipu_common.c
+++ b/drivers/mxc/ipu3/ipu_common.c
@@ -2565,6 +2565,16 @@ int32_t ipu_enable_channel(struct ipu_soc *ipu, ipu_channel_t channel)
if (ipu->ic_use_count > 0)
ipu_conf |= IPU_CONF_IC_EN;
break;
+ case CSI_VDI_MEM:
+ case CSI_VDI_PRP_VF_MEM:
+ if (ipu->vdi_use_count > 0) {
+ ipu_conf |= IPU_CONF_ISP_EN;
+ ipu_conf |= IPU_CONF_VDI_EN;
+ ipu_conf |= IPU_CONF_IC_INPUT;
+ }
+ if (ipu->ic_use_count > 0)
+ ipu_conf |= IPU_CONF_IC_EN;
+ break;
default:
break;
}
--
1.9.1

我检查了linux代码,在linux内核版本 L3.10.53 and L3.14.52中,ipu_enable_channel 函数没有这个问题,而在L4.1.15中这个函数修改了,才出现了这个问题。所以上述补丁是基于L4.1.15。

thank you very much。

Qiang_FSL‌  how can I use this patch to L3.0.35 ?  I found the libcamera2  in android BSP use the  following setting  in DeviceAdapter.cpp :

int input = 1;
ret = ioctl(mCameraHandle, VIDIOC_S_INPUT, &input);

Do I need to modify the  int input = 1;  to  int input = 3;   ?  how to improve the android vdi ?

hi brague

      你好,想问一下你测试用的是模拟摄像头吗?可以了解下你摄像头的分辨率是多少,具体显示的分辨率是多少?打了这个补丁后用app打开摄像头时显示的效果有所改善吗?你也是通过libcamera2这个hal层来显示摄像头的图像吗?我看到测试的命令“mxc-v4l2-tvin -ol 0 -ot 0 -ow 800 -oh 480 -i 3 -g2d”,i是设置成3的,而libcamera2上面是设置成1,那是否有必要修改libcamera2这个hal 层的代码?谢谢。

Yes, you can set input=3, for input=1, it means CSI->SMFC->MEM, it captured the original video data to memory.

我们的内核是4.1.15,你先用测试用例测试,如果效果是可行的,再把相应的设置参数改到HAL层就行了。

Version history
Revision #:
1 of 1
Last update:
‎04-12-2016 08:25 PM
Updated by: