How to configure the IPU to pack 8-bit bayer

bschuste · ‎05-30-2018

Dear Support,

I am trying to setup the registers of the video processing chain with a ov9732 sensor as the source.
The path is following:

ov9732 -> mipi/csi-2 -> csi/ipu gasket -> csi input -> sfmc -> idmac -> memory

The source can only be configured in this mode V4L2_IF_TYPE_BT656_MODE_BT_10BIT.
When storing in 16-bit format into memory, the images are good.
But when enabling the compander in the ipu to produce a 8-bit format, the images are noisy.

What is the proper IPU registers setup to get the bayer 8-bit per pixel format working?

The linux version is: 4.1.15-1.0.0 build using yocto.

Attached is a bayer BGGR image which can be visualized in 8-bit grey scale.

Below are some logs showing the content of registers:

setup_ifparm,1491: csi_param: vsync_pol(0) hsync_pol(0) ext_vsync(0)
ipu_csi_init_interface,104: setting cfg_param.data_fmt CSI_SENS_CONF_DATA_FMT_BAYER
ioctl_g_ifparm,777: u.bt656.mode 4, V4L2_IF_TYPE_BT656_MODE_BT_10BIT
setup_ifparm,1468: ifparm.u.bt656.clock_curr=22000000
setup_ifparm,1473: csi_param.clk_mode=1
setup_ifparm,1482: csi_param.data_width=IPU_CSI_DATA_WIDTH_10
setup_ifparm,1485: csi_param.pack_tight=0
setup_ifparm,1491: csi_param: vsync_pol(0) hsync_pol(0) ext_vsync(0)
csi_enc_setup,89:, sensor_protocol 1
csi_enc_setup,119: setting pixel_fmt = IPU_PIX_FMT_GENERIC
csi_enc_setup,145: pixel_fmt               IPU0
csi_enc_setup,146: v2f.fmt.pix.pixelformat BA81

_ipu_csi_init,940: Enabling 10-bit to 8-bit companding
CSI_SENS_CONF     = 0x00001b10
CSI_SENS_FRM_SIZE = 0x02cf04ff
CSI_ACT_FRM_SIZE = 0x02cf04ff
CSI_OUT_FRM_CTRL = 0x00000000
CSI_TST_CTRL      = 0x00000000
CSI0_CPD_CTRL     = 0x00000010
CSI1_CPD_CTRL     = 0x00000000
CSI_CPD_OFFSET1   = 0x00000000
CSI_CPD_OFFSET2   = 0x00000000
CSI_CPD_RC((0))   = 0x00200000
CSI_CPD_GRC((0)) = 0x00200000
CSI_CPD_GBC((0)) = 0x00200000
CSI_CPD_BC((0))   = 0x00200000
CSI_CPD_RC((1))   = 0x00600040
CSI_CPD_GRC((1)) = 0x00600040
CSI_CPD_GBC((1)) = 0x00600040
CSI_CPD_BC((1))   = 0x00600040
CSI_CPD_RC((2))   = 0x00a00080
CSI_CPD_GRC((2)) = 0x00a00080
CSI_CPD_GBC((2)) = 0x00a00080
CSI_CPD_BC((2))   = 0x00a00080
CSI_CPD_RC((3))   = 0x00e000c0
CSI_CPD_GRC((3)) = 0x00e000c0
CSI_CPD_GBC((3)) = 0x00e000c0
CSI_CPD_BC((3))   = 0x00e000c0
CSI_CPD_RC((4))   = 0x01200100
CSI_CPD_GRC((4)) = 0x01200100
CSI_CPD_GBC((4)) = 0x01200100
CSI_CPD_BC((4))   = 0x01200100
CSI_CPD_RC((5))   = 0x01600140
CSI_CPD_GRC((5)) = 0x01600140
CSI_CPD_GBC((5)) = 0x01600140
CSI_CPD_BC((5))   = 0x01600140
CSI_CPD_RC((6))   = 0x01a00180
CSI_CPD_GRC((6)) = 0x01a00180
CSI_CPD_GBC((6)) = 0x01a00180
CSI_CPD_BC((6))   = 0x01a00180
CSI_CPD_RC((7))   = 0x01e001c0
CSI_CPD_GRC((7)) = 0x01e001c0
CSI_CPD_GBC((7)) = 0x01e001c0
CSI_CPD_BC((7))   = 0x01e001c0
CSI_CPD_RS((0))   = 0x20202020
CSI_CPD_GRS((0)) = 0x20202020
CSI_CPD_GBS((0)) = 0x20202020
CSI_CPD_BS((0))   = 0x20202020
CSI_CPD_RS((1))   = 0x20202020
CSI_CPD_GRS((1)) = 0x20202020
CSI_CPD_GBS((1)) = 0x20202020
CSI_CPD_BS((1))   = 0x20202020
CSI_CPD_RS((2))   = 0x20202020
CSI_CPD_GRS((2)) = 0x20202020
CSI_CPD_GBS((2)) = 0x20202020
CSI_CPD_BS((2))   = 0x20202020
CSI_CPD_RS((3))   = 0x20202020
CSI_CPD_GRS((3)) = 0x20202020
CSI_CPD_GBS((3)) = 0x20202020
CSI_CPD_BS((3))   = 0x20202020

ipu_init_channel, 839: csi 0, channel 268435392, Calling _ipu_csi_init: params->csi_mem.mipi.en 1
ipu_init_channel_buffer,1461: channel=268435392, type=0, pixel_fmt=0x30555049, width=1280, height=720, stride=120
ipu_init_channel_buffer,1467: phyaddr_0 = 0x40800000, phyaddr_1 = 0x40800000, phyaddr_2 = 0x00000000, u=0, v=0
initializing idma ch 0 @ c08c0000
ipu_init_channel_buffer, 1560: reading burst_size 64
ipu_init_channel_buffer, 1569: IPU_PIX_FMT_GENERIC: setting SMFC burst_size 4
ch 0 word 0 - 00000000 00000000 00000000 E0002800 000B3C9F
ch 0 word 1 - 08100000 01020000 00CFC000 00013FC0 00000000
PFS 0x6,
BPP 0x5, NPB 0x3f
FW 1279,
FH 719, EBA0 0x40800000
EBA1 0x40800000
Stride 1279
scan_order 0
uv_stride 0
u_offset 0x0
v_offset 0x0
Width0 0+1,
Width1 0+1, Width2 0+1,
Width3 0+1, Offset0 0,
Offset1 0, Offset2 0,
Offset3 0

bschuste · ‎06-04-2018

Looked into FCW format converter write and forced the following values to CPMEM for the given channel (0):

- BPP: 0 (32-bit per pixel)

- PFS: RGB (although the data is generic, packing only works with PFS=7, see table 38-14. Channel Parameters Memory for interleaved, i.MX6DL RM rev. 3, 09/207)

- NBP: 15 (burst size 16)

I am hoping to get rid of the 8-bit LSB of a 16-bit word coming out of CSI as generic data without using the compander.

Defined width0 to width3 and offset 0 to offset3 using the following function:

_ipu_ch_params_set_packing(&params, 3, 0, 3, 4, 3, 16, 3, 20);

I also tried with BPP = 3 (16-bit per pixel) and burst size 32 and

_ipu_ch_params_set_packing(&params, 1, 0, 1, 2, 1, 4, 1, 6);

But have no success, the image looks like if bursts of data are repeated three times.

Any solution which would work with generic data?

igorpadykov · ‎06-04-2018

some IPU packing examples one can find in Github SDK
https://github.com/backenklee/swp-report/tree/master/iMX6_Platform_SDK

bschuste · ‎06-05-2018

This link does not seem to be active anymore.

igorpadykov · ‎06-05-2018

please check zip on

https://community.nxp.com/thread/432859

bschuste · ‎06-11-2018

Thanks Igor,

I have been looking at function ipu_idma_pixel_format_config() in file ipu_idmac.c, how two RGB formats are packed.

By setting following values, I now get an image that is duplicated vertically and horizontally.

BPP: 5 (8-bit)

NPB: 7 (idmac burst size of 8, any value different from this results in a noisy picture)

BPF: 7 (RGB)

Packing: _ipu_ch_params_set_packing(&params, 8, 0, 8, 8, 8, 16, 8, 24);

I would expect idmac to write only the byte at offset 0 to memory. Is this really the case?

SMFC burst size does not seem to have an impact. What should be the correct setting here?

igorpadykov · ‎06-13-2018

Hi Brigitte

one can also look in sect.18.10.1.4 Configuring the DC block

iMX6_Firmware_Guide.pdf included in SDK. Bayer sensors are not

supported in NXP official BSPs, so there are no validated or recommended settings.

Best regards
igor

bschuste · ‎06-18-2018

Hi Igor,

This section refers to the display path of the IPU and does not seem to apply to the capture path at all. It raises a question though, is there a specific firmware for the IPU capture side? If yes, what should be the latest version?

Thanks,

Brigitte

igorpadykov · ‎06-18-2018

Hi Brigitte

please check nxp official linux documentation on

i.MX Software|NXP

linux-imx - i.MX Linux kernel

Documentation describes software features which are "officially" supported:

that is tested and documented. All other sw efforts which are not listed in documentation,

such as custom drivers, can be supported using Professional Service

NXP Professional Services|NXP

Best regards
igor

igorpadykov · ‎05-31-2018

Hi Brigitte

unfortunately IPU does not support natively bayer format, may be useful to check below links

IPU - bayer pattern conversion

https://community.nxp.com/message/344529#344529

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

bschuste · ‎05-31-2018

Thanks Igor, I am not sure I understand your answer correctly.

Per the reference manual, the IPU cannot convert a Bayer format to RGB or YUV which is OK.

But I expect the IPU to be able to transfer Bayer data from the sensor to memory properly.

The only difference here with other applications is that the sensor provides 10-bit raw data that IPU is supposed to quantize down to 8-bit using the compander and store as 8-bit into memory.

Are you saying this is not going to work?

igorpadykov · ‎05-31-2018

IPU does not "know" about Bayer format, there is no special processing for that format.

For packing capabilities one can look at sect.37.4.2.3 FCW & FCR - Format converter

write and read i.MX6DQ Reference Manual

http://www.nxp.com/docs/en/reference-manual/IMX6DQRM.pdf

Best regards
igor