How to configure the IPU to pack 8-bit bayer

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

How to configure the IPU to pack 8-bit bayer

2,857 Views
bschuste
Contributor II

Dear Support,

I am trying to setup the registers of the video processing chain with a ov9732 sensor as the source.
The path is following:

 ov9732 -> mipi/csi-2 -> csi/ipu gasket -> csi input -> sfmc -> idmac -> memory

The source can only be configured in this mode V4L2_IF_TYPE_BT656_MODE_BT_10BIT.
When storing in 16-bit format into memory, the images are good.
But when enabling the compander in the ipu to produce a 8-bit format, the images are noisy.

What is the proper IPU registers setup to get the bayer 8-bit per pixel format working?

The linux version is: 4.1.15-1.0.0 build using yocto.

Attached is a bayer BGGR image which can be visualized in 8-bit grey scale.

Below are some logs showing the content of registers:
                                                             
setup_ifparm,1491: csi_param: vsync_pol(0) hsync_pol(0) ext_vsync(0)                                         
ipu_csi_init_interface,104: setting cfg_param.data_fmt  CSI_SENS_CONF_DATA_FMT_BAYER                             
ioctl_g_ifparm,777: u.bt656.mode 4, V4L2_IF_TYPE_BT656_MODE_BT_10BIT                                             
setup_ifparm,1468: ifparm.u.bt656.clock_curr=22000000                                                            
setup_ifparm,1473: csi_param.clk_mode=1                                                                          
setup_ifparm,1482: csi_param.data_width=IPU_CSI_DATA_WIDTH_10                                                    
setup_ifparm,1485: csi_param.pack_tight=0                                                                        
setup_ifparm,1491: csi_param: vsync_pol(0) hsync_pol(0) ext_vsync(0)                                             
csi_enc_setup,89:, sensor_protocol 1                                                                             
csi_enc_setup,119: setting pixel_fmt = IPU_PIX_FMT_GENERIC                                                       
csi_enc_setup,145: pixel_fmt               IPU0                                                                  
csi_enc_setup,146: v2f.fmt.pix.pixelformat BA81    

_ipu_csi_init,940: Enabling 10-bit to 8-bit companding
CSI_SENS_CONF     = 0x00001b10                                                                                   
CSI_SENS_FRM_SIZE = 0x02cf04ff                                                                                   
CSI_ACT_FRM_SIZE  = 0x02cf04ff                                                                                   
CSI_OUT_FRM_CTRL  = 0x00000000                                                                                   
CSI_TST_CTRL      = 0x00000000                                                                                   
CSI0_CPD_CTRL     = 0x00000010                                                                                   
CSI1_CPD_CTRL     = 0x00000000                                                                                   
CSI_CPD_OFFSET1   = 0x00000000                                                                                   
CSI_CPD_OFFSET2   = 0x00000000                                                                                   
CSI_CPD_RC((0))   = 0x00200000                                                                                   
CSI_CPD_GRC((0))  = 0x00200000                                                                                   
CSI_CPD_GBC((0))  = 0x00200000                                                                                   
CSI_CPD_BC((0))   = 0x00200000                                                                                   
CSI_CPD_RC((1))   = 0x00600040                                                                                   
CSI_CPD_GRC((1))  = 0x00600040                                                                                   
CSI_CPD_GBC((1))  = 0x00600040                                                                                   
CSI_CPD_BC((1))   = 0x00600040                                                                                   
CSI_CPD_RC((2))   = 0x00a00080                                                                                   
CSI_CPD_GRC((2))  = 0x00a00080                                                                                   
CSI_CPD_GBC((2))  = 0x00a00080                                                                                   
CSI_CPD_BC((2))   = 0x00a00080                                                                                   
CSI_CPD_RC((3))   = 0x00e000c0                                                                                   
CSI_CPD_GRC((3))  = 0x00e000c0                                                                                   
CSI_CPD_GBC((3))  = 0x00e000c0                                                                                   
CSI_CPD_BC((3))   = 0x00e000c0                                                                                   
CSI_CPD_RC((4))   = 0x01200100                                                                                   
CSI_CPD_GRC((4))  = 0x01200100                                                                                   
CSI_CPD_GBC((4))  = 0x01200100                                                                                   
CSI_CPD_BC((4))   = 0x01200100                                                                                   
CSI_CPD_RC((5))   = 0x01600140                                                                                   
CSI_CPD_GRC((5))  = 0x01600140                                                                                   
CSI_CPD_GBC((5))  = 0x01600140                                                                                   
CSI_CPD_BC((5))   = 0x01600140                                                                                   
CSI_CPD_RC((6))   = 0x01a00180                                                                                   
CSI_CPD_GRC((6))  = 0x01a00180                                                                                   
CSI_CPD_GBC((6))  = 0x01a00180                                                                                   
CSI_CPD_BC((6))   = 0x01a00180                                                                                   
CSI_CPD_RC((7))   = 0x01e001c0                                                                                   
CSI_CPD_GRC((7))  = 0x01e001c0                                                                                   
CSI_CPD_GBC((7))  = 0x01e001c0                                                                                   
CSI_CPD_BC((7))   = 0x01e001c0                                                                                   
CSI_CPD_RS((0))   = 0x20202020                                                                                   
CSI_CPD_GRS((0))  = 0x20202020                                                                                   
CSI_CPD_GBS((0))  = 0x20202020                                                                                   
CSI_CPD_BS((0))   = 0x20202020                                                                                   
CSI_CPD_RS((1))   = 0x20202020                                                                                   
CSI_CPD_GRS((1))  = 0x20202020                                                                                   
CSI_CPD_GBS((1))  = 0x20202020                                                                                   
CSI_CPD_BS((1))   = 0x20202020                                                                                   
CSI_CPD_RS((2))   = 0x20202020                                                                                   
CSI_CPD_GRS((2))  = 0x20202020                                                                                   
CSI_CPD_GBS((2))  = 0x20202020                                                                                   
CSI_CPD_BS((2))   = 0x20202020                                                                                   
CSI_CPD_RS((3))   = 0x20202020                                                                                   
CSI_CPD_GRS((3))  = 0x20202020                                                                                   
CSI_CPD_GBS((3))  = 0x20202020                                                                                   
CSI_CPD_BS((3))   = 0x20202020
                                                       
ipu_init_channel, 839: csi 0, channel 268435392, Calling _ipu_csi_init: params->csi_mem.mipi.en 1                
ipu_init_channel_buffer,1461: channel=268435392, type=0, pixel_fmt=0x30555049, width=1280, height=720, stride=120
ipu_init_channel_buffer,1467: phyaddr_0 = 0x40800000, phyaddr_1 = 0x40800000, phyaddr_2 = 0x00000000, u=0, v=0   
initializing idma ch 0 @ c08c0000                                                                                
ipu_init_channel_buffer, 1560: reading burst_size 64                                                             
ipu_init_channel_buffer, 1569: IPU_PIX_FMT_GENERIC: setting SMFC burst_size 4                                    
ch 0 word 0 - 00000000 00000000 00000000 E0002800 000B3C9F                                                       
ch 0 word 1 - 08100000 01020000 00CFC000 00013FC0 00000000                                                       
PFS 0x6,                                                                                                         
BPP 0x5, NPB 0x3f                                                                                                
FW 1279,                                                                                                         
FH 719, EBA0 0x40800000                                                                                          
EBA1 0x40800000                                                                                                  
Stride 1279                                                                                                      
scan_order 0                                                                                                     
uv_stride 0                                                                                                      
u_offset 0x0                                                                                                     
v_offset 0x0                                                                                                     
Width0 0+1,                                                                                                      
Width1 0+1, Width2 0+1,                                                                                          
Width3 0+1, Offset0 0,                                                                                           
Offset1 0, Offset2 0,                                                                                            
Offset3 0                                                                                                        

Labels (1)
0 Kudos
11 Replies

2,350 Views
bschuste
Contributor II

Looked into FCW format converter write and forced the following values to CPMEM for the given channel (0):

- BPP: 0 (32-bit per pixel)

- PFS: RGB (although the data is generic, packing only works with PFS=7, see table 38-14. Channel Parameters Memory for interleaved, i.MX6DL RM rev. 3, 09/207)

- NBP: 15 (burst size 16)

I am hoping to get rid of the 8-bit LSB of a 16-bit word coming out of CSI as generic data without using the compander.

Defined width0 to width3 and offset 0 to offset3 using the following function:

_ipu_ch_params_set_packing(&params, 3, 0, 3, 4, 3, 16, 3, 20);

I also tried with BPP = 3 (16-bit per pixel) and burst size 32 and

_ipu_ch_params_set_packing(&params, 1, 0, 1, 2, 1, 4, 1, 6);

But have no success, the image looks like if bursts of data are repeated three times.

Any solution which would work with generic data?

0 Kudos

2,350 Views
igorpadykov
NXP Employee
NXP Employee

some IPU packing examples one can find in Github SDK
https://github.com/backenklee/swp-report/tree/master/iMX6_Platform_SDK 

0 Kudos

2,350 Views
bschuste
Contributor II

This link does not seem to be active anymore.

0 Kudos

2,350 Views
igorpadykov
NXP Employee
NXP Employee
0 Kudos

2,350 Views
bschuste
Contributor II

Thanks Igor,

I have been looking at function ipu_idma_pixel_format_config() in file ipu_idmac.c, how two RGB formats are packed.

By setting following values, I now get an image that is duplicated vertically and horizontally.

BPP: 5 (8-bit)

NPB: 7 (idmac burst size of 8, any value different from this results in a noisy picture)

BPF: 7 (RGB)

Packing: _ipu_ch_params_set_packing(&params, 8, 0, 8, 8, 8, 16, 8, 24);

I would expect idmac to write only the byte at offset 0 to memory. Is this really the case?

SMFC burst size does not seem to have an impact. What should be the correct setting here?

0 Kudos

2,350 Views
igorpadykov
NXP Employee
NXP Employee

Hi Brigitte

one can also look in sect.18.10.1.4 Configuring the DC block

iMX6_Firmware_Guide.pdf included in SDK. Bayer sensors are not

supported in NXP official BSPs, so there are no validated or recommended settings.

Best regards
igor

0 Kudos

2,350 Views
bschuste
Contributor II

Hi Igor,

This section refers to the display path of the IPU and does not seem to apply to the capture path at all. It raises a question though, is there a specific firmware for the IPU capture side? If yes, what should be the latest version?

Thanks,

Brigitte

0 Kudos

2,350 Views
igorpadykov
NXP Employee
NXP Employee

Hi Brigitte

please check nxp official linux documentation on

i.MX Software|NXP 

linux-imx - i.MX Linux kernel 

Documentation describes software features which are "officially" supported:

that is tested and documented. All other sw efforts which are not listed in documentation,

such as custom drivers, can be supported using Professional Service

NXP Professional Services|NXP 

Best regards
igor

0 Kudos

2,351 Views
igorpadykov
NXP Employee
NXP Employee

Hi Brigitte

unfortunately IPU does not support natively bayer format, may be useful to check below links

IPU - bayer pattern conversion 

https://community.nxp.com/message/344529#344529 

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

0 Kudos

2,350 Views
bschuste
Contributor II

Thanks Igor, I am not sure I understand your answer correctly.

Per the reference manual, the IPU cannot convert a Bayer format to RGB or YUV which is OK.

But I expect the IPU to be able to transfer Bayer data from the sensor to memory properly.

The only difference here with other applications is that the sensor provides 10-bit raw data that IPU is supposed to quantize down to 8-bit using the compander and store as 8-bit into memory.

Are you saying this is not going to work?

0 Kudos

2,349 Views
igorpadykov
NXP Employee
NXP Employee

IPU does not "know" about Bayer format, there is no special processing for that format.

For packing capabilities one can look at sect.37.4.2.3 FCW & FCR - Format converter

write and read i.MX6DQ Reference Manual

http://www.nxp.com/docs/en/reference-manual/IMX6DQRM.pdf

Best regards
igor