IPU IDMAC support for 8-bit to 24-bit conversions

[vladimirzapolsk]

Hello,

is it technically possible to convert data by means of IPU from 8-bit per fixel format to 24-bit per pixel format? If yes, which CPMEM configuration is recommended to convert 8-bit grayscale data (Y8) to RGB24 or YUV?

With best wishes,

Vladimir

[joanxie]

refer to the chapter 33.2.5 Color Space Conversion of Reference Manual :

This block is responsible for carrying out the following video color space conversion

functions:

• RGB to/from YCbCr

• 4:2:2 to/from 4:4:4 up (pixel repetition or linear interpolation)/down-converter

• Limited to/from full quantization range conversion

[vladimirzapolsk]

Hello,

thank you for the response, however I still not sure what's the proper answer from your reply. If I understand correctly in general CSC could be used to convert not only RGB to YUV and vice versa, but more flexibly, because it is programmable. Currently in my query I'm mostly interested in data conversions, which packs/unpacks raw (or YUV) data, especially conversion from Y8 stream to any 24-bit or 32-bit per pixel format.

For instanse is it possible to unpack grayscale 8-bit per pixel data stream to YCbCr by means of IDMAC or not? CSC unit can't be exploited for this purpose, however my question is about IDMAC/CPMEM configuration in general.

Thank you in advance.

With best wishes,

Vladimir

[joanxie]

ok, I see , CSC can support YUV to RGB, but if input is Y8 grayscale, you must change them to yuv first(by your own software, just give some UV component), then use CSC to change YUV to RGB24bit, I mean that IPU couldn't support Y8 to RGB directly, because they need UV component.

[vladimirzapolsk]

Hi,

I have to repeat that I'm not interested in CSC utilization, which is a part of IC, my question is about IDMAC/CPMEM configuration. Even if I want to convert data from Y8 to RGB24 I shouldn't use CSC, and the question is about unpacking 8-bit per pixel data to 24-bit per pixel data by means of IDMAC.

With best wishes,

Vladimir

[joanxie]

I see, I mean that current IPU couldnt' supprt gray scale data to yuv24bit or RGB,  if you want to change to yuv, must expand y8 to yuv first(give some uv componet by software), current IPU couldn't support this.

[vladimirzapolsk]

Hi,

ok, thank you, I treat your answer as IDMAC can't unpack raw 8-bit data to 24-bit data.

With best wishes,

Vladimir

[DraganOstojic]

Vladimir, I solved that problem by using IPU in the following way: First I capture gray scale as RGB not as raw. I accomplished that by specifying that my horizontal resolution is 1/3 of what is coming from camera. To replicate 1 gray scale pixel into 3, I first upscale 3x horizontally followed by applying custom loaded CSC which averages 3 gray scale pixels into 3 identical gray scale pixels. If you're interested I could post the code fragment.

[vladimirzapolsk]

Hi Dragan,

I believe I got your nice idea, and it seems to be feasible. Can you check, does IPU and its correspondent driver work well, if a horizontal resolution is not a multiple of 3?

Generally I understand the method, but I would appreciate you so much, if you post a code snippet related to CSC setup and especially IPU DMAC/CPMEM configuration, if you do it.

Thank you.

With best wishes,

Vladimir

[DraganOstojic · Accepted solution]

I now have a solution that uses GPU. I will post details of the solution in the following thread:

https://community.freescale.com/thread/309817

[DraganOstojic]

I implemented a different method to convert 8-bit gray scale to 24-bit rgb from what I suggested above. This method converts 8-bit gray scale using a custom csc matrix and color space transformation from planar YUV to rgb24. Because there is no U,V component, coefficients are chosen such that:

R, G, B = Y * 1 + U * 0 + V * 0

Please see attached changes to the imx6 IPU BSP to enable custom csc matrices download.

The way 8-bit gray scale is converted is as follows:

First capture from gray scale camera using 24-bit rgb pixel format but specify h-res as 1/3 of original gray scale h-res. Make sure that the capture buffer is large enough for U/V planes. U/V bytes are not actually captured nor used but are needed for IPU unit to properly function during conversion YUV->RGB.

In the application use the following code to perform the conversion:

void CameraHal::setupCustomIpuMatrixForGrayscaleToRgbConversion()

    {

        for (int i = 0; i < 4; i++)

            for (int j = 0; j < 3; j++)

                mIpuTask.custom_coeff[i][j] = 0;    

            mIpuTask.custom_coeff[0][0] = 128;

            mIpuTask.custom_coeff[1][0] = 128;

            mIpuTask.custom_coeff[2][0] = 128;                          

    }

setupCustomIpuMatrixForGrayscaleToRgbConversion();

        mIpuTask.custom_matrix  = 1;

        mIpuTask.input.format   = IPU_PIX_FMT_YUV444P;

        mIpuTask.output.format  = IPU_PIX_FMT_RGB32;

        mIpuTask.output.rotate  = IPU_ROTATE_HORIZ_FLIP;

...

     if (ioctl(mIpuFd, IPU_QUEUE_TASK, &mIpuTask) < 0)

                CAMERA_LOG_ERR("%s: convert Y8->RGB32: ioct(IPU_QUEUE_TASK) fail", __func__);

Let me know if you need more clarification.

[marcus_c]

Hi Dragan,

Can you post more detail of ioctl command?

It looks like mIpuTask needs more information and system hangs with only input/output/format assignment.

I tried to assign as many information as possible but I don't know how to find out input/output dma_addr_t paddr.

[DraganOstojic]

Hi Marcus, you can look into how ipu task data structure should be initialized in the following examples:

https://github.com/rogeriorps/ipu-examples

If you are on Android 4.0.4 in order to be able to open and use ipu driver from the hal layer, you need to change this additional file:

~/myandroid/device/fsl/imx6/etc/ueventd.freescale.rc

and change the following line as follows:

/dev/mxc_ipu              0666   graphics   system

Examples in the link give enough detail about which fields need to be initialized.

On Android 4.0.4, you can get physical address you asked me about from the DMA_BUFFER data structure element:

DMA_BUFFER *buffer_in = ... // initialized to point to the memory that is input to the IPU

DMA_BUFFER *buffer_out = ... // initialized to point to the memory that is output of the IPU

...

mIpuTask.input.paddr  = buffer->phy_offset; // for memory that is input to the IPU

mIpuTask.output.paddr = buffer->phy_offset; // for memory that is output of the IPU

DMA_BUFFER data structure is used in the camera hal to store information about the allocated memory buffer. It is defined in ~/myandroid/hardware/imx/mx6/libcamera/Camera_utils.h

Let me know it you need more information.

[joanxie]

yes, couldn't change them directly