Rotating of Live video.

[johnturnur]

Hello

In one application we need to  scale and rotate the video data of type YUV422. We are using Platform SDK reference IPU code . We are not using V4l2 code.

To achieve above, we are capturing the video using OV5642 and directly display it on LVDS panels using View finder task.

OV5642  captures the video in YUV422 format and using IDMAC 21 channel and function ipu_resize_idmac_config() i am  able to re size video from 1024*768 to 640*480 resolutions.

We have used following path

CSI->IC( using channel 21)-> DP(Channel 23) . we are able to re- size video using this path and displays it on LVDS panels.

Now we want to rotate scaled video. To achieve this we used channel 46 as rotation input channel and 49 as rotation output channels using View finder task.

CSI->IC->IRT->DP . We are able to rotate the image but the display is garbage. Also rotation is performed on NON_INTERLEAVED_YUV420 data format.

Can IPU process rotation on YUV420 data?

Is all the channels and path are correct for rotation to work?

Guide me if i am wrong.

We are using Platform SDK V1.1.0 source code for IPU and camera capture drivers.

Thanks,

J

[juangutierrez]

Is it possible to first perform the rotation and the re-sizing?

[juangutierrez]

I noticed that for viewfinder mode the ipu_ic_task_enable function in SDK is not doing nothing when IC_ROT argument is passed. Maybe something like the next change is missing.

diff --git a/sdk/drivers/ipu/src/ipu_ic.c b/sdk/drivers/ipu/src/ipu_ic.c

index 14add06..2da4e54 100755

--- a/sdk/drivers/ipu/src/ipu_ic.c

+++ b/sdk/drivers/ipu/src/ipu_ic.c

@@ -498,6 +498,8 @@ int32_t ipu_ic_task_enable(int32_t ipu_index, int32_t task_type, int32_t task, i

             ipu_write_field(ipu_index, IPU_IC_CONF__PRPVF_CSC2, enable);

         else if (task == IC_PRPVF)

             ipu_write_field(ipu_index, IPU_IC_CONF__PRPVF_EN, enable);

+        else if (task == IC_ROT)

+            ipu_write_field(ipu_index, IPU_IC_CONF__PRPVF_ROT_EN, enable);

         else

             printf("Task Type is wrong!!\n");

I did the next modification (not applicable to you) to test Viewfinder mode in sdk_unit_test_ipu test and I was able to see both rotation in resize working

diff --git a/sdk/drivers/ipu/test/ips_resize.c b/sdk/drivers/ipu/test/ips_resize.c

index 1a00ab7..b446955 100644

--- a/sdk/drivers/ipu/test/ips_resize.c

+++ b/sdk/drivers/ipu/test/ips_resize.c

@@ -42,7 +42,7 @@

int32_t ips_resize_test(ips_dev_panel_t * panel)

{

     uint32_t ipu_index = 1;     // use ipu 1

-    uint32_t taskType = PP_TASK;

+    uint32_t taskType = PrP_VF_TASK;

     uint32_t rot_in_mem = CH28_EBA0, rot_out_mem = CH27_EBA0, res_out_mem = CH23_EBA0;

     ipu_rot_info_t rot_info;

     ipu_res_info_t res_info;

@@ -134,8 +134,8 @@ int32_t ips_resize_test(ips_dev_panel_t * panel)

                 ipu_ic_resize_config(ipu_index, taskType, res_info);

                 //enable ic task

-                ipu_ic_task_enable(ipu_index, PP_TASK, IC_PP, 1);

-                ipu_ic_task_enable(ipu_index, PP_TASK, IC_ROT, 1);

+                ipu_ic_task_enable(ipu_index, taskType, IC_PP, 1);

+                ipu_ic_task_enable(ipu_index, taskType, IC_ROT, 1);

                 ipu_ic_enable(ipu_index, 1, 1);

                 //enable rotate idma channel

@@ -175,8 +175,8 @@ int32_t ips_resize_test(ips_dev_panel_t * panel)

                 ipu_ic_rotation_config(ipu_index, taskType, 1, 0, 0);

                 //enable ic task

-                ipu_ic_task_enable(ipu_index, PP_TASK, IC_PP, 1);

-                ipu_ic_task_enable(ipu_index, PP_TASK, IC_ROT, 1);

+                ipu_ic_task_enable(ipu_index, taskType, IC_PP, 1);

+                ipu_ic_task_enable(ipu_index, taskType, IC_ROT, 1);

                 ipu_ic_enable(ipu_index, 1, 1);

                 //enable rotate idma

[johnturnur]

Hello Juan,

Thanks for your reply. I have already made the changes to enable IPU_IC_CONF__PRPVF_ROT_EN for View finder task.

I am working on YUV420 video data and using view finder task i am trying to rotate. I am able to rotate video but the output is not clear.

For your reference i am setting following iDMA channel parameters to rotate YUV420 data.

uint32_t rot_chnl_in = MEM_TO_IC_PRPVF_ROT_CH46;

    uint32_t rot_chnl_out = IC_PRPVF_ROT_TO_MEM_CH49;

memset(&rot_info, 0x00, sizeof(ipu_rot_info_t));

  rot_info.width_in =  res_info.width_out;//panel->width;

  rot_info.height_in = res_info.height_out; //panel->height;

  rot_info.width_out = rot_info.height_in;

  rot_info.height_out = rot_info.width_in;

  //rot_info.strideline_in = rot_info.width_in * 2;

  //rot_info.strideline_out = rot_info.width_out * 2;

  rot_info.pixel_format_in = NON_INTERLEAVED_YUV420;

  rot_info.pixel_format_out = NON_INTERLEAVED_YUV420;

  rot_info.rot = 1;

  rot_info.hf = 0;

  rot_info.vf = 0;

  rot_info.addr0_in =rot_in_mem;

  rot_info.addr0_out = vdi_out_mem;

  rot_info.strideline_in = rot_info.width_in;

  rot_info.strideline_out = rot_info.width_out;//panel->width;

  rot_info.u_offset_in = rot_info.width_in * rot_info.height_in;

  rot_info.u_offset_out = rot_info.height_out* rot_info.width_out;//panel->width * panel->height;

  ipu_rotate_idmac_config(ipu_index, rot_chnl_in, rot_chnl_out, rot_info);

please help if this parameters are wrong or something need to modify? Also, Can you please share your Idam channel parameters of your test?

Thanks,

J

[juangutierrez]

Hi

I tried using a yuv422 image and i got the same problem when rotated

This is my configuration

rot_info.width_in = 640;

rot_info.height_in = 480;

rot_info.width_out = rot_info.height_in;

rot_info.height_out = rot_info.width_in;

rot_info.strideline_in = panel->width * 2;

rot_info.strideline_out = panel->width * 2;

rot_info.pixel_format_in = NON_INTERLEAVED_YUV422;

rot_info.pixel_format_out = NON_INTERLEAVED_YUV422;

    rot_info.rot = 1;        

    rot_info.hf = 0;         

    rot_info.vf = 0;

rot_info.addr0_in =      

        rot_in_mem + (panel->height - rot_info.height_in) * panel->width + panel->width -

        rot_info.width_in;

    rot_info.addr0_out =

rot_out_mem + (panel->height - rot_info.height_out) * panel->width + panel->width -

        rot_info.width_out;

/*rotate idmac config */

ipu_rotate_idmac_config(ipu_index, channel_in, channel_out, rot_info);

One alternative solution might be to do color transformation from yuv to rgb.before rotating the image and then convert back to yuv.

I'm investigating if the yuv422 is supported by rotation or if some config is missing. I will let you know as soon as I have some info.

[juangutierrez]

Hi

I tested the rotation on Linux and it is working fine for yuv422p. So, the format is supported

I dumped the info from linux with the idma configuration

So, this can be helpful to see if there is something missing in the SDK.

Resizing is made first and then rotation.

Please take a look and check if you can find some idma config missing in the sdk.

imx-ipuv3 imx-ipuv3.0: [0xdf938000]input:

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     format = 0x50323234 <-- yuv422p in linux

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     width = 1024

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     height = 768

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     crop.w = 1024

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     crop.h = 768

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     crop.pos.x = 0

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     crop.pos.y = 0

imx-ipuv3 imx-ipuv3.0: [0xdf938000]input buffer:

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     paddr = 0x18a00000

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     i_off = 0x0

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     i_uoff = 0xc0000

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     i_voff = 0x120000

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     istride = 1024

imx-ipuv3 imx-ipuv3.0: [0xdf938000]output:

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     format = 0x50323234 <-- yuv422p in linux

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     width = 1024

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     height = 768

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     crop.w = 1024

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     crop.h = 768

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     crop.pos.x = 0

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     crop.pos.y = 0

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     rotate = 5

imx-ipuv3 imx-ipuv3.0: [0xdf938000]output buffer:

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     paddr = 0x18000000

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     o_off = 0x0

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     o_uoff = 0xc0000

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     o_voff = 0x120000

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     ostride = 1024

imx-ipuv3 imx-ipuv3.0: [0xdf938000]want task_id = 1

imx-ipuv3 imx-ipuv3.0: [0xdf938000]want task mode is 0x3

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     IC_MODE = 0x1

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     ROT_MODE = 0x2

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     VDI_MODE = 0x4

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     Task_no = 0x100

imx-ipuv3 imx-ipuv3.0: [0xdf938000]ic channel MEM_PRP_VF_MEM

imx-ipuv3 imx-ipuv3.0: [0xdf938000]rot channel MEM_ROT_VF_MEM

imx-ipuv3 imx-ipuv3.0: ipu_channel_status, dma_status:0.

imx-ipuv3 imx-ipuv3.0: ipu_channel_status, dma_status:0.

imx-ipuv3 imx-ipuv3.0: ipu_channel_status, dma_status:0.

imx-ipuv3 imx-ipuv3.0: [0xdf938000]ic + rot mode

imx-ipuv3 imx-ipuv3.0: [0xdf938000]rotation:

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     format = 0x50323234 <-- yuv422p in linux

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     width = 768

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     height = 1024

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     paddr = 0x18e00000

imx-ipuv3 imx-ipuv3.0: [0xdf938000]     rstride = 768

imx-ipuv3 imx-ipuv3.0: init channel = 5

imx-ipuv3 imx-ipuv3.0: resizing from 768 -> 1024 pixels, downsize=0, resize=0.7496 (reg=6141)

imx-ipuv3 imx-ipuv3.0: resizing from 1024 -> 768 pixels, downsize=0, resize=1.3337 (reg=10926)

imx-ipuv3 imx-ipuv3.0: initializing idma ch 12 @ ea8c0300

imx-ipuv3 imx-ipuv3.0: ch 12 word 0 - 00000000 60000000 00240000 E0000000 000BFC7F

imx-ipuv3 imx-ipuv3.0: ch 12 word 1 - 03140000 00000000 2023C000 0000FFC0 000001FF

imx-ipuv3 imx-ipuv3.0: PFS 0x1,

imx-ipuv3 imx-ipuv3.0: BPP 0x0,

imx-ipuv3 imx-ipuv3.0: NPB 0xf

imx-ipuv3 imx-ipuv3.0: FW 1023,

imx-ipuv3 imx-ipuv3.0: FH 767,

imx-ipuv3 imx-ipuv3.0: EBA0 0x18a00000

imx-ipuv3 imx-ipuv3.0: EBA1 0x0

imx-ipuv3 imx-ipuv3.0: Stride 1023

imx-ipuv3 imx-ipuv3.0: scan_order 0

imx-ipuv3 imx-ipuv3.0: uv_stride 511

imx-ipuv3 imx-ipuv3.0: u_offset 0xc0000

imx-ipuv3 imx-ipuv3.0: v_offset 0x120000

imx-ipuv3 imx-ipuv3.0: Width0 0+1,

imx-ipuv3 imx-ipuv3.0: Width1 0+1,

imx-ipuv3 imx-ipuv3.0: Width2 0+1,

imx-ipuv3 imx-ipuv3.0: Width3 0+1,

imx-ipuv3 imx-ipuv3.0: Offset0 31,

imx-ipuv3 imx-ipuv3.0: Offset1 15,

imx-ipuv3 imx-ipuv3.0: Offset2 0,

imx-ipuv3 imx-ipuv3.0: Offset3 0

imx-ipuv3 imx-ipuv3.0: initializing idma ch 21 @ ea8c0540

imx-ipuv3 imx-ipuv3.0: ch 21 word 0 - 00000000 60000000 00240000 E0000000 000FFC5F

imx-ipuv3 imx-ipuv3.0: ch 21 word 1 - 031C0000 00000000 2023C000 0000BFC0 0000017F

imx-ipuv3 imx-ipuv3.0: PFS 0x1,

imx-ipuv3 imx-ipuv3.0: BPP 0x0,

imx-ipuv3 imx-ipuv3.0: NPB 0xf

imx-ipuv3 imx-ipuv3.0: FW 767,

imx-ipuv3 imx-ipuv3.0: FH 1023,

imx-ipuv3 imx-ipuv3.0: EBA0 0x18e00000

imx-ipuv3 imx-ipuv3.0: EBA1 0x0

imx-ipuv3 imx-ipuv3.0: Stride 767

imx-ipuv3 imx-ipuv3.0: scan_order 0

imx-ipuv3 imx-ipuv3.0: uv_stride 383

imx-ipuv3 imx-ipuv3.0: u_offset 0xc0000

imx-ipuv3 imx-ipuv3.0: v_offset 0x120000

imx-ipuv3 imx-ipuv3.0: Width0 0+1,

imx-ipuv3 imx-ipuv3.0: Width1 0+1,

imx-ipuv3 imx-ipuv3.0: Width2 0+1,

imx-ipuv3 imx-ipuv3.0: Width3 0+1,

imx-ipuv3 imx-ipuv3.0: Offset0 31,

imx-ipuv3 imx-ipuv3.0: Offset1 11,

imx-ipuv3 imx-ipuv3.0: Offset2 0,

imx-ipuv3 imx-ipuv3.0: Offset3 0

imx-ipuv3 imx-ipuv3.0: init channel = 2

imx-ipuv3 imx-ipuv3.0: initializing idma ch 46 @ ea8c0b80

imx-ipuv3 imx-ipuv3.0: ch 46 word 0 - 00000000 60000000 00240000 E2A00000 000FFC5F

imx-ipuv3 imx-ipuv3.0: ch 46 word 1 - 031C0000 00000000 2021C000 0000BFC0 0000017F

imx-ipuv3 imx-ipuv3.0: PFS 0x1,

imx-ipuv3 imx-ipuv3.0: BPP 0x0,

imx-ipuv3 imx-ipuv3.0: NPB 0x7

imx-ipuv3 imx-ipuv3.0: FW 767,

imx-ipuv3 imx-ipuv3.0: FH 1023,

imx-ipuv3 imx-ipuv3.0: EBA0 0x18e00000

imx-ipuv3 imx-ipuv3.0: EBA1 0x0

imx-ipuv3 imx-ipuv3.0: Stride 767

imx-ipuv3 imx-ipuv3.0: scan_order 0

imx-ipuv3 imx-ipuv3.0: uv_stride 383

imx-ipuv3 imx-ipuv3.0: u_offset 0xc0000

imx-ipuv3 imx-ipuv3.0: v_offset 0x120000

imx-ipuv3 imx-ipuv3.0: Width0 0+1,

imx-ipuv3 imx-ipuv3.0: Width1 0+1,

imx-ipuv3 imx-ipuv3.0: Width2 0+1,

imx-ipuv3 imx-ipuv3.0: Width3 0+1,

imx-ipuv3 imx-ipuv3.0: Offset0 31,

imx-ipuv3 imx-ipuv3.0: Offset1 11,

imx-ipuv3 imx-ipuv3.0: Offset2 0,

imx-ipuv3 imx-ipuv3.0: Offset3 0

imx-ipuv3 imx-ipuv3.0: initializing idma ch 49 @ ea8c0c40

imx-ipuv3 imx-ipuv3.0: ch 49 word 0 - 00000000 60000000 00240000 E0200000 000BFC7F

imx-ipuv3 imx-ipuv3.0: ch 49 word 1 - 03000000 00000000 2021C000 0000FFC0 000001FF

imx-ipuv3 imx-ipuv3.0: PFS 0x1,

imx-ipuv3 imx-ipuv3.0: BPP 0x0,

imx-ipuv3 imx-ipuv3.0: NPB 0x7

imx-ipuv3 imx-ipuv3.0: FW 1023,

imx-ipuv3 imx-ipuv3.0: FH 767,

imx-ipuv3 imx-ipuv3.0: EBA0 0x18000000

imx-ipuv3 imx-ipuv3.0: EBA1 0x0

imx-ipuv3 imx-ipuv3.0: Stride 1023

imx-ipuv3 imx-ipuv3.0: scan_order 0

imx-ipuv3 imx-ipuv3.0: uv_stride 511

imx-ipuv3 imx-ipuv3.0: u_offset 0xc0000

imx-ipuv3 imx-ipuv3.0: v_offset 0x120000

imx-ipuv3 imx-ipuv3.0: Width0 0+1,

imx-ipuv3 imx-ipuv3.0: Width1 0+1,

imx-ipuv3 imx-ipuv3.0: Width2 0+1,

imx-ipuv3 imx-ipuv3.0: Width3 0+1,

imx-ipuv3 imx-ipuv3.0: Offset0 31,

imx-ipuv3 imx-ipuv3.0: Offset1 15,

imx-ipuv3 imx-ipuv3.0: Offset2 0,

imx-ipuv3 imx-ipuv3.0: Offset3 0

[juangutierrez · Accepted solution]

Good news. i was able to perform the rotation with the yuv422p image in the SDK :smileyhappy:

The trick is to set the U offset. Since the image is planar (non_interleaved) we can see it as three separate buffers: Y buffer, U buffer and V buffer. However, we need to indicate how far (the offset) the U buffer and Y buffer  are from the Y buffer. Actually set the u offset is enough.

Check the next config I used to perform first re-sizing and then rotation

                memset(&res_info, 0x00, sizeof(ipu_res_info_t));

                res_info.width_in = panel->width;

                res_info.height_in = panel->height;

                res_info.height_out = panel->width;

                res_info.width_out = panel->height;

                res_info.strideline_in = res_info.width_in;

                res_info.strideline_out = res_info.width_out;

                res_info.pixel_format_in = NON_INTERLEAVED_YUV422; // INTERLEAVED_RGB565;

                res_info.pixel_format_out = NON_INTERLEAVED_YUV422; // INTERLEAVED_RGB565;                                                                                      

                res_info.addr0_in = res_in_mem;

                res_info.addr0_out = res_out_mem;

                res_info.u_offset_in = 0xc0000;   // hardcoded for a 1024 x 768 image

                res_info.u_offset_out = 0xc0000;                                                                                                                                

                ipu_resize_idmac_config(ipu_index, res_chnl_in, res_chnl_out, res_info);                                                                                        

                //set rotate idma             

                memset(&rot_info, 0x00, sizeof(ipu_rot_info_t));

                rot_info.width_in = panel->height;

                rot_info.height_in = panel->width;

                rot_info.width_out = panel->width;

                rot_info.height_out = panel->height;

                rot_info.strideline_in = rot_info.width_in;

                rot_info.strideline_out = rot_info.width_out;

                rot_info.pixel_format_in = NON_INTERLEAVED_YUV422; //INTERLEAVED_RGB565;

                rot_info.pixel_format_out = NON_INTERLEAVED_YUV422; //INTERLEAVED_RGB565;                                                                                       

                rot_info.rot = 1;             

                rot_info.hf = 0;              

                rot_info.vf = 0;              

                rot_info.addr0_in = res_out_mem;

                rot_info.addr0_out = rot_out_mem;

                rot_info.ubo_in = 0xc0000;    

                rot_info.ubo_out = 0xc0000;

                ipu_rotate_idmac_config(ipu_index, rot_chnl_in, rot_chnl_out, rot_info);

[johnturnur]

Thanks for your reply. I have solved the problem same way you mentioned above.

[johnturnur]

Hi Juan,

I am able to rotate the video but the screen flickers when rotating the video.

We are using Platform SDK V1.1 driver code of IPU. We are not using V4L2 driver for above application.

Is rotation of View finder task introduce flickering of the screen? How to resolve this issue.

Are  you facing same issue?

Please help

Thanks

J.

[juangutierrez]

I can see the flicker too :smileysad:. However, in linux  the transition is smooth, so maybe some parameter is missing or not sure if related with cache.

Could you check with the rest of the parameters from the linux dump above?