i.MX6 IPU and rotating a frame buffer

[jasonhaedt]

I'll try to make a long story as short as possible :smileyhappy: (let me know if I skip any important details).  Hopefully this is an easy one, but we're still a little green on this part and the IPU is a pretty complex beast, so I'm banging my head at the moment.  The LCD we're planning on using for this product is FHD/portrait mode (so, 1080x1920) connected via DSI.  We've written a driver for this to glue in to the existing DSI driver (so, pretty much like the hx8369 driver down in drivers/video/mxc).  So far so good - everything works at this point.  However, the physical orientation of the display will be landscape i.e. 1920x1080, so we were hoping to use the IPU's IRT to rotate the final framebuffer 90 degrees.  In order to not have to write plugins for our userspace application(s) to worry about setting up the IPU to do this (and also just to try to get a proof-of-concept going), I've tried to modify the IPU framebuffer driver (mxc_ipuv3_fb.c) to accommodate this.  The quick hack to test this out was just to have the driver allocate an extra framebuffer down in mxcfb_map_video_memory (we'll call this new buffer A and the original buffer B).  This new buffer is the one that now gets attached to the kernel framebuffer i.e. fb_info->screen_base (so this is the one that will get rendered in to).  I set up DMA through the IRT using MEM_ROT_VF_MEM (channel 46 in, channel 49 out) with buffer A being the input and buffer B being the output (initializing both with ipu_init_channel_buffer) (the rotation parameter is set to IPU_ROTATE_90_RIGHT, input resolution is x=1920/y=1080/stride=(3*1920) (we're running rgb/24bpp), output resolution is x=1080/y=1920/stride=(3*1080)).  Finally I link the rotation channel to the display channel (ipu_link_channels, MEM_ROT_VF_MEM -> MEM_FG_SYNC).

With this setup, my rotated screen is pretty much garbage (so much so that there's no discernible pattern to guess as to what might be wrong).  The weird part though is if I cut down horizontal resolution of the input framebuffer (and of course the framebuffer structure that gets registered) to 1080x1080, the rotation works fine (I believe I've tried non-square resolutions too with a lower x resolution and they worked okay, but I'll have to double-check that one).

Okay, I just did some more digging in to the underlying registers that are getting configured by the IPU driver and I see that the width/height fields in IPUx_IC_IDMAC_3/4 are limited to 10 bits (1024 pixels), so I assume that I'd need to figure a way to break the 1080x1920 rotation in to four separate DMA transfers, correct?

[haff]

Hello, I am working now on the same problem.

Jason Haedt wrote:

Okay, I just did some more digging in to the underlying registers that are getting configured by the IPU driver and I see that the width/height fields in IPUx_IC_IDMAC_3/4 are limited to 10 bits (1024 pixels), so I assume that I'd need to figure a way to break the 1080x1920 rotation in to four separate DMA transfers, correct?

Have you found a way to do it? Or any other workaround?

[jasonhaedt]

Sorry to be a nuisance, but after some more digging I've got some more information on what we're seeing and I was sort hoping to do a sanity-check with you guys.  I wrote a little program to enable the profiler in the MMDC just to get a rough idea of what DDR utilization looks like in a few different scenarios.  I don't think I mentioned before, but we're running on the sabresd board right now with a little custom daughterboard to a mipi dsi display.  DDR clock is at 528MHz, and we're running around a 66MHz pixel clock.  I'm calculating utilization by just dividing MADPSR1/MADPSR0.  What I see is

display blanked - < 1%

display unblanked/unrotated - 28%

display unblanked/rotated - 50%

I was somewhat surprised by these numbers - the 28% for just the display seems a lot higher than I'd expect.  Given approximately a 66MHz pixel clock, the total bandwidth  from the memory should be around 264MByte/sec (66*4 - according to the reference manual the IPU translates everything in/out of memory to 8:8:8:8).  Peak rate for the ddr should be 528*2*64/8 = 8.4GByte/sec (and I realize peak bandwidth is a mostly useless metric here and that average throughput should be considerably lower, but it's a starting point).  Have you guys done any profiling at different resolutions/frame rates, and if so do the numbers we're seeing seem reasonable?

Like I said, sorry to be a nuisance, but we're trying to finalize our hardware design and we're trying to get a handle on whether there are pieces of the design we need to optimize by moving certain processing operations (like screen rotation) out in to our fpga.

Thanks again!

[rogerio_silva · Accepted solution]

Hi Jason,

You're right. The maximum IPU can rotate is 1024x1024. Linux kernel splits the image to show images bigger than 1024x1024.

You can check the file linux/drivers/mxc/ipu3/ipu_device.c about the way it splits. See the "create_split_task" function.

Rgds

Rogerio

[haff]

Hi,

But split task isn't applicable in case of rotate as far as I understand. For this case there is even special error IPU_CHECK_ERR_SPLIT_WITH_ROT. Am I mistaken? Is there any other way to rotate video bigger than 1024x1024?

Best regards,

Lavnikevich Dmitry