topic Re: i.MX6 IPU and rotating a frame buffer in i.MX Processors

i.MX6 IPU and rotating a frame buffer

jasonhaedt — Fri, 12 Jul 2013 13:38:48 GMT

I'll try to make a long story as short as possible :smileyhappy: (let me know if I skip any important details). Hopefully this is an easy one, but we're still a little green on this part and the IPU is a pretty complex beast, so I'm banging my head at the moment. The LCD we're planning on using for this product is FHD/portrait mode (so, 1080x1920) connected via DSI. We've written a driver for this to glue in to the existing DSI driver (so, pretty much like the hx8369 driver down in drivers/video/mxc). So far so good - everything works at this point. However, the physical orientation of the display will be landscape i.e. 1920x1080, so we were hoping to use the IPU's IRT to rotate the final framebuffer 90 degrees. In order to not have to write plugins for our userspace application(s) to worry about setting up the IPU to do this (and also just to try to get a proof-of-concept going), I've tried to modify the IPU framebuffer driver (mxc_ipuv3_fb.c) to accommodate this. The quick hack to test this out was just to have the driver allocate an extra framebuffer down in mxcfb_map_video_memory (we'll call this new buffer A and the original buffer B). This new buffer is the one that now gets attached to the kernel framebuffer i.e. fb_info->screen_base (so this is the one that will get rendered in to). I set up DMA through the IRT using MEM_ROT_VF_MEM (channel 46 in, channel 49 out) with buffer A being the input and buffer B being the output (initializing both with ipu_init_channel_buffer) (the rotation parameter is set to IPU_ROTATE_90_RIGHT, input resolution is x=1920/y=1080/stride=(3*1920) (we're running rgb/24bpp), output resolution is x=1080/y=1920/stride=(3*1080)). Finally I link the rotation channel to the display channel (ipu_link_channels, MEM_ROT_VF_MEM -> MEM_FG_SYNC).

With this setup, my rotated screen is pretty much garbage (so much so that there's no discernible pattern to guess as to what might be wrong). The weird part though is if I cut down horizontal resolution of the input framebuffer (and of course the framebuffer structure that gets registered) to 1080x1080, the rotation works fine (I believe I've tried non-square resolutions too with a lower x resolution and they worked okay, but I'll have to double-check that one).

Okay, I just did some more digging in to the underlying registers that are getting configured by the IPU driver and I see that the width/height fields in IPUx_IC_IDMAC_3/4 are limited to 10 bits (1024 pixels), so I assume that I'd need to figure a way to break the 1080x1920 rotation in to four separate DMA transfers, correct?

Re: i.MX6 IPU and rotating a frame buffer

rogerio_silva — Fri, 12 Jul 2013 19:34:17 GMT

Hi Jason,

You're right. The maximum IPU can rotate is 1024x1024. Linux kernel splits the image to show images bigger than 1024x1024.

You can check the file linux/drivers/mxc/ipu3/ipu_device.c about the way it splits. See the "create_split_task" function.

Rgds

Rogerio

Re: i.MX6 IPU and rotating a frame buffer

jasonhaedt — Fri, 19 Jul 2013 14:29:08 GMT

Sorry to be a nuisance, but after some more digging I've got some more information on what we're seeing and I was sort hoping to do a sanity-check with you guys. I wrote a little program to enable the profiler in the MMDC just to get a rough idea of what DDR utilization looks like in a few different scenarios. I don't think I mentioned before, but we're running on the sabresd board right now with a little custom daughterboard to a mipi dsi display. DDR clock is at 528MHz, and we're running around a 66MHz pixel clock. I'm calculating utilization by just dividing MADPSR1/MADPSR0. What I see is

display blanked - < 1%

display unblanked/unrotated - 28%

display unblanked/rotated - 50%

I was somewhat surprised by these numbers - the 28% for just the display seems a lot higher than I'd expect. Given approximately a 66MHz pixel clock, the total bandwidth from the memory should be around 264MByte/sec (66*4 - according to the reference manual the IPU translates everything in/out of memory to 8:8:8:8). Peak rate for the ddr should be 528*2*64/8 = 8.4GByte/sec (and I realize peak bandwidth is a mostly useless metric here and that average throughput should be considerably lower, but it's a starting point). Have you guys done any profiling at different resolutions/frame rates, and if so do the numbers we're seeing seem reasonable?

Like I said, sorry to be a nuisance, but we're trying to finalize our hardware design and we're trying to get a handle on whether there are pieces of the design we need to optimize by moving certain processing operations (like screen rotation) out in to our fpga.

Thanks again!

Re: i.MX6 IPU and rotating a frame buffer

haff — Thu, 16 Jan 2014 12:59:42 GMT

Hello, I am working now on the same problem.

Jason Haedt wrote:

Okay, I just did some more digging in to the underlying registers that are getting configured by the IPU driver and I see that the width/height fields in IPUx_IC_IDMAC_3/4 are limited to 10 bits (1024 pixels), so I assume that I'd need to figure a way to break the 1080x1920 rotation in to four separate DMA transfers, correct?

Have you found a way to do it? Or any other workaround?

Re: i.MX6 IPU and rotating a frame buffer

haff — Tue, 21 Jan 2014 09:46:50 GMT

Hi,

But split task isn't applicable in case of rotate as far as I understand. For this case there is even special error IPU_CHECK_ERR_SPLIT_WITH_ROT. Am I mistaken? Is there any other way to rotate video bigger than 1024x1024?

Best regards,

Lavnikevich Dmitry

Re: i.MX6 IPU and rotating a frame buffer

michelesponchia — Mon, 27 Jan 2014 16:34:11 GMT

Hi, I wonder if you can post some code snippet of your changes in the framebuffer driver: I am stuck in a similar situation, but I get always a "timeout fl*ip" error and I can't figure out why :smileysad:

edit: some words

Re: i.MX6 IPU and rotating a frame buffer

shanghuadai — Fri, 08 Aug 2014 09:50:04 GMT

Hello, Your changes can rotate the UI ?

Re: i.MX6 IPU and rotating a frame buffer

jérômepouiller — Thu, 23 Jul 2015 15:50:39 GMT

Based on architecture proposed by Jason, I wrote a patch to support framebuffer rotation on imx6. It may help someone. It is available here:

jerome-pouiller/mxc-video-rotate · GitHub

Obviously, there are a few limitations, but it works very well.

Re: i.MX6 IPU and rotating a frame buffer

wolnet — Wed, 29 Jul 2015 12:34:49 GMT

Dear Jerome,

thank you for your patch it works well for me when I use 90 degrees and 270 degrees,

but when I rotate 180 degrees the image is splitted into 5 columns.

Do you have any idea what could be the issue ?

Kind regards

Wolfgang

Re: i.MX6 IPU and rotating a frame buffer

marco_cassol — Mon, 14 Sep 2015 16:11:49 GMT

Hi Wolfgang,

The issue is that the code is assuming that every rotation will invert width and height, which is not true for a 180deg rotation where the resolution stays the same.

The following diff based on the Freescale Kernel 3.14.28 and the implementation from Jerome should work with the 8 different kinds of rotation.

Re: i.MX6 IPU and rotating a frame buffer

ivannikolaenko — Fri, 02 Oct 2015 07:04:55 GMT

Hi Marco!

Thank You very much! It works perfect!

Best regards,

Ivan.

Re: i.MX6 IPU and rotating a frame buffer

friederschrempf — Tue, 17 Nov 2015 15:58:18 GMT

Thank you Jérôme and Marco Antonio for providing the patch to enable rotation!

It works fine, but I'm getting error messages like this:

mxc_sdc_fb fb.17: Error updating SDC buf 1 to address=0x42F00000, current buf 2, buf0 ready 1, buf1 ready 0, buf2 ready 0

What do they mean? Is there any way to get rid of them?

Thanks and Regards

Frieder Schrempf

Re: i.MX6 IPU and rotating a frame buffer

jérômepouiller — Thu, 19 Nov 2015 16:09:37 GMT

Hello Frieder,

It means a part of IPU try to access a buffer that is not yet ready. I have just pushed (on my github) a fix hat should solve this issue.

Re: i.MX6 IPU and rotating a frame buffer

jérômepouiller — Thu, 19 Nov 2015 17:14:57 GMT

Hello Wolfgang,

Normally, it should be able to do 180° rotation. But, as Marco pointed out, I have only tested it on kernel 3.10.17 and I won't be surprised some changes will be necessary for other versions.

However, my patch should not be necessary for 180° rotation. Did you try to just call FBIOPUT_VSCREENINFO with rotate member of fb_var_screeninfo set to 1 or 2?

Re: i.MX6 IPU and rotating a frame buffer

vikaspatil — Thu, 26 Nov 2015 12:50:22 GMT

Hi Jerome, Marco Antonio,

Thanks a lot for sharing your work.

I tried attached patch with linux 3.14.28 and I can see the "boot logo" rotated by 180 degree after adding the "video=mxcfb2:dev=lcd,CLAA-WVGA,if=RGB24,rotate=3" in kernel command line.

However I don't see the QT application screens rotated by 180 degree. Do you know if this works with GUI toolkits (e.g. QT) and display server/compositor (wayaland/weston)? or Do I need to add such support in those components?

My platform is based on

i.MX6 Duallite

Linux 3.14.28

Vivante GPU driver imx-gpu-viv_5.0.11.p4.4

Weston 1.8.0

Wyaland-ivi-extension 1.4.0

Qt 5.4.2

QTWayland 5.4.2

Thanks & Regards,

Vikas

Re: i.MX6 IPU and rotating a frame buffer

vikaspatil — Thu, 26 Nov 2015 13:09:22 GMT

Hi Jerome, Marco Antonio,

QT app also works. I was using fb1 for weston, after chnaging it to fb0 it works.

Thanks.

Regards,

Vikas

Re: i.MX6 IPU and rotating a frame buffer

vikaspatil — Fri, 27 Nov 2015 06:06:59 GMT

This code rotates the fb0. Do you know if same way fb1 (overlay) also can be rotated?

Regards,

Vikash

Re: i.MX6 IPU and rotating a frame buffer

jérômepouiller — Tue, 09 Feb 2016 16:51:27 GMT

Hello Vikash,

I don't think it would be possible to also rotate fb1 without redesigning whole architecture. You can try to place `fb0` and `fb1` buffers in contiguous memory and rotate both buffer together, but I am not sure it would work.

Re: i.MX6 IPU and rotating a frame buffer

ivannikolaenko — Mon, 23 May 2016 13:51:32 GMT

Hi all!

We moved to a imx-3.14.52 kernel and I was surprised that it still not implemented rotation of the image. Moreover, I was surprised and very disappointed that patch proposed by Jerome Pouiller is not working anymore. That made me write to this topic.

So, does anyone have any ideas how, after all, to implement a hardware rotation of the image once and for all? jasonhaedt, could you share your solution?
Thank you.

Re: i.MX6 IPU and rotating a frame buffer

alexv — Fri, 03 Feb 2017 12:10:25 GMT

Hi all.

I use kernel 3.14.56 for Udoo. Have someone the patch to this version?

Thanks!