This is bit old question, but I've tried to compile 3-series kernel for i.MX53 hardware. I downloaded 3.8.2 and 3.16.4 from kernel.org and got them both compiled for i.MX53. Problem is that neither of them work. To be more precise they won't do anything ie. won't even boot.
I've seen mentions here people compiling and booting kernel 3.8.2 successfully for i.MX53 so it is possible.
I think on 3-series kernel you need to set LOADADDR that wasn't needed for example 2.6.35 - kernel. I've tried 0x70800000 and 0x70008000, but no help. u-boot loads the kernel image no problem and crc checks out so that's good, but then I get "Starting kernel..." - text and nothing else.
I understood that 3-series kernel support for i.MX53 isn't as good as for i.MX6, but it should still work? Is there Freescale version of that kernel for i.MX53 or patch set that can be applied for vanilla kernel like for 2.6.35? Or any instructions how to get the 3- series kernel compiled for i.MX53? I don't seem to be able to get it boot. I've tried with QSB and Sabre-boards.