We are just moving 1600bytes buffers from the FPGA to the imx6, we have seveal buffers sending data and we are trying to keep latency as low as possible.
If you were able to allocate a 8Meg buffer then there should be no problem to transfer such amount of memory.
We tested our driver against the 3.10 and 4.1 kernel and is an almost unmodified driver we developed for a PC platform. We only had to change DMA target address in the FPGA DMA controller from 64 bits to 32 bits to get it working.