You can customize the file ./imx50_rdp/required_hardware.xml to remove those feature you dont need in android framework.
And remove those service/daemon you dont need in init.rc
"pmem_adsp": is use as a physical-continuous buffer allocator for hardware video decoder and encoder. Since there is no vpu in mx50, you can remove the pmem_adsp in kernel
"pmem_gpu": is use as the physical-continuous buffer allocator for the android hal Gralloc. Android framework use the allocator to allocate the buffer for UI surface. It is highly dependents on your device display's resolution.
The size >= 5* display_w × display_h *sizeof(RGBA8888) * 2