I have a T2080QDS board and I purchased a 16 GB DDR chip for it. I am allocating 4 GB of DDR RAM to each Linux Guest OS. When I do this, my PCIe Ethernet NICs no longer function. They do not receive the MSI interrupt. If I allocate 2 GB to each Linux Guest, then it works fine.
I believe I've traced the issue to the fsl_pci.c file, function setup_pci_atmu(). In this function, it is checking the msi-address-64 property, which is set because I am running Topaz hypervisor. But since we have to extend the DDR ATMU to accommodate the MSIIR register, we now have greater than 4GB of DDR allocated which causes us to setup a 2nd PCI inbound window later in the function. I'm assuming this is where we have issues with the Hypervisor model, as I never see the MSI interrupt happen in linux (I have debug in the fsl_msi_cascade function).
This works fine if I just boot Linux and have 16GB of RAM allocated to it. It only doesn't work in the Hypervisor scenario.
Anyone have any ideas what might be the issue?