iMX8 BAR space increase (Kernel: 4.14.98)

[vinuchandran]

How to increase the BAR space on iMX8?

I'm getting the following error when trying to connect an FPGA over PCIe.

[ 4.877122] pci 0000:00:00.0: BAR 14: assigned [mem 0x18000000-0x185fffff]
[ 4.884071] pci 0000:00:00.0: BAR 0: assigned [mem 0x18600000-0x186fffff 64bit]
[ 4.891456] pci 0000:00:00.0: BAR 6: assigned [mem 0x18700000-0x1870ffff pref]
[ 4.898754] pci 0000:01:00.0: BAR 2: assigned [mem 0x18000000-0x183fffff 64bit]
[ 4.906116] pci 0000:01:00.0: BAR 0: assigned [mem 0x18400000-0x18407fff 64bit]
[ 5.122276] pci 0001:00:00.0: BAR 0: assigned [mem 0x20000000-0x200fffff 64bit]
[ 5.129598] pci 0001:00:00.0: BAR 14: assigned [mem 0x20100000-0x202fffff]
[ 5.136542] pci 0001:00:00.0: BAR 6: assigned [mem 0x20300000-0x2030ffff pref]
[ 5.143829] pci 0001:01:00.0: BAR 14: assigned [mem 0x20100000-0x201fffff]
[ 5.150767] pci 0001:01:00.0: BAR 0: assigned [mem 0x20200000-0x2021ffff]
[ 5.157572] pci 0001:01:00.1: BAR 0: assigned [mem 0x20220000-0x2023ffff]
[ 5.164432] pci 0001:02:01.0: BAR 14: assigned [mem 0x20100000-0x201fffff]
[ 5.171318] pci 0001:03:00.0: BAR 0: assigned [mem 0x20100000-0x2011ffff]
[ 5.178177] pci 0001:03:00.1: BAR 0: assigned [mem 0x20120000-0x2013ffff]
[ 239.307003] pcieport 0001:03:00.0: BAR 14: failed to assign [mem size 0x00300000]
[ 239.322155] pcieport 0001:04:00.0: BAR 14: failed to assign [mem size 0x00300000]
[ 239.337393] pci 0001:05:00.0: BAR 0: failed to assign [mem size 0x00100000 64bit]
[ 239.352532] pci 0001:05:00.0: BAR 2: failed to assign [mem size 0x00100000 64bit]
[ 239.367058] pci 0001:05:00.0: BAR 4: failed to assign [mem size 0x00001000]
[ 324.213416] pcieport 0001:04:00.0: BAR 14: failed to assign [mem size 0x00300000]
[ 324.228415] pci 0001:05:00.0: BAR 0: failed to assign [mem size 0x00100000 64bit]
[ 324.243452] pci 0001:05:00.0: BAR 2: failed to assign [mem size 0x00100000 64bit]
[ 324.257952] pci 0001:05:00.0: BAR 4: failed to assign [mem size 0x00001000]

Its also given in the dmesg that,

[    0.000000]     PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000   (    16 MB)

 the PCIe I/O size is 16MB.

What is the procedure to increase the 16MB limit to say 128MB?

[igorpadykov]

Hi Vinuchandran

>How to increase the BAR space on iMX8?

max. bar size is limited by the max. size of the PCIe memory space allocated by the SOC.

From sect.2.2 System Memory Map  i.MX 8DualXPlus/8QuadXPlus Applications Processor Reference Manual       256MB of address space reserved for PCIe in i.MX8X.
So max. bar size can be reached only one bar is required by EP.

Best regards
igor

[vinuchandran · Accepted solution]

Thanks @igorpadykov

My issue is resolved. I was getting the "No space for mem and Failed to assign mem" errors when I program the FPGA after booting the iMX8 (My total mem requirement is ~20MB) and do a PCIe rescan.

If the bitstream is loaded before the iMX is booting, this issue is not encountered.

Seems like iMX doesn't fully support hotplug, maybe? (CONFIG_HOTPLUG_PCI_PCIE is already enabled)

[abelal]

Hi @vinuchandran,

1. How did you manage to load bitstream before the iMX8 started booting?
   • JTAG?
   • U-Boot?
   • Program and reboot?
2. Also once this was done were you able to load a new bitstream after the iMX8 booted up and do a PCIe rescan?

[igorpadykov]

right, hotplug is not supported.

Best regards
igor