not able to have data transfer with EP FPGA in T208x based board

[monalihaware]

i have a custom board with T208x as RC and kintex FPGA as EP.  The RC is able to detect the FPGA as memory controller while booting. The uboot is as follows:

U-Boot 2016.092.0+ga06b209 (Dec 09 2021 - 15:03:17 +0530)
CPU0: T2081E, Version: 1.1, (0x85390011)
Core: e6500, Version: 2.0, (0x80400120)
Clock Configuration:
CPU0:933.333 MHz, CPU1:933.333 MHz, CPU2:933.333 MHz, CPU3:933.333 MHz,
CCB:533.333 MHz,
DDR:733.333 MHz (1466.667 MT/s data rate) (Asynchronous), IFC:533.333 MHz
FMAN1: 466.667 MHz
QMAN: 266.667 MHz
PME: 533.333 MHz
L1: D-cache 32 KiB enabled
I-cache 32 KiB enabled
Reset Configuration Word (RCW):
00000000: 080b0007 07000000 00000000 00000000
00000010: ab000002 00404000 ec027000 c1000000
00000020: 00000000 00000000 00000000 000305fc
00000030: 00000000 00000000 00000000 00000004
Board: T2081QDS, Board Version: ECRDSP1, boot from NOR FlashSERDES Reference Cl
ocks:
SD1_CLK1 = 100MHz,SD1_CLK2= Not available
I2C: ready
SPI: ready
DRAM: Initializing....using Fixed Parameter
Num of DDR controller is: 1
Configuring DDR for 1466.667 MT/s data rate
2 GiB (DDR3, 64-bit, CL=11, ECC off)
PCA: failed to select proper channel
PCA: failed to select proper channel
Warning: Adjusting core voltage failed.
PCA: failed to select proper channel
Flash:
256 MiB
L2: 2 MiB enabled
Corenet Platform Cache: 512 KiB enabled
Using SERDES1 Protocol: 171 (0xab)
SEC0: RNG instantiated
NAND: 0 MiB
MMC: FSL_SDHC: 0
MMC: no card present
EEPROM: Read failed.
PCIe1: disabled
PCIe2: disabled
PCIe3: Root Complex,
x4 gen2, regs @ 0xfe260000
01:00.0 - 10ee:0008 - Memory controller
PCIe3: Bus 00 - 01
PCIe4: Root Complex,
no link, regs @ 0xfe270000
PCIe4: Bus 02 - 02
In: serial
Out: serial
Err: serial
Net: SerDes1 protocol 0xab is not supported
Fman1: Uploading microcode version 108.4.5
FM1@DTSEC3, FM1@DTSEC4 [PRIME]
Hit any key to stop autoboot: 0

The commands for PCIe is as follows for bus 0 and bus-

pci 0 long =>
Scanning PCI devices on bus 0

Found PCI device 00.00.00:
vendor ID = 0x1957
device ID = 0x0838
command register ID = 0x0006
status register = 0x0010
revision ID = 0x11
class code = 0x0b (Processor)
sub class code = 0x20
programming interface = 0x00
cache line = 0x08
latency time = 0x00
header type = 0x01
BIST = 0x00
base address 0 = 0x00000000
base address 1 = 0x00000000
primary bus number = 0x00
secondary bus number = 0x01
subordinate bus number = 0x01
secondary latency timer = 0x00
IO base = 0x10
IO limit = 0x00
secondary status = 0x0000
memory base = 0xe000
memory limit = 0xe100
prefetch memory base = 0x1000
prefetch memory limit = 0x0000
prefetch memory base upper = 0x00000000
prefetch memory limit upper = 0x00000000
IO base upper 16 bits = 0x0000
IO limit upper 16 bits = 0x0000
expansion ROM base address = 0xe0000000
interrupt line = 0xff
interrupt pin = 0x01
bridge control = 0x0000

=> pci 1 long
Scanning PCI devices on bus 1

Found PCI device 01.00.00:
vendor ID = 0x10ee
device ID = 0x0008
command register ID = 0x0006
status register = 0x0010
revision ID = 0x00
class code = 0x05 (Memory controller)
sub class code = 0x80
programming interface = 0x00
cache line = 0x08
latency time = 0x00
header type = 0x00
BIST = 0x00
base address 0 = 0xe1000000
base address 1 = 0x00000000
base address 2 = 0x00000000
base address 3 = 0x00000000
base address 4 = 0x00000000
base address 5 = 0x00000000
cardBus CIS pointer = 0x00000000
sub system vendor ID = 0x10ee
sub system ID = 0x0007
expansion ROM base address = 0x00000000
interrupt line = 0xff
interrupt pin = 0x01
min Grant = 0x00
max Latency = 0x00

The configuration space -header 0 of EP i am able to read using pcie commands in the uboot as follows-

=> mw 0xfe260000 80010000
=> md.l fe260004 1
fe260004: ee100800 ....

The pcie regsiters inside the processor for PCIe3 and the LAWs are as follows-

md 0xfe260c20 50
fe260c20: 000e0000 00000000 000c0000 00000000 ................
fe260c30: 8004401b 00000000 00000000 00000000 ..@.............
fe260c40: 00000000 00000000 000f8020 00000000 ........... ....
fe260c50: 8008800f 00000000 00000000 00000000 ................
fe260c60: 00000000 00000000 00000000 00000000 ................
fe260c70: 00000000 00000000 00000000 00000000 ................
fe260c80: 00000000 00000000 00000000 00000000 ................
fe260c90: 00044027 00000000 00000000 00000000 ..@'............
fe260ca0: 00000000 00000000 00000000 00000000 ................
fe260cb0: 00000000 00000000 00000000 00000000 ................
fe260cc0: 00000000 00000000 00000000 00000000 ................
fe260cd0: 00000000 00000000 00000000 00000000 ................
fe260ce0: 00000000 00000000 00000000 00000000 ................
fe260cf0: 20f44017 00000000 00000000 00000000 .@.............
fe260d00: 00ffe000 00000000 00000000 00000000 ................
fe260d10: 00e44017 00000000 00000000 00000000 ..@.............
fe260d20: 00000000 00000000 00000000 00000000 ................
fe260d30: 00000000 00000000 00000000 00000000 ................
fe260d40: 00000000 00000000 00000000 00000000 ................
fe260d50: 00000000 00000000 00000000 00000000 ................
=> md 0xfe000c00 200
fe000c00: 0000000f e0000000 81f0001b 00000000 ................
fe000c10: 0000000f f4000000 81800018 00000000 ................
fe000c20: 0000000f f6000000 83c00018 00000000 ................
fe000c30: 0000000f 00000000 81d00018 00000000 ................
fe000c40: 0000000f ff800000 81f00013 00000000 ................
fe000c50: 00000000 c0000000 8020001b 00000000 ......... ......
fe000c60: 00000000 f8020000 8020000f 00000000 ......... ......
fe000c70: 00000000 d0000000 8030001b 00000000 .........0......
fe000c80: 0000000f f8030000 8030000f 00000000 .........0............
fe000de0: 00000000 00000000 00000000 00000000 ................
fe000df0: 00000000 00000000 8100001e 00000000 ................
fe000e00: 00000000 00000000 00000000 00000000 ................

AFter all these i am not able to read any content of the FPGA, nor able to write to it using following commands-

=>md 0xc1000000 1
c1000000:
=>md 0xc0000000 1
c0000000:

the uboot hands here. Can any one tell me where the problem could be. I am not able to trace any mistake.

[yipingwang]

According to the following configuration in include/configs/T208xRDB.h, the virtual address of PCIE3 should be 0xb0000000.

/* controller 3, Slot 1, tgtid 1, Base address 202000 */
#define CONFIG_SYS_PCIE3_MEM_VIRT 0xb0000000
#define CONFIG_SYS_PCIE3_MEM_PHYS 0xc30000000ull
#define CONFIG_SYS_PCIE3_IO_VIRT 0xf8020000
#define CONFIG_SYS_PCIE3_IO_PHYS 0xff8020000ull

[monalihaware]

Hello yiping, I did not understand how what you said is related to my problem.. am I accessing wrong address?

The t208xqds.h file I have not changed, except for the fact that I assigned 32 bit addresses to pcie BAr outbound registers.