PCI device (64bit memory problem)

manoloruiz · ‎04-26-2016

I am using a 64 bit memory FPGA connected by PCI in my T2080rdb. I am using the next function in a device driver

0002:01:00.0 Memory controller: Xilinx Corporation Device 7022 Subsystem: Xilinx Corporation Device
 0007 Flags: bus master, fast devsel, latency 0 
Memory at <ignored> (64-bit, prefetchable) 
Memory at c40000000 (64-bit, prefetchable) [size=64K] 
Capabilities: [40] Power Management version 3 
Capabilities: [48] MSI: Enable- Count=1/8 Maskable- 64bit+ 
Capabilities: [60] Express Endpoint, MSI 00 
Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00

however, when I use the next functions in my device driver:

unsigned long pci_resource_[start|len|end|flags](struct pci_dev *pdev, int bar);

I have got the next result:

______BAR0______

The value of pci_resource_start is: 0x0

The value of pci_resource_len is: 0x0

However, when I use the same device driver in a PCI device driver that use 32 bit memory, I have got:

______BAR0______

The value of pci_resource_start is: 0xc20000000

The value of pci_resource_len is: 0x100000

any idea why is it happening this?

BR.

manoloruiz · ‎04-29-2016

thank very much for your answer. That is true, however it must be a problem when a PCI device has a 64 bit memory.

Using the next function:

unsigned long pci_resource_[start|len|end|flags](struct pci_dev *pdev, int bar);

I have got the next result:

______BAR0______

The value of pci_resource_start is: 0x0

The value of pci_resource_len is: 0x0

However, when I use the same device driver in a PCI device driver that use 32 bit memory, I have got:

______BAR0______

The value of pci_resource_start is: 0xc20000000

The value of pci_resource_len is: 0x100000

I change dbt in other to say to the kernel that it is a 64 bit memory. But does not work. You can see:

PCI addresses are 64 bits wide, and are encoded into phys.mid and phys.low. However, the really interesting things are in phys.high which is a bit field:

n: relocatable region flag (doesn't play a role here)
p: prefetchable (cacheable) region flag
t: aliased address flag (doesn't play a role here)
ss: space code
- 00: configuration space
- 01: I/O space
- 10: 32 bit memory space
- 11: 64 bit memory space

Any idea?

manoloruiz · ‎05-05-2016

Any idea? Should I modify the u-boot?

ufedor · ‎05-05-2016

I see that U-Boot correctly configured BAR2.

I assume that the issue is connected with the FPGA.

To obtain additional information it is needed to debug U-Boot PCI enumeration code.

manoloruiz · ‎04-28-2016

where I can find that log?

ufedor · ‎04-28-2016

I need the console output (printed by U-Boot) before the "pci header" command.

manoloruiz · ‎04-28-2016

U-Boot 2015.01+SDKv1.9+geb3d4fc (Apr 28 2016 - 09:54:26)

CPU0: T2080E, Version: 1.1, (0x85380011)

Core: e6500, Version: 2.0, (0x80400120)

Clock Configuration:

CPU0:1799.820 MHz, CPU1:1799.820 MHz, CPU2:1799.820 MHz, CPU3:1799.820 MHz,

CCB:599.940 MHz,

DDR:933.310 MHz (1866.620 MT/s data rate) (Asynchronous), IFC:149.985 MHz

FMAN1: 699.930 MHz

QMAN: 299.970 MHz

PME: 599.940 MHz

L1: D-cache 32 KiB enabled

I-cache 32 KiB enabled

Reset Configuration Word (RCW):

00000000: 1207001b 15000000 00000000 00000000

00000010: 66150002 00000000 e8104000 c1000000

00000020: 00800000 00000000 00000000 000307fc

00000030: 00000000 00000000 00000000 00000004

Board: T2080RDB, Board rev: 0x01 CPLD ver: 0x03, boot from SD/MMC

SERDES Reference Clocks:

SD1_CLK1=156.25MHZ, SD1_CLK2=100.00MHZ

SD2_CLK1=100.00MHZ, SD2_CLK2=100.00MHZ

I2C: ready

SPI: ready

DRAM: Detected UDIMM D3XP56082XL10AA

4 GiB (DDR3, 64-bit, CL=13, ECC on)

Flash: 128 MiB

L2: 2 MiB enabled

Corenet Platform Cache: 512 KiB enabled

Using SERDES1 Protocol: 102 (0x66)

Using SERDES2 Protocol: 21 (0x15)

SEC0: RNG instantiated

NAND: 512 MiB

MMC: FSL_SDHC: 0

*** Warning - bad CRC, using default environment

EEPROM: Invalid ID (ff ff ff ff)

PCIe1: Endpoint, undetermined, regs @ 0xfe240000

PCIe1: Bus 00 - 00

PCIe2: Root Complex, x2 gen2, regs @ 0xfe250000

02:00.0 - 1957:0808 - Processor

PCIe2: Bus 01 - 02

PCIe3: disabled

PCIe4: Root Complex, x4 gen2, regs @ 0xfe270000

04:00.0 - 10ee:7022 - Memory controller

PCIe4: Bus 03 - 04

In: serial

Out: serial

Err: serial

=> pci header 4.0.0

vendor ID = 0x10ee

device ID = 0x7022

command register = 0x0007

status register = 0x0010

revision ID = 0x00

class code = 0x05 (Memory controller)

sub class code = 0x80

programming interface = 0x00

cache line = 0x08

latency time = 0x00

header type = 0x00

BIST = 0x00

base address 0 = 0xc000000c

base address 1 = 0x00001000

base address 2 = 0xe000000c

base address 3 = 0x00000000

base address 4 = 0x00000000

base address 5 = 0x00000000

cardBus CIS pointer = 0x00000000

sub system vendor ID = 0x10ee

sub system ID = 0x0007

expansion ROM base address = 0x00000000

interrupt line = 0xff

interrupt pin = 0x00

min Grant = 0x00

max Latency = 0x00

ufedor · ‎04-28-2016

Please also provide:

1) dump of the LAW registers by executing U-Boot command "md fe000c00 80"

2) dump of the PCIe4 registers by executing U-Boot command "md fe270000 400"

manoloruiz · ‎04-28-2016

=> md fe000c00 80

fe000c00: 0000000f e8000000 81f0001b 00000000 ................

fe000c10: 0000000f f4000000 81800018 00000000 ................

fe000c20: 0000000f f6000000 83c00018 00000000 ................

fe000c30: 0000000f ffdf0000 81f0000b 00000000 ................

fe000c40: 0000000f 00000000 81d00018 00000000 ................

fe000c50: 0000000f ff800000 81f00013 00000000 ................

fe000c60: 0000000c 00000000 8000001c 00000000 ................

fe000c70: 0000000f f8000000 8000000f 00000000 ................

fe000c80: 0000000c 20000000 8010001b 00000000 .... ...........

fe000c90: 0000000f f8010000 8010000f 00000000 ................

fe000ca0: 0000000c 40000000 8030001b 00000000 ....@....0......

fe000cb0: 0000000f f8030000 8030000f 00000000 .........0......

fe000cc0: 00000000 00000000 00000000 00000000 ................

fe000cd0: 00000000 00000000 00000000 00000000 ................

fe000ce0: 00000000 00000000 00000000 00000000 ................

fe000cf0: 00000000 00000000 00000000 00000000 ................

fe000d00: 00000000 00000000 00000000 00000000 ................

fe000d10: 00000000 00000000 00000000 00000000 ................

fe000d20: 00000000 00000000 00000000 00000000 ................

fe000d30: 00000000 00000000 00000000 00000000 ................

fe000d40: 00000000 00000000 00000000 00000000 ................

fe000d50: 00000000 00000000 00000000 00000000 ................

fe000d60: 00000000 00000000 00000000 00000000 ................

fe000d70: 00000000 00000000 00000000 00000000 ................

fe000d80: 00000000 00000000 00000000 00000000 ................

fe000d90: 00000000 00000000 00000000 00000000 ................

fe000da0: 00000000 00000000 00000000 00000000 ................

fe000db0: 00000000 00000000 00000000 00000000 ................

fe000dc0: 00000000 00000000 00000000 00000000 ................

fe000dd0: 00000000 00000000 00000000 00000000 ................

fe000de0: 00000000 00000000 00000000 00000000 ................

fe000df0: 00000000 00000000 8100001f 00000000 ................

=> md fe270000 400

fe270000: 8001ff00 ffffffff 00000000 0013ffff ................

fe270010: 0400ffff 00080028 00000000 00000000 .......(........

fe270020: 00000000 00000000 000cffff 00000000 ................

fe270030: 00000000 00000000 00000000 00000000 ................

fe270040: 00000184 00000000 00000000 00000000 ................

fe270050: 00000000 00000000 00000000 00000000 ................

fe270060: 00000000 00000000 00000000 00000000 ................

fe270070: 00000000 00000000 00000000 00000000 ................

fe270080: 00000000 00000000 00000000 00000000 ................

fe270090: 00000000 00000000 00000000 00000000 ................

fe2700a0: 00000000 00000000 00000000 00000000 ................

fe2700b0: 00000000 00000000 00000000 00000000 ................

fe2700c0: 00000000 00000000 00000000 00000000 ................

fe2700d0: 00000000 00000000 00000000 00000000 ................

fe2700e0: 00000000 00000000 00000000 00000000 ................

fe2700f0: 00000000 00000000 00000000 00000000 ................

fe270100: 00000000 00000000 00000000 00000000 ................

fe270110: 00000000 00000000 00000000 00000000 ................

fe270120: 00000000 00000000 00000000 00000000 ................

fe270130: 00000000 00000000 00000000 00000000 ................

fe270140: 00000000 00000000 00000000 00000000 ................

fe270150: 00000000 00000000 00000000 00000000 ................

fe270160: 00000000 00000000 00000000 00000000 ................

fe270170: 00000000 00000000 00000000 00000000 ................

fe270180: 00000000 00000000 00000000 00000000 ................

fe270190: 00000000 00000000 00000000 00000000 ................

fe2701a0: 00000000 00000000 00000000 00000000 ................

fe2701b0: 00000000 00000000 00000000 00000000 ................

fe2701c0: 00000000 00000000 00000000 00000000 ................

fe2701d0: 00000000 00000000 00000000 00000000 ................

fe2701e0: 00000000 00000000 00000000 00000000 ................

fe2701f0: 00000000 00000000 00000000 00000000 ................

fe270200: 00000000 00000000 00000000 00000000 ................

fe270210: 00000000 00000000 00000000 00000000 ................

fe270220: 00000000 00000000 00000000 00000000 ................

fe270230: 00000000 00000000 00000000 00000000 ................

fe270240: 00000000 00000000 00000000 00000000 ................

fe270250: 00000000 00000000 00000000 00000000 ................

fe270260: 00000000 00000000 00000000 00000000 ................

fe270270: 00000000 00000000 00000000 00000000 ................

fe270280: 00000000 00000000 00000000 00000000 ................

fe270290: 00000000 00000000 00000000 00000000 ................

fe2702a0: 00000000 00000000 00000000 00000000 ................

fe2702b0: 00000000 00000000 00000000 00000000 ................

fe2702c0: 00000000 00000000 00000000 00000000 ................

fe2702d0: 00000000 00000000 00000000 00000000 ................

fe2702e0: 00000000 00000000 00000000 00000000 ................

fe2702f0: 00000000 00000000 00000000 00000000 ................

fe270300: 00000000 00000000 00000000 00000000 ................

fe270310: 00000000 00000000 00000000 00000000 ................

fe270320: 00000000 00000000 00000000 00000000 ................

fe270330: 00000000 00000000 00000000 00000000 ................

fe270340: 00000000 00000000 00000000 00000000 ................

fe270350: 00000000 00000000 00000000 00000000 ................

fe270360: 00000000 00000000 00000000 00000000 ................

fe270370: 00000000 00000000 00000000 00000000 ................

fe270380: 00000000 00000000 00000000 00000000 ................

fe270390: 00000000 00000000 00000000 00000000 ................

fe2703a0: 00000000 00000000 00000000 00000000 ................

fe2703b0: 00000000 00000000 00000000 00000000 ................

fe2703c0: 00000000 00000000 00000000 00000000 ................

fe2703d0: 00000000 00000000 00000000 00000000 ................

fe2703e0: 00000000 00000000 00000000 00000000 ................

fe2703f0: 00000000 00000000 00000000 00000000 ................

fe270400: 00000000 00000000 00000000 00000000 ................

fe270410: 00000000 00000000 00000000 00000000 ................

fe270420: 00000000 00000000 00000000 00000000 ................

fe270430: 00000000 00000000 00000000 00000000 ................

fe270440: 00000000 00000000 00000000 00000000 ................

fe270450: 00000000 00000000 00000000 00000000 ................

fe270460: 00000000 00000000 00000000 00000000 ................

fe270470: 00000000 00000000 00000000 00000000 ................

fe270480: 00000000 00000000 00000000 00000000 ................

fe270490: 00000000 00000000 00000000 00000000 ................

fe2704a0: 00000000 00000000 00000000 00000000 ................

fe2704b0: 00000000 00000000 00000000 00000000 ................

fe2704c0: 00000000 00000000 00000000 00000000 ................

fe2704d0: 00000000 00000000 00000000 00000000 ................

fe2704e0: 00000000 00000000 00000000 00000000 ................

fe2704f0: 00000000 00000000 00000000 00000000 ................

fe270500: 00000000 00000000 00000000 00000000 ................

fe270510: 00000000 00000000 00000000 00000000 ................

fe270520: 00000000 00000000 00000000 00000000 ................

fe270530: 00000000 00000000 00000000 00000000 ................

fe270540: 00000000 00000000 00000000 00000000 ................

fe270550: 00000000 00000000 00000000 00000000 ................

fe270560: 00000000 00000000 00000000 00000000 ................

fe270570: 00000000 00000000 00000000 00000000 ................

fe270580: 00000000 00000000 00000000 00000000 ................

fe270590: 00000000 00000000 00000000 00000000 ................

fe2705a0: 00000000 00000000 00000000 00000000 ................

fe2705b0: 00000000 00000000 00000000 00000000 ................

fe2705c0: 00000000 00000000 00000000 00000000 ................

fe2705d0: 00000000 00000000 00000000 00000000 ................

fe2705e0: 00000000 00000000 00000000 00000000 ................

fe2705f0: 00000000 00000000 00000000 00000000 ................

fe270600: 00000000 00000000 00000000 00000000 ................

fe270610: 00000000 00000000 00000000 00000000 ................

fe270620: 00000000 00000000 00000000 00000000 ................

fe270630: 00000000 00000000 00000000 00000000 ................

fe270640: 00000000 00000000 00000000 00000000 ................

fe270650: 00000000 00000000 00000000 00000000 ................

fe270660: 00000000 00000000 00000000 00000000 ................

fe270670: 00000000 00000000 00000000 00000000 ................

fe270680: 00000000 00000000 00000000 00000000 ................

fe270690: 00000000 00000000 00000000 00000000 ................

fe2706a0: 00000000 00000000 00000000 00000000 ................

fe2706b0: 00000000 00000000 00000000 00000000 ................

fe2706c0: 00000000 00000000 00000000 00000000 ................

fe2706d0: 00000000 00000000 00000000 00000000 ................

fe2706e0: 00000000 00000000 00000000 00000000 ................

fe2706f0: 00000000 00000000 00000000 00000000 ................

fe270700: 00000000 00000000 00000000 00000000 ................

fe270710: 00000000 00000000 00000000 00000000 ................

fe270720: 00000000 00000000 00000000 00000000 ................

fe270730: 00000000 00000000 00000000 00000000 ................

fe270740: 00000000 00000000 00000000 00000000 ................

fe270750: 00000000 00000000 00000000 00000000 ................

fe270760: 00000000 00000000 00000000 00000000 ................

fe270770: 00000000 00000000 00000000 00000000 ................

fe270780: 00000000 00000000 00000000 00000000 ................

fe270790: 00000000 00000000 00000000 00000000 ................

fe2707a0: 00000000 00000000 00000000 00000000 ................

fe2707b0: 00000000 00000000 00000000 00000000 ................

fe2707c0: 00000000 00000000 00000000 00000000 ................

fe2707d0: 00000000 00000000 00000000 00000000 ................

fe2707e0: 00000000 00000000 00000000 00000000 ................

fe2707f0: 00000000 00000000 00000000 00000000 ................

fe270800: 00000000 00000000 00000000 00000000 ................

fe270810: 00000000 00000000 00000000 00000000 ................

fe270820: 00000000 00000000 00000000 00000000 ................

fe270830: 00000000 00000000 00000000 00000000 ................

fe270840: 00000000 00000000 00000000 00000000 ................

fe270850: 00000000 00000000 00000000 00000000 ................

fe270860: 00000000 00000000 00000000 00000000 ................

fe270870: 00000000 00000000 00000000 00000000 ................

fe270880: 00000000 00000000 00000000 00000000 ................

fe270890: 00000000 00000000 00000000 00000000 ................

fe2708a0: 00000000 00000000 00000000 00000000 ................

fe2708b0: 00000000 00000000 00000000 00000000 ................

fe2708c0: 00000000 00000000 00000000 00000000 ................

fe2708d0: 00000000 00000000 00000000 00000000 ................

fe2708e0: 00000000 00000000 00000000 00000000 ................

fe2708f0: 00000000 00000000 00000000 00000000 ................

fe270900: 00000000 00000000 00000000 00000000 ................

fe270910: 00000000 00000000 00000000 00000000 ................

fe270920: 00000000 00000000 00000000 00000000 ................

fe270930: 00000000 00000000 00000000 00000000 ................

fe270940: 00000000 00000000 00000000 00000000 ................

fe270950: 00000000 00000000 00000000 00000000 ................

fe270960: 00000000 00000000 00000000 00000000 ................

fe270970: 00000000 00000000 00000000 00000000 ................

fe270980: 00000000 00000000 00000000 00000000 ................

fe270990: 00000000 00000000 00000000 00000000 ................

fe2709a0: 00000000 00000000 00000000 00000000 ................

fe2709b0: 00000000 00000000 00000000 00000000 ................

fe2709c0: 00000000 00000000 00000000 00000000 ................

fe2709d0: 00000000 00000000 00000000 00000000 ................

fe2709e0: 00000000 00000000 00000000 00000000 ................

fe2709f0: 00000000 00000000 00000000 00000000 ................

fe270a00: 00000000 00000000 00000000 00000000 ................

fe270a10: 00000000 00000000 00000000 00000000 ................

fe270a20: 00000000 00000000 00000000 00000000 ................

fe270a30: 00000000 00000000 00000000 00000000 ................

fe270a40: 00000000 00000000 00000000 00000000 ................

fe270a50: 00000000 00000000 00000000 00000000 ................

fe270a60: 00000000 00000000 00000000 00000000 ................

fe270a70: 00000000 00000000 00000000 00000000 ................

fe270a80: 00000000 00000000 00000000 00000000 ................

fe270a90: 00000000 00000000 00000000 00000000 ................

fe270aa0: 00000000 00000000 00000000 00000000 ................

fe270ab0: 00000000 00000000 00000000 00000000 ................

fe270ac0: 00000000 00000000 00000000 00000000 ................

fe270ad0: 00000000 00000000 00000000 00000000 ................

fe270ae0: 00000000 00000000 00000000 00000000 ................

fe270af0: 00000000 00000000 00000000 00000000 ................

fe270b00: 00000000 00000000 00000000 00000000 ................

fe270b10: 00000000 00000000 00000000 00000000 ................

fe270b20: 00000000 00000000 00000000 00000000 ................

fe270b30: 00000000 00000000 00000000 00000000 ................

fe270b40: 00000000 00000000 00000000 00000000 ................

fe270b50: 00000000 00000000 00000000 00000000 ................

fe270b60: 00000000 00000000 00000000 00000000 ................

fe270b70: 00000000 00000000 00000000 00000000 ................

fe270b80: 00000000 00000000 00000000 00000000 ................

fe270b90: 00000000 00000000 00000000 00000000 ................

fe270ba0: 00000000 00000000 00000000 00000000 ................

fe270bb0: 00000000 00000000 00000000 00000000 ................

fe270bc0: 00000000 00000000 00000000 00000000 ................

fe270bd0: 00000000 00000000 00000000 00000000 ................

fe270be0: 00000000 00000000 00000000 00000000 ................

fe270bf0: 00000000 00000000 02080300 00000000 ................

fe270c00: 00000000 00000000 00000000 00000000 ................

fe270c10: 80044027 00000000 00000000 00000000 ..@'............

fe270c20: 000e0000 00000000 00c40000 00000000 ................

fe270c30: 8004401b 00000000 00000000 00000000 ..@.............

fe270c40: 00000000 00000000 00ff8030 00000000 ...........0....

fe270c50: 8008800f 00000000 00000000 00000000 ................

fe270c60: 00000000 00000000 00000000 00000000 ................

fe270c70: 00000000 00000000 00000000 00000000 ................

fe270c80: 00000000 00000000 00000000 00000000 ................

fe270c90: 00044027 00000000 00000000 00000000 ..@'............

fe270ca0: 00000000 00000000 00000000 00000000 ................

fe270cb0: 00000000 00000000 00000000 00000000 ................

fe270cc0: 00000000 00000000 00000000 00000000 ................

fe270cd0: 00000000 00000000 00000000 00000000 ................

fe270ce0: 00000000 00000000 00000000 00000000 ................

fe270cf0: 20f44017 00000000 00000000 00000000 .@.............

fe270d00: 00ffe000 00000000 00000000 00000000 ................

fe270d10: 00e44017 00000000 00000000 00000000 ..@.............

fe270d20: 00000000 00000000 00000000 00000000 ................

fe270d30: 00000000 00000000 00000000 00000000 ................

fe270d40: 00000000 00000000 00000000 00000000 ................

fe270d50: 00000000 00000000 00000000 00000000 ................

fe270d60: 00000000 00000000 00000000 00000000 ................

fe270d70: 00000000 00000000 00000000 00000000 ................

fe270d80: 00000000 00000000 00000000 00000000 ................

fe270d90: 20f44027 00000000 00000000 00000000 .@'............

fe270da0: 00000000 00000000 01000000 00000000 ................

fe270db0: a0f5501f 00000000 00000000 00000000 ..P.............

fe270dc0: 00000000 00000000 00000000 00000000 ................

fe270dd0: a0f5501f 00000000 00000000 00000000 ..P.............

fe270de0: 00ffe000 00000000 00000000 00000000 ................

fe270df0: 80e44017 00000000 00000000 00000000 ..@.............

fe270e00: 80020000 00000000 00fdfeb6 00000000 ................

fe270e10: 00000000 00000000 00000000 00000000 ................

fe270e20: 80000001 00000000 00000800 00000000 ................

fe270e30: 00000000 00000000 00000000 00000000 ................

fe270e40: 00000000 00000000 00000000 00000000 ................

fe270e50: 00000000 00000000 00000000 00000000 ................

fe270e60: 00000000 00000000 00000000 00000000 ................

fe270e70: 00000000 00000000 00000000 00000000 ................

fe270e80: 00000000 00000000 00000000 00000000 ................

fe270e90: 00000000 00000000 00000000 00000000 ................

fe270ea0: 00000000 00000000 00000000 00000000 ................

fe270eb0: 00000000 00000000 00000000 00000000 ................

fe270ec0: 00000000 00000000 00000000 00000000 ................

fe270ed0: 00000000 00000000 00000000 00000000 ................

fe270ee0: 00000000 00000000 00000000 00000000 ................

fe270ef0: 00000000 00000000 00000000 00000000 ................

fe270f00: 80400084 00000000 00000000 00000000 .@..............

fe270f10: cc800000 90000044 00ff0000 00000000 .......D........

fe270f20: 00008000 00000000 00000000 00000000 ................

fe270f30: 00000000 00000000 00000000 00000000 ................

fe270f40: 00000000 00000000 00000000 00000000 ................

fe270f50: 00000000 00000000 00000000 00000000 ................

fe270f60: 00000000 00000000 00000000 00000000 ................

fe270f70: 00000000 00000000 00000000 00000000 ................

fe270f80: 00000000 00000000 00000000 00000000 ................

fe270f90: 00000000 00000000 00000000 00000000 ................

fe270fa0: 00000000 00000000 00000000 00000000 ................

fe270fb0: 00000000 00000000 00000000 00000000 ................

fe270fc0: 00000000 00000000 00000000 00000000 ................

fe270fd0: 00000000 00000000 00000000 00000000 ................

fe270fe0: 00000000 00000000 00000000 00000000 ................

fe270ff0: 00000000 00000000 00000000 00000000 ................

thanks for your help. Do you see anything wrong?

ufedor · ‎04-29-2016

According to the provided dumps the U-Boot was able to configure BAR2 of the FPGA:

> base address 2 = 0xe000000c

in accordance with LAW:

> fe000ca0: 0000000c 40000000 8030001b 00000000

and Outbound ATMU:

> fe270c20: 000e0000 00000000 00c40000 00000000

> fe270c30: 8004401b 00000000 00000000 00000000

I.e physical address of the FPGA memory is 0xC_4000_0000.

ufedor · ‎04-26-2016

The issue definitely is not connected with the T2080 hardware.

Please check wheter the FPGA's BAR0 is a 32-bit memory space BAR and it is needed to use a 64-bit BAR.

Which sortware is in question?

Have you tried to enumerate the FPGA using U-Boot from the Freescale/NXP SDK 1.9?

manoloruiz · ‎04-27-2016

Thank very much for your answer.

-I haven not change anything in Yocto project source, neather dtb nor u-boot from SDK 1.8.

-The FPGA has a 64-bit memory space BAR. I have tried the same FPGA with 32-bit memory space BAR and work properly. It does not work when the FPGA has 64-bit memory space BAR.

I do not know where the problem is. Do I have to change the U-Boot? Do I have to make any change in the device Tree? If I have to, how can I do it?

ufedor · ‎04-27-2016

1) Have you tried to enumerate the FPGA using U-Boot?

What is the result?

2) Please provide the FPGA BARs description.

manoloruiz · ‎04-27-2016

I have not tried to enumerate the FPGA using U-boot. How can I do that?

I got from linux the following:

lspci

0001:00:00.0 PCI bridge: Freescale Semiconductor Inc Device 0830 (rev 11)

0001:01:00.0 Power PC: Freescale Semiconductor Inc Device 0808 (rev 10)

0002:00:00.0 PCI bridge: Freescale Semiconductor Inc Device 0830 (rev 11)

0002:01:00.0 Memory controller: Xilinx Corporation Device 7022

lspci -v

0002:01:00.0 Memory controller: Xilinx Corporation Device 7022

Subsystem: Xilinx Corporation Device 0007

Flags: bus master, fast devsel, latency 0

Memory at <ignored> (64-bit, prefetchable)

Memory at c40000000 (64-bit, prefetchable) [size=64K]

Capabilities: [40] Power Management version 3

Capabilities: [48] MSI: Enable- Count=1/8 Maskable- 64bit+

Capabilities: [60] Express Endpoint, MSI 00

Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00

Kernel driver in use: pci

lspci -x

0002:01:00.0 Memory controller: Xilinx Corporation Device 7022

00: ee 10 22 70 07 00 10 00 00 00 80 05 08 00 00 00

10: 0c 00 00 c0 00 10 00 00 0c 00 00 e0 00 00 00 00

20: 00 00 00 00 00 00 00 00 00 00 00 00 ee 10 07 00

30: 00 00 00 00 40 00 00 00 00 00 00 00 ff 00 00 00

from U-boot

PCIe1: Endpoint, no link, regs @ 0xfe240000

PCIe1: Bus 00 - 00

PCIe2: Root Complex, x2 gen2, regs @ 0xfe250000

02:00.0 - 1957:0808 - Processor

PCIe2: Bus 01 - 02

PCIe3: disabled

PCIe4: Root Complex, x4 gen2, regs @ 0xfe270000

04:00.0 - 10ee:7022 - Memory controller

/pcie@ffe270000: PCICSRBAR @ 0xff000007

/pcie@ffe270000: Setup 64-bit PCI DMA window

/pcie@ffe270000: WARNING: Outbound window cfg leaves gaps in memory map. Adjusting the memory map could reduce unnecessary bounce buffering.

/pcie@ffe270000: DMA window size is 0xe0000000

EDAC PCI2: Giving out device to module 'MPC85xx_edac' controller 'mpc85xx_pci_err': DEV 'ffe270000.pcie' (INTERRUPT)

MPC85xx_edac acquired irq 23 for PCI Err

MPC85xx_edac PCI err registered

software IO TLB [mem 0x083ec000-0x0c3ec000] (64MB) mapped at [c0000000083ec000-c00000000c3ebfff]

PCI: Probing PCI hardware

fsl-pci ffe250000.pcie: PCI host bridge to bus 0001:00

pci_bus 0001:00: root bus resource [io 0x10000-0x1ffff] (bus address [0x0000-0xffff])

pci_bus 0001:00: root bus resource [mem 0xc20000000-0xc2fffffff] (bus address [0xe0000000-0xefffffff])

pci_bus 0001:00: root bus resource [bus 00-01]

pci 0001:00:00.0: ignoring class 0x0b2000 (doesn't match header type 01)

pci 0001:00:00.0: PCI bridge to [bus 01-ff]

fsl-pci ffe270000.pcie: PCI host bridge to bus 0002:00

pci_bus 0002:00: root bus resource [io 0x21000-0x30fff] (bus address [0x0000-0xffff])

pci_bus 0002:00: root bus resource [mem 0xc40000000-0xc4fffffff] (bus address [0xe0000000-0xefffffff])

pci_bus 0002:00: root bus resource [bus 00-01]

pci 0002:00:00.0: ignoring class 0x0b2000 (doesn't match header type 01)

pci 0002:00:00.0: PCI bridge to [bus 01-ff]

PCI: Cannot allocate resource region 0 of device 0002:01:00.0, will remap

pci 0001:00:00.0: PCI bridge to [bus 01]

pci 0001:00:00.0: bridge window [io 0x10000-0x1ffff]

pci 0001:00:00.0: bridge window [mem 0xc20000000-0xc2fffffff]

pci 0002:00:00.0: BAR 9: can't assign mem pref (size 0x40000000)

pci 0002:01:00.0: BAR 0: can't assign mem pref (size 0x40000000)

pci 0002:00:00.0: PCI bridge to [bus 01]

pci 0002:00:00.0: bridge window [io 0x21000-0x30fff]

pci 0002:00:00.0: bridge window [mem 0xc40000000-0xc4fffffff]

pci_bus 0002:00: Some PCI device resources are unassigned, try booting with pci=realloc

Do you need more information? How can I get the information that you need?thanks very much

ufedor · ‎04-27-2016

1) Please provide output of the U-Boot command "pci header 4.0.0".

2) Is there a Reference Manual or User Guide for the FPGA PCIe implementation?

manoloruiz · ‎04-27-2016

1) Please provide output of the U-Boot command "pci header 4.0.0".

There is not device in 4.0.0 however vendor and device ID show up in 3.0.0. You can see:

=> pci header 3.0.0

vendor ID = 0x10ee

device ID = 0x7022

command register = 0x0007

status register = 0x0010

revision ID = 0x00

class code = 0x05 (Memory controller)

sub class code = 0x80

programming interface = 0x00

cache line = 0x08

latency time = 0x00

header type = 0x00

BIST = 0x00

base address 0 = 0xc000000c

base address 1 = 0x00001000

base address 2 = 0xe000000c

base address 3 = 0x00000000

base address 4 = 0x00000000

base address 5 = 0x00000000

cardBus CIS pointer = 0x00000000

sub system vendor ID = 0x10ee

sub system ID = 0x0007

expansion ROM base address = 0x00000000

interrupt line = 0xff

interrupt pin = 0x00

min Grant = 0x00

max Latency = 0x00

2) Is there a Reference Manual or User Guide for the FPGA PCIe implementation?

The PCI device FW is correct. I have tried in another system and works. I think the problem is the U-Boot or the device tree in my T2080RDB. The U-boot look like it does not realize that it a pci device working with 64 bits memories. You can see. There are 6 BAR. In a 64bit PCI device should be only 3 BAR.

Any solution?

ufedor · ‎04-27-2016

Please try the SDK 1.9 to eliminate a software bug possibility.

manoloruiz · ‎04-28-2016

Hi udefor,

I have tried the new version SDK 1.9. However, the problem persists.Any other solution? should I change anything in uboot, device tree or CCSR?

ufedor · ‎04-28-2016

Please provide SDK 1.9 U-Boot log and "pci header" command output.

manoloruiz · ‎04-28-2016

=> pci header 3.0.0

vendor ID = 0x10ee

device ID = 0x7022

command register = 0x0007

status register = 0x0010

revision ID = 0x00

class code = 0x05 (Memory controller)

sub class code = 0x80

programming interface = 0x00

cache line = 0x08

latency time = 0x00

header type = 0x00

BIST = 0x00

base address 0 = 0xc000000c

base address 1 = 0x00001000

base address 2 = 0xe000000c

base address 3 = 0x00000000

base address 4 = 0x00000000

base address 5 = 0x00000000

cardBus CIS pointer = 0x00000000

sub system vendor ID = 0x10ee

sub system ID = 0x0007

expansion ROM base address = 0x00000000

interrupt line = 0xff

interrupt pin = 0x00

min Grant = 0x00

max Latency = 0x00

ufedor · ‎04-28-2016

Please provide SDK 1.9 U-Boot log

PCI device (64bit memory problem)

PCI device (64bit memory problem)

QorIQ T2 Devices