Accessing on PCIe memory

[utkarsh_100]

Hi All 

I want to communicate T1042 processor with AXI fpga through pcie. I can able to access AXI fpga configuration details that are listed below

-> vendor ID =                   0x10ee

device ID =                   0x7022

command register =            0x0007

status register =             0x0010

revision ID =                 0x00

class code =                  0x05

sub class code =              0x80

programming interface =       0x00

cache line =                  0x10

latency time =                0x00

header type =                 0x00

BIST =                        0x00

base address 0 =              0xa0000000

base address 1 =              0x00000000

base address 2 =              0x00000000

base address 3 =              0x00000000

base address 4 =              0x00000000

base address 5 =              0x00000000

cardBus CIS pointer =         0x00000000

sub system vendor ID =        0x10ee

sub system ID =               0x0007

expansion ROM base address =  0x00000000

interrupt line =              0x29

interrupt pin =               0x01

min Grant =                   0x00

max Latency =                 0x00

Capabilities - Power Management

Capabilities - Message Signaled Interrupts: 0x48 control 0x80

But when I am writing onto the address 0xa000_0000 on the fpga side ,i am not able to access the data. 

Code is below

#define PCI_BAR0 0xA0000000

fpga_Write(0x0000,0x2222);

int fpga_Write(unsigned int Addr,unsigned long data){
*(unsigned int *)(PCI_BAR0+Addr) = data;
return 0;
}

Device tree configuration is below 

pci1: pcie@ffe250000 {
compatible = "fsl,qoriq-pcie";
device_type = "pci";
#size-cells = <2>;
#address-cells = <3>;
bus-range = <0 0xff>;
interrupts = <21 2 0 0>;
#interrupt-cells = <1>;
reg = <0xf 0xfe250000 0 0x10000>;
/*
* EEEEEEEEEE address properties encoded
* p PCI address upper 32 bits
* PPPPPPPPPP PCI address lower 32 bits
* ccc CPU address upper 32 bits
* CCCCCCCCCC CPU address lower 32 bits
* s range size upper 32 bits
* SSSSSSSSSS range size lower 32 bits
*
* EEEEEEEEEE p PPPPPPPPPP ccc CCCCCCCCCC s SSSSSSSSSS
*/
ranges = <0x02000000 0 0xa0000000 0xf 0xa0000000 0 0x10000000
0x01000000 0 0x00000000 0xf 0xffc10000 0 0x00010000>;
interrupt-map-mask = <0xfff800 0 0 7>;
interrupt-map = <
0000 0 0 1 &mpic 41 1 0 0
0000 0 0 2 &mpic 5 1 0 0
0000 0 0 3 &mpic 6 1 0 0
0000 0 0 4 &mpic 7 1 0 0
>;
};

I have few question that are highlighted below 

1. Do I need to send data though TLP layer. How should I do that

2. Checked PCI Express memory-mapped registers .Under that enable these registers 

         2.1 PCI Express configuration address register

         2.2 PCI Express configuration data register

but not sure on which address need to write on ,since  I am using pci2 controller 2 ,i am writing on 0x25_0000 or do I have to write on 0xffe250000 (highlighted in DTS configuration)

Thanks 

[yipingwang]

Hello utkarsh rawat,

1. Please check whether you could read the FPGA information through BAR address under u-boot, please refer to the following.

=> pci 2
Scanning PCI devices on bus 2
BusDevFun  VendorId   DeviceId   Device Class       Sub-Class
_____________________________________________________________
02.00.00   0x8086     0x107d     Network controller      0x00
=> pci head 02.00.00
  vendor ID =                   0x8086
  device ID =                   0x107d
  command register ID =         0x0006
  status register =             0x0010
  revision ID =                 0x06
  class code =                  0x02 (Network controller)
  sub class code =              0x00
  programming interface =       0x00
  cache line =                  0x08
  latency time =                0x00
  header type =                 0x00
  BIST =                        0x00
  base address 0 =              0xe0000000
  base address 1 =              0xe0020000
  base address 2 =              0x00001001
  base address 3 =              0x00000000
  base address 4 =              0x00000000
  base address 5 =              0x00000000
  cardBus CIS pointer =         0x00000000
  sub system vendor ID =        0x8086
  sub system ID =               0x1092
  expansion ROM base address =  0xe0040000
  interrupt line =              0x00
  interrupt pin =               0x01
  min Grant =                   0x00
  max Latency =                 0x00
=> pci display 02.00.00 0xe0000000
e0000000: 107d8086 00100006 02000006 00000008
e0000010: e0000000 e0020000 00001001 00000000
e0000020: 00000000 00000000 00000000 10928086
e0000030: e0040000 000000c8 00000000 00000100

2.  For PCIe device driver in u-boot to use "pci_map_bar" to map the BAR address to the local virtual address space, please refer to drivers/net/e1000.c or drivers/block/sata_sil.c.

In Linux, please refer to Documentation/PCI/pci.txt, please refer to the example code in drivers/net/ethernet/intel/e1000/e1000_main.c.

Memory (MMIO), and I/O port addresses should NOT be read directly from the PCI device config space. Use the values in the pci_dev structure as the PCI "bus address" might have been remapped to a "host physical" address by the arch/chip-set specific kernel support. See Documentation/io-mapping.txt for how to access device registers
or device memory.

For example in a PCIe device, bar 0 is used for Port IO, and bar 1 is used for the MMIO. So here we should read the physical base address from bar 1 and remap the MMIO region as the following.

mmio_start = pci_resource_start(dev, 1);

ioaddr = ioremap(mmio_start, mmio_len);

3. Regarding dts definition, the "ranges" property definition translates from a PCI address space to a CPU address space. Memory space translates from PCI memory space 0xe0000000 to processor memory space(physical) 0xc10000000. IO memory space from PCI address 0 mapped to CPU physical address 0xff8010000.

        pci1: pcie@ffe250000 {
                reg = <0xf 0xfe250000 0 0x10000>;
                ranges = <0x02000000 0x0 0xe0000000 0xc 0x10000000 0x0 0x10000000
                                 0x01000000 0x0 0x00000000 0xf 0xf8010000 0x0 0x00010000>;
                pcie@0 {
                        ranges = <0x02000000 0 0xe0000000
                                  0x02000000 0 0xe0000000
                                  0 0x10000000

                                  0x01000000 0 0x00000000
                                  0x01000000 0 0x00000000
                                  0 0x00010000>;
                };
        };

This often needs to be coincident with u-boot definition in include/configs/T104xRDB.h.

/* controller 2, Slot 2, tgtid 2, Base address 201000 */
#ifdef CONFIG_PCIE2
#define CONFIG_SYS_PCIE2_MEM_VIRT       0x90000000
#define CONFIG_SYS_PCIE2_MEM_BUS        0xe0000000
#define CONFIG_SYS_PCIE2_MEM_PHYS       0xc10000000ull
#define CONFIG_SYS_PCIE2_MEM_SIZE       0x10000000      /* 256M */
#define CONFIG_SYS_PCIE2_IO_VIRT        0xf8010000
#define CONFIG_SYS_PCIE2_IO_BUS         0x00000000
#define CONFIG_SYS_PCIE2_IO_PHYS        0xff8010000ull
#define CONFIG_SYS_PCIE2_IO_SIZE        0x00010000      /* 64k */
#endif

Have a great day,
TIC

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------