T2080 DMA transfer to xilinix FPGA driver

[sadashiva]

Hi,

We are using T2080 processor custom board which is connected through PCIe to Xilinx FPGA.

We need to validate the DMA transfer between T2080 and FPGA.

At u-boot, 

the memory and io is mapped and able to access

BAR0 is 0x81000000 & BAR1 is 0x82000000

After board boots, BAR0 and BAR 1 address space is mapped correctly and able to access in kernel (at user space) using pcimem utility.

We are in the process of validating the DMA transfer between processor and FPGA.

Not able to find exact drivers/control registers need to be enabled at processor (t2080) side.

Also not able to find the xilinx driver files xdma.c  in (SDK2.0, kernel version 4.1).

Can you please help us on the procedure and drivers has to be enabled to validate the DMA transfer between processor and FPGA?

Best Regards,

Sadashiva

[ufedor]

You wrote:

> After board boots, BAR0 and BAR 1 address space is mapped correctly

> and able to access in kernel (at user space) using pcimem utility.

Please provide PCI header of the initialized FPGA.

Which physical addresses were used to access the BARs from the T2080 local address space?

[sadashiva]

Hi Ufedor,

>Please provide PCI header of the initialized FPGA?

pci head 1.0.0
  vendor ID =                   0x10ee
  device ID =                   0x7024
  command register ID =         0x0006
  status register =             0x0010
  revision ID =                 0x00
  class code =                  0x05 (Memory controller)
  sub class code =              0x80
  programming interface =       0x00
  cache line =                  0x08
  latency time =                0x00
  header type =                 0x00
  BIST =                        0x00
  base address 0 =              0x81000000
  base address 1 =              0x82000000
  base address 2 =              0x00000000
  base address 3 =              0x00000000
  base address 4 =              0x00000000
  base address 5 =              0x00000000
  cardBus CIS pointer =         0x00000000
  sub system vendor ID =        0x10ee
  sub system ID =               0x0007
  expansion ROM base address =  0x00000000
  interrupt line =              0xff
  interrupt pin =               0x01
  min Grant =                   0x00
  max Latency =                 0x00

>Which physical addresses were used to access the BARs from the T2080 local address space?

My t2080 dts entry for the address map is

  pci3: pcie@ffe270000 {
        reg = <0xf 0xfe270000 0 0x10000>;
        ranges = <0x02000000 0 0x80000000 0xc 0x00000000 0 0x04000000
              0x01000000 0 0x00000000 0xf 0xf9000000 0 0x00100000>;
        pcie@0 {
            ranges = <0x02000000 0 0x80000000
                  0x02000000 0 0x80000000
                  0 0x10000000

                  0x01000000 0 0x00000000
                  0x01000000 0 0x00000000
                  0 0x00100000>;
        };
    };  

The kernel is mapped to the following address,

  MEM 0x0000000c00000000..0x0000000c03ffffff -> 0x0000000080000000
  IO 0x0000000ff9000000..0x0000000ff90fffff -> 0x0000000000000000

The pcimem utility output at kernel level is as follows,

root@t2080rdb:~#   ./powerpc_pci /sys/bus/pci/devices/0000\:01\:00.0/resource0 0
/sys/bus/pci/devices/0000:01:00.0/resource0 opened.
Target offset is 0x0, page size is 4096
mmap(0, 4096, 0x3, 0x1, 3, 0x0)
PCI Memory mapped to address 0xf7bd6000.
Value at offset 0x0 (0xf7bd6000): 0xFFFF0000


root@t2080rdb:~#   ./powerpc_pci /sys/bus/pci/devices/0000\:01\:00.0/resource1 0
/sys/bus/pci/devices/0000:01:00.0/resource1 opened.
Target offset is 0x0, page size is 4096
mmap(0, 4096, 0x3, 0x1, 3, 0x0)
PCI Memory mapped to address 0xf7a28000.
Value at offset 0x0 (0xf7a28000): 0x2000100

Best Regards,

Sadashiva

[ufedor]

Please provide U-Boot dump of the PCIe Outbound Windows CCSR area 0xfe270C00 - 0xfe270C9F

[sadashiva]

Hi Ufedor

=> md.b 0xfe270C00 0x9f
fe270c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
fe270c10: 80 04 40 27 00 00 00 00 00 00 00 00 00 00 00 00    ..@'............
fe270c20: 00 08 00 00 00 00 00 00 00 c0 00 00 00 00 00 00    ................
fe270c30: 80 04 40 1b 00 00 00 00 00 00 00 00 00 00 00 00    ..@.............
fe270c40: 00 00 00 00 00 00 00 00 00 ff 80 00 00 00 00 00    ................
fe270c50: 80 08 80 0f 00 00 00 00 00 00 00 00 00 00 00 00    ................
fe270c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
fe270c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
fe270c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................

fe270c90: 00 04 40 27 00 00 00 00 00 00 00 00 00 00 00       ..@'...........

[ufedor]

The Outbound Window in question is:

fe270c20: 00080000 00000000 00c00000 00000000
fe270c30: 8004401b 00000000 00000000 00000000

i.e. translation from T2080 local physical address to the PCIe address is:

0xC_0000_0000 -> 0x8000_0000

Considering that DMA operates with physical addresses, to write data from the beginning of BAR0 (for example) it is required to set DMA destination address 0xC_0100_0000.

[sadashiva]

Hi Ufedor,

For Host DMA transfer, as per the above memory bar assignment

DAR – 0x0010011c – set t0 0x0100 0000

DATR – 0x00100118 – set to 0x0000 000c

But the dma transfer is not success with status register is throwing the error, 0x10, i.e PE bit is set which is a programming error.

When i read the address of BAR0, no data is copied here from my source ddr address which is set to 0x1000 0000 , filled with data 0x11.

Can you please help us on the destination address to confiure on DAR and DATR correctly?

Thanks

Sadashiva

[ufedor]

> DAR – 0x0010011c

> DATR – 0x00100118

These addresses are incorrect - they are just CCSR offsets.

Must be:

DAR – 0xFE10011c

DATR – 0xFE100118

Also DMAx_DATRn[DWRITETTYPE] must not be zero - refer to the RM, 23.3.7 DMA destination attributes register (DMAx_DATRn)

[sadashiva]

Hi Ufedor

Thank you for the inputs.

We enabled the bits in Attribute registers

DMA1_SATR0 & DMA1_DATR0 for the field

SREADTTYPE & DWRITETTYPE

The memory dump for CCSR is as follows:

CCSRBAR=0xffe000000 mapped at: 0xf6aa1000, offset = 0x100100
ffe100100: 080001c5 00000000 00000000 00000000
ffe100110: 00040000 10100000 0004000c 01100000
ffe100120: 00000000 00000000 00000000 00000000
ffe100130: 00000000 00000000 00000000 00000000
ffe100140: 00000000 00000000 00000000 00000000
ffe100150: 00000000 00000000 00000000 00000000
ffe100160: 00000000 00000000 00000000 00000000

Now the DMA transfer is successful!

Thank you for the support!

Thanks,

Sadashiva Reddy

[sadashiva]

Hi Ufedor

Thank you for the response.

For t2080rdb, DMA access,

We are trying to read the DMA mode register, 0x100100

i. The physical address 0xfe100100 able to read value 0x08000000 in u-boot.

ii. Once the kernel is booted,
     Trying to read the DMA mode register using the physical address -> converted to virtual address using mmap
     when we try to do the kernel is crashing

As we searched in the forums, bus address has to be used to read the DMA mode register.
Please help us how to convert the physical address to bus address to access the DMA mode register in t2080rdb.

How to access the CCSBAR memory in kernel?

Thanks
Sadashiva

[sadashiva]

Hi Ufedor,

For Host DMA transfer, as per the above memory bar assignment

DAR – 0x0010011c – set t0 0x0100 0000

DATR – 0x00100118 – set to 0x0000 000c

But the dma transfer is not success with status register is throwing the error, 0x10, i.e PE bit is set which is a programming error.

When i read the address of BAR0, no data is copied here from my source ddr address which is set to 0x1000 0000 , filled with data 0x11.

Can you please help us on the destination address to confiure on DAR and DATR correctly?

Thanks

Sadashiva

[ufedor]

Please consider that DMA performs memory-to-memory transfer operations.

Source and destination memory areas must be properly allocated in the processor's local address space.

Linux DMA driver is described in the QorIQ SDK V2.0-1703 Documentation, 7.3.1 Direct Memory Access Driver (PowerQUICC, QorIQ) available at:

https://www.nxp.com/design/software/embedded-software/linux-software-and-development-tools/linux-sdk... 

[sadashiva]

Hi Ufedor

Thank you for the information.

I have enabled fsl dma driver as per the linux dma driver link specified,

The kernel log is

[ 1.565687] fsl-elo-dma ffe100300.dma: #0 (fsl,eloplus-dma-channel), irq 28

[ 1.572629] fsl-elo-dma ffe100300.dma: #1 (fsl,eloplus-dma-channel), irq 29

[ 1.579596] fsl-elo-dma ffe100300.dma: #2 (fsl,eloplus-dma-channel), irq 30

[ 1.586562] fsl-elo-dma ffe100300.dma: #3 (fsl,eloplus-dma-channel), irq 31

[ 1.593530] fsl-elo-dma ffe100300.dma: #4 (fsl,eloplus-dma-channel), irq 76

[ 1.600496] fsl-elo-dma ffe100300.dma: #5 (fsl,eloplus-dma-channel), irq 77

[ 1.607464] fsl-elo-dma ffe100300.dma: #6 (fsl,eloplus-dma-channel), irq 78

[ 1.614432] fsl-elo-dma ffe100300.dma: #7 (fsl,eloplus-dma-channel), irq 79

[ 1.621923] fsl-elo-dma ffe101300.dma: #0 (fsl,eloplus-dma-channel), irq 32

[ 1.628863] fsl-elo-dma ffe101300.dma: #1 (fsl,eloplus-dma-channel), irq 33

[ 1.635831] fsl-elo-dma ffe101300.dma: #2 (fsl,eloplus-dma-channel), irq 34

[ 1.642798] fsl-elo-dma ffe101300.dma: #3 (fsl,eloplus-dma-channel), irq 35

[ 1.649766] fsl-elo-dma ffe101300.dma: #4 (fsl,eloplus-dma-channel), irq 80

[ 1.656733] fsl-elo-dma ffe101300.dma: #5 (fsl,eloplus-dma-channel), irq 81

[ 1.663701] fsl-elo-dma ffe101300.dma: #6 (fsl,eloplus-dma-channel), irq 82

[ 1.670667] fsl-elo-dma ffe101300.dma: #7 (fsl,eloplus-dma-channel), irq 83

[ 1.678115] fsl-elo-dma ffe102300.dma: #0 (fsl,eloplus-dma-channel), irq 464

[ 1.685146] fsl-elo-dma ffe102300.dma: #1 (fsl,eloplus-dma-channel), irq 465

[ 1.692197] fsl-elo-dma ffe102300.dma: #2 (fsl,eloplus-dma-channel), irq 466

[ 1.699252] fsl-elo-dma ffe102300.dma: #3 (fsl,eloplus-dma-channel), irq 467

[ 1.706310] fsl-elo-dma ffe102300.dma: #4 (fsl,eloplus-dma-channel), irq 473

[ 1.713362] fsl-elo-dma ffe102300.dma: #5 (fsl,eloplus-dma-channel), irq 474

[ 1.720416] fsl-elo-dma ffe102300.dma: #6 (fsl,eloplus-dma-channel), irq 475

[ 1.727471] fsl-elo-dma ffe102300.dma: #7 (fsl,eloplus-dma-channel), irq 476

But to do the DMA transfer from the t2080rdb to the fpga,

What is the procedure and steps to follow at t2080 processor side?

Is there any sample source is available?

The address BAR0 and BAR1 is mapped as per the request from the FPGA and able to write and read back the data from the registers.

[ufedor]

> able to write and read back the data from the registers.

What do you mean?

How these read/write operations correspond to the DMA transfers?

[sadashiva]

Hi Ufedor

> What do you mean?

Able to talk to the FPGA over PCIe correctly

> How these read/write operations correspond to the DMA transfers?

It is not regarding DMA, it is about correct mapping of address bars.

Can you just direct us on the DMA sample code for t2080rdb to the FPGA?

Thank you.

Best Regards,

Sadashiva