Hi,
We are using T2080 processor custom board which is connected through PCIe to Xilinx FPGA.
We need to validate the DMA transfer between T2080 and FPGA.
At u-boot,
the memory and io is mapped and able to access
BAR0 is 0x81000000 & BAR1 is 0x82000000
After board boots, BAR0 and BAR 1 address space is mapped correctly and able to access in kernel (at user space) using pcimem utility.
We are in the process of validating the DMA transfer between processor and FPGA.
Not able to find exact drivers/control registers need to be enabled at processor (t2080) side.
Also not able to find the xilinx driver files xdma.c in (SDK2.0, kernel version 4.1).
Can you please help us on the procedure and drivers has to be enabled to validate the DMA transfer between processor and FPGA?
Best Regards,
Sadashiva
You wrote:
> After board boots, BAR0 and BAR 1 address space is mapped correctly
> and able to access in kernel (at user space) using pcimem utility.
Please provide PCI header of the initialized FPGA.
Which physical addresses were used to access the BARs from the T2080 local address space?
Hi Ufedor,
>Please provide PCI header of the initialized FPGA?
pci head 1.0.0
vendor ID = 0x10ee
device ID = 0x7024
command register ID = 0x0006
status register = 0x0010
revision ID = 0x00
class code = 0x05 (Memory controller)
sub class code = 0x80
programming interface = 0x00
cache line = 0x08
latency time = 0x00
header type = 0x00
BIST = 0x00
base address 0 = 0x81000000
base address 1 = 0x82000000
base address 2 = 0x00000000
base address 3 = 0x00000000
base address 4 = 0x00000000
base address 5 = 0x00000000
cardBus CIS pointer = 0x00000000
sub system vendor ID = 0x10ee
sub system ID = 0x0007
expansion ROM base address = 0x00000000
interrupt line = 0xff
interrupt pin = 0x01
min Grant = 0x00
max Latency = 0x00
>Which physical addresses were used to access the BARs from the T2080 local address space?
My t2080 dts entry for the address map is
pci3: pcie@ffe270000 {
reg = <0xf 0xfe270000 0 0x10000>;
ranges = <0x02000000 0 0x80000000 0xc 0x00000000 0 0x04000000
0x01000000 0 0x00000000 0xf 0xf9000000 0 0x00100000>;
pcie@0 {
ranges = <0x02000000 0 0x80000000
0x02000000 0 0x80000000
0 0x10000000
0x01000000 0 0x00000000
0x01000000 0 0x00000000
0 0x00100000>;
};
};
The kernel is mapped to the following address,
MEM 0x0000000c00000000..0x0000000c03ffffff -> 0x0000000080000000
IO 0x0000000ff9000000..0x0000000ff90fffff -> 0x0000000000000000
The pcimem utility output at kernel level is as follows,
root@t2080rdb:~# ./powerpc_pci /sys/bus/pci/devices/0000\:01\:00.0/resource0 0
/sys/bus/pci/devices/0000:01:00.0/resource0 opened.
Target offset is 0x0, page size is 4096
mmap(0, 4096, 0x3, 0x1, 3, 0x0)
PCI Memory mapped to address 0xf7bd6000.
Value at offset 0x0 (0xf7bd6000): 0xFFFF0000
root@t2080rdb:~# ./powerpc_pci /sys/bus/pci/devices/0000\:01\:00.0/resource1 0
/sys/bus/pci/devices/0000:01:00.0/resource1 opened.
Target offset is 0x0, page size is 4096
mmap(0, 4096, 0x3, 0x1, 3, 0x0)
PCI Memory mapped to address 0xf7a28000.
Value at offset 0x0 (0xf7a28000): 0x2000100
Best Regards,
Sadashiva
Please provide U-Boot dump of the PCIe Outbound Windows CCSR area 0xfe270C00 - 0xfe270C9F
Hi Ufedor
=> md.b 0xfe270C00 0x9f
fe270c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
fe270c10: 80 04 40 27 00 00 00 00 00 00 00 00 00 00 00 00 ..@'............
fe270c20: 00 08 00 00 00 00 00 00 00 c0 00 00 00 00 00 00 ................
fe270c30: 80 04 40 1b 00 00 00 00 00 00 00 00 00 00 00 00 ..@.............
fe270c40: 00 00 00 00 00 00 00 00 00 ff 80 00 00 00 00 00 ................
fe270c50: 80 08 80 0f 00 00 00 00 00 00 00 00 00 00 00 00 ................
fe270c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
fe270c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
fe270c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
fe270c90: 00 04 40 27 00 00 00 00 00 00 00 00 00 00 00 ..@'...........
The Outbound Window in question is:
fe270c20: 00080000 00000000 00c00000 00000000
fe270c30: 8004401b 00000000 00000000 00000000
i.e. translation from T2080 local physical address to the PCIe address is:
0xC_0000_0000 -> 0x8000_0000
Considering that DMA operates with physical addresses, to write data from the beginning of BAR0 (for example) it is required to set DMA destination address 0xC_0100_0000.
Hi Ufedor,
For Host DMA transfer, as per the above memory bar assignment
DAR – 0x0010011c – set t0 0x0100 0000
DATR – 0x00100118 – set to 0x0000 000c
But the dma transfer is not success with status register is throwing the error, 0x10, i.e PE bit is set which is a programming error.
When i read the address of BAR0, no data is copied here from my source ddr address which is set to 0x1000 0000 , filled with data 0x11.
Can you please help us on the destination address to confiure on DAR and DATR correctly?
Thanks
Sadashiva
> DAR – 0x0010011c
> DATR – 0x00100118
These addresses are incorrect - they are just CCSR offsets.
Must be:
DAR – 0xFE10011c
DATR – 0xFE100118
Also DMAx_DATRn[DWRITETTYPE] must not be zero - refer to the RM, 23.3.7 DMA destination attributes register (DMAx_DATRn)
Hi Ufedor
Thank you for the inputs.
We enabled the bits in Attribute registers
DMA1_SATR0 & DMA1_DATR0 for the field
SREADTTYPE & DWRITETTYPE
The memory dump for CCSR is as follows:
CCSRBAR=0xffe000000 mapped at: 0xf6aa1000, offset = 0x100100
ffe100100: 080001c5 00000000 00000000 00000000
ffe100110: 00040000 10100000 0004000c 01100000
ffe100120: 00000000 00000000 00000000 00000000
ffe100130: 00000000 00000000 00000000 00000000
ffe100140: 00000000 00000000 00000000 00000000
ffe100150: 00000000 00000000 00000000 00000000
ffe100160: 00000000 00000000 00000000 00000000
Now the DMA transfer is successful!
Thank you for the support!
Thanks,
Sadashiva Reddy
Hi Ufedor
Thank you for the response.
For t2080rdb, DMA access,
We are trying to read the DMA mode register, 0x100100
i. The physical address 0xfe100100 able to read value 0x08000000 in u-boot.
ii. Once the kernel is booted,
Trying to read the DMA mode register using the physical address -> converted to virtual address using mmap
when we try to do the kernel is crashing
As we searched in the forums, bus address has to be used to read the DMA mode register.
Please help us how to convert the physical address to bus address to access the DMA mode register in t2080rdb.
How to access the CCSBAR memory in kernel?
Thanks
Sadashiva
Hi Ufedor,
For Host DMA transfer, as per the above memory bar assignment
DAR – 0x0010011c – set t0 0x0100 0000
DATR – 0x00100118 – set to 0x0000 000c
But the dma transfer is not success with status register is throwing the error, 0x10, i.e PE bit is set which is a programming error.
When i read the address of BAR0, no data is copied here from my source ddr address which is set to 0x1000 0000 , filled with data 0x11.
Can you please help us on the destination address to confiure on DAR and DATR correctly?
Thanks
Sadashiva
Please consider that DMA performs memory-to-memory transfer operations.
Source and destination memory areas must be properly allocated in the processor's local address space.
Linux DMA driver is described in the QorIQ SDK V2.0-1703 Documentation, 7.3.1 Direct Memory Access Driver (PowerQUICC, QorIQ) available at:
Hi Ufedor
Thank you for the information.
I have enabled fsl dma driver as per the linux dma driver link specified,
The kernel log is
[ 1.565687] fsl-elo-dma ffe100300.dma: #0 (fsl,eloplus-dma-channel), irq 28
[ 1.572629] fsl-elo-dma ffe100300.dma: #1 (fsl,eloplus-dma-channel), irq 29
[ 1.579596] fsl-elo-dma ffe100300.dma: #2 (fsl,eloplus-dma-channel), irq 30
[ 1.586562] fsl-elo-dma ffe100300.dma: #3 (fsl,eloplus-dma-channel), irq 31
[ 1.593530] fsl-elo-dma ffe100300.dma: #4 (fsl,eloplus-dma-channel), irq 76
[ 1.600496] fsl-elo-dma ffe100300.dma: #5 (fsl,eloplus-dma-channel), irq 77
[ 1.607464] fsl-elo-dma ffe100300.dma: #6 (fsl,eloplus-dma-channel), irq 78
[ 1.614432] fsl-elo-dma ffe100300.dma: #7 (fsl,eloplus-dma-channel), irq 79
[ 1.621923] fsl-elo-dma ffe101300.dma: #0 (fsl,eloplus-dma-channel), irq 32
[ 1.628863] fsl-elo-dma ffe101300.dma: #1 (fsl,eloplus-dma-channel), irq 33
[ 1.635831] fsl-elo-dma ffe101300.dma: #2 (fsl,eloplus-dma-channel), irq 34
[ 1.642798] fsl-elo-dma ffe101300.dma: #3 (fsl,eloplus-dma-channel), irq 35
[ 1.649766] fsl-elo-dma ffe101300.dma: #4 (fsl,eloplus-dma-channel), irq 80
[ 1.656733] fsl-elo-dma ffe101300.dma: #5 (fsl,eloplus-dma-channel), irq 81
[ 1.663701] fsl-elo-dma ffe101300.dma: #6 (fsl,eloplus-dma-channel), irq 82
[ 1.670667] fsl-elo-dma ffe101300.dma: #7 (fsl,eloplus-dma-channel), irq 83
[ 1.678115] fsl-elo-dma ffe102300.dma: #0 (fsl,eloplus-dma-channel), irq 464
[ 1.685146] fsl-elo-dma ffe102300.dma: #1 (fsl,eloplus-dma-channel), irq 465
[ 1.692197] fsl-elo-dma ffe102300.dma: #2 (fsl,eloplus-dma-channel), irq 466
[ 1.699252] fsl-elo-dma ffe102300.dma: #3 (fsl,eloplus-dma-channel), irq 467
[ 1.706310] fsl-elo-dma ffe102300.dma: #4 (fsl,eloplus-dma-channel), irq 473
[ 1.713362] fsl-elo-dma ffe102300.dma: #5 (fsl,eloplus-dma-channel), irq 474
[ 1.720416] fsl-elo-dma ffe102300.dma: #6 (fsl,eloplus-dma-channel), irq 475
[ 1.727471] fsl-elo-dma ffe102300.dma: #7 (fsl,eloplus-dma-channel), irq 476
But to do the DMA transfer from the t2080rdb to the fpga,
What is the procedure and steps to follow at t2080 processor side?
Is there any sample source is available?
The address BAR0 and BAR1 is mapped as per the request from the FPGA and able to write and read back the data from the registers.
> able to write and read back the data from the registers.
What do you mean?
How these read/write operations correspond to the DMA transfers?
Hi Ufedor
> What do you mean?
Able to talk to the FPGA over PCIe correctly
> How these read/write operations correspond to the DMA transfers?
It is not regarding DMA, it is about correct mapping of address bars.
Can you just direct us on the DMA sample code for t2080rdb to the FPGA?
Thank you.
Best Regards,
Sadashiva