T2080 DMA to access FPGA BLOCKRAM over PCIe

[flabyjacob]

Hi all,

I have a t2080rdb based custom board

In UBoot, I have enabled DMA and it is able to access local DDR.

I need to access the FPGA BLOCKRAM connected on the PCIe3 using DMA.

PCIe3 memory map is done in the T208xRDB.h in the UBoot source code as follows,

#define CONFIG_SYS_PCIE3_MEM_VIRT       0xb0000000

#define CONFIG_SYS_PCIE3_MEM_BUS        0xe0000000

#define CONFIG_SYS_PCIE3_MEM_PHYS       0xc30000000ull

#define CONFIG_SYS_PCIE3_MEM_SIZE       0x10000000      /* 256M */

#define CONFIG_SYS_PCIE3_IO_VIRT        0xf8020000

#define CONFIG_SYS_PCIE3_IO_BUS         0x00000000

#define CONFIG_SYS_PCIE3_IO_PHYS        0xff8020000ull

#define CONFIG_SYS_PCIE3_IO_SIZE        0x00010000      /* 64k */

I am able to access BLOCKRAM from CPU as follows,

md  b0010000

But when I give the same address as the destination address for DMA, it is giving me an error(TE bit set in the DMA Status Register).

Can somebody help me to proceed further?

Regards

Flaby

[ufedor]

Please note that core uses MMU, so physical address of the PCIe Outbound window has to be obtained using 'ioremap'.

[flabyjacob]

Hi,

T2080Reference manual - section:23.3.23 says,

"The target interface is derived from the local access window and outbound ATMU mappings and

the transaction is obtained from the value specified in SATRn[SREADTTYPE]"

Do we need to configure any DMA registers to enable the above feature in UBoot sources - ie,DMA to see the LAW and outbound ATMU mappings?

Regards

Flaby

[ufedor]

No such DMA configuration is needed.

It is required to specify corrrect physical address for the DMA - in the discussed case PCIe3 memory window base address is 0xc30000000.

[flabyjacob]

Hi Fedor,

Thanks for your reply...

The above mentioned physical address is already handled in UBoot source code in the PCIe initialization. I understand that a translation happens on the PCIe outbound registers to get an access to the physical address 0xc30000000 and because of this I am able to access FPGA BLOCKRAM via CPU without any problem. Please correct me if I am wrong...

  Hence, can we give this physical address directly to DMA by skipping the translation part in the PCIe controller?

Regards

Flaby

[ufedor]

> I understand that a translation happens on the PCIe outbound

>registers to get an access to the physical address 0xc30000000

Your understanding is not correct.

The PCIe controller does not perform SOC-level address translation - i.e. DMA must use the physical address 0xc30000000 to access the PCIe3 Outbound window.

[flabyjacob]

Hi Fedor,

Thanks  for your reply...

When I give 0xc30000000 as the destination address for DMA, my DDR memory at address 0x3000000 is getting filled with the data from the source address given to the DMA

Regards

Flaby

[ufedor · Accepted solution]

This means that DMA is programmed incorrectly.

It is required to check that DMAx_DATRn[EDAD]=0xC.

[brett_p_stahlma]

ufedor wrote:

This means that DMA is programmed incorrectly.

It is required to check that DMAx_DATRn[EDAD]=0xC.

Is there something like this for the LS1012A? I don't see any such register in the eDMA module, and I need to specify 0x40_0000_0000 (PCIe config memory) as the source address.

Thanks,

Brett S.

[flabyjacob]

Hi Fedor,

Thanks a lot...

That solves my problem

In UBoot,

the following code part does the SATR and DATR initialization

                out_dma32(&dma->satr,

                         in_dma32(&dma->satr) | (u32)((u64)src >> 32));

               out_dma32(&dma->datr,

                        in_dma32(&dma->datr) | (u32)((u64)dest >> 32));

By some reason, 0xc of my physical address was not coming there properly. I had checked this part earlier and just left it assuming that there is nothing wrong in that. Currently, I hardcoded DATR with 0xc and see that FPGA BLOCKRAM is filled with data from the source address given to DMA.

Now, I need to look into my code part in which I am pushing destination address to DMA where something wrong happens

Next, I need to access DDR connected to the FPGA too...

Thank you once again for helping me to resolve the issue

Regards

Flaby

[flabyjacob]

Hi,

Thanks for your reply...

But how to obtain it in UBoot?

As I mentioned in the first mail,

CONFIG_SYS_PCIE3_MEM_PHYS is 0xc30000000ull in UBoot's T208xRDB.h file

Can I give the same address to DMA to access the BLOCKRAM?

Regards

Flaby