IMX8M- MINI PCIe throughput about vc0 Posted Buffer size

[YinDeng]

Hi all,

I have a xilinx k7 ep + imx8mm rc design with software release L5.4.70_2.3.0.

The throughput only 197MB/s when I try to transfer data from k7 to imx8mm by ep write.But when I used the same k7 to transfer data to ti am5728, I can get 333MB/s throughput.

I had also found the nxp i.MX8 MM PCIe test result in topic https://community.nxp.com/t5/i-MX-Processors/PCIe-Bandwidth/m-p/1063269#M156271 The ep dma write also just have 230MB/s，but there's no answer why the speed is so slow.

I do some debug work and I found that the ep write speed limite may be the little posted buffer size of vc0. The posted header size and data size only 12 credits and 25 credits。The ti am5728 and k7 is almost the doubule and more。

But When I try to expand the buffer size by change VC0_P_DATA_ SCALE, VC0_P_HDR_S CALE in "Segmented-Buffer VC0 Posted Receive Queue Control" Register before start LTSSM （Dose it should be done more earlier？） ，the buffer size had no change. Is there something wrong?  

There's the code

diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index a5e11f8..e01f326 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -1800,6 +1800,15 @@ static int imx6_pcie_establish_link(struct imx6_pcie *imx6_pcie)
int ret;

dw_pcie_dbi_ro_wr_en(pci);
+
+ tmp = dw_pcie_readl_dbi(pci, 0x748);
+ printk("tmp before: 0x%x\n", tmp);
+ //dw_pcie_writel_dbi(pci, 0x748, 0x4F20C019);
+ //dw_pcie_writel_dbi(pci, 0x748, 0x4A20C019);
+ dw_pcie_writel_dbi(pci, 0x748, 0x4520C019);
+ tmp = dw_pcie_readl_dbi(pci, 0x748);
+ printk("tmp after: 0x%x\n", tmp);
+

[timharvey]

YinDeng,

I have also been looking for the details of why the IMX8M PCIe performance is lower than expected. I managed to find AN13164 from NXPtitled 'iMX8MP PCIe Bandwith Analysis' yet I can not find this app note anywhere on NXP's site:  https://comm.eefocus.com/media/download/index/id-1021154

As shown in Table 2, one can notice that while in some cases the bandwidth fits Gen 3 expectations, in other cases it is limited
to the performance slightly better than the realistic Gen2 throughput.

This is due to an i.MX 8M Plus limitation decribed below.

When the outstanding inbound write data transfer size exceeds 400 Bytes, the number of inbound MWr TLP transactions that the
i.MX 8M Plus PCIe Controller can support is up to the combination of 12 headers and 400 bytes of data payload. This remains
true as long as neither is exceeded.

This limitation can also be noticed on i.MX 8M Mini and i.MX 8M Quad regardless of the PCIe Gen X.
NOTE
Higher performance can be obtained to achieve the same goal by having the i.MX 8M Plus issue outbound MRd transactions
instead of using inbound MWr.

I'm not sure if this explains what you are seeing. Please let us know if you learn anything more.

Tim

[YinDeng]

timharvey​,

  The inbound MWr TLP buffer size what I get is from my K7 design by the “[PG054] 7 Series Integrated Block for PCI Express IP”

  I had also found that the imx6 and imx7 serial platform can change this buffer size, but  imx8 is not. And VC0_P_DATA_ SCALE, VC0_P_HDR_S CALE are not useful and available in the true .
 I think swapping the PCIe roles or transmit direction can round this transfer limit.

 Finally, 《DWC_pcie_ctl_rc_databook in 2019》  is helpful.