AnsweredAssumed Answered

i.MX6 - How to enable PCIe 2.0 ?

Question asked by Brian Lee on Oct 8, 2018
Latest reply on Oct 10, 2018 by igorpadykov

Hi All,

 

I have a custom board design based on the i.MX6Q Sabresd board (MCIMX6Q-SDB) running Android 8.0.

 

The board has a full-size PCIe 2.0 x1 connection. I have confirmed that this is working by plugging in a PCIe 2.0 to USB 3.0 card (SilverStone SST-EC04-P, link). In Android, via the command line, I can mount an external USB SSD (MX500 120Gb) and see the files, write files etc.

 

I am seeing an issue whereby the PCIe connection is only coming up as Gen 1.

 

board_01:/ # dmesg | grep -i pci
[    0.723168] PCI: CLS 0 bytes, default 64
[    0.879982] OF: PCI: host bridge /soc/pcie@0x01000000 ranges:
[    0.879994] OF: PCI:   No bus range found for /soc/pcie@0x01000000, using [bus 00-ff]
[    0.880025] OF: PCI:    IO 0x01f80000..0x01f8ffff -> 0x00000000
[    0.880041] OF: PCI:   MEM 0x01000000..0x01efffff -> 0x01000000
[    0.935078] imx6q-pcie 1ffc000.pcie: Link up, Gen1
[    0.935287] imx6q-pcie 1ffc000.pcie: PCI host bridge to bus 0000:00
[    0.935301] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.935311] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    0.935320] pci_bus 0000:00: root bus resource [mem 0x01000000-0x01efffff]
[    0.935358] pci 0000:00:00.0: [16c3:abcd] type 01 class 0x060400
[    0.935382] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    0.935400] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
[    0.935471] pci 0000:00:00.0: supports D1
[    0.935480] pci 0000:00:00.0: PME# supported from D0 D1 D3hot D3cold
[    0.935741] PCI: bus0: Fast back to back transfers disabled
[    0.935954] pci 0000:01:00.0: [1912:0014] type 00 class 0x0c0330
[    0.936062] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00001fff 64bit]
[    0.936546] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    0.960535] PCI: bus1: Fast back to back transfers disabled
[    0.960722] pci 0000:00:00.0: BAR 0: assigned [mem 0x01000000-0x010fffff]
[    0.960738] pci 0000:00:00.0: BAR 8: assigned [mem 0x01100000-0x011fffff]
[    0.960749] pci 0000:00:00.0: BAR 6: assigned [mem 0x01200000-0x0120ffff pref]
[    0.960765] pci 0000:01:00.0: BAR 0: assigned [mem 0x01100000-0x01101fff 64bit]
[    0.960829] pci 0000:00:00.0: PCI bridge to [bus 01]
[    0.960842] pci 0000:00:00.0:   bridge window [mem 0x01100000-0x011fffff]
[    0.961106] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt
[    0.961117] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[    0.961129] pcie_pme 0000:00:00.0:pcie001: service driver pcie_pme loaded
[    0.961269] aer 0000:00:00.0:pcie002: service driver aer loaded
[    0.961444] pci 0000:01:00.0: enabling device (0140 -> 0142)
[    2.136127] ehci-pci: EHCI PCI platform driver

 

board_01:/ # lspci -k                                                         
00:00.0 Class 0604: 16c3:abcd pcieport
01:00.0 Class 0c03: 1912:0014 xhci_hcd

 

Looking at the driver code (pci-imx6.c), it appears that the process for entering PCIe 2.0 mode is as follows:

 

 

static int imx6_pcie_establish_link(struct imx6_pcie *imx6_pcie)
{
    struct pcie_port *pp = &imx6_pcie->pp;
    struct device *dev = pp->dev;
    u32 tmp;
    int ret;

    /*
     * Force Gen1 operation when starting the link.  In case the link is
     * started in Gen2 mode, there is a possibility the devices on the
     * bus will not be detected at all.  This happens with PCIe switches.
     */
    if (!IS_ENABLED(CONFIG_PCI_IMX6_COMPLIANCE_TEST)) {
        tmp = dw_pcie_readl_rc(pp, PCIE_RC_LCR);
        tmp &= ~PCIE_RC_LCR_MAX_LINK_SPEEDS_MASK;
        tmp |= PCIE_RC_LCR_MAX_LINK_SPEEDS_GEN1;
        dw_pcie_writel_rc(pp, PCIE_RC_LCR, tmp);
    }

    /* Start LTSSM. */
    if (imx6_pcie->variant == IMX7D)
        regmap_update_bits(imx6_pcie->reg_src, 0x2c, BIT(6), BIT(6));
    else
        regmap_update_bits(imx6_pcie->iomuxc_gpr, IOMUXC_GPR12,
                IMX6Q_GPR12_PCIE_CTL_2, 1 << 10);

    ret = imx6_pcie_wait_for_link(imx6_pcie);
    if (ret) {
        dev_info(dev, "Link never came up\n");
        goto err_reset_phy;
    }

    if (imx6_pcie->link_gen == 2) {

        /* Allow Gen2 mode after the link is up. */
        tmp = dw_pcie_readl_rc(pp, PCIE_RC_LCR);
        tmp &= ~PCIE_RC_LCR_MAX_LINK_SPEEDS_MASK;
        tmp |= PCIE_RC_LCR_MAX_LINK_SPEEDS_GEN2;
        dw_pcie_writel_rc(pp, PCIE_RC_LCR, tmp);
    } else {
        dev_info(dev, "Link: Gen2 disabled\n");
        goto out;
    }

    /*
     * Start Directed Speed Change so the best possible speed both link
     * partners support can be negotiated.
     */
    tmp = dw_pcie_readl_rc(pp, PCIE_LINK_WIDTH_SPEED_CONTROL);
    tmp |= PORT_LOGIC_SPEED_CHANGE;
    dw_pcie_writel_rc(pp, PCIE_LINK_WIDTH_SPEED_CONTROL, tmp);

    ret = imx6_pcie_wait_for_speed_change(imx6_pcie);
    if (ret) {
        dev_info(dev, "Roll back to GEN1 link!\n");
    }

    /* Make sure link training is finished as well! */
    ret = imx6_pcie_wait_for_link(imx6_pcie);
    if (ret) {
        dev_err(dev, "Failed to bring link up!\n");
        goto err_reset_phy;
    }

out:
    tmp = dw_pcie_readl_rc(pp, PCIE_RC_LCSR);
    dev_info(dev, "Link up, Gen%i\n", (tmp >> 16) & 0xf);
    return 0;

err_reset_phy:
    dev_dbg(dev, "PHY DEBUG_R0=0x%08x DEBUG_R1=0x%08x\n",
        dw_pcie_readl_rc(pp, PCIE_PHY_DEBUG_R0),
        dw_pcie_readl_rc(pp, PCIE_PHY_DEBUG_R1));
    imx6_pcie_reset_phy(imx6_pcie);

    if (!IS_ENABLED(CONFIG_PCI_IMX6_COMPLIANCE_TEST)) {
        clk_disable_unprepare(imx6_pcie->pcie);
        if (!imx6_pcie->ext_osc)
            clk_disable_unprepare(imx6_pcie->pcie_bus);
        clk_disable_unprepare(imx6_pcie->pcie_phy);
        if (imx6_pcie->variant == IMX6SX)
            clk_disable_unprepare(imx6_pcie->pcie_inbound_axi);
        release_bus_freq(BUS_FREQ_HIGH);
        if (imx6_pcie->pcie_phy_regulator != NULL)
            regulator_disable(imx6_pcie->pcie_phy_regulator);
        if (imx6_pcie->pcie_bus_regulator != NULL)
            regulator_disable(imx6_pcie->pcie_bus_regulator);
    }

    return ret;
}

 

I think this equates to:

 

1. Enter PCIe 1.0 mode

2. Wait for link to come up

3. Attempt to Enter PCI 2.0 mode

4. Report actual mode

 

For debug I have put print statements in the code, and I can see that it does enter the "if (imx6_pcie->link_gen == 2)" statement.

 

So I am wondering why I do not get Gen2 as the result in dmesg.

 

I see elsewhere that there is an issue with Jitter, whereby it is recommended that an external clock source is used for PCIe 2.0. This was not noticed when we were designing the board.

 

My questions are as follows:

 

1) It looks like the negotiation for PCIe 2.0 speeds is ultimately done by the hardware, and that it will automatically fall back to PCIe 1.0 if this does not succeed. Is this correct?

 

2) Would the jitter problem be the reason we cannot negotiate PCIe 2.0? i.e. the system is testing whether it is PCIe 2.0 capable and determining we are not as a result of the jittery clock?

 

Thanks,

 

Brian.

Outcomes