i.MX6Q: Using an external reference for PCIe

[kitz36]

I'm working on a design in which the i.MX6Q communicates with a single PCIe endpoint.  Unlike the SABRE reference designs the reference clock from the endpoint comes from an on-board oscillator, not the i.MX6.  We recently got the i.MX6Q to use this external clock as a reference for PCIe.  I wanted to share our experiences -- and hopefully compare/contrast to other solutions.

We were able to bring the PCIe link up initially using the PCIe "internal" reference.  The PCIe was driven by the ENET PLL.  This PLL provides clocks for Ethernet, PCIe, and SATA.  This wasn't a ideal setup, however, as the endpoint was on an asynchronous reference.

Our solution to using an external reference for PCIe has three parts:

1. Configure the CLK pins to accept a clock as input.

2. Bypass the ENET PLL, using the CLK pins as the alternate source.

3. Configure the PCIe PHY to accept a clock other than 125 MHz.

To configure CLK pins as input you must adjust the PMU_MISC1n register (reference section 50.7.6 in Rev. O of the reference manual).  Clear the appropriate OBEN bit and set the IBEN bit.  The clock source bits are don't-care; they're only used as an output.  In my case, I modified the _clk_pcie_enable() and _clk_pcie_disable() routines in the Freescale BSP @ linux/arch/arm/mach-mx6/clock.c.

The ENET PLL is bypassed by adjusting the CCM_ANALOG_PLL_ENETn register (section 18.7.15 in the Rev. O RM).  Set the BYPASS_CLK_SRC (bits [15:14]) to the source clock and set the BYPASS bit (bit 16).  I updated the _clk_pll_enable() routine in clock.c to make these adjustments.  See below.

static int _clk_pll_enable(struct clk *clk)

{

    unsigned int reg;

    void __iomem *pllbase;

    pllbase = _get_pll_base(clk);

    reg = __raw_read(pll_base);

    /* Bypass the ENET PLL; use CLK1 as the source */

    if (clk == &pll8_enet_main_clk) {

        reg &= ~ANADIG_PLL_BYPASS_CLK_SRC_MASK;

        reg |= (0x1 << ANADIG_PLL_BYPASS_CLK_SRC_OFFSET);

    }

    /* power-up the PLL */

    reg &= ~ANADIG_PLL_POWER_DOWN;

    /* ...snip... */

    /* Enable the PLL output now */

    reg = __raw_readl(pllbase);

    if (clk != &pll8_enet_main_clk)

        reg &= ~ANADIG_PLL_BYPASS;

    reg |= ANADIG_PLL_ENABLE;

    __raw_writel(reg, pllbase);

    return 0;

}

Finally, the PCIe PHY needs to be adjusted to accept a clock at the provided frequency.  In our case, the reference clock is 100MHz.  By default the PCIe PHY wants a 125MHz reference.  To accept another rate adjust the MPLL according to table 49-1 in the reference manual.  Use the ATEOVRD (49.5.8) and MPLL_OVRD_IN_LO  (49.5.9) registers.  NOTE:  The ATEOVRD shows bit 0 as reserved.  I determined through experimentation that this isn't the case.  Bit 0 is ref_clkdiv2, bit 1 is ref_usb2_en, and bit 2 is ateovrd_en.  See code below.

#define PCIE_PHY_ATEOVRD          (0x10)

#define PCIE_PHY_MPLL_OVRD_IN_LO  (0x11)

static int imx_pcie_override_phy_mpll(u32 mpll_multiplier, u32 ref_clkdiv2)

{

    u32 ref_usb2_en;

    u32 reg;

    int ret;

    pr_info("overriding PCIe PHY MPLL config: multiplier = %d, clkdiv2 = %d\n",

        mpll_multiplier, ref_clkdiv2);

    /* set the MPLL override value to 'disabled' */

    pcie_phy_cr_read(PCIE_PHY_MPLL_OVRD_IN_LO, &reg);

    reg &= ~(0x1 << 1);

    pcie_phy_cr_write(PCIE_PHY_MPLL_OVRD_IN_LO, reg);

    /* enable MPLL override */

    reg |= (0x1 << 0);

    pcie_phy_cr_write(PCIE_PHY_MPLL_OVRD_IN_LO, reg);

    /* set the new MPLL multiplier */

    reg &= ~(0x7F << 2);

    reg |=  (mpll_multiplier << 2);

    pcie_phy_cr_write(PCIE_PHY_MPLL_OVRD_IN_LO, reg);

    /* enable multiplier override */

    reg |= (0x1 << 9);

    pcie_phy_cr_write(PCIE_PHY_MPLL_OVRD_IN_LO, reg);

    /*

     * set the ref_clkdiv2.  when this override is enabled it

     * overrides both ref_clkdiv2 and ref_usb2_en.  make sure

     * the overriden ref_usb2_en reflects the original value.

     */

     pcie_phy_cr_read(PCIE_PHY_ATEOVRD, &reg);

     ref_usb2_en = (reg >> 3) & 0x1;

     /* set the current value of ref_usb2_en as the override */

     reg &= ~(0x1 << 1);

     reg |=  (ref_usb2_en << 1);

     /* set the ref_clkdiv2 override */

     reg &= ~(0x1 << 0);

     reg |=  (ref_clkdiv2 << 0);

     pcie_phy_cr_write(PCIE_PHY_ATEOVRD, reg);

     /* enable the ref_clkdiv2 override */

     reg |= (0x1 << 2);  /* ateovrd_en */

     pcie_phy_cr_write(PCIE_PHY_ATEOVRD, reg);

     /* disable the MPLL override */

     pcie_phy_cr_read(PCIE_PHY_MPLL_OVRD_IN_LO, &reg);

     reg &= ~(0x1 << 0);

     reg |=  (0x1 << 1);

     pcie_phy_cr_write(PCIE_PHY_MPLL_OVRD_IN_LO, reg);

     return 0;

}

I call this routine from the imx_pcie_pltfm_probe() routine in linux/arch/arm/mach-mx6/pcie.c just before enabling the LTSSM.

That's it.  This seems to work for me.  If anyone has other experiences I'd love to hear about it.

-Charlie

[mitchellcox]

Sorry to resurrect this - although it's still a good reference for most people!

I'm using the newer kernel, 3.10, which uses device trees for setup. The clock code is very different to what it used to be and I can't seem to figure out how to apply your patches. I'm guessing that they may already be applied and I just need to set up a different clock in the dtb...

Could someone please shed some light on using an external clock for PCIe with the newer kernels (3.10 perhaps)? There is basically no documentation :smileysad:

Thanks!

[keytwo]

Hello,

I'm currently trying to use the sabrelite board to act as a PCIe EndPoint. The thing is that the sabrelite has on its PCIe connector (j23) only pin TX+/TX- and RX+/RX- (as well as gnd/3.3v) but no CLK pair. If I understand correctly, by default, the PCIe core is calibrated at 125Mhz and therefor, without any CLK pair comming from the PCIe RC, I would have to get it to work on a 100Mhz so change the PLL.

I was wondering if this code you posted here could help for this purpose as well.

Looking at this place: i.MX6Q PCIe EP/RC Validation System They use 2 boards that are the same and therefor, both are calibrated at 125Mhz. I wanted to use it with a real PCIe bus coming from a PC.

When looking at the patch "0001-ENGR00268112-pcie-emaluate-the-pcie-ep-as-ram-device.patch.zip" of the same page, I see:

-------

/*

* configure the class_rev(emaluate one memory ram ep device),

* bar0 and bar1 of ep

*/

writel(0xdeadbeaf, dbi_base + PCI_VENDOR_ID);

writel( readl(dbi_base + PCI_CLASS_REVISION)  |  (PCI_CLASS_MEMORY_RAM << 16), dbi_base + PCI_CLASS_REVISION);

--------

and when I run it with the patch, and do a memtool:

$ ./memtool -32 0x01FFC000 4

ABCD16C3

Knowing that 0x01FFC000 is the address of the PCIe PID/VID in EP mode and ABCD16C3 is the default PID/VID according to 48.8.1 of the documentation.

So I'm a little bit lost out there, and would like any kind of opinion about it.

regards

[karthikas]

Hello

I am using I.mx6 ARM processor, I want to configure PCIe clock frequency to 100MHZ.

1.I need to write standalone program to set PCIe clock frequency,and I need to generate binary file of this program.

2.I have decided to use above mentioned functions,If so what are the header file I need to be include and please suggest me how generate binary file