i.MX8QM PCIe Couldn't get pcie-phy

[aloshchilov]

Hi, i work with iMX8QM based board and currently i am trying to get PCIe working and connect a USB3.0 hub in further. 

Could you please advise, what is wrong and where should i find some related docs?

kernel v 5.4.24

Currently I va got fails on a very first stage. dmesg says kernel couldn't get pcie phy:

[ 1.594490] imx6q-pcie 5f010000.pcie: couldn't get pcie-phy
[ 1.600012] imx6q-pcie 5f010000.pcie: No cache used with register defaults set!
[ 1.612598] imx6q-pcie 5f010000.pcie: host bridge /bus@5f000000/pcie@0x5f010000 ranges:
[ 1.620283] imx6q-pcie 5f010000.pcie: IO 0x7ff80000..0x7ff8ffff -> 0x00000000
[ 1.627663] imx6q-pcie 5f010000.pcie: MEM 0x70000000..0x7fefffff -> 0x70000000
[ 1.743667] imx6q-pcie 5f010000.pcie: PCIe PLL locked after 0 us.
[ 2.747695] imx6q-pcie 5f010000.pcie: Phy link never came up
[ 2.753501] imx6q-pcie 5f010000.pcie: failed to initialize host
[ 2.759431] imx6q-pcie 5f010000.pcie: unable to add pcie port.
[ 3.483527] ehci-pci: EHCI PCI platform driver

Same for both, pciea@5f000000 and pcieb@5f010000.

DT:

pciea: pcie@0x5f000000 {
compatible = "fsl,imx8qm-pcie","snps,dw-pcie";
reg = <0x5f000000 0x10000>, /* Controller reg */
<0x6ff00000 0x80000>, /* PCI cfg space */
<0x5f080000 0xf0000>; /* lpcg, csr, msic, gpio */
reg-names = "dbi", "config", "hsio";
#address-cells = <3>;
#size-cells = <2>;
device_type = "pci";
bus-range = <0x00 0xff>;
ranges = <0x81000000 0 0x00000000 0x6ff80000 0 0x00010000 /* downstream I/O */
0x82000000 0 0x60000000 0x60000000 0 0x0ff00000>; /* non-prefetchable memory */
num-lanes = <1>;
num-viewport = <4>;
interrupts = <GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 72 IRQ_TYPE_LEVEL_HIGH>; /* eDMA */
interrupt-names = "msi", "dma";
#interrupt-cells = <1>;
interrupt-map-mask = <0 0 0 0x7>;
interrupt-map = <0 0 0 1 &gic 0 73 4>,
<0 0 0 2 &gic 0 74 4>,
<0 0 0 3 &gic 0 75 4>,
<0 0 0 4 &gic 0 76 4>;
/*
* Set these clocks in default, then clocks should be
* refined for exact hw design of imx8 pcie.
*/
clocks = <&pciea_lpcg 0>,
<&pciea_lpcg 1>,
<&pciea_lpcg 2>,
<&phyx2_lpcg 0>,
<&phyx2_crr0_lpcg 0>,
<&pciea_crr2_lpcg 0>,
<&misc_crr5_lpcg 0>;
clock-names = "pcie", "pcie_bus", "pcie_inbound_axi",
"pcie_phy", "phy_per", "pcie_per", "misc_per";
power-domains = <&pd IMX_SC_R_PCIE_A>,
<&pd IMX_SC_R_SERDES_0>,
<&pd IMX_SC_R_HSIO_GPIO>;
power-domain-names = "pcie", "pcie_phy", "hsio_gpio";
fsl,max-link-speed = <3>;
hsio-cfg = <PCIEAX1PCIEBX1SATA>;
local-addr = <0x40000000>;
status = "disabled";
};

&pciea{
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_pciea>;
reset-gpio = <&lsio_gpio4 29 GPIO_ACTIVE_LOW>;
disable-gpio = <&lsio_gpio4 9 GPIO_ACTIVE_LOW>;
ext_osc = <1>;
epdev_on-supply = <&epdev_on>;
reserved-region = <&rpmsg_reserved>;
status = "okay";
};

pinctrl_pciea: pcieagrp{
fsl,pins = <
IMX8QM_PCIE_CTRL0_WAKE_B_LSIO_GPIO4_IO28 0x04000021
IMX8QM_PCIE_CTRL0_PERST_B_LSIO_GPIO4_IO29 0x06000021
IMX8QM_USDHC2_RESET_B_LSIO_GPIO4_IO09 0x06000021
>;
};

pcieb: pcie@0x5f010000 {
compatible = "fsl,imx8qm-pcie","snps,dw-pcie";
reg = <0x5f010000 0x10000>, /* Controller reg */
<0x7ff00000 0x80000>, /* PCI cfg space */
<0x5f080000 0xf0000>; /* lpcg, csr, msic, gpio */
reg-names = "dbi", "config", "hsio";
#address-cells = <3>;
#size-cells = <2>;
device_type = "pci";
bus-range = <0x00 0xff>;
ranges = <0x81000000 0 0x00000000 0x7ff80000 0 0x00010000 /* downstream I/O */
0x82000000 0 0x70000000 0x70000000 0 0x0ff00000>; /* non-prefetchable memory */
num-lanes = <1>;
num-viewport = <4>;
interrupts = <GIC_SPI 102 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 104 IRQ_TYPE_LEVEL_HIGH>; /* eDMA */
interrupt-names = "msi", "dma";
#interrupt-cells = <1>;
interrupt-map-mask = <0 0 0 0x7>;
interrupt-map = <0 0 0 1 &gic 0 105 4>,
<0 0 0 2 &gic 0 106 4>,
<0 0 0 3 &gic 0 107 4>,
<0 0 0 4 &gic 0 108 4>;
clocks = <&pcieb_lpcg 0>,
<&pcieb_lpcg 1>,
<&pcieb_lpcg 2>,
<&phyx2_lpcg 1>,
<&phyx2_lpcg 0>,
<&phyx2_crr0_lpcg 0>,
<&pcieb_crr3_lpcg 0>,
<&pciea_crr2_lpcg 0>,
<&misc_crr5_lpcg 0>;
clock-names = "pcie", "pcie_bus", "pcie_inbound_axi",
"pcie_phy", "pcie_phy_pclk", "phy_per",
"pcie_per", "pciex2_per", "misc_per";
power-domains = <&pd IMX_SC_R_PCIE_B>,
<&pd IMX_SC_R_PCIE_A>,
<&pd IMX_SC_R_SERDES_0>,
<&pd IMX_SC_R_HSIO_GPIO>;
power-domain-names = "pcie", "pcie_per", "pcie_phy",
"hsio_gpio";
fsl,max-link-speed = <3>;
hsio-cfg = <PCIEAX1PCIEBX1SATA>;
local-addr = <0x80000000>;
status = "disabled";
};

&pcieb{
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_pcieb>;
reset-gpio = <&lsio_gpio5 0 GPIO_ACTIVE_LOW>;
ext_osc = <1>;
status = "okay";
};

pinctrl_pcieb: pciebgrp{
fsl,pins = <
IMX8QM_PCIE_CTRL1_WAKE_B_LSIO_GPIO4_IO31 0x04000021
IMX8QM_PCIE_CTRL1_PERST_B_LSIO_GPIO5_IO00 0x06000021
>;
};

added to defconfig:

CONFIG_PCI=y
+CONFIG_PCI_DOMAINS=y
+CONFIG_PCI_SYSCALL=y

CONFIG_PCI_IOV=y
CONFIG_PCI_HOST_GENERIC=y
CONFIG_PCI_IMX6=y
+CONFIG_PCIE_DW=y
+CONFIG_PCIEPORTBUS=y
+CONFIG_PHY_FSL_IMX_PCIE=y

+CONFIG_HAS_DMA=y
+CONFIG_HAS_IOMEM=y

+CONFIG_USB_EHCI_HCD=y

[carloscs11]

Hi buddy,
I know this post was a long time ago, but did you fix it?

If yes please let me know how, I'm facing a similar problem.

Thank you!

[igorpadykov]

Hi Andrey

may be recommended to try latest nxp 5.10 kernel and test with several pcie cards

https://source.codeaurora.org/external/imx/linux-imx/tree/arch/arm64/boot/dts/freescale/imx8qm-mek.d...

Check hardware using Table 11. PCIe recommendations  i.MX 8QuadMax/i.MX 8QuadXPlus Hardware Developer’s Guide

Best regards
igor

[aloshchilov]

Hi Igor, unfortunatelly it is pretty hard to move to another kernel. And switching to another card is not possible - it is a hard-wired device.

I traced the driver a bit and found the execution stops after the dw_pcie_link_up returns False (Link Up bit (it is a bit 4) in PCIE_PORT_DEBUG1 register stays low). My modified function:

int dw_pcie_link_up(struct dw_pcie *pci)

{

    u32 val0, val1;

    if (pci->ops->link_up)

        return pci->ops->link_up(pci);

    val0 = readl(pci->dbi_base + PCIE_PORT_DEBUG0);

    val1 = readl(pci->dbi_base + PCIE_PORT_DEBUG1);

    dev_info(

        pci->dev, 

        "PCIE_DBG[0..1]: 0x%08X 0x%08X ", 

        val0, val1);

    return ((val1 & PCIE_PORT_DEBUG1_LINK_UP) &&

        (!(val1 & PCIE_PORT_DEBUG1_LINK_IN_TRAINING)));

}

I tried to find the detailed description of both debug registers, Debug Register 0 (PL_DEBUG0_OFF, 728h) and Debug Register 1 (PL_DEBUG1_OFF, 72Ch), but it looks like the Reference Manual doesn't contain it.

Anyway I've got number of various DBG0 values and constant DBG1.

[ 21.909436] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x00006300 0x08200000
[ 22.016518] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x00008600 0x08200000
[ 22.123597] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x0000D700 0x08200000
[ 22.230671] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x0000EB00 0x08200000
[ 22.337746] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x0000EE00 0x08200000
[ 22.444817] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x00000D00 0x08200000
[ 22.551890] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x00005E00 0x08200000
[ 22.658967] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x0000B000 0x08200000
[ 22.766046] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x00009300 0x08200000
[ 22.873120] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x00000800 0x08200000
[ 22.980195] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x0000BC00 0x08200000
[ 23.087268] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x00000B00 0x08200000
[ 23.194340] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x0000A100 0x08200000
[ 23.301417] imx6q-pcie 5f010000.pcie: PCIE_DBG[0..1]: 0x00005A00 0x08200000

I found the decoder (https://github.com/xobs/imx6-pcie/blob/master/imx6-pcie-decoder.c) but I am not sure it works properly for IMX8QM. 

it outputs something like 

LTSSM current state: 0x0 (S_DETECT_QUIET)
PIPE transmit K indication: 0
PIPE Transmit data: 0xa1
Receiver is receiving logical idle: no
Second symbol is also idle (16-bit PHY interface only): no
Currently receiving k237 (PAD) in place of link number: no
Currently receiving k237 (PAD) in place of lane number: no
Link control bits advertised by link partner: 0x0
Receiver detected lane reversal: no
TS2 training sequence received: no
TS1 training sequence received: no
Receiver reports skip reception: no
LTSSM reports PHY link up: no
A skip ordered set has been transmitted: no
Link number advertised/confirmed by link partner: 0
Application request to initiate training reset: no
PIPE transmit compliance request: no
PIPE transmit electrical idle request: yes
PIPE receiver detect/loopback request: no
LTSSM-negotiated link reset: yes
LTSSM testing for polarity reversal: no
LTSSM performing link training: no
LTSSM in DISABLE state; link inoperable: no
Scrambling disabled for the link: no

Could you please advise what else should I look at, what conditions should bring the LINK UP bit in PCIE_PORT_DEBUG1 register high? 

[igorpadykov]

>Could you please advise what else should I look at, what conditions should bring

>the LINK UP bit in PCIE_PORT_DEBUG1 register high? 

may be suggested to test with several PCIe cards and check PCIe signals quality using

AN12444 PCIe Certification Guide for i.MX8 Serials

Best regards
igor