Unable to setup PCIe on imx6Q SabreSD board

[cbonniot]

Hello,

I have a SabreSD and I am trying to use a Wireless PCIe card.

My issue, for now, is that I am unable to setup correctly the PCIe bus.

I have encountered several issue until now, one of them being one of the PINs used by the PCI (power-on) was claimed by "SD2" and I had to change the DTS file to disable this "SD2".

lspci is not failing but not outputing anything either.

Here is what I have when I grep dmesg with pci (note: the logs are the same whether I connect the card or not, here it is not connected):

<7>fixed up name for pcie@0x01000000 -> pcie

<7>fixed up name for minipcie_ctrl -> minipcie_ctrl

<7>   create child: /soc/pcie@0x01000000

<7>of_platform_bus_create() - skipping /minipcie_ctrl, no compatible prop

<7>comparing lvds_gate with pcie_ref_125m

<7>comparing lvds_gate with pcie_axi

<7>comparing pcie_ref_125m with pcie_ref_125m

<7>comparing pcie_axi with pcie_ref_125m

<7>comparing pcie_axi with sata_ref_100m

<7>comparing pcie_axi with pcie_axi

<7>comparing sata_ref_100m with pcie_ref_125m

<3>imx6q-pcie 1ffc000.pcie: phy link never came up

<3>imx6q-pcie 1ffc000.pcie: failed to initialize host

<7>PCI: CLS 0 bytes, default 64

<6>ehci-pci: EHCI PCI platform driver

Here is my kernel configuration regarding PCI:

CONFIG_MIGHT_HAVE_PCI=y

CONFIG_PCI_QUIRKS=y

CONFIG_PCI=y

CONFIG_PCI_DOMAINS=y

CONFIG_PCI_SYSCALL=y

CONFIG_PCI_DEBUG=y

# PCI host controller drivers

CONFIG_PCIE_DW=y

CONFIG_PCI_IMX6=y

CONFIG_PCIEPORTBUS=y

CONFIG_PCIEAER=y

CONFIG_PCIEASPM=y

CONFIG_PCIEASPM_DEFAULT=y

CONFIG_PCIE_PME=y

CONFIG_OF_PCI=y

CONFIG_OF_PCI_IRQ=y

CONFIG_SND_PCI=y

CONFIG_USB_EHCI_PCI=y

CONFIG_GENERIC_PCI_IOMAP=y

According to this log "phy link never came up", and from what I can understand in the code, it seems that the PCI controller is not answering (drivers/pci/host/pci-imx6.c).

So if I understand what is happening, I am not even at the connection of the Wireless Card.

I think I need to be able to see the PCI bridge before being able to plug in the card. Can anyone confirm that ?

I am fairly new to kernel world, configuration... And I basically know nothing about PCI...

I am using kernel 3.10.53 and Android 5.0.2.

This is the setup given by Freescale User Guides.

I completely exhausted my knowledge to fix this and I went through all the posts I could find.

None of them helping me, either because the issues don't look the same, or because I don't understand the answers...

I am aware that some information is missing in this post, so please do not hesitate to ask me for additional data.

Thank you in advance for you help !

Regards,

[igorpadykov]

HI Christophe

for PCIe usage there is no need for modifications of Linux sources,

please use guidelines in attached Linux Manual

sect.42.4 Using PCIe Endpoint and Running Tests

and use Freescale BSP from i.MX6Q product page

Board Support Packages (27)

L3.14.28_1.0.0_iMX6QDLS_BUNDLE (REV L3.14.28_1.0.0)

SABRE Board Reference Design|Freescale

Note, wireless PCIe card should be connected to Sabre board before running

linux, recommended to try several cards for testing phy link.

Best regards

igor

-----------------------------------------------------------------------------------------------------------------------

Note: If this post answers your question, please click the Correct Answer button. Thank you!

-----------------------------------------------------------------------------------------------------------------------

[christophebonni]

Hi Igor,

Thank you for your quick answer.

As I said, I am a novice, can you please elaborate when you say "use Freescale BSP from i.MX6Q product page", what does it mean exactly ? I am sorry if this is a dumb question but I am kind of lost.

Every source package I am using comes directly from Freescale, it has not been modified. It is the package that comes when following the Android User Guides, isn't that BSP part of it ?

To be the clearest possible, here is exactly what I have done to retrieve the sources:

$ mkdir myandroid

$ cd myandroid

$ ~/bin/repo init -u https://android.googlesource.com/platform/manifest -b android-5.0.2_r1

$ ~/bin/repo sync -j4 # this command loads most needed repos. Therefore, it can take several hours to load.

$ cd ~/myandroid/prebuilts/gcc/linux-x86/arm

$ git clone https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.6

$ cd arm-eabi-4.6

$ git checkout android-4.4.3_r1

$ cd ~/myandroid

$ git clone git://git.freescale.com/imx/linux-2.6-imx.git kernel_imx # the kernel repo is large. Therefore, this process can take a while.

$ cd kernel_imx

$ git checkout l5.0.0_1.0.0-ga

$ cd ~/myandroid/bootable

$ cd bootloader

$ git clone git://git.freescale.com/imx/uboot-imx.git uboot-imx

$ cd uboot-imx

$ git checkout l5.0.0_1.0.0-ga

$ cd ~

$ mkdir Patch_Android

$ cd Patch_Android

$ cp path/to/patch/from/freescale/android_L5.0.0_1.0.0-ga_core_source.gz .

$ tar xzvf android_L5.0.0_1.0.0-ga_core_source.gz

$ cd android_L5.0.0_1.0.0-ga_core_source/code/

$ tar xzvf l5.0.0_1.0.0-ga.tar.gz

$ cd ~/myandroid

$ source ~/Patch_Android/android_L5.0.0_1.0.0-ga_core_source/code/l5.0.0_1.0.0-ga/and_patch.sh

$ c_patch ~/Patch_Android/android_L5.0.0_1.0.0-ga_core_source/code/l5.0.0_1.0.0-ga imx_l5.0.0_1.0.0-ga

The only modifications I made are on the kernel configuration file kernel_imx/arch/arm/configs/imx_v7_android_defconfig in order to include PCI configurations and the modification I describe below.

Now, about your recommendations.

I followed those guidelines and as you can see in my first post, the kernel has the requested CONFIG_PCI...

Also, the PCIe card I am using is not part of the list provided in the manual.

I understand that hot plug is not supported, but I thought that before trying to use the card, I should at least be able to see the PCI host bridge when I do lspci, isn't that right ?

If I remove the modification I made, which is:

android/kernel_imx/arch/arm/boot/dts/imx6q-sabresd.dts:

&usdhc2 {

    status = "disabled";

};

Here is the log I have at startup in dmesg:

<4>!!request miniPCIE Power On gpio

<3>imx6q-pcie 1ffc000.pcie: unable to get power-on gpio

<4>imx6q-pcie: probe of 1ffc000.pcie failed with error -16

<7>PCI: CLS 0 bytes, default 64

<6>ehci-pci: EHCI PCI platform driver

Thank you.

Regards

[evgenyerlihman]

Hey Christophe,

I am trying to build the Android image, but it seems like i am missing the imx_v7_android_defconfig file. Did you have it in the configs directory after just following the steps you described or did you do something additional?

Thanks,

Evgeny

[igorpadykov]

HI Christophe

could you try with Linux L3.14.28_1.0.0

without any modifications in dts files (only add PCIe

support to defconfig). PCI host will not be detected by lspci,

if no PCIe card is connected to board.

Best regards

igor

[christophebonni]

Hi Igor,

So I downloaded the BSP L3.14.28.

I followed the Freescale_Yocto_Project_User's_Guide.pdf to build my image.

Without any modification, I flashed the image and successfully booted Linux.

After that, I used

bitbake -c menuconfig linux-imx

I configured PCI and PCIe

But then, when trying to copy the saved configuration file to the source directory (in order to rebuild the kernel with it), I was not able to find the directory corresponding to the version 3.14.28. Here is the content of the folder:

linux-fslc          linux-imx-mfgtool-3.10.17  linux-fslc.inc                linux-imx-rt_3.10.17.bb

linux-imx           linux-imx-rt-3.10.17       linux-imx_2.6.35.3.bb         linux-ls1_3.12.bb

linux-imx-2.6.35.3  linux-ls1                  linux-imx_3.10.17.bb          linux-ls1.inc

linux-imx-3.10.17   linux-timesys-3.0.15       linux-imx.inc                 linux-mfgtool.inc

linux-fslc_3.17.bb         linux-imx-mfgtool_3.10.17.bb  linux-timesys_3.0.15.bb

I thought that for each linux built, there would be a folder containing the board configuration, maybe I'm wrong ?

I checked using uname -r and version is 3.14.28.

I searched on the internet for similar questions but couldn't find an answer.

I tried to copy the configuration file to a new folder I created named linux-imx-3.14.28/mx6/defconfig (as it is done for the other versions) and rebuilt it:

bitbake -c cleansstate linux-imx

bitbake fsl-image-gui

But when I looked in dmesg, there was absolutely no mention of PCI. So I guess that my new configuration file was not used.

Do you know what the issue is here and how I can use the config file ?

I tried to search this information in the manuals attached to the bundle but was unable to find the answer.

Thank you very much for your help !

Regards,

[christophebonni]

Hi Igor,

Finally, I replaced the configuration file I found in linux-imx-3.10.17/mx6/ by the one I updated with PCI support and rebuilt.

I think that the kernel got compiled with the correct one since I am able to do lspci and here is what is displayed in dmesg:

root@imx6qsabresd:~# dmesg | grep -e pci -e PCI

PCI: CLS 0 bytes, default 64

imx6q-pcie 1ffc000.pcie: phy link never came up

imx6q-pcie 1ffc000.pcie: Failed to bring link up!

imx6q-pcie 1ffc000.pcie: failed to initialize host

imx6q-pcie: probe of 1ffc000.pcie failed with error -22

ehci-pci: EHCI PCI platform driver

It seems that there is the same issue.

PCIe card is connected and lspci shows nothing.

Thank you.

[igorpadykov]

HI Christophe

for yocto building one can refer to attached yocto guide and below links

Freescale Yocto Project main page

The Yocto Project™ and Linux® Software Development for i.MX Application Processors

Best regards

igor

[christophebonni]

Hi Igor,

Any update ?

I really exhausted all the leads I had.

Your help is greatly appreciated.

Thank you.

[igorpadykov]

HI Christophe

for yocto building problems please

create new thread.

Best regards

igor

[christophebonni]

Hi Igor,

I no longer have issues to build Yocto.

Here is what is displayed in dmesg:

root@imx6qsabresd:~# dmesg | grep -e pci -e PCI

PCI: CLS 0 bytes, default 64

imx6q-pcie 1ffc000.pcie: phy link never came up

imx6q-pcie 1ffc000.pcie: Failed to bring link up!

imx6q-pcie 1ffc000.pcie: failed to initialize host

imx6q-pcie: probe of 1ffc000.pcie failed with error -22

ehci-pci: EHCI PCI platform driver

It seems that there is the same issue.

PCIe card is connected and lspci shows nothing.

Thank you.

[igorpadykov]

HI Christophe

is this one card, could you try some other card ?

Best regards

igor

[christophebonni]

Hi Igor,

Thank you for your answer.

Sorry for the delay, I had to buy another one.

With the other card, PCI bus seems to initialize itself correctly. So I guess the issue came from the first card.

I thank you for your continued support !

Regards,

[christophebonni]

Hello Igor,

I am sorry, I may have made a mistake in telling you that there is no longer any issue.

Still with my kernel 3.14.28 and with the new miniPCIe card.

Here is now the output of lspci.

00:00.0 PCI bridge: Device 16c3:abcd (rev 01)

01:00.0 Network controller: Qualcomm Atheros AR93xx Wireless Network Adapter (rev 01)

However, I don't have a network interface created in ifconfig.

Here is now the pci initialization (which seems to complete successfully):

imx6q-pcie 1ffc000.pcie: PCI host bridge to bus 0000:00

pci_bus 0000:00: root bus resource [io  0x1000-0x10000]

pci_bus 0000:00: root bus resource [mem 0x01000000-0x01efffff]

pci_bus 0000:00: root bus resource [bus 00-ff]

pci 0000:00:00.0: [16c3:abcd] type 01 class 0x060400

pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]

pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]

pci 0000:00:00.0: supports D1

pci 0000:00:00.0: PME# supported from D0 D1 D3hot D3cold

PCI: bus0: Fast back to back transfers disabled

pci 0000:01:00.0: [168c:0030] type 00 class 0x028000

pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0001ffff 64bit]

pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]

pci 0000:01:00.0: supports D1

pci 0000:01:00.0: PME# supported from D0 D1 D3hot

PCI: bus1: Fast back to back transfers disabled

pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01

pci 0000:00:00.0: BAR 0: assigned [mem 0x01000000-0x010fffff]

pci 0000:00:00.0: BAR 8: assigned [mem 0x01100000-0x011fffff]

pci 0000:00:00.0: BAR 9: assigned [mem 0x01200000-0x012fffff pref]

pci 0000:00:00.0: BAR 6: assigned [mem 0x01300000-0x0130ffff pref]

pci 0000:01:00.0: BAR 0: assigned [mem 0x01100000-0x0111ffff 64bit]

pci 0000:01:00.0: BAR 6: assigned [mem 0x01200000-0x0120ffff pref]

pci 0000:00:00.0: PCI bridge to [bus 01]

pci 0000:00:00.0:   bridge window [mem 0x01100000-0x011fffff]

pci 0000:00:00.0:   bridge window [mem 0x01200000-0x012fffff pref]

pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt

pci 0000:01:00.0: Signaling PME through PCIe PME interrupt

pcie_pme 0000:00:00.0:pcie01: service driver pcie_pme loaded

aer 0000:00:00.0:pcie02: service driver aer loaded

However, here is what happens when the kernel seems to try and load the driver :

PCI: enabling device 0000:01:00.0 (0140 -> 0142)

------------[ cut here ]------------

WARNING: CPU: 0 PID: 1 at kernel/irq/manage.c:1412 request_threaded_irq+0xfc/0x124()

Modules linked in:

CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.14.28-1.0.0_ga+g91cf351 #5

[<80015638>] (unwind_backtrace) from [<80011714>] (show_stack+0x10/0x14)

[<80011714>] (show_stack) from [<8073be5c>] (dump_stack+0x7c/0xbc)

[<8073be5c>] (dump_stack) from [<8002d718>] (warn_slowpath_common+0x70/0x8c)

[<8002d718>] (warn_slowpath_common) from [<8002d7d0>] (warn_slowpath_null+0x1c/0x24)

[<8002d7d0>] (warn_slowpath_null) from [<80069ce8>] (request_threaded_irq+0xfc/0x124)

[<80069ce8>] (request_threaded_irq) from [<80420764>] (ath_pci_probe+0x198/0x2e4)

[<80420764>] (ath_pci_probe) from [<802bfa9c>] (pci_device_probe+0x74/0xc8)

[<802bfa9c>] (pci_device_probe) from [<80352a2c>] (driver_probe_device+0x110/0x24c)

[<80352a2c>] (driver_probe_device) from [<80352c38>] (__driver_attach+0x8c/0x90)

[<80352c38>] (__driver_attach) from [<80350f84>] (bus_for_each_dev+0x6c/0xa0)

[<80350f84>] (bus_for_each_dev) from [<803521e4>] (bus_add_driver+0x148/0x1f0)

[<803521e4>] (bus_add_driver) from [<80353234>] (driver_register+0x78/0xf8)

[<80353234>] (driver_register) from [<80e016d0>] (ath9k_init+0x2c/0x68)

[<80e016d0>] (ath9k_init) from [<8000889c>] (do_one_initcall+0xf8/0x154)

[<8000889c>] (do_one_initcall) from [<80dd0c54>] (kernel_init_freeable+0x138/0x1d8)

[<80dd0c54>] (kernel_init_freeable) from [<80737548>] (kernel_init+0x8/0xe8)

[<80737548>] (kernel_init) from [<8000e538>] (ret_from_fork+0x14/0x3c)

---[ end trace 6a544c6d7fd3de66 ]---

ath9k 0000:01:00.0: request_irq failed

ath9k: probe of 0000:01:00.0 failed with error -22

Here is what is enabled in the kernel configuration:

Bus Support --->

     <*> PCI support

     PCI host controller drivers --->

          Freescale i.MX6 PCIe controller

     -*- PCI Express Port Bus Support

     [*] PCI Express ASPM control

Networking support --->

     -*- Wireless --->

          <*> cfg 80211 - wireless configuration API

          [*] enable powersave by default

          <*> Generic IEEE 802.11 Networking Stack (mac80211)

          -*- Enable LED triggers

          -*- Export mac80211 interals in DebugFS

Device Drivers --->

     [*] Network device support --->

          [*] Wireless LAN --->

               [*] Atheros Wireless Cards --->

                    [*] Atheros bluetooth coexistence support

                    <*> Atheros 802.11n wireless cards support

                    [*] Atheros ath9k PCI/PCIe bus support

                    [*] Atheros ath9k AHB bus support

                    [*] Atheros ath9k debugging

                    [*]   Detailed station statistics

                    [*] Atheros ath9k rate control

I am currently trying to go through the back trace and see where exactly it is failing.

[madisox]

Christophe,

As I've mentioned over in this thread: Re: Has anyone verified PCI driver on yocto 3.14.28-1.0.1 branch?

there are two modifications I had to make to the 3.14.28 kernel to get my PCIe device (also an Atheros wi-fi card) to work on my board:

- modify the device tree to fix the broken definition supplied by Freescale for the PCI interrupt controller

- disable MSI in the kernel configuration

If you haven't already done so, give those changes a try.

[christophebonni]

Hello Igor, Madisox,

Thank you both for your answers.

Madisox, I had read this post and thought it was not applicable to my issue since I have no log saying fixup IRQ...

I still tried changing the dtsi file and removing MSI from kernel configuration and here is the log I got:

PCI: enabling device 0000:01:00.0 (0140 -> 0142)

------------[ cut here ]------------

WARNING: CPU: 0 PID: 1 at kernel/irq/manage.c:1412 request_threaded_irq+0xfc/0x124()

Modules linked in:

CPU: 0 PID: 1 Comm: swapper/0 Tainted: G        W    3.14.28-1.0.0_ga+g91cf351 #2

[<80015638>] (unwind_backtrace) from [<80011714>] (show_stack+0x10/0x14)

[<80011714>] (show_stack) from [<807396d8>] (dump_stack+0x7c/0xbc)

[<807396d8>] (dump_stack) from [<8002d718>] (warn_slowpath_common+0x70/0x8c)

[<8002d718>] (warn_slowpath_common) from [<8002d7d0>] (warn_slowpath_null+0x1c/0x24)

[<8002d7d0>] (warn_slowpath_null) from [<80069ce8>] (request_threaded_irq+0xfc/0x124)

[<80069ce8>] (request_threaded_irq) from [<8041e108>] (ath_pci_probe+0x198/0x2e4)

[<8041e108>] (ath_pci_probe) from [<802bfa48>] (pci_device_probe+0x74/0xc8)

[<802bfa48>] (pci_device_probe) from [<803503d0>] (driver_probe_device+0x110/0x24c)

[<803503d0>] (driver_probe_device) from [<803505dc>] (__driver_attach+0x8c/0x90)

[<803505dc>] (__driver_attach) from [<8034e928>] (bus_for_each_dev+0x6c/0xa0)

[<8034e928>] (bus_for_each_dev) from [<8034fb88>] (bus_add_driver+0x148/0x1f0)

[<8034fb88>] (bus_add_driver) from [<80350bd8>] (driver_register+0x78/0xf8)

[<80350bd8>] (driver_register) from [<80dfc55c>] (ath9k_init+0x2c/0x68)

[<80dfc55c>] (ath9k_init) from [<8000889c>] (do_one_initcall+0xf8/0x154)

[<8000889c>] (do_one_initcall) from [<80dcbc54>] (kernel_init_freeable+0x138/0x1d8)

[<80dcbc54>] (kernel_init_freeable) from [<80734dc4>] (kernel_init+0x8/0xe8)

[<80734dc4>] (kernel_init) from [<8000e538>] (ret_from_fork+0x14/0x3c)

---[ end trace 0a639a22a670d711 ]---

ath9k 0000:01:00.0: request_irq failed

ath9k: probe of 0000:01:00.0 failed with error -22

Igor, I am looking for correct drivers and will let you know if anything changes.

Thank you both.

Regards,

[shivakumar]

Hi Christophe Bonniot,

Even I am facing this issue. I am also facing this crash while booting up

PCI: enabling device 0000:01:00.0 (0140 -> 0142)

------------[ cut here ]------------

WARNING: CPU: 0 PID: 1 at kernel/irq/manage.c:1412 request_threaded_irq+0xfc/0x124()

You got any solution on this ? 

-Regards,

Shiva Kumar M S

[igorpadykov]

HI Christophe

so this card is detected by linux, correct ?

"Network controller: Qualcomm Atheros AR93xx Wireless Network Adapter"

I think one can ask atheros for most updated AR93xx driver for 3.14 kernel.

~igor

[christophebonni]

Hi Igor,

I followed this User Guide and managed to use the modified custom kernel configuration file.

Please see my last post to see the output.

The issue seems to be reproduced with this BSP.

Thank you.

[christophebonni]

Hi Igor,

I'm trying with Yocto and the Linux version you mentioned, it is taking some time to set up the environment.

I'll let you know once it's done.

Thank you.