PCIe rescan in Linux

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

PCIe rescan in Linux

Jump to solution
27,677 Views
jeffreycoffman
Contributor III

Hello,

We are developing an FPGA device, which is going to be connected to an i.MX6 as a PCIe endpoint device.  I'd like to configure the FPGA once the system is up and running, and of course at this point the kernel has already probed the PCIe bus and hasn't see anything yet.  I'd like to be able to rescan the PCIe bus once the device is configured and the PCIe core is running.

I've done some searching and seen some reference to doing an echo 1 > /sys/bus/pci/rescan.  This doesn't really have any effect and I am told you need hotplug support for this to work.  So I am wondering the following:

1.  Does 3.0.35 Linux kernel support hotplug for the i.MX6?  I don't see it as option in the menuconfig.

2.  Does my device need to support hotplug, I assume it has to assert PERST#.

3.  My device isn't going to be hot swappable, I just need to delay the Kernels initial probing of the PCIe bus until the FPGA is configured.  Is this possible todo without hotplug, or are pretty much stuck needing the hotplug service?

If someone out there can help explain this in more detail, that would be great.

Thanks

1 Solution
8,872 Views
kitz36
Contributor III

That is correct.  I added the re-initialization code to my arch/arm/mach-mx6/pcie.c module.  Actually, my pcie.c module has quite a few updates versus the 1.1.0 BSP.  See attached.  Let me know if you have questions.

-Charlie

View solution in original post

0 Kudos
9 Replies
8,872 Views
randysilagi
Contributor II

Hi Jeffery,

Back in 2013 you were having a problem with PICe rescan.  I'm wondering how you loaded the FPGA.  Did you use one of the SPI channels or GPIO pins?  I'm working on a new product based on the i.MX6sx and we are planning on the loading the FPGA using ecspi0 and then doing the rescan.

Randy S.

0 Kudos
8,872 Views
jeffrey_coffman
Contributor II

Hi,

We load the FPGA using ecspi2.  We never ended needing to use rescan, for our application.

0 Kudos
8,872 Views
hendriklugtenbu
Contributor I

Hi,

These forum posts are all more then a year old. I am wondering if in newer Linux kernels these problems are solved. I found these commits to the kernel:

https://github.com/torvalds/linux/commits/master/drivers/pci/host/pci-imx6.c

History for drivers/pci/host - torvalds/linux · GitHub

I could not find literally that these commits resolve the rescan issues. Has anyone experience with this?

Thanks!

-Hendrik

0 Kudos
8,872 Views
richard_zhu
NXP Employee
NXP Employee

As I known that the hot-plug is not supported by imx6 pcie.

0 Kudos
8,872 Views
jeffreycoffman
Contributor III

Thanks again!  I will give this a try on my system.

Jeff

0 Kudos
8,872 Views
kitz36
Contributor III

I had a similar issue.  We wanted to reload an FPGA, which is our PCIe endpoint.  This would effectively take the link down.  Bringing the link back up turned out to be a bit of a problem.  The solution was two-fold.

First, I sort of had to hack the Linux kernel to force a true "rescan".  Originally the kernel wouldn't truly probe the endpoint on a rescan.  I ended up forcing the kernel to "forget" the bus by adding a call to "pci_remove_bus()" in drivers/pci/probe.c :: pci_rescan_bus().  See attached.

Second, I had to do another hack to re-initialize the i.MX6 PCIe hardware.  I described that in an earlier post

-Charlie

8,872 Views
jeffreycoffman
Contributor III

Charles,

Thanks for providing your solution.  I was hoping there was a less invasive way of doing this via the hot-plug mechanism, but I still haven't figured out whats involved with that approach.

I assume you made the changes to the re-initialize the i.MX6 PCIe hardware in imx_pcie_pltfm_reinit?

Thanks Again,

Jeff

0 Kudos
8,873 Views
kitz36
Contributor III

That is correct.  I added the re-initialization code to my arch/arm/mach-mx6/pcie.c module.  Actually, my pcie.c module has quite a few updates versus the 1.1.0 BSP.  See attached.  Let me know if you have questions.

-Charlie

0 Kudos
8,872 Views
gavinli
Contributor I

We found out it use different way to configure PCIe: It configures RC to use clock from EP

As we use RC clock to drive EP, so I disabled the PHY_MPLL override code in pcie.c

#define ZEP_DEBUG_DONT_INIT_PCI 1
#undef ZEP_DEBUG_DONT_INIT_PCI

#define ZEP_PCIE_OVERRIDE_PHY_MPLL 1
#undef ZEP_PCIE_OVERRIDE_PHY_MPLL     /* !!!!! */

#define IMX_PCIE_CORE_DEBUG
#undef  IMX_PCIE_CORE_DEBUG

#ifdef  IMX_PCIE_CORE_DEBUG
#define IMX_PCIE_PL_DEBUG_BUFFER_SZ (1024)

I did “echo 1 > /sys/bus/pci/devices/0000\:01\:00.0/remove” ,  then I recycled the EP power, and did “echo 1 > /sys/bus/pci/rescan”
The result is Linux kernel crash:

Unhandled fault: imprecise external abort (0x1406) at 0x000c4000
Internal error: : 1406 [#1] PREEMPT SMP
Modules linked in:
CPU: 0    Not tainted  (3.0.35-2508-g54750ff #77)

It also has lots of changed in the pcie.c comparing to my 4.0.0 release.

Would you please help us find if there is a solution for the 4.0.0 release?


Thanks.

0 Kudos