LS1027/LS1028 PCIe hotplug support not working

[mikelooijmans]

We want to use a CF express card in combination with an LS1027 (same as LS1028, just without GPU).

This is basically an NVME device that can be hot-plugged.

I'm experimenting with the LS1208ARDB board, but cannot get PCIe to "find" anything after boot. I'm running a 6.6 kernel with PCIe hotplug enabled in the kernel configuration.

If the NVME device is present at (very early) boot, it works fine.

The behavior when plugging it in later is that it doesn't work, even when forcing a PCIe bus scan using "echo 1 >/sys/bus/pci/rescan".

The behavior we see is that quickly after boot, the system turns off the PCIe reference clock to the device, and never turns it on again.

According to the datasheet, the PCIe core supports hot plugging, so what did we miss?

[LFGP]

dear @mikelooijmans ,

I have double checked with the internal team and in the reference manual, and unfortunately the chip doesn't support the Hot-plug capability, there is a  typo error in the reference manual ( 1.12.1 PCI Express ).

The right note is in the section ""25.4.1.60 PCI Express Slot Capabilities Register""

field                 description

HPD                Hot-Plug Capable
                         Note: This chip does not support hot-plug capabilities

[LFGP]

You need to check the Kernel Configuration related to PCIe bus Hot-Plug.

[mikelooijmans]

Yes, as I mentioned, I enabled PCIe hotplug support in the kernel.

The behavior is that if there's no device present on the PCIe port on power-on, nothing will ever be detected on the PCIe port, no matter what, only a reboot will make it appear.

If there is a device there at power on or reboot, it is possible to remove it and add it back and have it detected. Though usually it doesn't actually work, but that can be device (driver) related.

Our analysis shows that the PCIe reference clock is turned off when there's nothing present at boot. And there's apparently no way to turn that clock back on.