PCIe hotplug support

TrinathK · ‎01-27-2022

Hi,

Does ls1043a or ls1046a has support for pcie hotplug ? If yes, let me know how to use the hotplug feature.

I have already tried using below commands

echo 1 > /sys/bus/pci/devices/<device id>/remove

echo 1 > /sys/bus/pci/rescan

And i am able to find pci devices properly but memory allocation for BARs is not happening on rescan.

Thanks

Trinath K

yipingwang · ‎01-27-2022

1. Suppose the PCIe device under /sys/bus/pci/devices/0001:01:00.0 is the Intel PCIe e1000e network card, recognized as eth6. The /sys/bus/pci/devices/0001:00:00.0 is the bus of network card. Configure an ip and ping another host which is in the same subnet, make sure the network card works well.
# ls /sys/bus/pci/devices/0001:01:00.0/net
eth6
# ifconfig eth6 192.168.2.36
# ping 192.168.2.1 -I eth6
PING 192.168.2.1 (192.168.2.1): 56 data bytes
64 bytes from 192.168.2.1: seq=0 ttl=64 time=0.454 ms
64 bytes from 192.168.2.1: seq=1 ttl=64 time=0.251 ms
64 bytes from 192.168.2.1: seq=2 ttl=64 time=0.272 ms

2. Remove the PCIe network card from system.
# echo 1 > /sys/bus/pci/devices/0001:01:00.0/remove
[ 422.446123] e1000e 0001:01:00.0 eth6: NIC Link is Down

3. Check whether the PCIe network card still exists in system. All should fail.
# ifconfig eth6
eth6: error fetching interface information: Device not found
# ls /sys/bus/pci/devices/0001:01:00.0
ls: cannot access '/sys/bus/pci/devices/0001:01:00.0': No such file or directory

4. Rescan it from the bus.
# echo 1 > /sys/bus/pci/devices/0001:00:00.0/rescan
[ 665.279319] pci 0001:01:00.0: [8086:107d] type 00 class 0x020000
[ 665.285425] pci 0001:01:00.0: reg 0x10: [mem 0x4840000000-0x484001ffff]
[ 665.292074] pci 0001:01:00.0: reg 0x14: [mem 0x4840020000-0x484003ffff]
[ 665.298725] pci 0001:01:00.0: reg 0x18: [io 0x10000-0x1001f]
[ 665.304591] pci 0001:01:00.0: reg 0x30: [mem 0x4840040000-0x484005ffff pref]
[ 665.311841] pci 0001:01:00.0: PME# supported from D0 D3hot D3cold
[ 665.318167] pci 0001:01:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force'
[ 665.329029] pci 0001:01:00.0: BAR 0: assigned [mem 0x4840000000-0x484001ffff]
[ 665.336176] pci 0001:01:00.0: BAR 1: assigned [mem 0x4840020000-0x484003ffff]
[ 665.343329] pci 0001:01:00.0: BAR 6: assigned [mem 0x4840040000-0x484005ffff pref]
[ 665.350904] pci 0001:01:00.0: BAR 2: assigned [io 0x10000-0x1001f]
[ 665.357464] e1000e 0001:01:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
[ 665.526288] e1000e 0001:01:00.0 eth0: (PCI Express:2.5GT/s:Width x1) 00:15:17:c6:48:a2
[ 665.534226] e1000e 0001:01:00.0 eth0: Intel(R) PRO/1000 Network Connection
[ 665.541189] e1000e 0001:01:00.0 eth0: MAC: 1, PHY: 4, PBA No: E55239-001

5. Check whether the device is rescanned and works well.
# ls /sys/bus/pci/devices/0001:01:00.0/net

eth0

# ifconfig eth0 192.168.2.36
# # ping 192.168.2.1 -I eth0
PING 192.168.2.1 (192.168.2.1): 56 data bytes
64 bytes from 192.168.2.1: seq=0 ttl=64 time=0.470 ms
64 bytes from 192.168.2.1: seq=1 ttl=64 time=0.284 ms
64 bytes from 192.168.2.1: seq=2 ttl=64 time=0.297 ms

TrinathK · ‎01-27-2022

Hi Yiping Wang,

Thanks for the reply. I have established PCIe communication between ls1043ardb (layerscape board as Root complex) and Beagleboard x15 (as endpoint). I enabled PCIe hotplug in kernel configuration of ls1043ardb.

And i have already used "remove" and "rescan" and everything is working fine. I can able to detect my endpoint after rescan. but sometimes BARs are being unassigned.

Firstly, how may i know if a board support hotplug feature or not. In reference manual of ls1043 and ls1046, nothing is mentioned about pcie hotplug.

Also as per my understanding, hardware support is also required for hotplug feature to work. please explain on it.

Thanks in advance.

yipingwang · ‎01-27-2022

Would you please create a new thread to ask the hardware team to explain it.

TrinathK · ‎01-28-2022

Ok