AnsweredAssumed Answered

MSI Still Broken on NXP BSP L4.14.98_2.3.1_patch?

Question asked by Marcel Ziswiler on Mar 26, 2020

Since a while, we carry the pci=nomsi kernel boot command-line argument in our defaults. After our recent re-base on-top-of NXP BSP L4.14.98_2.3.1_patch we did some more PCIe tests but it looks like enabling MSI (e.g. leaving that pci=nomsi away) together with the PCIe switch on our Apalis Evaluation board still fails on the i.MX 8QuadMax B0 silicon chips:

[ 7.634640] imx6q-pcie 5f000000.pcie: 5f000000.pcie supply epdev_on not found, using dummy regulator
[ 7.653008] OF: PCI: host bridge /pcie@0x5f000000 ranges:
[ 7.660167] OF: PCI: IO 0x6ff80000..0x6ff8ffff -> 0x00000000
[ 7.667846] OF: PCI: MEM 0x60000000..0x6fefffff -> 0x60000000
[ 7.782667] imx6q-pcie 5f000000.pcie: pcie phy pll is locked.
[ 7.924853] imx6q-pcie 5f000000.pcie: Link: Gen2 disabled
[ 7.933635] imx6q-pcie 5f000000.pcie: Link up, Gen2
[ 7.943544] imx6q-pcie 5f000000.pcie: PCI host bridge to bus 0000:00
[ 7.953249] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 7.962069] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 7.971528] pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fefffff]
[ 8.064416] pci 0000:01:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 8.080064] pci 0000:02:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 8.091303] pci 0000:02:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 8.102556] pci 0000:02:03.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 8.114902] pci 0000:04:00.0: enabling Extended Tags
[ 8.132732] pci 0000:00:00.0: bridge configuration invalid ([bus 00-ff]), reconfiguring
[ 8.143227] pci 0000:00:00.0: BAR 0: assigned [mem 0x60000000-0x60ffffff 64bit]
[ 8.152326] pci 0000:00:00.0: BAR 6: assigned [mem 0x61000000-0x61ffffff pref]
[ 8.161305] pci 0000:00:00.0: BAR 14: assigned [mem 0x62000000-0x623fffff]
[ 8.169922] pci 0000:00:00.0: BAR 15: assigned [mem 0x62400000-0x625fffff 64bit pref]
[ 8.179511] pci 0000:00:00.0: BAR 13: assigned [io 0x1000-0x1fff]
[ 8.187445] pci 0000:01:00.0: BAR 14: assigned [mem 0x62000000-0x622fffff]
[ 8.196068] pci 0000:01:00.0: BAR 15: assigned [mem 0x62400000-0x625fffff 64bit pref]
[ 8.205636] pci 0000:01:00.0: BAR 0: assigned [mem 0x62300000-0x62303fff]
[ 8.214155] pci 0000:01:00.0: BAR 13: assigned [io 0x1000-0x1fff]
[ 8.222057] pci 0000:02:01.0: BAR 14: assigned [mem 0x62000000-0x621fffff]
[ 8.230643] pci 0000:02:01.0: BAR 15: assigned [mem 0x62400000-0x625fffff 64bit pref]
[ 8.240174] pci 0000:02:02.0: BAR 14: assigned [mem 0x62200000-0x622fffff]
[ 8.248717] pci 0000:02:01.0: BAR 13: assigned [io 0x1000-0x1fff]
[ 8.256558] pci 0000:02:01.0: PCI bridge to [bus 03]
[ 8.263189] pci 0000:02:01.0: bridge window [io 0x1000-0x1fff]
[ 8.270974] pci 0000:02:01.0: bridge window [mem 0x62000000-0x621fffff]
[ 8.279456] pci 0000:02:01.0: bridge window [mem 0x62400000-0x625fffff 64bit pref]
[ 8.288904] pci 0000:04:00.0: BAR 0: assigned [mem 0x62200000-0x62203fff 64bit]
[ 8.297930] pci 0000:02:02.0: PCI bridge to [bus 04]
[ 8.304572] pci 0000:02:02.0: bridge window [mem 0x62200000-0x622fffff]
[ 8.313070] pci 0000:02:03.0: PCI bridge to [bus 05]
[ 8.319748] pci 0000:01:00.0: PCI bridge to [bus 02-05]
[ 8.326611] pci 0000:01:00.0: bridge window [io 0x1000-0x1fff]
[ 8.334369] pci 0000:01:00.0: bridge window [mem 0x62000000-0x622fffff]
[ 8.342780] pci 0000:01:00.0: bridge window [mem 0x62400000-0x625fffff 64bit pref]
[ 8.352185] pci 0000:00:00.0: PCI bridge to [bus 01-05]
[ 8.358998] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff]
[ 8.366649] pci 0000:00:00.0: bridge window [mem 0x62000000-0x623fffff]
[ 8.374954] pci 0000:00:00.0: bridge window [mem 0x62400000-0x625fffff 64bit pref]
[ 8.385018] pcieport 0000:00:00.0: enabling device (0000 -> 0003)
[ 8.392965] pcieport 0000:00:00.0: Signaling PME with IRQ 454
[ 8.400450] pcieport 0000:00:00.0: AER enabled with IRQ 455
[ 8.408135] pcieport 0000:01:00.0: enabling device (0000 -> 0003)
[ 8.417018] pcieport 0000:02:01.0: enabling device (0000 -> 0003)
[ 8.432548] pciehp 0000:02:01.0:pcie204: Slot #1 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise- Interlock- NoCompl- LLActRep+
[ 8.446701] Unable to handle kernel NULL pointer dereference at virtual address 0000003c
[ 8.456496] Mem abort info:
[ 8.460976] Exception class = DABT (current EL), IL = 32 bits
[ 8.468595] SET = 0, FnV = 0
[ 8.473353] EA = 0, S1PTW = 0
[ 8.478202] Data abort info:
[ 8.482811] ISV = 0, ISS = 0x00000004
[ 8.488352] CM = 0, WnR = 0
[ 8.492992] [000000000000003c] user address but active_mm is swapper
[ 8.501053] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[ 8.508361] Modules linked in:
[ 8.513128] Process kworker/1:3 (pid: 1974, stack limit = 0xffff000012668000)
[ 8.522062] CPU: 1 PID: 1974 Comm: kworker/1:3 Not tainted 4.14.170-07228-g3a6cd4e88715 #18
[ 8.532249] Hardware name: Toradex Apalis iMX8QM/QP on Apalis Evaluation Board (DT)
[ 8.541797] Workqueue: events deferred_probe_work_func
[ 8.548828] task: ffff8008794b3600 task.stack: ffff000012668000
[ 8.556653] PC is at msi_set_mask_bit.isra.8+0x10/0xb8
[ 8.563700] LR is at pci_msi_unmask_irq+0x1c/0x28
[ 8.570307] pc : [<ffff0000084efb08>] lr : [<ffff0000084efbf4>] pstate: 400000c5
[ 8.579675] sp : ffff00001266b450
[ 8.584950] x29: ffff00001266b450 x28: ffff800874180800
[ 8.592251] x27: 0000000000000040 x26: ffff0000092d08d8
[ 8.599540] x25: ffff80087412caa4 x24: ffff80087412cb58
[ 8.606821] x23: 0000000000000000 x22: 0000000000000001
[ 8.614113] x21: ffff80087412ca28 x20: ffff80087412ca00
[ 8.621417] x19: 0000000000000000 x18: 00000000fffffffb
[ 8.628724] x17: 00000000139eb36a x16: 0000000000000000
[ 8.636005] x15: 0000000000000000 x14: ffffffffffffffff
[ 8.643274] x13: ffff80087417c91c x12: 0000000000000000
[ 8.650536] x11: 0000000000000000 x10: 0000000000000040
[ 8.657808] x9 : 0000000000000000 x8 : ffff800878cc3100
[ 8.665069] x7 : 0000000000000000 x6 : 000000000000003f
[ 8.672328] x5 : 0000000000000040 x4 : 0000000000000000
[ 8.679578] x3 : 0000000000000040 x2 : 0000000000000000
[ 8.686786] x1 : ffff80087412ca00 x0 : 00000000000001ca
[ 8.693989] Call trace:
[ 8.698304] Exception stack(0xffff00001266b310 to 0xffff00001266b450)
[ 8.706690] b300: 00000000000001ca ffff80087412ca00
[ 8.716483] b320: 0000000000000000 0000000000000040 0000000000000000 0000000000000040
[ 8.726301] b340: 000000000000003f 0000000000000000 ffff800878cc3100 0000000000000000
[ 8.736112] b360: 0000000000000040 0000000000000000 0000000000000000 ffff80087417c91c
[ 8.745919] b380: ffffffffffffffff 0000000000000000 0000000000000000 00000000139eb36a
[ 8.755715] b3a0: 00000000fffffffb 0000000000000000 ffff80087412ca00 ffff80087412ca28
[ 8.765499] b3c0: 0000000000000001 0000000000000000 ffff80087412cb58 ffff80087412caa4
[ 8.775203] b3e0: ffff0000092d08d8 0000000000000040 ffff800874180800 ffff00001266b450
[ 8.784837] b400: ffff0000084efbf4 ffff00001266b450 ffff0000084efb08 00000000400000c5
[ 8.794448] b420: ffff00000907e680 0000000000000000 ffffffffffffffff 0000000000000000
[ 8.804051] b440: ffff00001266b450 ffff0000084efb08
[ 8.810675] [<ffff0000084efb08>] msi_set_mask_bit.isra.8+0x10/0xb8
[ 8.818651] [<ffff0000084efbf4>] pci_msi_unmask_irq+0x1c/0x28
[ 8.826200] [<ffff00000811d408>] irq_enable+0x38/0x70
[ 8.833050] [<ffff00000811d4b4>] __irq_startup+0x74/0x98
[ 8.840160] [<ffff00000811d53c>] irq_startup+0x64/0x120
[ 8.847167] [<ffff00000811b744>] __setup_irq+0x734/0x778
[ 8.854229] [<ffff00000811b908>] request_threaded_irq+0xd8/0x188
[ 8.861988] [<ffff0000084ec380>] pcie_init_notification+0x48/0xd8
[ 8.869813] [<ffff0000084ea0a0>] pciehp_probe+0x170/0x338
[ 8.876902] [<ffff0000084e5d04>] pcie_port_probe_service+0x34/0x60
[ 8.884745] [<ffff0000086dc6bc>] driver_probe_device+0x20c/0x2c8
[ 8.892388] [<ffff0000086dc8f0>] __device_attach_driver+0xb8/0xe8
[ 8.900111] [<ffff0000086da9c8>] bus_for_each_drv+0x68/0xa8
[ 8.907294] [<ffff0000086dc390>] __device_attach+0xc0/0x130
[ 8.914477] [<ffff0000086dc970>] device_initial_probe+0x10/0x18
[ 8.921978] [<ffff0000086db8a0>] bus_probe_device+0x90/0x98
[ 8.929112] [<ffff0000086d98d0>] device_add+0x328/0x5b0
[ 8.935893] [<ffff0000086d9b74>] device_register+0x1c/0x28
[ 8.942939] [<ffff0000084e6000>] pcie_port_device_register+0x228/0x490
[ 8.951040] [<ffff0000084e6504>] pcie_portdrv_probe+0x34/0xc0
[ 8.958351] [<ffff0000084d77bc>] local_pci_probe+0x3c/0xb0
[ 8.965402] [<ffff0000084d84bc>] pci_device_probe+0xe4/0x188
[ 8.972617] [<ffff0000086dc6bc>] driver_probe_device+0x20c/0x2c8
[ 8.980201] [<ffff0000086dc8f0>] __device_attach_driver+0xb8/0xe8
[ 8.987881] [<ffff0000086da9c8>] bus_for_each_drv+0x68/0xa8
[ 8.995031] [<ffff0000086dc390>] __device_attach+0xc0/0x130
[ 9.002188] [<ffff0000086dc410>] device_attach+0x10/0x18
[ 9.009095] [<ffff0000084cccac>] pci_bus_add_device+0x4c/0xa8
[ 9.016452] [<ffff0000084ccd40>] pci_bus_add_devices+0x38/0x88
[ 9.023903] [<ffff0000084ccd70>] pci_bus_add_devices+0x68/0x88
[ 9.031350] [<ffff0000084ccd70>] pci_bus_add_devices+0x68/0x88
[ 9.038780] [<ffff0000084fc63c>] dw_pcie_host_init+0x2c4/0x4f0
[ 9.046207] [<ffff0000084fe2b4>] imx_pcie_probe+0x60c/0xa08
[ 9.053354] [<ffff0000086de268>] platform_drv_probe+0x58/0xb8
[ 9.060641] [<ffff0000086dc6bc>] driver_probe_device+0x20c/0x2c8
[ 9.068180] [<ffff0000086dc8f0>] __device_attach_driver+0xb8/0xe8
[ 9.075818] [<ffff0000086da9c8>] bus_for_each_drv+0x68/0xa8
[ 9.082932] [<ffff0000086dc390>] __device_attach+0xc0/0x130
[ 9.090043] [<ffff0000086dc970>] device_initial_probe+0x10/0x18
[ 9.097496] [<ffff0000086db8a0>] bus_probe_device+0x90/0x98
[ 9.104610] [<ffff0000086dc09c>] deferred_probe_work_func+0xa4/0x148
[ 9.112523] [<ffff0000080e69b0>] process_one_work+0x1c8/0x328
[ 9.119816] [<ffff0000080e6d50>] worker_thread+0x240/0x450
[ 9.126844] [<ffff0000080ecae0>] kthread+0x128/0x130
[ 9.133319] [<ffff000008084e98>] ret_from_fork+0x10/0x18
[ 9.140128] Code: a9be7bfd 910003fd f9000bf3 f9400833 (3940f261)
[ 9.147734] ---[ end trace fd7940b8b69853f2 ]---
[ 9.153866] note: kworker/1:3[1974] exited with preempt_count 1

Could NXP please confirm whether or not MSI indeed should work? Have you guys ever tried this with a PCIe switch setup similar to what we have on our EvalBoard?

 

For reference, I attached full log files both on our EvalBoard with and without MSI as well as on Ixora (which is another carrier board of ours however without any PCIe switch) with MSI.

Outcomes