topic Re: IMX8MM kernel hang while 3 SPIs are enabled in i.MX Processors

IMX8MM kernel hang while 3 SPIs are enabled

mahesh_rd — Fri, 09 Oct 2020 07:35:57 GMT

Hi,

We are working on a custom board based on the i.MX 8M Mini Processor[MIMX8MM3CVTKZAA]. We require the use of 3 SPIs for certain functionality. When we tried to enable SPI1, SPI2 and SPI3 one by one, we noticed that the kernel hangs midway when the 3rd SPI is enabled.

On further investigation, we noticed that there is no problem with the kernel when I enable just 2 SPIs (ie, SPI1-SPI2, SPI1-SPI3, SPI2-SPI3), but I cannot have 3 SPIs simultaneously, since the kernel always hangs(kernel_spi123_hang.txt). I have included the kernel log for the hang when 3 SPI's are enabled, I cant seem to find the root cause of this problem

I have also included the log when only 2 SPIs are enabled (kernel_spi12_ok.txt)

Re: IMX8MM kernel hang while 3 SPIs are enabled

igorpadykov — Fri, 09 Oct 2020 10:18:38 GMT

Hi mahesh_rd

one can recheck what " SPI" are really used in the case: as there is

FlexSPI module with pin names as "QSPI":

pinctrl_flexspi0: flexspi0grp {
fsl,pins = <MX8MM_IOMUXC_NAND_ALE_QSPI_A_SCLK 0x1c2

https://source.codeaurora.org/external/imx/linux-imx/tree/arch/arm64/boot/dts/freescale/imx8mm-evk.dts?h=imx_5.4.47_2.2.0

and 3 eCSPI modules, seems by default they are disabled in imx8mm-evk.dts

eCSPI1 is enabled in imx8mm-ddr3l-val.dts

https://source.codeaurora.org/external/imx/linux-imx/tree/arch/arm64/boot/dts/freescale/imx8mm-ddr3l-val.dts?h=imx_5.4.47_2.2.0

Best regards
igor

Re: IMX8MM kernel hang while 3 SPIs are enabled

mahesh_rd — Fri, 09 Oct 2020 17:51:30 GMT

Hi igor,

Thanks for the info. But I have tried disabling flexspi(status=disabled) and commenting out iomux definitions in pinmux. This issue still persists even after that.

I have no problem in enabling 2 SPI at a time, it can be ECSPI1 and ECSPI3, ECSPI1 and ECSPI2, ECSCI1 and ECSPI2. I can see these nodes listed in Linux /dev as "spidev0.0/1.0/2.0"

But whenever I try to enable all 3 nodes in device tree files, I get kernel hang. I have attached the log in my post. I can't seem to find out the problem.

Re: IMX8MM kernel hang while 3 SPIs are enabled

igorpadykov — Sat, 10 Oct 2020 05:19:50 GMT

Hi mahesh_rd

in attached log actually there are no any "ecspi" entries, only flex-spi.

Suggest to rebuild all image from scratch and recheck if ecspi pins were not

used by other peripherals.

Best regards
igor

Re: IMX8MM kernel hang while 3 SPIs are enabled

mahesh_rd — Sat, 10 Oct 2020 11:08:27 GMT

Hi, igor

In the log attached (kernel_spi123_hang.txt), they are listed,I have attached the images for reference

It gets stuck after this message (CAN driver interface). I dont have any problem with 2 SPI devices at a time as shown in (kernel_spi12_ok.txt)

Re: IMX8MM kernel hang while 3 SPIs are enabled

igorpadykov — Sun, 11 Oct 2020 12:53:09 GMT

Hi mahesh_rd

>It gets stuck after this message (CAN driver interface).

i.MX8M Mini has not CAN interface.

Below comparison table describing all available interfaces in i.MX8 procesors:

https://www.nxp.com/docs/en/supporting-information/IMX8LAYERCMPR.pdf

Best regards
igor

Re: IMX8MM kernel hang while 3 SPIs are enabled

mahesh_rd — Mon, 12 Oct 2020 08:22:07 GMT

Hi Igor,

I followed your advice and disabled CAN interface. Now I get the following message after 3 ECSPIs are probed. I have attached the log. Is this maybe a issue of LPDDR RAM?

[ 2.431801] spi_imx 30840000.ecspi: probed [ 2.444707] libphy: Fixed MDIO Bus: probed [ 2.450342] NXP PHY: loading NXP PHY driver: [autonomous mode] [ 2.456880] tun: Universal TUN/TAP device driver, 1.6 [ 2.472215] Unable to handle kernel read from unreadable memory at virtual address ffff0000096e4000 [ 2.472226] Unable to handle kernel NULL pointer dereference at virtual address 0000002d [ 2.472241] Mem abort info: [ 2.481442] Mem abort info: [ 2.489399] Exception class = DABT (current EL), IL = 32 bits [ 2.489406] SET = 0, FnV = 0 [ 2.489412] EA = 0, S1PTW = 0 [ 2.489420] Data abort info: [ 2.489426] ISV = 0, ISS = 0x00000004 [ 2.489432] CM = 0, WnR = 0 [ 2.489441] [000000000000002d] user address but active_mm is swapper [ 2.489450] Internal error: Oops: 96000004 [#1] PREEMPT SMP [ 2.489459] Modules linked in: [ 2.492292] Exception class = IABT (current EL), IL = 32 bits [ 2.495073] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G W 4.14.98+g5910884 #1 [ 2.495080] Hardware name: FSL i.MX8MM EVK board (DT) [ 2.495091] task: ffff80007a3c1b00 task.stack: ffff000009798000 [ 2.501034] SET = 0, FnV = 0 [ 2.504092] PC is at handle_irq_event_percpu+0x24/0x58 [ 2.507258] EA = 0, S1PTW = 0 [ 2.510128] LR is at handle_irq_event_percpu+0x1c/0x58 [ 2.510135] pc : [<ffff00000811bc0c>] lr : [<ffff00000811bc04>] pstate: 600001c5 [ 2.510140] sp : ffff00000800bee0 [ 2.510147] x29: ffff00000800bee0 [ 2.514010] swapper pgtable: 4k pages, 48-bit VAs, pgd = ffff000009705000 [ 2.516963] x28: ffff80007a3c1b00 [ 2.523344] [ffff0000096e4000] *pgd=00000000bfffe803 [ 2.528901] x27: 0000000000000000 x26: ffff00000800c000 [ 2.528911] x25: ffff000008008000 x24: ffff000009699000 [ 2.531999] , *pud=00000000bfffd803 [ 2.537906] x23: ffff80007a00f480 x22: ffff80007a628a28 [ 2.537914] x21: ffff80007a628aa4 x20: 0000000000000001 [ 2.537926] x19: 0000000000000404 [ 2.545955] , *pmd=00000000bfff8803 [ 2.550991] x18: 0000000000000007 [ 2.550998] x17: 0000000000000001 x16: 0000000000000019 [ 2.551006] x15: 0000000000000033 x14: 000000000000004c [ 2.556960] , *pte=00f8000041ae4f13 [ 2.560000] x13: 0000000000000000 x12: 0000000000000000 [ 2.560010] x11: 0000000000000000 x10: 0000000000000040 [ 2.568308] x9 : ffff0000094f2a68 x8 : ffff800076000248 [ 2.568317] x7 : 0000000000000000 x6 : 0000000000000000 [ 2.673233] x5 : 0000000000000000 x4 : 0000000000000000 [ 2.678559] x3 : 0000000000000000 x2 : 0000000000000404 [ 2.683887] x1 : 0000000000000404 x0 : 0000000000000404 [ 2.689218] Process swapper/1 (pid: 0, stack limit = 0xffff000009798000) [ 2.695933] Call trace: [ 2.698397] Exception stack(0xffff00000800bda0 to 0xffff00000800bee0) [ 2.704856] bda0: 0000000000000404 0000000000000404 0000000000000404 0000000000000000 [ 2.712699] bdc0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 2.720542] bde0: ffff800076000248 ffff0000094f2a68 0000000000000040 0000000000000000 [ 2.728387] be00: 0000000000000000 0000000000000000 000000000000004c 0000000000000033 [ 2.736230] be20: 0000000000000019 0000000000000001 0000000000000007 0000000000000404 [ 2.744074] be40: 0000000000000001 ffff80007a628aa4 ffff80007a628a28 ffff80007a00f480 [ 2.751917] be60: ffff000009699000 ffff000008008000 ffff00000800c000 0000000000000000 [ 2.759759] be80: ffff80007a3c1b00 ffff00000800bee0 ffff00000811bc04 ffff00000800bee0 [ 2.767602] bea0: ffff00000811bc0c 00000000600001c5 ffff80007a00f480 ffff000009699000 [ 2.775445] bec0: 0000ffffffffffff ffff00000800c000 ffff00000800bee0 ffff00000811bc0c [ 2.783292] [<ffff00000811bc0c>] handle_irq_event_percpu+0x24/0x58 [ 2.789486] [<ffff00000811bc88>] handle_irq_event+0x48/0x78 [ 2.795076] [<ffff00000811fa88>] handle_fasteoi_irq+0xa8/0x180 [ 2.800924] [<ffff00000811ac1c>] generic_handle_irq+0x24/0x38 [ 2.806685] [<ffff00000811b29c>] __handle_domain_irq+0x5c/0xb8 [ 2.812536] [<ffff000008081960>] gic_handle_irq+0x78/0x174 [ 2.818034] Exception stack(0xffff00000979be20 to 0xffff00000979bf60) [ 2.824487] be20: 0000000000000000 0000000000000000 0000000000000001 0000000000000000 [ 2.832334] be40: ffff0000094c0388 ffff00000979bf50 0000800076ae4000 0000000000000000 [ 2.840178] be60: ffff80007a3c24e0 ffff80007ffaade8 ffff80007a780e00 0000000000000000 [ 2.848023] be80: 0000000000000000 0000000000000000 000000000000004c 0000000000000033 [ 2.855865] bea0: 0000000000000019 0000000000000001 0000000000000007 ffff0000094ba018 [ 2.863709] bec0: ffff0000094d8000 ffff0000094d8000 ffff0000094c5cc0 ffff0000094d8e9c [ 2.871551] bee0: 0000000000000000 0000000000000000 ffff80007a3c1b00 0000000000000000 [ 2.879394] bf00: 0000000000000000 ffff00000979bf60 ffff00000808581c ffff00000979bf60 [ 2.887236] bf20: ffff000008085820 0000000000000045 0000000000000000 0000000000000000 [ 2.895079] bf40: ffffffffffffffff ffff000008141ad4 ffff00000979bf60 ffff000008085820 [ 2.902923] [<ffff000008083230>] el1_irq+0xb0/0x124 [ 2.907818] [<ffff000008085820>] arch_cpu_idle+0x10/0x18 [ 2.913147] [<ffff00000810c928>] do_idle+0x120/0x1e0 [ 2.918125] [<ffff00000810cb80>] cpu_startup_entry+0x20/0x28 [ 2.923802] [<ffff00000808faa0>] secondary_start_kernel+0x110/0x120 [ 2.930087] Code: b81fcc3f 97ffffa8 2a0003f3 b9402fa1 (b9402e80) [ 2.936203] ---[ end trace ad9723dcb0436f4a ]--- [ 2.940840] Kernel panic - not syncing: Fatal exception in interrupt [ 2.947209] SMP: stopping secondary CPUs [ 3.951150] SMP: failed to stop secondary CPUs 0-1 [ 3.955956] Kernel Offset: disabled [ 3.959460] CPU features: 0x080200c [ 3.962963] Memory Limit: none [ 3.966034] ---[ end Kernel panic - not syncing: Fatal exception in interrupt

Re: IMX8MM kernel hang while 3 SPIs are enabled

igorpadykov — Mon, 12 Oct 2020 11:16:27 GMT

Hi mahesh_rd

one can check if they are not used by M4 core, in SDK_EVK-MIMX8MM folder ..driver_examples/ecspi/

https://mcuxpresso.nxp.com/en/welcome

Best regards
igor

Re: IMX8MM kernel hang while 3 SPIs are enabled

mahesh_rd — Mon, 12 Oct 2020 16:22:34 GMT

Hi Igor,

I have double checked.There is no image running in the cortex M4 core. I still cant figure out why this is happening. I will proceed to do a DDR stress test tomorrow, since we have 4GB ram installed, compared to 2GB of evaluation module( although we have used only 2GB of 4GB). Do you think something else is wrong?

Re: IMX8MM kernel hang while 3 SPIs are enabled

igorpadykov — Tue, 13 Oct 2020 01:06:32 GMT

Hi mahesh_rd

if you did not test ddr memory, then this may be reason for that issue.

Best regards
igor

Re: IMX8MM kernel hang while 3 SPIs are enabled

wdunkley — Fri, 08 Jan 2021 03:24:23 GMT

I'm having a hard time getting my other SPI devices working, could I see what you did in your DTS file to get your second and third SPI device to work?

Thanks