"Synchronous Abort" handler, esr 0x96000210 on S32G2 Custom HW - Crash while access to LPDDR4 Dear Community Members, We are encountering a boot issue on our S32G274A-based hardware, where the bootloader fails with a data abort exception. Setup Details: Boot Method: QSPI Image: Custom FIP image with integrated ramdisk Flashing Method: S32DS Flash Tool and Lauterbach (issue occurs with both) BSP Version: 43.0 Bootloader: U-Boot 2022.04+gc01b3bf7b1+p0 SoC: S32G274A We are able to configure the DDR and able to boot till uboot prompt. But after that an access to DDR get crash , ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Please find the below details for reference. => md 80000000 1000 "Synchronous Abort" handler, esr 0x96000210 elr: 00000000ff91cb68 lr : 00000000ff91cab8 (reloc) elr: 00000000ff91cb68 lr : 00000000ff91cab8 x0 : 0000000000000009 x1 : 00000000ffbc66e8 x2 : 00000000fffffffe x3 : 0000000000000020 x4 : 0000000000000000 x5 : 00000000ffbc66e0 x6 : 0000000000000030 x7 : 00000000ffbc6630 x8 : 0000000000000010 x9 : 0000000000000002 x10: 00000000ffffffd8 x11: 0000000000000010 x12: 0000000000000001 x13: 0000000000000001 x14: 0000000000000030 x15: 0000000000000021 x16: 00000000ff8af778 x17: 0000000000000000 x18: 00000000ffbcfd80 x19: 0000000000000004 x20: 0000000000000004 x21: 0000000000000004 x22: 0000000080000000 x23: 00000000ffbc66e9 x24: 0000000000000000 x25: 00000000ffbc6698 x26: 00000000ff9398a7 x27: 0000000000000008 x28: 0000000000000004 x29: 00000000ffbc6630 Code: 2a0403f3 17ffffcb 7100129f 54000181 (b94002c3) Resetting CPU ... resetting ... NOTICE: Reset status: Power-On Reset NOTICE: BL2: v2.10.7(release):bsp43.0-2.10.7-dirty NOTICE: BL2: Built : 08:17:37, Nov 26 2024 NOTICE: BL2: Booting BL31 ------------------------------------------------------------------------------------------ The complete boot-log is as below, where the crash can be found while Uboot trying to access DDR. ------------------------------------------------------------------------------------------ Please find the boot logs attached below for reference. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ U-Boot 2022.04+gc01b3bf7b1+p0 (Nov 21 2024 - 14:00:36 +0000) SoC: NXP S32G274A rev. 2.1 CPU: ARM Cortex-A53 r0p4 @ max 1000 MHz Model: S32G2-platform DRAM: 4 GiB Inside dm_init_and_scan............ pfeng_mdio mdio@2: EMAC MDIO id out of range 88Q5152 reset Core: 291 devices, 24 uclasses, devicetree: board MMC: FSL_SDHC: 0 Loading Environment from SPIFlash... SF: Detected mt35xu01gbba with page size 256 Bytes, erase size 4 KiB, total 128 MiB *** Warning - bad CRC, using default environment s32cc_serdes_phy serdes@40480000: Using mode 0 for SerDes subsystem pci_s32cc pcie@40400000: DBI R/W is not being enabled pci_s32cc pcie@40400000: PCI Device and Vendor IDs could not be set pci_s32cc pcie@40400000: DBI R/W is not being enabled s32cc_serdes_phy serdes@40480000: Failed to lock PCIe phy s32cc_serdes_phy serdes@40480000: PHY: Failed to power on serdes@40480000: -110. pci_s32cc pcie@40400000: Failed to power on PHY 'serdes_lane0' pci_s32cc pcie@40400000: Failed to set PCIe host settings In: serial@401c8000 Out: serial@401c8000 Err: serial@401c8000 Board revision: RDB2 Net: Found PFE version 0x50300 (S32G2) pfeng pfeng-base: Uploading CLASS firmware pfeng pfeng-base: EMAC0 block was initialized pfeng pfeng-base: EMAC1 block was initialized pfeng pfeng-base: EMAC2 block was initialized pfeng pfeng-base: Enabling the CLASS block pfeng pfeng-base: PFE Platform started successfully (mask: 7) pfeng_netif pfe0: Failed to get 'emac0_xpcs' PHY eth1: pfe0pfeng_netif pfe1: Failed to get 'emac1_xpcs' PHY , eth2: pfe1, eth3: pfe2 Hit any key to stop autoboot: 0 switch to partitions #0, OK mmc0(part 0) is current device ** No partition table - mmc 0 ** Couldn't find partition mmc 0:2 Can't set block device mcr=cc000f03 ipcr=3400000f flashcr=3030100 buf0cr=3f000000 buf1cr=3f000000 buf2cr=3f000000 buf3cr=800080 bfgencr=e00000 buf0ind=0 buf1ind=0 buf2ind=0 awrcr=0 dllcra=d0080c2 paritycr=0 sfar=2c03 sfacr=0 smpr=33 rbsr=0 rbct=1f000000 awrsr=0 dllsr=f4c00080 dlcr=ff40ff40 dlsr_fa=0 tbsr=0 tbdr=0 tbct=0 sr=380002 fr=1000009 rser=0 sptrclr=0 dlpr=433455aa => run flashboot Booting from flash... Reading 17629184 byte(s) at offset 0x00000000 "Error" handler, esr 0xbf000002 elr: 00000000ff8f5f6c lr : 00000000ff8f5e68 (reloc) elr: 00000000ff8f5f6c lr : 00000000ff8f5e68 x0 : 0000000000000040 x1 : 0000000000000000 x2 : 0000000000000ba0 x3 : 000000000000003f x4 : 0000000000000002 x5 : 000000001fe00411 x6 : 0000000000000000 x7 : 0000000000000011 x8 : 0000000000200000 x9 : 0000000000000008 x10: 0000000000000002 x11: 000000001fe00000 x12: 0000000000000001 x13: 000000000000003f x14: 0000000000000000 x15: 0000000000000021 x16: 00000000ff8f4e9c x17: 0000000000000000 x18: 00000000ffbcfd80 x19: 00000000ffbf33f0 x20: 00000000010d0000 x21: 00000000002b0000 x22: 0000000080000000 x23: 00000000007d2dc2 x24: 0000000000000007 x25: 0000000000000000 x26: 0000000000000001 x27: 0000000000000001 x28: 00000000010d0000 x29: 00000000ffbc6130 Code: d1000694 380016c0 17ffffbd f8606aa1 (f8206ac1) Resetting CPU ... Removing MTD device #5 (Kernel) with use count 1 Error when deleting partition "Kernel" (-16) resetting ... NOTICE: Reset status: Destructive Reset (RUN) NOTICE: BL2: v2.10.7(release):bsp43.0-2.10.7-dirty NOTICE: BL2: Built : 08:17:37, Nov 26 2024 NOTICE: BL2: Booting BL31 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ We would appreciate any guidance or suggestions to help resolve this issue. Regards, Vishal G Re: "Synchronous Abort" handler, esr 0x96000210 on S32G2 Custom HW - Crash while access to Hello, @vishalg
Thanks for your reply.
From the log, it is "Internal error: synchronous external abort", this indicates that a exception occurred while accessing certain address, which is likely related to accessing unmapped memory or inconsistent permissions.
Not sure if you modified default dts for GIC part? have you examined the memory used in gic_iterate_rdists?
BR
Chenyin Re: "Synchronous Abort" handler, esr 0x96000210 on S32G2 Custom HW - Crash while access to Hi Chenyin, Thankyou for the quick reply. I am able to resolve the issue by performing the DDR Validation test using S32 Design Studio and configuring the DDR correctly with regenerated configurations.. Now i am successfully able to boot uboot come to kernel where i am observing kernel panic at boot, triggered by an "Internal error: synchronous external abort" while initializing the GICv3 interrupt controller on our Custom S32G274A platform running Linux 6.6.52-rt43. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- NOTICE: Reset status: Power-On Reset NOTICE: BL2: v2.10.7(release):bsp43.0-2.10.7-dirty NOTICE: BL2: Built : 08:17:37, Nov 26 2024 NOTICE: BL2: Booting BL31 U-Boot 2022.04+gc01b3bf7b1+p0 (Nov 21 2024 - 14:00:36 +0000) SoC: NXP S32G274A rev. 2.1 CPU: ARM Cortex-A53 r0p4 @ max 1000 MHz Model: S32G2-platform DRAM: 3.5 GiB Inside dm_init_and_scan............ pfeng_mdio mdio@2: EMAC MDIO id out of range 88Q5152 reset Core: 291 devices, 24 uclasses, devicetree: board MMC: FSL_SDHC: 0 Loading Environment from SPIFlash... SF: Detected mt35xu01gbba with page size 256 Bytes, erase size 4 KiB, total 128 MiB *** Warning - bad CRC, using default environment s32cc_serdes_phy serdes@40480000: Using mode 0 for SerDes subsystem pci_s32cc pcie@40400000: DBI R/W is not being enabled pci_s32cc pcie@40400000: PCI Device and Vendor IDs could not be set pci_s32cc pcie@40400000: DBI R/W is not being enabled s32cc_serdes_phy serdes@40480000: Failed to lock PCIe phy s32cc_serdes_phy serdes@40480000: PHY: Failed to power on serdes@40480000: -110. pci_s32cc pcie@40400000: Failed to power on PHY 'serdes_lane0' pci_s32cc pcie@40400000: Failed to set PCIe host settings In: serial@401c8000 Out: serial@401c8000 Err: serial@401c8000 Board revision: RDB2 Net: Found PFE version 0x50300 (S32G2) pfeng pfeng-base: Uploading CLASS firmware pfeng pfeng-base: EMAC0 block was initialized pfeng pfeng-base: EMAC1 block was initialized pfeng pfeng-base: EMAC2 block was initialized pfeng pfeng-base: Enabling the CLASS block pfeng pfeng-base: PFE Platform started successfully (mask: 7) pfeng_netif pfe0: Failed to get 'emac0_xpcs' PHY eth1: pfe0pfeng_netif pfe1: Failed to get 'emac1_xpcs' PHY , eth2: pfe1, eth3: pfe2 Hit any key to stop autoboot: 0 switch to partitions #0, OK mmc0(part 0) is current device ** No partition table - mmc 0 ** Couldn't find partition mmc 0:2 Can't set block device mcr=cc000f03 ipcr=3400000f flashcr=3030100 buf0cr=3f000000 buf1cr=3f000000 buf2cr=3f000000 buf3cr=800080 bfgencr=e00000 buf0ind=0 buf1ind=0 buf2ind=0 awrcr=0 dllcra=d0080c2 paritycr=0 sfar=2c03 sfacr=0 smpr=33 rbsr=0 rbct=1f000000 awrsr=0 dllsr=15c00080 dlcr=ff40ff40 dlsr_fa=0 tbsr=0 tbdr=0 tbct=0 sr=380002 fr=1000009 rser=0 sptrclr=0 dlpr=433455aa => run flashboot Booting from flash... Reading 17629184 byte(s) at offset 0x00000000 Reading 196608 byte(s) at offset 0x00000000 Reading 32571392 byte(s) at offset 0x00000000 ## Loading init Ramdisk from Legacy Image at 90000000 ... Image Name: fsl-image-base-s32g2-platfo Image Type: AArch64 Linux RAMDisk Image (uncompressed) Data Size: 29917545 Bytes = 28.5 MiB Load Address: 00000000 Entry Point: 00000000 Verifying Checksum ... OK ## Flattened Device Tree blob at 83000000 Booting using the fdt blob at 0x83000000 Using Device Tree in place at 0000000083000000, end 00000000830120ae Starting kernel ... unable to select a mode : -70 device_remove: Device '
[email protected]' failed to remove, but children are gone [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034] [ 0.000000] Linux version 6.6.52-rt43-g1a29a32be610-dirty (oe-user@oe-host) (aarch64-fsl-linux-gcc (GCC) 11.4.0, GNU ld (GNU Binutils) 2.38.20220708) #1 SMP PREEMPT_RT Thu Nov 21 09:35:09 UTC 2024 [ 0.000000] Machine model: s32g274 Platform [ 0.000000] earlycon: linflex0 at MMIO 0x00000000401c8000 (options '') [ 0.000000] printk: legacy bootconsole [linflex0] enabled [ 0.000000] OF: reserved mem: 0x0000000034000000..0x000000003407ffff (512 KiB) nomap non-reusable pfebufs@34000000 [ 0.000000] OF: reserved mem: 0x0000000034080000..0x000000003409ffff (128 KiB) nomap non-reusable pfebufs@34080000 [ 0.000000] Reserved memory: created DMA memory pool at 0x0000000083200000, size 3 MiB [ 0.000000] OF: reserved mem: initialized node pfebufs@83200000, compatible id shared-dma-pool [ 0.000000] OF: reserved mem: 0x0000000083200000..0x00000000835dffff (3968 KiB) nomap non-reusable pfebufs@83200000 [ 0.000000] OF: reserved mem: 0x00000000835e0000..0x00000000835fffff (128 KiB) map non-reusable pfebufs@835e0000 [ 0.000000] OF: reserved mem: 0x0000000084000000..0x0000000084ffffff (16384 KiB) nomap non-reusable shm@84000000 [ 0.000000] OF: reserved mem: 0x00000000c0000000..0x00000000c03fffff (4096 KiB) nomap non-reusable shm@c0000000 [ 0.000000] OF: reserved mem: 0x00000000c0400000..0x00000000c07fffff (4096 KiB) nomap non-reusable shm@c0400000 [ 0.000000] OF: reserved mem: 0x00000000d0000000..0x00000000d000007f (0 KiB) nomap non-reusable shm@d0000000 [ 0.000000] OF: reserved mem: 0x00000000d0000080..0x00000000d00000ff (0 KiB) nomap non-reusable shm@d0000080 [ 0.000000] OF: reserved mem: 0x00000000ff600000..0x00000000ff7fffff (2048 KiB) nomap non-reusable atf@ff600000 [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000080000000-0x00000000ffffffff] [ 0.000000] DMA32 empty [ 0.000000] Normal [mem 0x0000000100000000-0x00000008dfffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000080000000-0x00000000831fffff] [ 0.000000] node 0: [mem 0x0000000083200000-0x00000000835dffff] [ 0.000000] node 0: [mem 0x00000000835e0000-0x0000000083ffffff] [ 0.000000] node 0: [mem 0x0000000084000000-0x0000000084ffffff] [ 0.000000] node 0: [mem 0x0000000085000000-0x00000000bfffffff] [ 0.000000] node 0: [mem 0x00000000c0000000-0x00000000c07fffff] [ 0.000000] node 0: [mem 0x00000000c0800000-0x00000000cfffffff] [ 0.000000] node 0: [mem 0x00000000d0001000-0x00000000ff5fffff] [ 0.000000] node 0: [mem 0x00000000ff600000-0x00000000ff7fffff] [ 0.000000] node 0: [mem 0x00000000ff800000-0x00000000ffffffff] [ 0.000000] node 0: [mem 0x0000000880000000-0x00000008dfffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000008dfffffff] [ 0.000000] On node 0, zone DMA: 1 pages in unavailable ranges [ 0.000000] cma: Reserved 256 MiB at 0x00000000ef600000 on node -1 [ 0.000000] psci: probing for conduit method from DT. [ 0.000000] psci: PSCIv1.1 detected in firmware. [ 0.000000] psci: Using standard PSCI v0.2 function IDs [ 0.000000] psci: MIGRATE_INFO_TYPE not supported. [ 0.000000] psci: SMC Calling Convention v1.4 [ 0.000000] percpu: Embedded 20 pages/cpu s42112 r8192 d31616 u81920 [ 0.000000] pcpu-alloc: s42112 r8192 d31616 u81920 alloc=20*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [ 0.000000] Detected VIPT I-cache on CPU0 [ 0.000000] CPU features: detected: GIC system register CPU interface [ 0.000000] CPU features: detected: ARM erratum 845719 [ 0.000000] CPU features: detected: NXP erratum ERR050481 (TLBI by VA handled incorrectly) [ 0.000000] CPU features: detected: ARM errata 1165522, 1319367, 1530923, or 1530924 [ 0.000000] alternatives: applying boot alternatives [ 0.000000] Kernel command line: console=ttyLF0,115200 root=/dev/ram rw earlycon [ 0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) [ 0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 903167 [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.000000] software IO TLB: area num 4. [ 0.000000] software IO TLB: mapped [mem 0x00000000eb600000-0x00000000ef600000] (64MB) [ 0.000000] Memory: 3194416K/3670012K available (9856K kernel code, 686K rwdata, 3012K rodata, 1856K init, 245K bss, 213452K reserved, 262144K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] rcu: Preemptible hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4. [ 0.000000] rcu: RCU priority boosting: priority 1 delay 500 ms. [ 0.000000] rcu: RCU_SOFTIRQ processing moved to rcuc kthreads. [ 0.000000] No expedited grace period (rcu_normal_after_boot). [ 0.000000] Trampoline variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode [ 0.000000] GICv3: 544 SPIs implemented [ 0.000000] GICv3: 0 Extended SPIs implemented [ 0.000000] GICv3: MBI range [167:182] [ 0.000000] GICv3: Using MBI frame 0x0000000050800000 [ 0.000000] Root IRQ handler: gic_handle_irq [ 0.000000] Internal error: synchronous external abort: 0000000096000010 [#1] PREEMPT_RT SMP [ 0.000000] Modules linked in: [ 0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.6.52-rt43-g1a29a32be610-dirty #1 [ 0.000000] Hardware name: s32g274 Platform (DT) [ 0.000000] pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 0.000000] pc : gic_iterate_rdists+0x4c/0x114 [ 0.000000] lr : gic_init_bases+0x198/0x664 [ 0.000000] sp : ffffffc080e83b80 [ 0.000000] x29: ffffffc080e83b80 x28: 0000000080ca6074 x27: 0000000000000000 [ 0.000000] x26: 0000000000000000 x25: ffffffc080e8c208 x24: 0000000000000018 [ 0.000000] x23: 000000000000ffe8 x22: 0000000000000000 x21: ffffffc08040ed84 [ 0.000000] x20: ffffffc0809f9b60 x19: ffffffc081000000 x18: 0000000000000030 [ 0.000000] x17: 6a20303031207369 x16: 2079616c65642074 x15: ffffffffffff1660 [ 0.000000] x14: 0000000000000000 x13: 7172695f656c646e x12: 61685f636967203a [ 0.000000] x11: ffffffffffff0000 x10: 000000000000000a x9 : ffffffc080e9aa78 [ 0.000000] x8 : 00000000fffff7ff x7 : ffffffc080ec6a78 x6 : 000000000000004d [ 0.000000] x5 : ffffff885f8479c8 x4 : 0000000000000000 x3 : 0000000000000000 [ 0.000000] x2 : 0000000000000000 x1 : ffffff88000030e0 x0 : ffffffc08100ffe8 [ 0.000000] Call trace: [ 0.000000] gic_iterate_rdists+0x4c/0x114 [ 0.000000] gic_init_bases+0x198/0x664 [ 0.000000] gic_of_init+0x2a8/0x308 [ 0.000000] of_irq_init+0x318/0x3a8 [ 0.000000] irqchip_init+0x18/0x24 [ 0.000000] init_IRQ+0xac/0xc0 [ 0.000000] start_kernel+0x24c/0x5f0 [ 0.000000] __primary_switched+0xb4/0xbc [ 0.000000] Code: d29ffd17 9b387eda f87a6833 8b170260 (b9400000) [ 0.000000] ---[ end trace 0000000000000000 ]--- [ 0.000000] Kernel panic - not syncing: Attempted to kill the idle task! [ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]--- -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Any input what can cause this issue ? Regards, Vishal Re: "Synchronous Abort" handler, esr 0x96000210 on S32G2 Custom HW - Crash while access to Hello, @vishalg
Thanks for your post.
In common, such issues are caused by accessing the protect or not-mapped memory(like reserved one), may I know if you had modified the memory map of default BSP43? adding more reserved address range?
Have you tested more addresses? are there any other ranges of address are valid for loading images?
BR
Chenyin