Retrieve crash dump on IMX7 SDK

[jpr_83]

Hi,

I am trying to get the crash dump when kernel panic occurs. I am using meta-bsp-imx7 gatesgarth branch board support package

Linux version: 5.10.9 {i.MX Release Distro 5.10-gatesgarth (gatesgarth)}

When I run my app service and I get below kernel panic issue, to debug this I need the crash dump. I followed the link [https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/Kdump-Crash-in-I-MX/ta-p/1124321] [Note: Also followed the ppt attached in this link] and it’s not so clear also not able to get the crash dump.

Here is the IMX7 SDK board crash dump kernel configuration settings:
CONFIG_PREEMPT_NONE=y
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT is not set
CONFIG_KEXEC=y
CONFIG_SYSFS=y
CONFIG_DEBUG_INFO=y
CONFIG_CRASH_DUMP=y
# CONFIG_WLAN is not set
CONFIG_PROC_VMCORE=y
CONFIG_PSTORE=y
CONFIG_PSTORE_DEFLATE_COMPRESS=y
# CONFIG_PSTORE_LZO_COMPRESS is not set
# CONFIG_PSTORE_LZ4_COMPRESS is not set
# CONFIG_PSTORE_LZ4HC_COMPRESS is not set
# CONFIG_PSTORE_842_COMPRESS is not set
# CONFIG_PSTORE_ZSTD_COMPRESS is not set
CONFIG_PSTORE_COMPRESS=y
CONFIG_PSTORE_DEFLATE_COMPRESS_DEFAULT=y
CONFIG_PSTORE_COMPRESS_DEFAULT="deflate"
CONFIG_PSTORE_CONSOLE=y
CONFIG_PSTORE_PMSG=y
CONFIG_PSTORE_RAM=y
CONFIG_PSTORE_ZONE=y
CONFIG_PSTORE_BLK=y
CONFIG_PSTORE_BLK_BLKDEV=""
CONFIG_PSTORE_BLK_KMSG_SIZE=64
CONFIG_PSTORE_BLK_MAX_REASON=2
CONFIG_PSTORE_BLK_PMSG_SIZE=64
CONFIG_PSTORE_BLK_CONSOLE_SIZE=64

Packages included to get the crash dump in the driver-image.bb file
IMAGE_INSTALL += "tcpdump"
IMAGE_INSTALL += " kexec-tools makedumpfile"

Note1: I am not able to update device tree file for this crash dump configuration due to unable to find its location on recipe-kernel.

Some info about my trails for getting the crash dump:

root@cl-som-imx7:~# crashkernel=512M
root@cl-som-imx7:~# kexec -lpi /boot/Image --append="root=/dev/mmcblk0p2 rootwait rootfstype=ext4 maxcpus=0 reset_devices"
Memory for crashkernel is not reserved
Please reserve memory by passing"crashkernel=X@Y" parameter to kernel
Then try to loading kdump kernel
root@cl-som-imx7:~#
root@cl-som-imx7:~# crashkernel=512M@128M
root@cl-som-imx7:~# kexec -lpi /boot/Image --append="root=/dev/mmcblk0p2 rootwait rootfstype=ext4 maxcpus=0 reset_devices"
Memory for crashkernel is not reserved
Please reserve memory by passing"crashkernel=X@Y" parameter to kernel
Then try to loading kdump kernel
root@cl-som-imx7:~# 

Note2: Unable to set the crashkernel memory.

Please let me know if any additional information required.

Thanks in advance for your response.

[Bio_TICFSL]

Hello jpr_83,

The L5.10.35 and L5.10.52, run without crash please check your board or compile your project again. Also you are asigned too much memory to the kernel crash but this doesn't not happend on EVK.

Regards

[jpr_83]

Thanks for the response and I will try to update the Kernel version as you suggested, Any instructions set available for upgrading the kernel version?? Also please see my further investigation notes below.

Note: Updated crashkernel to 128M as suggested.

we are running NXP i.MX Release Distro 5.10-gatesgarth on NXP imx7 dual.

Note : compulab imx7 official version is linux imx7 4.14.98 version, and we took the development branch gatesgarth

we are using kexec toos 2.0.22 version on it, to analyze the kernel panic issue and we are getting below error [lzma_decompress_file: read on /boot/zImage of 1996179303 bytes failed].

command given to load kexec:

kexec -dcip /boot/zImage --command-line="console=ttymxc0,115200 root=/dev/mmcblk0p2 rw rootwait"

Cmdline o/p{

Try gzip decompression

Try LZMA decompression.

lzma_decompress_file: read on /boot/zImage of 1996179303 bytes failed

kernel: 0x765d3008 kernel_size: 0x8705d0

MEMORY RANGES

0000000080000000-00000000bfffffff (0)

zImage header: 0x016f2818 0x00000000 0x008705d0

zImage size 0x8705d0, file size 0x8705d0

offset 0x00001960 tag 0x5a534c4b size 24

zImage requires 0x008815d0 bytes

Decompressed kernel sizes:

text+data 0x014b31b4 bss 0x00075e6c total 0x01529020

Resulting kernel space: 0x01d34784

Reserved memory ranges

00000000b0000000-00000000b7ffffff (0)

Coredump memory ranges

0000000080000000-00000000afffffff (0)

00000000b8000000-00000000bfffffff (0)

kernel symbol _stext vaddr = 80100000

phys offset = 0x80000000, page offset = 80000000

Using 32-bit ELF core format

get_crash_notes_per_cpu: crash_notes addr = bf6c1b00, size = 180

Elf header: p_type = 4, p_offset = 0xbf6c1b00 p_paddr = 0xbf6c1b00 p_vaddr = 0x0 p_filesz = 0xb4 p_memsz = 0xb4

get_crash_notes_per_cpu: crash_notes addr = bf6d5b00, size = 180

Elf header: p_type = 4, p_offset = 0xbf6d5b00 p_paddr = 0xbf6d5b00 p_vaddr = 0x0 p_filesz = 0xb4 p_memsz = 0xb4

vmcoreinfo header: p_type = 4, p_offset = 0x8434a000 p_paddr = 0x8434a000 p_vadd r = 0x0 p_filesz = 0x1024 p_memsz = 0x1024

Elf header: p_type = 1, p_offset = 0x80000000 p_paddr = 0x80000000 p_vaddr = 0x8 0000000 p_filesz = 0x30000000 p_memsz = 0x30000000

Elf header: p_type = 1, p_offset = 0xb8000000 p_paddr = 0xb8000000 p_vaddr = 0xb 8000000 p_filesz = 0x8000000 p_memsz = 0x8000000

elfcorehdr: 0xb7f00000

crashkernel: [0xb0000000 - 0xb7ffffff] (128M)

memory range: [0x80000000 - 0xafffffff] (768M)

memory range: [0xb8000000 - 0xbfffffff] (128M)

kernel command line: "console=ttymxc0,115200 root=/dev/mmcblk0p2 rw rootwait elf corehdr=0xb7f00000 mem=130048K"

Kernel: address=0xb0008000 size=0x01d34784

DT : address=0xb1d3e000 size=0x0000c024

kexec_load: entry = 0xb0008000 flags = 0x280001

nr_segments = 3

segment[0].buf = 0x765d3008

segment[0].bufsz = 0x8705d0

segment[0].mem = 0xb0008000

segment[0].memsz = 0x871000

segment[1].buf = 0xb6b960

segment[1].bufsz = 0xc024

segment[1].mem = 0xb1d3e000

segment[1].memsz = 0xd000

segment[2].buf = 0xb67de8

segment[2].bufsz = 0x400

segment[2].mem = 0xb7f00000

segment[2].memsz = 0x1000

root@cl-som-imx7:~#

we can see that kexec entry has been loaded and crashkernel also been set from the above info.

but when we issue kexec -e, after this command it is throwing error saying nothing has been loaded. and it was unable to load crashkernel.

1)Since kexec 2.0.22 is official released version of kexec tools which supports ARM architecture as well but why for IMX7 arm zimage decompression technique is failing?

2)Any idea why the crash kernel is not being executed?

[shiva_bhargav_y]

I m also facing the same error with same board