kernel crash with a panic message on our custom board based on imx6

[shivashankar]

Hi,

We are using i.MX6D processor on a Nano SOM and a DDR mounted on it.

The DDR is NT6TL64T64CR from Nanya Technologies.

We have generated the new u-boot after generating the new .inc file and it is booting fine till u-boot.

I am using sd card booting, but it is failing to load the kernel and generating a crash message.

The below is the sample boot log after which kernel is not loading. PFA for complete boot log.

Kindly shed some light on this issue.

Sample boot log :

mc0: queuing unknown CIS tuple 0x80 (7 bytes)
mmc0: queuing unknown CIS tuple 0x80 (6 bytes)
mmc1: SDHCI controller on 2194000.usdhc [2194000.usdhc] using ADMA
mmc2: no vqmmc regulator found
mmc2: no vmmc regulator found
mmc2: SDHCI controller on 2198000.usdhc [2198000.usdhc] using ADMA
mmc0: queuing unknown CIS tuple 0x91 (3 bytes)
mmc0: new high speed SDIO card at address 0001
Galcore version 5.0.11.25762
Unable to handle kernel paging request at virtual address d7f9d009
pgd = 80004000
[d7f9d009] *pgd=00000000
Internal error: Oops: 5 [#1] PREEMPT SMP ARM
Modules linked in:
CPU: 1 PID: 1 Comm: swapper/0 Not tainted 3.14.28+gfd07114f13f2 #1
task: 8c0a0000 ti: 8c09e000 task.ti: 8c09e000
PC is at gckOS_FreeNonPagedMemory+0x40/0x108

Regards,

Shiva Shankar K.

[igorpadykov]

Hi Shiva

error "Unable to handle kernel paging request at virtual address d7f9d009" may be due to

memory errors. One can run ddr test

https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/i-MX-6-7-DDR-Stress-Test-Tool/ta-p/11082...

and update uboot dcd header with new ddr settings found from ddr test

https://source.codeaurora.org/external/imx/uboot-imx/tree/board/freescale/mx6sabresd/mx6dlsabresd.cf...

Best regards
igor

[shivashankar]

Hi Igor,

As you suggested I tried two methods :

1) DDR_Stress_Tester_V1.0.3.1 tool is used since my ddr is configured in interleaved mode

Here I have attached the DDR_Stress_Tester_V1.0.3.1.zip file which I used for ddr stress test from windows machine. I repeatedly updated the NT-6T-L64T64CR-GO.cfg (renamed in the attachment as NT-6T-L64T64CR-GO.txt file)  file DCD values in the uboot source code by the updated read and write calibration values obtained from the stress test. Although, I am getting the crash message as I said in the beginning of our conversation. 

One more info to you is after calibration of the read and write operations, the ddr stress test keeps running in loop in iteration more than 200 times and was not able to reach the end of the test. 

2) Stress test from the u-boot

I used the stress test binary files present in the DDR_Stress_Tester_V1.0.3.1.zip file and it is getting stuck after the 

## Starting application at 0x00907000 ...

Please find the attachment for the log 

Kindly let me know what could be issue and if I am using proper ddr stress tool version.

Regards,

Shiva Shankar K.

[igorpadykov]

Hi Shiva

>One more info to you is after calibration of the read and write operations, the ddr stress test

>keeps running in loop in iteration more than 200 times and was not able to reach the end of the test. 

seems board did not pass ddr test. One can try to change drive strength of ddr signals described in

Chapter IOMUX Controller (IOMUXC) Reference Manual.

Best regards
igor

[shivashankar]

Hi Ignor,

I tried the DDR stress test. Meanwhile, I have observed one thing when my custom board with new ddr booted up. i.e.,

The new DDR NT6TL64T64CR-GO is of 512 MBytes on the whole as against  my old DDR

 MT42L128M64LL-25  which is of 1GBytes.

The old DDR prints as it is 1GB in the u-boot whereas the new DDR prints only 256MB rather printing 512MB.

This is what is baffling me and I am using the same .xls sheet attached here, with 14 bit row and 9 bit column despite having that, it is giving me 256 MBytes, I strongly believe that if this is resolved hopefully the kernel panic issue could be sorted out, please help me to understand this. 

I am attaching the .xls sheet here for your reference once again.

Regards,

Shiva Shankar K.

[shivashankar]

Hi Igor,

The below is my observation once again : 

1) u-boot 2014.04

In this we have boards.cfg file to pass .cfg and DDR memory size as below

Active arm armv7 mx6 mistral mx6nano mx6qnano_interleave mx6nano:IMX_CONFIG=board/mistral/mx6nano/mx6q_42_128m64d2ll-25_4kb.cfg,MX6Q,DEFAULT_FDT_FILE="imx6q-nano.dtb",DDR_INTERLEAVE,DDR_MB=256 Vijai Kumar K <vijaikumar.k@mistralsolutions.com>

 a) In this When I give DDR_MB=512 the u-boot gets stuck as below :

U-Boot 2017.03-nano_v2014.03_3.14_ra+g442f54f6bc (Apr 19 2021 - 14:01:41 +0530)

CPU: Freescale i.MX6D rev1.5 at 792MHz
CPU: Extended Commercial temperature grade (-20C to 105C) at 30C
Reset cause: POR
show_board_info 20
Model: Freescale i.MX6 Dual NANO SOM Board
Board: MX6-NANO
DRAM: 512 MiB

b) When I give DDR_MB =256 the uboot executes fine and the kernel crashes
 

U-Boot 2017.03-nano_v2014.03_3.14_ra+g442f54f6bc (Apr 18 2021 - 11:03:45 +0530)

CPU: Freescale i.MX6D rev1.5 at 792MHz
CPU: Extended Commercial temperature grade (-20C to 105C) at 30C
Reset cause: POR
show_board_info 20
Model: Freescale i.MX6 Dual NANO SOM Board
Board: MX6-NANO
DRAM: 256 MiB

PMIC: PFUZE100! DEV_ID=0x10 REV_ID=0x21
MMC: FSL_SDHC: 0, FSL_SDHC: 1
No panel detected: default to Hannstar-XGA
Display: Hannstar-XGA (1024x768)
In: serial
Out: serial
Err: serial
Net:
Error: ethernet@02188000 address not set.
No ethernet found.
Normal Boot
Hit any key to stop autoboot: 3

2) u-boot 2017.04

We pass the config file as below for the DDR,

CONFIG_SYS_EXTRA_OPTIONS="IMX_CONFIG=board/mistral/mx6nano/mx6q_42_128m64d2ll-25_4kb.cfg,MX6Q"

without even mentioning the DDR size here the u-boot by itself calculates the size as 256 MB and crashes in the kernel.

How come the u-boot here calculates the memory as 256MB despite having a physical 512 MB memory and why does it gets stuck if I manually give DDR SIZE as 512MB in the u-boot 2014.04 version.   

3) I took my old DDR of 1GB from micron and reduced the size to 512MB in the u- boot. This booted u-boot and kernel completely and everything was fine. Where as when I reduced it further down to 256MB then the kernel crash issue appeared.

Kindly shed some light on this issue.

Regards,

Shiva Shankar K.

[igorpadykov]

Hi Shiva

ddr size issue can be debugged in function imx_ddr_size()

https://source.codeaurora.org/external/imx/uboot-imx/tree/arch/arm/imx-common/cpu.c?h=imx_v2017.03_4...

Best regards
igor

[shivashankar]

Hi Igor,

We have the DDR working now, as you said it was due to the signal drive strength. 

Initially, I made manual changes in the xls sheet rather changing input parameters in the configuration sheet. Finally, got it working by changing in the configuration sheet.

Thanks a lot for all the support.

Regards,

Shiva Shankar K.