i.MX8MQ is hanging in busfreq-imx8mq.c driver

[harshitshah]

Hi NXP Team,

We are trying to port the i.MX8MQ custom board to Linux kernel 5.4.3. We are using the 5.4.3 kernel provided by the NXP team.

Please note that on the same board, previous version 4.7 is working well and it is giving file system prompt.

On 5.4 kernel with the same configuration, we are getting the hang. Please find attached logs about the hang.

When debugging the kernel hang we have found that the kernel is hanging/crashing in the busfreq-imx8mq.c driver. While adding the prints I found that it is hanging at busfreq_probe() function. In the probe function, it is hanging at the first iteration where the arm_smccc_smc() is called. 

Attached is device tree, defconfig file and serial logs that I am getting on 5.4.

Please let us know what are the next steps I need to do to get the file-system prompt.

Regards.

[harshitshah]

Hi Igor,

I have performed the DDR Stress test and applied the changes to the lpddr4_timing_b0.c file on 2 different U-boot versions. On top of that, the following are the changes that were done to support the 4GB DDR.

===========================================================

# Change - 1 (I have made this change as I thought it was required to be changed even without this change as well we are facing same hang)

diff --git a/arch/arm/dts/fsl-imx8mq.dtsi b/arch/arm/dts/fsl-imx8mq.dtsi
index cf8b41689f..d6808f34d8 100644
--- a/arch/arm/dts/fsl-imx8mq.dtsi
+++ b/arch/arm/dts/fsl-imx8mq.dtsi
@@ -51,7 +51,7 @@

memory@40000000 {
device_type = "memory";
- reg = <0x00000000 0x40000000 0 0xc0000000>;
+ reg = <0x00000000 0x40000000 0x00000001 0x00000000>;
};

# Change - 2

diff --git a/include/configs/imx8mq_evk.h b/include/configs/imx8mq_evk.h
index f153337a89..6caf2f249d 100644
--- a/include/configs/imx8mq_evk.h
+++ b/include/configs/imx8mq_evk.h
@@ -206,7 +206,7 @@

#define CONFIG_SYS_SDRAM_BASE 0x40000000
#define PHYS_SDRAM 0x40000000
-#define PHYS_SDRAM_SIZE 0xC0000000 /* 3GB DDR */
+#define PHYS_SDRAM_SIZE 0x100000000 /* 4GB DDR */

#define CONFIG_SYS_MEMTEST_START PHYS_SDRAM
#define CONFIG_SYS_MEMTEST_END (CONFIG_SYS_MEMTEST_START + \

# Change - 3 

diff --git a/board/freescale/imx8mq_evk/imx8mq_evk.c b/board/freescale/imx8mq_evk/imx8mq_evk.c
index 941620eedf..5a4dece8ee 100644
--- a/board/freescale/imx8mq_evk/imx8mq_evk.c
+++ b/board/freescale/imx8mq_evk/imx8mq_evk.c
@@ -75,6 +75,17 @@ int dram_init(void)
return 0;
}

+/* Get the top of usable RAM */
+ulong board_get_usable_ram_top(ulong total_size)
+{
+ //printf("board_get_usable_ram_top total_size is 0x%lx \n", total_size);
+
+ if(gd->ram_top > 0x100000000)
+ gd->ram_top = 0x100000000;
+
+ return gd->ram_top;
+}
+

===========================================================

-=================

Logs

1) The version that was working was the 2018.03 version.

===========================================================

U-Boot SPL 2018.03-master_v2018.03+g79594fe (Jan 23 2020 - 17:37:20 +0000)
PMIC:  PFUZE100 ID=0x10
DDRINFO: start lpddr4 ddr init
DDRINFO:ddrphy calibration done
DDRINFO: ddrmix config done
Normal Boot
Trying to boot from MMC1
U-Boot 2018.03-master_v2018.03+g79594fe (Jan 23 2020 - 17:37:20 +0000)CPU:   Freescale i.MX8MQ rev2.0 1300 MHz (running at 800 MHz)
CPU:   Industrial temperature grade (-40C to 105C) at 37C
Reset cause: POR
Model: i.MX8MQ
DRAM:  4 GiB
MMC:   FSL_SDHC: 0, FSL_SDHC: 1
Loading Environment from MMC... OK
No panel detected: default to HDMI
Display: HDMI (1280x720)
In:    serial
Out:   serial
Err:   serial BuildInfo:
  - ATF a-20190
  - U-Boot 2018.03-master_v2018.03+g79594feswitch to partitions #0, OK
mmc0(part 0) is current device
flash target is MMC:0
Net:   No ethernet found.
Fastboot: Normal
Normal Boot
Hit any key to stop autoboot:  0 

==========================================================

2) The logs which are creating hang are of the 2019.04 version.

  ==========================================================  

U-Boot SPL 2019.04-lf-5.4.y_v2019.04+g228843cdf5 (Jul 06 2020 - 15:40:39 +0000)
PMIC:  PFUZE100 ID=0x10
DDRINFO: start DRAM init
DDRINFO:ddrphy calibration done
DDRINFO: ddrmix config done
Normal Boot
Trying to boot from MMC2
U-Boot 2019.04-lf-5.4.y_v2019.04+g228843cdf5 (Jul 06 2020 - 15:40:39 +0000)CPU:   Freescale i.MX8MQ rev2.0 1300 MHz (running at 800 MHz)
CPU:   Industrial temperature grade (-40C to 105C) at 37C
Reset cause: POR
Model: i.MX8MQ
DRAM:  ��

==========================================================

Regards.

[igorpadykov]

Hi Harshit

one can try :

1. just for test use default EVK option with 3GB :

#define PHYS_SDRAM_SIZE            0xC0000000 /* 3GB DDR */

2. try to change uboot ddr firmware binaries using sect.4.3 Building u-boot image

MSCALE_DDR_Tool_User_Guide.pdf included in ddr test package.

Best regards
igor

[harshitshah]

Hi Igor,

With only below change, I am able to get the U-boot prompt (DDR Stress Test configuration are of 4GB as already attached in the original thread).

#define PHYS_SDRAM_SIZE            0xC0000000 /* 3GB DDR */

Please have a look at the below logs.

U-Boot SPL 2019.04-lf-5.4.y_v2019.04+g228843cdf5 (Jul 06 2020 - 21:26:22 +0000)
PMIC: PFUZE100 ID=0x10
DDRINFO: start DRAM init
DDRINFO:ddrphy calibration done
DDRINFO: ddrmix config done
Normal Boot
Trying to boot from MMC2

U-Boot 2019.04-lf-5.4.y_v2019.04+g228843cdf5 (Jul 06 2020 - 21:26:22 +0000)

CPU: Freescale i.MX8MQ rev2.0 1300 MHz (running at 800 MHz)
CPU: Industrial temperature grade (-40C to 105C) at 43C
Reset cause: POR
Model: i.MX8MQ

DRAM: 3 GiB
MMC: FSL_SDHC: 0, FSL_SDHC: 1
Loading Environment from MMC... Run CMD11 1.8V switch
*** Warning - bad CRC, using default environment

No panel detected: default to HDMI
Display: HDMI (1280x720)
In: serial
Out: serial
Err: serial

BuildInfo:
- ATF 7b3389d
- U-Boot 2019.04-lf-5.4.y_v2019.04+g228843cdf5

Run CMD11 1.8V switch
switch to partitions #0, OK
mmc1 is current device
Run CMD11 1.8V switch
Net:
Error: ethernet@30be0000 address not set.

Error: ethernet@30be0000 address not set.
eth-1: ethernet@30be0000
Fastboot: Normal
Normal Boot
Hit any key to stop autoboot: 0
u-boot=>

Now, the question is why we are not getting hang when we change the same to 3GB? 

Regards.

[igorpadykov]

Hi Harshit

so it works fine with 3GB and hangs with 4GB, right ?

I sent you 4GB patches via mail, please recheck.

[harshitshah]

Hi Igor,

With the mentioned patchsets of 4GB DDR  Linux kernel and U-boot on i.MX8MQ , I am getting below crash in the kernel.

====================================================================

U-Boot 2019.04-lf-5.4.y_v2019.04+g228843cdf5 (Jul 07 2020 - 18:49:20 +0000)

CPU: Freescale i.MX8MQ rev2.0 1300 MHz (running at 800 MHz)
CPU: Industrial temperature grade (-40C to 105C) at 45C
Reset cause: POR
Model: i.MX8MQ
DRAM: 4 GiB
MMC: FSL_SDHC: 0, FSL_SDHC: 1
Loading Environment from MMC... Run CMD11 1.8V switch
*** Warning - bad CRC, using default environment

No panel detected: default to HDMI
Display: HDMI (1280x720)
In: serial
Out: serial
Err: serial

BuildInfo:
 - ATF 7b3389d
 - U-Boot 2019.04-lf-5.4.y_v2019.04+g228843cdf5

Run CMD11 1.8V switch
switch to partitions #0, OK
mmc1 is current device
Run CMD11 1.8V switch
Net: 
Error: ethernet@30be0000 address not set.

Error: ethernet@30be0000 address not set.
eth-1: ethernet@30be0000
Fastboot: Normal
Normal Boot
Hit any key to stop autoboot: 0 
Run CMD11 1.8V switch
switch to partitions #0, OK
mmc1 is current device
Run CMD11 1.8V switch
** Unrecognized filesystem type **
** Unrecognized filesystem type **

switch to partitions #0, OK
mmc0(part 0) is current device
26550784 bytes read in 183 ms (138.4 MiB/s)
Booting from mmc ...
35301 bytes read in 5 ms (6.7 MiB/s)
## Flattened Device Tree blob at 43000000
 Booting using the fdt blob at 0x43000000
 Using Device Tree in place at 0000000043000000, end 000000004300b9e4

Starting kernel ...

[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[ 0.000000] Linux version 5.4.3-lts-lf-5.4.y+ge14cc4dc7e5f (oe-user@oe-host) (gcc version 9.2.0 (GCC)) #1 SMP PREEMPT Mon Jul 6 22:08:38 U0
[ 0.000000] Machine model: i.MX8MQ
[ 0.000000] earlycon: ec_imx6q0 at MMIO 0x0000000030860000 (options '115200')
[ 0.000000] printk: bootconsole [ec_imx6q0] enabled
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: UEFI not found.
[ 0.000000] Reserved memory: created CMA memory pool at 0x0000000044000000, size 960 MiB
[ 0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[ 0.000000] Internal error: synchronous external abort: 96000210 [#1] PREEMPT SMP
[ 0.000000] Modules linked in:
[ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.3-lts-lf-5.4.y+ge14cc4dc7e5f #1
[ 0.000000] Hardware name: i.MX8MQ Communication Processor (DT)
[ 0.000000] pstate: 20000085 (nzCv daIf -PAN -UAO)
[ 0.000000] pc : __create_pgd_mapping+0x190/0x748
[ 0.000000] lr : __create_pgd_mapping+0x150/0x748
[ 0.000000] sp : ffff8000118b3d60
[ 0.000000] x29: ffff8000118b3d60 x28: ffff800010080000 
[ 0.000000] x27: ffff800010080000 x26: fffffdfffe436800 
[ 0.000000] x25: 00000000bdfc3568 x24: fffffdfffe437000 
[ 0.000000] x23: ffff800010080000 x22: 0000000000000040 
[ 0.000000] x21: 0000000000f60000 x20: 0000000040480000 
[ 0.000000] x19: ffff800040000000 x18: 0000000000000010 
[ 0.000000] x17: 0000000000001400 x16: 0000000000001c00 
[ 0.000000] x15: ffff8000118c22f0 x14: 642d646572616873 
[ 0.000000] x13: 20646920656c6269 x12: 0000000000000000 
[ 0.000000] x11: 0000000000000004 x10: 0000000000000018 
[ 0.000000] x9 : 0000000000000000 x8 : fffffdfffe43a000 
[ 0.000000] x7 : 0000000000000000 x6 : 0000000000000000 
[ 0.000000] x5 : ffff8000119d5000 x4 : 000fffffdfffe437 
[ 0.000000] x3 : 0000000000000037 x2 : ffff800010fdffff 
[ 0.000000] x1 : ffff80003fffffff x0 : 000000013ffff003 
[ 0.000000] Call trace:
[ 0.000000] __create_pgd_mapping+0x190/0x748
[ 0.000000] map_kernel_segment+0x70/0xa4
[ 0.000000] paging_init+0x12c/0x5d0
[ 0.000000] setup_arch+0x268/0x5e8
[ 0.000000] start_kernel+0x88/0x44c
[ 0.000000] Code: f94053e2 92628673 d1000661 eb02003f (f9400301) 
[ 0.000000] random: get_random_bytes called from print_oops_end_marker+0x28/0x50 with crng_init=0
[ 0.000000] ---[ end trace 0000000000000000 ]---
[ 0.000000] Kernel panic - not syncing: Attempted to kill the idle task!
[ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---

====================================================================
Please let me know further steps.

[harshitshah]

Hi Igor,

A gentle reminder to share the further steps for the last error.

Regards.

[igorpadykov]

Hi Harshit

nxp has special service for helping customers to port software

Professional Engineering Services | NXP 

Best regards
igor