imx8mp custom board boot process hang at running /sbin/init

[npavel]

Hi,

We have a custom board based on imx8mp. The board it's pretty simple and versus the evk version we don't have pcie, sound, any kind of output beside uart, spi/can.  We have an usb mini port for debugging/console and instead of  6GB we have a 8GB module. We added lpddr4_timing.c from MXC ddr memory tool in uboot.

We are building core-image-base with modifications in uboot and kernel DTS files, where we removed pcie regulator can1_stby, can2_stby, reg_audio_pwr bt_sco_codec sound-bt-sco sound-hdmi regulator etc for things that don't exits. Attached are the DTS files, configs and multiple boot logs from serial.

In 90% our boots the booting stops at running init sometime the boot hands a bit sooner. We are out of ideas to try or where to look, we'd appreciate some ideas how to debug this issue further.

We are trying to boot from SD and eMMC with same results. We can use uuu to flash the yocto uboot and image to eMMC.

Kernel config: https://pastebin.com/b1barZrd
u-boot config: https://pastebin.com/Wn4HUFTD

Thank you.

[harisislam]

Hello everyone,

I've gone through the discussion above, and while many of you are further ahead, I'm currently stuck at a more basic stage.

I'm working with a custom board based on the NXP i.MX 8M Plus processor. My goal for now is simply to boot the board and get UART logs, but unfortunately, I'm not seeing any output.

Interestingly, if I flash the same flash.bin image to the i.MX 8M Plus EVK, it boots successfully and UART logs are shown.

The main difference between the EVK and my custom board is the RAM configuration. My board uses 8GB of LPDDR4, while the EVK has 6GB.

To support the 8GB RAM on my board:

I used the NXP DDR Tool to generate a new lpddr4_timing.c file.

I replaced the existing one in u-boot/board/freescale/imx8mp_evk/lpddr4_timing.c.

I also updated the device tree to reflect the change from 6GB to 8GB in the memory node.

I then rebuilt the image and flashed it to my custom board.

Despite these changes, I'm still not getting any UART output when booting the custom board. However, the same image continues to work on the EVK.

Can anyone help me figure out what might be going wrong? Any guidance on debugging this or checking the DDR initialization would be really helpful.

Thanks in advance!

Haris

[Bassam]

If someone still interested, I had the same issue where the boot stopped after "running /sbin/init"

I also had this error in the logs:

"imx-pgc imx-pgc-domain.8: failed to command PGC"

I use imx8mp, with kernel version 6.1.xx

And I was able to solve it when I added a dummy node for the pgc power domains in imx8mp.dtsi like in the attached photo

[maxvde]

Hi @npavel,

I'm curious if this issue was ever resolved? I have the exact same setup, i.MX8M Plus on a custom board, hanging in exactly the same way when starting /sbin/init or /bin/bash.

In my case, we're using eMMC only.

Build config is the same:

Build Configuration:
BB_VERSION = "2.2.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal"
TARGET_SYS = "aarch64-poky-linux"
MACHINE = "imx8mp-lpddr4-evk"
DISTRO = "fsl-imx-wayland"
DISTRO_VERSION = "6.1-langdale"
TUNE_FEATURES = "aarch64 armv8a crc crypto"
TARGET_FPU = ""

Makes no sense to me. The filesystem appears to mount fine.

[ 1.419295] mmc0: new DDR MMC card at address 0001
[ 1.425723] mmcblk0: mmc0:0001 008GB0 7.28 GiB
[ 1.436345] mmcblk0: p1 p2
[ 1.440356] mmcblk0boot0: mmc0:0001 008GB0 8.00 MiB
[ 1.446808] mmcblk0boot1: mmc0:0001 008GB0 8.00 MiB
[ 1.453136] mmcblk0rpmb: mmc0:0001 008GB0 4.00 MiB, chardev (234:0)
...
[ 1.608639] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Quota mode: none.
[ 1.617236] VFS: Mounted root (ext4 filesystem) on device 179:2.
[ 1.624285] devtmpfs: mounted
[ 1.628062] Freeing unused kernel memory: 3328K
[ 1.632711] Run /bin/bash as init process
[ 1.636730] with arguments:
[ 1.639705] /bin/bash
[ 1.642334] with environment:
[ 1.645479] HOME=/
[ 1.647836] TERM=linux

But it proceeds no further.

Max

[npavel]

Yes. I cleaned up everything from the dts/dtsi from the kernel. This means even removing the other dts files (for other boards) that had different things defined in dtsi files. Basically starting a new dts/dtsi platform definition with only the basic things like CPU, memory, emmc and board main power domain definition. After this booted, I added other needed devices.  In my case there were issues with other power domains.

[maxvde]

Hi. Thanks for the reply. Interesting, I basically did the same with my .dts, commenting out almost everything to begin with, but I still was including imx8mp.dtsi.

So with the clean dts/dtsi you are able to get to boot past /sbin/init and into a normal system? 

For me, U-Boot works fine, I checked the memory there with mtest and that was fine, and Fastboot can load all of core-image-minimal into the eMMC chip, but somehow it breaks when Linux is trying to run /sbin/init after mounting the device.

I'll take a closer look at the power domain definitions.

[npavel]

Yes, removing eveything from dts/dtsi include chain and leaving only basic cpu, memory, emmc worked.

[Dhruvit]

Hi @npavel,

I hope you are doing well.

Please make sure you have correctly formatted and partitioned SDcard & the correct rootfs is present.
Please provide me with schematics of emmc/SD card connections.

Please refer to the below-mentioned file for u-boot environment variables.

/include/configs/imx8mp_evk.h

Are you using yocto to build a custom image (linux,rootfs, etc)?

Thanks & Regards,
Dhruvit Vasavada

[npavel]

Hi Dhruvit,

Thanks for looking into this issue.  We are using yocto  and writing the sd or mmc from the core-image-base wic file using dd. The contents seems to be correct the /sbin/init being /lib/systemd/systemd binary correctly compiled for arm64. The yocto build is the default for core-image-base but with the DTS file changes for uboot/kernel as attached in the previous message. Nothing else is changed in the build (6.1-langdale).
The build was initialized with this command: 

DISTRO=fsl-imx-wayland MACHINE=imx8mp-lpddr4-evk   source imx-setup-release.sh -b build-aib-v1

 Build information:

Build Configuration:
BB_VERSION           = "2.2.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-poky-linux"
MACHINE              = "imx8mp-lpddr4-evk"
DISTRO               = "fsl-imx-wayland"
DISTRO_VERSION       = "6.1-langdale"
TUNE_FEATURES        = "aarch64 armv8a crc crypto"

I'm going to send the schematic as private message if it's ok.

Thanks.

[Dhruvit]

Hi @npavel,

I hope you are doing well.

From the logs, it seems the console kernel parameter is declared multiple times in mmcroot env.

One should specify the console parameter in bootargs env.

Please specify which uart port is used as debug console in the custom board.

Please see /include/configs/imx8mp_evk.h for specifying environment variables.

Please make sure the card detect pin is working properly.

Thanks & Regards,
Dhruvit Vasavada

[npavel]

Hi Dhruvit,

> From the logs, it seems the console kernel parameter is declared multiple times in mmcroot env.

Yes we added this after we tested a few times with just one console declaration in bootargs just to make sure that init doesn't change to another console

> Please make sure the card detect pin is working properly.

We believe that the CD pin is working. Would this pin make the init process stop/not execute  without any feedback ?

Thanks,

[Dhruvit]

Hi @npavel,

I hope you are doing well.

Please share the updated logs and printenv output for further debugging.

Please mention which UART is being used as a console.

We believe that the CD pin is working. Would this pin make the init process stop/not execute without any feedback?
[Ans]: it could make rootfs inaccessible or can create problems while mounting it.

Thanks & Regards,
Dhruvit Vasavada