LayerscapeのトピックRe: Kernel Hangs at Boot With Secure Boot Enabled

Kernel Hangs at Boot With Secure Boot Enabled

brandensherrell — Tue, 21 Jun 2016 20:55:33 GMT

I feel as though I am flooding this community with questions regarding Secure Boot, so for that I apologize. At any rate, now that I finally have the device booting in a secure state I have run into yet another issue; the kernel is hanging after loading and fails to boot.

All validation succeeds for U-boot, PPA, bootscript, and the kernel and evidence of this can be seen in the log below. My bootscript simply does the following:

cp.b 0x60A00000 0x81000000 0x3500000
esbc_validate 0x63F40000
bootm 0x81000000

If I flash a RCW with SBEN=0 then I can successfully validate all of these components individually and boot the kernel without issue (successful boot log shown after failed boot log below). It seems that when I am booting with Secure Boot enabled that the kernel is hanging. To provide a bit more information, I the core never releases any other core (i.e. only core0 is running) and is stuck at a yield instruction:

Additionally, we are operating in a secure state at this point per the HPSR register.

Here is the U-boot log for reference. I noticed that PPA seems to issue "PPA validation started .. " quite a few times.

=> cpld reset altbank
U-Boot 2016.012.0+ga9b437f (Jun 03 2016 - 13:05:36 -0400)
SoC: unknown (0x87920410)
Clock Configuration:
       CPU0(A53):1500 MHz CPU1(A53):1500 MHz CPU2(A53):1500 MHz
       CPU3(A53):1500 MHz
       Bus:      400 MHz DDR:      1600 MT/s FMAN:     500 MHz
Reset Configuration Word (RCW):
       00000000: 0810000f 0c000000 00000000 00000000
       00000010: 14550002 80004012 e0625000 61002000
       00000020: 00000000 00000000 00000000 00038800
       00000030: 00000000 00001100 00000096 00000001
I2C:   ready
Model: LS1043A RDB Board
Board: LS1043ARDB, boot from vBank 4
CPLD: V1.4
PCBA: V3.0
SERDES Reference Clocks:
SD1_CLK1 = 156.25MHZ, SD1_CLK2 = 100.00MHZ
DRAM: Initializing DDR....
Detected UDIMM Fixed DDR on board
2 GiB (DDR4, 32-bit, CL=11, ECC off)
SEC0: RNG instantiated
PPA validation startedPPA validation Successful
PPA Firmware: Version 0.2
Firmware 'Microcode version 0.0.1 for LS1021a r1.0' for 1021 V1.0
QE: uploading microcode 'Microcode for LS1021a r1.0' version 0.0.1
PPA validation startedPPA validation Successful
Using SERDES1 Protocol: 5205 (0x1455)
Flash: 128 MiB
NAND: 512 MiB
MMC:   FSL_SDHC: 0
Using default environment
EEPROM: NXID v1
PCIe1: disabled
PCIe2: Root Complex no link, regs @ 0x3500000
PCIe3: Root Complex no link, regs @ 0x3600000
In:    serial
Out:   serial
Err:   serial
SCSI: Error: SCSI Controller(s) 1B4B:9170 not found
Net:   Fman1: Uploading microcode version 106.4.17
FM1@DTSEC1, FM1@DTSEC2, FM1@DTSEC3 [PRIME], FM1@DTSEC4, FM1@DTSEC5, FM1@DTSEC6, FM1@TGEC1
Hit any key to stop autoboot: 0
esbc_validate command successful
## Executing script at 60060000
esbc_validate command successful
## Loading kernel from FIT Image at 81000000 ...
   Using 'config@1' configuration
   Trying 'kernel@1' kernel subimage
     Description: ARM64 Linux kernel
     Type:         Kernel Image
     Compression: gzip compressed
     Data Start:   0x810000d4
     Data Size:    5384092 Bytes = 5.1 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x80080000
     Entry Point: 0x80080000
   Verifying Hash Integrity ... OK
## Loading ramdisk from FIT Image at 81000000 ...
   Using 'config@1' configuration
   Trying 'ramdisk@1' ramdisk subimage
     Description: Ramdisk
     Type:         RAMDisk Image
     Compression: uncompressed
     Data Start:   0x8152914c
     Data Size:    26922046 Bytes = 25.7 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: unavailable
     Entry Point: unavailable
   Verifying Hash Integrity ... OK
## Loading fdt from FIT Image at 81000000 ...
   Using 'config@1' configuration
   Trying 'fdt@1' fdt subimage
     Description: Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression: uncompressed
     Data Start:   0x81522924
     Data Size:    26534 Bytes = 25.9 KiB
     Architecture: AArch64
   Verifying Hash Integrity ... OK
   Loading fdt from 0x81522924 to 0x90000000
   Booting using the fdt blob at 0x90000000
   Uncompressing Kernel Image ... OK
   Loading Ramdisk to ce653000, end cffffc3e ... OK
   Loading Device Tree to 00000000ce639000, end 00000000ce6527a5 ... OK
PPA validation startedPPA validation Successful
PPA validation startedPPA validation Successful
Starting kernel ...

Successful booting of this exact same kernel image (but with SBEN=0) is shown below. It does appear to be a little different.

=> cpld reset altbank
U-Boot 2016.012.0+ga9b437f (Jun 03 2016 - 13:05:36 -0400)
SoC: unknown (0x87920410)
Clock Configuration:
       CPU0(A53):1500 MHz CPU1(A53):1500 MHz CPU2(A53):1500 MHz
       CPU3(A53):1500 MHz
       Bus:      400 MHz DDR:      1600 MT/s FMAN:     500 MHz
Reset Configuration Word (RCW):
       00000000: 0810000f 0c000000 00000000 00000000
       00000010: 14550002 80004012 e0425000 61002000
       00000020: 00000000 00000000 00000000 00038800
       00000030: 00000000 00001101 00000096 00000001
I2C:   ready
Model: LS1043A RDB Board
Board: LS1043ARDB, boot from vBank 4
CPLD: V1.4
PCBA: V3.0
SERDES Reference Clocks:
SD1_CLK1 = 156.25MHZ, SD1_CLK2 = 100.00MHZ
DRAM: Initializing DDR....
Detected UDIMM Fixed DDR on board
2 GiB (DDR4, 32-bit, CL=11, ECC off)
SEC0: RNG instantiated
PPA Firmware: Version 0.2
Firmware 'Microcode version 0.0.1 for LS1021a r1.0' for 1021 V1.0
QE: uploading microcode 'Microcode for LS1021a r1.0' version 0.0.1
Using SERDES1 Protocol: 5205 (0x1455)
Flash: 128 MiB
NAND: 512 MiB
MMC:   FSL_SDHC: 0
Using default environment
EEPROM: NXID v1
PCIe1: disabled
PCIe2: Root Complex no link, regs @ 0x3500000
PCIe3: Root Complex no link, regs @ 0x3600000
In:    serial
Out:   serial
Err:   serial
SCSI: Error: SCSI Controller(s) 1B4B:9170 not found
Net:   Fman1: Uploading microcode version 106.4.17
FM1@DTSEC1, FM1@DTSEC2, FM1@DTSEC3 [PRIME], FM1@DTSEC4, FM1@DTSEC5, FM1@DTSEC6, FM1@TGEC1
Hit any key to stop autoboot: 0
=> source 0x60060000
## Executing script at 60060000
esbc_validate command successful
## Loading kernel from FIT Image at 81000000 ...
   Using 'config@1' configuration
   Trying 'kernel@1' kernel subimage
     Description: ARM64 Linux kernel
     Type:         Kernel Image
     Compression: gzip compressed
     Data Start:   0x810000d4
     Data Size:    5384092 Bytes = 5.1 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: 0x80080000
     Entry Point: 0x80080000
   Verifying Hash Integrity ... OK
## Loading ramdisk from FIT Image at 81000000 ...
   Using 'config@1' configuration
   Trying 'ramdisk@1' ramdisk subimage
     Description: Ramdisk
     Type:         RAMDisk Image
     Compression: uncompressed
     Data Start:   0x8152914c
     Data Size:    26921871 Bytes = 25.7 MiB
     Architecture: AArch64
     OS:           Linux
     Load Address: unavailable
     Entry Point: unavailable
   Verifying Hash Integrity ... OK
## Loading fdt from FIT Image at 81000000 ...
   Using 'config@1' configuration
   Trying 'fdt@1' fdt subimage
     Description: Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression: uncompressed
     Data Start:   0x81522924
     Data Size:    26534 Bytes = 25.9 KiB
     Architecture: AArch64
   Verifying Hash Integrity ... OK
   Loading fdt from 0x81522924 to 0x90000000
   Booting using the fdt blob at 0x90000000
   Uncompressing Kernel Image ... OK
   Using Device Tree in place at 0000000090000000, end 00000000900197a5
Starting kernel ...
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Initializing cgroup subsys cpu
...

Re: Kernel Hangs at Boot With Secure Boot Enabled

yipingwang — Tue, 28 Jun 2016 07:12:51 GMT

Hello Branden Sherrell,

The cause of the problem is that device tree download to the incorrect address at the end of the Linux Kernel log with secure boot enabled, please refer to to the following lines from the log, in the normal secure boot log there should be no these lines.

Uncompressing Kernel Image ... OK

Loading Ramdisk to ce653000, end cffffc3e ... OK

Loading Device Tree to 00000000ce639000, end 00000000ce6527a5 ... OK

Please try the following boot script.

cp.b 0x60A00000 0x81000000 0x3500000

esbc_validate 0x63F40000

setenv bootargs "console=ttyS0,115200 root=/dev/ram0 earlycon=uart8250,0x21c0500";

setenv fdt_high "0xffffffffffffffff";

setenv initrd_high "0xffffffffffffffff";

bootm $img_addr

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

Re: Kernel Hangs at Boot With Secure Boot Enabled

brandensherrell — Tue, 28 Jun 2016 19:03:16 GMT

Hi Yiping, that was it! How could you tell from the logs that this was happening?

I read from uboot documentation that setting fdt_high and initrd_high to 0xffffffffffffffff indicates that they should not be copied to RAM. This makes sense considering the fdt and initrd are contained within the FIT image.

However, both of those environment variables were already defined in my uboot environment. The only difference between your script and mine was that the default environment bootargs was:

bootargs=console=ttyS0,115200 root=/dev/ram0 earlycon=uart8250,mmio,0x21c0500 mtdparts=60000000.nor:1m(nr_bank0_rcw),1m(nor_bank0_uboot),1m(nor_bank0_uboot_env),1m(nor_bank0_fman_uconde),40m(nor_bank0_fit),1mnor_bank4_rcw),1m(nor_bank4_uboot),1m(nor_bank4_uboot_env),1m(nor_bank4_fman_ucode),40m(nor_bank4_fit);7800000.flash:1m(nand_uboot),1m(nand_uboot_env),20m(nand_fit);spi0.0:1m(uboot),5m(kernel),1m(dtb),9m(filesystem)

And you issued a

bootm $img_addr

rather than directly using the load address.

bootm 0x81000000

Would you mind providing a bit of info about why this change was necessary and why the default bootargs string did not work properly?

Re: Kernel Hangs at Boot With Secure Boot Enabled

brandensherrell — Thu, 30 Jun 2016 12:58:56 GMT

It seems that the Secure Boot u-boot does not use the environment variables? If so, what tells Secure Boot to ignore its environment variable set?

If I flash a RCW with SBEN=0 then I can trap into the u-boot shell and print the default environment variable. It also gives 10 seconds to do so, while the Secure Boot u-boot gives 0 seconds (i.e. no chance to trap into a shell).

Re: Kernel Hangs at Boot With Secure Boot Enabled

yipingwang — Fri, 01 Jul 2016 01:59:36 GMT

Hello Branden Sherrell,

In secure boot mode, u-boot cannot be stopped to edit the environment variable, u-boot will execute the boot script to boot Linux Kernel directly.

It's is necessary to add the following bootargs parameters in the boot script. Even through these parameters have already been saved in the u-boot environment space, in secure boot mode, these parameter will not be used.

setenv bootargs "console=ttyS0,115200 root=/dev/ram0 earlycon=uart8250,0x21c0500";

setenv fdt_high "0xffffffffffffffff";

setenv initrd_high "0xffffffffffffffff";

Have a great day,
Yiping