topic Re: Cannot Boot A53 Linux from M7 on RDB3 in S32G

Cannot Boot A53 Linux from M7 on RDB3

lihongjun — Mon, 06 Nov 2023 11:04:14 GMT

Dear NXP,

I am trying to let RDB3 board boot from M7/nor, launch linux from A53/SD_card and run multicore IPCF demos.

The details of the images I built are as below:
1) M7_0 bootloader: secure boot, Crypto and Rm are all disabled;
2) IVT and bt_m7_blob.bin: boot target is M7, boot device is QSPI serial flash, DCD configured, HSE not configured.
3) u-boot: version = bsp35.0-2020.04, board = s32g399ardb3;
4) kernel: version = bsp35.0-5.10.145-rt, defconfig = s32cc_defconfig;
5) ATF: version = bsp35.0-2.5, plat = s32g399ardb3, BL33 = /home/lhj/fsl-auto-yocto-bsp/u-boot/u-boot-nodtb.bin
For dtc, current version is 1.6.0, meanwhile version 1.5.0 or 1.7.0 has been tried too.

The issues are:
1) When set board boot from NOR Flash:
It cannot boot and shows below messages:
NOTICE: Reset status: Power-On Reset
ERROR: Failed to check FDT integrity
PANIC at PC : 0x0000000034306878

2) When set board boot from SD card, use the pre-built image:
It can boot and launch kernel successfully.

3) When set board boot from SD card, use the pre-built image and only replace the manually-built fip.s32 file in SD card:
It can boot and launch kernel successfully.

4) When set board boot from SD card, use the pre-built image and replace the manually-built Image or s32g399a-rdb3.dtb file in SD card:
It can boot, but u-boot cannot launch the kernel, shows below messages:

NOTICE: Reset status: Power-On Reset
NOTICE: BL2: v2.5(release):bsp35.0-2.5-dirty
NOTICE: BL2: Built : 15:44:39, Nov 6 2023
NOTICE: BL2: Booting BL31

U-Boot 2020.04 (Nov 06 2023 - 14:46:46 +0800)

CPU: NXP S32G399A rev. 1.1
Model: NXP S32G399A-RDB3
DRAM: 3.5 GiB
Ignore unsupported SCMI protocol 19
MMC: FSL_SDHC: 0
Loading Environment from MMC... OK
Configuring PCIe0 as RootComplex
PCIe0: Failed to get link up
PCI: Failed autoconfig bar 1c
In: serial@401c8000
Out: serial@401c8000
Err: serial@401c8000
Board revision: RDB3 Revision F
Net: EQOS phy: rgmii @ 1

Warning: eth_eqos (eth0) using random MAC address - da:09:e1:22:aa:ae
eth0: eth_eqosFailed to get speed of XPCS for emac1_xpcs PFE: emac0: sgmii emac1: sgmii emac2: rgmii
PFEng firmware file 'mmc@0:1:s32g_pfe_class.fw' loading failed: -2

Hit any key to stop autoboot: 0
Failed to get speed of XPCS for emac1_xpcsPFEng firmware file 'mmc@0:1:s32g_pfe_class.fw' loading failed: -2
PFE: emac0: sgmii emac1: sgmii emac2: rgmii
pfeng_cfg_mode_enable: Invalid PFE device
switch to partitions #0, OK
mmc0 is current device
Booting from net ...
Failed to get speed of XPCS for emac1_xpcs PFE: emac0: sgmii emac1: sgmii emac2: rgmii
PFEng firmware file 'mmc@0:1:s32g_pfe_class.fw' loading failed: -2
eth_eqos Waiting for PHY auto negotiation to complete......... TIMEOUT !
phy_startup() failed: -110FAILED: -110Failed to get speed of XPCS for emac1_xpcsPFEng firmware file 'mmc@0:1:s32g_pfe_class.fw' loading failed: -2
eth_eqos Waiting for PHY auto negotiation to complete......... TIMEOUT !
phy_startup() failed: -110FAILED: -110Failed to get speed of XPCS for emac1_xpcsPFEng firmware file 'mmc@0:1:s32g_pfe_class.fw' loading failed: -2
Bad Linux ARM64 Image magic!
=>

It seems that the M7 bootloader works, but the kernel and device-tree I built are not correct, isn't it? If so, please tell me why and how to build them correctly.

Thanks.

Re: Cannot Boot A53 Linux from M7 on RDB3

Daniel-Aguirre — Mon, 06 Nov 2023 17:13:02 GMT

Hi,

For the option in which your image is failing, it seems that you are possibly adding steps, due to the BSP version difference.

Can you help us confirm if not copying both image and dtb to your SD card provide a correct outcome? We are not copying both image and dtb, since we are not modifying any of these.

Please, let us know.

Re: Cannot Boot A53 Linux from M7 on RDB3

lihongjun — Tue, 07 Nov 2023 01:18:48 GMT

Dear Daniel,

If just write prebuilt image to SD card and set board boot from SD card, it will bootup successfully, and login prompt will be shown at last.

But it will show "failed to check FDT integrity" if the board is set to boot from nor-flash.

Thanks.

Re: Cannot Boot A53 Linux from M7 on RDB3

Daniel-Aguirre — Tue, 07 Nov 2023 15:56:17 GMT

Hi,

Thanks for your feedback.

To verify if the fip.bin was built correctly, you could try to create an IVT with the fip.bin as the Application Bootloader and A53_0 as boot core. If there is no problem with the binary, then you should be able to boot from NOR Flash and see an output in the console.

Also, we may have not explained adequately, we were referring as if in the following step:

4) When set board boot from SD card, use the pre-built image and replace the manually-built Image or s32g399a-rdb3.dtb file in SD card:

Have you tried without "replace the manually-built Image or s32g399a-rdb3.dtb". Just modify the fip.s32 under the pre-built image, don't change any other file.

Please, let us now.

Re: Cannot Boot A53 Linux from M7 on RDB3

lihongjun — Wed, 08 Nov 2023 11:59:10 GMT

Dear Daniel,

I made the IVT with fip.bin I built and A53_0 as boot core, and the board can boot from NOR flash and run linux kernel successfully.

If I only replace the fip.s32 file in SD card and keep all other images unchanged, it can boot directly from SD card, and the kernel can be launched too.

Thanks.

Re: Cannot Boot A53 Linux from M7 on RDB3

lihongjun — Thu, 09 Nov 2023 10:38:19 GMT

Dear Daniel,

I reconfigured my M7 bootloader in EB, and a ram address overlapping mistake, pointed out by NXP Chinese FAE, was corrected. Now the u-boot is running, but error messages still come out when loading kernel:

...
U-Boot 2020.04 (Nov 09 2023 - 14:33:40 +0800)

This issue exists either use prebuilt kernel/dtb image file or the images built by myself.

So I still need your help.

Thanks.

Re: Cannot Boot A53 Linux from M7 on RDB3

Daniel-Aguirre — Thu, 09 Nov 2023 16:00:52 GMT

Hi,

Thanks for your feedback.

We may not have understood the log you shared, since we are not seeing any error, can you help us elaborate more on this last error?

Can you help us share the whole console log?

Please, let us know.

Re: Cannot Boot A53 Linux from M7 on RDB3

lihongjun — Fri, 10 Nov 2023 00:43:12 GMT

Dear Daniel,

When boot from Nor flash, It hangs up after show "pcie@40400000: RC/EP configuration not set correctly". I think a serious issue occurs when initializing PCIe interface. Is it possible that the issue is caused by confliction between data area used by PCIe driver and the one used by application running on M7 cores?

As a contrast, when boot from SD card directly, although it shows error messages about PCIe, as shown below, the boot process can continue and the kernel can be loaded completely with login prompt.
...
Configuring PCIe0 as RootComplex
PCIe0: Failed to get link up
PCI: Failed autoconfig bar 1c
In: serial@401c8000
Out: serial@401c8000
Err: serial@401c8000
...

Thanks.

Re: Cannot Boot A53 Linux from M7 on RDB3

Daniel-Aguirre — Fri, 10 Nov 2023 15:38:26 GMT

Hi,

Which application are you using under your M7 core? Are you using/configuring anything on regards of PCIe?

How are you configuring your Bootloader? Your application load addresses, to better understand your setup.

Since we are not able to reproduce your outcome, we would like to know how you are doing your setup.

Please, let us know.

Re: Cannot Boot A53 Linux from M7 on RDB3

lihongjun — Sun, 12 Nov 2023 04:14:53 GMT

Dear Daniel,

The applications I used under M7 core are IPCF_Example_multi_instance_S32G399_M7_0/1 included in S32G3 software bundle.

The M7 BootLoader creation and configuration steps are:

1. Import the Bootloader_S32G3XX_ASR_4.4_M7 project from Integration_Reference_Examples_S32G3_2023_02 folder into EB;

2. Load configuration, disable Rm item and Rm_Init item in SysDal plugin;

3. Edit boot sources as below:

The reset handler of those boot sources are gotten respectively from the ATF building output, the Reset_Handler keyword of IPCF_Example_multi_instance_S32G399_M7_0.map file and the intc_vector keyword of IPCF_Example_multi_instance_S32G399_M7_1.map file.

4. Configure Boot image fragments of linux_bsp_atf as below:
The load address is gotten from ATF building output.

5. Configure Boot image fragments of ipc_app_m7_0 as below:

The address is gotten from int_sram_c0 keyword in IPCF_Example_multi_instance_S32G399_M7_0.map. The size of IPCF_Example_multi_instance_S32G399_M7_0.bin file is 768KB.

6. Configure Boot image fragments of ipc_app_m7_1 as below:

The address is gotten from int_sram_c1 keyword in IPCF_Example_multi_instance_S32G399_M7_1.map. The size of IPCF_Example_multi_instance_S32G399_M7_1.bin file is 768KB too.

7. Configure the boot core as below:

8. Delete project output, generate project, then build the BootLoader by calling launch.bat in Cygwin.

BTW, I have tried to disable secure boot, but it has no affect on board bootup result.

Then I burned all the images mentioned above to Nor flash, wrote prebuilt image to SD card and replaced the fip.s32 file in card. The board can bootup OK in SD card boot mode but failed in Nor flash boot mode.

FYI.

Thanks.

Re: Cannot Boot A53 Linux from M7 on RDB3

Daniel-Aguirre — Mon, 13 Nov 2023 16:29:35 GMT

Hi,

Thanks for your feedback. For what we can see, we understand with the following configuration:

That you are using the default linker provided under the IPCF demo for S32G399. If so, we understand that the example is not prepared for using it as per the AN13750, since the "description.txt" file describes the expected steps to be followed, which are to load the binaries directly from u-boot, not through the usage of the NXP bootloader.

Since we understand you are trying the first steps to use the NXP bootloader, we recommend using the linker provided under the AN13750SW (the ones available under the IPCF examples) and modify a blink example with this linker. If done correctly, you should see a led being toggled under your RDB3 and Linux booting.

Again, IPCF examples were not prepared to be used as mentioned under the AN13750, that is why AN13750SW provides the examples to be used.

Please, let us know.

Re: Cannot Boot A53 Linux from M7 on RDB3

lihongjun — Tue, 14 Nov 2023 01:18:07 GMT

Dear Daniel,

I will try as you told, but might this is not the root cause of the issue. Because I have tried to configure the BL only boot ATF on A53_0 core, not boot the two M7_0/1 demos, and the issue is still there.

Thanks.

Re: Cannot Boot A53 Linux from M7 on RDB3

lihongjun — Tue, 14 Nov 2023 10:31:43 GMT

Dear Daniel,

I do not understand the meaning of "using the linker provided under the AN13750SW (the ones available under the IPCF examples) and modify a blink example with this linker", could you please tell me the operation steps?

Thanks.

Re: Cannot Boot A53 Linux from M7 on RDB3

Daniel-Aguirre — Tue, 14 Nov 2023 15:34:27 GMT

Hi,

With this we are referring to using the provided linker under the AN13750SW IPCF example, and modify the "Siul2_Dio_ToggleLed_S32G399A_M7" example's linker with it, something as shown below:

If done correctly, your M7_0 source should look as follows under the Bootloader:

This will only be a first step, so that we know it is working.

As for why when only the A53_0 core was enabled it didn't work, we assume that M7_0 core was left unattended (since there was no app linked to it) hence it may have cause the behavior.

Please, let us know.

Re: Cannot Boot A53 Linux from M7 on RDB3

lihongjun — Wed, 15 Nov 2023 09:54:48 GMT

Dear Daniel,

I have tried as what you told, and the LED toggle demo is running (I modified source code to let the blue LED flicker for 1000 times, and it works).

But for A53, u-boot cannot load kernel and always halt after showing "pcie@40400000: RC/EP configuration not set correctly", just the same as before.

I think there must be something wrong during ATF building. Please help me to find the reason.

And I have 2 more questions here:

1) Could you please tell me the u-boot source version, the ATF source version, the dtc version and the version of any other components/tools needed by ATF building which are used by your teams for IPCF application on RDB3 board?

2) In AN13750, it says I must apply the patch to modify the alignment of ATF. In fact I just modified the s32_common.mk file and didn't execute the patch command, because I don't know if I must do this for RDB3, and where and how to get the patch. Is it possible that this situation cause the ATF issue?

Thanks.

Re: Cannot Boot A53 Linux from M7 on RDB3

Daniel-Aguirre — Wed, 15 Nov 2023 16:59:46 GMT

Hi,

Thanks for your feedback. Can you help us share the Boot Sources tab of your bootloader? Since there we can see where the Reset Handler is being addresses. We do apologize.

As for your output, we are unable to replicate it from our side. Can you help us understand the steps you are following on how you are generating your Linux image? With this, we can try to look into any difference with your current steps.

As for your questions, the first one should be able to be answered better once we have the feedback of your current build steps. As from our side as a summary, we are using the pre-built NXP BSP provided inside a *.tgz file. We apply the path for the ATF for 64-bit alignment and then modify the fip.s32 under the pre-built image (again, the one provided inside a *.tgz file). We confirm that we can boot the image from SD card, to confirm there is no problem with our build, we verify the generated fip.bin with the creation of an IVT with A53_0 core as the boot core and if everything went right, we proceed to the bootloader. If the first 2 steps went well, we assume that any situation after that is related to our Bootloader/IVT configurations.

For the second, we apply the patch only for the 64-bit alignment.

Please, let us know.

Re: Cannot Boot A53 Linux from M7 on RDB3

lihongjun — Thu, 16 Nov 2023 02:19:21 GMT

Dear Daniel,

The boot source configure of my BL (ATF and LED toggle demo) is as below:

Boot image fragments of Linux ATF:

For Linux image building, the steps I followed are:

1) Download toolchain;

2) Build u-boot:
A) Clone u-boot source code, checkout to bsp35.0-2020.04;
B) Build u-boot:
make CROSS_COMPILE=/home/lhj/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu- s32g399ardb3_defconfig
make CROSS_COMPILE=/home/lhj/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-

3) Build ATF
A) Install libssl-dev and openssl;
B) Install dtc
C) Clone ATF source code and checkout to bsp35.0-2.5;
D) Edit s32_common.mk file set FIP_ALIGN := 64;
E) Build ATF
make CROSS_COMPILE=/home/lhj/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu- ARCH=aarch64 PLAT=s32g399ardb3 BL33=/home/lhj/fsl-auto-yocto-bsp/u-boot/u-boot-nodtb.bin
The result of ATF building is:
Image Layout
IVT: Offset: 0x1000 Size: 0x100
AppBootCode Header: Offset: 0x1200 Size: 0x40
Application: Offset: 0x1240 Size: 0x30800

Boot Core: A53_0
IVT Location: SD/eMMC
Load address: 0x342faf00
Entry point: 0x34302000

4) Build Linux
A) Clone Kernel source code and checkout to bsp35.0-5.10.145-rt;
B) Build:
make ARCH=arm64 CROSS_COMPILE=/home/lhj/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu- s32cc_defconfig
make ARCH=arm64 CROSS_COMPILE=/home/lhj/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-
Note: Although the kernel image and dtb files are generated successfully, I did not use them so far.

5) For usage of image files:
A) The fip.bin file built by myself was burned to nor flash at beginning address 0x100000;
B) For SD card, the prebuilt image fsl-image-auto-s32g399ardb3.sdcard (unzipped from binaries_auto_linux_bsp35.0_s32g3_pfe.tgz file included in S32G3 software bundle) was written first, then the fip.s32 file built by myself was replaced. The Image and s32g399a-rdb3.dtb file built by myself were not used yet.

FYI

Thanks.

Re: Cannot Boot A53 Linux from M7 on RDB3

Daniel-Aguirre — Thu, 16 Nov 2023 19:03:40 GMT

Hi,

Given the steps you are mentioning, we believe you have not used the AN13750SW linker, as per our previous description, since the address you show is the following:

And the one that from using the AN13750SW linker should be the following:

Still, we have changed our linker to, as per our current knowledge, be the same the one you are using (the one available under the IPCF_Example_multi_instance_S32G399_M7_0 example). Our result is the following:

With this, we compile the Bootloader. Create the following IVT:

Where the DCD and QSPI parameters can't be seen, but are the ones provided under the following paths:

C:\nxp\Integration_Reference_Examples_S32G3_2022_06\code\framework\realtime\swc\bootloader\platforms\S32G3XX\res\flash\S32G3XX_QuadSPI_133MHz_DDR_configuration.bin

C:\nxp\Integration_Reference_Examples_S32G3_2023_02\code\framework\realtime\swc\bootloader\platforms\S32G3XX\res\flash\S32G3XX_DCD_InitSRAM.bin

Since we disable HSE under the Bootloader, we do not provide any HSE-FW under the IVT:

Our Bootloader.c file looks as follows (only modifications done to it):

Then flash as per AN13750 explains:

Erase from 0x0 to 0x400000. Then flash as follows:

0x0 -> Blob of the IVT shown previously
0x100000 -> fip.bin
0x200000 -> Siul2_Dio_ToggleLed_S32G399A_M7.bin (where Reset_Handler address is 0x34100010, this can be confirmed under the *.map file)

With this, we are able to see the LED (under PA_06) being toggled under uboot and see the full Linux boot (when the Linux boot starts, the LED stops toggling, since we did not reconfigure anything under the Linux DTB):

The full log will be attached to this post.

At the end, we understand we are following your setup, as per our current understanding, and can see both cores running no problem.

Please, let us know.

Re: Cannot Boot A53 Linux from M7 on RDB3

lihongjun — Fri, 17 Nov 2023 03:34:18 GMT

Dear Daniel,

Thank you for your reply.

I checked my BL project in EB and demo project in S32DS carefully, all of them are just the same as yours. This situation confused me very much.

So I still think that this issue is caused by steps-missing during ATF building. Is there any modification or patch you have applied after checked out the ATF source code from git server? Should I apply the 0001-fip-align-and-mmc-init.patch as document AN13750 said? Or, should I apply the modification to remove the clock conflictions between M7 core and A53 core, as introduced in community topic S32G Bootloader Customzition - NXP Community?

Thanks.

Re: Cannot Boot A53 Linux from M7 on RDB3

Daniel-Aguirre — Fri, 17 Nov 2023 16:01:08 GMT

Hi,

As for the test I provided before, I only applied the FIP_ALIGN patch to be equal to 64. After that, I build the uboot and ATF as per AN13750 (with the branch set to BSP35.0, without the mmc patch, the platform to be RDB3 and the links to the github repository instead of CodeAurora).

With that, we are seeing the correct outcome from our side.

If you are only doing a toggle, toggle example does not configure any clocks, hence there should not be a problem. If you are configuring clocks, then all clock inits should be done under the Bootloader.

Still, we can recommend also opening an NXP online ticket, if you require any code/file sharing.

Please, let us know.