S32G_Put_UBOOT_PFEFW_into_QSP

[#1]

本文说明在S32G3 RDB3板上，Uboot中使能PFE驱动时，需要加载PFE FW，默认Uboot中，PFE FW是放在SD/eMMC的FAT分区，通过文件系统访问来读取。本文说明如何修改为从QSPI NOR中读取。主要的应用场景是：

•  在烧写镜像时，需要Uboot通过网口来烧写内核镜像及rootfs。而此时SD/eMMC还没有分区，所以无法将PFE网口需要的FW放在FAT分区中。

• 目录

  1    背景与相关资料... 2

  1.1  问题背景... 2

  1.2  需要的软件，工具与文档... 3

  2    将Uboot PFE FW放在QSPI Nor上... 4

  2.1  Uboot代码说明与修改... 4

  2.2  测试... 6

[eburley]

S32G Put Uboot PFE FW into QSPI Nor

This article explains that on the S32G3 RDB3 board, when the PFE driver is enabled in Uboot, the PFE FW needs to be loaded. By default, in Uboot, the PFE FW is placed in the FAT partition of SD/eMMC and read through file system access. This article explains how to modify it to read from QSPI NOR.

The main application scenarios are:  When burning the image, Uboot needs to burn the kernel image and rootfs through the network port. At this time, SD/eMMC has no partition, so the FW required by the PFE network port cannot be placed in the FAT partition.

The software version is BSP41 UBoot.

History               Description                    Author

V1                        Created this article                John.Li

1 Background and related information

1.1 Problem background

The Linux image burning of S32G is usually as follows:

1. Use the burning tool (flash tools+uart or lauterbach program+jtag) to burn the bootloader into QSPI NOR.
2. Use the burning tool (flash tools+uart or lauterbach program+jtag) to burn the fip image into QSPI NOR;

Or: Use the burning tool (flash tools+uart or lauterbach program+jtag), or bootloader+CAN,

to burn ATF BL2 into QSPI NOR and burn Uboot into eMMC.

1. Uboot burns the kernel image and rootfs into eMMC through network devices (such as: tftp+NFS).

Assuming that the only network device supported by Uboot is PFE, it is necessary to develop a method to load PFE FW, because NXP's default Uboot for SD/eMMC startup only supports storing PFE FW in the FAT partition, and the above process cannot partition SD/eMMC.

So this article explains a way to solve this problem.

First, analyze all the ways to load FW for PFE drivers:

Serial No. Method Location Problem

1 The M core PFE MCAL Master driver compiles FW as an array into the \example_application\pfe_firmware \s32g_pfe_class.c program file.

 2 The A core Uboot driver reads FW binary from the SD/eMMC FAT partition and puts it together with the kernel image and DTB: /boot# ls s32g_pfe_class.fw … No FAT partition during programming

3 The A core PFE master kernel module driver reads FW binary from the file system root file system /lib/firmware# ls s32g_pfe_class.fw s32g_pfe_util.fw The module driver starts slowly

4 The A core PFE master kernel driver reads FW binary from the kernel image and compiles it into the kernel. Refer to the following document [3] The kernel driver starts quickly

This article is mainly to solve the problem of item 2. The solution ideas can be:

No. Method Reference Problem

1 If there is a GMAC hardware connection, change to use GMAC as the network port for burning the image. Uboot has a GMAC driver, which can be used directly. 1: GMAC is usually used by the M core, and customers do not want to use GMAC in Linux. 2: Using GMAC requires changing another network interface, which is sometimes not allowed by the hardware.

2 Compile PFE FW into Uboot as an array. Refer to the method of MCAL driver. 1: There is a certain amount of development work. 2: Updating FW requires updating Uboot, or even the entire FIP.

 3 Prepare a minimum linux system based on qspi and use the kernel driver directly to burn. NXP has released Uboot support based on QSPI Nor. 1: There is a certain amount of development work. 2: It cannot be reused with the image that is officially started, and it will occupy a certain amount of QSPI NOR space.

4 Store PFE FW in QSPI NOR, and change PFE FW loading from SD/eMMC FAT partition to loading from QSPI NOR. NXP has released Uboot support based on QSPI NOR, in which PFE FW is also stored in QSPI NOR. This method supports separate upgrade of PFE FW. 1: Uboot needs to support QSPI driver, which may conflict with MCAL QSPI driver (but generally speaking, M core is in suspended state during Linux burning process).

Compared with item 4, this method is more reasonable, so this article explains it.

1.2 Required software, tools and documents

Software,          Tools,   Documents               Name Description

Linux

BSP

BSP41

Reference Documentation：

《S32G3_LinuxBSP_41.0_User_Manual.pdf》

从www.nxp.com/s32g个人帐号中获得

App

notes

S32G Uboot customization doc

https://community.nxp.com/t5/

NXP-Designs-Knowledge-Base/

S32G-Uboot-customization-application-doc/

ta-p/1399900

App

notes

S32G_Compiled_PFE_into_Kernel

https://community.nxp.com/

t5/NXP-Designs-Knowledge-Base/

S32G-Compiled-PFE-into-Kernel/

ta-p/1726770

Create a BSP41 Uboot development environment according to the instructions in item 1/2 of the document.

2 Put Uboot PFE FW on QSPI Nor

2.1 Uboot code description and modification

1. Compare configs\s32g399ardb3_defconfig and configs\s32g399ardb3_qspi_defconfig

The difference is that one defines CONFIG_SD_BOOT=y and the other defines CONFIG_QSPI_BOOT=y

1. drivers\net\nxp_pfeng\kconfig:

choice

prompt "PFE firmware location"

depends on NXP_PFENG_STANDALONE

default NXP_PFENG_FW_LOC_SDCARD if SD_BOOT //So here we can change it to NXP_PFENG_FW_LOC_QSPI

default NXP_PFENG_FW_LOC_QSPI if QSPI_BOOT

config NXP_PFENG_FW_LOC_SDCARD

bool "SD card"

depends on NXP_PFENG_STANDALONE

help

The firmware file is located on SD card.

config NXP_PFENG_FW_LOC_QSPI

bool "QSPI flash"

depends on NXP_PFENG_STANDALONE

help

The firmware is on the QSPI memory.

endchoice

1. drivers\net\nxp_pfeng\pfeng_fw.c

int pfeng_fw_set_from_env_and_load(struct pfeng_priv *priv)

{

/* Parse fw destination from environment */

env_fw = env_get(PFENG_ENV_VAR_FW_SOURCE);

#define PFENG_ENV_VAR_FW_SOURCE "pfengfw"

Therefore, the pfengfw parameter should be configured in uboot to directly point to the fw address, as follows:

According to code analysis：

#if CONFIG_IS_ENABLED(NXP_PFENG_FW_LOC_QSPI)

fw_part = env_fw;

#endif

/* FW load */

ret = pfeng_fw_load(fw_name, fw_int, fw_part, fw_type, priv);

static int pfeng_fw_load(char *fname, char *iface, const char *part, int ftype,

struct pfeng_priv *priv)

{…

val = simple_strtoul(part, NULL, 16);

ret = pfeng_fw_check_qspi_addr(val);

qspi_addr = val;

ret = setup_flash_device(&flash);

ret = load_pfe_fw(flash, qspi_addr, &fw_buffer, &elf_size);

So pfengfw should be set to the offset address of QSPI Nor, for example, pfengfw = "0xa00000" // 10MB address.

1. Missing macro definition：include\spi_flash.h

/* by default ENV use the same parameters than SF command */

#ifndef CONFIG_ENV_SPI_BUS

# define CONFIG_ENV_SPI_BUS CONFIG_SF_DEFAULT_BUS

#endif

#ifndef CONFIG_ENV_SPI_CS

# define CONFIG_ENV_SPI_CS CONFIG_SF_DEFAULT_CS

#endif

#ifndef CONFIG_ENV_SPI_MAX_HZ

# define CONFIG_ENV_SPI_MAX_HZ CONFIG_SF_DEFAULT_SPEED

#endif

#ifndef CONFIG_ENV_SPI_MODE

# define CONFIG_ENV_SPI_MODE CONFIG_SF_DEFAULT_MODE

#endif

1. Other qspi related variables in the code can be referenced

#define PFENG_FW_MAX_ELF_SIZE 0xffffU

#define PFENG_FW_MAX_QSPI_ADDR 0xf0000000U

1. Arch\arm\mach-s32\Kconfig

The QSPI driver is supported by default:

config NXP_S32CC

bool

…

imply DM_SPI

imply DM_SPI_FLASH

2.2 Test

1. Use Flash tool to burn PFE FW to the fixed offset address of QSPI NOR:

Switch RDB3 to download mode, connect the serial port and power supply, run

C:\NXP\S32DS.3.5\S32DS\tools\S32FlashTool\GUI\s32ft.exe, and burn s32g_pfe_class.fw to 0xa00000.

S32G Uboot PFE FW QSPI

7

1. After updating fip.s32, start Uboot and configure pfengfw parameters:

Switch RDB3 to SDcard boot mode, connect the serial port and power supply, stop the serial port terminal in Uboot, and configure pfengfw

Parameters:

Environment size: 2494/8188 bytes

=> setenv pfengfw "0xa00000"

=> sav

Saving Environment to MMC... Writing to MMC(0)... OK

=>

1. Print:

 Default mirror:

Warning: ethernet@4033c000 (eth0) using random MAC address - 36:1e:e2:b4:ea:a9

eth0: ethernet@4033c000

Found PFE version 0x0101 (S32G3)

pfeng pfeng-base: Uploading CLASS firmware

pfeng pfeng-base: EMAC0 block was initialized

pfeng pfeng-base: EMAC1 block was initialized

pfeng pfeng-base: EMAC2 block was initialized

pfeng pfeng-base: Enabling the CLASS block

pfeng pfeng-base: PFE Platform started successfully (mask: 7)

s32cc_serdes_phy serdes@44180000: Using mode 3 for SerDes subsystem

s32cc_serdes_phy serdes@44180000: Unstable RX detected on XPCS1

s32cc_serdes_phy serdes@44180000: Unstable RX detected on XPCS0

, eth1: pfe0, eth2: pfe1, eth3: pfe2

Remove s32g_pfe_class.fw from the default image:

Warning: ethernet@4033c000 (eth0) using random MAC address - c2:5a:fc:70:04:bc

eth0: ethernet@4033c000

Found PFE version 0x0101 (S32G3)

PFEng firmware file 'mmc@0:1:s32g_pfe_class.fw' loading failed: -2

Found PFE version 0x0101 (S32G3)

PFEng firmware file 'mmc@0:1:s32g_pfe_class.fw' loading failed: -2

Found PFE version 0x0101 (S32G3)

PFEng firmware file 'mmc@0:1:s32g_pfe_class.fw' loading failed: -2

After placing FW in QSPI Nor:

Found PFE version 0x0101 (S32G3)

SF: Detected mx25uw51245g with page size 256 Bytes, erase size 64 KiB, total 64 MiB

pfeng pfeng-base: Uploading CLASS firmware

pfeng pfeng-base: EMAC0 block was initialized

pfeng pfeng-base: EMAC1 block was initialized

pfeng pfeng-base: EMAC2 block was initialized

pfeng pfeng-base: Enabling the CLASS block

pfeng pfeng-base: PFE Platform started successfully (mask: 7)

s32cc_serdes_phy serdes@44180000: Using mode 3 for SerDes subsystem

s32cc_serdes_phy serdes@44180000: Unstable RX detected on XPCS1

s32cc_serdes_phy serdes@44180000: Unstable RX detected on XPCS0

, eth1: pfe0, eth2: pfe1, eth3: pfe2

[eburley]

I've got the pfe firmware to load from qspi.

Now I need to enable pfe0.

pfeng pfeng-base: Uploading CLASS firmware
pfeng pfeng-base: EMAC0 not used, skipped
pfeng pfeng-base: EMAC1 block was initialized
pfeng pfeng-base: EMAC2 block was initialized
pfeng pfeng-base: Enabling the CLASS block

Shouldn't this be using the device tree from Linux?

When I update my dts file and rebuild my image, the fip.s32-sdcard file is not getting updated.

What do I need to do to get EMAC0 initialized?

[Johnli]

my uboot log on RDB3 as follows, pls check the bsp rm for the uboot configuration of PFE:

NOTICE: Reset status: Power-On Reset
NOTICE: BL2: v2.10.0 (release):bsp41.0-2.10
NOTICE: BL2: Built : 03:12:33, Jul 19 2024
NOTICE: BL2: Booting BL31

U-Boot 2022.04-dirty (Jul 19 2024 - 12:03:42 +0900)

SoC: NXP S32G399A rev. 1.1
CPU: ARM Cortex-A53 r0p4 @ max 1300 MHz
Model: NXP S32G399A-RDB3
DRAM: 3.5 GiB
Core: 306 devices, 25 uclasses, devicetree: board
MMC: FSL_SDHC: 0
Loading Environment from MMC... OK
s32cc_serdes_phy serdes@40480000: Using mode 0 for SerDes subsystem
pci_s32cc pcie@40400000: Configuring as RootComplex
pci_s32cc pcie@40400000: Failed to get link up
In: serial@401c8000
Out: serial@401c8000
Err: serial@401c8000
Board revision: RDB3 Revision F
PCIe: BusDevFun VendorId DeviceId Device Class Sub-Class
__________________________________________________________________________
pcie@40400000 RootComplex
| `-- 01:00.00 0x1957 0x4300 Bridge device 0x04
Net: eth0: ethernet@4033c000
Found PFE version 0x0101 (S32G3)
SF: Detected mx25uw51245g with page size 256 Bytes, erase size 64 KiB, total 64 MiB
pfeng pfeng-base: Uploading CLASS firmware
pfeng pfeng-base: EMAC0 block was initialized
pfeng pfeng-base: EMAC1 block was initialized
pfeng pfeng-base: EMAC2 block was initialized
pfeng pfeng-base: Enabling the CLASS block
pfeng pfeng-base: PFE Platform started successfully (mask: 7)
s32cc_serdes_phy serdes@44180000: Using mode 3 for SerDes subsystem
s32cc_serdes_phy serdes@44180000: Unstable RX detected on XPCS1
s32cc_serdes_phy serdes@44180000: Unstable RX detected on XPCS0
, eth1: pfe0, eth2: pfe1, eth3: pfe2
Hit any key to stop autoboot: 0
=> pri
baudrate=115200
board_rev=F
boot_mtd=booti
bootargs=root=/dev/ram rw earlycon loglevel=7
bootcmd=mmc dev ${mmcdev}; if mmc rescan; then if run loadimage; then run mmcboot; fi; fi
bootdelay=2
console=ttyLF0
eth1addr=00:04:9f:be:ef:00
eth2addr=00:04:9f:be:ef:01
eth3addr=00:04:9f:be:ef:02
ethaddr=a6:70:59:8b:53:cb
fdt_addr=0x83000000
fdt_enable_hs400=fdt addr ${fdt_addr}; fdt rm /soc/mmc no-1-8-v; fdt resize;
fdt_file=s32g399a-rdb3.dtb
fdt_high=0xffffffffffffffff
fdt_override=;
fdtcontroladdr=ff891000
flashboot=echo Booting from flash...; run flashbootargs;mtd read Kernel ${loadaddr};mtd read DTB ${fdt_addr};mtd read Rootfs ${ramdisk_addr};${boot_mtd} ${loadaddr} ${ramdisk_addr} ${fdt_addr};
flashbootargs=setenv bootargs console=${console},${baudrate} root=/dev/ram rw earlycon ;setenv flashsize 0x04000000;
hwconfig=serdes0:mode=pcie,clock=ext;pcie0:mode=rc;serdes1:mode=xpcs0&xpcs1,clock=ext,fmhz=125;xpcs1_0:speed=2G5;xpcs1_1:speed=2G5
image=Image
initrd_high=0xffffffffffffffff
ipaddr=10.0.0.100
loadaddr=0x80000000
loadfdt=fatload mmc ${mmcdev}:${mmcpart} ${fdt_addr} ${fdt_file}; run fdt_override;
loadimage=fatload mmc ${mmcdev}:${mmcpart} ${loadaddr} ${image}
loadtftpfdt=tftp ${fdt_addr} ${fdt_file};
loadtftpimage=tftp ${loadaddr} ${image};
mmcargs=setenv bootargs console=${console},${baudrate} root=${mmcroot} earlycon
mmcboot=echo Booting from mmc ...; run mmcargs; if run loadfdt; then run fdt_fixups; ${boot_mtd} ${loadaddr} - ${fdt_addr}; else echo WARN: Cannot load the DT; fi;
mmcdev=0
mmcpart=1
mmcroot=/dev/mmcblk0p2 rootwait rw
netargs=setenv bootargs console=${console},${baudrate} root=/dev/nfs ip=dhcp nfsroot=${serverip}:${nfsroot},v3,tcp earlycon
netboot=echo Booting from net ...; run netargs; if test ${ip_dyn} = yes; then setenv get_cmd dhcp; else setenv get_cmd tftp; fi; ${get_cmd} ${image}; if test ${boot_fdt} = yes || test ${boot_fdt} = try; then if ${get_cmd} ${fdt_addr} ${fdt_file}; then ${boot_mtd} ${loadaddr} - ${fdt_addr}; else if test ${boot_fdt} = try; then ${boot_mtd}; else echo WARN: Cannot load the DT; fi; fi; else ${boot_mtd}; fi;
netmask=255.255.255.0
nfsboot=echo Booting from net using tftp and nfs...; run nfsbootargs;run loadtftpimage; run loadtftpfdt;${boot_mtd} ${loadaddr} - ${fdt_addr};
nfsbootargs=setenv bootargs console=${console},${baudrate} root=/dev/nfs rw ip=${ipaddr}:${serverip}::${netmask}::eth0:off nfsroot=${serverip}:/tftpboot/rfs,nolock,v3,tcp earlycon
pfe1_phy_addr=8
pfengfw=0xa00000
ramdisk_addr=0x90000000
script=boot.scr
serverip=10.0.0.1
stderr=serial@401c8000
stdin=serial@401c8000
stdout=serial@401c8000

Environment size: 2592/8188 bytes
=>

[eburley]

Does the fip file contain the dtb from linux?

How do I force atf to rebuild the fip when I change my dts file?

[eburley]

U-Boot is definately using the devicetree data from arm-trusted-firmware, specifically the arm-trusted-firmware/fdts/<custom>.dtsi file is being used.

I am trying to use a KSZ9131 on PFE0 that was on GMAC0 on the Rev A of the board (and working).  I copied the settings to pfe_netif0.

When I ping, it attempts on gmac0 and fails (it should), but doesn't attempt pfe0.

If I disable gmac0 in the devicetree, atf doesn't run at all.

Output:

NOTICE: BL2: v2.10.0 (release):bsp41.0-2.10-dirty
NOTICE: BL2: Built : 09:32:10, May 27 2024
NOTICE: BL2: Booting BL31

U-Boot 2022.04+g4744d0e2c8+p0 (May 27 2024 - 09:33:20 +0000)

SoC: NXP S32G399A rev. 1.1
CPU: ARM Cortex-A53 r0p4 @ max 1300 MHz
Model: NXP S32G399A xxxxxxxxx CGM
DRAM: 3.5 GiB
Core: 305 devices, 25 uclasses, devicetree: board
MMC: FSL_SDHC: 0
Loading Environment from MMC... OK
s32cc_serdes_phy serdes@40480000: Using mode 2 for SerDes subsystem
s32cc_serdes_phy serdes@40480000: XPCS1 is in reset
s32cc_serdes_phy serdes@40480000: XPCS init failed
pci_s32cc pcie@40400000: Failed to get PHY 'serdes_lane0'
s32cc_serdes_phy serdes@44180000: Using mode 2 for SerDes subsystem
s32cc_serdes_phy serdes@44180000: XPCS1 is in reset
s32cc_serdes_phy serdes@44180000: XPCS init failed
pci_s32cc pcie@44100000: Failed to get PHY 'serdes_lane0'
In: serial@401c8000
Out: serial@401c8000
Err: serial@401c8000
Board revision: xxxxxxxxx Rev B
Net:
Warning: ethernet@4033c000 (eth0) using random MAC address - b2:f8:e3:e6:66:de
eth0: ethernet@4033c000
Found PFE version 0x0101 (S32G3)
SF: Detected mx25uw51245g with page size 256 Bytes, erase size 64 KiB, total 64 MiB
pfeng pfeng-base: Uploading CLASS firmware
pfeng pfeng-base: EMAC0 block was initialized
pfeng pfeng-base: EMAC1 not used, skipped
pfeng pfeng-base: EMAC2 not used, skipped
pfeng pfeng-base: Enabling the CLASS block
pfeng pfeng-base: PFE Platform started successfully (mask: 1)
, eth1: pfe0
Hit any key to stop autoboot: 0
switch to partitions #0, OK
mmc0(part 0) is current device
** No partition table - mmc 0 **
Couldn't find partition mmc 0:1
Can't set block device
=> setenv ipaddr 192.168.254.197
=> setenv serverip 192.168.254.199
=> ping 192.168.254.197
Could not get PHY for mdio@4033c200: addr 1
dm_eth_phy_connect() failed. no phy foundFAILED: 0Using ethernet@4033c000 device

ARP Retry count exceeded; starting again
ping failed; host 192.168.254.197 is not alive

Device tree settings in fdts/<custom>.dtsi :

&pfe {
status = "okay";
pinctrl-0 = <&pfe0_mdio_pins>, <&pfe0_pins>;
pinctrl-1 = <&pfe0_mdio_pins>;
pinctrl-names = "default", "pfe_sgmii";
};

&pfe_mdio0 {
status = "okay";
#address-cells = <1>;
#size-cells = <0>;
/* KSZ9131 */
mdio_b_phy1: ethernet-phy@1 {
max-speed = <1000>;
reg = <1>;
};
};

&pfe_netif0 {
phy-mode = "rgmii";
phy-handle = <&mdio_b_phy1>;
};

Any advice will be appreciated!

[eburley]

The last issue had to do with initializing the pfe0 clocks in the bootloader.

Now pfe0 is working, but I cannot run the kernel:

Load address: 0x83000000
Loading: ###
5.1 MiB/s
done
Bytes transferred = 43037 (a81d hex)
## Flattened Device Tree blob at 83000000
Booting using the fdt blob at 0x83000000
Using Device Tree in place at 0000000083000000, end 000000008300d81c

Starting kernel ...

clk pfe0_tx_rgmii already disabled
clk pfe0_rx_rgmii already disabled
"Synchronous Abort" handler, esr 0x02000000

Is the kernel trying to re-initialize pfe0, which is in use?

Wouid pre-slave mode work?

[eburley]

I've got it to the point where I "run nfsboot" and the device tree and kernel loads. 

When the kernel starts, it fails to mount the rootfs.  Probably because pfe0 is not back up yet.

Is there a way to fix this?

When the kernel boots, it fails here:

[ 115.678056] VFS: Unable to mount root fs via NFS.
[ 115.682434] devtmpfs: mounted
[ 115.686193] Freeing unused kernel memory: 2496K
[ 115.690379] Run /sbin/init as init process
[ 115.694512] Run /etc/init as init process
[ 115.698438] Run /bin/init as init process
[ 115.702435] Run /bin/sh as init process
[ 115.706251] Kernel panic - not syncing: No working init found. Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance.
[ 115.720324] SMP: stopping secondary CPUs
[ 115.724235] Kernel Offset: disabled
[ 115.727700] CPU features: 0x1,00000008,00020000,1000420b
[ 115.732994] Memory Limit: none
[ 115.736036] ---[ end Kernel panic - not syncing: No working init found. Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance. ]---