S32G_Compiled_PFE_into_Kernel

[#1]

This doc explain how to compile PFE driver into kernel to accelerate the network boot time, chinese version:

目录

1    需要的软件，工具与文档... 2

2    目前PFE驱动的情况... 2

3    将PFE Slave驱动编译进内核... 3

3.1  将PFE驱动代码加入内核... 3

3.2  开发Makefile文件... 6

3.3  编译与测试... 8

4    将PFE Master驱动编译进内核... 10

4.1  编译Master工程... 10

4.2  测试... 11

4.3  解决FW的加载问题... 11

Contents

1    Related Materials. 2

2    Current PFE Driver 2

3    Compiled PFE Slave into Kernel 3

3.1  Put the PFE driver source into kernel source. 3

3.2  Develop the Makefile. 6

3.3  Compilation and Testing. 8

4    Compiled PFE Master Driver into Kernel 10

4.1  Compiling the Master driver 10

4.2  Testing. 11

4.3  Solve the FW loading fail issue. 11

[yezhang]

Hello

According to your guidance, we have successfully compiled the PFE server and firmware into the kernel. The loading speed of the PFE driver has indeed increased, but we have discovered an issue where PFe1 and PFe2 set their MAC addresses after 10 seconds. What is the reason for this? And report Driver runtime error at 2.433127 seconds? Does PFE support sleep wake-up?

Thank you! Looking forward to your reply

[ 0.000000] OF: reserved mem: initialized node pfebufs@83200000, compatible id shared-dma-pool
[ 1.699473] pfeng 46000000.pfe: PFEng ethernet driver loading ...
[ 1.705679] pfeng 46000000.pfe: Version: 1.2.0 RC2
[ 1.710557] pfeng 46000000.pfe: Driver commit hash: M4_DRIVER_COMMIT_HASH
[ 1.717464] pfeng 46000000.pfe: Multi instance support: disabled (standalone)
[ 1.724725] pfeng 46000000.pfe: Compiled by: 11.3.0
[ 1.729713] pfeng 46000000.pfe: Cbus addr 0x46000000 size 0x1000000
[ 1.736100] pfeng 46000000.pfe: nxp,fw-class-name: s32g_pfe_class.fw
[ 1.742569] pfeng 46000000.pfe: nxp,fw-util-name: s32g_pfe_util.fw
[ 1.748935] pfeng 46000000.pfe: irq 'hif0' : 37
[ 1.753566] pfeng 46000000.pfe: irq 'hif1' : 38
[ 1.758195] pfeng 46000000.pfe: irq 'hif2' : 39
[ 1.762810] pfeng 46000000.pfe: HIF channels mask: 0x0007
[ 1.768327] pfeng 46000000.pfe: netif name: pfe0
[ 1.773034] pfeng 46000000.pfe: DT mac addr: 00:01:be:be:ef:11
[ 1.778977] pfeng 46000000.pfe: netif(pfe0) mode: std
[ 1.784124] pfeng 46000000.pfe: netif(pfe0) EMAC: 0
[ 1.789096] pfeng 46000000.pfe: netif(pfe0) HIFs: count 1 map 01
[ 1.795222] pfeng 46000000.pfe: EMAC0 interface mode: 4
[ 1.800689] pfeng 46000000.pfe: netif name: pfe1
[ 1.805400] pfeng 46000000.pfe: DT mac addr: 00:01:be:be:ef:22
[ 1.811342] pfeng 46000000.pfe: netif(pfe1) mode: std
[ 1.816487] pfeng 46000000.pfe: netif(pfe1) EMAC: 1
[ 1.821455] pfeng 46000000.pfe: netif(pfe1) HIFs: count 1 map 02
[ 1.827613] pfeng 46000000.pfe: EMAC1 interface mode: 4
[ 1.833040] pfeng 46000000.pfe: netif name: pfe2
[ 1.837747] pfeng 46000000.pfe: DT mac addr: 00:01:be:be:ef:33
[ 1.843689] pfeng 46000000.pfe: netif(pfe2) mode: std
[ 1.848834] pfeng 46000000.pfe: netif(pfe2) EMAC: 2
[ 1.853801] pfeng 46000000.pfe: netif(pfe2) HIFs: count 1 map 04
[ 1.859922] pfeng 46000000.pfe: EMAC2 interface mode: 4
[ 1.865374] pfeng 46000000.pfe: PFE port coherency enabled, mask 0x1e
[ 1.872195] pfeng 46000000.pfe: Clocks: sys=300MHz pe=600MHz
[ 1.877988] pfeng 46000000.pfe: Interface selected: EMAC0: 0x4 EMAC1: 0x4 EMAC2: 0x4
[ 1.886542] pfeng 46000000.pfe: PFE controller reset done
[ 1.892130] pfeng 46000000.pfe: TX clock on EMAC0 for interface sgmii installed
[ 1.899632] pfeng 46000000.pfe: RX clock on EMAC0 for interface sgmii installed
[ 1.907158] pfeng 46000000.pfe: TX clock on EMAC1 for interface sgmii installed
[ 1.914657] pfeng 46000000.pfe: RX clock on EMAC1 for interface sgmii installed
[ 1.922167] pfeng 46000000.pfe: TX clock on EMAC2 for interface sgmii installed
[ 1.929668] pfeng 46000000.pfe: RX clock on EMAC2 for interface sgmii installed
[ 1.937287] pfeng 46000000.pfe: assigned reserved memory node pfebufs@83100000
[ 1.944700] pfeng 46000000.pfe: assigned reserved memory node pfebufs@83180000
[ 1.952114] pfeng 46000000.pfe: assigned reserved memory node pfebufs@83200000
[ 1.959494] pfeng 46000000.pfe: assigned reserved memory node pfebufs@835e0000
[ 1.966964] pfeng 46000000.pfe: Firmware: CLASS s32g_pfe_class.fw [46564 bytes]
[ 1.974406] pfeng 46000000.pfe: Firmware: UTIL s32g_pfe_util.fw [20332 bytes]
[ 1.981677] pfeng 46000000.pfe: PFE CBUS p0x46000000 mapped @ v0xffffffc00b000000 (0x1000000 bytes)
[ 1.990884] pfeng 46000000.pfe: HW version 0x101
[ 1.995607] pfeng 46000000.pfe: Silicon S32G3
[ 2.000048] pfeng 46000000.pfe: Fail-Stop mode disabled
[ 2.008218] pfeng 46000000.pfe: PFE_ERRORS:Parity instance created
[ 2.014521] pfeng 46000000.pfe: PFE_ERRORS:Watchdog instance created
[ 2.020995] pfeng 46000000.pfe: PFE_ERRORS:Bus Error instance created
[ 2.027553] pfeng 46000000.pfe: PFE_ERRORS:FW Fail Stop instance created
[ 2.034378] pfeng 46000000.pfe: PFE_ERRORS:Host Fail Stop instance created
[ 2.041375] pfeng 46000000.pfe: PFE_ERRORS:Fail Stop instance created
[ 2.047936] pfeng 46000000.pfe: PFE_ERRORS:ECC Err instance created
[ 2.054322] pfeng 46000000.pfe: BMU1 buffer base: p0xc0000000
[ 2.060273] pfeng 46000000.pfe: BMU2 buffer base: p0x83100000 (0x80000 bytes)
[ 2.068915] pfeng 46000000.pfe: register IRQ 41 by name 'PFE BMU IRQ'
[ 2.075661] pfeng 46000000.pfe: BMU_EMPTY_INT (BMU @ p0x(____ptrval____)). Pool ready.
[ 2.075664] pfeng 46000000.pfe: Firmware .elf detected
[ 2.075675] pfeng 46000000.pfe: Uploading CLASS firmware
[ 2.083732] pfeng 46000000.pfe: BMU_EMPTY_INT (BMU @ p0x(____ptrval____)). Pool ready.
[ 2.088966] pfeng 46000000.pfe: Selected FW loading OPs to load 8 PEs in parallel
[ 2.130712] pfeng 46000000.pfe: pfe_ct.h file version"92367c0e25f21f49217a9b08168ad2c8"
[ 2.154221] pfeng 46000000.pfe: [FW VERSION] 1.5.0, Build: Nov 23 2022, 09:48:33 (), ID: 0x31454650
[ 2.163755] pfeng 46000000.pfe: Uploading UTIL firmware
[ 2.169076] pfeng 46000000.pfe: Selected FW loading OPs to load 1 PEs in parallel
[ 2.179283] pfeng 46000000.pfe: pfe_ct.h file version"92367c0e25f21f49217a9b08168ad2c8"
[ 2.190990] pfeng 46000000.pfe: VLAN ID incorrect or not set. Using default VLAN ID = 0x01.
[ 2.199499] pfeng 46000000.pfe: VLAN stats size incorrect or not set. Using default VLAN stats size = 20.
[ 2.209335] pfeng 46000000.pfe: Fall-back bridge domain @ 0x20000a34 (class)
[ 2.216510] pfeng 46000000.pfe: Default bridge domain @ 0x20000a2c (class)
[ 2.223861] pfeng 46000000.pfe: Routing table created, Hash Table @ p0x83180000, Pool @ p0x83188000 (65536 bytes)
[ 2.234571] pfeng 46000000.pfe: Feature err051211_workaround: DISABLED
[ 2.242853] pfeng 46000000.pfe: MDIO bus 0 disabled: Not found in DT
[ 2.249331] pfeng 46000000.pfe: MDIO bus 1 disabled: Not found in DT
[ 2.259052] pfeng 46000000.pfe: MDIO bus 2 enabled
[ 2.264362] pfeng 46000000.pfe: HIF0 enabled
[ 2.269056] pfeng 46000000.pfe: HIF1 enabled
[ 2.273765] pfeng 46000000.pfe: HIF2 enabled
[ 2.278117] pfeng 46000000.pfe: HIF3 not configured, skipped
[ 2.284811] pfeng 46000000.pfe pfe0: registered
[ 2.289456] pfeng 46000000.pfe pfe0: Subscribe to HIF0
[ 2.295133] pfeng 46000000.pfe pfe0: Enable HIF0
[ 2.300042] pfeng 46000000.pfe pfe0: setting MAC addr: 00:01:be:be:ef:11
[ 2.306889] pfeng 46000000.pfe pfe0: PTP HW addend 0x80000000, max_adj configured to 46566128 ppb
[ 2.315924] pfeng 46000000.pfe: IEEE1588: Input Clock: 200000000Hz, Output: 100000000Hz, Accuracy: 10.0ns
[ 2.326149] pfeng 46000000.pfe pfe0: Registered PTP HW clock successfully on EMAC0
[ 2.334539] pfeng 46000000.pfe pfe1: registered
[ 2.339160] pfeng 46000000.pfe pfe1: Subscribe to HIF1
[ 2.344814] pfeng 46000000.pfe pfe1: Enable HIF1
[ 2.349689] pfeng 46000000.pfe pfe1: setting MAC addr: 00:01:be:be:ef:22
[ 2.356529] pfeng 46000000.pfe pfe1: PTP HW addend 0x80000000, max_adj configured to 46566128 ppb
[ 2.365572] pfeng 46000000.pfe: IEEE1588: Input Clock: 200000000Hz, Output: 100000000Hz, Accuracy: 10.0ns
[ 2.375773] pfeng 46000000.pfe pfe1: Registered PTP HW clock successfully on EMAC1
[ 2.384135] pfeng 46000000.pfe pfe2: registered
[ 2.388755] pfeng 46000000.pfe pfe2: Subscribe to HIF2
[ 2.394407] pfeng 46000000.pfe pfe2: Enable HIF2
[ 2.399319] pfeng 46000000.pfe pfe2: setting MAC addr: 00:01:be:be:ef:33
[ 2.406162] pfeng 46000000.pfe pfe2: PTP HW addend 0x80000000, max_adj configured to 46566128 ppb
[ 2.415194] pfeng 46000000.pfe: IEEE1588: Input Clock: 200000000Hz, Output: 100000000Hz, Accuracy: 10.0ns
[ 2.425397] pfeng 46000000.pfe pfe2: Registered PTP HW clock successfully on EMAC2
[ 2.433127] pfeng 46000000.pfe: (PFENG_DEV) event 1 - Driver runtime error: [pfeng-drv.c:864]
[ 10.394090] pfeng 46000000.pfe pfe1: setting MAC addr: 02:48:5a:01:03:20
[ 10.407065] pfeng 46000000.pfe: HIF1 started
[ 10.411380] pfeng 46000000.pfe pfe1: configuring for fixed/sgmii link mode
[ 10.418267] pfeng 46000000.pfe pfe1: Link is Up - 2.5Gbps/Full - flow control off
[ 10.427109] pfeng 46000000.pfe pfe2: setting MAC addr: 00:01:be:be:ef:33
[ 10.439587] pfeng 46000000.pfe: HIF2 started
[ 10.782874] pfeng 46000000.pfe pfe2: PHY [PFEng Ethernet MDIO.2:02] driver [RTL9010AX Ethernet] (irq=POLL)
[ 10.792177] pfeng 46000000.pfe pfe2: configuring for phy/sgmii link mode
[ 10.798923] pfeng 46000000.pfe pfe2: (PFENG_NETDEV) event 1 - Driver runtime error: [pfeng-phylink.c:37] Speed not supported
[ 10.810905] pfeng 46000000.pfe pfe2: Link is Up - 1Gbps/Full - flow control off
[ 11.273105] IPv6: ADDRCONF(NETDEV_CHANGE): pfe1: link becomes ready
[ 11.279623] IPv6: ADDRCONF(NETDEV_CHANGE): pfe2: link becomes ready

[rqyang]

Hi, 我按照上面的文档成功将pfe编译进了内核，但是板卡启动后发现网络功能失效了，pfe0 pfe1和pfe2都没有办法ping通外部。 然后我对比dmesg的的log，发现是xpcs初始化比pfe初始化要晚，于是我修改了pfe的加载时间，将其延后。 板卡起来后pfe1和pfe2能正常通讯，但pfe0还是无法通讯，我对比dmesg的log，一切都很正常，和用ko加载的log没有区别。 我的pfe1和pfe2用的都是sgmii连接 pfe0的话依然是sgmii直连到了SJA1110上面。pfe驱动和fw都是用的1.0.0版本，请问一下有什么可能导致pfe0失效的原因吗？ 感恩！

[Johnli]

is it possible to sja1110 do not lock the clock to s32g?

as follows:

S32G_PFE_Master Demo on M7 This doc explain how to build a PFE master project on M7 and how to integration. chinese version https://community.nxp.com/t5/NXP-Designs-Knowledge-Base/S32G-PFE-Master-Demo-building-on-M7/ta-p/160...

[rqyang]

Hi Johnli

        我感觉像是RX-CLOCK的问题，因为我再查看pfe0的收发包计数时，能看到tx在增加，而rx时钟为0.

        我去查看了芯片手册，看到pfe0的rx-clk用的是clk47

由于pfe使用的是MC_CGM_2,于是我找到相应的寄存器0x44018480

在pfe初始化时增加了读取写入寄存器的值并轮询的操作

这么修改之后板卡内核启动就崩溃了

请问一下这么修改有问题吗？

如果有的话该怎么修改？

没有问题话是什么原因导致崩溃的呢？

best regards

Yang