enabling pfe module for userspace ls1012

[lhazim-c]

I am trying to bring up the phys on a custom board using the ls1012 processor, it has been giving me a bit of trouble. the physical link appears to be good, with auto negotiation succeeding but the MAC layer appears to be failing. I am able to bring up the interfaces if I assign them static ips, however they are unable to find anyone else on the network. If I set them to use dhcp it sends dhcp discoveries but the dhcp server on the other side never receives them.  I am using LSDK 21.08 to build the images, using the machine type ls1012afwy, with changes to the linux device tree I will paste the node entries below. 

&pfe {
	status = "okay";
	#address-cells = <1>;
	#size-cells = <0>;

	pfe_mac0: ethernet@0 {
		compatible = "fsl,pfe-gemac-port";
		#address-cells = <1>;
		#size-cells = <0>;
		reg = <0x0>;	/* GEM_ID */
		fsl,gemac-bus-id = <0x0>;	/* BUS_ID */
		fsl,mdio-mux-val = <0x0>;
		phy-mode = "sgmii";
		phy-handle = <&sgmii_phy0>;
	};

	pfe_mac1: ethernet@1 {
		compatible = "fsl,pfe-gemac-port";
		#address-cells = <1>;
		#size-cells = <0>;
		reg = <0x1>;	/* GEM_ID */
		fsl,gemac-bus-id = <0x1>;	/* BUS_ID */
		fsl,mdio-mux-val = <0x0>;
		phy-mode = "sgmii";
		phy-handle = <&sgmii_phy1>;
	};

	mdio@0 {
		#address-cells = <1>;
		#size-cells = <0>;

		sgmii_phy0: ethernet-phy@0 {
			reg = <0x0>;
			fsl,mdio-phy-mask = <0xFFFFFFF9>;
		};

		sgmii_phy1: ethernet-phy@1 {
			reg = <0x1>;
			fsl,mdio-phy-mask = <0xFFFFFFF9>;
		};
	};
};

(is the phy-mask correct? I changed it after seeing many recommendations under other peoples' posts to do so but I'm not sure that it has had any effect)

I went digging into the pfe kernel module code and found the "us" parameter which by default is set to 0, only allow networking for kernel. My attempts to update this value both through kernel defconfig config_cmdline options and u boot defconfig bootargs option have failed. I am not entirely sure that this is the only problem but it seems like a lead. Where is the default value for this parameter specified? 

Here is some relevant dmesg output. My concerns based on these messages is that pfe_eth
_init_one is unable to print out the base address pointer value which seems strange, and also based on that message it appears to be executing this chunk of code, which seems strange: 

	if ((!(pfe_use_old_dts_phy) && !(priv->phy_node)) ||
	    ((pfe_use_old_dts_phy) &&
	      (priv->einfo->phy_flags & GEMAC_NO_PHY))) {
		pr_info("%s: No PHY or fixed-link\n", __func__);
		goto skip_phy_init;
	}

pfe_hw_init
[ 2.569020] CLASS version: 20
[ 2.572014] TMU version: 1011231
[ 2.575273] BMU1 version: 21
[ 2.578172] BMU2 version: 21
[ 2.581080] EGPI1 version: 50
[ 2.584122] EGPI2 version: 50
[ 2.587143] HGPI version: 50
[ 2.590070] HIF version: 10
[ 2.592931] HIF NOPCY version: 10
[ 2.596276] bmu_init(1) done
[ 2.599472] bmu_init(2) done
[ 2.604829] class_init() done
[ 2.617895] tmu_init() done
[ 2.620721] gpi_init(1) done
[ 2.623631] gpi_init(2) done
[ 2.626532] gpi_init(hif) done
[ 2.629630] bmu_enable(1) done
[ 2.632808] bmu_enable(2) done
[ 2.635899] pfe_hif_lib_init
[ 2.639034] pfe_hif_init
[ 2.641603] pfe_hif_alloc_descr
[ 2.644967] pfe_hif_init_buffers
[ 2.648993] pfe_firmware_init
[ 2.652089] CLASS fw of length 53124 bytes loaded from FDT.
[ 2.657752] TMU fw of length 11592 bytes loaded from FDT.
[ 2.663203] pfe_load_elf
[ 2.665766] pe_load_ddr_section: load address(3fb0000) and elf file address(fffffdfffe80891c) rcvd
[ 2.674886] mmc0: new high speed SDHC card at address 5048
[ 2.687524] mmcblk0: mmc0:5048 SD32G 28.8 GiB
[ 2.700815] mmcblk0: p1 p2 p3 p4
[ 2.713483] PFE binary version: pfe_ls1012a_slowpath_10
[ 2.718811] pfe_load_elf
[ 2.722297] pfe_ctrl_init
[ 2.724941] pfe_ctrl_init finished
[ 2.728372] pfe_eth_init
[ 2.730914] pfe_eth_init: lhazim: cbus_emac_base[0]=13200000
[ 2.730931] pfe 4000000.pfe: pfe_eth_mdio_init: mdc_div: 64, phy_mask: fffffff9
[ 2.743970] usb 1-1: new high-speed USB device number 2 using xhci-hcd
[ 2.751096] libphy: ls1012a MDIO Bus: probed
[ 2.757195] pfe 4000000.pfe: pfe_eth_mdio_init: mdc_div: 64, phy_mask: fffffff9
[ 2.766153] libphy: ls1012a MDIO Bus: probed
[ 2.772009] pfe_phy_init interface 4
[ 2.776436] pfe 4000000.pfe eth0: pfe_eth_init_one: lhazim: test debug print
[ 2.783534] pfe 4000000.pfe eth0: pfe_eth_init_one: created interface, baseaddr: (____ptrval____)
[ 2.793812] pfe_phy_init interface 4
[ 2.798214] pfe 4000000.pfe eth1: pfe_eth_init_one: lhazim: test debug print
[ 2.805328] pfe 4000000.pfe eth1: pfe_eth_init_one: created interface, baseaddr: (____ptrval____)
[ 2.814268] pfe_debugfs_init

any assistance would be greatly appreciated ~

[LFGP]

On LS1012AFRWY, the Ethernet PHY driver does not work well in U-Boot.
Open LSDK 21.08, which may have impact on this issue, so can you please try to use the LLDP 6.1.55 instead LSDK 21.08

[lhazim-c]

I am not trying to use the PHY in U-Boot, only in userspace. It appears the fsl,pfe-phy-if-flags property is very important, I was able to get some (but not all) of my interfaces by adding this property to the device tree. Can you please explain this property?

Thanks,

[LFGP]

dear @lhazim-c 

in the next link you will find an example that will help you with your issue

https://community.nxp.com/t5/Layerscape-Knowledge-Base/Packet-Forwarding-Engine-PFE-Ethernet-Ports-S...

[LFGP]

regarding the device tree, please see the provided NXP LS1012ARDB device tree 

https://github.com/nxp-qoriq/linux/blob/lf-6.1.y/arch/arm64/boot/dts/freescale/fsl-ls1012a-rdb.dts

review in your image if the DPDK library is loaded and check if the PFE is enabled in the kernel

[lhazim-c]

Hello, thank you for your reply. I am looking at this device tree and wondering what does this section indicate? 

		sgmii_phy: ethernet-phy@2 {
			reg = <0x2>;
		};

Here are the kernel defconfig lines related to PPFE

CONFIG_FSL_PPFE=y
CONFIG_FSL_PPFE_UTIL_DISABLED=y

I am not sure about the DPDK library, how can I test for that? I don't see any entry for that in the defconfig. A search of my rfs image files does show that the dpdk library is loaded, how can I use this library?

Thanks again. 

[lhazim-c]

I am reading the README of the DPDK library and as mentioned under the section related to the PPFE platform the result of 

cat /sys/module/pfe/parameters/us 

should be 1 and I am instead getting 0 and my attempts to change this value to 1 have not been successful as I have mentioned in the original post.