ls1043 DPAA and DPDK

bhargavjayswal · ‎09-14-2023

I have build kernel (linux-qoriq - kenel version 6.1) and core-minimal-rootfs for ls1043ARDB using yocto base layerscape SDK and also enabled FMC and dpdk packages. I had refere Layerscape_Linux_distribution_POC_user_guide (LLDPU0G_RevL6.1.1_1.0.0_23.08.pdf) for setup and cross-compilation process.

I am trying to integrate DPAA network engine with DPDK but facing problem to bind ethernet interface with DPDK and set IPv4 address for those interfaces.

I have boot through fsl-ls1043a-rdb-usdpaa.dtb and run below command after getting control as root login.

>> fmc -x

>> fmc -c /usr/share/dpdk/dpaa/usdpaa_config_ls1043.xml -p /usr/share/dpdk/dpaa/usdpaa_policy_hash_ipv4_1queue.xml

When I run dpdk-tools i can see log and observe that 4 interface configured in dtb is available and rest 2 interface are attached with linux network stack as below.

================================================================================

root@ls1043ardb:~ dpdk-testpmd

EAL: Detected CPU lcores: 4
EAL: Detected NUMA nodes: 1
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: lsx_pciep:SoC(0x87920000) not supported
EAL: Scan for (lsx_pciep) bus failed.
EAL: Selected IOVA mode 'PA'
EAL: No available 32768 kB hugepages reported
EAL: No available 64 kB hugepages reported
EAL: No available 1048576 kB hugepages reported
EAL: VFIO support initialized
Nov 8 14:44:12 ls1043ardb daemon.info dpdk-testpmd[349]: EAL: VFIO support initialized
EAL: DPAA Bus Detected
Nov 8 14:44:12 ls1043ardb daemon.notice dpdk-testpmd[349]: EAL: DPAA Bus Detected
PMD: net: dpaa: fm1-mac1: 00:04:9F:04:F0:D4
Nov 8 14:44:12 ls1043ardb daemon.info dpdk-testpmd[349]: PMD: net: dpaa: fm1-mac1: 00:04:9F:04:F0:D4
PMD: net: dpaa: fm1-mac2: 00:04:9F:04:F0:D5
Nov 8 14:44:12 ls1043ardb daemon.info dpdk-testpmd[349]: PMD: net: dpaa: fm1-mac2: 00:04:9F:04:F0:D5
PMD: net: dpaa: fm1-mac5: 00:04:9F:04:F0:D8
Nov 8 14:44:12 ls1043ardb daemon.info dpdk-testpmd[349]: PMD: net: dpaa: fm1-mac5: 00:04:9F:04:F0:D8
PMD: net: dpaa: fm1-mac6: 00:04:9F:04:F0:D9
Nov 8 14:44:12 ls1043ardb daemon.info dpdk-testpmd[349]: PMD: net: dpaa: fm1-mac6: 00:04:9F:04:F0:D9
PMD: net: dpaa: fm1-mac9: 00:04:9F:04:F0:DA
Nov 8 14:44:12 ls1043ardb daemon.info dpdk-testpmd[349]: PMD: net: dpaa: fm1-mac9: 00:04:9F:04:F0:DA
PMD: dpaa_sec-1 cryptodev init
Nov 8 14:44:13 ls1043ardb daemon.info dpdk-testpmd[349]: PMD: dpaa_sec-1 cryptodev init
PMD: dpaa_sec-2 cryptodev init
Nov 8 14:44:13 ls1043ardb daemon.info dpdk-testpmd[349]: PMD: dpaa_sec-2 cryptodev init
PMD: dpaa_sec-3 cryptodev init
Nov 8 14:44:13 ls1043ardb daemon.info dpdk-testpmd[349]: PMD: dpaa_sec-3 cryptodev init
PMD: dpaa_sec-4 cryptodev init
Nov 8 14:44:13 ls1043ardb daemon.info dpdk-testpmd[349]: PMD: dpaa_sec-4 cryptodev init
TELEMETRY: No legacy callbacks, legacy socket not created
Nov 8 14:44:13 ls1043ardb daemon.warn dpdk-testpmd[349]: TELEMETRY: No legacy callbacks, legacy socket not created
EAL: Error - exiting with code: 1
Cause: Nov 8 14:44:13 ls1043ardb daemon.crit dpdk-testpmd[349]: EAL: Error - exiting with code: 1 Cause:
No cores defined for forwarding
Check the core mask argument
Nov 8 14:44:13 ls1043ardb daemon.crit dpdk-testpmd[349]: No c[ 2639.670223] audit: type=1701 audit(1667918653.412:24): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=349 comm="dpdk-1
ores defined for forwarding Check the core mask argument
[ 2643.058669] USDPAA release: 16417 FQIDs
[ 2643.246818] USDPAA release: 5 CGRIDs
Segmentation fault

================================================================================

root@ls1043ardb:~ ifconfig
eth0 Link encap:Ethernet HWaddr 00:04:9F:04:F0:D6
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Memory:1ae4000-1ae4fff

eth1 Link encap:Ethernet HWaddr 00:04:9F:04:F0:D7
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Memory:1ae6000-1ae6fff

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000

================================================================================

Below bind command is not working

root@ls1043ardb:~ dpdk-devbind.py --bind=uio_pci_generic 0000:01:00.0 root@ls1043ardb:~ dpdk-devbind.py --bind=uio_pci_generic fm1-mac6 root@ls1043ardb:~ dpdk-devbind.py --bind=uio_pci_generic eth0

================================================================================

My question are below.

1. How I can bind eth0 and eth1 with DPDK and remove from network stack ?

2. How to set ip address of all 6 interface in DPDK mode ?

3 How to check ping response of all 6 interface in DPDK mode?

4. how I can forward packet from one ethernet interface to another one interface using DPDK ?

5. What is optimal way todo packet forward using DPAA network engine ?

6 which driver name should i have to use in below command

>> dpdk-devbind.py --bind=driver_name eth0

SebastianG · ‎09-20-2023

Hello @bhargavjayswal,

1. How I can bind eth0 and eth1 with DPDK and remove from network stack ?

For this purpose you can use ethtool and dpdk_devbind.py

2. How to set ip address of all 6 interface in DPDK mode ?

For set the ip address you can use the tool ifconfig

3 How to check ping response of all 6 interface in DPDK mode?

For this question you can do ping between target and host to test the 6 interfaces

4. how I can forward packet from one ethernet interface to another one interface using DPDK ?

In the following link you can find a way to forward packets between eth0 and eth1:

https://doc.dpdk.org/guides-20.11/sample_app_ug/skeleton.html

5. What is optimal way todo packet forward using DPAA network engine ?

You can create a object with DPNI using the DPAA

6 which driver name should I have to use in below command

In the following post I will tell you the name of the driver

>> dpdk-devbind.py --bind=driver_name eth0

Regards,

Sebastian

bhargavjayswal · ‎09-21-2023

Hi Sebastian,

Thanks for quick reply.

I can't see network interface in output of "dpdk_devbind.py -s". So how to find network interface list. Request you to share exact process to get network interface detect through DPDK utilities . If any setup or command require to execute before DPDK then please share with us.

I already ask this question at below link and also attached the log.

https://community.nxp.com/t5/Processor-Expert-Software/DPDK-unable-to-list-DPAA-network-interface-in...

Regards,

Bhargav Jayswal

SebastianG · ‎09-15-2023

Hello @bhargavjayswal,

I would like to inform you that I'm working on your question, I will let you know as soon as I have an update.

Thank you so much for your patience

Regards,

Sebastian

bhargavjayswal · ‎09-19-2023

Hi Sebastian,

Thank you for support. Have you got chance to look in to my queries ? Since LS1024 processor support will stop soon, so we are changing our entire product line to LS1043. So hope you understand how much critical it is for us. Hope you will provide an solution as soon as possible.

Regards,

Bhargav Jayswal

hemantagrawal · ‎10-19-2023

dpdk_bind only work for PCI based ports and not for DPAA (fman) ports - please follow the LSDK userguide section:

https://docs.nxp.com/bundle/GUID-C3A436DA-E944-4F73-9811-2335DEBD04D6/page/GUID-DD36538C-486D-43D4-8...
https://docs.nxp.com/bundle/GUID-C3A436DA-E944-4F73-9811-2335DEBD04D6/page/GUID-24522929-0187-48ED-A...
https://docs.nxp.com/bundle/GUID-C3A436DA-E944-4F73-9811-2335DEBD04D6/page/GUID-B386D634-B9B1-4980-A...

also check: https://github.com/NXP/dpdk/blob/19.11-qoriq/nxp/README_dpaa1_shared_mac for shared mode port. i.e. shared port between kernel and userspace.

bhargavjayswal · ‎10-27-2023

Thanks Hemant,

Link provided by you is very helpful. T Dpaa-shared is very helpful and what i am exactly looking for.

Could you please share dts file for ls1043, currently dpaa-shared dts files are only available for ls1046 as below.

1. fsl-ls1046a-rdb-usdpaa-shared.dts

2. fsl-ls1046a-rdb-usdpaa-shared-mac9-only.dts

3. fsl-ls1046a-rdb-usdpaa-shared-mac10.dts

hemantagrawal · ‎10-27-2023

Please compare the changes done for LS1046 and do the same for LS1043.

bhargavjayswal · ‎10-31-2023

Hi @hemantagrawal ,

If I wanted to forward packets of selected ip address and protocol type (UDP type), then what will be the best way to do.

Let's say TCP and UDP packets are received by ls1043ardb system and I want to pass all TCP and UDP packet through kernel network stack , but UDP packets coming from ip address 192.168.1.100 and 192.168.1.200 on RGMII1 wanted to forward on RGMII2 at to destination 192.168.50.100 and 192.168.50.200 respectively then how it can be achieved through user-space.

Similarly wanted to forward UDP packets coming from RGMII2 with IP address 192.168.50.100 and 192.168.50.200 to the destination RGMII1 to 192.168.1.100 and 192.168.1.20 using user-space , where apart from this TCP and UDP packets pass through kernel network stack

Regards,

BhargavJayswal

hemantagrawal · ‎11-02-2023

Multiple options:

1.you can receive all UDP packets in userspace. Use l2fwd to forward the relavant packets. Send unwanted packets back to Linux using Tun/Tap or KNI interface.
https://github.com/NXP/dpdk/blob/19.11-qoriq/nxp/README_pkt_split_app_demo
(ignore the FMAN Micro-code and OL port type)
https://github.com/NXP/dpdk/tree/19.11-qoriq/examples/pkt_split_app

2. You will create RGMII1 and RGMII2 as shared interfaces. Modify the policy file statically to send the packets to DPDK VSP/queues for the given IP address
you can refer to https://www.nxp.com/docs/en/application-note/AN4760.pdf to understand how to modify the fmc vsp_classif files for the same