移植实时Linux方案Xenomai到i.MX ARM64平台 (Enable Xenomai on i.MX ARM64 Platform)

cancel
Showing results for 
Search instead for 
Did you mean: 

移植实时Linux方案Xenomai到i.MX ARM64平台 (Enable Xenomai on i.MX ARM64 Platform)

100% helpful (1/1)

移植实时Linux方案Xenomai到i.MX ARM64平台 (Enable Xenomai on i.MX ARM64 Platform)

    Xenomai is real-time framework, which can run seamlessly side-by-side Linux as a co-kernel system, or natively over mainline Linux kernels (with or without PREEMPT-RT patch). The dual kernel nicknamed Cobalt, is a significant rework of the Xenomai 2.x system. Cobalt implements the RTDM specification for interfacing with real-time device drivers. The native linux version, an enhanced implementation of the experimental Xenomai/SOLO work, is called Mercury. In this environment, only a standalone implementation of the RTDM specification in a kernel module is required, for interfacing the RTDM-compliant device drivers with the native kernel. You can get more detailed information from Home · Wiki · xenomai / xenomai · GitLab  

    I have ported xenomai 3.1 to i.MX Yocto 4.19.35-1.1.0, and currently support ARM64 and test on i.MX8MQ EVK board. I did over night test( 5 real-time threads + GPU SDK test case) and stress test by tool stress-ng on i.MX8MQ EVK board. It looks lile pretty good. Current version (20200730) also support i.MX8MM EVK.

    You need git clone https://gitee.com/zxd2021-imx/xenomai-arm64.git, and git checkout xenomai-4.19.35-1.1.0-20200818 (which inlcudes all patches and bb file) and add the following variable in conf/local.conf before build xenomai by command bitake xenomai. 

XENOMAI_KERNEL_MODE = "cobalt" 

PREFERRED_VERSION_linux-imx = "4.19-${XENOMAI_KERNEL_MODE}"

IMAGE_INSTALL_append += " xenomai"

or
XENOMAI_KERNEL_MODE = "mercury"

PREFERRED_VERSION_linux-imx = "4.19-${XENOMAI_KERNEL_MODE}"

IMAGE_INSTALL_append += " xenomai"

If XENOMAI_KERNEL_MODE = "cobalt", you can build dual kernel version. And If XENOMAI_KERNEL_MODE = "mercury", it is single kernel with PREEMPT-RT patch.

The following is test result by the command (/usr/xenomai/demo/cyclictest -p 99 -t 5 -m -n -i 1000  -l 100000

//Over normal Linux kernel without GPU SDK test case

T: 0 ( 4220) P:99 I:1000 C: 100000 Min: 7 Act: 10 Avg: 9 Max: 23
T: 1 ( 4221) P:99 I:1500 C: 66672 Min: 7 Act: 10 Avg: 10 Max: 20
T: 2 ( 4222) P:99 I:2000 C: 50001 Min: 7 Act: 12 Avg: 10 Max: 81
T: 3 ( 4223) P:99 I:2500 C: 39998 Min: 7 Act: 11 Avg: 10 Max: 29
T: 4 ( 4224) P:99 I:3000 C: 33330 Min: 7 Act: 13 Avg: 10 Max: 26

//Over normal Linux kernel with GPU SDK test case

T: 0 ( 4177) P:99 I:1000 C: 100000 Min: 7 Act: 10 Avg: 11 Max: 51
T: 1 ( 4178) P:99 I:1500 C: 66673 Min: 7 Act: 12 Avg: 10 Max: 35
T: 2 ( 4179) P:99 I:2000 C: 50002 Min: 7 Act: 12 Avg: 11 Max: 38
T: 3 ( 4180) P:99 I:2500 C: 39999 Min: 7 Act: 12 Avg: 11 Max: 42
T: 4 ( 4181) P:99 I:3000 C: 33330 Min: 7 Act: 12 Avg: 11 Max: 36

 

//Cobalt with stress-ng --cpu 4 --io 2 --vm 1 --vm-bytes 512M --timeout 600s --metrics-brief
T: 0 ( 4259) P:50 I:1000 C:3508590 Min:      0 Act:    0 Avg:    0 Max:      42
T: 1 ( 4260) P:50 I:1500 C:2338831 Min:      0 Act:    1 Avg:    0 Max:      36
T: 2 ( 4261) P:50 I:2000 C:1754123 Min:      0 Act:    1 Avg:    1 Max:      42
T: 3 ( 4262) P:50 I:2500 C:1403298 Min:      0 Act:    1 Avg:    1 Max:      45
T: 4 ( 4263) P:50 I:3000 C:1169415 Min:      0 Act:    1 Avg:    1 Max:      22

 

//Cobalt without GPU SDK test case
T: 0 ( 4230) P:50 I:1000 C: 100000 Min: 0 Act: 0 Avg: 0 Max: 4
T: 1 ( 4231) P:50 I:1500 C:   66676 Min: 0 Act: 1 Avg: 0 Max: 4
T: 2 ( 4232) P:50 I:2000 C:   50007 Min: 0 Act: 1 Avg: 0 Max: 8
T: 3 ( 4233) P:50 I:2500 C:   40005 Min: 0 Act: 1 Avg: 0 Max: 3
T: 4 ( 4234) P:50 I:3000 C:   33338 Min: 0 Act: 1 Avg: 0 Max: 5


//Cobalt with GPU SDK test case
T: 0 ( 4184) P:99 I:1000 C:37722968 Min: 0 Act: 1 Avg: 0 Max: 24
T: 1 ( 4185) P:99 I:1500 C:25148645 Min: 0 Act: 1 Avg: 0 Max: 33
T: 2 ( 4186) P:99 I:2000 C:18861483 Min: 0 Act: 1 Avg: 0 Max: 22
T: 3 ( 4187) P:99 I:2500 C:15089187 Min: 0 Act: 1 Avg: 0 Max: 23
T: 4 ( 4188) P:99 I:3000 C:12574322 Min: 0 Act: 1 Avg: 0 Max: 29


//Mercury without GPU SDK test case
T: 0 ( 4287) P:99 I:1000 C:1000000 Min: 6 Act: 7 Avg: 7 Max: 20
T: 1 ( 4288) P:99 I:1500 C:  666667 Min: 6 Act: 9 Avg: 7 Max: 17
T: 2 ( 4289) P:99 I:2000 C:  499994 Min: 6 Act: 8 Avg: 7 Max: 24
T: 3 ( 4290) P:99 I:2500 C:  399991 Min: 6 Act: 9 Avg: 7 Max: 19
T: 4 ( 4291) P:99 I:3000 C:  333322 Min: 6 Act: 8 Avg: 7 Max: 21


//Mercury with GPU SDK test case
T: 0 ( 4222) P:99 I:1000 C:1236790 Min: 6 Act: 7 Avg: 7 Max: 55
T: 1 ( 4223) P:99 I:1500 C:  824518 Min: 6 Act: 7 Avg: 7 Max: 44
T: 2 ( 4224) P:99 I:2000 C:  618382 Min: 6 Act: 8 Avg: 8 Max: 88
T: 3 ( 4225) P:99 I:2500 C:  494701 Min: 6 Act: 7 Avg: 8 Max: 49
T: 4 ( 4226) P:99 I:3000 C:  412247 Min: 6 Act: 7 Avg: 8 Max: 53

//////////////////////////////////////// Update for Yocto L5.4.47 2.2.0  ///////////////////////////////////////////////////////////

New release for Yocto release L5.4.47 2.2.0 and it supports i.MX8M series (8MQ,8MM,8MN and 8MP). You need to git clone https://gitee.com/zxd2021-imx/xenomai-arm64.git,  and git checkout xenomai-5.4.47-2.2.0.

You need to add the following variable in conf/local.conf before build xenomai by command bitake imx-image-multimedia

XENOMAI_KERNEL_MODE = "cobalt" 

PREFERRED_VERSION_linux-imx = "5-${XENOMAI_KERNEL_MODE}"

IMAGE_INSTALL_append += " xenomai"

or
XENOMAI_KERNEL_MODE = "mercury"

PREFERRED_VERSION_linux-imx = "5-${XENOMAI_KERNEL_MODE}"

IMAGE_INSTALL_append += " xenomai"

//////////////////////////////////////// Update for Yocto L5.4.70 2.3.0  ///////////////////////////////////////////////////////////

New release  for Yocto release L5.4.70 2.3.0 and it supports i.MX8M series (8MQ,8MM,8MN and 8MP) and i.MX8QM/QXP. You need to git clone https://gitee.com/zxd2021-imx/xenomai-arm64.git and git checkout xenomai-5.4.70-2.3.0.

Updating:

1, Support i.MX8QM and i.MX8QXP

2, Fix altency's the issue which uses legacy API to get time

 

//////////////////////////////////////// update for Yocto L5.4.70 2.3.2  ///////////////////////////////////////////////////////////

New release for Yocto release L5.4.70 2.3.2. You need to git clone https://gitee.com/zxd2021-imx/xenomai-arm64.git, and git checkout xenomai-5.4.70-2.3.2.

Updating:

1, Enable Xenomai RTDM driver in Linux Kernel

2, Currently CAN, UART, GPIO,  SPI and Ethernet (in debug for RTNet)  are added in Xenomai.

3, Add KERNEL_DEVICETREE += " freescale/imx8mp-rt-evk.dtb " in sources/meta-imx/meta-bsp/conf/machine/imx8mpevk.conf to enable relative device in Xenomai domain, for example rt-imx8mp-flexcan.

 

//////////////////////////////////////// Update for Yocto L5.4.70 2.3.3  ///////////////////////////////////////////////////////////

New release for Yocto release L5.4.70 2.3.3. You need to git clonehttps://gitee.com/zxd2021-imx/xenomai-arm64.git and git checkout xenomai-5.4.70-2.3.3.

Updating:

1, Enable RTNet FEC driver

2, Currently CAN, UART, GPIO,  SPI and Ethernet ( FEC Controller)  are added in Xenomai.

3, Add KERNEL_DEVICETREE += " freescale/imx8mp-rt-evk.dtb " in sources/meta-imx/meta-bsp/conf/machine/imx8mpevk.conf and KERNEL_DEVICETREE += " freescale/imx8mm-rt-ddr4-evk.dtb " in sources/meta-imx/meta-bsp/conf/machine/imx8mmddr4evk.conf to enable rt_fec device in Xenomai domain.

Verifying the network connection by RTnet Ping Between i.MX8M Mini EVK and i.MX8M Plus EVK

a, Setup test environment

1, Connect ENET1 of  i.MX8M Plus EVK (used as a master) and  ENET of i.MX8M Mini EVK (used as a slave) of  to a switch or hub

2, Modify /usr/xenomai/etc/rtnet.conf in i.MX8M Plus EVK board as the following:

@@ -16,7 +16,7 @@ MODULE_EXT=".ko"


# RT-NIC driver
-RT_DRIVER="rt_eepro100"
+RT_DRIVER="rt_fec"
RT_DRIVER_OPTIONS=""

# PCI addresses of RT-NICs to claim (format: 0000:00:00.0)
@@ -30,8 +30,8 @@ REBIND_RT_NICS=""
# The TDMA_CONFIG file overrides these parameters for masters and backup
# masters. Leave blank if you do not use IP addresses or if this station is
# intended to retrieve its IP from the master based on its MAC address.
-IPADDR="10.0.0.1"
-NETMASK=""
+IPADDR="192.168.100.101"
+NETMASK="255.255.255.0"

# Start realtime loopback device ("yes" or "no")
RT_LOOPBACK="yes"
@@ -65,7 +65,7 @@ TDMA_MODE="master"
# Master parameters

# Simple setup: List of TDMA slaves
-TDMA_SLAVES="10.0.0.2 10.0.0.3 10.0.0.4"
+TDMA_SLAVES="192.168.100.102"

# Simple setup: Cycle time in microsecond
TDMA_CYCLE="5000"

3, Modify /usr/xenomai/etc/rtnet.conf in i.MX8M Mini EVK board as the following:

@@ -16,7 +16,7 @@ MODULE_EXT=".ko"


# RT-NIC driver
-RT_DRIVER="rt_eepro100"
+RT_DRIVER="rt_fec"
RT_DRIVER_OPTIONS=""

# PCI addresses of RT-NICs to claim (format: 0000:00:00.0)
@@ -30,8 +30,8 @@ REBIND_RT_NICS=""
# The TDMA_CONFIG file overrides these parameters for masters and backup
# masters. Leave blank if you do not use IP addresses or if this station is
# intended to retrieve its IP from the master based on its MAC address.
-IPADDR="10.0.0.1"
-NETMASK=""
+IPADDR="192.168.100.102"
+NETMASK="255.255.255.0"

# Start realtime loopback device ("yes" or "no")
RT_LOOPBACK="yes"
@@ -59,13 +59,13 @@ STAGE_2_CMDS=""
# TDMA mode of the station ("master" or "slave")
# Start backup masters in slave mode, it will then be switched to master
# mode automatically during startup.
-TDMA_MODE="master"
+TDMA_MODE="slave"


# Master parameters

# Simple setup: List of TDMA slaves
-TDMA_SLAVES="10.0.0.2 10.0.0.3 10.0.0.4"
+TDMA_SLAVES="192.168.100.102"

# Simple setup: Cycle time in microsecond
TDMA_CYCLE="5000"

4, rename imx8mm-rt-ddr4-evk.dtb to imx8mm-ddr4-evk.dtb in /run/media/mmcblk1p1,  rename imx8mp-rt-evk.dtb to imx8mp-evk.dtb in /run/media/mmcblk1p1, and reboot board.

5, Run the below command on i.MX8M Mini EVK board.

cd /usr/xenomai/sbin/
./rtnet start &

5, Run the below command on i.MX8M Plus EVK board.

cd /usr/xenomai/sbin/
./rtnet start &

When you see the log (rt_fec_main 30be0000.ethernet (unnamed net_device) (uninitialized): Link is Up - 100Mbps/Full - flow control rx/tx) and you can run command "./rtroute" to check route table if the slave IP (192.168.100.102) is in route..

b, Verify the network connection using the command below:

./rtping -s 1024 192.168.100.102

//////////////////////////////////////// Update for Yocto L5.10.52 2.1.0  ///////////////////////////////////////////////////////////

New release for Yocto release L10.52 2.1.0. You need to git clonehttps://gitee.com/zxd2021-imx/xenomai-arm64.git and git checkout xenomai-5.10.52-2.1.0.

Updating:

1, Upgrade Xenomai to v3.2

2, Enable Dovetail instead of ipipe.

Copy xenomai-arm64 to <Yocto folder>/sources/meta-imx/meta-bsp/recipes-kernel, and add the following variable in conf/local.conf before build Image with xenomai enable by command bitake imx-image-multimedia.

XENOMAI_KERNEL_MODE = "cobalt"

IMAGE_INSTALL_append += " xenomai"

or

XENOMAI_KERNEL_MODE = "mercury"

IMAGE_INSTALL_append += " xenomai"

Notice: If XENOMAI_KERNEL_MODE = "cobalt", you can build dual kernel version. And If XENOMAI_KERNEL_MODE = "mercury", it is single kernel with PREEMPT-RT patch.

 Latency testing of Xenomai3.2+Dovetail with isolating CPU 2,3 ( Xenomai 3.2 on 8MM DDR4 EVK with GPU test case (GLES3/S08_EnvironmentMappingRefraction_Wayland) + iperf3 + 2 ping 65000 size + stress-ng --cpu 2 --io 2 --vm 1 --vm-bytes 256M --metrics-brief )

The following is test result by the command (/usr/xenomai/demo/cyclictest -a 2,3 -p 50 -t 5 -m -n -i 1000)

root@imx8mmddr4evk:~# /usr/xenomai/demo/cyclictest -a 2,3 -p 50 -t 5 -m -n -i 1000
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 5.96 6.04 6.03 7/155 1349

T: 0 ( 615) P:50 I:1000 C:63448632 Min: 0 Act: 0 Avg: 0 Max: 55
T: 1 ( 616) P:50 I:1500 C:42299087 Min: 0 Act: 0 Avg: 1 Max: 43
T: 2 ( 617) P:50 I:2000 C:31724315 Min: 0 Act: 0 Avg: 1 Max: 51
T: 3 ( 618) P:50 I:2500 C:25379452 Min: 0 Act: 0 Avg: 1 Max: 53
T: 4 ( 619) P:50 I:3000 C:21149543 Min: 0 Act: 0 Avg: 1 Max: 47

 

 

 

 

Comments

Thank you Xiao, you are awesome. Xenomai is much better than PREEMPT RT kernel, with much lower latencies.

I'll test this !

Regards,

Laci

CPU isolation will help to improve the latency of Xenomai. Like:

1, Add isolcpus=2,3 in bootargs of uboot.

2, Run cyclictest  with option -a 2,3

 

Version history
Revision #:
18 of 18
Last update:
‎11-29-2021 11:33 PM
Updated by: