1. Introduction
1.1. Purpose
This application note introduces a procedure of how to port AVB/TSN stack and run referring feature demos on i.MX8DXL board. This can help users who want to run AVB/TSN demos to quickly understand and customized their own codes. Since many of the standards are only for TSN switch/bridges and i.MX8DXL is design to be a TSN/AVB endpoint, the demos did not implement a full stack or full standards. They only demonstrated the basic end-to-end point (talker to listener) A/V streaming without bridge or switch.
The software used for example in this documentation are based on the opensource such as gstreamer and alsa utils.
1.2. Overview
1.2.1. AVB/TSN
AVB (Audio Video Bridging) is a common name for the set of technical standards which provide improved synchronization, low-latency, and reliability for switched ethernet network. AVB was initially developed by the IEEE Audio Video Bridging task group of the IEEE 802.1 standards committee. In November 2012, AVB group was renamed to TSN (Time-Sensitive Networking) task group to reflect the expanded scope of its work, which is to provide the specifications that will allow time-synchronized low latency streaming services through IEEE 802 networks.
The referring standards shows as follows:
TSN protocol additions
QoS components supported in HW TSN MAC + SW driver
Managed Object components expose i/f to allow support of standardized network config protocols (local & remote)
Transport API to allow other transport layer to use TSN QoS
Stack extensions to map traffic priority to application task scheduling
Real Time, gPTP based, Best Effort
1.2.2. Demo introduction
The two streams are defined as below to grantee time sensitive (sub-microsecond synchronization), low latency and bandwidth on the ethernet:
Stream A: SR class A, AVTP Audio Format, PCM 16-bit sample, 48 kHz, stereo, 12 frames per AVTPDU.
Stream B: SR class B, AVTP Compressed Video Format, H.264 profile High, 1920x1080, 30 fps.
The two TSN streams would be allocated into different TC (traffic control) class for egress. Different TC class would be mapped to different hardware queues with specific DMA channel which supported by ENET_OoS IP.
The demos were built by follow blocks:
Linux Traffic Control: streams egress control
Linux ptp: clock sync in network
Libavtp: Time Sensitive Applications AV Transport protocol
Gstreamer: avtp plugin uses the libavtp to transmit and receive AVTP audio/video (audio pcm, video h264).
1.2.3. Traffic control
Multiply queue qdiscs + CBS:
The CBS class is actually handled by hardware IP to select which queue for transmitting.
CBS parameters come straight from the IEEE 802.1Q-2018 specification. They are the following:
idleSlope: rate credits are accumulated when queue isn’t transmitting;
sendSlope: rate credits are spent when queue is transmitting;
hiCredit: maximum amount of credits the queue is allowed to have;
loCredit: minimum amount of credits the queue is allowed to have;
2. Build demo
2.1. Build yocto
$ DISTRO=fsl-imx-xwayland MACHINE=imx8dxlevk source imx-setup-release.sh -b ./xwayland
$ bitbake imx-image-full
Prepare a SD card and burn it with the built out images.
2.2. Rebuild kernel
Rebuild the kernel after applying the 0001-qenet-add-queue-avoid-panic.patch, and overwrite the Image and imx8dxl-evk.dtb on the boot partition of the SD card.
2.3. Install the toolchain
$ bitbake -f fsl-image-validation-imx -c populate_sdk
$ sh tmp/deploy/sdk/fsl-imx-xwayland-glibc-x86_64-fsl-image-validation-imx-aarch64-imx8dxlevk-toolchain-5.4-zeus.sh
The toolchain would be installed into /opt/fsl-imx-xwayland/5.4-zeus
2.4. Create a install folder
$ mkdir <your install folder>
Create a folder to install all of the shared libraries, binaries and configure files which built out manually in this doc. After built done, you should copy all of the contents in this folder to target board root.
2.5. Build libavtp
$ source /opt/fsl-imx-xwayland/5.4-zeus/environment-setup-aarch64-poky-linux
$ git clone https://github.com/Avnu/libavtp.git
$ cd libavtp
$ meson build --prefix=<your install folder>/usr
$ ninja -C build
Copy the built out .so and .pc into the toolchain rootfs:
$ sudo cp build/libavtp.so* /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/lib
$ sudo cp build/meson-private/*.pc /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/lib/pkgconfig/
Copy the .so into the install folder:
$ cp build/libavtp.so* <install folder>/usr/lib/
To make sure you have avtp package installed correctly:
$ pkg-config --list-all | grep avtp
2.6. Build ALSA aaf plugin
$ cd <yocto build>/tmp/work/aarch64-poky-linux/alsa-plugins/1.1.9-r0/alsa-plugins-1.1.9
$ ./configure --build=x86_64-linux --host=aarch64-poky-linux --target=aarch64-poky-linux --prefix=<install folder>/usr --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot=<yocto build>/xwayland/tmp/work/aarch64-poky-linux/alsa-plugins/1.1.9-r0/recipe-sysroot --disable-static --enable-aaf --disable-jack --disable-libav --disable-maemo-plugin --disable-maemo-resource-manager --enable-pulseaudio --enable-samplerate --with-speex=lib
$ make
$ make install
2.7. Build Gstreamer AVTP plugins （1.17.x）
2.7.1. Build Gstreamer core
$ git clone https://gitlab.freedesktop.org/gstreamer/gstreamer.git
$ patch -p1 < gstreamer-1.0-pass-build.patch
$ meson build --prefix=<install folder>/usr
$ ninja -C build
$ sudo ninja -C build install
After Gstreamer is installed into <your install folder>, please fix the “prefix” path in the .pc files by, and copy to the toolchain folders:
$ cd <your install folder>
$ grep -lR <your install folder> ./lib/pkgconfig/ | xargs sed -i 's/<your install folder>/\/usr/g'
$ cp -rf ./usr/* /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/
2.7.2. Build gst-plugins-base
$ git clone https://gitlab.freedesktop.org/gstreamer/gst-plugins-base.git
$ cd gst-plugins-base
$ patch -p1 < gst-plugins-base-pass-build.patch
$ meson build --prefix=<your install folder>/usr
$ ninja -C build
$ sudo ninja -C build install
2.7.3. Build gst-plugins-bad
$ git clone https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad.git
$ cd gst-plugins-bad
$ meson build --prefix=<your install folder>/usr
$ ninja -C build
$ sudo ninja -C build install
After gst-plugins-base and gst-plugins-bad installed into <your install folder>, please fix the “prefix” path in the .pc files and copy them into the toolchain folders:
$ cd <your install folder>
$ grep -lR <your install folder> ./lib/pkgconfig/ | xargs sed -i 's/<your install folder>/\/usr/g'
$ cp -rf ./usr/* /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/
2.8. Build H.264 SW plugins
2.8.1. Build x264
As the yocto actually has the x264 recipes, but not included in our bblayers, we need to copy the x264 source into our bblayers path under <yocto>/source to build:
$ cp -rf ./poky/meta/recipes-multimedia/x264 ./meta-openembedded/meta-multimedia/recipes-multimedia/
$ vi ./meta-openembedded/meta-multimedia/recipes-multimedia/x264_git.bb
Remove the LICENSE_FLAGS line
$ bitbake -f x264 -c do_install
$ sudo cp -rf tmp/work/aarch64-poky-linux/x264/r2917+gitAUTOINC+72db437770-r0/image/usr/* /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/
2.8.2. Build gst-plugins-ugly
$ git clone https://gitlab.freedesktop.org/gstreamer/gst-plugins-ugly.git
$ cd gst-plugins-ugly
$ meson build --prefix=<your install folder>/usr
$ ninja -C build
$ sudo ninja -C build install
2.8.3. Build libav
$ cp -rf poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav meta-openembedded/meta-multimedia/recipes-multimedia/gstreamer-1.0/
Remove the LICENSE_FLAGS line
$ vim ./poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav_1.16.2.bb
$ bitbake -f gstreamer1.0-libav -c do_install
$ cp /opt/samba/nxf39444/imx-yocto-bsp-i.mx8dxl/xwayland/tmp/work/aarch64-poky-linux/gstreamer1.0-libav/1.16.2-r0/image/usr/lib/gstreamer-1.0/libgstlibav.so <your install folder>/usr/lib/gstreamer-1.0
2.8.4. Install binaries
Final step is to copy all of your built out files from <your install folder> into your board / root, and boot up the board.
$ export GST_PLUGIN_PATH=/usr/lib/gstreamer-1.0/
$ gst-inspect-1.0
To check if the above Gstreamer plugins we built out can be found by gst-instpect.
3. System Setup
3.1. VLAN
The ENTE_QoS is assigned to eth0 instance. So create eth0.5 for vlan id 5:
$ ip link add link eth0 name eth0.5 type vlan id 5 egress-qos-map 2:2 3:3
$ ip link set eth0.5 up
3.2. Qdiscs
The TSN control plane is implemented through the TC (Traffic Control) system. The transmission algorithms specified in the FQTSS (Forwarding and Queuing for Time-Sensitive Streams) chapter of IEEE 802.1Q-2018 are supported via TC Qdiscs (Queuing Discipline).
3.2.1. MQPRIO qdisc
$ tc qdisc add dev eth0 parent root handle 100 mqprio num_tc 3 map 0 0 2 1 0 0 0 0 0 0 0 0 0 0 0 0 queues 1@0 1@1 1@2 hw 1
3.2.2. CBS qdisc
Q1 CBS for audio, Q2 CBS for video:
$ tc qdisc replace dev eth0 parent 100:3 handle 888 cbs idleslope 3648 sendslope -996352 hicredit 12 locredit -113 offload 1
$ tc qdisc replace dev eth0 parent 100:2 handle 777 cbs idleslope 98688 sendslope -901312 hicredit 153 locredit -1389 offload 1
3.2.3. TimeSync
Run the ptp4l and phc2sys in background, and use check_clocks to check the ptp sync works.
$ ptp4l -i eth0 -f ./gPTP.cfg --step_threshold=1 &
$ pmc -u -b 0 -t 1 "SET GRANDMASTER_SETTINGS_NP clockClass 248 clockAccuracy 0xfe offsetScaledLogVariance 0xffff currentUtcOffset 37 leap61 0 leap59 0 currentUtcOffsetValid 1 ptpTimescale 1 timeTraceable 1 frequencyTraceable 0 timeSource 0xa0"
$ ./check_clocks -d eth0
4. Run demo
4.1. ALSA AAF audio
To run the alsa AAF demo, please add aaf0 and converter0 plugin device into /etc/asound.conf:
pcm.aaf0 {
type aaf
ifname eth0.5
addr 01:AA:AA:AA:AA:AA
prio 2
streamid AA:BB:CC:DD:EE:FF:000B
mtt 50000
time_uncertainty 1000
frames_per_pdu 12
ptime_tolerance 100
}
pcm.converter0 {
type linear
slave {
pcm "hw:0,0"
format S16_LE
}
}
The “aaf0” plugin device defines the ethernet interface which AAF runs on, the socket priority which mapping to Traffic Class in kernel TC, the stream-id for the aaf streaming. The “converter0” plugin device is used for convert the S16_BE format to S16_LE for the wm8960 PCM audio.
Select one device as AVB talker, and run:
$ speaker-test -p 25000 -F S16_BE -c 2 -r 48000 -D aaf0
Select one device as AVB listener, and run:
$ arecord -F 25000 -t raw -f S16_BE -c 2 -r 48000 -D aaf0 | aplay -F 25000 -t raw -f S16_BE -c 2 -r 48000 -D converter0
You can hear the sound on the listener device.
You can also check which qdisc queue is used for AAF by:
$ tc -s qdisc
4.2. Gstreamer AAF audio
Select one device as AVB talker, and run:
$ gst-launch-1.0 clockselect. \( clock-id=realtime audiotestsrc samplesperbuffer=12 is-live=true ! audio/x-raw,format=S16BE,channels=2,rate=48000 ! avtpaafpay mtt=50000000 tu=1000000 streamid=0xAABBCCDDEEFF000B processing-deadline=0 ! avtpsink ifname=eth0.5 address=06:98:c0:22:df:35 priority=3 processing-deadline=0 \)
Select one device as AVB listener, and run:
$ gst-launch-1.0 clockselect. \( clock-id=realtime avtpsrc ifname=eth0.5 ! avtpaafdepay streamid=0xAABBCCDDEEFF000B ! queue max-size-bytes=0 max-size-buffers=0 max-size-time=0 ! audioconvert ! audioresample ! alsasink device="hw:0,0" \)
5. Packet sniffer
Use tcpdump on board to dump the L2 ethernet packet:
$ tcpdump -i eth0 ether proto 0x22f0 -w dump.pcap
The AVTP ether protocol code is 0x22f0 embedded inside the ether frame, or you can use "vlan 5" VLAN id for tcpdump parameters to dump.
Then open this dump.pcap in the windows/Linux PC by the wireshark tool, it will automatically show the protocol inside the package, it can also parser the IEEE1722 (AVTP) CVF/AFF package header as below:
To measure the package latency from transmit port (talker) to receive port (listener), you can use the tcpdump on both end-points. And compare the Epoch Time the packet dumped: "Epoch Time: 1596252905.688243000 seconds". The delta of the epoch time of the same packet is around 100us~500us. This latency actually includes the AF_PACKET clone cost in kernel netfilter, also the tcpdump application schedule latency.
6. Revision history
summarizes the changes done to this document since the initial release.
Table2. Revision history
Revision number
Date
Substantive changes
1
5/2021
Initial release
View full article
