AVB/TSN demo on i.MX8MP

Showing results for 
Search instead for 
Did you mean: 

AVB/TSN demo on i.MX8MP

AVB/TSN demo on i.MX8MP



Wikipedia: Audio Video Bridging (AVB) is a common name for the set of technical standards which provide improved synchronization, low-latency, and reliability for switched Ethernet networks. AVB was initially developed by the Institute of Electrical and Electronics Engineers (IEEE) Audio Video Bridging task group of the IEEE 802.1 standards committee. In November 2012, Audio Video Bridging task group was renamed to Time-Sensitive Networking (TSN) task group to reflect the expanded scope of its work, which is to "provide the specifications that will allow time-synchronized low latency streaming services through IEEE 802 networks". Further standardization efforts are ongoing in IEEE 802.1 TSN task group.


AVB and TSN technologies and standards:


Area of Definition

Title of Standard


IEEE 802.1AS

Timing and synchronization

Timing and Synchronization for Time-Sensitive Applications (gPTP)



IEEE 802.1Qav

Forwarding and queuing

Forwarding and Queuing for Time-Sensitive Streams (FQTSS)


IEEE 802.1Qat

Path control and reservation

Stream Reservation Protocol (SRP)


IEEE 802.1BA


Audio Video Bridging (AVB) Systems



AV transport

Layer 2 Transport Protocol for Time Sensitive Applications



Device manage and control

Device Discovery, Enumeration, Connection Management and Control Protocol


IEEE 802.1Qbu and IEEE 802.3br

Forwarding and queuing

Frame preemption


IEEE 802.1Qbv

Forwarding and queuing

Enhancements for scheduled traffic


IEEE 802.1Qca

Path control and reservation

Path control and reservation


IEEE 802.1Qcc

Central configuration method

Enhancements and performance improvements


IEEE 802.1Qci

Time-based ingress policing

Per-stream filtering and policing


IEEE 802.1CB

Seamless redundancy

Frame replication and elimination for reliability



Since many of the standards are only for TSN switch/bridges and i.MX8MP is design to be a TSN/AVB endpoint, this demo does not implement a full stack or full standards. It only demonstrates the basic end-to-end point (talker to listener) A/V streaming without bridge or switch.


Below table shows what this demo supports:




IEEE 802.1AS



IEEE 802.1Qav/Qbu/Qbv

TC qdisc (taprio, mqprio, etf, cbs)

ENET_QoS IP (multi queue + EST, FPE, CBS)





Demo introduction

Two i.MX8MP EVK boards are used for this demo, one act as a AVB talker to send A/V streams, the other one act as a AVB listener to receive A/V streams who can be playback to audio codec and sink video to screen. The two boards are connected by a RJ45 ethernet cable on each ENET2 port (only ENET2 port has TSN features).


Three streams’ type and SR (Stream Reservation) class are defined as below to grantee time sensitive (sub-microsecond synchronization), low latency and bandwidth on the ethernet:

  • Stream A: SR class A, AVTP Compressed Video Format, H.264 profile High, 1920x1080, 30 fps.
  • Stream B: SR class B, AVTP Audio Format, PCM 16-bit sample, 48 kHz, stereo, 12 frames per AVTPDU.
  • Other stream: Best-effort streams

These three TSN streams would be allocated into different traffic control (TC) class for egress in Linux. Different TC class would be mapped to different hardware queues with dedicated DMA channel, thanks to the multi-queue support by ENET_QoS IP. Then the traffic control apply different scheduling policy on each queues for different types of streams egress, to make sure highest priority or critical packet can be sent in the correct time slot.

The TSN streams are transmitted over Virtual LANs (VLANs). Bridges use the VLAN priority information (PCP) to identify Stream Reservation (SR) traffic classes which are handled according to the Forward and Queuing Enhancements for Time-Sensitive Streams (FQTSS) mechanisms described in Chapter 34 of the IEEE 802.1Q standard.


The demo is built up by following blocks:

  • Linux TC (traffic control): streams egress control to meet AVB/TSN requirements, which take advantage of the i.MX8MP TSN ENET IP.
  • Linux PTP: clock sync in network, which take advantage of the i.MX8MP TSN ENET IP.
  • Libavtp: Time Sensitive Applications AV Transport protocol.
  • ALSA: AVTP audio format plugin uses the libavtp to transmit and receive AVTP audio PCM streams.
  • Gstreamer: avtp plugin uses the libavtp to transmit and receive AVTP audio/video streams (video should be encoded as H264, audio PCM). x264enc and libav is a software H.264 video encoder/decoder, which alternative to the on chip VPU acceleration plugin, due to the VPU plugin is not supported in the latest Gstreamer version (1.17.x).


Traffic control

This demo can use two different traffic control qdisc settings:

  • mqprio + cbs: use CBS features
  • taprio + pfifo: use EST and FPE features (802.1Qbu/bv).

The pfifo is the default traffic control class, which use FIFO schedule policy for egress packets. The CBS class is actually handled by hardware IP to select which queue for transmit in a certain time slot.


Credit Base Shaper (CBS)

CBS parameters come straight from the IEEE 802.1Q-2018 specification. They are the following:

  • idleSlope: rate credits are accumulated when queue isn’t transmitting;
  • sendSlope: rate credits are spent when queue is transmitting;
  • hiCredit: maximum amount of credits the queue is allowed to have;
  • loCredit: minimum amount of credits the queue is allowed to have;


Enhancements to Scheduled Traffic (EST)

The IEEE 802.1Qbv defines the schedule for each of the queues on every egress port which makes the implementation aware of traffic arrival schedule. This information can be used to block the lower priority traffic from transmission in this time window/slot. This ensures that scheduled traffic is forwarded from sender to receiver through all the network nodes with a deterministic delay. The i.MX8MP uses the gate control list with configurable time slot and frame preempt (IEEE 802.1Qbu) features to support EST.


Other than the CBS, the gate control list control the egress transmission by a fixed open interval for that queue.


Build demo

Build L5.4.24_2.1.0

$MACHINE=imx8mpevk DISTRO=fsl-imx-xwayland source imx-setup-release.sh -b build-8mp

$bitbake  fsl-image-validation-imx

Prepare a SD card and burn it with the built out images.

Rebuild kernel

Rebuild the kernel after applying the 0001-qenet-add-queue-avoid-panic.patch (attached), and overwrite the Image and imx8mp-evk.dtb on the boot partition of the SD card.

Install the toolchain

$bitbake -f fsl-image-validation-imx -c populate_sdk

$sh tmp/deploy/sdk/fsl-imx-xwayland-glibc-x86_64-fsl-image-validation-imx-aarch64-imx8mpevk-toolchain-5.4-zeus.sh

The toolchain would be installed into /opt/fsl-imx-xwayland/5.4-zeus/

Create a install folder

$mkdir <your install folder>

Create a folder to install all of the shared libraries, binaries and configure files which built out manually in this doc. After built done, you should copy all of the contents in this folder to target board root.

Build libavtp

$source /opt/fsl-imx-xwayland/5.4-zeus/environment-setup-aarch64-poky-linux

$git clone https://github.com/Avnu/libavtp.git

$cd libavtp

$meson build

$ninja -C build

Copy the built out .so and .pc into the toolchain rootfs:

$sudo cp build/libavtp.so* /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/lib/

$sudo cp build/meson-private/*.pc /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/lib/pkgconfig/

To make sure you have avtp package installed correctly:

$pkg-config --list-all | grep avtp

Copy the .so into the install folder:

$cp build/libavtp.so* <install folder>/usr/lib/

Build ALSA aaf plugin

The AAF (ALSA AVTP Audio Format) plugin is a PCM plugin that uses Audio Video Transport Protocol (AVTP) to transmit/receive audio samples through a Time-Sensitive Network (TSN) capable network. The plugin enables media applications to easily implement AVTP Talker and Listener functionalities.


$cd <yocto build>/tmp/work/aarch64-mx8mp-poky-linux/alsa-plugins/1.1.9-r0/alsa-plugins-1.1.9

$./configure --build=x86_64-linux --host=aarch64-poky-linux --target=aarch64-poky-linux --prefix=<your install folder>/usr --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot=/opt/samba/nxa23059/jailhouse/yocto-5.x/build-8mp/tmp/work/aarch64-mx8mp-poky-linux/alsa-plugins/1.1.9-r0/recipe-sysroot  --disable-static  --enable-aaf --disable-jack --disable-libav --disable-maemo-plugin --disable-maemo-resource-manager --enable-pulseaudio --enable-samplerate --with-speex=lib


$make install

Build Gstreamer AVTP plugins

Build Gstreamer 1.17.x

$git clone https://gitlab.freedesktop.org/gstreamer/gstreamer.git

$cd gstreamer

$patch -p1 < gstreamer-1.0-pass-build.patch

$meson build --prefix=<your install folder>/usr

$ninja -C build

$ninja -C build install

After Gstreamer is installed into <your install folder>, please fix the “prefix” path in the .pc files by, and copy to the toolchain folders:

$cd <your install folder>

$grep -lR <your install folder> ./lib/pkgconfig/ | xargs sed -i 's/<your install folder>/\/usr/g'

NOTE: Make sure you use the ‘\’ for ‘/’ convert in your path.

$cp -rf ./usr/* /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/

Build gst-plugins-base

$git clone https://gitlab.freedesktop.org/gstreamer/gst-plugins-base.git

$cd gst-plugins-base

$patch -p1 < gst-plugins-base-pass-build.patch

$meson build --prefix=<your install folder>/usr

$ninja -C build

$ninja -C build install

Build gst-plugins-bad

$git clone https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad.git

$cd gst-plugins-bad

$meson build --prefix=<your install folder>/usr

$ninja -C build

$ninja -C build install

  • If you met gdbus-codegen issue, please remove the “--c-generate-autocleanup” and “--output-directory” parameters in the build/build.ninja
  • If you met issue of include file like: “…/gst/gl/gstglapi.h:24:10: fatal error: gst/gl/gstglconfig.h: No such file or directory.” Please modify the build/build.ninja to correct the -I header file parameter of the build.

After gst-plugins-base and gst-plugins-bad installed into <your install folder>, please fix the “prefix” path in the .pc files and copy them into the toolchain folders:

$cd <your install folder>

$grep -lR <your install folder> ./lib/pkgconfig/ | xargs sed -i 's/<your install folder>/\/usr/g'

NOTE: Make sure you use the ‘\’ for ‘/’ convert in your path.

$cp -rf ./usr/* /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/


Build H.264 SW plugins

AVTPDU uses H.264 as payload for streaming, this requires H.264 encoder/decoder plugins, either software or hardware accelerations. Since we upgrade the Gstreamer and its plugins to a new version, the VPU plugins cannot be used anymore. So software H.264 plugins are required: x264 for encoder, libav for decoder.

Build x264

As the yocto actually has the x264 recipes, but not included in our bblayers, we need to copy the x264 source into our bblayers path under <yocto>/source to build:

$cp -rf ./poky/meta/recipes-multimedia/x264 ./meta-openembedded/meta-multimedia/recipes-multimedia/

$vi ./meta-openembedded/meta-multimedia/recipes-multimedia/x264_git.bb

Remove the LICENSE_FLAGS line

$bitbake -f x264 -c do_install

$sudo cp -rf tmp/work/aarch64-poky-linux/x264/r2917+gitAUTOINC+72db437770-r0/image/usr/* /opt/fsl-imx-xwayland/5.4-zeus/sysroots/aarch64-poky-linux/usr/

Build gst-plugins-ugly

$git clone https://gitlab.freedesktop.org/gstreamer/gst-plugins-ugly.git

$cd gst-plugins-ugly$meson build --prefix=<your install folder>/usr

$ninja -C build

$ninja -C build install

Build libav

As the yocto actually has the libav recipes, but not included in our bblayers, we need to copy the its source into our bblayers path under <yocto>/source:

$cp -rf ./poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-libav ./meta-openembedded/meta-multimedia/recipes-multimedia/gstreamer

$vi ./meta-openembedded/meta-multimedia/recipes-multimedia/gstreamer/gstreamer1.0-libav_1.16.1.bb

Remove the LICENSE_FLAGS line

$bitbake -f gstreamer1.0-libav -c do_install

$cp tmp/work/aarch64-mx8mp-poky-linux/gstreamer1.0-libav/1.14.0-r0/image/usr/lib/gstreamer-1.0/libgstlibav.so <your install folder>/usr/lib/gstreamer-1.0

Now you have all of the Gstreamer plugins which required for AVB/TSN audio/video demo:

  • avtpsrc, avtpsink, avtpaafpay, avtpaafdepay, avtpcvfpay, avtpcvfdepay
  • x264_enc (encoder), avdec_h264 (decoder)


Install binaries

Final step is to copy all of your built out files from <your install folder> into your board / root, and boot up the board.

Verify the Gstreamer plugins install correctly or not:

$export GST_PLUGIN_PATH= /usr/lib/gstreamer-1.0/


Check if the above Gstreamer plugins we built out can be found by gst-instpect.

System Setup


The ENTE_QoS is assigned to eth1 instance. So create eth1.5 for vlan id 5:

$ip link add link eth1 name eth1.5 type vlan id 5 \ 

       egress-qos-map 2:2 3:3

$ip link set eth1.5 up


The TSN control plane is implemented through the Linux Traffic Control (TC) System. The transmission algorithms specified in the Forwarding and Queuing for Time-Sensitive Streams (FQTSS) chapter of IEEE 802.1Q-2018 are supported via TC Queuing Disciplines (qdiscs). Linux currently provides the following qdiscs relating to TSN:

  • Multiply queue qdiscs (These two qdiscs are mutually exclusive, select one for your demo):
    • MQPRIO: Mapping the TC class into different hardware queue in IP.
    • TAPRIO: Implements the Enhancements for Scheduled Traffic introduced by IEEE 1Qbv/Qbu. This is supported by i.MX8MP ENET_QoS IP through EST features:
      • Qbv – Time aware shaper
      • Qbu - frame preemption.
  • CBS qdisc: Implements the Credit-Based Shaper introduced by the IEEE 802.1Qav This is supported by i.MX8MP ENET_QoS IP through EST (Enhancements to Scheduled Traffic) features. It works with mqprio qdisc together.
  • ETF qdisc: While not an FQTSS feature, Linux also provides the Earliest TxTime First (ETF) qdisc which enables the LaunchTime Since the i.MX8MP ENET_QoS IP does not support LaunchTime feature, this qdisc configurations would be ignored.


MQPRIO qdisc

$tc qdisc add dev eth1 parent root handle 100 mqprio \

        num_tc 3 \

        map 0 0 2 1 0 0 0 0 0 0 0 0 0 0 0 0 \

        queues 1@0 1@1 1@2 \

        hw 1

NOTE, since ENET_QoS Q0 does not support hardware CBS, we have to avoid using Q0 for AVB streaming. Here’s the mapping:

  • socket SO_PRIORITY 2 -> TC CLASS 2 -> Q2
  • socket SO_PRIORITY 3 -> TC CLASS 1 -> Q1
  • Other socket priority -> TC CLASS 0 -> Q0

TAPRIO qdisc

Create a root qdisc handle to map the different CLASS streams to hardware queues w/ GCL (gate control list). This root handle maps the CLASS A stream to queue Q1, CLASS B stream to Q2, and others to Q0 by the “map” and “queues” parameters, same as mqprio above. The Q1/Q2(CLASS1/2) gates are open in the first 300us interval, then only Q1(CLASS1) gate is open in the next 300us with all other CLASS gated. The last sched-entry defines in after Q1 is open after 300us, all of the CLASS gates are open for next 200us. Then loopback to the first sched-entry for gate control.

Please note in this demo, we only enable features to support the 802.1Qbv, the Qbu (Frame preempt) requires patches for iproute2.

$tc qdisc add dev eth1 parent root handle 100 taprio \

        num_tc 3 \

        map 0 0 2 1 0 0 0 0 0 0 0 0 0 0 0 0 \

        queues 1@0 1@1 1@2 \

        base-time 1000000000 \

        sched-entry S 0x6 300000 \

        sched-entry S 0x2 300000 \

        sched-entry S 0x7 200000 \

        flags 0x2

CBS qdisc

Q1 CBS for video, Q2 CBS for audio:

$tc qdisc replace dev eth1 parent 100:2 handle 777 cbs \

        idleslope 98688 sendslope -901312 hicredit 153 locredit -1389 \

        offload 1

$tc qdisc replace dev eth1 parent 100:3 handle 888 cbs \

        idleslope 3648 sendslope -996352 hicredit 12 locredit -113 \

       offload 1

NOTE: By default, the Q0 will be allocated for pfifo qdisc class if we do not define them.

ETF qdisc

As ENET_QOS does not support hardware launch time in IP, the ETF qdisc would not be used here.


Run the ptp4l and phc2sys in background, and use check_clocks to check the ptp sync works.

$ptp4l -i eth1 -f ./gPTP.cfg --step_threshold=1 &

$pmc -u -b 0 -t 1 "SET GRANDMASTER_SETTINGS_NP clockClass 248 \

        clockAccuracy 0xfe offsetScaledLogVariance 0xffff \

        currentUtcOffset 37 leap61 0 leap59 0 currentUtcOffsetValid 1 \

        ptpTimescale 1 timeTraceable 1 frequencyTraceable 0 \

        timeSource 0xa0"

$phc2sys -s eth1 -c CLOCK_REALTIME --step_threshold=1 \

        --transportSpecific=1 -w & 

$./check_clocks -d eth1

The PTP Hardware Clock (PHC)  synced between PTP master/slave means the RMS offset between PHC and GM (master clock) is < 100ns. PHC and system clock (CLOCK_REALTIME) synced means the clock offset < 100ns


  • The check_clocks source code can be downloaded from here.
  • cfg is described below, if you want to identify which is master and which is slave, use “masterOnly 1” or “slaveOnly 1” in this configuration file.


# 802.1AS example configuration containing those attributes which

# differ from the defaults.  See the file, default.cfg, for the  

# complete list of available options.                             



gmCapable               1                                        

priority1               248                                       

priority2               248                                      

logAnnounceInterval     0                                        

logSyncInterval         -3                                       

syncReceiptTimeout      3                                         

neighborPropDelayThresh 800                                      

min_neighbor_prop_delay -20000000                                

assume_two_step         1                                        

path_trace_enabled      1                                        

follow_up_info          1                                        

transportSpecific       0x1                                       

ptp_dst_mac             01:80:C2:00:00:0E                        

network_transport       L2                                       

delay_mechanism         P2P                              


Run demo

ALSA AAF audio

To run the alsa AAF demo, please add aaf0 and converter0 plugin device into /etc/asound.conf:

pcm.aaf0 {

        type aaf

        ifname eth1.5

        addr 01:AA:AA:AA:AA:AA

        prio 2

        streamid AA:BB:CC:DD:EE:FF:000B

        mtt 50000

        time_uncertainty 1000

        frames_per_pdu 12

        ptime_tolerance 100


pcm.converter0 {

        type linear

        slave {

                pcm "hw:2,0"

                format S16_LE




The “aaf0” plugin device defines the ethernet interface which AAF runs on, the socket priority which mapping to Traffic Class in kernel TC, the stream-id for the aaf streaming.

The “converter0” plugin device is used for convert the S16_BE format to S16_LE for the wm8960 PCM audio.


Select one device as AVB talker, and run:

$speaker-test -p 25000 -F S16_BE -c 2 -r 48000 -D aaf0

Select one device as AVB talker, and run:

$arecord -F 25000 -t raw -f S16_BE -c 2 -r 48000 -D aaf0 | aplay -F 25000 -t raw -f S16_BE -c 2 -r 48000 -D converter0

You can hear the sound on the listener device. You can also check which qdisc queue is used for AAF by:

$tc -s qdisc


Gstreamer AAF audio

Select one device as AVB talker, and run:

$gst-launch-1.0 clockselect. \( clock-id=realtime \

    audiotestsrc samplesperbuffer=12 is-live=true ! \

    audio/x-raw,format=S16BE,channels=2,rate=48000 ! \

    avtpaafpay mtt=50000000 tu=1000000 streamid=0xAABBCCDDEEFF000B processing-deadline=0 ! \

    avtpsink ifname=eth1.5 address=01:AA:AA:AA:AA:AA priority=2 processing-deadline=0 \)


Select one device as AVB listener, and run:

$gst-launch-1.0 clockselect. \( clock-id=realtime \

    avtpsrc ifname=eth1.5 ! avtpaafdepay streamid=0xAABBCCDDEEFF000B ! \

    queue max-size-bytes=0 max-size-buffers=0 max-size-time=0 ! \

    audioconvert ! audioresample !  alsasink device="hw:2,0" \)

Gstreamer CVF video

Select one device as AVB talker, and run:

$gst-launch-1.0 clockselect. \( clock-id=realtime \

    videotestsrc is-live=true ! video/x-raw,width=480,height=320,framerate=15/1 ! \

    clockoverlay ! x264enc bframes=0  key-int-max=1 speed-preset=1 tune=4 ! h264parse config-interval=-1 ! \

    avtpcvfpay processing-deadline=20000000 mtu=1400 mtt=2000000 tu=125000 streamid=0xAABBCCDDEEFF000A ! \

    avtpsink ifname=eth1.5 priority=3 processing-deadline=20000000 \)


  • To eliminate the h.264 software encoding/decoding overhead with acceptable latency for this demo, we use several parameters for x264enc element:
    • bframes: x264enc and avdec_h264 together was found to have issues, remove bframes in the stream would help.
    • key-int-max: decoder can only decode the frame when a keyframe is present on stream to make sure decoder can work as faster as it can, the distance between two keyframes must be set to closest.
    • speed-preset: to low down the CPU loading for encoding, we use the option to make encoding as faster as we can. (=1 means ultrafast)
    • tune: 4 means zero latency
  • ‘mtu=1400’ parameters for avtpcvfpay element is very important, if using the default mtu=1500, the listener cannot get the AVTPDUs package correctly from VLAN. The reason is unknown yet.


Select one device as AVB listener, and run:

$gst-launch-1.0 clockselect. \( clock-id=realtime \

    avtpsrc ifname=eth1.5 ! avtpcvfdepay streamid=0xAABBCCDDEEFF000A ! \

    queue max-size-bytes=0 max-size-buffers=0 max-size-time=0 ! \

    avdec_h264 ! videoconvert ! clockoverlay halignment=right ! waylandsink \)


The demo screenshot below: there are two clocks show on the videotestsrc stream: left one is the timestamp recorded before x264enc encoding on the AVB talker side, right one is the timestamp recorded after avdec_h265 decoding and do video convert to YUV frames on AVB listener side. You can see the timestamp is sync in seconds.


Deep dive

Packet sniffer

Use tcpdump on board to dump the L2 ethernet packet:

./tcpdump -i eth1 ether proto 0x22f0 -w dump.pcap

The AVTP ether protocol code is 0x22f0 embedded inside the ether frame, or you can use "vlan 5" VLAN id for tcpdump parameters to dump.

Then open this dump.pcap in the windows/Linux PC by the wireshark tool, it will automatically show the protocol inside the package, it can also parser the IEEE1722 (AVTP) CVF/AFF package header as below:


Precise latency measurement

  • The clockoverlay plugin used in the above talker/listener is actually seconds level precision, which can not reflect the latency from talker videotestsrc -> listener avdec_h264 decoding finish. Here need a little hack to the clockoverlay element in the gst-plugins-base to get the millisecond precision. The patch is attached (gst-plugins-base-clockoverlay-us.patch), please apply and rebuild the gst-plugins-base, then replace the libgstpango.so on the board /usr/lib/gstreamer-1.0/. When doing CVF demo, you can take a picture of the screen, and check the two clock's diff. During my test, the latency is about 50ms, which include all the cost of encoding, AVTP packaging, streaming, ethernet transmit, ethernet receive, AVTP unpack and frame decoding.


  • To measure the package latency from transmit port (talker) to receive port (listener), you can use the tcpdump on both end-points. And compare the Epoch Time the packet dumped: "Epoch Time: 1596252905.688243000 seconds". The delta of the epoch time of the same packet is around 100us~500us. This latency actually includes the AF_PACKET clone cost in kernel netfilter, also the tcpdump application schedule latency.


Version history
Revision #:
1 of 1
Last update:
‎08-11-2020 07:49 PM
Updated by: