Unstable latency when using GStreamer to decode live video from AVTP stream(H.264)

[killedByBugs]

I am trying this TSN/AVB demo (link: https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/AVB-TSN-demo-on-i-MX8MP/ta-p/1123791#toc... ) with an imx8qxp MEK board.

An imx8qxp MEK board is used as the AVB listener. The following GStreamer command is used to decode and play the live video(H.264) from the AVTP stream (sent from another board connected with a camera):

gst-launch-1.0 clockselect. \( clock-id=realtime avtpsrc ifname=eth1.5 ! avtpcvfdepay streamid=0xAABBCCDDEEFF000A ! queue max-size-bytes=0 max-size-buffers=0 max-size-time=0 ! h264parse ! v4l2h264dec ! imxvideoconvert_g2d ! clockoverlay halignment=right ! waylandsink \)

It successfully plays the video. However, the latency is unstable. Sometimes, the latency is about 500ms. After a reboot, the latency may change to around 4s.

The output info from GStreamer is like:

====== V4L2DEC: 1.20.3 build on Sep 15 2022 02:27:06. ======
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: DebugGstSystemClock
Redistribute latency...
Redistribute latency...
Redistribute latency...

0:00:00.0 / 99:99:99.

I am unsure why the output has 3 "Redistribute latency...". The timer "0:00:00.0 / 99:99:99." shows instantaneously when the talker starts to send the AVTP stream. However, this timer starts to count until about 4s past.

I also tried to catch latency for each element by using gst-shark. It turns out that:

1. when the total latency is about 500ms: the queue element has the highest latency at about 90.86ms.
2. when the total latency is about 4s: the queue element has the highest latency at about 336.99 ms.

Strangely, the total latency is not much changed when I don't use the queue element on the GStreamer command. This high latency is unacceptable.

Can you please give me some suggestions to solve the problem?

[Bio_TICFSL]

Hello,

Make sure that you are using GPU or you are using the hardware VPU since this is a lot of time.

Regards

[killedByBugs]

Hi,

You are right. Those commands already have elements "v4l2h264dec", and "imxvideoconvert_g2d". They should use VPU, but I also would like to make sure if they did. Do you have any methods to see if the VPU is being used?
Occasionally, the latency is not much. The latency may change after a reboot of the boards. It's unstable. So, I think the problem probably is not about using VPU. Do you have any other ideas about the unstable latency? Thank you.

Regards