Potential Issue when using H.264 encoding when combined with RTP on i.MX8MMini

[vincentz63]

Hi,

I’m working on an app streaming live video encoded to H.264 over RTP on the i.MX8MMini LPDDR4 EVK.

I’m experiencing dropped frames: 1 frame is lost frame every 128 frames precisely.

This can be reproduced with the pipeline below:

gst-launch-1.0 -v videotestsrc is-live=true ! video/x-raw,width=640,height=480,pixelformat=YUY2,framerate=30/1 ! vpuenc_h264 gop-size=30 bitrate=5000 ! queue ! rtph264pay ! queue ! rtph264depay ! queue ! fpsdisplaysink text-overlay=false video-sink=fakesink sync=false

Running this command for 2min, you should see the average framerate stabilising to 29.77Hz, which corresponds precisely to 1 frame being dropped every 128.

I’ve also observed the frame drop using a custom app based on GStreamer and looking at the PTS after the ‘rtph264depay’ element and could see a jump of 33.33ms in the PTS, indicating a frame has been dropped.

Running the pipeline without the RTP payloader and depayloader does not show the problem.

gst-launch-1.0 -v videotestsrc is-live=true ! video/x-raw,width=640,height=480,pixelformat=YUY2,framerate=30/1 ! vpuenc_h264 gop-size=30 bitrate=5000 ! queue ! fpsdisplaysink text-overlay=false video-sink=fakesink sync=false

Has anyone seen this before?

Thanks

JP

[vincentz63]

I've now found out that the H.264 stream generated by the i.MX8MMini VPU encoder contains one Filler NALU every 128 NALUs.

The NALUs surrounding the Filler look fine, no different from the other VCL NALUs in the stream.

This would seem to be related to the problem.

Has anyone not noticed this problem already? Am I the only one to be streaming VPU-encoded H.264 over RTP on the i.MX8MMini??