Display output causes jumps in the encoded video

[isaacnickaein]

I have been trying to run these two tasks simultaneously on i.MX6DL:

1. a GUI (Qt) application (Outputs to DISP0 using BT1120 patch)

2. Encoding the input video (Input video from CSI0)

Specifically, the following two commands is executed:

1. ./openglwindow -platform eglfs

(This is the Qt example on OpenGL)

2. gst-launch -v mfw_v4lsrc capture-mode=0 ! queue2 ! vpuenc codec=6 bitrate=5000000 seqheader-method=3 ! queue2 ! mpegtsmux ! udpsink host=10.1.1.169  port=6666

The problem is when I start the GUI application, it causes sudden "jump"s in the encoded video. I have attached the encoded video.

The jumps happens whenever there is some change in the output video. I have tried several ways to generate an image output and the only case that distortions disappear was:

1. First, running this pipleline as the video output:

gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16

2. Then, running the encode pipleline as before.

For some reason, using /dev/video16 as device output solves the issue. Also, the order of running the commands is important: Running the encoding pipeline first and then switching to video output still lead to distortions in the encoded video.

Based on these observations, I have two questions:

1. How can I run the Qt application in the same manner as the last pipleline (with device=/dev/video16) so that it doesn't affect the encoding process. I have tried EGLFS options (e.g. QT_QPA_EGLFS_FB) but those didn't help.

2. What is the root cause of this issue and how this can be prevented?

Bests,

Isaac

Original Attachment has been moved to: output.mkv

[art]

The root cause of the issue is high amount of data transferring on the memory

bus that sometimes causes a video data loss. Seems that it cannot be prevented on

a given hardware.

Have a great day,
Artur

[isaacnickaein]

But this issue doesn't happen when I use the following pipeline:

gst-launch videotestsrc ! mfw_v4lsink device=/dev/video16

for generating the output video (more details in my first post). What is the significance of /dev/video16 that suppresses the issue? Can that be replicated for a Qt EGLFS application?