Hi everyone,
Release: L4.14.98_2.0.0_ga_images_MX8MMEVK.zip
Board: i.MX8MM EVK
I found that adding imxvideoconvert_g2d element to GStreamer will cause frame loss. Here are some tests I did using the official GStreamer RTSP Demo.
Test conditions:
Input source: /dev/video0 1080p 60fps uyvy format
Player: liveplayer (bin) 20170408
rtsp demo code：https://github.com/GStreamer/gst-rtsp-server/blob/master/examples/test-launch.c
1.Do not add imxvideoconvert_g2d element, direct transmission after encoding
Enter command:
./test-launch -p 554 "v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=1920,height=1080 ! queue ! vpuenc_h264 bitrate=4096 ! rtph264pay name=pay0 pt=96"
Test results:
It is normal that the input source is 60fps and it is still 60fps after transmission
2.Add imxvideoconvert_g2d element, transmission after encoding
Enter command:
./test-launch -p 554 "v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=1920,height=1080 ! queue ! imxvideoconvert_g2d ! video/x-raw,width=1920,height=1080 ! videorate ! video/x-raw,framerate=(fraction)60/1 ! vpuenc_h264 bitrate=4096 ! rtph264pay name=pay0 pt=96"
Test results:
The input source is 60fps. After transmission, it becomes 52 ~ 54fps, so the frame is lost. Moreover, the 4m code rate was originally set, but the player displays only about 3.8m code rate.
3.Add imxvideoconvert_g2d element, add a videorate element to set the frame rate, and then transmit after encoding
Enter command:
./test-launch -p 554 "v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=1920,height=1080 ! queue ! imxvideoconvert_g2d ! video/x-raw,width=1920,height=1080 ! videorate ! video/x-raw,framerate=(fraction)60/1 ! vpuenc_h264 bitrate=4096 ! rtph264pay name=pay0 pt=96"
Test results:
The input source is 60fps, and it is still 60fps after transmission, but the delay will add about 60ms. It is possible to use the video rate element to copy frames until 60fps is met. Therefore, the delay is too large.
My current imxvideoconvert_g2d element version:
My current videorate element version:
My current vpuenc_h264 element version:
My question:
- How to solve the problem of losing frame rate after adding imxvideoconvert_g2d element without adding additional delay?
- Can replacing imxvideoconvert_g2d element with the latest version solve this problem? If yes, please provide the corresponding version number?
- I found that this problem only exists at 1080p and 60fps. Can the hardware performance only reach this level and cannot be optimized?
