How to improve fps when using OpenGL ES with Gstreamer

[toshinariagata]

I am using i.MX8X C0 board with 5.4.70-2.3.0 bsp and devicetree is imx8qxp-mek-lcdif-rpmsg.dtb.
I execute the following command, fps will be 8.42.

gst-launch-1.0 -v v4l2src device=/dev/video3 ! 'video/x-raw,format=(string)YUY2,width=640,height=480,framerate=(fraction)30/1' ! glupload ! glcolorconvert ! gldownload ! v4l2h264enc ! avimux ! fpsdisplaysink sync=false video-sink="appsink"

I convert the USB camera image format and then convert it to h264.
Although I'd like to use a shader, the fps drops just by converting the format.
Would you please tell me how to improve fps when using OpenGL ES with Gstreamer?

[Bio_TICFSL]

Hello toshinariagata,

The MX8X has 1 shader for opengl 3.0 and 3.1, and have a VPU hardware encoder to 1080p, you are converting the video that's the reason the fps drop, once you have converted the video you will have the fps at least 30fps, since it depend on the USB speed.

Regards

[toshinariagata]

Hi. @Bio_TICFSL 

Thank you for the reply.

I used the following command to import from a file instead of a USB camera and converted them to h264.
But the fps was 8.22 with no improvement. Also, the CPU usage was 100%.
Could you please tell me how I can reduce CPU usage and improve fps?

I don't want to give up on using Opengl ES.

gst-launch-1.0 -v filesrc location='/home/root/test.yuy2' ! rawvideoparse format=yuy2 width=640 height=480 framerate=30/1 ! glupload ! glcolorconvert ! gldownload ! v4l2h264enc ! avimux ! fpsdisplaysink sync=false video-sink="appsink"

I thought that using dmabuf would improve the CPU usage and fps, so I tried the following command, but it resulted in an error.

gst-launch-1.0 -v filesrc location='/home/root/test.yuy2' ! rawvideoparse format=yuy2 width=640 height=480 framerate=30/1 ! glupload ! glcolorconvert ! gldownload ! v4l2h264enc output-io-mode=dmabuf-import ! avimux ! fpsdisplaysink sync=false video-sink="appsink"

(gst-launch-1.0:764): GStreamer-CRITICAL **: 21:10:25.828: gst_buffer_resize_range: assertion 'bufmax >= bufoffs + offset + size' failed
ERROR: from element /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0: Failed to process frame.
Additional debug info:
../sys/v4l2/gstv4l2videoenc.c(813): gst_v4l2_video_enc_handle_frame (): /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0:
Maybe be due to not enough memory or failing driver
ERROR: pipeline doesn't want to preroll.