How to make a 30fps video with imxcompositor_g2d

[toshinariagata]

I made one video using imxcompositor_g2d from four RGB videos (720x480, 30fps), but I could not make a 30fps video.
I would like to know how I can make a 30fps video with imxcompositor_g2d.
I'm using i.MX8QuadXPlusMEK+L5.4.70_2.3.

The command is as follows.

gst-launch-1.0 -v imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=720 sink_0::height=480 \
sink_1::xpos=0 sink_1::ypos=480 sink_1::width=720 sink_1::height=480 \
sink_2::xpos=720 sink_2::ypos=0 sink_2::width=720 sink_2::height=480 \
sink_3::xpos=720 sink_3::ypos=480 sink_3::width=720 sink_3::height=480 \
! 'video/x-raw,format=(string)RGB16,width=1440,height=960' ! rawvideoparse format=rgb16 width=1440 height=960 framerate=30/1 ! fpsdisplaysink video-sink="appsink" \
filesrc location='/home/root/1.rgb16' ! rawvideoparse format=rgb16 width=720 height=480 framerate=30/1 ! comp.sink_0 \
filesrc location='/home/root/2.rgb16' ! rawvideoparse format=rgb16 width=720 height=480 framerate=30/1 ! comp.sink_1 \
filesrc location='/home/root/3.rgb16' ! rawvideoparse format=rgb16 width=720 height=480 framerate=30/1 ! comp.sink_2 \
filesrc location='/home/root/4.rgb16' ! rawvideoparse format=rgb16 width=720 height=480 framerate=30/1 ! comp.sink_3

Result of fpsdisplaysink

/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 65, dropped: 0, current: 19.71, average: 18.07

[b36401]

Although VPU of i.MX8X supports up to 30fps 1080p encoding, possibly when you run these 4 720x480 streams the processor has no power for the total load (including some background processes).

[toshinariagata]

Hi. @b36401.

When I run the same command on the i.MX8QuadXPlusMEK+LF_v5.10.72-2.2.0, I can make a 25fps video.
Do you know the reason for the improvement?

[toshinariagata]

Hi. @b36401.

When I run the same command on the i.MX 8M Min EVK, I can make a 26fps video.
Does the difference in VPU make this much of a difference?
Is there any difference in the memory size used by imxcompositor_g2d?
Would you please advise if you notice anything?

[toshinariagata]

Hi.@b36401

Thank you for your reply.

I tested the fps of 1 720x480, 2 720x480, and 3 720x480.
You are right, the fps is down. So this is the power of VPU, right?
Is there any way to create 1/1440x960/30fps video from 4/720x480/30fps videos?

Create 1/720x480/30fps video from 1/720x480/30fps video.

gst-launch-1.0 -v imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=720 sink_0::height=480 \
! 'video/x-raw,format=(string)RGB16,width=720,height=480' ! rawvideoparse format=rgb16 width=720 height=480 framerate=30/1 ! fpsdisplaysink video-sink="appsink" \
filesrc location='/home/root/1.rgb16' ! rawvideoparse format=rgb16 width=720 height=480 framerate=30/1 ! comp.sink_0

/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 66, dropped: 0, current: 30.01, average: 31.19

Create 1/720x960/30fps video from 2/720x480/30fps videos.

gst-launch-1.0 -v imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=720 sink_0::height=480 \
sink_1::xpos=0 sink_1::ypos=480 sink_1::width=720 sink_1::height=480 \
! 'video/x-raw,format=(string)RGB16,width=720,height=960' ! rawvideoparse format=rgb16 width=720 height=960 framerate=30/1 ! fpsdisplaysink video-sink="appsink" \
filesrc location='/home/root/1.rgb16' ! rawvideoparse format=rgb16 width=720 height=480 framerate=30/1 ! comp.sink_0 \
filesrc location='/home/root/2.rgb16' ! rawvideoparse format=rgb16 width=720 height=480 framerate=30/1 ! comp.sink_1

/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 64, dropped: 0, current: 29.97, average: 31.03

Create 1/1440x960/30fps video from 3/720x480/30fps videos.

gst-launch-1.0 -v imxcompositor_g2d name=comp \
sink_0::xpos=0 sink_0::ypos=0 sink_0::width=720 sink_0::height=480 \
sink_1::xpos=0 sink_1::ypos=480 sink_1::width=720 sink_1::height=480 \
sink_2::xpos=720 sink_2::ypos=0 sink_2::width=720 sink_2::height=480 \
! 'video/x-raw,format=(string)RGB16,width=1440,height=960' ! rawvideoparse format=rgb16 width=1440 height=960 framerate=30/1 ! fpsdisplaysink video-sink="appsink" \
filesrc location='/home/root/1.rgb16' ! rawvideoparse format=rgb16 width=720 height=480 framerate=30/1 ! comp.sink_0 \
filesrc location='/home/root/2.rgb16' ! rawvideoparse format=rgb16 width=720 height=480 framerate=30/1 ! comp.sink_1 \
filesrc location='/home/root/3.rgb16' ! rawvideoparse format=rgb16 width=720 height=480 framerate=30/1 ! comp.sink_2

/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 66, dropped: 0, current: 20.62, average: 20.87