imxcompositor_g2d taking the slowest input framerate

[itaigolan]

Hello,

I'm using an imx8mp on a variscite dart board and a basler camera. I'm trying to get an overlay I generate in code to be displayed on top of the video.

In the code i created 2 general pipelines:

v4l2src -> queue1 -> compositor and appsrc -> queue2 -> compositor. The compositor is then connected to a waylandsink.

I'm able to see the overlay on top of the video but the framerate radically decreases. When I don't push up buffers to appsrc the framerate is 60fps, which drops to 30fps when I start sending empty buffers and drops more if the buffers have the overlay data in it. I've tried playing around with videorate and caps for the appsrc, v4l2src, and sink but haven't gotten the framerate up past 30fps when sending buffers. My understanding is that imxcompositor_g2d should take the framerate of the fastest pipeline as output but that doesn't seem to be the case. Any help would be appreciated.

[itaigolan]

I'm still seeing this issue. Does anyone have any ideas about how I can fix it?

[Bio_TICFSL]

Hello,

Yes, that is decrease by camera, gstreamer has nothing to do with it.

Regards

[itaigolan]

I tried 2 new pipelines, both connecting to a waylandsink wrapped around an fpsdisplaysink to get the framerate:

appsrc -> imxvideoconvert_g2d -> videorate -> queue -> imxcompositor_g2d -> sink

The above pipeline has a framerate of 30fps. The following pipeline is the same but removes imxcompositor_g2d
appsrc -> imxvideoconvert_g2d -> videorate -> queue -> sink

That pipeline has a framerate of about 50 - 60 fps so it looks like imxcompositor_g2d is causing a significant drop in framerate.

[itaigolan]

The camera runs consistently at 60fps unless I start pushing up buffers to appsrc so to me it looks like its a problem with gstreamer and not the camera. 

Update:

I just ran my application with GST_DEBUG=2 and I'm consistently getting the following warnings:

0:00:37.796856205 2317816 0xaaaabdfd4580 WARN v4l2bufferpool gstv4l2bufferpool.c:501:gst_v4l2_buffer_pool_alloc_buffer:<v4l2src0:pool0:src> newly allocated buffer 2 is not free

0:00:37.821350099 2317816 0xaaaabdfd4580 WARN imxcompositor
gstimxcompositor.c:1545:gst_imxcompositor_aggregate_frames:<imxcompositor_g2d0:sink_0> set rotate failed 0xffff6000a7e0

Do you know what these mean?