How to create a pipeline to overlay an image on a rtp video source with Gstreamer?

[fedpol]

Hi all,
I would like to overlay a logo on a video played from an IP camera.
The following pipeline from a test source works:
gst-launch-1.0 videotestsrc pattern=0 ! gdkpixbufoverlay location=/filepath/logo.png ! overlaysink

But making the same operation on a rtp stream I get only few frames at the begin and then the stream stops:

gst-launch-1.0 \
rtspsrc location=rtsp://login:pass@IP:port/channel ! rtph264depay ! vpudec \
! queue ! gdkpixbufoverlay location=/filepath/logo.png ! overlaysink

With --gst-debug-level=2 I see the following errors: 

0:00:00.914122666 2329 0xb5e890 ERROR vpu_dec_object gstvpudecobject.c:779:gst_vpu_dec_object_release_frame_buffer_to_vpu:<vpudecobject0> clearing display framebuffer failed: invalid param
0:00:00.914209333 2329 0xb5e890 ERROR vpu_dec_object gstvpudecobject.c:1016:gst_vpu_dec_object_get_gst_buffer:<vpudecobject0> gst_vpu_dec_object_release_frame_buffer_to_vpu fail.
0:00:00.914250666 2329 0xb5e890 ERROR vpu_dec_object gstvpudecobject.c:1228:gst_vpu_dec_object_decode:<vpudecobject0> gst_vpu_dec_object_get_gst_buffer fail: error

I even tried with imxcompositor_ipu to combine two videos:

gst-launch-1.0 -v \
rtspsrc location=rtsp://login:pass@IP:port/channel ! rtph264depay ! vpudec name=r \
filesrc location=/filepath/logo.png ! pngdec ! imxvideoconvert_ipu ! imagefreeze name=u \
imxcompositor_ipu start-time-selection=1 name=mix \
sink_0::xpos=0 sink_0::ypos=0 sink_0::zorder=0\
sink_1::xpos=0 sink_1::ypos=0 sink_1::zorder=1\
! overlaysink \
r. ! mix.sink_0 \
u. ! mix.sink_1

But the video is not rendered. With --gst-debug-level=2 I get many warn like:

0:00:01.803386000 2360 0xdc9950 WARN imxaggregator gstaggregator.c:1368:gst_aggregator_query_latency_unlocked:<mix> Latency query failed
and the playback hags there.#

Do someone have any idea how to properly overlay a simple image on a rtp video stream?

Thanks,

Federico

[igorpadykov]

Hi Federico

one can try pipelines described in sect.7.5 Overlaysink usage attached Linux Guide

and nxp bsps described on

https://www.nxp.com/support/developer-resources/run-time-software/i.mx-developer-resources/i.mx-6ser...

Best regards
igor
-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[fedpol]

Thank you for the answer.

I didn't know that is possible to use Overlaysink for the purpose.

A possible pipeline is then:

gst-launch-1.0 -v \
rtspsrc location=rtsp://login:pass@IP:port/channel ! rtph264depay ! vpudec ! imxvideoconvert_ipu !
overlaysink overlay-width = 1280 overlay-height = 800 zorder = 0 
filesrc location=/filepath/logo.png ! pngdec ! imxvideoconvert_ipu ! imagefreeze !
overlaysink overlay-top = 400 overlay-left = 640 overlay-width = 32 overlay-height = 32 zorder = 1

But the transparency of the 'png' is not shown.
Is it possible to overlay the image maintaining the transparency?

Thank you in advance,

Federico