imxeglvivsink inside a GTK window in Wayland

[sjimenez]

We are using an imx6-based board running Linux with Wayland. I need use the Vivante GPU's imxeglvivsink to display video inside a GTK+3 window with GStreamer. The problem is that the video with the imxeglvivsink plugin does not anchor to my app's window when running in Wayland platform. As a result, the video surface appears detached from the GTK window. Looking at the imxeglvivsink plugin source (gst_imx_egl_viv_sink_egl_platform_init_window()) for wayland platform (egl_platform_wayland), a window_handle is passed but never used.

My GTK app passes the wl_surface of the app's window to the GStreamer bus sync handler, and the imxeglvivsink sees this. However, I am not sure how to take use the window_handle inside the gst_imx_egl_viv_sink_egl_platform_init_window() function to display video inside my GTK app's window, similar to how the x11 (platform egl_platform_x11). Any guidelines on how to do this?

Thank you

[Bio_TICFSL]

Hello,

imxeglvivsink is not nxp GST plugin, please report to github.

Regards

[Bio_TICFSL]

Hello Samuel,

I didn't see imxeglvivsink plugin in 4.14.98 release.  There are sinks like xvimagesink, waylandsink, overlysink imxv4l2sink  glimagesink etc.

What do you want to know?  I am sorry I cannot understand your questions well.

what do you mean by "the gst_imx_egl_viv_sink_egl_platform_init_window() function to display video inside my GTK app's window, similar to how the x11 (platform egl_platform_x11)"

We cannot develop code for you, please be specific to what you want to know. Or send us test reproducible code.

Thanks

[Bio_TICFSL]

is something similar to this (c - Gstreamer video in GTK Window - Stack Overflow ) you want to achieve ?

[sjimenez]

Hello,

Thanks for the reply.

In general, I need to use a GPU-based video sink to render video inside a GTK application. I am using a Wayland, so I cannot use ximagesink or xvimagesink, aside from the fact that these are CPU-intensive.

I started by using a sample app using waylandsink from the gst-plugins-bad repo (here). This works fine. This would play a video test (videotestsrc) displayed in a GTK app window, like so:

Next, I replace waylandsink (which is CPU-intensive) by a GPU-based video sink. The imxeglvivsink plugin, from the GStreamer-imx plugins (here), serves this purpose. However, when using imxeglvivsink, the video does not get overlayed on a GTK window. Instead, it will create its own surface while leaving the GTK window blank.

When looking at the code for the imxeglvivsink, I noticed that the window_handle that gets passed by my GTK application through the GStreamer 'prepare-window-handle' bus sync handler message, does not get used (see egl_platform_wayland.c#L510).

This is in contrast to when the imxeglvivsink is built for the x11 platform. In x11 platform, the window_handle is used as the parent x11 window (see egl_platform_x11.c#L141). As a result, the imxeglvivsink plugin works in x11 platform. This was tested on an imx6qsasbresd reference board.

Since I need to use Wayland, is it possible to make use of the window_handle passed to the imxeglvivsink in Wayland platform to render video inside a GTK application (similar to how waylandsink does)?

Thank you.