SWAY + gstreamer on IMX8MP (Only wayland)

[jayganatra]

Hi,

I'm trying to replace the Weston compositor with Sway on an IMX8MP device. As part of this effort, I've included the Sway package for kirkstone = L5.15.52_2.1.0
in my Yocto build, as shown below:

IMAGE_INSTALL:append = " sudo sway "

Additionally, I created a new user because Sway doesn't support logging in as the root user, and I added this new user to the necessary groups:

inherit extrausers
PASSWD="\$5\$9gounUy.ZkcjiF4H\$/Xy3R25TknKrWmv/rCOJfXuDE9wecgyaVilSEymEF1C"
DEV_PASSWD="\$5\$C6cr07SRVZhrxjIm\$2OOBCoEQ.IFORLhzc1KGgXaIqU3ru1Lr9gWYvzeR8E8"
EXTRA_USERS_PARAMS = "usermod -p '${PASSWD}' root; \
useradd -m -p '${DEV_PASSWD}' -d /home/dev dev; \
groupadd devs; \
usermod -a -G sudo,video,tty,input,devs dev; \
"

Running GStreamer without initiating Sway works fine, and VPU support is confirmed to be available. I modified the permissions with the following commands:

sudo chgrp video /dev/mxc_*
sudo chmod 660 /dev/mxc_*

Here's the GStreamer command I used:

sudo -E gst-launch-1.0 playbin uri=<filename.mp4>

This setup operates smoothly, although there's a scaling issue that I'm currently overlooking.

However, when attempting to execute the same command within Sway, I encountered issues. Below are the steps I took:

1. Start seatd with the non-root user: `sudo seatd -u dev &`
2. Launch Sway: `sway &`
3. Set the Wayland display: `export WAYLAND_DISPLAY=wayland-1`
4. Run the GStreamer command: `gst-launch-1.0 playbin uri=<filename.mp4>`

In this scenario, the frame rate drops to 1.2 FPS, unlike the 30 FPS achieved outside of Sway. Attempting to launch GStreamer with `sudo -E` results in Sway crashing and the display turning off.

Below are the logs from running the GStreamer command and the sway + kernel logs:

(GStreamer logs)
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...

====== AIUR: 4.7.1 build on Jul 19 2022 06:15:03. ======
Core: MPEG4PARSER_06.20.06 build on May 7 2022 07:52:31
file: /usr/lib/imx-mm/parser/lib_mp4_parser_arm_elinux.so.3.2
------------------------
Track 00 [video_0] Enabled
Duration: 0:00:30.033333000
Language: und
Mime:
video/x-h264, parsed=(boolean)true, alignment=(string)au, stream-format=(string)avc, width=(int)1920, height=(int)720, framerate=(fra
------------------------
------------------------
Track 01 [audio_0] Enabled
Duration: 0:00:30.549333000
Language: und
Mime:
audio/mpeg, mpegversion=(int)4, channels=(int)2, rate=(int)48000, bitrate=(int)2276, stream-format=(string)raw, codec_data=(buffer)11
------------------------
XDG_RUNTIME_DIR (/run/user/1000) is not owned by us (uid 0), but by uid 1000! (This could e.g. happen if you try to connect to a non-root Pul)

====== BEEP: 4.7.1 build on Jul[ 4572.364687] remoteproc remoteproc0: powering up imx-dsp-rproc
19 2022 06:15:03. ======
Core: DSP decoder Wrapper build on [ 4572.376352] remoteproc remoteproc0: Booting fw image imx/dsp/hifi4.bin, size 804421
May 19 2022 08:49:06
file: /usr[ 4572.387226] imx-dsp-rproc 3b6e8000.dsp: Allocated carveout doesn't fit device address request
/lib/imx-mm/audio-codec/wrap/lib_[ 4572.397921] imx-dsp-rproc 3b6e8000.dsp: Allocated carveout doesn't fit device address request
dsp_wrap_arm_elinux.so
[ 4572.408902] remoteproc remoteproc0: bad phdr da 0x92400000 mem 0x408cc
[ 4572.417484] remoteproc remoteproc0: Failed to load program segments: -22
[ 4572.424384] remoteproc remoteproc0: Boot failed: -22
Error: Invalid argument
remote proc is not ready

====== BEEP: 4.7.1 build on Jul 19 2022 06:15:03. ======
Core: AAC decoder Wrapper build on Jul 25 2022 15:45:05
file: /usr/lib/imx-mm/audio-codec/wrap/lib_aacd_wrap_arm_elinux.so.3
CODEC: BLN_MAD-MMCODECS_AACD_ARM_03.09.00_ARMV8 build on Jul 25 2022 10:05:48.

====== V4L2DEC: 1.20.0 build on Jul 28 2022 06:57:16. ======
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...
Pipeline is PREROLLED ...
g pipeline to PLAYING ...

(Sway + kernel logs)
[ 4572.408902] remoteproc remoteproc0: bad phdr da 0x92400000 mem 0x408cc
[ 4572.417484] remoteproc remoteproc0: Failed to load program segments: -22
[ 4572.424384] remoteproc remoteproc0: Boot failed: -2
[ 4573.688552] audit: type=1701 audit(1709290634.022:3): auid=4294967295 uid=1000 gid=1000 ses=4294967295 pid=545 comm="sway" ex1
[wlr] [types/wlr_surface.c:748] New wlr_surface 0xaaaad6768a50 (res 0xaaaad66aca30)
 [wlr] [types/wlr_surface.c:748] New wlr_surface 0xaaaad6768d90 (res 0xaaaad66ab4e0)
 [wlr] [types/xdg_shell/wlr_xdg_surface.c:414] new xdg_surface 0xaaaad67690d0 (res 0xaaaad66c8000)
[sway/desktop/xdg_shell.c:514] New xdg_shell toplevel title='(null)' app_id='(null)'
 [wlr] [wayland] listener function for opcode 4 of zwp_linux_buffer_params_v1 is NULL
Gdk-Message: 10:57:14.036: Error reading events from display: Broken pipe
[ 4573.789319] drm_panel_disable
[ 4573.792318] panel_simple_disable enabled=1
[ 4573.796549] panel_simple_disable - Backlihgt(off) to stop lvds signal time - STBYB pull L to RSTB time: 167
[ 4574.124807] imx8mp_ldb_encoder_disable
[ 4574.128692] drm_panel_unprepare
[ 4574.131832] panel_simple_unprepare prepared=1
[ 4574.136308] panel_simple_unprepare - Restart VCC time: 500
[ 4574.678516] drm_panel_prepare
[ 4574.681512] panel_simple_prepare prepared=0
[ 4574.685836] imx8mp_ldb_encoder_enable

It appears that while GStreamer commands run correctly without Sway, integrating them into a Sway session leads to performance degradation and crashes. Any advice or solutions would be greatly appreciated.

IMX8MPLUS  

[Bio_TICFSL]

Hello,

Make sure that sway runs on AARCH64 architectures it looks like they don't, or the MX8Mplus it simple don't support it.

Regards