Hello,
We face an issue with weston, and/or potentially imx_drm driver. Executing following script, which emulates fast HDMI cable reconnection cycles, leads to weston crash, which, in turn, might be the result of misbehaving imx_drm driver.
Test script:
#/bin/bash
loops=10
iters=10
l=1
while [ $l -le $loops ]
do
echo "Iteration: $l/$loops"
i=1
while [ $i -le $iters ]
do
echo "Reconnecting display: $i/$iters"
echo off > /sys/class/drm/card0-HDMI-A-1/status
sleep 0.5;
echo on > /sys/class/drm/card0-HDMI-A-1/status
sleep 0.5;
i=$((i + 1))
done
sleep 5;
l=$((l + 1))
done
Kernel log:
[ 37.589206] audit: type=1701 audit(1720002160.627:20): auid=0 uid=0 gid=0 ses=2 pid=752 comm="weston" exe="/usr/bin/weston" sig=11 res=1
[ 37.722022] ------------[ cut here ]------------
[ 37.722031] [CRTC:33:crtc-0] vblank wait timed out
[ 37.722073] WARNING: CPU: 3 PID: 1 at drivers/gpu/drm/drm_atomic_helper.c:1542 drm_atomic_helper_wait_for_vblanks.part.0+0x280/0x2ac
[ 37.722094] Modules linked in: rfcomm algif_hash algif_skcipher af_alg fsl_jr_uio caam_jr caamkeyblob_desc caamhash_desc caamalg_desc crypto_engine authenc libdes mwifiex_sdio crct10dif_ce hci_uart snd_soc_imx_hdmi mwifiex btmrvl_sdio btqca btbcm btmrvl cfg80211 dw_hdmi_cec bluetooth snd_soc_fsl_sai snd_soc_fsl_aud2htx imx_sdma rfkill ina2xx snd_soc_wm8904 lm75 flexcan caam can_dev secvio error galcore(O) fuse
[ 37.722218] CPU: 3 PID: 1 Comm: systemd Tainted: G O 5.15.148-6.7.0-devel+git.bfdbfb2c85fb #1
[ 37.722225] Hardware name: Toradex Verdin iMX8M Plus WB on Dahlia Board (DT)
[ 37.722229] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 37.722236] pc : drm_atomic_helper_wait_for_vblanks.part.0+0x280/0x2ac
[ 37.722246] lr : drm_atomic_helper_wait_for_vblanks.part.0+0x280/0x2ac
[ 37.722260] sp : ffff800009c4b730
[ 37.722262] x29: ffff800009c4b780 x28: 0000000000000c36 x27: 0000000000000000
[ 37.722276] x26: 0000000000000001 x25: 0000000000000038 x24: ffff0000c0fed800
[ 37.722289] x23: 0000000000000001 x22: 0000000000000000 x21: ffff0000cc33f900
[ 37.722303] x20: ffff0000c0c3b088 x19: 0000000000000000 x18: ffffffffffffffff
[ 37.722317] x17: 0000000000000000 x16: 0000000000000000 x15: ffff800009bb5478
[ 37.722331] x14: 0000000000000000 x13: 0a74756f2064656d x12: 6974207469617720
[ 37.722343] x11: ffff800009a02690 x10: 000000000000003a x9 : 0000000000000021
[ 37.722357] x8 : 00000000ffffffff x7 : 6974207469617720 x6 : 000000000000000c
[ 37.722371] x5 : ffff0000ff7739f0 x4 : 0000000000000000 x3 : 0000000000000027
[ 37.722384] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff0000c0098000
[ 37.722399] Call trace:
[ 37.722402] drm_atomic_helper_wait_for_vblanks.part.0+0x280/0x2ac
[ 37.722413] drm_atomic_helper_wait_for_vblanks+0x18/0x30
[ 37.722424] lcdifv3_drm_atomic_commit_tail+0x54/0x70
[ 37.722432] commit_tail+0xa0/0x190
[ 37.722440] drm_atomic_helper_commit+0x170/0x190
[ 37.722448] drm_atomic_commit+0x4c/0x60
[ 37.722458] drm_client_modeset_commit_atomic+0x1d0/0x274
[ 37.722467] drm_client_modeset_commit_locked+0x5c/0x19c
[ 37.722477] drm_fb_helper_set_par+0x98/0x120
[ 37.722487] fb_set_var+0x174/0x354
[ 37.722495] fbcon_blank+0x160/0x240
[ 37.722504] do_unblank_screen+0x90/0x150
[ 37.722513] vt_ioctl+0x3dc/0x1620
[ 37.722523] tty_ioctl+0x24c/0xc40
[ 37.722532] __arm64_sys_ioctl+0xa8/0xf0
[ 37.722539] invoke_syscall+0x48/0x114
[ 37.722550] el0_svc_common.constprop.0+0x44/0xfc
[ 37.722562] do_el0_svc+0x28/0xa0
[ 37.722570] el0_svc+0x28/0x80
[ 37.722583] el0t_64_sync_handler+0xa4/0x130
[ 37.722595] el0t_64_sync+0x1a0/0x1a4
[ 37.722605] ---[ end trace c03bfda25e7292c9 ]---
[ 58.137025] [drm:drm_crtc_commit_wait] *ERROR* flip_done timed out
[ 58.143332] [drm:drm_atomic_helper_wait_for_dependencies] *ERROR* [CRTC:33:crtc-0] commit wait timed out
[ 68.376810] [drm:drm_crtc_commit_wait] *ERROR* flip_done timed out
[ 68.383119] [drm:drm_atomic_helper_wait_for_dependencies] *ERROR* [CONNECTOR:35:HDMI-A-1] commit wait timed out
[ 78.616721] [drm:drm_crtc_commit_wait] *ERROR* flip_done timed out
[ 78.623026] [drm:drm_atomic_helper_wait_for_dependencies] *ERROR* [PLANE:31:plane-0] commit wait timed out
Systemd log:
Jul 03 10:22:40 verdin-imx8mp-07174536 systemd[1]: weston.service: Main process exited, code=killed, status=11/SEGV
Jul 03 10:22:40 verdin-imx8mp-07174536 systemd[1]: weston.service: Failed with result 'signal'.
Hardware:
- iMX8MP
Software:
- Linux: NXP downstream 5.15-2.2.x
- Weston: NXP downstream 10.0.1
Please let me know if you need further details.
In some occurrences, the kernel log is different:
[ 303.042081] ------------[ cut here ]------------
[ 303.046775] [CRTC:33:crtc-0] vblank wait timed out
[ 303.051677] WARNING: CPU: 0 PID: 1 at drivers/gpu/drm/drm_atomic_helper.c:1542 drm_atomic_helper_wait_for_vblanks.part.0+0x280/0x2ac
[ 303.063742] Modules linked in: rfcomm algif_hash algif_skcipher af_alg fsl_jr_uio caam_jr caamkeyblob_desc caamhash_desc caamalg_desc crypto_engine authenc libdes crct10dif_ce snd_soc_imx_hdmi mwifiex_sdio hci_uart btqca mwifiex snd_soc_fsl_aud2htx btmrvl_sdio btbcm btmrvl dw_hdmi_cec snd_soc_fsl_sai bluetooth cfg80211 rfkill imx_sdma snd_soc_wm8904 ina2xx flexcan lm75 can_dev caam secvio error galcore(O) fuse
[ 303.100639] CPU: 0 PID: 1 Comm: systemd Tainted: G W O 5.15.148-6.7.0-devel+git.bfdbfb2c85fb #1
[ 303.110582] Hardware name: Toradex Verdin iMX8M Plus WB on Dahlia Board (DT)
[ 303.117712] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 303.124762] pc : drm_atomic_helper_wait_for_vblanks.part.0+0x280/0x2ac
[ 303.131374] lr : drm_atomic_helper_wait_for_vblanks.part.0+0x280/0x2ac
[ 303.137985] sp : ffff800009c4b990
[ 303.141341] x29: ffff800009c4b9e0 x28: 0000000000003402 x27: 0000000000000000
[ 303.148571] x26: 0000000000000001 x25: 0000000000000038 x24: ffff0000c1015800
[ 303.155798] x23: 0000000000000001 x22: 0000000000000000 x21: ffff0000c17b5b00
[ 303.163025] x20: ffff0000c1014888 x19: 0000000000000000 x18: 0000000000000001
[ 303.170252] x17: 0000000000000004 x16: 0000000000000000 x15: 0000000000000000
[ 303.177480] x14: 0000000000000020 x13: ffff800009a026a0 x12: 0000000000000651
[ 303.184709] x11: 000000000000021b x10: ffff800009a026a0 x9 : ffff800009a026a0
[ 303.191936] x8 : 00000000ffffefff x7 : ffff800009a5a6a0 x6 : ffff800009a5a6a0
[ 303.199164] x5 : 0000000000000000 x4 : ffff0000ff7379f0 x3 : ffff0000ff73a9e0
[ 303.206393] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff0000c0098000
[ 303.213620] Call trace:
[ 303.216097] drm_atomic_helper_wait_for_vblanks.part.0+0x280/0x2ac
[ 303.222360] drm_atomic_helper_wait_for_vblanks+0x18/0x30
[ 303.227827] lcdifv3_drm_atomic_commit_tail+0x54/0x70
[ 303.232942] commit_tail+0xa0/0x190
[ 303.236479] drm_atomic_helper_commit+0x170/0x190
[ 303.241246] drm_atomic_commit+0x4c/0x60
[ 303.245220] drm_client_modeset_commit_atomic+0x1d0/0x274
[ 303.250694] drm_client_modeset_commit_locked+0x5c/0x19c
[ 303.256076] drm_client_modeset_commit+0x30/0x60
[ 303.260757] drm_fbdev_client_restore+0x74/0xd0
[ 303.265347] drm_client_dev_restore+0x80/0xe0
[ 303.269762] drm_lastclose+0x6c/0x7c
[ 303.273387] drm_release+0x110/0x130
[ 303.277012] __fput+0x70/0x260
[ 303.280112] ____fput+0x10/0x20
[ 303.283297] task_work_run+0x80/0xe0
[ 303.286920] do_notify_resume+0x214/0x1280
[ 303.291072] el0_svc+0x6c/0x80
[ 303.294169] el0t_64_sync_handler+0xa4/0x130
[ 303.298499] el0t_64_sync+0x1a0/0x1a4
[ 303.302208] ---[ end trace 813df8e5850192da ]---
[ 313.418126] [drm:drm_crtc_commit_wait] *ERROR* flip_done timed out
[ 313.424425] [drm:drm_atomic_helper_wait_for_dependencies] *ERROR* [CRTC:33:crtc-0] commit wait timed out
[ 323.658159] [drm:drm_crtc_commit_wait] *ERROR* flip_done timed out
[ 323.664454] [drm:drm_atomic_helper_wait_for_dependencies] *ERROR* [CONNECTOR:35:HDMI-A-1] commit wait timed out
[ 333.898205] [drm:drm_crtc_commit_wait] *ERROR* flip_done timed out
[ 333.904501] [drm:drm_atomic_helper_wait_for_dependencies] *ERROR* [PLANE:31:plane-0] commit wait timed out