GPU hang after multiple suspend/resume cycles

[zbigniewbauer]

We are running Android application that uses AlarmManager and RTC_WAKEUP to wakeup the system every
30 seconds from deep sleep and perform some simple tasks.

We discovered that Android 7 and 8 hangs after ~700 suspend/resume cycles.
The issue was reproduced on Sabre SD iMX6q platform running Android n7.1.2_2.0.0-ga and mx-o8.0.0_1.0.0_ga.

After adding some debug we traced the failure to kernel_imx/drivers/mxc/gpu-viv/hal/os/linux/kernel/gc_hal_kernel_os.c file.
After adding debug, the last few lines captured just before the system system hang and before (imX6 watchdog triggers) are:
__set_current_state(TASK_INTERRUPTIBLE);
spin_unlock_irq(&signal->obj.wait.lock);
timeout = schedule_timeout(timeout);

This looks like the problem is related to gpu-viv driver or to Freescale/NXP proprietary graphics libraries.

What can be the cause of this issue?

Regards,
Z

[zbigniewbauer]

The GPU hung issue can be reproduced on Sabre iMX6Q development board.
Image used in test: android_N7.1.2_2.0.0_image_6dqpsabresd.tar.gz downloaded from NXP website (https://www.nxp.com/webapp/Download?colCode=IMX_N7.1.2_2.0.0_ANDROID_DEMO_SD_BSP&appType=license&loc...)

Attached is MainActivity.java file from Android test application that was used to test suspend/resume. The test sets an alarm to wake up the system (partial wake up, display stays off).

On alarm the app runs a simple task (2 seconds sleep, set new alarm). To speed up the test process the application sets new alarm every 12 seconds.
During the test the USB client cable should be disconnected to allow the system to go into full sleep mode.
Attached is a partial log file (first boot, a few first suspend resume cycles, and a few last suspend/resume cycles before the system hung and reset).

The boot arguments were modified to print debug messages during suspend/resume:
setenv bootargs console=ttymxc0,115200 earlyprintk=serial,ttymxc0,115200 initcall_debug ignore_loglevel no_console_suspend init=/init video=mxcfb0:dev=ldb,fbpix=RGB32,bpp=32 video=mxcfb1:off video=mxcfb2:off video=mxcfb3:off vmalloc=128M androidboot.console=ttymxc0 consoleblank=0 androidboot.hardware=freescale androidboot.selinux=permissive cma=448M buildvariant=user

The system hang during resume process in gpu driver.

The same test run on Sabre DP with android_N7.1.1_1.0.0_image_6dqpsabresd.tar.gz image works fine. The test run for a several days without any issues.

[Bio_TICFSL]

Sorry, we don´t have any way to reproduced this but your issue is hardware. Please check it.

Regards

[zbigniewbauer]

This can't be a hardware issue. I tested this on two different platforms with same result.

Android 7.1.1 release works fine, Android 7.2.2 release hung/resets after ~700 suspend/resume cycles.

This issue can be reproduced on NXP SABRE iMX6Q reference platform.

Can you please try to reproduce this on your side. This is a show stopper issue for us.

Regards,

Z

[Bio_TICFSL]

Can you please check whether attached patch is helpful?

[zbigniewbauer]

The patch fixed GPU hung issue. I was able to run suspend/resume test for several days without any problems.

Thanks,

Z

[Bio_TICFSL]

Hello,

Can you please state the detailed duplicated steps?

Can you please provide the detailed failure log for the failure? It hangs before suspend or after resume?

Regards