freescale android kernel have scheduling problem !!!

[yangaaron]

I wrote a very simple Android application(AndroidGLSurfaceViewActivity.apk).mainly use opengl use muti-thread to draw frame on the screen

I tried to run on the freescale platform ( i.mx6q , 1GB Ram , android 4.0.4 , hdmi 1080p output )

It can't run more than one hour will cause the system to crash. (Kernel panic or rcu_preempt_state detected stalls on CPUs/tasks).

but it can run all night on the Android Emulator and Samsung Galaxy Note 8.0

PS : Android apk and program source code and detailed error messages, see attached file

Original Attachment has been moved to: rcu_preempt_state-detected-stalls-on-CPUstasks.txt.zip

Original Attachment has been moved to: AndroidGLSurfaceViewActivity.apk.zip

Original Attachment has been moved to: AndroidGLSurfaceViewActivity.7z.zip

Original Attachment has been moved to: Kernel-panic--not-syncing-Fatal-exception.txt.zip

[david5icp]

Dear All,

The scheduling problem means Android Kernel will crash anytime, anywhere when Device is running ?  Because our product also have similar problem during burn-in test.

Thanks!

David Wu

[Chris1z]

David,

Do you have a test that you can share that causes a crash?  Any logs?

[david5icp]

Sorry... I didn't in charge this issue In past few months, but i am back now...

I do the patch and test 2 devices, 1 device is normal and still running (more then 15 hr) , 1 device is stop in my test program after 15hr . The full log as attached files.

These 2 files is in same device, i just separate it. R105C-003.txt is begin form test and R105C-003-2.txt has error message as below:

---------------------------------------------------------------------------------------------------------------------

30|root@android:/ #

130|root@android:/ # BUG: scheduling while atomic: swapper/0/0x00000000

Modules linked in: dhd

[<c0052f64>] (unwind_backtrace+0x0/0x138) from [<c069296c>] (__schedule+0x828/0x

960)

[<c069296c>] (__schedule+0x828/0x960) from [<c004c714>] (cpu_idle+0xa4/0xfc)

[<c004c714>] (cpu_idle+0xa4/0xfc) from [<10689b54>] (0x10689b54)

request_suspend_state: sleep (0->3) at 56609932567084 (2014-05-06 01:43:15.82892

1065 UTC)

flashlight_control: off

dhd_early_suspend

dhd_suspend_resume_helper

---------------------------------------------------------------------------------------------------------------------

It seems still Scheduling issue...

I redo the same test with problem device again.... report it tomorrow.

BR.

David Wu

[fabio_estevam]

Does this patch help?

linux-2.6-imx.git - Freescale i.MX Linux Tree

[DraganOstojic]

Hi Fabio, in my case, just applying this patch didn't help.

[jiujinhong]

Hi Dragan,

Did you fix this issue?

[XianzhongLi]

"rcu_preempt_state detected stalls" could be fixed with the followng patch, please try

--- linux-3.0.35/arch/arm/plat-mxc/time.c.orig	2013-12-11 09:25:29.518910709 +1300
+++ linux-3.0.35/arch/arm/plat-mxc/time.c	2013-12-11 09:26:12.958912361 +1300

@@ -165,7 +165,8 @@

__raw_writel(tcmp, timer_base + V2_TCMP);

-

return 0;

+        return evt < 0x7fffffff &&

+            (int)(tcmp - __raw_readl(timer_base + V2_TCN)) < 0 ? ETIME : 0;

}

#ifdef DEBUG

[yangaaron]

Hello , Xianzhong

I patch to our kernel and try run again on our freescale platform board ( i.mx6q , 1GB Ram , hdmi 1080p output )

Test results are as follows:
system hang up and no error message within an hour (android version 4.0.4)
system reboot and no error message within an hour    (android version 4.2.2)
system reboot and no error message about two days  (android version 4.3)

[lily_zhang]

Hello, Aaron:

The above patch targets to resolve the issue "rcu_preempt_state detected stalls on CPUstasks". So from your test, do you still see "CPU stall" issue?

If it's resolved, the left issue is "system reboot and no error message sometimes". Can you please try to disable Watchdog? So we can check the sitatuions in this case.

[yangaaron]

Hi , Xiaoli

I think it is random ~

Its system crash situation occurs in the second performance without spending too much time (android jb 4.3)

About error message I only saw "rcu_preempt_state detected stalls on CPUstasks" message on android 4.0.4  before i patching it.

system hang up and no error message after patching it (android ics 4.0.4)

usually system would be direct reboot and no any error message when running on android jb 4.2 & 4.3 (whether patch it or not)

system hang up and no error message after i disabled watchdog(android jb 4.2 & 4.3)

[lily_zhang]

To disable watchdog is to check the situation when the problem happens.

After reboot, it's difficult to get more useful information.

So to disable watchdog is not solution, just for debug purpose.

[yangaaron]

Hello , Xiaoli

Yes , I know

Thank you

[qiang_li-mpu_se]

I tried this APK with "maxcpus=1", then only one CPU will be loaded in kernel, and I had also disabled the WDOG, running this APK, the system still will hang up without error messages.

[yangaaron]

Hello, Qiang

Yes, I know ~

Before I have done similar experiments

Thanks

[LeonardoSandova]

Aaron, have you tried with maxcpus=0 ? this is not the same as =1, because =0 turns off SMP.

[yangaaron]

Hello, Leonardo

Have any update for this issue ?

Thanks~

[qiang_li-mpu_se · Accepted solution]

Hi Aaron, please apply the attached two patches, from our test, the issue can be fixed.

[DraganOstojic]

Hi Qiang, I can still reproduce the issue with the patch applied. In my case it's a hard hang with no message on the serial terminal or logcat. I wonder, do you have some other patch I could try?

[qiang_li-mpu_se]

Hi Dragan, if there is no messages output from serial terminal, your issue is not kernel schedule issue.

Did you update your GPU lib and driver to lastest 4.6.9p13? (You can search "GPU P13" in community, there's GPU update for R13.4.1, JB4.2.2, JB4.3 BSP)

[DraganOstojic]

Hi Qiang, please check what I did since you replied last time. Today I re-enabled 0027-ENGR00306276 on top of p13 and I get the crash. Please see attached. Do you have any other suggestions. It appears that none of the suggestions so far work.