[IMX8MM] Android 14 GPU crashed when first boot

[mayqueen]

Platform: IMX8MM

OS: Android 14

Kernel: 6.1.55

Good news is our IMX8MP product already done using Android 14, but bad news is we face another issue for IMX8MM product as title.

That strange because it only happens on two conditions:

1. first boot after UUU image flashing (it's fixed, 100% happened), it happened after boot complete and initialing the GUI desktop.

2. Playing 1080P video for about 30mins, and press back for the instance time (randomly)

current CMA size is 550MB, I'm tried change to 800MB but the issue still the same, note that Android 13 has no this problem, so I think it's GPU driver bug? The crash log as attached file, have any directions? Because I have no IMX8MM EVK board, could you have the same issue on the EVK?

I checked the dumpsys meminfo, it seems no memory leak.

Another concern is I saw the imx8mm.dtsi, the GPU power domain already change to GPC based, is it the root cause? BTW, my DRAM controller frequency is 1200MHz (2400MTS), I don't know GPU or VPU pll need change or not, BUT Android 13 is normal!! I have no any change on it.

Thanks a lot

[AshutoshNama]

@mayqueen Try below way, I hope this will help you:
Driver Analysis: Examine the GPU driver for incompatibilities between IMX8MM and Android 14, paying particular attention to startup and edge cases for video playback.

CMA Configuration: Verify that the CMA size is enough in high-memory circumstances; 800MB might not resolve because of memory requirements unique to the GPU.

Power Domain: Check for configuration errors or regressions affecting the stability of Android 14 by looking at GPU power domain changes in imx8mm.dtsi.

Clock Frequencies: Verify GPU/VPU PLL settings against Android 14 specifications; discrepancies might cause instability even if Android 13 is functioning well.

EVK Validation: To identify hardware-specific problems and collect repeatable data, test on an EVK board.

Best Regards,
Ashutosh Nama.

[mayqueen]

I aleady resolved about few months ago, thanks

[prakashm]

@mayqueen have you managed to solve this issue on android14 ..because i'm facing the first time boot problem and my gpu driver crashing. it would be helpful if you could share your experience on this. thanks in advance

[mayqueen]

Yes, I was fixed.

Crashed at the 1nd boot is must be happened, actually it still trigger random crashed since 2nd boot.

In my experience, it's due to memory and CMA allocation does not stable, I guess you adapt a special memory chip, right? Double check the memory calibration or hardware signal has no problem. Secondly, a workaround is trying to reduce the usable memory size in Android (modify the dts and increase the reserve memory), it will works fine.

My solution is change back to micron memory chip which NXP EVK used.

[prakashm]

in my case the android 13 works with cma that i used on adnroid14 ..but wondering how it could be the memory problem..

[mayqueen]

YES, Android 13 works because the GPU driver and power domain driver were still old structure, but it's a big jump so cannot downgrade to the same driver on Android 14, detail can follow Ashutosh Nama comment as above.

I think memory allocation is very sensitive on new drivers, this is why I decide change the memory chip or decrease memory usage to fix this issue, thanks.

[prakashm]

thank you so much for your reply mayqueen

[Zhiming_Liu]

Hi @mayqueen 

Can you share your Android13 version?

The GPU library in  Android13_2.2.0 is same as Android14, but not different in Android13_1.0.0.

I can't reproduce same issue with Android14_1.2.0. Can you also try this verison?

Thanks

Zhiming

[mayqueen]

android-14.0.0_1.2.0 still has this issue, but it can observe the happen probability is lower than 1.0.0.
My MIPI-DSI panel is 1280x800 base on panel-simple.c, is it the root cause?

But anyway, Android 13 is works perfectly, so I'm so confusing...

So I have to give up Android 14 and lose the order?

[Zhiming_Liu]

Hi @mayqueen 

As we can't reproduce same issue when first boot on EVK. Here is GPU SE team suggestion:

Customer can use Android13_1.2.0 GPU library in Android14. If they use Android 14 image, they have to use android 14 environment to compile Android 13 galcore.ko, otherwise it won't start. If you can't compile Android 13 galcore.ko in Android 14, please create a case and SE team can help(Please create case to request this in SFDC).

[mayqueen]

Update status:

1. I adapt NON-GKI defconfig as default setting, if use GKI kernel, the first boot into the desktop is no problem, but when check Android 14 version in Setting -> About, it still stuck and happen GPU crash, using webview browser also trigger this issue.

2. I tried to downgrade GPU driver to andrord-13-1.2.0 version in Android 14, I use non-GKI kernel but the issue still existed, because I face compile failed using GKI version to build ko, so just pending this way, because after observed, GPU version should not the root cause, looks like power domain issue, NON-GKI and GKI have difference crash points, e.g.

3. If I use non-GKI defconfig, it means maybe I can revert your GPCV2, power domain and clock tree modification for GKI? BUT in Android 13, you also have those chagnes but I never revert them, it works perfectly.

[mayqueen]

My Android 13 version: android-13.0.0_1.2.0

Sure, I'll try Android 14 1.2.0, my current Android 14 is 1.0.0, I think this version has unstable on GPU.