iMX8M Plus Freezes During Kernel Boot Even with GPU Disabled and WDT Disabled

[kuri]

Dear NXP Support,

I am encountering a freezing issue on the iMX8M Plus during the kernel boot process, persisting across different boards. The system consistently freezes right after the DRM initialization log message:

[drm] Initialized imx-drm 1.0.0 20120507 for display-subsystem on minor 1

Notably, the freeze occurs at a temperature of 50°C and is not consistent across all boards; some exhibit this behavior while others do not. Attempts to mitigate the issue by lowering the DDR clock from 1600M to 1200M, disabling the GPU, and disabling the Watchdog Timer (WDT) have been unsuccessful.

Here are additional specifics about the situation:

• The freeze occurs immediately following the DRM initialization.
• The issue has been observed on multiple boards, indicating variability in hardware or configuration.
• Disabling the GPU and the WDT does not prevent the freeze.

Could you provide any insights or recommendations on how to further diagnose or resolve this issue? Any guidance on additional steps or configurations to try would be greatly appreciated.

Thank you for your support.

Best regards,

IMX8MPLUS 

[kuri]

Dear NXP Support,

Following up on my previous inquiry regarding the freezing issue on the iMX8M Plus during the kernel boot process, I have conducted further testing and observed an interesting behavior related to the display subsystem configuration.

Specifically, when the display subsystem is configured to use HDMI exclusively, the system freeze occurs consistently immediately after the DRM initialization log message. However, when the system is configured to use LVDS exclusively, the freeze does not occur, and the system continues to boot normally. Here is a snippet of the device tree configuration related to the display subsystem for reference:

dts

display-subsystem {

compatible = "fsl,imx-display-subsystem";

ports = <&lcdif1_disp>,

<&lcdif2_disp>,

<&lcdif3_disp>;

};

This differential behavior based on the display interface in use (HDMI vs. LVDS) suggests that the issue may be related to the specific configuration or handling of the HDMI output. Given this observation, could you provide any insights or recommendations on potential causes for this behavior? Additionally, any advice on how to further diagnose this issue or adjust configurations to prevent the system freeze when would be greatly appreciated.

Thank you for your continued support.

Best regards,

[Chavira]

Hi @kuri!

Could you please confirm they are using our L5.15.32_2.0.0 release (lf-5.15.32-2.0.0) as baseline?

[kuri]

Hi. @Chavira ,

Thank you for your answer. The base version we are using is Linux version 5.15.32-00057-gbc700a9c9ab0. We will try switching to L5.15.32_2.0.0.

Are there any differences between 0057 and 2.0.0 that could potentially cause this issue?

Best regards,

[Chavira]

Hi @kuri!

I've been trying to reproduce the issue by stressing the processor until it reaches 55-60°C in our EVK without success. I couldn't find a similar report of this issue either in the SW team database.

I checked the execution call trace of the display subsystem when using HDMI and the first places I suggest looking are in drivers/gpu/drm/imx/dw_hdmi-imx.c and drivers/phy/freescale/phy-fsl-samsung-hdmi.c which are the HDMI Controller and HDMI PHY drivers.

The reason behind is that the only difference that I see for now between LVDS and HDMI is that HDMI has its own internal PLL. If the PLL is not locked due to high temperature, that may be the cause of the issue.

Functions samsung_hdmi_phy_clk_set_rate and imx8mp_hdmi_phy_init are called after this log message is shown:

 Initialized imx-drm 1.0.0 20120507 for display-subsystem on minor 1

Could you can try adding log messages like the patch attached to check if the system hangs in those modules?

[kuri]

Hi. @Chavira ,

I've investigated the issue with the provided patch and added logging.

It appears that the system is hanging at console_lock.

Please refer to the attached image and log for more details.

What might be the possible reasons for this behavior?

Best regards,

[Chavira]

Hi @kuri!

I found a kernel thread in which they were discussing a deadlock in one of the 5.15 releases in the same function as what the customer reported. This was reproducible in some ARM-based boards. Please check:

[BUG] rockpro64 board hangs in console_init() after commit 10e14073107d - Alexandru Elisei (kernel.o...

Re: [PATCH 5.15 000/251] 5.15.47-rc2 review - Thomas Backlund (kernel.org)

The patch is contained in this message:

Re: [PATCH 5.15 000/251] 5.15.47-rc2 review - Jan Kara (kernel.org).

Could you try this patch?

[kuri]

Hi. @Chavira ,
Thank you for your reply.

I tried the patch you gave me, but there was no change in my symptoms.

Upon reviewing the logs, I have found additional information and have some questions to ask.

Additional information:
There are differences in the logs below between high temperature and room temperature.

Normal temperature: (Normal temperature, normal operation.txt L2918)

[drm:drm_helper_probe_single_connector_modes] [CONNECTOR:40:HDMI-A-1] disconnected

High temperature log: (High temperature, error operation.txt L2868)

[drm:drm_helper_probe_single_connector_modes] [CONNECTOR:40:HDMI-A-1] status updated from unknown to connected

・HDMI interface is not used
・Hardware pattern: All HDMI signals are open
However, at the time of the error, the log outputs that the HDMI was found even though the HDMI monitor was not connected.

Question:
Given the above, is there a possibility that something like this event will occur?
Please let me know if there are any other points that I should investigate.

Thank you for your support.

Best regards,

[Chavira]

Hi @kuri!

Sorry for the late reply. I managed to reproduce the issue only once. Given that the symptoms only occur when the HDMI cable is not connected, I suspect that there may be noise in the your board when the temperature is high enough.  I was shortcutting pins 18 (HDMI_5V) and 19 (HPD_CN) when the board was booting with LVDS active (using imx8mp-evk-it6263-lvds-dual-channel.dtb) and the console got stuck in the same place as what your log showed. Please see log attached. Chip was at 39°C.

Could you please check for noise in pin HPD_CN of your board in the conditions the problem occurs?

[kuri]

Hi. @Chavira ,

I have an additional question regarding specifications.

Could you tell me under what conditions the following log ( HDMI connection) appears?

Does it involve the state of REXT and HPD terminals?

[drm:drm_helper_probe_single_connector_modes] [CONNECTOR:40:HDMI-A-1] status updated from unknown to connected

 Best regards,

[kuri]

Hi. @Chavira ,

I apologize for the confusion. Upon further review, it turns out that the base version was lf-5.15.32-2.0.0.

Best regards,

[Chavira]

Hi @kuri!
Thank you for contacting NXP Support!

Are you using a custom board?

I can not reproduce the error.

Can you try to reproduce the error on an EVK board?

Best Regards!

Chavira

[kuri]

@Chavira ,
Thank you for your response.

Yes, I am using a custom board. We also have not been able to replicate the issue on an EVK board.

I would appreciate any insights into possible causes, modules, or functional blocks that might be responsible for this issue, as well as any other advice you might have.

Best regards,