Access to eMMC device on S32G2-RDB2 PASSES BUT FAILS FOR S32G3-RDB3 BOARD

[vishalg]

Dear Community members,

I am trying to load and run a custom OS on S32G3-RDB3 hardware but it fails to access the eMMC device.

But the same custom OS when i try to run on S32G2-RDB2 passes and able to access the eMMC.

Seems like some clock or GPIO is not properly initialized in case on S32G3-RDB3 , because  the first access prints of the eMMC-driver to the board already fails.

Boards setup :
I am loading the custom OS image from u-boot using below command:
=> fatload mmc 0:1 83000000 s32g399a-rdb3.dtb
54165 bytes read in 17 ms (3 MiB/s)
=> fatload mmc 0:1 81200000 bootstrap_customOS.uimage
6663564 bytes read in 297 ms (21.4 MiB/s)

And after this i boot the uimage , which gets stuck at the eMMC device on S32G3-RDB3 but passes on S32G2-RDB2.

I have tried with BSP38/BSP42 on RDB3 but the issue remains the same(no access to emmc device)

Any inputs on how to resolve this error, will be really helpful and appreciated.

Regards,
Vishal G

[chenyin_h]

Hello, @vishalg 

I feel very sorry for it.

I checked it again internally, and has confirmed that the issue had been handled by supporting engineer from your region, he has sent email to your side for further supporting. then you may directly discuss the issue with him with details.

I do apologize for your inconvenience.

BR

Chenyin 

[chenyin_h]

Hello, @vishalg 

You are welcome

I have checked the case you mentioned, due our internal policy changes, the private case you created has been assigned to your regional team for further support.

Seems they were under heavy load in recent days, which may be the reason that not already handled your case, let me try delivering the information to them.

Apologize for your inconvenience.

BR

Chenyin  

[vishalg]

Hi Chenyin,

Thankyou for your reply.

My Case Number 00704716, still did not got any response or any online debug call support to show the on bench scenario.

Meanwhile for our custom hardware we are able to build BSP43.

Now we have more clarity on this issue, after several trials. We have our custom OS launched by NXP’s u-boot:
1. When we use u-boot from BSP34 (u-boot version 2020) we can access SD/eMMC on our S32G3 based h/w.
2. When we use u-boot from BSP43 (u-boot version 2022) we cannot access SD/eMMC on our S32G3 based h/w.

On NXP’s RDB3 h/w we tested with BSP42 (u-boot 2022) it failed to access SD/eMMC.
We could not access the same using BSP34 because we couldn’t download and build image for RDB3.
https://community.nxp.com/t5/S32G/BSP34-Yocto-build-fails-for-S32G3-RDB3-hardware-at-repo-sync/m-p/2...

• The only explanation is that some clocks or some power domain for this device is not properly enabled. Our emmc-driver relies on that this was already properly done by the boot loader (u-boot).
• The boot loader is accessing the MMC device so the device is at least initialized by the boot loader but maybe it also performs some "de-initialization" before starting the actual boot process in u-boot 2022 & not in u-boot 2020

Can you please let me know how shall i proceed.
Regards,
Vishal

[chenyin_h]

Hello, @vishalg 

Thanks for your clarification.

I checked the log you shared again, and found that from the failed log, the images for booting the board are also loaded from eMMC with the commands(fatload) under u-boot, which could help to confirm that the eMMC works fine under u-boot, which means the u-boot code of BSP42 could support the eMMC correctly in your case.

The issue actually occurred on your OS/supervisor booting phase, which mostly depend on the OS/supervisor driver, I understand that some OS/supervisor may rely on some configurations done by bootloader for a successfully boot, but since I am not aware of your custom OS/supervisor and the issue could not be reproduced on my side, currently it is difficult for me to directly debug on it.

From my working experience, it is better to directly debug it on your hardware setup, for the current status, I suggest raising a private ticket via support.nxp.com, our colleagues from your region would directly support on it, it is more convenient to share codes/binaries via that private portal to help on the support.

Apologize for your inconvenience.

BR

Chenyin

[vishalg]

Hi Chenyin,

Thankyou for your prompt reply.

As per your inputs, i have created a private ticket using support.nxp.com.
My Case Number 00704716, its been a week but still i did not got any response neither any updates on the ticket, with respect to the NXP colleague assigned to us as per our region. 

Even we tried writing to the old support emails, but still no response.

Can you please lets me know, how shall i proceed further to resolve this issue.

Regards,

Vishal G

[chenyin_h]

Hello, @vishalg 

Sorry for the delay from my side due on leave.

OK, I could understand the setups of your side, seems your custom OS had some issues when accessing the eMMC part, may I know if you have checked the device driver status in your custom OS? 

And since the eMMC part has been changed during the update of the RDB, may I know your board version of both RDB2 and RDB3 used in the tests?

BR

Chenyin

[chenyin_h]

Hello, @vishalg 

Thanks for sharing the log

So the requirement is to boot your images correctly based on BSP42 on RDB3? I mentioned that the successful log is based on BSP34 on RDB2, may I know if your images based on BSP42 could run correctly on RDB2? 

I suggest firstly to determine whether the issue is caused by the differences between RDB2 and RDB3 or the BSP differences, from my opinion, it is more likely the compatible issue of your own code during the BSP upgrading.

BR

Chenyin

[vishalg]

Hi Chenyin,
Thankyou for your reply.

Yes i have checked the device driver status in my customOS image for both the scenario(BSP34- "U-Boot 2020.04+g6391b468b1" and BSP42- "U-Boot 2022.04+g5a6f62071f")
with respect to the configuration and it is fine.

The observation remains same when i switch to SD card, so with eMMC and SD interface both case, the error remains same as explained in below provided boot logs.

The SDHCI driver is same for SD card and eMMC, that i am using in my customOS image.

The version of my S32g3-RDB3 hardware is "SCH-53060 REV E2" (S32G3-VPN-RDB3)
The S32G2-RDb2 based hardware is used by my counter part which is in Germany, whose version info i do not have as of now.
I tried to run the same customOS.uimage with a Bosch Harwadre that is made out of NXP S32G3-RDB3 design whose schematic and board revision details are provided above.
The processor used in Bosch hardware is S32G399. Observaions remains exactly the same.

With Bosch Hardware using BSP34- "U-Boot 2020.04+g6391b468b1" the customOS image boots and i am able to load and access eMMC.

But with BSP42- "U-Boot 2022.04+g5a6f62071f" it fails .

This test narrows down the conclusion which is the comparison is between the uboot of BSP34 and BSP42.
My suspect is there is something in BSP42 bootloader "U-Boot 2022.04+g5a6f62071f" say some clock or GPIO is not properly initialized
because the first access prints of the eMMC-driver to the board fails.

Note : Now we have tested both the uboot of BSP34 and BSP42 on S32G3 based hardware and the results are same, as it was in case of S32G2-RDB2.

Can you please help me to debug this on BSP42- "U-Boot 2022.04+g5a6f62071f" in which eMMC first access itself is not working.

Looking for your response .

Regards,
Vishal

[vishalg]

Hi Chenyin,

Thankyou for your reply.

Adding more details to explain our setup.

Our customOS Image (bootstrap_customOS.uimage) only use the bootloader while booting, as shown in our logs below, it dose not use rest of the BSP(kernel and filesystem).

Our custom image (bootstrap_customOS.uimage) is independent of BSP42 or BSP34.

We use the BSP42 bootloader or BSP34 bootloader, just to load the image into the RAM and boot from it.

So my suspect is there is something in BSP42 bootloader "U-Boot 2022.04+g5a6f62071f" say some clock or GPIO is not properly initialized because the first access prints of the eMMC-driver to the board fails.

While in case of BSP34 the bootloader which is "U-Boot 2020.04+g6391b468b1 , our custom image(bootstrap_customOS.uimage) is able to detect and access the eMMC device.

So the comparison is between the uboot of BSP34 and BSP42.

Context: Why are we using BSP42 bootloader "U-Boot 2022.04+g5a6f62071f" because  it has support for EL2 mode in S32G3, (And in bootloader of BSP34, EL2 mode is not supported) which is our use case once the eMMC is up we will host applications in EL2 mode of S32G3.

Hope this clarifies the setup and requirements details.

Awaiting for your response.

Regards,
Vishal G

[chenyin_h]

Hello, @vishalg 

Thanks for the post.

Would you mind sharing us the full log from your test?

BR

Chenyin

[vishalg]

Hi Chenyin,

Here with the Full log of my test in both the pass and fail case for your reference.

Case 1: S32G2RDB2 - S32G2-RDB2-With-BSP34-bootlogDD-eMMC-PASS.txt
Case 2: S32G3RDB3 - S32G3-RDB3-With-BSP42-bootlogDD-eMMC-fails.txt

Regards,
Vishal G