Incorrect reset cause reported when performing functional reset on S32G

[kawin]

Hi there,

Information provided by internal and external teams indicates that something is wrong with the way U-Boot reports reset source. When performing a functional reset from Linux, VxWorks, and U-Boot, sometimes powered-on reset is reported instead. This behavior is reproducible across our organization on S32G-VNP-RDB2 Reference Board.

My own testing on BSP 30.0 U-Boot 2020.04+g7eba18e1c0 (Aug 19 2021 - 15:37:50 +0000), I saw the same behavior—reset cause is incorrectly reported after the second reset command. Subsequent resets correctly report "Functional reset".

• Power supply enabled
  • U-Boot "Reset cause" == Power-On Reset
• U-Boot reset command `reset`
  • U-Boot "Reset cause" == Functional Reset
• U-Boot reset command `reset`
  • U-Boot "Reset cause" == Power-On Reset
• U-Boot reset command `reset`
  • U-Boot "Reset cause" == Functional Reset

[tristan_mcdaniel]

Hi Kawin,

We ultimately found the root cause of the POR and it turns out to be the VR5510 PMIC. Overall, this POR is not an issue with the S32G2 or the PMIC, but it is due to the default configuration of the PMIC. I have attached the VR5510's reference manual since the below description includes several references to it.

For the RDB2/GoldBox, the default One-Time Programming (OTP) for the VR5510 sets a fault error counter (section 22.7.1 of VR5510 RM) to have a threshold of 3 as part of its fault management features. This fault error counter's job is to count the number of faults that occur both internally to and externally from the VR5510. If the threshold is met, then it will implement the configured impact. In this case, the default configured impact is for the VR5510 to assert its FS0B and RSTB pins when the threshold is met.

Based upon table 69 of the VR5510 RM, external resets that occur and that are connected through the RSTB pin of the VR5510 are considered external faults and result in the increment of the fault counter, as shown below. In the RDB2/GoldBox schematic, the RSTB pin of the VR5510 is connected to the RESET_B line, which is also connected to the RESET pin of the S32G274A. When a functional reset occurs, the RESET_B line will go low and the VR5510's fault counter will be incremented.

When the second functional reset is performed by the S32G274A, the VR5510's fault counter will reach the value of 3 and the VR5510 will assert both its RSTB (RESET_B) and FS0B pins. Due to another default OTP configuration on the VR5510, the PMIC will also assert its PGOOD pin, which turns out to be connected to the POR_B line of the RDB2/GoldBox and causes the functional reset in question to become a POR. The assertion of the PGOOD pin on the VR5510 side is covered in the excerpt below from section 27.7.2 of the VR5510 RM.

Overall, this comes down to the configuration of the VR5510 based upon the OTP programmed to it. Please let us know if you have any further questions.

Thanks,
Tristan

[kawin]

Hi Tristan, 

First of all, thank you for the detailed response. If you wouldn't mind, I have two followup questions:

1. Why was default the configuration chosen to be this way?
2. Suggestion on how to achieve consistent functional resets without triggering POR while still enabling all other safety features?

Although, as you said, the POR happens after X number of functional resets due to the default configuration of the PMIC, the confusion is what launched this series of investigations. It was strange the first I encountered it. If I issue functional reset commands with the intention of performing functional reset, getting instead a one-time POR after 2 attempts until the next power cycle of the PMIC itself is not consistent with my expectation. 

Looking at the RM, there doesn't seem to be a way to stop the PMIC from incrementing fault-counter due to external RESET. 

[tristan_mcdaniel · Accepted solution]

Hi Kawin,

Let me first try to clear-up some of the confusion around the debugging steps that we took and then come back to your two main questions.

When debugging this issue, we knew the the POR_B pin on the S32G274A is input only. This is highlighted within the IOMUX spreadsheet attached to the S32G2 RM. Since this is input only, by looking at the POR_B test point on the RDB2 and seeing that it is being asserted whenever the POR in question occurs, we knew that the root cause was outside of the scope of the S32G274A. To further support this, the reset escalations described in the reset chapter (chapter 28) of the S32G2 RM shows that functional resets can be escalated to destructive resets, but not a POR. If a destructive reset was occurring, we would be able to see it through the MC_RGM DES register, but we did not see any indication of that. In addition, we also knew that the problem was not with u-boot since the functional reset is being correctly requested and if it was u-boot, the POR_B test point on the board would not be asserted.

Looking into the RDB2 schematic, there are ultimately two main sources connected to the POR_B line that could assert POR_B: (1) the VR5510 PMIC PGOOD line and (2) the POR button on the board. Since the button is not causing the issue, we started looking into the PMIC and studying its register values before and after the POR in question. It is through this that we found the overall cause.

Moving forward to your two questions:

1. Why was default the configuration chosen to be this way?
   • In this case, the fault error counter on the VR5510 is configurable through a set of bits in the FS_I_FSSM register. These include:
     • FLT_ERR_CNT_LIMIT: dictates the max value and intermediate value for the fault error counter
     • FLT_ERR_IMPACT: dictates the reaction the VR5510 takes once the fault error counter's value reaches the configured intermediate value
   • Ultimately, the default value for the fault error counter is there only to have something as a default and to show how the fault error counter works. Since its operation is dictated by registers, the user can freely conform it to their desired behavior.
   • More information about the fault error counter and these configurations can be found in section 22.7.1 of the VR5510 data sheet
2. Suggestion on how to achieve consistent functional resets without triggering POR while still enabling all other safety features?
   • For the fault error counter, you can use the FLT_ERR_IMPACT bits of the VR5510's FS_I_FSSM register that was mentioned above. Table 68 from the VR5510 Data Sheet details the different reactions that the VR5510 can take once the fault error counter reaches the intermediate value, as shown below. If you do not want the VR5510 to react to the fault error counter, setting these two bits to 00 will be the best option.

• Please note that in setting the value of the FS_I_FSSM register, you will also have to set the FS_I_NOT_FSSM register with the 1's complement value of the FS_I_FSSM register.

Please let us know if you have any additional questions.

Thanks,
Tristan

[kawin]

Hi Tristan,

Thank you for going above and beyond investigating this issue. It's safe to say that you have addressed all of my concerns. I think we can close this ticket now.

Thank you,

Kawin

[tristan_mcdaniel]

Hi Kawin,

We replicated the issue and analyzed the state of the system after the reset in question. Ultimately, the issue is not with u-boot as it is accurately reporting the reset that occurred. Instead, the system is actually performing a power-on reset when you have the specific sequence of:

1. Power-on board using power switch (will not occur if you simply use the POR button on the board)
2. Perform functional reset request either through the u-boot command "reset" or through the RESET_B button on the board
3. Board performs a functional reset and boots into the A53 cores again
4. Perform functional reset request either through the u-boot command "reset" or through the RESET_B button on the board
5. Board performs a POR and boots into the A53 cores again

The MC_RGM FES and DES registers indicate that a POR is actually occurring when they are polled after the POR in step 5. By studying the status of the PMC registers and ruling out any issues with the schematic (ex. looking into the pull-up resistors on the POR_B line), we determined that the VR5510 PMIC is likely causing the POR. This logic comes from the fact that the PGOOD line from the VR5510 is one of only a few sources of input for POR_B in the schematic (page 12 of S32G2 RDB2 Rev. D schematic, as shown below). Along with this, we are also observing RESET_B being asserted low during the entire duration of POR_B being asserted low, which is consistent with the configuration of the VR5510 for the RDB2 if a POR is issued due to a reaction to a fault.

At this point, we are working with the PMIC team to determine the fault observed by the VR5510 that causes it to issue a POR. As a result, we should have more details to share soon.

Thanks,
Tristan

[kawin]

Hi Tristan,

Truly appreciate you going above and beyond investigating this issue. Looking forward to your next response. 

Thanks,

Kawin 

[tristan_mcdaniel]

Hi Kawin,

I apologize for the delay. This ticket has been escalated to me and I am currently looking into why this is happening. On my end, I am noticing that every reset performed through the u-boot command after the power-on reset in question always shows as a functional reset, so I am looking into why only the third reset shows-up as a "power-on reset". Will have more details to share soon.

Thanks,
Tristan

[kawin]

Any update on this?