Device: S32K312
Toolchain: Green Hills ELXR (compiler)
HSE Firmware: s32k312_hse_fw_0.13.0_2.55.0_pb250129.bin
Debugger: Lauterbach TRACE32
Software: AUTOSAR RTD-based Bootloader (FBL) + Application (APP), two-image structure
ISSUE SUMMARY
On a subset of production units, the CPU hangs immediately after a Functional
(software) reset. The same units always boot correctly after a Destructive
(power-on) reset. The hang does not reproduce on our reference/known-good units.
EVIDENCE THAT AN NMI OCCURS BEFORE ANY APPLICATION CODE EXECUTES
1) CPU context captured at the hang point (auto-stacked exception frame):
- R0-R3 = 0x00000000, R12 = 0x00000000
- LR = 0xFFFFFFFF (reset default -> no BL has executed yet)
- PC = 0x00416904 (the very first instruction address of our Reset_Handler)
- xPSR = 0x01000000
2) SCB->ICSR = 0x00000802
- VECTACTIVE[8:0] = 2 -> NMI is the currently active exception
- RETTOBASE = 1
This confirms the CPU is currently executing inside the NMI handler.
3) Our vector table entry for the NMI offset correctly points to our own
default exception handler, so this is a genuine NMI event, not vector
table corruption.
REGISTERS CHECKED AT THE SAME HANG STATE (all read as clean / inactive)
- MC_RGM_DES = 0x00000000 (not a destructive reset)
- MC_RGM_FES = 0x20000000 (bit 29 only) (only "software functional reset"
flag set, no other functional
reset source flagged)
- FCCU: STAT, N2AF_STATUS, A2FF_STATUS, N2FF_STATUS, NCF_S0, IRQ_STAT
all = 0x00000000
- CMU_FC instances 0, 3, 4: SR = 0x00000000 (no frequency high/low fault)
- PMC LVSC = 0x00000000 (no LVD/HVD flag, latched or live)
- ERM (0x4025C000): could not be read on either good or failing units
(likely clock-gated in our configuration), so ERM status is unverified.
QUESTIONS
1. Are there any NMI sources -- other than FCCU / CMU_FC / PMC / MC_RGM --
that could fire before the application's Reset_Handler executes its
first instruction?
2. Since the HSE subsystem runs independently of the application core, is
it possible for an application-core Functional reset (which does not
reset HSE) to create a state mismatch that triggers an NMI on the
application core?
3. Is there a known errata for S32K312 matching this symptom (NMI only on
functional/software reset, never on power-on reset)?
Any guidance on additional registers to check, or documentation covering
NMI sources outside FCCU / ERM / CMU_FC / PMC, would be greatly appreciated.
Hi @Chibeom,
Could you please read registers MU_0.MUB CSSR0 and MU_1.MUB CSSR0 at the hang state, and confirm whether bit 0 (NMIC) is set in either of them?
Thank you
Hi @danielmartynek ,
Thank you for pointing us to MU_0.MUB / MU_1.MUB CSSR0.
CSSR0 (bit 0, NMIC) on both MU_0.MUB and MU_1.MUB reads 0x00000000 on the
failing unit at the hang state, so the MU->NMI request path (CCR0[NMI] /
CSSR0[NMIC]) does not appear to be pending.
However, while comparing MU registers between a known-good unit and a
failing unit (both captured at the identical hang-state address range),
we found a consistent difference:
Good unit Failing unit
MU_0.MUB VER 0x0300000F 0x0300000F (identical)
MU_0.MUB PAR 0x20200404 0x20200404 (identical)
MU_0.MUB CR 0x00000000 0x00000000 (identical)
MU_0.MUB SR 0x00000000 0x00000002 <- MURIP set
MU_1.MUB VER/PAR/CR: identical between good and failing units
MU_1.MUB SR 0x00000000 0x00000002 <- MURIP set
So on BOTH MU instances, SR bit 1 (MURIP) is set only on the failing
unit, consistently. Per the reference manual, MURIP indicates that
"processor A" has issued an MU reset, and can only be cleared by a
system reset (not by an MU reset).
Since the CPU is frozen inside the NMI handler before executing any
application code, it could not have cleared this flag itself, so it
must have been set prior to (or as part of) this boot sequence.
We'd appreciate your input on the following:
1. For MU_0.MUB and MU_1.MUB, which processor is "processor A" (i.e.
who sets MURIP)? Our header only exposes the "MUB" register block at
the application-core-accessible address -- does this imply the
application core is always "processor B" and HSE is "processor A"
for these instances?
2. Does "system reset" (required to clear MURIP) include a Functional/SW
reset of the application core, or only a Destructive/POR reset? If
MURIP is not cleared by our functional reset, that would explain why
it stays set across SW reset while it is clear after power-on.
3. Independent of the NMI question: is a set/stuck MURIP flag itself
expected or considered anomalous during normal operation?
4. Since CSSR0[NMIC] currently reads 0, is it possible for hardware to
auto-clear NMIC upon NMI exception entry, or does it only clear via
an explicit software write (in which case NMIC=0 would mean the
MU->NMI channel was never asserted in the first place)?
Thanks again for your help so far.
Hi @Chibeom,
I'm sorry for the delay. I was out of office for two days.
1. Yes, the HSE_B core controls the MUA interfaces of MU_0 and MU_1.
2. Any system reset should reset MURIP.
3. I would consider this an anomaly, as I do not have much information about it.
4. It requires an explicit write, as it is a W1C register.
Can you make sure that HSE_B is inactive at the time the functional reset is triggered?
Also, what is the state of HSE_B while the application is stuck in the NMI handler?
Can you read the standard HSE GPR (0x4039_C028), FSR, and GSR registers on the MU_0 B side?
Do you use the NMI pin in the application?
Regards,
Daniel
Hi Daniel,
Please find three combined register-dump screenshots attached, followed
by our findings organized by your questions.
--------------------------------------------------------
ATTACHMENTS
--------------------------------------------------------
Attachment 1: GOOD unit (Secure Debug enabled, running normally)
Attachment 2: FAILING unit, immediately BEFORE the functional reset is
triggered (normal operation)
Attachment 3: FAILING unit, AFTER the functional reset, stuck in the
NMI handler (hang state)
--------------------------------------------------------
FINDINGS
1) HSE_B activity at the time the functional reset is triggered, and
2) state of HSE_B while stuck in the NMI handler:
Comparing Attachment 2 (before reset) and Attachment 3 (after reset,
hang state) on the failing unit, every register we checked reads
IDENTICALLY before and after the reset:
- MU_0.MUB / MU_1.MUB TSR = 0x0000000F, RSR = 0x00000000 (no pending
messages on transmit/receive channels, unchanged by the reset)
- MU_0.MUB GSR = 0x00000000 (unchanged)
- MU_0.MUB FSR = 0x03600000 (unchanged)
- HSE GPR (0x4039C028) = 0x000001C1 (unchanged)
- MU_0.MUB / MU_1.MUB SR bit 1 (MURIP) = 0x00000002 -- already set
BEFORE the reset is triggered, and remains set, unchanged, after
the reset
So MURIP was already set prior to this reset cycle, and the
functional reset itself does not change any of these HSE-related
registers.
For reference, on a good unit with the same Secure Debug
configuration (Attachment 1), MURIP reads 0x00000000 on both
MU_0.MUB and MU_1.MUB, while HSE GPR and WKPU NCR read the same
values as the failing unit.
3) Regarding whether a set/stuck MURIP is anomalous:
Understood, thank you for confirming.
4) NMI pin usage:
We do not use the WKPU-routed NMI path (WKPU_IP_USED is not enabled;
no WKPU driver code is compiled into either our bootloader or
application image). WKPU NCR (0x402B4008) = 0x60000000 identically
across all three attachments. NSR = 0x00000000 in all cases. Since
this is unchanged across all units and conditions, we don't believe
an external/WKPU-routed NMI source is involved.
SUMMARY OF FINDINGS SO FAR
MURIP (MU_0.MUB and MU_1.MUB SR bit 1) is already set on the failing
unit BEFORE the functional reset is even triggered, and remains
unchanged throughout the hang. It reads 0 on a good unit with the same
Secure Debug configuration. This is the only consistent, reproducible
difference we have found across every register we've compared (FCCU,
CMU_FC, PMC, WKPU, and MU CSSR0/GSR/TSR/RSR/GPR/FSR).
Since MURIP is set by "processor A" (HSE_B) and should be cleared by
"any system reset" per your answer, and since it is already set before
our functional reset is triggered (and the reset itself does not appear
to change it), this suggests HSE_B issued an MU reset at some earlier
point that was never cleared by a "system reset" recognized by HSE_B.
QUESTIONS
1. Is there a way to determine, from the HSE side, what would cause
HSE_B (processor A) to issue an MU reset in the first place? We'd
like to understand why MURIP gets set at all.
2. Is there a recommended way for us to trigger a reset that HSE_B
recognizes as a "system reset" (to clear MURIP) from application
software, short of a full power cycle?
3. Could a stuck MURIP flag on the application-core side be related to
the NMI we are observing, or are these more likely two independent
symptoms of the same earlier event?
Thanks again for your continued help with this.
Hello @Chibeom,
Thank you for the detailed register dumps. I have escalated the questions around MURIP behavior and the potential NMI path between HSE_B and CM7_0 to our internal HSE team, as this seems to be not documented. I will get back to you once I have their input.
Hello @danielmartynek,
Thank you for the update, and for escalating the MURIP / NMI path
question to your internal HSE team. We appreciate it, and we'll wait for
their input.
In the meantime, we found an additional data point that may be relevant,
so we wanted to share it now rather than wait.
While comparing OTP fields in the UTEST Flash area between a good unit
and a failing unit, we found a difference in the Lifecycle slots.
CUST_DEL (0x1B000220-22F) and OEM_PROD (0x1B000230-23F) are identically
programmed (0x55AA50AF across all words) on both the good unit and the
failing unit.
The difference is in the IN_FIELD slot (0x1B000240-24F):
- Good unit: begins being programmed
- Failing unit: reads as unprogrammed (0xFFFFFFFF)
We are still double-checking the exact byte pattern within the IN_FIELD
slot on our side, but the good/failing difference at this slot appears
consistent.
Could you clarify:
1. Does this suggest that the failing unit's configuration became
corrupted or incomplete partway through the transition into
IN_FIELD?
2. Could an incomplete or missing lifecycle advancement to IN_FIELD
explain the NMI/hang behavior we have been investigating in this
thread?
3. Is there a safe way to check or complete this lifecycle advancement
on the failing units, without a full production re-flow?
Thanks again for your help.
Hi @Chibeom,
Based on the memory view, OEM_PROD = Inactive, IN_FIELD = Erased.
Can you please first read the DCM registers: RM, rev.12, Section 39.3.1 DCM memory map.
And Section 38.2.3 Read-Only GPR On Destructive Reset 3 (DCMROD3)?
You can also use the HSE_FW APIs to get the LC attribute?
Thank you
Hi @danielmartynek ,
Thank you for pointing us to the DCM memory map and DCMROD3. We captured DCMSTAT (0h), DCMLCS (8h), DCMLCS_2 (80h), and DCMROD3 (208h) on both units, and decoded them against RM rev.9.
----------------------------------------------------
CAPTURED VALUES
----------------------------------------------------
Good unit:
- DCMSTAT (0h) = 0x00000E11
- DCMLCS (8h) = 0x00000000
- DCMLCS_2 (80h) = 0x00000000
- DCMROD3 (208h) = 0x00000000
Failing unit:
- DCMSTAT (0h) = 0x00000E03
- DCMLCS (8h) = 0x06184104
- DCMLCS_2 (80h) = 0x00000006
- DCMROD3 (208h) = 0x00400000
----------------------------------------------------
DECODED FIELDS (FAILING UNIT ONLY, since good unit reads all-zero)
----------------------------------------------------
DCMSTAT:
- bit1 DCMERR = 1 (DCM completed with error) -- good unit has this bit = 0
- bit4 DCMLCST = 0 (LC scanning status not "completed successfully") -- good unit has this bit = 1
DCMLCS:
- bits 21-19 DCMLCC4 (IN_FIELD Marking) = 011b = "Region is erased/virgin"
- bits 15-13 DCMLCC3 (OEM_PROD Marking) = 010b = "Marked as inactive"
- bits 27-25 DCMLCC5 (Pre-FA Marking) = 011b = "erased/virgin"
- All associated *_ECE/*_CFE/*_CSS bits = 0.
DCMLCS_2:
- bits 3-1 DCMLCC6 (FA Marking) = 011b = "erased/virgin"
DCMROD3:
- bit22 LC_ERR = 1 ("Error In Life Cycle Scanning")
This is consistent with the UTEST OTP dump we shared earlier: the IN_FIELD slot on the failing unit reads as erased/virgin.
----------------------------------------------------
HSE_FW API RESULT (HseReadLifecycle) ON THE FAILING UNIT
----------------------------------------------------
HseReadLifecycle() returns 0x10 = HSE_LC_IN_FIELD. So from the HSE firmware's point of view, the current lifecycle is already IN_FIELD.
This appears to conflict with the DCM/OTP data above: DCM's DCMLCC4 field reads IN_FIELD marking as "erased/virgin," and the UTEST OTP IN_FIELD slot (0x1B000240h onward) reads as unprogrammed (0xFFFFFFFF), yet the HSE API reports the lifecycle as confirmed IN_FIELD.
We wanted to share this as-is rather than draw a conclusion, since we don't know whether HSE tracks lifecycle through a separate/secure store independent of the DCM flash marking, or whether this indicates the marking itself is the problem.
Regards,
Chibeom
Hello @Chibeom,
Thanks for the data.
Since the IN_FIELD slot is still in the erased state, could you try setting the attribute again to advance it?
As I mentioned, the case is currently under internal discussion.
I will update this thread as soon as I have any new information.
Hi @Chibeom,
Probably the HSE service responsible for advancing the Life Cycle (LC) was interrupted, leaving the LC in this state.
The LC and LC Control (DCMLCC) register reports 0x77 (IN_FIELD) as the HSE_FW does, but the UTEST area is not programmed correctly. In theory, you could program the UTEST IN_FIELD slot using a debugger, which should clear the DCM error.
Regards,
Daniel
Hello @danielmartynek
We tried setting the IN_FIELD attribute again on the failing unit, as suggested.
Result: HSE_SRV_RSP_NOT_ALLOWED (0xAA55A21C)
Regards,
Chibeom
Thank you for the suggestion to program the UTEST IN_FIELD slot using a debugger.
We checked our internal OTP field reference table, and the IN_FIELD lifecycle slot (1B00_0240-024F) is listed as write-protected for any master except HSE once LC > MCU_PROD (OEM_PROD). Since HseReadLifecycle() on this unit already reports IN_FIELD, this LC condition appears to already be met.
Could you clarify how a debugger write to this slot would be expected to succeed under this protection rule? Is there a specific procedure, mode, or authentication step required for the debugger to be treated as an allowed master in this case?
Separately, do you have any findings yet on why the LC advancement to IN_FIELD was left in this partial state in the first place? We'd like to understand the root cause, not just the recovery step, if that analysis is available.
Thank you,
Chibeom
Hi @Chibeom,
Thank you for the information.
It seems there is no option to recover the MCU at this point.
One possibility is that the HSE set attribute service request to advance the LC was interrupted by a system reset (I understand the LC was not advanced using the LCW within the IVT).:
- Do you read the HSE response of the service request? Do you log whether there was an error?
- Before triggering the service, do you verify that HSE_STATUS_INIT_OK is set?
How many boards/MCUs are affected by this issue? Is it limited to a few units, or have you observed it across a larger number of devices?
Thank you,
Daniel
Hi @Chibeom,
Do you have any update?
Thank you
Hi @danielmartynek ,
Apologies for the delayed response, and thank you for the questions. Here are our answers:
1. For the LC advance sequence, we read the HSE service responses (from the DebugAuth, AdvanceLifecycle, and ReadLifecycle calls) and use them to determine an overall success/fail status. We don't log which specific call failed or the individual error code -- only the overall OK/FAIL result exists, and it is not persisted anywhere. So we have no record of what happened during the original LC advancement on these units.
2. Neither our LC-update function nor its caller explicitly checks HSE_STATUS_INIT_OK before triggering the AdvanceLifecycle service. We check HSE_STATUS_INIT_OK once at the beginning of our download flow, by reading the MU_0.MUB FSR register (0x4038C104) and deriving hseStatus_t from it (mask/shift on FSR bits 16-31, as done in Hse_Ip_GetHseStatus). This check is not repeated before the lifecycle advancement step later in the flow.
For additional context, our production equipment log for the failing units shows the following sequence: FSR check completed (OK) -> App download -> Secure Debug Enable -> FAIL Note that "Secure Debug Enable" in our equipment log refers to the entire procedure that includes the LC advancement (DebugAuth, AdvanceLifecycle, and ReadLifecycle together) -- it's logged as a single pass/fail step, so we cannot tell from this log which of the sub-steps actually failed.
Our debug equipment (TRACE32) has checked HSE_STATUS_INIT_OK as part of our existing debug flow, but our download/programming equipment (production line) may not have consistently checked this at the point in the sequence where the LC advancement is triggered.
Regarding the number of affected units: we currently have 2 boards/MCUs showing this issue.
We also have two follow-up questions:
- Is reading the FSR register and deriving hseStatus_t from it this way a valid/recommended way to check HSE_STATUS_INIT_OK?
- We currently check HSE_STATUS_INIT_OK once at the beginning of our download flow, via this register read. Should it also be checked specifically before triggering the lifecycle advancement?
Thank you, Chibeom
Hi @danielmartynek ,
I hope you're doing well. I wanted to check in and see if there have been any updates on this thread.
We'd appreciate any guidance you can share when you get a chance.
Thank you,
Chibeom
Hi @Chibeom,
Apologies for the delay — I have been waiting for feedback from the HSE team, but have not received a clear explanation for the NMI.
The LC_ERR flag in DCMROD3: it can trigger an NMI only if FCCU NCF 3 is configured to generate one.
To your questions:
-
Yes, that is correct.
-
The HSE_STATUS_INIT_OK flag should be checked after any system reset. The application must wait for this flag before changing system clocks or using any HSE service. Once set, it remains set. The application can additionally poll the WFI flag of the HSE_B core (PRTN0_CORE2_STAT[WFI]) to determine whether the HSE_B is busy or idle.
On the topic of the interrupted LC advancement — there is one more possible root cause worth investigating. Changing the lifecycle modifies the contents of the UTEST flash, and the UTEST flash resides in the same RWW (Read-While-Write) partition as Code Flash Block 0. If any code is executing from Block 0 during the UTEST write, an RWW error will occur and can prevent the lifecycle change from completing successfully. To avoid this, the application must ensure there is no concurrent access to Block 0 while the LC advancement service is in progress. Note that the cache may mask this issue in most cases, but in certain corner cases, particularly when the application uses a non-synchronized event, it can result in a cache miss, making the problem visible.
Regards,
Daniel