S32K358 + FreeRTOS: Random HardFault during PendSV_Handler

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

S32K358 + FreeRTOS: Random HardFault during PendSV_Handler

103 Views
nirmal_masilamani
Contributor IV

Hi Team,

I am facing a random HardFault issue on an S32K358 running FreeRTOS.

The application runs normally for a long duration and then suddenly hangs. After the system stops executing, the Software Watchdog (SWT) is not serviced and eventually resets the controller.

The failure is not immediate—it occurs after approximately 1 to 2 hours of continuous execution

When halted after the failure, the call stack shows:

PendSV_Handler()

HardFault_Handler()

Register values:

LR = 0xA5A5A5A5

PC = 0x00407BD9

LR = 0xA5A5A5A5 looks like a memory initialization pattern rather than a valid return address.

Other registers:

R0 = 0x204011A8

R3 = 0x2040012C

R12 = 0x20400010

Any suggestions or debugging recommendations would be greatly appreciated.

0 Kudos
Reply
3 Replies

51 Views
danielmartynek
NXP TechSupport
NXP TechSupport

Hi @nirmal_masilamani,

It looks like the task context saved during the FreeRTOS context switch has been corrupted.
PendSV is used by FreeRTOS for context switching, therefore, if the HardFault occurs inside PendSV_Handler(), it often means the scheduler is attempting to restore an invalid task context.
One possible root cause is a task stack overflow. I would recommend increasing the stack size of the tasks and enabling FreeRTOS stack overflow detection:

configCHECK_FOR_STACK_OVERFLOW

Implement: vApplicationStackOverflowHook().

Additionally, you can periodically monitor the remaining stack space of each task using uxTaskGetStackHighWaterMark(). This can help identify tasks that are running close to their stack limits before the fault occurs.

Regards,

Daniel

 

 

 

0 Kudos
Reply

47 Views
nirmal_masilamani
Contributor IV

Hello @danielmartynek ,

Thank you for response,

I already tried increased stack size, enabled stack overflow hook.

Added debug CAN msg in overflow hook, I am not receiving that message when fault occur.

Also monitoring uxTaskGetStackHighWaterMark(), when fault occur

Task 1 : 1977 × 4 ≈ 7908 bytes free

Task 2: 1971 × 4 ≈ 7884 bytes free

Task 3: 3988 × 4 ≈ 15952 bytes free

0 Kudos
Reply

16 Views
danielmartynek
NXP TechSupport
NXP TechSupport

Hi @nirmal_masilamani,

So we can probably exclude a stack overflow as the root cause.
However, the task context is still getting corrupted. The processor is restoring LR = 0xA5A5A5A5, which results in a UsageFault. 0xA5A5A5A5 is the pattern generated from tskSTACK_FILL_BYTE (0xA5U) and is used by FreeRTOS to fill task stacks when they are created.

danielmartynek_0-1784709240297.png

Therefore, if LR becomes 0xA5A5A5A5, the context is restored from a location that still contains the original stack fill pattern rather than a valid register value.

This could happen if the SP gets corrupted. In that case, PendSV_Handler() would restore the task context from a wrong location in RAM.

You should be able to identify the SRAM region from the stack pointer address. 

I would recommend that you properly protect the region by MPUs and XRDC. 

Also, are any interrupts calling FreeRTOS APIs? If so, are they using the FromISR() variants, and are their priorities configured correctly with respect to configMAX_SYSCALL_INTERRUPT_PRIORITY?

 

Regards,

Daniel

 

0 Kudos
Reply
%3CLINGO-SUB%20id%3D%22lingo-sub-2396864%22%20slang%3D%22en-US%22%20mode%3D%22CREATE%22%3ES32K358%20%2B%20FreeRTOS%3A%20Random%20HardFault%20during%20PendSV_Handler%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2396864%22%20slang%3D%22en-US%22%20mode%3D%22CREATE%22%3E%3CP%20class%3D%22%22%3EHi%20Team%2C%3C%2FP%3E%3CP%3EI%20am%20facing%20a%20random%20HardFault%20issue%20on%20an%20%3CSTRONG%3ES32K358%3C%2FSTRONG%3E%20running%20%3CSTRONG%3EFreeRTOS.%3C%2FSTRONG%3E%3C%2FP%3E%3CP%20class%3D%22%22%3EThe%20application%20runs%20normally%20for%20a%20long%20duration%20and%20then%20suddenly%20hangs.%20After%20the%20system%20stops%20executing%2C%20the%20Software%20Watchdog%20(SWT)%20is%20not%20serviced%20and%20eventually%20resets%20the%20controller.%3C%2FP%3E%3CP%3EThe%20failure%20is%20not%20immediate%E2%80%94it%20occurs%20after%20approximately%20%3CSTRONG%3E1%20to%202%20hours%3C%2FSTRONG%3E%20of%20continuous%20execution%3C%2FP%3E%3CP%20class%3D%22%22%3EWhen%20halted%20after%20the%20failure%2C%20the%20call%20stack%20shows%3A%3C%2FP%3E%3CP%20class%3D%22%22%3E%3CSTRONG%3EPendSV_Handler()%20%3C%2FSTRONG%3E%3C%2FP%3E%3CP%20class%3D%22%22%3E%3CSTRONG%3E%E2%86%93%20%3C%2FSTRONG%3E%3C%2FP%3E%3CP%20class%3D%22%22%3E%3CSTRONG%3EHardFault_Handler()%3C%2FSTRONG%3E%3C%2FP%3E%3CP%20class%3D%22%22%3E%3CSPAN%3ERegister%20values%3A%3C%2FSPAN%3E%3C%2FP%3E%3CP%20class%3D%22%22%3E%3CSTRONG%3ELR%20%3D%200xA5A5A5A5%20%3C%2FSTRONG%3E%3C%2FP%3E%3CP%20class%3D%22%22%3E%3CSTRONG%3EPC%20%3D%200x00407BD9%3C%2FSTRONG%3E%3C%2FP%3E%3CDIV%20class%3D%22%22%3E%3CDIV%20class%3D%22%22%3E%3CDIV%20class%3D%22%22%3E%3CDIV%20class%3D%22%22%3E%3CDIV%20class%3D%22%22%3E%3CDIV%20class%3D%22%22%3ELR%20%3D%200xA5A5A5A5%3CSPAN%3E%20looks%20like%20a%20memory%20initialization%20pattern%20rather%20than%20a%20valid%20return%20address.%3C%2FSPAN%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FDIV%3E%3CP%3EOther%20registers%3A%3C%2FP%3E%3CP%3E%3CSTRONG%3ER0%20%3D%200x204011A8%20%3C%2FSTRONG%3E%3C%2FP%3E%3CP%3E%3CSTRONG%3ER3%20%3D%200x2040012C%20%3C%2FSTRONG%3E%3C%2FP%3E%3CP%3E%3CSTRONG%3ER12%20%3D%200x20400010%3C%2FSTRONG%3E%3C%2FP%3E%3CP%3E%3CSTRONG%3EAny%20suggestions%20or%20debugging%20recommendations%20would%20be%20greatly%20appreciated.%3C%2FSTRONG%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2397098%22%20slang%3D%22en-US%22%20mode%3D%22CREATE%22%20translate%3D%22no%22%3ERe%3A%20S32K358%20%2B%20FreeRTOS%3A%20Random%20HardFault%20during%20PendSV_Handler%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2397098%22%20slang%3D%22en-US%22%20mode%3D%22CREATE%22%3E%3CP%3EHi%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fcommunity.nxp.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F233316%22%20target%3D%22_blank%22%3E%40nirmal_masilamani%3C%2FA%3E%2C%3C%2FP%3E%0A%3CP%3EIt%20looks%20like%20the%20task%20context%20saved%20during%20the%20FreeRTOS%20context%20switch%20has%20been%20corrupted.%3CBR%20%2F%3EPendSV%20is%20used%20by%20FreeRTOS%20for%20context%20switching%2C%20therefore%2C%20if%20the%20HardFault%20occurs%20inside%20PendSV_Handler()%2C%20it%20often%20means%20the%20scheduler%20is%20attempting%20to%20restore%20an%20invalid%20task%20context.%3CBR%20%2F%3EOne%20possible%20root%20cause%20is%20a%20task%20stack%20overflow.%20I%20would%20recommend%20increasing%20the%20stack%20size%20of%20the%20tasks%20and%20enabling%20FreeRTOS%20stack%20overflow%20detection%3A%3C%2FP%3E%0A%3CP%3E%3CSPAN%3EconfigCHECK_FOR_STACK_OVERFLOW%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%3CSPAN%3EImplement%3A%26nbsp%3B%3C%2FSPAN%3EvApplicationStackOverflowHook().%3C%2FP%3E%0A%3CP%3EAdditionally%2C%20you%20can%20periodically%20monitor%20the%20remaining%20stack%20space%20of%20each%20task%20using%20uxTaskGetStackHighWaterMark().%20This%20can%20help%20identify%20tasks%20that%20are%20running%20close%20to%20their%20stack%20limits%20before%20the%20fault%20occurs.%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%3ERegards%2C%3C%2FP%3E%0A%3CP%3EDaniel%3C%2FP%3E%0A%3CBR%20%2F%3E%0A%3CBR%20%2F%3E%0A%3CBR%20%2F%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2397124%22%20slang%3D%22en-US%22%20mode%3D%22CREATE%22%20translate%3D%22no%22%3ERe%3A%20S32K358%20%2B%20FreeRTOS%3A%20Random%20HardFault%20during%20PendSV_Handler%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2397124%22%20slang%3D%22en-US%22%20mode%3D%22CREATE%22%3E%3CP%3EHello%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fcommunity.nxp.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F160001%22%20target%3D%22_blank%22%3E%40danielmartynek%3C%2FA%3E%26nbsp%3B%2C%3C%2FP%3E%3CP%3EThank%20you%20for%20response%2C%3C%2FP%3E%3CP%3EI%20already%20tried%20increased%20stack%20size%2C%20enabled%20stack%20overflow%20hook.%3C%2FP%3E%3CP%3EAdded%20debug%20CAN%20msg%20in%20overflow%20hook%2C%20I%20am%20not%20receiving%20that%20message%20when%20fault%20occur.%3C%2FP%3E%3CP%3EAlso%20monitoring%26nbsp%3B%3CSPAN%3EuxTaskGetStackHighWaterMark()%2C%20when%20fault%20occur%3C%2FSPAN%3E%3C%2FP%3E%3CP%3ETask%201%20%3A%201977%20%C3%97%204%20%E2%89%88%207908%20bytes%20free%3C%2FP%3E%3CP%3ETask%202%3A%201971%20%C3%97%204%20%E2%89%88%207884%20bytes%20free%3C%2FP%3E%3CP%3ETask%203%3A%203988%20%C3%97%204%20%E2%89%88%2015952%20bytes%20free%3C%2FP%3E%3C%2FLINGO-BODY%3E