HSE Stuck only on first startup

[strofald]

Hello,

I successfully installed HSE FW (v0.2.55) on my S32K312 micro, and I am experiencing some issues.

I will explain the sequence that I am using right now.
I am using an 'installer' firmware that handles HSE FW installation and initial configuration.

Starting from a clean state, it performs the following steps:

- Install HSE FW through IVT method, also enabling HSE FW Feature Flag in UTEST when needed
- Check that installation is successful and read FW Version (success, I can read v0.2.55)
- Read MU Status and based on that, perform FormatKeyCatalog (success, catalog is correctly formatted)

After that, I erase DFLASH and PFLASH completely, and I flash my official AUTOSAR FW, issuing a Reset just before running it.
When I do so, I encounter the following scenario.
- After startup, the HSE receives a request for a MAC verify operation. Anyway this is targetting an empty key slot, so the HSE responds with error code HSE_SRV_RSP_KEY_EMPTY (as I would expect).
- After this operation, I perform the KeyElementSet for the key that was found empty before. Anyway when I do so, I do not receive any response from HSE and I am stuck waiting for it.
If I check the FSR and GSR registers, I can see that:

Also, at address 0x4039C028:

What can be the issue of this behavior? 
Would this be solved if I call KeyElementSet before trying to perform the Mac Verify job with that same key?
Also note that I am using the RTD MCAL drivers v6.0.0, and actually I do not perform any kind of waiting during startup to check if the HSE is alive before performing clock initialization through Mcu.

Kind regards,

Aldo

[strofald]

I also noticed that the value of the CONFIG_GPR3 register right after sending the HSE request is turned to the one below:

If I am not mistaken, the bit 27 set in the register means that the HSE FW was performing some operations in the BLOCK 2 (Data Flash) and the Host should not interfere.
Anyway I am thinking about the fact that during first startup we execute the NvM_ReadAll that fetches the data from the data flash... could this be triggering some conflicts between host and hse access?

[lukaszadrapa]

Hi @strofald 

This sounds like synchronization issue. There are shared resources between application core and HSE, so application is supposed to perform appropriate steps. It is described in section "14.6.5 Synchronizing flash read/write access between HSE and application core" in HSE firmware reference manual v2.6.

In case of clocks, the synchronization was already added to RTD drivers in version 5.0.0 (and higher). The driver is waiting for WFI bit before changing clocks because it is not allowed to change the HSE clocks while HSE is running.
But the synchronization between application core(s) and HSE is not implemented in the drivers. It's up to user to follow the recommendations described in mention section.

Regards,
Lukas

[strofald]

I understand that it is up to the application to take care of the synchronization issues but you also have to consider that access to flash memory is completely handled by MCAL drivers and in the upper layer modules of the AUTOSAR stack.

Application really does not have a mean to handle this issue without inserting some 'user-defined' code like the approach I mentioned.
Is there any plan to provide MCAL drivers that handle this kind of issue?

[lukaszadrapa]

I checked latest release of RTD 6.0.0 drivers and it's even mentioned in the driver limitations in user manual:

"3.5.5 Ensure that HSE CPU is in the WFI state before programming, erasing or execute code."

I believe it will be fixed in next releases. I will ask SW team for details. 

Regards,

Lukas

[strofald]

Another detail that I was able to see is that nothing changes if I try to switch the order of the operations:
Even if I do first the KeyElementSet and then the MacVerify operation, the error still remains.

It seems that any operation that is made right after the FW installation and key catalog formatting, triggers a fault in the HSE.
That fault is recovered if I issue a reset with the debugger, in fact after that the KeyElementSet is performed correctly (return code 0x55A5AA33 - HSE_SRV_RSP_OK)
 

[strofald]

One important thing to note is that in that scenario, if I issue a reset with the debugger and I retry the same steps a second time, they are executed successfully and I see that the HSE is not stuck.
So it seems that this behavior is triggered only at the first time after installation.

[strofald]

For those who may be helped by this post: the issue occurs because in my case the host application , during startup, is is trying to read/write from the Data Flash (BLOCK 2).
it seems that during this time the HSE is also trying to access data flash.

Currently I was not able to find any resource in Fls (4.4) or more recent driver MEM_43_INFLS that implements a mechanism of synchronization for this.
(The Mem_43_INFLS UM is saying that a synchronization mechanism should exist, but I was not able to find it in the driver)

The only solution that was working for me was to use the configuration parameter of the Fls module: FlsStartFlashAccessNotif 
Using this, I can specify a callback function to be called before every Flash access is made.
In that function I implemented a check to verify that the HSE is in WFI mode.

By doing so I was able to overcome the issue I described.

I still would like to ask: is there any implementation mechanism that NXP is planning to provide to overcome this issue? I saw that in the Mcu driver files something is actually present (check that the HSE is in WFI before modifying clock configuration)
Is there a plan to port this kind of mechanism also for the Flash drivers?

Thanks

[ovidiubriscan]

We experienced the same problem, issue is that due to cost reduction even if there is in theory a dedicated DFLASH for HSE beneath is only one dflash controller (this in the past with Calypso did not happen), and logical steps would be NXP to implement one of these:
1) Bus arbitration at the bus level, control access of the masters - transparent for the end user
2) Semaphore in the HSE FW
3) Semaphore in the FLS driver
None of these will happen in the near future unfortunately. We went with a solution to have a SW semaphore in the pre/post of CSM API calls so no NVM / FLS operations are allowed during the CSM calls.

[strofald]

Hello,
thanks for sharing your experience.
Honestly it's a bit sad because I expected that this issue would be handled by the RTD driver.
Anyway, if I may ask, how did you solve the synchronization issue that happens when during startup the host is accessing data flash (e.g. for NvM_ReadAll) and the HSE is accessing it too?

[lukaszadrapa]

I discussed this with SW team. They told me that they already added semaphores for basic synchronization to RTD 6.0.0. 

And the plan is to add more sophisticated synchronization to RTD 7.0.0 which should be release in about two months. 

To avoid synchronization issues during startup, the host should not access flash blocks used by HSE until the HSE enters idle state (wait-for-interrupt mode), so host should poll for  PRTN0_CORE2_STAT[WFI].

Regards,

Lukas