topic Re: OTFAD Problem in i.MX RT Crossover MCUs

OTFAD Problem

LeslieLee — Sun, 27 Dec 2020 05:35:07 GMT

My previous question: Use FlexSPI while decrypted with OTFAD

Sorry for posting a new thread for my problem, I am doing this as I found there are actually multiple problems stack together with OTFAD.

Right now I suspect that the way I change FlexRam is conflicting with OTFAD.

Here is some observation of my problem

Result	Modify FlexRam	Calling ITCM Ram Func	Using OTFAD
Not Running	Yes	Yes	Yes
Running	No	Yes	Yes
Running	Yes	No	Yes
Running	Yes/No	Yes/No	No

So I believe FlexRam is the first problem avoiding the RT1011 running encrypted image.

I am using MCUXpresso IDE for development and I use the following codes in ResetISR function to change FlexRam Setting.

__attribute__ ((section(".after_vectors.reset")))
void ResetISR(void) {

// Disable interrupts
__asm volatile ("cpsid i");

/* SP reset */
__asm volatile ("MSR msp, %0" : : "r" (&_vStackTop) : );
__asm volatile ("MSR psp, %0" : : "r" (&_vStackTop) : );

/* config FlexRam */
FLEXRAM->TCM_CTRL = 4;
IOMUXC_GPR->GPR17 = 0xAB;//0xAA;//0x6B;
IOMUXC_GPR->GPR16 |= 0x7;

The whole startup.c file is attached.

I use MCUBootUtility to build and flash the OTFAD encrypted image and reset the SP register suggested here.

May I know what is the proper way to change FlexRam when using OTFAD encryption at the same time?

Thanks

P.S.

Update 1:

The ram function can be executed if I add an SDK_DelayAtLeastUs before it.

But OTFAD will still have a problem when the frequency adjustment code is placed in SRAM_ITC instead of non-encrypted flash region.

Update 2:

I create a new project in case there is other stuff in my original project causing the problem.

The new project shows that when the FlexRam is modified and only two peripheral(adc, adc_etc) is inited using MCUXpresso GUI, the program will work with no problem. However, when I added one more peripheral(lpi2c or lpspi), the program will crash.

Update 3:

When the encryption range is 0x60001000-0x60004000, the program based on the new project will work, but if the encryption range is 0x60001000-0x60005000, the program won't work.

Update 4:

By toggling the LED around the code and placing the trap code, I found that for the not working program, they crash at this function in clock_config.c

CLOCK_SetMux(kCLOCK_PeriphMux, 0);

And I found the reply about XIP through OTFAD problems on RT1011, this reply said that it is not working when they place the clock adjustment code in ram? So is placing clock adjustment code in not encrypted flash is the only way to keep OTFAD working? This could be a security problem as hackers may modify that not encrypted region and execute their own code to read the decrypted code. (consider no HAB enabled).

Update 5:

It turns out changing the FlexRam layout is not the actual cause of this problem, when I replace the three lines of code for changing FlexRam with this

__asm volatile ("nop");
__asm volatile ("nop");
__asm volatile ("nop");
__asm volatile ("nop");
__asm volatile ("nop");
__asm volatile ("nop");

The program will still crash.

Update 6:

I think one of the functions will have a problem when it is placing at one specific address. When I added three nop assembly in the main function, the chip won't work, but if I remove it, it works again. I attached the working project and not the working project. Please help me to find what causes this problem.

Update 7:

I had tested placing nop assembly according to the image table, and I observe this

Thanks

Re: OTFAD Problem

mjbcswitzerland — Fri, 25 Dec 2020 23:54:19 GMT

When doing things like changing FlexRAM layout or enabling/disabling encryption make sure that you work with data and instruction barriers,
eg.
asm volatile ("dsb 0xf":::"memory");
asm volatile ("isb 0xf":::"memory");

to ensure that the commanded change actually terminated before subsequent code is executed (this might explain some dependencies on delays in the code).

Also take a look at https://www.utasker.com/iMX/RT1010.html where all of your difficultes have been solved (allowing on-the-fly FlexRAM changes and safe, dynamic control of on-the-fly decryption) and also giving full compatibility with all i.MX RT 10xx devices (using either OTFAD or BEE). It is designed for professionals who need a proven solution rather than needing to re-solve typical real-world complications and thus achieving shortest time to market at lowest costs.

It includes an OTA and field updating of encrypted code, which adds flexibility to the basic concept (removes AES key management difficulties), whilst greatly reducing the technical complications for developers.

Regards

Mark

Re: OTFAD Problem

LeslieLee — Sat, 26 Dec 2020 03:12:22 GMT

Hi,

Adding data and instruction barriers still don't work.

I had tried adding

__asm volatile ("dsb 0xF":::"memory");
__asm volatile ("isb 0xF":::"memory");

after the FlexRam layout is changed.

And I also tried changing OTFAD w1 to w0 and add barriers afterward, and then execute

CLOCK_SetMux(kCLOCK_PeriphMux, 0);

But it still doesn't work.

Thanks

Re: OTFAD Problem

LeslieLee — Sat, 26 Dec 2020 10:42:27 GMT

I don't think it is the memory barrier problem.

Please check the attached two project, you will see that their difference is the three additional nop assemble in the main function, but one will work one won't.

Thanks a lot for your help

Re: OTFAD Problem

kerryzhou — Tue, 29 Dec 2020 07:27:18 GMT

Hi LeslieLee,

Thanks a lot for your effort, your question conclusion is very good.

In your newest attached otfad_work and otfad_not_work project, just the main code with or without the "nop"? I find in two projects, both the ResetISR didn't add the configure FlexRAM code.

When you check the code is working or not working, just check the GPIO is a toggle or not toggle?

If you write the code like this:

while(1)
{
GPIO1->DR_TOGGLE = 1<<23;
SDK_DelayAtLeastUs(500000, 500000000);
}
ram_test();

The ram_test will not be called, or you just want to use :

while(1)
{
GPIO1->DR_TOGGLE = 1<<23;
SDK_DelayAtLeastUs(500000, 500000000);
}

Seems in your code, you didn't modify the flexRAM, or your newest code just for the code in ITCM not working with OTFAD?

BTW, as this question is a little complicated, we need more time to do the checking.

So, if you are working on this case for your company, could you please use your company's email create one new nxp.com account, then create the case about this question, that will help you level up the case, and we can put more time in your case.

After you create the new company email account in the www.nxp.com

You can use this method to create a new high-level case:

1. Open below SUPPORT site, click blue "Go to Tickets" in the middle.
http://www.nxp.com/support/support:SUPPORTHOME

2.Then you will be requested to Login, if you have no an account, please first Register with your business email.

3.After login, please "Create New Cases" button in the middle, then you can submit your question.

If you still want me to provide service for you, in the case content, you can write assign to kerry Zhou. Then I will help you do more checking, after I reproduce your issues, I will also check with our internal expert.

Thanks a lot for your understanding.

Best Regards,

Kerry

Re: OTFAD Problem

LeslieLee — Tue, 29 Dec 2020 07:50:38 GMT

Hi Kerry,

My current discovery is that the program not working had nothing to do with the FlexRam reconfiguration. So placing the same amount of nop instruction to replace the FlexRam layout configuration will reproduce the same problem.

And yes, I check if the project working or not by checking if the LED toggling. And the two LED toggle code wrapping BOARD_InitBootClocks is used to check if the MCU can execute that function successfully. In all the failure cases, the LED will always on, showing that it is failing somewhere inside BOARD_InitBootClocks.

So there is nothing to do with the ram_test function call after the while(1) loop, it is just some leftover code I used to test whether the program failed due to the code in ITCM. (It is not the cause)

This is not the case for any company, right now I'm still an MPhil student, but I planned to sell this project once it is finished, so the failure in OTFAD encryption is an important problem to me.

And for your information, I am using MCUBootUtilityv2.4.0 to do the encryption and flashing, MCUXpresso IDE v11.2.0 for development.

Regards,

Leslie

Re: OTFAD Problem

kerryzhou — Tue, 29 Dec 2020 08:10:42 GMT

Hi LeslieLee

OK, thanks a lot for your information.

If it is just related to the clock, this is the known knowledge.

I get the information from our internal side in the previous time, BOARD_BootClockRUN code in the flash area can't do the OTFAD encryption. You can put the BOARD_BootClockRUN code in the not secured area.

Wish it helps you!

Best Regards,

Kerry

Re: OTFAD Problem

LeslieLee — Tue, 29 Dec 2020 08:18:55 GMT

Hi Kerry,

Yeah, I already know the clock function cannot be run in an encrypted region.

That's why in the project there are extra linker scripts in the linkscripts folder, they are used to move all the clock function into ITCM and run it. I had debugged in non-encryption environment and proved that all the function is already executed in ITCM.

Also if the problem is just placing code in an encrypted region, then I will not faced a successful run when no nop instruction is added.

Regards

Leslie

Re: OTFAD Problem

kerryzhou — Tue, 29 Dec 2020 10:14:38 GMT

Hi LeslieLee

If you add the nop in the unsecured flash, and add other function code, whether it has issues or not?

In fact, when you call BOARD_BootClockRUN , not just one API, this API also contains a lot of other API, all that related API need to in the internal ITCM when you don't want to put in the flash. Do you also checked it?

Best Regards,

Kerry

Re: OTFAD Problem

LeslieLee — Tue, 29 Dec 2020 12:14:56 GMT

Hi Kerry,

If you check my update 7 you will see that I actually tried to place nop instruction everywhere in the program and only adding nop before one specific address will crash the program. And in update 3 you will also see that if I don't encrypt that address (0x60004000-0x60005000) no problem will happen.

And if I replace the nop with another function code, the error will still happen, for example, if I placed nop in after 0x60004000 and add a while loop trap at the nop's original place, the program will still not work.

I know that it is not enough to put the only BOARD_BootClockRUN in ITCM, so in the linker scripts, I put the clock_config.o and fsl_clock.o in ITCM to ensure all functions in clock_config.c and fsl_clock.c are in ITCM.

And I also checked that the suspect failing code

CLOCK_SetMux(kCLOCK_PeriphMux, 0);

even in disassembly, all the code is executing in ITCM(0x0000xxxx)

And there is one extra information you may want to know, my setting for OTFAD in MCUBootUtitlity are

Protected Region start: 0x60001000

Protected Region length: 0xFF000

Regards,

Leslie

Re: OTFAD Problem

kerryzhou — Wed, 30 Dec 2020 01:39:14 GMT

Hi LeslieLee,

Thanks for your updated information.

Now talk about your :

In update 3 you will also see that if I don't encrypt that address (0x60004000-0x60005000) no problem will happen.

if the encryption range is 0x60001000-0x60005000, the program won't work.

Do you try to divide the OTFAD region to two: one is 0X60001000-0X60004000, another one is 0x60004000-0x60005000, as you know, in the MCUBootUtility, there has 4 OTFAD region. if you use two region, whether the issue happens or not?

Any updated information, please kindly let me know.

Kerry

Re: OTFAD Problem

LeslieLee — Wed, 30 Dec 2020 02:50:59 GMT

Hi Kerry,

Yes, I did try encrypting the program with a different region. I had tried the following two settings.

0x60001000-0x60004000 + 0x60004000-0x60005000 -> not working

0x60001000-0x60004000 + 0x60005000-0x60006000 -> working

Regards,

Leslie

Re: OTFAD Problem

kerryzhou — Mon, 04 Jan 2021 09:59:34 GMT

Hi LeslieLee,

Thanks for your testing.

Do you check the memory map, which detail code in the 0x60004000-0x60005000?

Best Regards,

Kerry

Re: OTFAD Problem

LeslieLee — Mon, 04 Jan 2021 10:06:55 GMT

Hi Kerry,

You may see the screenshot in Update 7 about what code is included in that region.

I don't see the relationship of those codes related to the crash.

And I am guessing the problem is the region boundary 0x60004000 with my nop insert test.

Regards,

Leslie

Re: OTFAD Problem

kerryzhou — Tue, 05 Jan 2021 06:07:33 GMT

Hi LeslieLee,

Thanks for your reply.

Maybe really still have any other important points need to care about it.

When our secure expert back, I will check with him again.

Any updated information from my side, I will let you know.

Best Regards,

Kerry

Re: OTFAD Problem

kerryzhou — Thu, 25 Feb 2021 05:34:35 GMT

Hi LeslieLee,

About the OTFAD issues, please note these information:

1. Reason

When OTFAD is enabled, if the encrypted app is XIP running, in the app, core frequency need to be higher than FlexSPI frequency. If core frequency is lower than flexSPI, when core access the encrypted Flash area, OTFA can't decrypte the code, and will case the instruction messing issues, will case the hardfault.

In the BOARD_BootClockRUN(), we can see the core frequency from 396Mhz to external OSC24 Mhz switch, it will make the core smaller than FlexSPI frequency, as you know, the slowest FlexSPI is 30Mhz. This is the root issue caused the OTFAD limit issues.

2. Solution

When configure the clock, don't switch to the external OSC then switch to the PLL, we can switch to PLL directly. Put the Audio(Enet) Pll init code to the front side. The modification code is:

void BOARD_BootClockRUN(void)
{
// 此处略去...
/* Set Oscillator ready counter value. */
CCM->CCR = (CCM->CCR & (~CCM_CCR_OSCNT_MASK)) | CCM_CCR_OSCNT(127);
- /* Setting PeriphClk2Mux and PeriphMux to provide stable clock before PLLs are initialed */
- CLOCK_SetMux(kCLOCK_PeriphClk2Mux, 1); /* Set PERIPH_CLK2 MUX to OSC */
- CLOCK_SetMux(kCLOCK_PeriphMux, 1); /* Set PERIPH_CLK MUX to PERIPH_CLK2 */

// delete...
/* Set IPG_PODF. */
CLOCK_SetDiv(kCLOCK_IpgDiv, 3);
+ /* Init Enet PLL. */
+ CLOCK_InitEnetPll(&enetPllConfig_BOARD_BootClockRUN);
+ /* Set preperiph clock source. */
+ CLOCK_SetMux(kCLOCK_PrePeriphMux, 3);

// delete...
/* Enable Audio PLL output. */
CCM_ANALOG->PLL_AUDIO |= CCM_ANALOG_PLL_AUDIO_ENABLE_MASK;
- /* Init Enet PLL. */
- CLOCK_InitEnetPll(&enetPllConfig_BOARD_BootClockRUN);
- /* Set preperiph clock source. */
- CLOCK_SetMux(kCLOCK_PrePeriphMux, 3);

// delete...
/* Set SystemCoreClock variable. */
SystemCoreClock = BOARD_BOOTCLOCKRUN_CORE_CLOCK;
}

Re: OTFAD Problem

LeslieLee — Fri, 05 Mar 2021 07:45:04 GMT

Okay, so two months later I finally found the solution to this problem.

Following this article, I modify the BOARD_BootClockRUN function instead of putting it into the ITCM.

And Wala, it works now.

Re: OTFAD Problem

kerryzhou — Mon, 08 Mar 2021 02:18:33 GMT

Yes, that article is from our AE, and it also report from you and I transfer to him, recently, our AE have the time, so he writes that article.

If you find any issues, welcome to report to us, thanks for your contribution!

Best Regards,

kerry