MIMXRT1064CVJ5A DTCM / ITCM access error happens, Application unstable

[fdm_85]

We have a project with a TFT display (800 * 480) .
Grafik content is managed by embedded wizard.
We are using FreeRTOS V8.2.3.
Using SDK 2.8.1
Using J-Link plus 10.1

Ported the complete project from the 1064 EvaKit board to our custom board. No ext ram or flash available.

Initialy the project went along fine. After adding more features the software became unstable.
Having troubles starting at all, becoming very slow and unresponsive/laggy when debugging.

So i stripped down to a minimal approach.
As we do not have an external RAM for the display content, we use the OC_RAM for framebuffer and dynamic heap for embedded wizard lots.

DTCM is used for stack, heap, bss, data.
FLEX-RAM allocation is left at standard as read from the related fuses (verified as 0b0000 -> 128KB ITCM, 128KB DTCM, 256KB OC-Flex Ram + 512KB OC-Ram). No changes were made.
https://www.nxp.com/docs/en/application-note/AN12077.pdf

Our minimal program features:
- internal watchdog
- FLEXRAM_EnableAccessError ISRs enabled and serviced.
- UART for debug printf in blocking mode

When the software accesses some peripherals an ITCM error occurs. (Like enable watchdog, read PLL...) [ ITCM access error happens mostly when reading Clocks or enabling clocks]
Enabling/disabling the I or D Chache has some impact.

So i tried the following steps: (each time recompiling and flashing, let the software run, disconnect debugger, 10 second power down, POR reset so SW is free running)
1. Disabled Caches:
already boots with one ITCM access error (when calling BOARD_DebugConsoleSrcFreq, WDOG_Init), only uses watchdog thread, crashes after about 14 seconds with repeated DTCM access errors

2. explizit set of "no power down of flex ram when cpu does power down", does not improve things
3. enabled instruction cache, no improvement
4. enabled data cache, seems to improve things (no more crashes after 14 seconds, but still has ITCM access errors when calling BOARD_DebugConsoleSrcFreq or WDOG_Init. But i do wonder why enabling Data Cache it fixes DTCM access errors, maybe this is related to freeRTOS scheduler? I could not find the exact spot were the error occurs.
5. added cyclic print to have some indication of system alive time (print all 5 secconds "ping" in watchdog trigger thread"

Up to this point i used this MPU config: (only en/disabling SCB_EnableDCache and SCB_EnableICache)

/* MPU configuration. */
void BOARD_ConfigMPU(void)
{
    /* Disable I cache and D cache */
    if (SCB_CCR_IC_Msk == (SCB_CCR_IC_Msk & SCB->CCR))
    {
        SCB_DisableICache();
    }
    if (SCB_CCR_DC_Msk == (SCB_CCR_DC_Msk & SCB->CCR))
    {
        SCB_DisableDCache();
    }

    /* Disable MPU */
    ARM_MPU_Disable();

    /* MPU configure:
     * Use ARM_MPU_RASR(DisableExec, AccessPermission, TypeExtField, IsShareable, IsCacheable, IsBufferable,
     * SubRegionDisable, Size)
     * API in mpu_armv7.h.
     * param DisableExec       Instruction access (XN) disable bit,0=instruction fetches enabled, 1=instruction fetches
     * disabled.
     * param AccessPermission  Data access permissions, allows you to configure read/write access for User and
     * Privileged mode.
     *      Use MACROS defined in mpu_armv7.h:
     * ARM_MPU_AP_NONE/ARM_MPU_AP_PRIV/ARM_MPU_AP_URO/ARM_MPU_AP_FULL/ARM_MPU_AP_PRO/ARM_MPU_AP_RO
     * Combine TypeExtField/IsShareable/IsCacheable/IsBufferable to configure MPU memory access attributes.
     *  TypeExtField  IsShareable  IsCacheable  IsBufferable   Memory Attribtue    Shareability        Cache
     *     0             x           0           0             Strongly Ordered    shareable
     *     0             x           0           1              Device             shareable
     *     0             0           1           0              Normal             not shareable   Outer and inner write
     * through no write allocate
     *     0             0           1           1              Normal             not shareable   Outer and inner write
     * back no write allocate
     *     0             1           1           0              Normal             shareable       Outer and inner write
     * through no write allocate
     *     0             1           1           1              Normal             shareable       Outer and inner write
     * back no write allocate
     *     1             0           0           0              Normal             not shareable   outer and inner
     * noncache
     *     1             1           0           0              Normal             shareable       outer and inner
     * noncache
     *     1             0           1           1              Normal             not shareable   outer and inner write
     * back write/read acllocate
     *     1             1           1           1              Normal             shareable       outer and inner write
     * back write/read acllocate
     *     2             x           0           0              Device              not shareable
     *  Above are normal use settings, if your want to see more details or want to config different inner/outter cache
     * policy.
     *  please refer to Table 4-55 /4-56 in arm cortex-M7 generic user guide <dui0646b_cortex_m7_dgug.pdf>
     * param SubRegionDisable  Sub-region disable field. 0=sub-region is enabled, 1=sub-region is disabled.
     * param Size              Region size of the region to be configured. use ARM_MPU_REGION_SIZE_xxx MACRO in
     * mpu_armv7.h.
     */
static uint8_t c = 0u;
    /*
     * Add default region to deny access to whole address space to workaround speculative prefetch.
     * Refer to Arm errata 1013783-B for more details.
     *
     */
    /* Region 0 setting: Instruction access disabled, No data access permission. */
    MPU->RBAR = ARM_MPU_RBAR(c, 0x00000000U);
    MPU->RASR = ARM_MPU_RASR(1, ARM_MPU_AP_NONE, 2, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_4GB);
    ++c;

    /* Region 2 setting: Memory with Device type, not shareable,  non-cacheable. */
    MPU->RBAR = ARM_MPU_RBAR(c, 0x60000000U);
    MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_FULL, 2, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_512MB);
    ++c;

#if defined(XIP_EXTERNAL_FLASH) && (XIP_EXTERNAL_FLASH == 1)
    /* Region 3 setting: Memory with Normal type, not shareable, outer/inner write back. */
    MPU->RBAR = ARM_MPU_RBAR(c, 0x70000000U);
    MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_RO, 0, 0, 1, 1, 0, ARM_MPU_REGION_SIZE_4MB);
    ++c;
#endif

    /* Region 4 setting: Memory with Device type, not shareable, non-cacheable. */
    MPU->RBAR = ARM_MPU_RBAR(c, 0x00000000U);
    MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_FULL, 2, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_1GB);
    ++c;

    /* Region 5 setting: Memory with Normal type, not shareable, outer/inner write back */
    MPU->RBAR = ARM_MPU_RBAR(c, 0x00000000U);
    MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_FULL, 0, 0, 1, 1, 0, ARM_MPU_REGION_SIZE_128KB);
    ++c;

    /* Region 6 setting: Memory with Normal type, not shareable, outer/inner write back */
    MPU->RBAR = ARM_MPU_RBAR(c, 0x20000000U);
    MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_FULL, 0, 0, 1, 1, 0, ARM_MPU_REGION_SIZE_128KB);
    ++c;

    /* Region 7 setting: Memory with Normal type, not shareable, outer/inner write back */
    MPU->RBAR = ARM_MPU_RBAR(c, 0x20200000U);
    MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_FULL, 0, 0, 1, 1, 0, ARM_MPU_REGION_SIZE_512KB);
    ++c;

    /* Region 8 setting: Memory with Normal type, not shareable, outer/inner write back */
    MPU->RBAR = ARM_MPU_RBAR(c, 0x20280000U);
    MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_FULL, 0, 0, 1, 1, 0, ARM_MPU_REGION_SIZE_256KB);
    ++c;

    /* Region 11 setting: Memory with Device type, not shareable, non-cacheable */
    MPU->RBAR = ARM_MPU_RBAR(c, 0x40000000);
    MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_FULL, 2, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_4MB);
    ++c;

    /* Region 12 setting: Memory with Device type, not shareable, non-cacheable */
    MPU->RBAR = ARM_MPU_RBAR(c, 0x42000000);
    MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_FULL, 2, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_1MB);
    ++c;

    /* Enable MPU */
    ARM_MPU_Enable(MPU_CTRL_PRIVDEFENA_Msk);

    /* Enable I cache and D cache */
//    SCB_EnableDCache();
//    SCB_EnableICache();
}

6. adapted mpu configuration, app seems stable but throws DTCM access errors when using printf

/* MPU configuration. */
void BOARD_ConfigMPU(void)
{
    /* Disable I cache and D cache */
    if (SCB_CCR_IC_Msk == (SCB_CCR_IC_Msk & SCB->CCR))
    {
        SCB_DisableICache();
    }
    if (SCB_CCR_DC_Msk == (SCB_CCR_DC_Msk & SCB->CCR))
    {
        SCB_DisableDCache();
    }

    /* Disable MPU */
    ARM_MPU_Disable();

    /* MPU configure:
     * Use ARM_MPU_RASR(DisableExec, AccessPermission, TypeExtField, IsShareable, IsCacheable, IsBufferable,
     * SubRegionDisable, Size)
     * API in mpu_armv7.h.
     * param DisableExec       Instruction access (XN) disable bit,0=instruction fetches enabled, 1=instruction fetches
     * disabled.
     * param AccessPermission  Data access permissions, allows you to configure read/write access for User and
     * Privileged mode.
     *      Use MACROS defined in mpu_armv7.h:
     * ARM_MPU_AP_NONE/ARM_MPU_AP_PRIV/ARM_MPU_AP_URO/ARM_MPU_AP_FULL/ARM_MPU_AP_PRO/ARM_MPU_AP_RO
     * Combine TypeExtField/IsShareable/IsCacheable/IsBufferable to configure MPU memory access attributes.
     *  TypeExtField  IsShareable  IsCacheable  IsBufferable   Memory Attribtue    Shareability        Cache
     *     0             x           0           0             Strongly Ordered    shareable
     *     0             x           0           1              Device             shareable
     *     0             0           1           0              Normal             not shareable   Outer and inner write
     * through no write allocate
     *     0             0           1           1              Normal             not shareable   Outer and inner write
     * back no write allocate
     *     0             1           1           0              Normal             shareable       Outer and inner write
     * through no write allocate
     *     0             1           1           1              Normal             shareable       Outer and inner write
     * back no write allocate
     *     1             0           0           0              Normal             not shareable   outer and inner
     * noncache
     *     1             1           0           0              Normal             shareable       outer and inner
     * noncache
     *     1             0           1           1              Normal             not shareable   outer and inner write
     * back write/read acllocate
     *     1             1           1           1              Normal             shareable       outer and inner write
     * back write/read acllocate
     *     2             x           0           0              Device              not shareable
     *  Above are normal use settings, if your want to see more details or want to config different inner/outter cache
     * policy.
     *  please refer to Table 4-55 /4-56 in arm cortex-M7 generic user guide <dui0646b_cortex_m7_dgug.pdf>
     * param SubRegionDisable  Sub-region disable field. 0=sub-region is enabled, 1=sub-region is disabled.
     * param Size              Region size of the region to be configured. use ARM_MPU_REGION_SIZE_xxx MACRO in
     * mpu_armv7.h.
     */
    /*
     * Add default region to deny access to whole address space to workaround speculative prefetch.
     * Refer to Arm errata 1013783-B for more details.
     *
     */
    /* Region 0 setting: Instruction access disabled, No data access permission. */
    MPU->RBAR = ARM_MPU_RBAR(0, 0x00000000U);
    MPU->RASR = ARM_MPU_RASR(1, ARM_MPU_AP_NONE, 2, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_4GB);

    // Document AN12042 'Using the i.MXRT L1 Cache'
   // page 5 gives the default configuration, which is overlaid with this one
   // where differences exist (i.e. if a region is not specified below,
   // it retains it's default configuration)
   // ARM_MPU_RASR(DisableExec, AccessPermission, TypeExtField, IsShareable,
   // IsCacheable, IsBufferable, SubRegionDisable, Size)

   // 0x80000000-0xDFFFFFFF is external memory like SDRAM etc. Not used.
   // give external memories no access (1.5Gb total 0x80000000-0xdfffffff)
   MPU->RBAR = ARM_MPU_RBAR(1, 0x80000000U);
   MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_NONE, 0, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_1GB);

   // External flash at 0x60000000-0x7fffffff. Cacheable and executable.
   MPU->RBAR = ARM_MPU_RBAR(2, 0x60000000U);
   MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_NONE, 0, 0, 1, 1, 0, ARM_MPU_REGION_SIZE_512MB);

   // This is all of RAM, which we will disable, then make sub-regions with the
   // appropriate attributes. 0x00000000-0x3fffffff
   MPU->RBAR = ARM_MPU_RBAR(3, 0x00000000U);    // default = no access
   MPU->RASR = ARM_MPU_RASR(1, ARM_MPU_AP_NONE, 2, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_1GB);

   /* I-TCM FlexRam mapped */
   // MAX: 512KB -> 0000_0000 - 0007_FFFF, executable, cacheable
   MPU->RBAR = ARM_MPU_RBAR(4, 0x00000000U);
   MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_FULL, 0, 0, 1, 1, 0, ARM_MPU_REGION_SIZE_128KB);

   /* D-TCM FlexRam mapped */
   // MAX: 512KB -> 2000_0000 - 2007_FFFF, Not executable, cacheable
   MPU->RBAR = ARM_MPU_RBAR(5, 0x20000000U);
   MPU->RASR = ARM_MPU_RASR(1, ARM_MPU_AP_FULL, 0, 0, 1, 1, 0, ARM_MPU_REGION_SIZE_128KB);

   /* OCRAM 1064 default 512KB */
   // 0x20200000-0x2027ffff, Not executable, cacheable
   MPU->RBAR = ARM_MPU_RBAR(6, 0x20200000U);
   MPU->RASR = ARM_MPU_RASR(1, ARM_MPU_AP_FULL, 0, 0, 1, 1, 0, ARM_MPU_REGION_SIZE_512KB);

   /* OCRAM FlexRam mapped */
   // MAX: 512KB -> 0x2028_0000-0x202F_FFFF, Not executable, cacheable
   MPU->RBAR = ARM_MPU_RBAR(7, 0x20280000U);
   MPU->RASR = ARM_MPU_RASR(1, ARM_MPU_AP_FULL, 0, 0, 1, 1, 0, ARM_MPU_REGION_SIZE_256KB);

   /* AIPS-1 to AIPS-4 1MB each */
   // 4000_0000 403F_FFFF 4MB
   MPU->RBAR = ARM_MPU_RBAR(8, 0x40000000);
   MPU->RASR = ARM_MPU_RASR(1, ARM_MPU_AP_FULL, 2, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_4MB);

   /* AIPS-5 */
   // 4200_0000-420F_FFFF 1MB
   MPU->RBAR = ARM_MPU_RBAR(9, 0x42000000);
   MPU->RASR = ARM_MPU_RASR(1, ARM_MPU_AP_FULL, 2, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_1MB);

   /* Flash QSPI */
   // MAX: 512KB -> 0x7000_0000-0x7400_0000, executable, cacheable
   MPU->RBAR = ARM_MPU_RBAR(10, 0x70000000U);
   MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_RO, 0, 0, 1, 1, 0, ARM_MPU_REGION_SIZE_4MB);

   /* Enable MPU */
   ARM_MPU_Enable(MPU_CTRL_PRIVDEFENA_Msk);

   /* Enable I cache and D cache */
   SCB_EnableDCache();
   SCB_EnableICache();
}

7. adapted mpu configuration, adapted OCRAM config, framebuffer area is non cacheable, no changes (as expected)

/* OCRAM 1064 default 512KB */
// 0x20200000-0x2027ffff, Not executable, non cacheable as it is accessed by dma and core
MPU->RBAR = ARM_MPU_RBAR(6, 0x20200000U);
MPU->RASR = ARM_MPU_RASR(1, ARM_MPU_AP_FULL, 1, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_512KB);

/* OCRAM FlexRam mapped */
// MAX: 512KB -> 0x2028_0000-0x202F_FFFF, Not executable, shareable cacheable
MPU->RBAR = ARM_MPU_RBAR(7, 0x20280000U);
MPU->RASR = ARM_MPU_RASR(1, ARM_MPU_AP_FULL, 0, 1, 1, 0, 0, ARM_MPU_REGION_SIZE_256KB);

8. adapted mpu configuration, adapted D-TCM, can execute code, fixed the DTCM error when printf is active

/* D-TCM FlexRam mapped */
// MAX: 512KB -> 2000_0000 - 2007_FFFF, Not executable, cacheable
MPU->RBAR = ARM_MPU_RBAR(5, 0x20000000U);
MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_FULL, 0, 0, 1, 1, 0, ARM_MPU_REGION_SIZE_128KB);

9. adapted mpu configuration, improved dtcm and itcm mpu config to represent the implizit setting (as described in the added comment) No changes in software behaivour

// For Cortex®-M7, TCMs memories always behave as non-cacheable, non-shared normal memories, irrespective
// of the memory type attributes defined in the MPU for a memory region containing addresses held in the TCM.
// configure it anyway because of Arm errata 1013783-B workarround
/* I-TCM FlexRam mapped */
// MAX: 512KB -> 0000_0000 - 0007_FFFF, executable, cacheable
MPU->RBAR = ARM_MPU_RBAR(4, 0x00000000U);
MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_FULL, 1, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_128KB);

/* D-TCM FlexRam mapped */
// MAX: 512KB -> 2000_0000 - 2007_FFFF, Not executable, cacheable
MPU->RBAR = ARM_MPU_RBAR(5, 0x20000000U);
MPU->RASR = ARM_MPU_RASR(0, ARM_MPU_AP_FULL, 1, 0, 0, 0, 0, ARM_MPU_REGION_SIZE_128KB);

The ITCM access error keeps occuring with each peripheral initialised ( CAN, WDG, I2C...)
Non the less, the Init seems to work, as the watchdog and CAN are working as expected. I disabled IRQ for ITCM and left only DTCM and OC_RAM IRQ enabled, as ITCM reporting seemed to worsen SW stability.
If i add more features, like TFT and embedded wizard, the software just crashes at some point, display content gets overlaid with static noise or does not start at all... I could not pin down the exact effects of disalbing or enabling certain tasks or features, some time a minor change has a fatal effect, sometime not...
Changing some minor part and recompiling sometimes results in a runnable SW, then reverting the steps recompiling and doing it again may result in a different SW behaviour...

Currently i use 4 tasks:
CAN with some cyclic messages
ADC reading 2 signals cyclical
embedded wizzard gui Task
Watchdog trigger task

The applications seems stable and all tasks perform as expected (ADC reads correct values, CAN transmits and receives, GUI shows the correct content and reacts to changes)
BUT i get about 110 DTCM access errors in about TWO seconds...

I do not have allocated any code to ITCM.
Only heap, bss, stack and data to DTCM.
This is the memory setting:

 <chip>
  <memory id='Flash' type='Flash' is_ro='true' can_program='true'></memory>
  <memory id='RAM' type='RAM'></memory>
   <memoryInstance id='PROGRAM_FLASH' derived_from='Flash' location='0x70000000' size='0x400000' driver='MIMXRT1064.cfx'  edited='true'  />
   <memoryInstance id='SRAM_DTC' derived_from='RAM' location='0x20000000' size='0x20000'   edited='true'  />
   <memoryInstance id='SRAM_ITC' derived_from='RAM' location='0x0' size='0x20000'   edited='true'  />
   <memoryInstance id='SRAM_OC' derived_from='RAM' location='0x20200000' size='0xc0000'   edited='true'  />
 </chip>

Is there any more information or some application notes on this topic?
What am I missing?
Is there some specific alignment to be respected for variables at data or bss?
Are there any differences when using OC-Ram or DTCM?
Is my MPU config correct?
Is there any chance to resolve where the ITCM or DTCM access error occured, other than manualy stepping through code and watching the related FLEX-RAM Status register?

Relates to Case: 00330679

Kind regards
Clemens

[victorjimenez]

Hello Clemens, 

As we discussed in the case, here's the last information. 

I verified if you had something in the ITCM memory but you don't have anything on this space of the FlexRAM, see the below image.

So, it still doesn't make any sense that you activate this interrupt and that it is triggered since you are never accessing this space of memory.

As I mentioned in the case, the example that you are using was created by embedded wizard, hence I wasn't familiar with the overall structure. I decided to change the example code that you shared to match how we activate the FlexRAM interrupt in the SDK example, flexram_access. I commented out your interrupt handler, FLEXRAM_IRQHandler as well as the call to the function FLEXRAM_EnableAccessErrorIrq inside the ResetISR. Instead of that, I activated the interrupt inside the main and the interrupt handler is declared in the main.c file. Once I made these changes, everything worked perfectly, even though I still have the ITCM interrupt enabled, the application never jumped to the interrupt handler.

Regards,
Victor