Code relocation and caches on Vybrid-A5 MQX; hard reboot issue.

[billpringlemeir]

I have written pc-relative routines in assembler.  I want to relocate them to different memory devices on the tower board.  I have selected both SDRAM and internal SRAM.  In both cases, the relocation works after the 2nd attempt to use the memory after a hard reboot (disconnect power).  If I do a soft reboot, then the routines are fine the first time.  I have mqx running and implemented some primitive tz-monitor handling (the same issue happens with/without tz).  I can see the code gets a DFSR of '0x00001c06'; the IFSR is 0x5 maybe these are irrelevant. I have examined the memory before/after the code update and the d-cache seems to say the code has been updated.  I am using the following mqx code to try and synchronize the caches,

       memcpy(p, (void*)target, size);

       asm("dsb"); asm("isb");
       _DCACHE_FLUSH();
       asm("dsb"); asm("isb");
       _ICACHE_INVALIDATE();
       asm("dsb"); asm("isb");
       _DCACHE_INVALIDATE();
       asm("dsb"); asm("isb");
       
       if(memcmp(p, (void*)target, size))
           printf("Memory mismatch\n");
       else
           printf("Memory OK!\n");

       /* Clear data/instruction fault registers. */
       asm(" mcr p15, 0, %0, c6, c0, 0\n" :: "r"(0xaa55aa55UL));
       asm(" mcr p15, 0, %0, c5, c0, 0\n" :: "r"(0xaa55aa55UL));
       asm(" mcr p15, 0, %0, c6, c0, 2\n" :: "r"(0xaa55aa55UL));
       asm(" mcr p15, 0, %0, c5, c0, 1\n" :: "r"(0xaa55aa55UL));

I have tried many permutations, with/without the final _DCACHE_INVALIDATE.  I looked at the Cortex-A5-MPcore TRM and it seems that the Vybrid-A5 has no ACP or SCU active.  The MQX code looks fine for flushing, although the L2 is a unified cache.  I see the manual has some L1 cache debug registers.  However, they take 'sets' and 'ways' as parameters.  I have also tried to alter the MMU permissions to change the memory to 'device' so that it should be write-through, but I think that these are intended more for data only.  I have also tried to disable the branch prediction, but I think this is cache related.

I think that the SDRAM is pre-populated with the routines during the soft boots, so some cache is setup with some latent routine that was copied via previous attempts to run the routine.  I think the memory is almost '0xffffffff' after a hard reset. Is there any functional block that I have missed that I need to update to relocate and/or load code?  Or source code examples I should look at?  I have looked at the u-boot source and didn't see anything obvious.  u-boot seems to work fine on hard resets; maybe MQX has enabled some extra functionality?  This is on a Tower system.  If I call the routines directly (without tz), then MQX will crash (or dabort loops forever).  A soft reset and the direct calls to the relocated location are fine.  If I call the routine at the original location as loaded by u-boot, then it executes always.  With the tz, I have code to dump the contexts and I can see the 'PC' is set to the MQX abort_handler, with the abort lr set to the first instruction (+8), which seems to indicate the first code access fails.

A relocatable routines I use is below; I have many different implementations (using UART, SDRAM/iRAM, etc) and I don't think the issue is here, but I provide it for reference.

        .align 8
.equ gpio0_base, 0x400ff000
ASM_PUBLIC(led_poll)
ASM_PUBLIC(led_poll_size)
ASM_PUBLIC_BEGIN(led_poll) 
ASM_PUBLIC_FUNC(led_poll) 
ASM_LABEL(led_poll)
     movw r1, #:lower16:gpio0_base
     movt r1, #:upper16:gpio0_base
     mov  r2, #24*2        @ 24 blinks. 
1:  mov  r0, #(15<<22)    @ gpio0-22 to 25.
     str  r0, [r1, #12]    @ toggle leds.\
     mov  r0, #0x80000     @ spin loop delay. 
2:  subs r0, r0, #1
     bne  2b
     subs r2, r2, #1
     bne  1b
     bx lr
     .ltorg
ASM_PUBLIC_END(led_poll)
led_poll_size:         .word . - led_poll

Possibly, I can provide a binary; I would have to remove the tz support to give source.

EDIT: I retried with MQX 4.1 and the same issue seems to occur.

EDIT2: There appears to be some bugs in _a5_dcache_flush().  The shifts should not be hexadecimal and we should select the data cache with,

MCR(15, 2, 0, 0, 0, 0);

/* Select data cache */

before looking at the sets/ways information.  I still don't know if there are additional problems...

[karina_valencia]

timesyssupport can you help to attend this case?

[timesyssupport]

Hello Karina,

Other than U-Boot source, we do not have any code examples of this. Does the MQX team have any code samples for the A5 core they can share?

Thanks,

Timesys Support

[karina_valencia]

cyborgnegotiator can you  check this  case to see if you have information to share please?

[karina_valencia]

cyborgnegotiator any comment?

[billpringlemeir · Accepted solution]

Thanks Karina.  I have verified that it is just the implementation of _a5_dcache_flush() and _a5_dcache_invalidate().  The code shifts by hex 0x13 and it should be decimal 13.  The error means that the L1 cache was not flushed (and I thought it was).  I asked in the Vybrid forum as I thought there might be some other memory consistency issue I needed to attend to.  It is really just an MQX issue.  Should I change the community?

diff --git a/mqx/source/psp/cortex_a/cache_a5.c b/mqx/source/psp/cortex_a/cache_a5.c
index 4780077..a85088b 100644
--- a/mqx/source/psp/cortex_a/cache_a5.c
+++ b/mqx/source/psp/cortex_a/cache_a5.c
@@ -71,13 +71,15 @@ void _a5_dcache_invalidate(void)
     uint32_t csid, wayset;  /* Cache Size ID , wayset parameter */
     int32_t num_sets, num_ways; /* number of sets  */
 
+    MCR(15, 2, 0, 0, 0, 0);       /* Select data cache */
+    ISB();
     MRC(15, 1, csid, 0, 0, 0);    /* Read Cache Size ID */
     /* Fill number of sets  and number of ways from csid register  This walues are decremented by 1*/
-    num_ways = (csid >> 0x03) & 0x3FFu; //((csid& csid_ASSOCIATIVITY_MASK) >> csid_ASSOCIATIVITY_SHIFT`)
+    num_ways = (csid >> 3) & 0x3FFu; //((csid& csid_ASSOCIATIVITY_MASK) >> csid_ASSOCIATIVITY_SHIFT`)
     /* Invalidation all lines (all Sets in all ways) */
     while (num_ways >= 0)
     {
-        num_sets = (csid >> 0x13) & 0x7FFFu; //((csid & csid_NUMSETS_MASK)      >> csid_NUMSETS_SHIFT       )
+        num_sets = (csid >> 13) & 0x7FFFu; //((csid & csid_NUMSETS_MASK)      >> csid_NUMSETS_SHIFT       )
         while (num_sets >= 0 )
         {
             wayset = (num_sets << 5u) | (num_ways << 30u); //(num_sets << SETWAY_SET_SHIFT) | (num_sets << 3SETWAY_WAY_SHIFT)
@@ -140,12 +142,14 @@ void _a5_dcache_flush(void)
     uint32_t csid, wayset;  /* Cache Size ID */
     int32_t num_sets, num_ways; /* number of sets  */
 
+    MCR(15, 2, 0, 0, 0, 0);       /* Select data cache */
+    ISB();
     MRC(15, 1, csid, 0, 0, 0);    /* Read Cache Size ID */
-    /* Fill number of sets  and number of ways from csid register  This walues are decremented by 1*/
-    num_ways = (csid >> 0x03) & 0x3FFu;
+    /* Fill number of sets  and number of ways from csid register  The values are decremented by 1*/
+    num_ways = (csid >> 3) & 0x3FFu;
     while (num_ways >= 0)
     {
-        num_sets = (csid >> 0x13) & 0x7FFFu;
+        num_sets = (csid >> 13) & 0x7FFFu;
         while (num_sets >= 0 )
         {
             wayset = (num_sets << 5u) | (num_ways << 30u);

[karina_valencia]

Hi Bill,

cyborgnegotiator can provide some guidelines about MQX here and you also can ask  directly into MQX space MQX Software Solutions