MPC5748G: dual core projects only work when using the first and 3rd core

[pmrlcbtcphtzjdq]

I'm trying to develop a simple multi-core application with freeRTOS.

The idea is simple: core Z4_0 controls the LEDs and waits for signals from other tasks that run on other cores. After receiving a signal, core Z4_0 blinks an LED.

The inter-task communication is done with queues. I set up a shared memory segment in the linker file, and put the QueueHandle_t at the memory location accessible for all other cores. The application starts a task on the first core (Z4_0) that controls the LEDs. This task reads from the queue. The other task, which runs on a different core,  packs an integer onto the queue. This integer is then being read by the task running on core 1 (Z4_0) and a diode lights up. There are three cores in the project, each core runs a freeRTOS, but one core has no tasks (meaning the task scheduler runs the 'idle' task).

The problem is that this only works only if the sending task (producer task, task that packs the values onto the queue) is located on the third core - Z2. If the sending task is located on the second core (Z4_1) then the application doesn't work (no LEDs blink).

With debugging I can decide which core should start first. If I put the sending task onto the second core, and then unfreeze it as the LAST core, the application works fine. However, if I unfreeze the third, empty core as the last one, the application doesn't work.

It seems that the third core undergoes some initialization that disrupts the functioning of the second core. I don't understand it, because the core is 'empty', it runs only the task scheduler with idle task. Furthermore, only the first core (Z4_0) has any pins enabled - there should be no conflict with the hardware.

How to develop an application that uses all three cores, and is independent from the startup sequence?

I am including memory maps and code snippets.

CORE Z4_0:

- Memory map of /Project_Settings/Linker_Files/linker_flash.ld:

/* Define SRAM */
SRAM_BASE_ADDR = DEFINED(__sram_base_addr__) ? __sram_base_addr__ : 0x40000000;
SRAM_SIZE = DEFINED(__sram_size__) ? __sram_size__ : 256K;

MEMORY
{
    flash_rchw : org = 0x00FA0000, len = 0x4
    cpu0_reset_vec : org = 0x00FA0000+0x10, len = 0x4
    cpu1_reset_vec : org = 0x00FA0000+0x14, len = 0x4
    cpu2_reset_vec : org = 0x00FA0000+0x04, len = 0x4
    rappid_boot_data : org = 0x00FA0000+0x08, len = 0x8

    m_text : org = FLASH_BASE_ADDR, len = FLASH_SIZE 
    m_data : org =   0x40001000, len = 252K
   /************************************************/
   shared :    org = 0x40000000,   len = 4k   
   /************************************************/ 
}

- main.c snippet

/* toggles the LED received from the task */
 void vToggleLEDonNotification(void* pvParamaters)
 {
	 //8 LEDs: pa4, pa0, pj4,ph5,pc4,ph13,pa7,pa10
	 while(1){
		 uint32_t receivedLEDno;
		 /* read the LED number from the queue */
		 if(xQueueReceive( *LEDQueue, &receivedLEDno, 500 ) == pdTRUE){
			 switch(receivedLEDno){
				   case 0: //PA4
					   PINS_DRV_TogglePins(PTA, 0x10) ;
					   break;
				   case 1: //PA0
					   PINS_DRV_TogglePins(PTA, 0x1) ;
					   break;
				   case 2: //PJ4
					   PINS_DRV_TogglePins(PTJ, 0x10) ;
					   break;
				   default:
					     break;
			 }
		}
	 }
 }

/*place the queue in commonly shared piece of memory */
QueueHandle_t *LEDQueue =(QueueHandle_t*) 0x40000000;

int main(void)
{

  #ifdef PEX_RTOS_INIT
    PEX_RTOS_INIT();                   /* Initialization of the selected RTOS. Macro is defined by the RTOS component. */
  #endif

   /* Initialize and configure clocks */
   CLOCK_SYS_Init(g_clockManConfigsArr, (uint8_t)CLOCK_MANAGER_CONFIG_CNT, g_clockManCallbacksArr, (uint8_t)CLOCK_MANAGER_CALLBACK_CNT);
   CLOCK_SYS_UpdateConfiguration(0U, CLOCK_MANAGER_POLICY_AGREEMENT);

   /* Initialize pins */
   PINS_DRV_Init(NUM_OF_CONFIGURED_PINS, g_pin_mux_InitConfigArr);

   /* queue used to send LED numbeers from one task to another */
   *LEDQueue = xQueueCreate( 10, sizeof(uint32_t) );

   xTaskCreate(vToggleLEDonNotification, "ToggleLEDonNotificationTask", 128U, NULL, 2, NULL);
   vTaskStartScheduler();

  #ifdef PEX_RTOS_START
    PEX_RTOS_START();                  /* Startup of the selected RTOS. Macro is defined by the RTOS component. */
  #endif
  /*** Processor Expert end of main routine. DON'T MODIFY THIS CODE!!! ***/
  for(;;) {
    if(exit_code != 0) {
      break;
    }
  }
  return exit_code;
} /*** End of main routine. DO NOT MODIFY THIS TEXT!!! ***/

CORE Z2:

- Memory map of /Project_Settings/Linker_Files/linker_flash.ld:

/* Define SRAM */
SRAM_BASE_ADDR = DEFINED(__sram_base_addr__) ? __sram_base_addr__ : 0x40080000;
SRAM_SIZE = DEFINED(__sram_size__) ? __sram_size__ : 256K;

MEMORY
{
    flash_rchw : org = 0x00FA0000, len = 0x4
    cpu0_reset_vec : org = 0x00FA0000+0x10, len = 0x4
    cpu1_reset_vec : org = 0x00FA0000+0x14, len = 0x4
    cpu2_reset_vec : org = 0x00FA0000+0x04, len = 0x4

    m_text : org = FLASH_BASE_ADDR, len = FLASH_SIZE
    m_data : org = SRAM_BASE_ADDR, len = SRAM_SIZE
    /************************************************/
    shared :    org = 0x40000000,   len = 4k   
    /************************************************/ 
}

- main.c snippet

/* sends LED number on the queue */
 void vSendLEDnumber(void* pvParamaters)
 {
	 uint32_t LEDnumber = 0;
	 //8 LEDs: pa4, pa0, pj4,ph5,pc4,ph13,pa7,pa10
	 while(1){
		 vTaskDelay( 763 / portTICK_PERIOD_MS );
		 /* pack the LED number on to the queue */
		 xQueueSendToBack( *LEDQueue, &LEDnumber, 0 );
		 LEDnumber++;
		 if (LEDnumber > 7){LEDnumber = 0;}
	 }
 }

/*place the queue in commonly shared piece of memory */
QueueHandle_t *LEDQueue =(QueueHandle_t*) 0x40000000;

int main(void)
{
  #ifdef PEX_RTOS_INIT
    PEX_RTOS_INIT();                   
  #endif

    /* Initialize and configure clocks */
    CLOCK_SYS_Init(g_clockManConfigsArr, (uint8_t)CLOCK_MANAGER_CONFIG_CNT, g_clockManCallbacksArr, (uint8_t)CLOCK_MANAGER_CALLBACK_CNT);
    CLOCK_SYS_UpdateConfiguration(0U, CLOCK_MANAGER_POLICY_AGREEMENT);

    xTaskCreate(vSendLEDnumber, "SendLEDnumber", 128U, NULL, 1, NULL);
    vTaskStartScheduler();

  /*** Don't write any code pass this line, or it will be deleted during code generation. ***/

  #ifdef PEX_RTOS_START
    PEX_RTOS_START();                  /* Startup of the selected RTOS. Macro is defined by the RTOS component. */
  #endif
  /*** Processor Expert end of main routine. DON'T MODIFY THIS CODE!!! ***/
  for(;;) {
    if(exit_code != 0) {
      break;
    }
  }
  return exit_code;
} 

CORE Z4_1

- Memory map of /Project_Settings/Linker_Files/linker_flash.ld:

/* Define SRAM */
SRAM_BASE_ADDR = DEFINED(__sram_base_addr__) ? __sram_base_addr__ : 0x40040000;
SRAM_SIZE = DEFINED(__sram_size__) ? __sram_size__ : 256K;

MEMORY
{
    flash_rchw : org = 0x00FA0000, len = 0x4
    cpu0_reset_vec : org = 0x00FA0000+0x10, len = 0x4
    cpu1_reset_vec : org = 0x00FA0000+0x14, len = 0x4
    cpu2_reset_vec : org = 0x00FA0000+0x04, len = 0x4

    m_text : org = FLASH_BASE_ADDR, len = FLASH_SIZE
    m_data : org = SRAM_BASE_ADDR, len = SRAM_SIZE  
   /************************************************/
   shared :    org = 0x40000000,   len = 4k   
   /************************************************/ 
}

- main.c

int main(void)
{
  #ifdef PEX_RTOS_INIT
    PEX_RTOS_INIT();                  
  #endif

    /* Initialize and configure clocks */
    CLOCK_SYS_Init(g_clockManConfigsArr, (uint8_t)CLOCK_MANAGER_CONFIG_CNT, g_clockManCallbacksArr, (uint8_t)CLOCK_MANAGER_CALLBACK_CNT);
    CLOCK_SYS_UpdateConfiguration(0U, CLOCK_MANAGER_POLICY_AGREEMENT);

    vTaskStartScheduler();

  #ifdef PEX_RTOS_START
    PEX_RTOS_START();                 
  #endif

  for(;;) {
    if(exit_code != 0) {
      break;
    }
  }
  return exit_code;
} 

One more detail: I have not been able to flash all three cores onto the board using the 'Flash'  option of the S32DS. It would only write to the first core. The way I program all three cores of the board is:

1. creating a 'launch group' with all the cores.

2. clicking 'run configuration' and selecting the launch group with three cores.

3. Unplugging the device - the program now resides in the flash.

[pmrlcbtcphtzjdq]

Thank you for the reply. Unfortunately it didn't work.

Your suggestions:

1. Data Cache

In all cores data cache is disabled (strange that although the Z2_2 core doesn't have the data cache, its startup_MPC5748G.S still has an option for enabling...). It still doesn't work. The producer task (that sends the variables for the first core to read) still needs to be on Z2_2. Otherwise, it doesn't work - as described above.

2. Cache inhibit with the SMPU

If I set the cache inhibit for the shared memory (where the queue is) no LEDs being turn on - it is strange, as the default value for the LED is ON, and they remain dead (If I unfreeze just the first core in the debugging view, then I can see the LEDs turn on, however as soon as I reset the board they are dead).

I tried two approaches - setting the SMPU in the same way in all 3 main.c files for all of the cores, as well as setting the cache inhibit separately for each of the core.

At first I tried setting the cache inhibit manually. I couldn't compile the following code:

    /* configure the SMPU to set cache inhibition for shared SRAM */
    SMPU_1.RGD[0].WORD0.R = 0x40000000 		// start address
    SMPU_1.RGD[0].WORD1.R = 0x40000fff;          // end address
    SMPU_1.RGD[0].WORD3.B.CI = 1;		// set cache inhibit
    SMPU_1.CES0.B.GVLD =1;			// enable smpu_1

The compiler couldn't find the structs. I tried to locate the SMPU header files, but I haven't seen any do any specific files included in examples for SMPU usage...

Anyway, I used the component option from the GUI and set it this way:

file smpu1.c:

const smpu_user_config_t smpu1_UserConfig0[] =
{
    /*! Region number 0
        Name: Region 0 */
  {
	    .startAddr           = 0x40000000, 	                     /*!< Memory region start address */
	    .endAddr             = 0x40000fff,                     /*!< Memory region end address */
	    .specAccessEnable    = false,                           /*!< Specific access enable */
	    .specAccessSet       = NULL,                            /*!< Specific access configurations */
	    .masterAccRight      = smpu1_AccessRightConfig0,        /*!< Master access right */
	    .cacheInhibitEnable  = true,                           /*!< Cache inhibit enable */
	    .processIdEnable     = false,                           /*!< Process identifier enable */
	    .processIdentifier   = 0x00U,                           /*!< Process identifier */
	    .processIdMask       = 0x00U,                           /*!< Process identifier mask */
	    .lockConfig          = SMPU_UNLOCK                      /*!< Lock configuration */
	    }
};

initializing in main.c:

  SMPU_DRV_Init(INST_SMPU1, 1U, smpu1_UserConfig0);

It doesn't work. No matter if the SPMU_DRV_Init() is called on all the cores, only on the first core or only on the producer core the LEDs remain off.

Do you have any other suggestions?

[lukaszadrapa]

Hi,

SMPU is a system feature, not core feature. SMPU is sitting on a crossbar switch. That means no individual initialization of the same descriptor is needed on all cores.

One important point – be aware that once the SMPU is enabled, all accesses that do not hit a descriptor are considered as access violation. In other words, all used resources must be covered by descriptors.

In case of MPC5748G, there are two SMPU modules. SMPU1 covers SRAM only. It’s OK to use descriptor 0 as you did but you need to add second one which will cover rest of the RAM.

This explains why it doesn’t work ever after this initialization.

If the cache is disabled, it’s not necessary to do this initialization.

Well, that’s about the SMPU but it doesn’t solve the problem. Is it possible to share the project? If yes, please upload the compiled one, so I can load it directly to the mcu without modifications.

Thanks,

Lukas

[pmrlcbtcphtzjdq]

thanks a lot. I am attaching the code (I left out all the SMPU related changes, as it is irrelevant when the cache is disabled).

[lukaszadrapa]

Thanks. But give me more time please, I'm totally overloaded now. I will try to allocate some time to test it in next days.

Regards,

Lukas

[pmrlcbtcphtzjdq]

Of course, thank you very much for your support.
I kept on trying other things and created the same application, but this time just running on two cores only.

I used the same code, I just created new projects. I figured out that there is a correlation. The application works only when the consumer task (LED blinking), is placed on Z4_0, even if there are just two cores.

The correlation looks like this for two core applications:

Z4_0: producer, Z4_1 consumer - doesn't work

Z4_0: consumer, Z4_1 producer - works

Z4_0: producer, Z2_2 consumer - doesn't work

Z4_0: consumer, Z2_2 producer - works

with producer being a task that sends a variable onto the queue and consumer being a task that reads the variable from the queue and blinks LED, according to the number that has been read.

Apart from that, the code is almost identical - the only other difference is that the producer core doesn't initialize pins (it doesn't have to, as it only sends a variable over the shared RAM)..

[lukaszadrapa · Accepted solution]

Hi,

finally I found some time today. I was playing with your project and I was facing very weird behavior of freertos. I saw no problem with initialization performed by startup files, so I focused on freertos initialization. Then I found:

in ..\three_cores_led_blink_Z4_0\Generated_Code\FreeRTOSConfig.h:

     /* PowerPC specific: pit channel to use 0-15 */
     #define configUSE_PIT_CHANNEL (3)

in ..\three_cores_led_blink_Z4_1\Generated_Code\FreeRTOSConfig.h

     /* PowerPC specific: pit channel to use 0-15 */
     #define configUSE_PIT_CHANNEL (3)

in ..\three_cores_led_blink_Z2_2\Generated_Code\FreeRTOSConfig.h

     /* PowerPC specific: pit channel to use 0-15 */
     #define configUSE_PIT_CHANNEL (3) 

Each freertos instance should use own PIT channel. I believe everything will work as expected if you fix this configuration.

Regards,

Lukas

[pmrlcbtcphtzjdq]

thanks a lot!
That was it. I changed the PIT Channel of every core to a different one (precisely speaking I assigned the Z4_0 to use channel 3-  as in the original setting, core Z4_1 to use channel 4, and core Z2_2 to use channel 5).

Now all the cores work as they should, and the consumer task can be placed at any core.

[lukaszadrapa]

Hi,

my first idea was SRAM initialization but you changed the linker files in the right way.

Second idea I have is related to data cache. Is it disabled on both Z4 cores? If not, could you try to disabled it? There's no cache on Z2 core, so this could explain the difference. Another option is to use SMPU module to make the shared memory cache inhibited. Be aware that there's no HW coherency unit, so the best way is to configure shared area as cache inhibited.

Not sure if this will solve the problem but these are the main points when we are talking about startup files.

Regards,

Lukas