topic Re: A53 core suspend in i.MX Processors

A53 core suspend

Lakshmi_AG — Wed, 14 Jun 2023 10:17:50 GMT

Hi,

I am using imxqm with only A53 cores enabled.

Presently trying to self suspend A53 cores(0,1,2,3) using below mentioned scu API but i see cores are not suspending and DDR also not in refresh mode.

/*************************************sleep sequence****************************************************/

sc_irq_enable(ipc_handle, 297U, SC_IRQ_GROUP_WAKE, SC_IRQ_PAD, SC_TRUE);
    /* Enable GPIO pad wakeup */
    sc_pad_set_wakeup(ipc_handle, 40U, 6U);//SC_PAD_WAKEUP_RISE_EDGE 6U

    /* make sure system sources power ON in low power mode by default */
    sc_pm_req_low_power_mode(ipc_handle, SC_R_A53_0, SC_PM_PW_MODE_ON);
    /* Config system interface HPM, LPM */
    sc_pm_req_sys_if_power_mode(ipc_handle, SC_R_A53, SC_PM_SYS_IF_DDR, SC_PM_PW_MODE_ON, SC_PM_PW_MODE_OFF);
    sc_pm_req_sys_if_power_mode(ipc_handle, SC_R_A53, SC_PM_SYS_IF_OCMEM, SC_PM_PW_MODE_ON, SC_PM_PW_MODE_OFF);
    sc_pm_req_sys_if_power_mode(ipc_handle, SC_R_A53, SC_PM_SYS_IF_MU, SC_PM_PW_MODE_OFF, SC_PM_PW_MODE_OFF);
    sc_pm_req_sys_if_power_mode(ipc_handle, SC_R_A53, SC_PM_SYS_IF_INTERCONNECT, SC_PM_PW_MODE_ON,
        SC_PM_PW_MODE_OFF);
    sc_pm_set_cpu_resume(ipc_handle, SC_R_A53_0,true, 0x80000000);//
    sc_pm_req_cpu_low_power_mode(ipc_handle, 1U,SC_PM_PW_MODE_OFF, SC_PM_WAKE_SRC_SCU);

__asm__("dsb sy");
__asm__("wfi");

while(1);

/************************************************************************************************/

/***************************wake-up sequence******************************************/

/* Query SCU wakeup event status */
#if 1
    uInt32 status;
    sc_irq_status(ipc_handle, 297U, SC_IRQ_GROUP_WAKE, &status);
    /* Check for pad wakeup */
    if (status & SC_IRQ_PAD)
    {
        /* Check for GPIO pad wakeup */
        /* Note: SCFW updates pending pad wakeup config to SC_PAD_WAKEUP_OFF */
        uInt8 gpio_wakeup;
        //sc_pad_get_wakeup(ipc_handle, 40U, &gpio_wakeup);
        if (gpio_wakeup == SC_PAD_WAKEUP_OFF)
        {
            /* GPIO pad generated wake event */
            print(DEBUG_PRINT_ALWAYS, "GPIO wakeup\r\n");

} }

Is the suspend sequnce all good, because if i add any print statemnt inside while(1), is it still executing that mean A53 are still alive.

Is there any other steps do we need to follow before suspending, or is there any method to check if cores are suspend mode.

Re: A53 core suspend

AldoG — Fri, 16 Jun 2023 22:08:08 GMT

Hello,

Your sequence looks correct, just that you didn't set the A53 Core into low power mode, you may take the following sample code as reference.
https://github.com/nxp-imx/imx-atf/blob/lf_v2.6/plat/imx/imx8qm/imx8qm_psci.c#L195

So you're missing this one:
sc_pm_req_low_power_mode(ipc_handle, SC_R_A53, SC_PM_PW_MODE_OFF);

All other seems correct, please try it and let us know.

Best regards,
Aldo.

Re: A53 core suspend

Lakshmi_AG — Mon, 19 Jun 2023 05:09:46 GMT

Hi Aldo,

I tried by adding the API but still the issue is same.

below is the updated sequence.

//suspend//

/* Register for PAD wakeup */
    sc_irq_enable(ipc_handle, 297U, SC_IRQ_GROUP_WAKE, SC_IRQ_PAD, SC_TRUE);
    /* Enable GPIO pad wakeup */
    sc_pad_set_wakeup(ipc_handle, 40U, 6U);//SC_PAD_WAKEUP_RISE_EDGE 6U

    /* make sure system sources power ON in low power mode by default */
    sc_pm_req_low_power_mode(ipc_handle, SC_R_A53, SC_PM_PW_MODE_OFF);
    /* Config system interface HPM, LPM */
    sc_pm_req_sys_if_power_mode(ipc_handle, SC_R_A53, SC_PM_SYS_IF_DDR, SC_PM_PW_MODE_ON, SC_PM_PW_MODE_OFF);
    sc_pm_req_sys_if_power_mode(ipc_handle, SC_R_A53, SC_PM_SYS_IF_OCMEM, SC_PM_PW_MODE_ON, SC_PM_PW_MODE_OFF);
    sc_pm_req_sys_if_power_mode(ipc_handle, SC_R_A53, SC_PM_SYS_IF_MU, SC_PM_PW_MODE_OFF, SC_PM_PW_MODE_OFF);
    sc_pm_req_sys_if_power_mode(ipc_handle, SC_R_A53, SC_PM_SYS_IF_INTERCONNECT, SC_PM_PW_MODE_ON,
        SC_PM_PW_MODE_OFF);
    sc_pm_set_cpu_resume(ipc_handle, SC_R_A53_0,true, 0x80000000);//
    sc_pm_req_cpu_low_power_mode(ipc_handle, SC_R_A53_0,SC_PM_PW_MODE_OFF, SC_PM_WAKE_SRC_SCU);
    sc_pm_req_cpu_low_power_mode(ipc_handle, SC_R_A53_1, SC_PM_PW_MODE_OFF, SC_PM_WAKE_SRC_SCU);
    sc_pm_req_cpu_low_power_mode(ipc_handle, SC_R_A53_2, SC_PM_PW_MODE_OFF, SC_PM_WAKE_SRC_SCU);
    sc_pm_req_cpu_low_power_mode(ipc_handle, SC_R_A53_3, SC_PM_PW_MODE_OFF, SC_PM_WAKE_SRC_SCU);

__asm__("dsb");
    __asm__("isb");
    __asm__("wfi");
    //__asm__("wfi");
    while (1)
    {
        print(DEBUG_PRINT_ALWAYS, ".\r\n");
    }

Re: A53 core suspend

AldoG — Tue, 20 Jun 2023 01:04:24 GMT

Hello,

I see no problems with the sequence you have shared it looks practically the same with the sample code we have available I have shared previously.

Also, you may be able to enable Debug Monitor on the SCFW, so you may check the state of the resources while they are in LPM, just for debugging purpose.

Best regards,
Aldo.

Re: A53 core suspend

Lakshmi_AG — Tue, 20 Jun 2023 05:31:42 GMT

Do we need to flush cache before executing wfi or do we any errata for A53 on why WFI does not work.

and also for enabling Debug Monitor, i went through the document and it specifies "The debug monitor
allows command-line interaction via the SCU UART" : which are hardware pins we need connect in soc to for SCU UART.

Re: A53 core suspend

AldoG — Tue, 20 Jun 2023 23:37:29 GMT

Hi,

As stated in the reference manual for the i.MX8QM, chapter 11.1.1.5 Interface:
Assuming that a simple standby state with no state loss is required, there is no preparation required, and the core may simply perform the WFI operation. The caches are kept coherent in this state, and resumption is caused by an interrupt to the core.

If transition is required to lower power states for the core, then before the WFI is signaled, the core must ensure that any architectural state is saved to memory, cache is disabled, flushed, and then removed from the coherency domain. The SCU must also have been told that the intent is to transition to this lower power state.

Also, regarding debug monitor, if you are using our MEK board, it has two serial ports, the first one (usually ttyUSB0) is used by the A cores (u-boot/Linux in this case), the second one is used by one of the M4 cores or the SCFW when debug monitor is enabled.

Best regards,
Aldo.

Re: A53 core suspend

Lakshmi_AG — Wed, 21 Jun 2023 16:39:52 GMT

Hi,

Thanks for the confirmation,

To further understand why cores are not suspending after WFI, arm for a53 suggests as below

/*********************************/

On entry into WFI low-power state, STANDBYWFI for that core is asserted. Assertion of STANDBYWFI guarantees that the core is in idle and low-power state.

/***************************/

On imx8qm how to check the STANDBYWFI signal. is there any register available.

-Lakshmi

Re: A53 core suspend

Lakshmi_AG — Thu, 22 Jun 2023 16:30:52 GMT

Hi,

We are looking for the immediate support on wfi not working on A53 cores (imx8qm),

Please help us to identify the issue.

Regards

Lakshmi

Re: A53 core suspend

AldoG — Tue, 27 Jun 2023 01:12:39 GMT

Hello,

Sorry for the delayed response, please note that it is the SCFW which will handle the system power states, the SCFW also monitors the state of the system to enter the lowest possible power state. SCFW uses the following to evaluate the system power state :

Executing environments (EEs) manage individual resources using SCFW APIs for resources (sc_pm_set_resource_power_mode). After all resources within a subsystem are powered down, SCFW will automatically power down the entire subsystem.

EEs manage shared system resources such as DDR, on-chip memory, and system interconnect using SCFW APIs for system interfaces (sc_pm_req_sys_if_power_mode). EEs specify power requirements for these shared resources during high-power mode (HPM) and low-power mode (LPM). If any EE is in HPM, SCFW will keep the system in HPM and set the system interfaces to the highest HPM power mode among all EEs currently in HPM. If all EEs are in LPM, the SCFW will transition the system to LPM and set the system interfaces to the highest LPM power mode for all EEs.

Cortex-A EEs manage their LPM configuration using SCFW APIs (sc_pm_req_cpu_low_power_mode). This API is also used to signal the SCFW to monitor individual CPU WFI signals and apply the LPM configuration once WFI has been reached.

There is the call sc_pm_get_resource_power_mode() to retrieve a resource current power mode, this for debugging purpose just to be sure the resource current state.

So I would like to know how are you testing going into low power mode?
Did you try by enabling debug monitor and check that it does not go into low power?

Best regards,
Aldo.

Re: A53 core suspend

Lakshmi_AG — Fri, 30 Jun 2023 18:35:19 GMT

Hi Aldo,

Thanks for the explanation.

SCU API are all good.

what we suspect is we are not able to put cores in WFI state. We feel that cores are getting continuous interrupts in our system.

Can we identify which interrupt been served to cores? Do we have such provision?

Re: A53 core suspend

AldoG — Tue, 04 Jul 2023 23:40:26 GMT

Hello,

I'll double check with internal team if we have something available.
Could you share how are you testing this?
Maybe sharing some rough steps to try to replicate.

Thank you,
Best regards,
Aldo.

Re: A53 core suspend

Lakshmi_AG — Wed, 05 Jul 2023 09:02:11 GMT

steps followed:

Power down sequence.

1. Detect the powerd down.

2. prepare scu for core low-power

3. disable irq_steer

4.disable irq,fiq

5. execute wfi(dsb and isb before wfi)

6. once core is out from wfi scu, poll for scu irq status to check if wake up from configured pad.

7.resuem back core to normall mode.

Re: A53 core suspend

qiang_li-mpu_se — Wed, 05 Jul 2023 09:43:27 GMT

For iMX8QM, when suspend the A53 core, the SCFW will turn off the A53 core power.

I think you need reference to the ATF "arm-trusted-firmware" code, function imx_domain_suspend() of file "arm-trusted-firmware\plat\imx\imx8qm\imx8qm_psci.c". Before suspend, ATF will set the resume address to SCFW, it is the ATF base address in DDR, then after suspend, SCFW will track the GPIO PAD wakeup event, after get the wake up signal, SCFW will turn on the A53 power, ATF will start to run again, ATF will check, it is cold boot or resume boot, for resume it will call imx_domain_suspend_finish() "in imx8qm_psci.c" to do the resume tasks.

The ATF code can be found at https://github.com/nxp-imx/imx-atf

ATF suspend main sequence: imx_domain_suspend() -> imx_pwr_domain_pwr_down_wfi(), they are called from function psci_cpu_suspend_start() of file "arm-trusted-firmware\lib\psci\psci_suspend.c"

Re: A53 core suspend

Lakshmi_AG — Wed, 05 Jul 2023 10:17:32 GMT

Thanks for the quick response.

I tried the sequence as mentioned in the ATF build, still core suspend and resume is not working.

is there any pre condition we need to have before sending commands to SCU for suspend

Re: A53 core suspend

qiang_li-mpu_se — Thu, 06 Jul 2023 03:11:09 GMT

Hi @Lakshmi_AG ,

For quickly method, I think you can use the ATF directly, then put your own image to address 0x80020000 like the Uboot. The mkimage command can be "./mkimage_imx8 -soc QM -rev B0 -append mx8qmb0-ahab-container.img -c -scfw scfw_tcm.bin -ap bl31.bin a53 0x80000000 your_image.bin --data 0x80020000 -out flash.bin"

After ATF successfully booted, it will jump to address 0x80020000 to run your_image.bin.

Then in your own image, you can call ATF's psci APIs with smccc method. Uboot example can be followed:

#define PSCI_SYSTEM_SUSPEND_AARCH64 0xc400000e

int do_board_suspend(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])

{

struct udevice *dev;

uclass_get_device_by_name(UCLASS_FIRMWARE, "psci", &dev);

invoke_psci_fn(PSCI_SYSTEM_SUSPEND_AARCH64, 0, 0, 0);

return 1;

}

U_BOOT_CMD(

suspend, 1, 1, do_board_suspend,

"suspend\n",

"system board suspend for i.MX 8 Quad devices \n"

);

If you can't use ATF directly, the you need porting the ATF code, it needs time to debug in your side. For such kind of debug, you can build scfw with "M=1" to enable the SCFW debug UART, then after run the suspend in your software, you can run command "power.r" in SCFW debug UART to check which module hasn't been turned off, then you need check the related code.

Re: A53 core suspend

Lakshmi_AG — Mon, 10 Jul 2023 03:36:40 GMT

Hi qiang_li-mpu_se

Thanks for the response.

Presently we are in 32bit mode, can i get macro for 32bit,

and also Do we have a example or a document to talk to ATF.

or Uboot code path for reference i can use.

Regards

Lakshmi

Re: A53 core suspend

Lakshmi_AG — Mon, 10 Jul 2023 09:01:01 GMT

Hi @qiang_li-mpu_se ,

I have started integrating the code to talk to atf,

invoke_psci_fn(PSCI_SYSTEM_SUSPEND_AARCH64, 0, 0, 0);

defined the invoke_psci_fn as below.. but got a doubt how does it gets invoked with the bl31.bin

is there any linker updates we need to do, or below code will work fine for me

****************************************************************************

typedef unsigned long (psci_fn)(unsigned long, unsigned long,
unsigned long, unsigned long);
static psci_fn *invoke_psci_fn;

#define SMCCC_VERSION 0x80000000
void suspend_using_atf(void)
{

    struct udevice *dev;
    invoke_psci_fn(SMCCC_VERSION , 0, 0, 0);

return 1;
}

*********************************************************************************************

Re: A53 core suspend

qiang_li-mpu_se — Mon, 10 Jul 2023 09:47:08 GMT

Hi @Lakshmi_AG

If your code is 32 bits, I think it can't work, because the ATF is 64 bits.

In Uboot, some files can be referenced for psci:

uboot-imx\arch\arm\cpu\armv8\psci.S

uboot-imx\arch\arm\cpu\armv8\smccc-call.S

uboot-imx\arch\arm\cpu\armv8\cpu.c

uboot-imx\drivers\firmware\psci.c

The smccc, psci method can be found from ARM website:

https://developer.arm.com/Architectures/Secure%20Monitor%20Calling%20Convention

https://developer.arm.com/documentation/den0028/latest/

From my understand, ATF has the handler to process smccc function calls.

Re: A53 core suspend

qiang_li-mpu_se — Tue, 11 Jul 2023 03:21:37 GMT

After ATF (bl31.bin) run, it will register service and handler for smccc. Then when kernel, Uboot or other software calls invoke_psci_fn(), it will goto arm_smccc_smc() as followed, the ATF handler and service will get the function call. No linker updates are needed.

ENTRY(__arm_smccc_smc)

SMCCC smc

ENDPROC(__arm_smccc_smc)

Re: A53 core suspend

Lakshmi_AG — Wed, 12 Jul 2023 06:24:30 GMT

Hi @qiang_li-mpu_se

If we are in 32bit mode, and if we cannot communicate with ATF(since atf running in 64 bit) with smccc method, is there any other way to do power management.

or any other mode to talk to ATF from 32 bit system

Regards

Lakshmi