imx8mm: Wake Linux from M4 core

[DavidC1]

We are developing a product that has an i.MX8 mini SoC in it (imx8mm).

There is an IMU (MEMS motion sensor) connected to it, which we are reading using code running the M4 core.  We use RPMSG to communicate with the product's main application, running on Linux on the A53 cores.

We already know from experience that the flow of data from the M4 to Linux via RPMSG prevent Linux from going to sleep.  So we have code where we tell the M4 code to stop sending messages before we attempt to enter sleep.  We then send it a command to resume sending messages after waking from sleep.  This code works.  We can wake it from sleep using the same button/interrupt we use to power our device on and off.

The next step, however, is to enter/leave sleep based on lack of motion, as determined by the IMU chip.  I have already written the code so we can tell the M4 to stop sending messages until significant motion is detected, and then resume.  This works great for temporarily suspending the data flow from the M4 to Linux, but the resumption of data flow does not cause Linux to wake from sleep.

It seems that the RPMSG activity is sufficient to prevent Linux from going to sleep, but they are not sufficient to wake it from sleep.

I don't yet know the reason why, but I assume the interrupt used to signal RPMSG data to Linux isn't able to wake the A53 cores from sleep.  I assume I need to configure something in order to tell the IMX that its interrupt should be treated as a wakeup event, but I don't know what it is that I need to configure.

I already looked at application note AN12195 (low power audio) which talks about doing something very similar (for audio), but it doesn't include any code fragments.  It only has a high-level description and links to Git repositories for massive components (like the Linux kernel), without specifying where the relevant bits of code or patch files may be found.

Update:

We've recently done some more testing, connecting to the M4's debug console and adding trace messages and it seems that the M4 core is being put to sleep along with the A53 cores.

If we start our application immediately after Linux boots, and it goes to sleep soon afterward (we've been using a 10s timeout for testing), and then move the device very soon afterward, then it will wake from sleep.

If, however, we allow the device to be running long enough for Linux to finish its startup sequences (including starting a few kernel device drivers that don't seem to start until after 60 seconds), then no luck.  The device can not be awakened.  Messages on the M4 console and GPIO activity from the M4 application both stop as well.  It appears that the M4 has been put to sleep.  So only an external interrupt (e.g. our power button) can wake it.

Similarly, if our device goes to sleep before Linux completes its startup, but remains asleep until after the kernel drivers all start themselves, kernel messages indicate that part of the sleep process waits for the kernel drivers to start, and then it goes completely to sleep.  At which point, M4 activity also stops.

So the question is no longer how to get the M4 to generate an interrupt that will wake up Linux.  That seems to be working fine.  The question now is how to put the Linux cores (the A53) to sleep without putting the M4 to sleep.

[jimmychan]

This may help you.

https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/M4-Low-Power-Demo-on-i-MX8MM/ta-p/110110...

[DavidC1]

No luck with the 5.10.52 ("Hardknott") BSP distribution.

The person who posted the help to that thread shared code for the 4.14.98 ("Sumo") kernel, which works great, but his patch for the 5.4 kernel did not work when I tried to apply it to a 5.10 kernel.

It appears that I can no longer reply to that thread (it must be too old).  Can you help out?  Or can @terry_lv possibly help out here as he did in the original thread?  I am interested in getting this to work on 5.10.52 (right now) and later for 5.15 ("Kirkstone"), which we are evaluating for use in the future.

Attached are the patch files I (unsuccessfully) tried to use on my 5.10.52 kernel.  The "enable-a53-fast-wakeup..." patch file is for the imx-atf component.  The "imx8mm-gir-wakeup" patch is for the linux-imx component.

When I used these patches, not only could the M4 not wake the device, but it seems that my other method of waking it (the interrupt from our power-on button) also didn't work.  I was forced to use our device's force-power-off in order to recover.

[terry_lv]

Hi,

  Please refer to the AN13400:
https://www.nxp.com.cn/docs/en/application-note/AN13400.pdf

  The implementation mentioned in Chapter 5.3 might help you.

  Thanks!

Regards

Terry

[DavidC1 · Accepted solution]

I think I've finally got it working for real (again, for the Linux 5.10.52 "Hardknott" kernel).

Since the last patch files I uploaded, I made the following changes:

• Get rid of the functions in mx8_mu.c.  As you suggested, I wrote (new and better) versions in imx-mailbox.c
• Set the GIE flag to enable the GP interrupt.  Do it in imx_mu_probe
• In imx_mu_isr, if we receive an interrupt that isn't being handled by other code (that is, not for messaging), check to see if there's an unhandled GIR interrupt.  If so, clear the flag and let Linux know that it has been handled.  (The patch file has a warning message in there for debugging, which I'll remove once I'm a bit more confident that the code works.)

@terry_lv: Please review the two patch files and let me know if I'm doing anything wrong that I should change.  It seems to work, but I'm still very new to Linux kernel development so I might have missed something important.

To everybody else reading this, I hope it helps you with your problems.

[DavidC1]

After a lot of study, I am starting to understand more.

It appears that the imx-mailbox.c file already loads the DeviceTree object when the driver starts up (via the .of_match_table field in the imx_mu_driver struct).  So the imx_mu_probe function already has it, included in the paltform_device pointer it is given.

The existing call to devm_platform_ioremap_resource appears to fetch the correct address, and maps it onto virtual address space (hence the reason why dumping the pointer doesn't show 0x30aa0000, but a different address).

I added this line just before the call to pm_runtime_enable():

MU_EnableGIRInt(priv->base, 0x8);

With this, I can wake from sleep once, but not again, because I'm not clearing the interrupt.  So my next step is to figure out where to add the call to MU_ClearGIRInt().

Attached are the two patches I've got so far.  Patch 0001 is for ATF.  Patch 0002 is for Linux.

Please take a look and let me know if I'm doing this right.

[DavidC1]

Hi, Terry,

I'm re-posting my reply because my previous one got deleted for some reason.

AN13400 explains what needs to be done at a register level, but it doesn't talk about how to patch the Linux kernel in order to implement the changes, which is where I need help.

I did find AN13340 (code here: https://source.codeaurora.org/external/imxsupport/cortex-M4_Cortex-A53_low_power_use_cases/tree/), which is designed for use with the 5.10 Linux kernel.  It was able to help me with half the solution.  Based on it, I was able to create a patch for the ATF code (attached).  With this, the M4 keeps running while the A53 sleeps, but additional work is needed to enable the interrupt that the M4 generates in order to wake the A53.

In another discussion thread (https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/M4-Low-Power-Demo-on-i-MX8MM/ta-p/110110...), you provided working patches for the 4.14 Linux kernel.  They work great for me, Thanks.

You discuss patches to the imx_mailbox.c source file in order to enable the wakeup interrupt, but the kernel 5.4 patches you provide do not work on my 5.10 kernel (and I'm not sure how they would work on 5.4 either).

In following your advice, however, I have ported the two MU_ClearGIRInt and MU_EnableGIRInt functions.  I have two questions, however, which I'm hoping you can answer:

• When calling MU_EnableGIRInt from the imx_mu_probe() function, where do I get the base address?  Should I hard-code the constant 0x30aa0000?  Should I read it from the DeviceTree?  Is it already available somewhere?
• I see (from the 4.14 kernel patches), that I need to call MU_ClearGIRInt in the MU interrupt handler, in order to clear the GIR bit.  Should I do this from imx_mu_isr()?  If not, where?  And where should I insert the call?

Finally, the MU base address is in the DeviceTree, but I don't know how to read it correctly.  I tried the following code:

struct device_node *np_mu;
void __iomem *mu_base;
...
np_mu = of_find_compatible_node(NULL, NULL, "fsl,imx8mm-mu");
mu_base = of_iomap(np_mu);
pr_info("MU base is 0x%px\n", mu_base);

but it did not print the expected value (0x30aa0000).  Instead, it printed 0xffff80001b1b0000.

So am I reading the value incorrectly?  If so, how should I read it?

Thank in advance,

-- David

[terry_lv]

Hi,

  In your patches, the mailbox code is changed for wakeup. It's an optional way in AN13400. See Chapter 5.3.2 in AN13400.

  I think two changes are needed to wakeup A53 from M4:

1. Define a LPA flag in your M4 project.
   If your M4 project don't need to use DDR, use DSP_LPA_ACTIVE in your project, otherwise, use M4_LPA_ACTIVE (Chapter 5.2.2 in AN13400).
2. See AN13400 5.2.3, the M core wakeup A53 by the changes in M core and ATF.
   

  Pls refer to software package of AN13400 for patches.

  Thanks!

Regard

Terry

[DavidC1]

Hi, Terry,

I read through AN13400.  It describes what needs to be done, but doesn't talk about how to integrate this into the Linux kernel. which is where I need help.

I did, however, find AN13340 (https://www.nxp.com/webapp/Download?colCode=AN13340&location=null), and its code (https://source.codeaurora.org/external/imxsupport/cortex-M4_Cortex-A53_low_power_use_cases/tree/), which has been a bit helpful.

After examining its ATF patch, I was able to create the "enable a53 fast wakeup stop when m4 run" patch (attached).  This solves half the problem - the M4 no longer sleeps when the A53 cores sleep.

I am still having a problem with the Linux patch.  I know that I need to set a bit in the MUA_ACR register to enable a GIE bit, and I know that I need to add something to its interrupt handler to clear it from MUA_ASR after the interrupt arrives.

Your comments in this thread: https://community.nxp.com/t5/i-MX-Processors-Knowledge-Base/M4-Low-Power-Demo-on-i-MX8MM/ta-p/110110... say that I should do this in the imx-mailbox.c file, but they aren't clear about exactly what needs to be done there (and the patch file you uploaded doesn't seem to work):

• Where do I get the mu_base address?  In the 4.14 kernel, the code gets it from the DeviceTree.  For the 5.10 kernel, I see that same address in the DeviceTree (imx8mm.dtsi) as "mu : mailbox@30aa0000".  The base address (30aa0000) is in the "reg" field (reg = <0x30aa0000 0x10000>;), but I don't know how I'm supposed to read it from there.
  • I tried the following (in the probe function), but it didn't give me the expected value, so I don't know if it is correct:

np_mu = of_find_compatible_node(NULL, NULL, "fsl,imx8mm-mu");
mu_base = of_iomap(np_mu, 0);

When I use this code and print the pointer (using the "%px" format spec, to avoid seeing a hashed pointer value), I don't see the expected value (0x30aa0000), but I get something completely different (0xffff80001b1b0000).

So, how should I get that base address?  Either I'm reading it wrong or I'm printing it wrong.

• As you suggested, I ported the two ASR/ACR accessor functions (MU_EnableGIRInt() and MU_ClearGIRInt()).
• Assuming I can get the correct base address, I know I should call MU_EnableGIRInt() in imx_mu_probe().  This shouldn't be a problem.
• I will, I assume, need to add a call to MU_GlearGIRInt() to the GP interrupt handler function.  Is there an existing function I can use?  Would it be imx_mu_isr()?  And how should I call it from there?  This function is significantly more complicated than the RPMSG isr function from the 4.14 kernel.

I'm looking forward to your assistance here.  I think I am very close to a working solution, but this is not easy code to understand.

Thanks in advance,

-- David

[DavidC1]

Thanks much.  The solution presented there works great with the 4.14 (Sumo) kernel.

I will now try and apply it to the 5.10 (Hardknott) kernel.  The solution doesn't mention this version, but it does provide alternative code for 5.4.  Hopefully those changes will work.  If not, I may have to ask for additional help.