Reading writing CAAM MCFGR, DECORR from Linux when TrustZone is enabled

[bryan_odonoghue]

Hello.

On an i.MX7Solo I find that when TrustZone is enabled MCFGR, DECORR return zeros instead of valid data when read/written from the upstream Linux CAAM driver.

The problem is described here : Kernel panic with CAAM on IMX6UL EVK? · Issue #1408 · OP-TEE/optee_os · GitHub  and you can see I've added my own input.

The only description of this behaviour I can find similar in "Security Reference Manual for i.MX 7Dual and 7Solo Applications Processors, Rev. 0, 03/2017" is at 9.9 SNVS Register Descriptions which states

"Privileged read/write accessible registers can only be accessed for read/write by
privileged software. Unauthorized write accesses are ignored, and unauthorized read
accesses return zero"

I can confirm however that the SNVS Command Register has the "Non-Privileged Software Access Enable" bit set to 1.

HPCOMPR = 0x80002100

It is possible to read valid data from the alias registers later on in the memory map. For example CAAM Status (CSTA) @ 0xFD4 returns valid and expected data.

Could NXP provide some guidance on how and why the MCFGR and DECORR registers are returning zero to Linux when TrustZone is enabled and how to correctly map those registers into a non-TrustZone context ?

[bryan_odonoghue]

Poking about about I've managed to get this working.

There are two things that need to happen on MX7 at least for me

#1 Assign ownership of the job-ring registers to non-secure mode - prior to enabling trust zone

In u-boot I do this.

diff --git a/drivers/crypto/fsl/jr.c b/drivers/crypto/fsl/jr.c
index 986eabf..ad2bd43 100644
--- a/drivers/crypto/fsl/jr.c
+++ b/drivers/crypto/fsl/jr.c
@@ -573,6 +573,15 @@ int sec_init_idx(uint8_t sec_idx)
uint32_t liodn_ns;
uint32_t liodn_s;
#endif
+ uint32_t ns_toggle;
+
+ printf("JR0 ms=0x%08x ls=0x%08x\n",
+ sec->jrliodnr[0].ms,
+ sec->jrliodnr[0].ls);
+
+ printf("JR1 ms=0x%08x ls=0x%08x\n",
+ sec->jrliodnr[1].ms,
+ sec->jrliodnr[1].ls);

if (!(sec_idx < CONFIG_SYS_FSL_MAX_NUM_OF_SEC)) {
printf("SEC initialization failed\n");
@@ -621,6 +630,22 @@ int sec_init_idx(uint8_t sec_idx)
#endif
#endif

+ ns_toggle = sec_in32(&sec->jrliodnr[0].ms);
+ ns_toggle |= 0x08;
+ sec_out32(&sec->jrliodnr[0].ms, ns_toggle);
+
+ ns_toggle = sec_in32(&sec->jrliodnr[1].ms);
+ ns_toggle |= 0x08;
+ sec_out32(&sec->jrliodnr[1].ms, ns_toggle);
+
+ printf("JR0 ms=0x%08x ls=0x%08x\n",
+ sec->jrliodnr[0].ms,
+ sec->jrliodnr[0].ls);
+
+ printf("JR1 ms=0x%08x ls=0x%08x\n",
+ sec->jrliodnr[1].ms,
+ sec->jrliodnr[1].ls);
+

#2 Skip RNG initialisation in Linux (thanks to Peng Fan) in NXP for this suggestion

diff --git a/drivers/crypto/caam/ctrl.c b/drivers/crypto/caam/ctrl.c
index 98468b9..a79e11e 100644
--- a/drivers/crypto/caam/ctrl.c
+++ b/drivers/crypto/caam/ctrl.c
@@ -693,6 +693,7 @@ static int caam_probe(struct platform_device *pdev)
kick_trng(pdev, ent_delay);
ent_delay += 400;
}
+ #if 0
/*
* if instantiate_rng(...) fails, the loop will rerun
* and the kick_trng(...) function will modfiy the
@@ -708,6 +709,9 @@ static int caam_probe(struct platform_device *pdev)
* so don't hog the CPU
*/
cpu_relax();
+ #else
+ dev_info(dev, "Relying on bootloader CAAM RNG init\n");
+ #endif
} while ((ret == -EAGAIN) && (ent_delay < RTSDCTL_ENT_DLY_MAX));
if (ret) {
dev_err(dev, "failed to instantiate RNG");

[bryan_odonoghue]

So this text in the NXP document describes the behaviour I'm seeing exactly

NOTE
The CAAM address space is divided into 16 4 KB pages to
match the access granularity of the MMU. Registers that are
intended to be accessed by a specific processor or process are
grouped into one of these 16 pages so that access to these
registers can be restricted via CAAM's own MID-based access
control mechanism, or via the CPU's MMU. For instance, the
general configuration and status registers are located within
page 0 and are intended to be accessed only by privileged
software. The registers that control each job ring are located in
a separate address block so that access to each job ring can be
restricted to a particular process. Some registers, such as the
version ID registers, are intended to be shared among processes.
Rather than require each CAAM driver process to have two
MMU page entries, one page for its private registers and one
for the shared registers, CAAM "aliases" these shared registers
into the upper section of each of the 16 address blocks. Reading
any one of the address aliases for the same register returns the
same information. Some of these aliased registers are writable,
so access to these registers may require that software implement
a concurrency control construct, as would be the case with any
register that is read/write accessible by multiple processes.

The only question is - how do we tell the MMU (or is it the CAAM MID ) to assign control of page 0 over to Linux instead of TEE/TrustZone....

[jamesbone]

Hello Bryan,

We are discussing your issue internally, but since this are holidays, the answer can be a little delay.  I apologize for the inconvenience.

Happy New Year 2018