Trusty OS SS app can't create TP port

[rob_mclean]

I'm trying to use Trusty Tee with the Android 9.0 release and a 4.14 Linux kernel.  I do see that Trusty starts properly from SPL/U-BOOT, and the device boots to the Android home screen.  However, in the log console every 5 seconds I get these messages (see attached log files for more details):

[ 452.893314] trusty: ss-ipc: 382: Failed to create port com.android.trusty.storage.client.tp -14
[ 452.893317] trusty: ss: client_create_port: failure initializing client port (-14)
[ 452.893320] trusty: ss: proxy_connect: block_device_tipc_init failed (-14)
[ 452.893323] trusty: ss-ipc: 105: do_connect: failure initializing channel state (1002)
[ 452.894885] init: Service 'storageproxyd' (pid 4027) exited with status 1
[ 452.961482] init: Sending signal 9 to service 'storageproxyd' (pid 4027) process group...
[ 452.969879] libprocessgroup: Successfully killed process cgroup uid 0 pid 4027 in 0ms

I have been reading (and following) the IMX_ANDROID_SECURITY_USERS_GUIDE but I haven't generated the AVB key or programmed the attestation key.  I did program the RPMB fuses in the eMMC, and I think that the RPMB storage proxy service is available from U-Boot because I see this (seemingly error-free) output from u-boot prior to starting Linux:

INFO Initializing Trusty device
INFO selected trusty api version: 3 (requested 3)
INFO Initializing Trusty IPC device
INFO Initializing RPMB storage proxy service
INFO Initializing Trusty AVB client
INFO Initializing Trusty Keymaster client
INFO Initializing Trusty Hardware Crypto client

I did trace the code in the Trusty ss app all the way to an assembly call to "_trusty_port_create" where it puts the "__NR_port_create" value (0x10) into r12, and then makes a call to SVC to communicate back to the unsecure world where I think the Linux driver should be listening to create the TP port that the Trusty OS needs created for communication to happen between the secure and unsecure worlds.  I'm not sure what is blocking that from happening.

I'm also wondering, do the attestation keys need to be programmed before the Trusty OS can create the TP port, and/or is there something else that I'm missing?

It just occurred to me.  I built the Trusty OS LK outside of my Android build tree. Potentially with a different compiler than my Linux Kernel, but I did use the NXP Android 9.0 repo for the Trusty sources, so I am assuming that is a reasonable way to build it.

[rob_mclean]

By sprinkling both the Trusty OS code and the Linux kernel code, I've finally found the problem is not that Trusty OS SS can't create the TP port.  It is that there is a problem with mmc rpmb device that isn't enumerating as a block device.

I don't think I realized how all these pieces interact, but now I understand that the Linux kernel driver only maintains the connection channels that allow the Trusty OS (and it's apps) to communicated with Android apps.

In my case Trusty needs Android to perform an action for it.  The action that is failing is "rpmb_send".  Trusty packages up a request that gets sent via the Linux provided channel to storageproxyd which is an app that Android fires off.  That request seems to be getting properly received by storageproxyd, and in trying to perform the requested action, storageproxyd needs to send the MMC_IOC_MULTI_CMD ioctl to the mmc rpmb device.  That command is failing because the mmc rpmb device in my kernel is only getting enumerated as a character device and not as a block device.  I was working around the problem by copying the device node into the /dev/block directory, but I see now that the real solution is to understand why the rpmb device isn't getting properly enumerated in the block device directory.

This problem isn't solved but I'm willing to abandon this posting in favor of another one that is more appropriately titled.  I'll post a link to that new question when it becomes available.

[rob_mclean]

I figured out how to start the storageproxyd with the character RPMB eMMC device that was getting enumerated, and it looks like the trusty ss app is able to communicate with the storageproxyd enough to get the storageproxyd to read the requested block from the RPMB.  After that it appears as if the storageproxyd is unable to communicate back to trusty ss app the data from the read.  That is where the cant' create TP port shows up.

The error code -14 comes from the trusty ss app, and it is defined in a header file as "ERR_ALREADY_EXISTS".  Which makes me wonder why is the trusty ss app trying to recreate a port that it already created?  Is this a trusty ss app bug?  If so I'm probably posting to the wrong forum for help.

[rob_mclean · Accepted solution]

Turns out, the problem was the rpmb_proxy in u-boot was causing Trusty to create the port in the first place, and the way we are having u-boot boot the system was not shutting down the u-boot rpmb_proxy connection.  So when the storageproxyd came along from within Android and it attempted to create a new rpmb_proxy, that attempt was failing because the u-boot rpmb_proxy never disconnected.

To prove this I added the "trusty_ipc_shutdown" call to the u-boot boot command that we are using, and that fixed this problem.  I found that function call in the "do_boota" command, which is what should be used to boot android from mmc.

We might need to reconsider how we are telling u-boot to boot Android as a result of this finding.

[rob_mclean]

I turned on the debug output from the trusty kernel driver, and posted the entire boot log (including some of my own debug info).

Something that shows up more prominently now is that I did nothing to format the RPMB partition prior to burning the fuses in the eMMC.  It shows up as the Trusty OS  SS app error:

[ 61.385246] trusty: _Bool super_block_valid(const struct block_device *, const struct super_block *):154: bad magic, 0x8c6a2ba476cc72f0

I'm not sure if that is the root cause of this problem or not.

This link shows what a typical eMMC block 0 looks like before and after the first write, and the magic number reported in that error message isn't what it should be.