topic Re: OPTEE with Android on IMX6 quad in i.MX Processors

OPTEE with Android on IMX6 quad

saidgani_musaev — Thu, 23 Aug 2018 18:51:14 GMT

Hello everyone, I am using Nitrogen6x board from boundarydevice. It's based on IMX6Q chip. I am working on studying project related to Trusted execution environment. I could boot OPTEE with Linux kernel from linaro repo. When I try to boot Android with OPTEE it is stuck. I see logs from OPTEE itself: the last log message is "Primary CPU switching to normal world boot". I got android sources from boundary device private repo. Without OPTEE Android boots correctly. With the same OPTEE kernel, Linux kernel boots correctly as well( all client tests pass ). I recompiled Android kernel with CONFIG_EARLY_PRINTK and CONFIG_EARLY_PRINTK_DIRECT. Also I disabled SMP to not boot secondary CPUs. I don't see any message after OPTEE executes "smc #0" for first time. I assume Android booting code does do something which is forbidden in normal world. I wrote to Boundary Device support (Android Getting Started Guide - Boundary Devices ), but they directed me here :smileyhappy:. What could be a good starting point for me to get deep and to port Android to my board? As I understand this Android distribution will not work with "Trusty"-TEE, right?

Re: OPTEE with Android on IMX6 quad

chenlifu2015 — Fri, 24 Aug 2018 03:43:28 GMT

Hi,Saidgani Musaev

I am also trying to load Android img with OP-TEE,but how do you boot tee.bin and boot.img, use boota cmd or bootm? can you tell me the environment in U-Boot ? I have been blocked by this problem, Thanks

Re: OPTEE with Android on IMX6 quad

saidgani_musaev — Fri, 24 Aug 2018 06:51:01 GMT

Hello,

Which chip do you use? In general it is similar to what you are doing for booting Linux Kernel. I modified 6x_bootscript which is located on boot partition on my boot flash( note that I am working with ready sdcard. I put link to instruction in my question). In this script I have changed addresses of zImage, fdt, ramdisk and script. It really depends on how did you compile OPTEE( in my case FDT is supposed to be on address 0x18000000, NS_Entry_addr is 0x12000000 ), so in my case I used these addresses:

setexpr a_script 0x10000000
setexpr a_zImage 0x12000000
setexpr a_fdt 0x18000000
setexpr a_ramdisk 0x13800000

just before booting Android Kernel I loaded tee.bin (I used address 0x20000000 for this purpose), and then I just changed bootz {a_zImage}... to bootm 0x20000000. I hope it helped, Please let me know about your progress and which board do you use. Anyway I am going to put step by step guide to boot it on for my board. Thank you.

Re: OPTEE with Android on IMX6 quad

chenlifu2015 — Mon, 27 Aug 2018 01:44:20 GMT

Hi，Saidgani Musaev

Thanks for your apply，the type of board which I am using is IMX6Q，the method of booting the IMX6Q board is by sdcard，the address of FDT and NS_ENTRY are the same with your board.I do not understand the meaning of "then I just changed bootz {a_zImage}... to bootm 0x20000000", like bootm 0x20000000 - 0x18000000 ? and I can not find your 6x_bootscript too, can you just copy and paste your bootscript in the answer,thank you very mach!

Re: OPTEE with Android on IMX6 quad

saidgani_musaev — Mon, 27 Aug 2018 10:42:04 GMT

Hello, yes, like bootm 0x20000000 ..., but you miss ramdisk. I have attached my script here.

# ==================== script ================================

setenv bootpart 1

setexpr rval *0x020CC068 \& 0x180
if itest.s "x$rval" -eq "x100"; then
   echo "Starting fastboot..."
   fastboot 0
else
   if itest.s "x$rval" -eq "x80"; then
       setenv bootpart 2;
   fi
fi
mw.l 0x020cc068 0 1

setenv bootargs ''

m4=''
a_base=0x10000000
if itest.s x51 == "x${cpu}" ; then
   a_base=0x90000000
elif itest.s x53 == "x${cpu}"; then
   a_base=0x70000000
elif itest.s x6SX == "x${cpu}" || itest.s x7D == "x${cpu}"; then
   a_base=0x80000000
   if itest.s "x1" == "x$m4enabled" ; then
       run m4boot;
       m4='-m4';
   fi
fi

setexpr a_script 0x10000000
setexpr a_zImage 0x12000000
setexpr a_fdt 0x18000000
setexpr a_ramdisk 0x13800000

if itest.s "x" == "x${board}" ; then
echo "!!!! Error: Your u-boot is outdated. Please upgrade.";
exit;
fi

if itest.s "x" == "x${dtbname}" ; then
   if itest.s x6SOLO == "x${cpu}" ; then
       dtbname=imx6dl-${board}.dtb;
   elif itest.s x6DL == "x${cpu}" ; then
       dtbname=imx6dl-${board}.dtb;
   elif itest.s x6QP == "x${cpu}" ; then
       dtbname=imx6qp-${board}.dtb;
   elif itest.s x6SX == "x${cpu}" ; then
       dtbname=imx6sx-${board}${m4}.dtb;
   elif itest.s x7D == "x${cpu}" ; then
       dtbname=imx7d-${board}${m4}.dtb;
   elif itest.s x51 == "x${cpu}" ; then
       dtbname=imx51-${board}${m4}.dtb;
   elif itest.s x53 == "x${cpu}" ; then
       dtbname=imx53-${board}${m4}.dtb;
   else
       dtbname=imx6q-${board}.dtb;
   fi
fi

if load ${dtype} ${disk}:${bootpart} ${a_script} uEnv.txt ; then
env import -t ${a_script} ${filesize}
fi

if load ${dtype} ${disk}:${bootpart} ${a_fdt} ${bootdir}${dtbname} ; then
   fdt addr ${a_fdt}
   setenv fdt_high 0xffffffff
else
   echo "!!!! Error loading ${bootdir}${dtbname}";
   exit;
fi

cmd_xxx_present=
fdt resize
if itest.s "x" != "x${cmd_custom}" ; then
run cmd_custom
cmd_xxx_present=1;
fi

# ------------------- Displays detection
if itest.s "x" != "x${cmd_hdmi}" ; then
   run cmd_hdmi
   cmd_xxx_present=1;
   if itest.s x == x${allow_noncea} ; then
       setenv bootargs ${bootargs} mxc_hdmi.only_cea=1;
       echo "only CEA modes allowed on HDMI port";
   else
       setenv bootargs ${bootargs} mxc_hdmi.only_cea=0;
       echo "non-CEA modes allowed on HDMI, audio may be affected";
   fi
fi
if itest.s "x" != "x${cmd_lcd}" ; then
   run cmd_lcd
   cmd_xxx_present=1;
   setenv bootargs $bootargs tsc2004.calibration=-67247,-764,272499173,324,69283,-8653010,65536
fi
if itest.s "x" != "x${cmd_lvds}" ; then
   run cmd_lvds
   cmd_xxx_present=1;
fi
if itest.s "x" != "x${cmd_lvds2}" ; then
   run cmd_lvds2
   cmd_xxx_present=1;
fi
if itest.s "x" == "x${cmd_xxx_present}" ; then
   echo "!!!!!!!!!!!!!!!!"
   echo "warning: your u-boot may be outdated, please upgrade"
   echo "!!!!!!!!!!!!!!!!"
fi

setenv bootargs "$bootargs console=${console},115200 vmalloc=128M consoleblank=0 androidboot.selinux=permissive androidboot.selinux=enforcing rootwait"
setenv bootargs $bootargs androidboot.hardware=freescale

if itest.s "xsata" == "x$dtype" ; then
   bootdev=sda
elif itest.s "xusb" == "x$dtype" ; then
   bootdev=sda
else
   bootdev=mmcblk${disk}
fi
setenv bootargs $bootargs androidboot.bootdev=$bootdev

if itest.s "x" != "x$selinux" ; then
setenv bootargs $bootargs androidboot.selinux=$selinux
fi

# add serial number based on MAC address
setexpr mac_hi *0x021bc630
setexpr mac_lo *0x021bc620
setenv bootargs $bootargs androidboot.serialno="$mac_hi$mac_lo"

if itest.s "x" != "x$hwrotation" ; then
setenv bootargs $bootargs androidboot.hwrotation=$hwrotation
fi

if itest.s "x" != "x$disable_giga" ; then
setenv bootargs $bootargs fec.disable_giga=1
fi

if itest.s "x" != "x$wlmac" ; then
   setenv bootargs $bootargs wlcore.mac=$wlmac
   setenv bootargs $bootargs androidboot.wlan.mac=$wlmac
   # create a bt mac address from the wlan one
   setexpr btmac sub "(..:..:..:).(.*)" "\\1b\\2" ${wlmac}
   setenv bootargs $bootargs androidboot.btmacaddr=${btmac}
fi

if itest.s "x" == "x$cma" ; then
cma=448M
fi
setenv bootargs $bootargs cma=$cma

if itest.s "x" != "x$gpumem" ; then
setenv bootargs $bootargs galcore.contiguousSize=$gpumem
fi

if itest.s "x" != "x$show_fdt" ; then
fdt print /
fi

if itest.s "x" != "x$show_env" ; then
printenv
fi

tftp 0x20000000 uTee

if ${fs}load ${dtype} ${disk}:${bootpart} ${a_zImage} zImage ; then
   if ${fs}load ${dtype} ${disk}:${bootpart} ${a_ramdisk} uramdisk.img ; then
       bootm 0x20000000 ${a_ramdisk} ${a_fdt} ;
   fi
fi

echo "Error loading the OS"

Re: OPTEE with Android on IMX6 quad

chenlifu2015 — Tue, 28 Aug 2018 07:45:36 GMT

Hi，Saidgani Musaev

Thanks for your apply，I just load optee on IMX6Q by referencing your idea，I just modify the function ：do_boota，because do_boota finally used function :do_bootm，so I fixed the args for the do_bootm like:"sprintf(boot_addr_start, "0x%lx", tee_entry_addr);" (tee_entry_addr is 0x4E000000).In this way，I just load boot.img and the use boota cmd，then，uboot will load the optee correctly，but I also stuck in the log：“Primary CPU switching to normal world boot”， and I think the question is not about android code，I think the reason is we changed the process of loading kernel(like analysis zImge or boot.img)，so when optee has been load and return to normal world for loading kernel，can not get the args(like length load address entry address and bootargs) for loading kernel

Re: OPTEE with Android on IMX6 quad

saidgani_musaev — Tue, 28 Aug 2018 20:09:17 GMT

I don't think so.

Here is code from core/arch/arm/kernel/geberic_entry_a32.S:

/*
      * Save boot arguments
      * entry r0, saved r4: pagestore
      * entry r1, saved r7: (ARMv7 standard bootarg #1)
      * entry r2, saved r6: device tree address, (ARMv7 standard bootarg #2)
      * entry lr, saved r5: non-secure entry address (ARMv7 bootarg #0)
      */
     .macro bootargs_entry
#if defined(CFG_NS_ENTRY_ADDR)
     ldr     r5, =CFG_NS_ENTRY_ADDR
#else
     mov     r5, lr
#endif
#if defined(CFG_PAGEABLE_ADDR)
     ldr     r4, =CFG_PAGEABLE_ADDR
#else
     mov     r4, r0
#endif
#if defined(CFG_DT_ADDR)
     ldr     r6, =CFG_DT_ADDR
#else
     mov     r6, r2
#endif
     mov     r7, r1
     .endm
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

So as you see bootargs are being saved for normal world. Moreover - I could boot Android with OPTEE. But I have weird problem with Android itself - it remounts filesystem, restart services and then reboots itself. I used Android kernel from linaro for hikey( which is ported for using with OPTEE). (GitHub - linaro-swg/optee_android_manifest: Android manifest for building OP-TEE in AOSP )

Re: OPTEE with Android on IMX6 quad

chenlifu2015 — Wed, 29 Aug 2018 06:23:15 GMT

Hi,

It is my bad,but I still think the problem is not about the Android code,maybe we should read the code of the bootloader(u-boot) more carefully，or OP-TEE may not be load correctly（I supprt the SMP，the Second Cpu init will print another log，but I do not see it）because in this case，system.img is load behind boot.img, when we resolve this problem,system.img maybe load.

Re: OPTEE with Android on IMX6 quad

saidgani_musaev — Wed, 29 Aug 2018 19:05:49 GMT

Hi, I could boot everything with different version of the kernel. But currently I have another problem - healthd daemon and some other services don't work. I am still looking for different configurations of AOSP( but it takes a lot of time to download it, build it, reconfigure it). But at least I could boot it. The main problem with current build is that I can't enable adb shell. And I can't get root access through serial console. So, as far as different Kernel works ( with absolutely the same environment, filesystem, init scripts etc.) - I think the problem is in Kernel(boot) code :smileyhappy: .

P.S. You can find link to the working kernel in my previous message.

Re: OPTEE with Android on IMX6 quad

chenlifu2015 — Thu, 30 Aug 2018 01:34:42 GMT

Hi，Thanks for your share，I will use your kernel code to build boot.img and try again，The link to working kernel is GitHub - linaro-swg/optee_android_manifest: Android manifest for building OP-TEE in AOSP？Should I download another version of the kernel，because you said " I could boot everything with different version of the kernel",can you tell me the link to working OP-TEE and Android Code，I want to use the same code exactly like you，so I will rule out the possibilities of wrong code。

Re: OPTEE with Android on IMX6 quad

chenlifu2015 — Fri, 31 Aug 2018 07:00:27 GMT

Hi,I have used the kernel with the link： GitHub - linaro-swg/optee_android_manifest: Android manifest for building OP-TEE in AOSP，but it does not work ，too.Should I download another version of the kernel,I have been drived mad by this question，I am looking forward to your apply，thanks for your help in this week。

Re: OPTEE with Android on IMX6 quad

chenlifu2015 — Thu, 06 Sep 2018 03:24:25 GMT

Hi，I'm sorry to bother you，but I have tried so many versions of kernel, but I still can not boot optee and boot.img correctly,so,can you tell me the link to the working kernel ,and the version you use, Thanks a lot

Re: OPTEE with Android on IMX6 quad

saidgani_musaev — Thu, 06 Sep 2018 15:56:37 GMT

Hello , I am really sorry for "ignoring" your questions. I was busy recent days. I would really love to help you with that. Could we move our discussion in other chat. Because of it's hard to discuss it here. You can find me on Facebook by my name. Or Send me a pm, please. Thank you.

Re: OPTEE with Android on IMX6 quad

chenlifu2015 — Mon, 10 Sep 2018 06:46:20 GMT

Hi,Saidgani Musaev

I have send a message to you in facebook,my name is the same with the username in NXP: 陈礼夫，I enable the CONFIG_DEBUG_PRINTK and CONFIG_DEBUG_LL,I got more details about the problem:

Unhandled fault: imprecise external abort (0xc06) at 0x12402d8c
10:33:41.297: pgd = c0004000
10:33:41.301: [12402d8c] *pgd=00000000
10:33:41.301: Internal error: : c06 [#1] PREEMPT ARM
10:33:41.305: Modules linked in:
10:33:41.309: CPU: 0 PID: 0 Comm: swapper Tainted: G W 4.1.15-dirty #1
10:33:41.313: Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
10:33:41.318: task: c110a3f8 ti: c1100000 task.ti: c1100000
10:33:41.323: PC is at find_static_vm_vaddr+0x18/0x6c
10:33:41.326: LR is at __iounmap+0x14/0x50
10:33:41.329: pc : [<c01141d0>] lr : [<c011439c>] psr: 20000093
10:33:41.333: sp : c1101fa0 ip : 000000c0 fp : 00000000
10:33:41.338: r10: ee7572c0 r9 : c10846e0 r8 : c11f4000
10:33:41.342: r7 : ffffffff r6 : c1102000 r5 : c11f4000 r4 : f4a02000
10:33:41.347: r3 : eefffff8 r2 : eeffffd8 r1 : c110bf0c r0 : f4a02000
10:33:41.352: Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment kernel
10:33:41.359: Control: 10c57c7d Table: 10004059 DAC: 00000015

so I am sure the the problem is in Kernel(boot) code,so can you tell me the right version for kernel code you use

Thanks a lot