I.MX8MP boot CM7 via linux kernel remoteproc

[LihuaLiu123]

hi all, 

i'm trying to boot cm7 via remoteproc driver "imx_rproc" but failed with both TCM and ddr_debug build.

1. boot with TCM build:

• i disabled the imx_rpmsg driver and enable imx_rproc driver from devices tree.
• system hang happen when loading the first fw segment to the TCM area.

echo start > /sys/class/remoteproc/remoteproc0/state
[ 38.348966] remoteproc remoteproc0: powering up imx-rproc
[ 38.357736] remoteproc remoteproc0: Firmware is an elf32 file
[ 38.363508] remoteproc remoteproc0: Booting fw image rpmsg_lite_pingpong_rtos_linux_remote.elf, size 164408
[ 38.373276] imx-rproc imx8mp-cm7: lihual2,iommu not present
[ 38.378875] remoteproc remoteproc0: lihual, rproc_fw_boot, fw boot entry 51d
[ 38.385958] imx-rproc imx8mp-cm7: lihual, add carveout m7, base 0x80000000 da 0x80000000, size 0x1000000
[ 38.395482] imx-rproc imx8mp-cm7: lihual, add carveout vdev0vring0, base 0x55000000 da 0x55000000, size 0x8000
[ 38.405501] imx-rproc imx8mp-cm7: lihual, add carveout vdev0vring1, base 0x55008000 da 0x55008000, size 0x8000
[ 38.415525] imx-rproc imx8mp-cm7: lihual, add carveout vdevbuffer, base 0x55400000 da 0x55400000, size 0x100000
[ 38.425640] lihual, imx_rproc_parse_fw, before set 0xa8
[ 38.430885] lihual, imx_rproc_parse_fw, after set 0xaa
[ 38.436071] remoteproc remoteproc0: lihual, rproc_handle_resources, enter
[ 38.442888] remoteproc remoteproc0: rsc: type 3
[ 38.447443] remoteproc remoteproc0: vdev rsc: id 7, dfeatures 0x1, cfg len 0, 2 vrings
[ 38.455437] remoteproc remoteproc0: vdev rsc: vring0: da 0x55000000, qsz 256, align 4096
[ 38.463567] remoteproc remoteproc0: vdev rsc: vring1: da 0x55008000, qsz 256, align 4096
[ 38.471696] imx-rproc imx8mp-cm7: map memory: 00000000b1f29fcf+1000000
[ 38.478327] imx-rproc imx8mp-cm7: map memory: 0000000063fce603+8000
[ 38.484646] imx-rproc imx8mp-cm7: map memory: 00000000616f1cee+8000
[ 38.490952] imx-rproc imx8mp-cm7: map memory: 00000000e585adc7+100000
[ 38.497448] remoteproc remoteproc0: imx_rproc_elf_load_segments, phdr: type 1 da 0x0 memsz 0x2a8 filesz 0x2a8
[ 38.507392] remoteproc remoteproc0: da = 0x0 sys_addr: 0x7e0000 len = 0x2a8 va = 0x00000000ed360d72   // va is maped via devm_ioremap

I doubt there are some issues from TCM acess as i am not sure if  A core can acess TCM directly or protected by some module, so i tried ddr build:

• From the log, it seems that fw segments have loaded to ddr 0x80000000 sucessfully and everything looks fine.
• But no "hello world" print to cm7 uart port and seems cm7 isn't started.
• By default, cm7 is booted from uboot via "bootaux" command, and i can boot cm7 with it.
• Are there any guides or docs for booting cm7 via remoteproc?
• software i run: linux-imx branch lf-5.10.y

logs:

root@imx8mpevk:~# echo start > /sys/class/remoteproc/remoteproc0/state
[ 761.015820] remoteproc remoteproc0: powering up imx-rproc
[ 761.021502] remoteproc remoteproc0: Firmware is an elf32 file
[ 761.027310] remoteproc remoteproc0: Booting fw image rpmsg_lite_pingpong_rtos_linux_remote.elf, size 164408
[ 761.037080] imx-rproc imx8mp-cm7: lihual2,iommu not present
[ 761.042679] remoteproc remoteproc0: lihual, rproc_fw_boot, fw boot entry 8000051d
[ 761.050194] imx-rproc imx8mp-cm7: lihual, add carveout m7, base 0x80000000 da 0x80000000, size 0x1000000
[ 761.059743] imx-rproc imx8mp-cm7: lihual, add carveout vdev0vring0, base 0x55000000 da 0x55000000, size 0x8000
[ 761.069783] imx-rproc imx8mp-cm7: lihual, add carveout vdev0vring1, base 0x55008000 da 0x55008000, size 0x8000
[ 761.079823] imx-rproc imx8mp-cm7: lihual, add carveout vdevbuffer, base 0x55400000 da 0x55400000, size 0x100000
[ 761.089954] remoteproc remoteproc0: lihual, rproc_handle_resources, enter
[ 761.096769] remoteproc remoteproc0: rsc: type 3
[ 761.101329] remoteproc remoteproc0: vdev rsc: id 7, dfeatures 0x1, cfg len 0, 2 vrings
[ 761.109325] remoteproc remoteproc0: vdev rsc: vring0: da 0x55000000, qsz 256, align 4096
[ 761.117439] remoteproc remoteproc0: vdev rsc: vring1: da 0x55008000, qsz 256, align 4096
[ 761.125563] imx-rproc imx8mp-cm7: map memory: 00000000b2faa939+1000000
[ 761.132130] imx-rproc imx8mp-cm7: map memory: 000000006cc4dfc6+8000
[ 761.138440] imx-rproc imx8mp-cm7: map memory: 000000005e49fc1d+8000
[ 761.144755] imx-rproc imx8mp-cm7: map memory: 000000001ec5a9c7+100000
[ 761.151269] remoteproc remoteproc0: phdr: type 1 da 0x80000000 memsz 0x2a8 filesz 0x2a8
[ 761.159295] remoteproc remoteproc0: da = 0x80000000 len = 0x2a8 va = 0x000000006c96a30c
[ 761.167322] remoteproc remoteproc0: phdr: type 1 da 0x80000400 memsz 0x460c filesz 0x460c
[ 761.175527] remoteproc remoteproc0: da = 0x80000400 len = 0x460c va = 0x00000000317ae83d
[ 761.183677] remoteproc remoteproc0: phdr: type 1 da 0x80004a0c memsz 0xab00 filesz 0x68
[ 761.191716] remoteproc remoteproc0: da = 0x80004a0c len = 0xab00 va = 0x0000000031885095
[ 761.199883] remoteproc remoteproc0: da = 0x80000400 len = 0x58 va = 0x00000000317ae83d
[ 761.207821] imx-rproc imx8mp-cm7: lihual, imx_rproc_start, method 0
[ 761.214120] lihual, imx_rproc_ready, wait for remote ready
[ 761.269630] lihual, imx_rproc_ready, timerout for remote ready   // rxdb irq timerout for cm7 ready,fw may not started.
[ 761.275477] remoteproc0#vdev0buffer: lihual, rproc_add_virtio_dev, try to find vdev0buffer from carveout!
[ 761.285170] virtio virtio0: reset !
[ 761.288697] virtio virtio0: status: 1
[ 761.292560] virtio_rpmsg_bus virtio0: status: 3
[ 761.297147] remoteproc remoteproc0: vring0: va 0000000095de817d qsz 256 notifyid 0 ring_size 10246
[ 761.306146] remoteproc remoteproc0: vring1: va 00000000fb2ff26e qsz 256 notifyid 1 ring_size 10246
[ 761.315128] virtio_rpmsg_bus virtio0: lihual, rpmsg_probe buf[512, 512]
[ 761.323636] virtio_rpmsg_bus virtio0: buffers: va 0000000013568248, dma 0x00000000d49c0000
[ 761.332195] (null): lihual, rpmsg_ctrldev_probe
[ 761.337996] virtio_rpmsg_bus virtio0: lihual, rpmsg_probe, has feature VIRTIO_RPMSG_F_NS
[ 761.338805] imx6q-pcie 33800000.pcie: supply epdev_on not found, using dummy regulator
[ 761.346295] rpmsg_ns virtio0.rpmsg_ns.53.53: lihual, rpmsg_ns_probe
[ 761.557637] virtio_rpmsg_bus virtio0: status: 7
[ 761.562246] remoteproc remoteproc0: kicking vq index: 0
[ 761.679572] imx-rproc imx8mp-cm7: imx_rproc_kick: failed (0, err:-62) //mailbox is not available
[ 761.686034] virtio_rpmsg_bus virtio0: rpmsg host is online
[ 761.692799] remoteproc0#vdev0buffer: lihual,registered virtio0 (type 7)
[ 761.699546] remoteproc remoteproc0: remote processor imx-rproc is now up

[LihuaLiu123]

I tried to understand how u-boot boot up cm7.

Following is the uboot code of bootaux.

https://source.codeaurora.org/external/imx/uboot-imx/tree/arch/arm/mach-imx/imx_bootaux.c?h=imx_v202...

Actually, it just copy stack and pc address to MCU_BOOTROM_BASE_ADDR(0x7e0000) and call SMCC API. Then i try to do the same things in driver.

void __iomem *io_tcm_addr = ioremap(0x7e0000, 8);

writel(stack, io_tcm_addr);

writel(pc, io_tcm_addr + 4);

However, it didn't bring any lucky, system hang happened when writting the stack addr to tcm addr, it looks like the same result as loading fw failed with TCM build since they both do the write operation to the address 0x7e0000.

There're more reason to belive that A core can not access the TCM area at the time. 

Since uboot has the permisson to access TCM, i did test below:

I tried to boot cm7 with uboot, and stop it via below commands when kernel boot up.

echo stop > /sys/class/remoteproc/remoteproc0/state

and then start it via

echo hello_world.elf > /sys/class/remoteproc/remoteproc0/firmware

echo start > /sys/class/remoteproc/remoteproc0/state

As as result, CM7 is booted up successfully and running.

i belive that the kick command works because the stack and pc are already written to 0x7e0000 on uboot phase.

Booting up cm7 is not my purpose, i'm going to move forward with this workaround. However, if anyone of you know the detail, pls help comment. 

#ifndef CONFIG_IMX8
int arch_auxiliary_core_up(u32 core_id, ulong addr)
{
	u32 stack, pc;

	if (!addr)
		return -EINVAL;

#ifdef CONFIG_IMX8M
	stack = *(u32 *)addr;
	pc = *(u32 *)(addr + 4);
#else
	/*
	 * handling ELF64 binaries
	 * isn't supported yet.
	 */
	if (valid_elf_image(addr)) {
		stack = 0x0;
		pc = load_elf_image_phdr(addr);
		if (!pc)
			return CMD_RET_FAILURE;

	} else {
		/*
		 * Assume binary file with vector table at the beginning.
		 * Cortex-M4 vector tables start with the stack pointer (SP)
		 * and reset vector (initial PC).
		 */
		stack = *(u32 *)addr;
		pc = *(u32 *)(addr + 4);
	}
#endif
	printf("## Starting auxiliary core stack = 0x%08X, pc = 0x%08X...\n",
	       stack, pc);

	/* Set the stack and pc to MCU bootROM */
	writel(stack, MCU_BOOTROM_BASE_ADDR);
	writel(pc, MCU_BOOTROM_BASE_ADDR + 4);

	flush_dcache_all();

	/* Enable MCU */
#ifdef CONFIG_IMX8M
	call_imx_sip(IMX_SIP_SRC, IMX_SIP_SRC_MCU_START, 0, 0, 0);
#else
	clrsetbits_le32(SRC_BASE_ADDR + SRC_M4_REG_OFFSET,
			SRC_M4C_NON_SCLR_RST_MASK, SRC_M4_ENABLE_MASK);
#endif

	return 0;
}