Xen port to Colibri IMX8X

[dwagner]

Hello,

I am trying to use the XEN Hypervisor on a Toradex Colibri iMX8X, which features a i.MX8QXP.

My approach is to use the linux-toradex kernel from Toradex BSPs (Details below), made a new Devicetree-File (fsl-imx8qxp-colibri-eval-v3-dom0.dtb) similar to the imx8qm-mek-dom0.dtb and use the same U-Boot Boot-script colibri xenmmcboot - Pastebin.com.

Without Xen the Image boots just fine with the fsl-imx8qxp-colibri-eval-v3-dom0.dtb.

As xen-binary I tried both the imx-xen from your BSPs and the xen-colibri-imx8x from OE/meta-virtualisation and got different resulting errors.

imx-xen

(XEN) *** LOADING DOMAIN 0 ***
(XEN) Missing kernel boot module?
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Could not set up DOM0 guest OS
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

complete boot log: xen boot log - Pastebin.com 

xen-colibri-imx8x

[ 0.098234] Xen: initializing cpu0
[ 0.106238] Hierarchical SRCU implementation.
[ 0.119764] Error FSL_HVC_SC -1
[ 0.122594] Cannot request PMIC0_TEMP interrupt
[ 0.127125] Error FSL_HVC_SC -1
[ 0.130206] Cannot request PMIC1_TEMP interrupt
[ 0.134759] Error FSL_HVC_SC -1‍‍‍‍‍‍‍‍‍‍‍‍‍‍

looks like Xen is running, but at some point "Error FSL_HVC_SC -1" starts appearing. Sadly I couldn't get the (XEN) outputs to show up.

complete boot log: xen-colibri-imx8x boot log - Pastebin.com 

Any suggestions on how to fix/workaround either of both issues?

I would greatly appreciate your inputs!

Setup:

imx-xen

I have build Xen as described in the current https://www.nxp.com/docs/en/user-guide/IMX_YOCTO_PROJECT_USERS_GUIDE.pdf (Rev. L5.4.24_2.1.0, 06/2020)

/imx-yocto-bsp$ MACHINE=imx8qxpmek DISTRO=fsl-imx-wayland source imx-setup-release.sh -b build_imx8/
/imx-yocto-bsp/build_imx8$ bitbake xen‍‍‍‍

Dom0

To build Dom0 I use the linux-toradex kernel image provided by toradex OE repo as described here.

Added layer meta-virtualization  = "zeus:1bb515609594215aaf950555211e18c6c9a8e308" to Toradex OE and these to local.conf:

DISTRO_FEATURES_append = " xen virtualization"
IMAGE_INSTALL_append = " xen-base"‍‍‍‍

[Bio_TICFSL]

Hello Daniel,

Looks like toradex missing the PMIC or the PMIC is not on the board, the kernel from NXP is for the EVK is not going to work with toradex board.

Regards

[dwagner]

Hello,

thanks for your answer. 
I'm not using NXP's linux-kernel for the EVK, but the one that I built with the Toradex-BSPs, which are based on FSL-/NXP-BSPs. And this kernel is indeed working when booted without the hypervisor. 

Also when booted without hypervisor none of the "Cannot request ___ interrupt" Errors is printed. On top of that I'm sure Toradex has a PMIC on their board, as I have worked with that in the past.

The only thing I am currently using directly from the NXP-BSPs (as described in the Yocto Project Users guide) is the xen binary.

best regards

Daniel

[Bio_TICFSL]

Hi, Let´s see is someone of toradex‌ can help.

Regards

[dwagner]

Hello,

i was hoping to reach an imx-xen developer or someone that got the Xen part of the i.MX Yocto Project User's Guide running on an board other that the MEK. I can't find any information on what steps are generally nessecary to implement it on an imx-plattform other than in this doc which is specific to the MEK and not much detail is given.

Are the imx-xen developers at nxp(freescale)?

[Peng_Fan]

For dom0 dts, did you add something as below?

"

chosen {
#address-cells = <2>;
#size-cells = <2>;

stdout-path = &lpuart0;

/* Could be updated by U-Boot */
module@0 {
bootargs = "earlycon=xen console=hvc0 loglevel=8 root=/dev/mmcblk1p2 rw rootwait";
compatible = "xen,linux-zimage", "xen,multiboot-module";
reg = <0x00000000 0x80a00000 0x00000000 0xf93a00>;
};
};

"

Please also check uboot "run xenmmcboot" or "run xennetboot" to see how it works.

BTW: what usecase would you use xen on 8QX?

[dwagner]

Yes, we have added the following to the dts, which is based on fsl-imx8qxp-mek-dom0.dts - Pastebin.com. 

chosen {
    #address-cells = <2>;
    #size-cells = <2>;
    module@0 {
          bootargs = "earlycon=xen console=hvc0 loglevel=8 root=/dev/mmcblk0p2 rootwait $
          xen,dom0-bootargs = "console=dtuart dtuart=ttyLP3 earlycon=lpuart32,0x5a090000$
          compatible = "xen,linux-zimage", "xen,multiboot-module";
          /* The size will be override by uboot command */
          reg = <0x00000000 0x80a00000 0x00000000 0xf93a00>;
    };
};
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Additionally to the above, we also adopted all other nodes from mek-dom0.dts to our dts (fsl-imx8qxp-colibri-eval-v3-dom0.dts - Pastebin.com).

I have dissected the xenmmcboot script and to my understanding xenboot_common, which is called by xenmmcboot, also modifies the dtb via fdt set commands.

baudrate=115200
dom0fdt_file=fsl-imx8qxp-colibri-eval-v3-dom0.dtb
fdt_addr=0x83000000
image=Image
loadaddr=0x80280000
mmcdev=0
mmcpart=1
mmcroot=/dev/mmcblk0p2 rootwait rw
xenhyper_bootargs=console=dtuart dtuart=ttyLP3,115200 dom0_mem=2048M dom0_max_vcpus=2 dom0_vcpus_pin=true
xenlinux_addr=0x92000000
xenlinux_console=hvc0 earlycon=xen
 
xenmmcboot=
    setenv get_cmd "fatload mmc ${mmcdev}:${mmcpart}";
    setenv console ${xenlinux_console};
    run mmcargs;
    run xenboot_common;
 
mmcargs=
    setenv bootargs console=${console},${baudrate} root=${mmcroot} video=imxdpufb5:off video=imxdpufb6:off video=imxdpufb7:off
 
xenboot_common=
    ${get_cmd} ${loadaddr} xen;
    ${get_cmd} ${fdt_addr} ${dom0fdt_file};
    ${get_cmd} ${xenlinux_addr} ${image};
    fdt addr ${fdt_addr};
    fdt resize 256;
    fdt set /chosen/module@0 reg <0x00000000 ${xenlinux_addr} 0x00000000 0x${filesize}>;
    fdt set /chosen/module@0 bootargs "${bootargs} ${xenlinux_bootargs}";
    setenv bootargs ${xenhyper_bootargs};
    booti ${loadaddr} - ${fdt_addr};
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Our usecase will be to segregate safety critical applications from those that aren't safety critical.

[jaskis]

Hello,

We never did something regarding Xen Supervisor in Toradex. Which Toradex branch did you use to compile the Colibri iMX8X Image.

Best regards, 

Jaski

[dwagner]

Hi,

we are using the 3.0 Version (-b LinuxImage3.0) of your repo to build the linux-kernel

Best regards,

Daniel

[jaskis]

Hi, Thanks for the answer. For Colibri iMX8X PMIC is controlled by the SIU Firmware. Do you have driver/API which make calls from Xen SuperVisor to the SIU Firmware?

Best regards, 

Jaski