How to Set up Embedded Hypervisor Topaz

[amittomar]

I have started using SDK1.6 and wanted to play around the Embedded Hyper visor "Topaz".

I compiled minimal image using bit-bake but could not see any Embedded Hyper visor source compiled with it.

Can anyone help me setting up the Embedded Hyper visor ,is it required to have separate hv DTS file?

Thanks

Amit

[yipingwang]

Hello Amit Tomar,

You need hv.dtb and hv.uImage, it is better to download the pre-built images ISO from the public Website, which contains all the images used for hypervisor.

Please refer to the attached document about hypervisor deployment, for Nor Flash memory map please refer to SDK documents.

You could find more information from Freescale Technical Information Center.

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[amittomar]

Thanks Yiping for quick response.

But is it must to flash these hv images,can't we do tftp to "ram" location and boot it up?

Thanks

Amit.

[yipingwang]

Hello Amit,

It is possible but you need to modify the default hv.dtb file.

First please modify u-boot environment(config-addr), here are physical addresses.

=>setenv bootargs config-addr=0xfe8900000 console=ttyS0,115200

=>setenv bootcmd 'bootm 0xfe8700000 - 0xfe8800000'

=>saveenv

Then you need to modify hv.dts provided in hv-cfg package, please refer to the following

	part1 {
	// Indicates that it is a partition node
	compatible = "partition";
	label = "p1-linux";

	// CPUs #0 to #2 are assigned to this partition
	cpus = <0 3>;
	guest-image = <0xf 0xe8020000 0 0 0 0x700000>;
	linux-rootfs = <0xf 0xe9300000 0 0x01300000 0 0x02800000>;
	dtb-window = <0 0x1000000 0 0x20000>;

	// =====================================================
	part2 {
	compatible = "partition";
	label = "p2-linux";
	// CPU #3 is assigned to this partition
	cpus = <3 1>;
	guest-image = <0xf 0xe8020000 0 0 0 0x700000>;
	linux-rootfs = <0xf 0xe9300000 0 0x01300000 0 0x02800000>;
	dtb-window = <0 0x1000000 0 0x20000>;

There is a thread would be helpful P2041RDB can't boot from RAM with hypervisor.

If you need further assistance, we could discuss more later.

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[amittomar]

Hi Yiping

Couple of little doubts regarding config-addr=0xfe8900000 and hv.dtb while booting from RAM.

1. What should be the config-addr,do we need to provide same config-addr=0xfe8900000 in case of booting from RAM?
2. Again,for guest-image(0xfe802000),linux-rootfs(0xe9300000) an dtb-window(0x1000000) do we need to give same address as mentioned in hv.dtb you provided?

     Or do we need to choose the addresses according to RAM, where we place these guest-images in RAM using tftp?

Hope not taking much of you time.

Thanks,

Amit.

[yipingwang]

Hello Amit,

Except Scott's comments, I add more explanation for you.

In the default configuration Hypervisor images are deployed on Nor Flash, if you want to use RAM boot, you need to change these images addresses according to your RAM deployment, please refer to the attachment I sent to you, it should be helpful a little bit.

config-addr is HV dtb address

setenv bootcmd 'bootm 0xfe8700000 - 0xfe8800000' here are address of hv.uImage and Kernel dtb.

"guest-image" is Kernel uImage address.

"linux-rootfs" is ramdisk filesystem.

In addition, you could use the following method to modify hv.dts and rebuild hv.dtb again.

1. Enter into bitbake hv-cfg -c patch -f.

2. Enter into build_<platform>_release/tmp/work/t4240qds-fsl_networking-linux/hv-cfg/git-r7/git/ and modify the dts file.

3. Rebuild dtb file

bitbake hv-cfg -c compile -f

bitbake hv-cfg

Get images in build_<platform>_release/tmp/deploy/images/hv-cfg

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[amittomar]

Thanks Yiping for your time and help on this.

I did the following as you suggested.

##############################################

setenv bootargs config-addr=0x20000666 console=ttyS0,115200

tftp 1000000 uImage

tftp 2000000 rootfs

tftp c00000 board FDT

tftp 20000666 hv.dtb

tftp 22a6a266 hv.uImage

bootm 1000000 2000000 c00000 20000666 22a6a266

###################################

hv.uImage did get running "bitbake hypervisor"

Now Host kernel is booted up(t4240rdb) but could not find Hypervisor as I see below message in dmesg logs

Freescale hypervisor management driver 

fsl-hv: no hypervisor found

Also in boot logs I did not see below messages

[0] Freescale Hypervisor 3.0-001

[0] Hypervisor command line: config-addr=0x20000666 console=ttyS0,115200

Below is what I provided in hv.dts file

part1 {

                // Indicates that it is a partition node

                compatible = "partition";

                label = "p1-linux";

                // CPUs #0 to #22 are assigned to this partition

                cpus = <0 23>;

                guest-image = <0xf 0x22a72866 0 0 0 0x700000>;

                linux-rootfs = <0xf 0x2006a066 0 0x01300000 0 0x02800000>;

                dtb-window = <0 0x1000000 0 0x20000>;

part2 {

                compatible = "partition";

                label = "p2-linux";

                // CPU #23 is assigned to this partition

                cpus = <23 1>;

                guest-image = <0xf 0x22a72866 0 0 0 0x700000>;

                linux-rootfs = <0xf 0x2006a066 0 0x01300000 0 0x02800000>;

                dtb-window = <0 0x1000000 0 0x20000>;

Would you please let me know what I'm doing wrong here?

Also,what is the way to confirm that hypervisor is booted up well?

Thanks

Amit.

[yipingwang]

Hello Amit,

I assume that you want to deploy images at  memory after 0x20000000.

1. hv.dts

part1 {

                // Indicates that it is a partition node

                compatible = "partition";

                label = "p1-linux";

                // CPUs #0 to #22 are assigned to this partition

                cpus = <0 23>;

                guest-image = <0x0 0x22a72866 0 0 0 0x700000>;

                linux-rootfs = <0x0 0x2006a066 0 0x01300000 0 0x02800000>;

                dtb-window = <0 0x1000000 0 0x20000>;

part2 {

                compatible = "partition";

                label = "p2-linux";

                // CPU #23 is assigned to this partition

                cpus = <23 1>;

                guest-image = <0x0 0x22a72866 0 0 0 0x700000>;

                linux-rootfs = <0x0 0x2006a066 0 0x01300000 0 0x02800000>;

                dtb-window = <0 0x1000000 0 0x20000>;

2. Deploy images

setenv bootargs config-addr=0x20000666 console=ttyS0,115200

tftp 0x22a72866 uImage

tftp 0x2006a066 rootfs

tftp 0x20c00000 board FDT

tftp 20000666 hv.dtb

tftp 22a6a266 hv.uImage

bootm 0x22a6a266 - 0x20c00000

Here only a example for you, you could adjust addresses according to your requirement.

3. About how to setup hypervisor consoles

Please refer to the beginning of the document I attached for you, you need to use "mux_server" to setup there consoles for hv and two guests.

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[amittomar]

Hello Yiping

I though I was close booting up the Hypervisor until I see following kernel crash

WARNING: adjusting available memory to 30000000

## Booting kernel from Legacy Image at 22a6a266 ...

   Image Name:  

   Created:      2014-08-25   8:14:32 UTC

   Image Type:   PowerPC Linux Kernel Image (uncompressed)

   Data Size:    426752 Bytes = 416.8 KiB

   Load Address: 00000000

   Entry Point:  00000000

   Verifying Checksum ... OK

## Flattened Device Tree blob at 20c00000

   Booting using the fdt blob at 0x20c00000

   Loading Kernel Image ... OK

OK

   Loading Device Tree to 03fde000, end 03fffc40 ... OK

[0] =======================================

[0] Freescale Hypervisor 1.3-009

[0] Hypervisor command line: config-addr=0x20000666 console=ttyS0,115200

[0] malloc_init: using 14348 KiB at 0x7f130150 - 0x7ff32fff

[0] malloc_init: using 812 KiB at 0x7ff34000 - 0x7fffefff

[0] dt_read_aliases: Alias pci1 points to non-existent /pcie@ffe250000

[0] assign_callback: device buffer-pool@7 in buffer-pool@7 not found

[0] assign_callback: device buffer-pool@8 in buffer-pool@8 not found

[0] assign_callback: device buffer-pool@9 in buffer-pool@9 not found

[0] assign_callback: device buffer-pool@16 in buffer-pool@16 not found

[0] assign_callback: device buffer-pool@17 in buffer-pool@17 not found

[0] assign_callback: device /fsl,dpaa/ethernet@16 in dpa-ethernet@16 not found

[0] assign_callback: device /fsl,dpaa/dpa-fman0-oh@1 in dpa-fman0-oh@1 not found

HV>                                                                                                                                  

[0] Load error report

[0] powerpc_mchk_interrupt: machine check interrupt!

[0] mcheck registers: mcsr = 8000, mcssr0 = 4060b0, mcsrr1 = 29002

[0] machine check

[0] NIP 0x004060b0 MSR 0x00029002 LR 0x00421c4c ESR 0x00800000 EXC 1

CTR 0x00000010 CR 0x40004024 XER 0x00000000 DEAR 0x14040080 PIR 0

guest-image in hv.dts signifies the kernel image booting up inside guest is same as board's uImage?

Also,could not see below message at boot time

[0] loading binary image from 0x2006a066 to 0x1300000

[0] loading binary image from 0x22a72866 to 0

Thanks,

Amit.

[yipingwang · Accepted solution]

Hello Amit,

I do some verification on my target and finally boot hv from RAM successfully, please refer to the following.

1. Please note in hv.dts there is a node used form boot from RAM, so please put images inside these range.

	images_pma: images_pma {
	compatible = "phys-mem-area";
	addr = <0x0 0x78000000>;	// Used for boot-from-RAM
	size = <0x0 0x04000000>;	// 64MB
	};

2. Modify address parts in hv.dts

  part1 {

                // Indicates that it is a partition node

                compatible = "partition";

                label = "p1-linux";

                // CPUs #0 to #22 are assigned to this partition

                cpus = <0 23>;

                guest-image = <0x0 0x78020000 0 0 0 0x700000>;

                linux-rootfs = <0x0 0x79300000 0 0x01300000 0 0x02800000>;

                dtb-window = <0 0x1000000 0 0x20000>;

                ... ...

	part2 {
	compatible = "partition";
	label = "p2-linux";
	// CPU #23 is assigned to this partition
	cpus = <23 1>;
	guest-image = <0x0 0x78020000 0 0 0 0x700000>;
	linux-rootfs = <0x0 0x79300000 0 0x01300000 0 0x02800000>;
	dtb-window = <0 0x1000000 0 0x20000>;

3. On the target board

tftp 78020000 wyp/t4240/uImage-t4240rdb.bin

tftp 79300000 wyp/t4240/fsl-image-core-t4240rdb.ext2.gz.u-boot

tftp 78700000 wyp/t4240/hv.uImage

tftp 0x78800000 wyp/t4240/uImage-t4240rdb-usdpaa.dtb

tftp 0x78900000 wyp/t4240/hv.dtb

=>setenv bootargs config-addr=0x78900000 console= ttyS0,115200

=>bootm 0x78700000 - 0x78800000

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[amittomar]

Thanks Yiping for taking out your time on this.

But still,I'm out of luck and seeing same kernel crash

=> setenv bootargs config-addr=0x78900000 console= ttyS0,115200

=> bootm 0x78700000 - 0x78800000

WARNING: adjusting available memory to 30000000

## Booting kernel from Legacy Image at 78700000 ...

   Image Name:  

   Created:      2014-08-25   8:14:32 UTC

   Image Type:   PowerPC Linux Kernel Image (uncompressed)

   Data Size:    426752 Bytes = 416.8 KiB

   Load Address: 00000000

   Entry Point:  00000000

   Verifying Checksum ... OK

## Flattened Device Tree blob at 78800000

   Booting using the fdt blob at 0x78800000

   Loading Kernel Image ... OK

OK

   Loading Device Tree to 03fde000, end 03fffc40 ... OK

[0] =======================================

[0] Freescale Hypervisor 1.3-009

[0] Hypervisor command line: config-addr=0x78900000 console= ttyS0,115200

[0] malloc_init: using 14348 KiB at 0x7f130150 - 0x7ff32fff

[0] malloc_init: using 812 KiB at 0x7ff34000 - 0x7fffefff

[0] dt_read_aliases: Alias pci1 points to non-existent /pcie@ffe250000

[0] assign_callback: device buffer-pool@7 in buffer-pool@7 not found

[0] assign_callback: device buffer-pool@8 in buffer-pool@8 not found

[0] assign_callback: device buffer-pool@9 in buffer-pool@9 not found

[0] assign_callback: device buffer-pool@16 in buffer-pool@16 not found

[0] assign_callback: device buffer-pool@17 in buffer-pool@17 not found

[0] assign_callback: device /fsl,dpaa/ethernet@16 in dpa-ethernet@16 not found

[0] assign_callback: device /fsl,dpaa/dpa-fman0-oh@1 in dpa-fman0-oh@1 not found

HV>                                                                                                                                  

[0] Load error report

[0] powerpc_mchk_interrupt: machine check interrupt!

[0] mcheck registers: mcsr = 8000, mcssr0 = 406050, mcsrr1 = 29002

[0] machine check

[0] NIP 0x00406050 MSR 0x00029002 LR 0x00421c4c ESR 0x00800000 EXC 1

CTR 0x00000004 CR 0x20004024 XER 0x00000000 DEAR 0x14040080 PIR 0

Prev trap level 0

Do we need to remove pma0 node from hv.dts as per P2041RDB can't boot from RAM with hypervisor

Any things else we need to configure?

Thanks ,

Amit

[yipingwang]

Hello Amit,

I have verified the above procedure on my T4240RDB without any problem, it should work for you.

Do you use T4240RDB board?

Do you use the images provide in the pre_built images ISO except hv.dtb? And also SDK 1.6 u-boot?

Are you sure that hv.dts file is got rebuilt?

I attached my hv.dtb file, please try to use it on your target.

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[amittomar]

Hello Yiping

I just checked my U-BOOT is from U-Boot 2013.01QorIQ-SDK-V1.5,would it make any difference?

Thanks

Amit.

[yipingwang]

Hello Amit,

I am not very sure, it's better to use all the images from SDK 1.6.

Please try whether my hv.dtb could make some difference on your target.

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[amittomar]

Oh yes Yiping ,your hv.dtb made a difference on my target.

Not sure why my hv.dtb gives a trouble ,verified it quite a times on U-boot by dumping it.

Now I'm on HV prompt ,would you let me know what needs to be done next?

HV>

Also,mux_servers need to be set up at U-boot prompt ,right?

Thanks

Amit.

[yipingwang]

Hello Amit,

If you only use the normal console(without mux_servers), you will see hv boot first, then guest1 Kernel booting and geuset2 Kernel booting, and boot logs are mixed together.

So you need to setup 3 consoles to see boot consoles separately.

For any additional assistance, please feel free to let me know.

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[amittomar]

Thanks Yiping for your valuable help.

I think I still didn't get the exact behaviour here.

In your last comment you said.

"If you only use the normal console(without mux_servers), you will see hv boot first, then guest1 Kernel booting and geuset2 Kernel booting, and boot logs are mixed together."

With the normal screen, I can only see hv boots and comes to hv prompt ,guest1 Kernel and guest2 Kernel doesn't boot up for me.

So as of now ,there is no question for mixed logs for me

From hv prompt I tried booting guest1 kernel running

HV> start load p1-linux

Couldn't start partition.

Also I tried set up mux_server and after launching it from host machine I again comes to HV prompt,I have two HV prompt running now

./mux_server -s  115200 /dev/ttyS0 15000 & socat -,raw,echo=0 tcp:0:15000

Thanks,

Amit

[yipingwang]

Hello Amit,

Please use these commands as the following.

mux_server "/dev/ttyS0" 15000  15001 15002&

Setup hv console:

socat -,raw,echo=0 tcp:0:15000

Setup guest 1:

socat -,raw,echo=0 tcp:0:15001

Setup guest 2:

socat -,raw,echo=0 tcp:0:15002

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[amittomar]

Hello Yiping

As per your suggestion, I set up the mux server.

But only at hv console,could see response and able to execute Hypervisor commands on "socat -,raw,echo=0 tcp:0:15000" hv console.

Can't see guest1 kernel and guest2 kernel booted up on guest1 console and guest2 console.

Both the socat -,raw,echo=0 tcp:0:15001 and socat -,raw,echo=0 tcp:0:15002 are just waiting without any output

Thanks

Amit.

[yipingwang]

Hello Amit,

I used boardfarm board, and used the command

mux_server -exec "bft connect T4240RDB-3" 15000 15001 15002 &

Please try whether you could use the following similar command to connect your target board.

mux_server -exec "kermit -c /dev/ttyS0" 15000 15001 15002 &

Then use socat to connect to the guest consoles, please make sure mux_server is running at the background.

If your problem remains, would you please provide you boot and setup log?

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[amittomar]

Hello Yiping

I did try following command but still facing the issue.

sudo ./mux_server -exec "kermit -c /dev/ttyS0" 15000 15001 15002&

[1] 23592

user@user-OptiPlex-7010:~$ ./mux_server: IO Channel 0 listening on port 15000 for stream 0.

./mux_server: IO Channel 1 listening on port 15001 for stream 1.

./mux_server: IO Channel 2 listening on port 15002 for stream 2.

./mux_server: using speed: 115200.

./mux_server: EOF on read from target.

It seems like mux_server is not started with it.

Thanks

Amit.