Virtual address mapping during hypervisor initialization.

traviswheatley — Thu, 20 Jun 2013 16:05:00 GMT

I am trying to bring up hypervisor on a P4080PCIe board. During initialization the hypervisor code maps the physical address of the device trees and then reads the fdt header information to determine the size of the tree. I have verified that the physical addresses are being passed into the initialization function properly and have verified that the device trees have valid headers. However, after calling map_phys() the data at the resulting virtual address is not what is expected.

I could use some help in figuring out where to look for the root cause of physical to virtual address mapping issues.

Following is a more detailed description of the behavior along with code snippets and log information:

During boot time I am receiving the message: "panic: fdt too large" error". This message is found in the hypervisor code in a function called map_fdt located in the init.c file. This function reads the .fdt header in the device tree blob and uses it to calculate the total size of the device tree.

I can use u-boot to dump the fdt header information of both the hardware .dtb as well as the guest .dtb files and find them both to contain valid totalsize members. However, in the hypervisor map_fdt function these values are not being read correctly. I put some debug print statements in this function and have found that the physical addresses of the .dtb files are getting passed in properly. However, the function calls a "map_phys" function to get a virtual address for the device trees. This virtual address is then used as the location from which to read the fdt header info.

So... that tells me that the virtual address mapping is going awry. The obvious questions is what could cause this virtual addressing operation to go wrong? Could it be some initialization problem with u-boot?

--------------------------------------------------------------------------------------------

Here is the hypervisor function from init.c (with my debug print statements):

static void *map_fdt(phys_addr_t treephys)

{

const size_t mapsize = 4 * 1024 * 1024;

phys_addr_t mapaddr = treephys & ~((phys_addr_t)mapsize - 1);

size_t len = mapsize;

size_t total = 0;

void *vaddr;

printf("TRW - in %s: treephys = %x\n",__FUNCTION__,(unsigned int)treephys);

printf("TRW - in %s: mapaddr = %x\n",__FUNCTION__,(unsigned int)mapaddr);

printf("TRW - in %s: mapsize = %x\n",__FUNCTION__,(unsigned int)mapsize);

/* Map guarded, as we don't know where the end of memory is yet. */

vaddr = map_phys(TEMPTLB1, mapaddr, temp_mapping[0], &len,

TLB_TSIZE_4M, TLB_MAS2_MEM | MAS2_G, TLB_MAS3_KDATA);

map_phys(TEMPTLB2, mapaddr + mapsize, temp_mapping[0] + mapsize,

&len, TLB_TSIZE_4M, TLB_MAS2_MEM | MAS2_G, TLB_MAS3_KDATA);

vaddr += treephys & (mapsize - 1);

printf("ftd_totalsize(vaddr) = %x\n",fdt_totalsize(vaddr));

/* We handle trees that straddle one 4MiB boundary, but we

* don't support trees larger than 4MiB.

if (mapsize * 2 - (treephys & (mapsize - 1)) < fdt_totalsize(vaddr)) {

if (cpu->crashing)

return NULL;

panic("fdt too large: %zu bytes\n", fdt_totalsize(vaddr));

}

return vaddr;

}

-------------------------------------------------------------------------------------

Here is the header information for both the hardware and hypervisor device trees:

=> fdt addr e8800000

=> fdt header

magic: 0xd00dfeed

totalsize: 0x9694 (38548)

off_dt_struct: 0xb8

off_dt_strings: 0x90b0

off_mem_rsvmap: 0x28

version: 17

last_comp_version: 16

boot_cpuid_phys: 0x0

size_dt_strings: 0x5e4

size_dt_struct: 0x8ff8

number mem_rsv: 0x0

=> fdt addr e8900000

=> fdt header

magic: 0xd00dfeed

totalsize: 0xc92 (3218)

off_dt_struct: 0x38

off_dt_strings: 0xbbc

off_mem_rsvmap: 0x28

version: 17

last_comp_version: 16

boot_cpuid_phys: 0xfeedbeef

size_dt_strings: 0xd6

size_dt_struct: 0xb84

number mem_rsv: 0x0

----------------------------------------------------------------------------------------------------

And last but not least, here is the output of the boot process with the debug print statements showing things going awry in the map_fdt function:

=> setenv bootargs config-addr=0xe8900000

=> bootm e8700000 - e8800000

WARNING: adjusting available memory to 30000000

## Booting kernel from Legacy Image at e8700000 ...

Image Name:

Created: 2013-06-19 14:29:31 UTC

Image Type: PowerPC Linux Kernel Image (uncompressed)

Data Size: 420000 Bytes = 410.2 KiB

Load Address: 00000000

Entry Point: 00000000

Verifying Checksum ... OK

## Flattened Device Tree blob at e8800000

Booting using the fdt blob at 0xe8800000

Loading Kernel Image ... OK

Loading Device Tree to 00ff3000, end 00fff693 ... OK

=======================================

Freescale Hypervisor 0.8-004

TRW - in map_fdt: treephys = ff3000

TRW - in map_fdt: mapaddr = c00000

TRW - in map_fdt: mapsize = 400000

ftd_totalsize(vaddr) = b000

TRW - Hypervisor command line: config-addr=0xe8900000

TRW - calling init_hv_mem. cfg_addr = e8900000

TRW - Calling map_fdt. cfg_addr: e8900000

TRW - in map_fdt: treephys = e8900000

TRW - in map_fdt: mapaddr = e8800000

TRW - in map_fdt: mapsize = 400000

ftd_totalsize(vaddr) = fffffffe

panic: fdt too large: 4294967294 bytes

TRW - in map_fdt: treephys = ff3000

TRW - in map_fdt: mapaddr = c00000

TRW - in map_fdt: mapsize = 400000

ftd_totalsize(vaddr) = b000

Re: Virtual address mapping during hypervisor initialization.

scottwood — Wed, 26 Jun 2013 16:36:39 GMT

You need to pass in the physical address of the config tree, not the U-Boot effective address. If the address space is laid out like p4080ds, then the physical address is 0xfe8900000, not 0x0e8900000.

Re: Virtual address mapping during hypervisor initialization.

traviswheatley — Wed, 26 Jun 2013 17:04:45 GMT

Thanks Scott.

This does the trick. One needs to specify the complete 36 bit address for in both the config-addr item in bootargs as well as the addresses used in the boom command.

setenv bootargs=0xfe8900000

bootm 0xfe8700000 - 0xfe8800000

Ironically, I just learned this less than an hour before receiving this reply. :smileywink:

I'm still working through some issues booting the OS image in the primary partition on the P4080PCIe, but I am now successfully getting to a hypervisor prompt.

Re: Virtual address mapping during hypervisor initialization.

scottwood — Wed, 26 Jun 2013 17:14:55 GMT

This only applies to the config tree, since that address gets passed directly to the hypervisor via bootargs. The arguments to bootm are U-Boot effective addresses (if they weren't, the hypervisor would not have received control at all and thus you would not have seen any messages). Since effective addresses are 32-bit, the upper 0xf is probably just getting ignored.

Re: Virtual address mapping during hypervisor initialization.

traviswheatley — Fri, 28 Jun 2013 14:52:41 GMT

On a related note, do you happen to know where I can find the device tree source file hv-4p-lnx-lnx-lwe-lwe.dts?

The older .ltib BSP has an example device tree binary that does almost what I want, two linux partitions plus a few managed LWE partitions that I can launch binaries on. It would be a heck of a lot easier to create the configuration I want using the source for this .dtb as a starting point rather than having to build one from scratch.

Re: Virtual address mapping during hypervisor initialization.

scottwood — Fri, 28 Jun 2013 16:30:01 GMT

I don't think current SDKs support LWE. USDPAA has replaced it. If you're using an older SDK that does support LWE, I suggest posting a separate question with a title that will attract the attention of people that might know the answer.

topic Virtual address mapping during hypervisor initialization. in P-Series

Virtual address mapping during hypervisor initialization.

Re: Virtual address mapping during hypervisor initialization.

Re: Virtual address mapping during hypervisor initialization.

Re: Virtual address mapping during hypervisor initialization.

Re: Virtual address mapping during hypervisor initialization.

Re: Virtual address mapping during hypervisor initialization.