Enable UEFI Firmware BIOS and Debug Xen/EFI on VMware Player

Since VMware product (e.g. Player) now supports nested virtualization VT-x/VMX , recently I wanted to use this feature to debug XEN w/ EFI on top of VMware Player. This post details some best practices, and some issues I encountered. 

Enable emulated UEFI BIOS firmware
When you're creating a new virtual machine in VMware Player, by default the emulated firmware is legacy BIOS. In order to enable UEFI BIOS firmware for that virtual machine, you need to do a tricky thing: 

Use an Editor to open the corresponding *.vmx file (for example, Ubuntu-64-bit-uefi.vmx in my setting) for that virtual machine you created, then append a line as below in that file. 

firmware="efi"

Relaunch that virtual machine after saving and closing the vmx configuration file, then you can see the EFI BIOS will be emulated by VMware Player, and you can also enter UEFI Shell Command line. See below snapshots

Virtualization features nested virtualized by VMware
From my observation, most of key virtualization features are nested virtualized by VMware, for example as below.

• EPT or NPT, hardware assisted paging (or stage 2 translation MMU).
• Unrestricted guest, allow guest software natively to execute code in real mode or page-disabled protected mode.
• VPID, Virtual-Processor Identifiers for TLB tagging (better performance)
•  Virtual NMI
• APIC TPR shadow

However, not all the virtualization features are nest-supported by VMware Player. For example, I just found that "Load/Store PAT MSR" is not virtualized, as a matter of fact, reading PAT MSR even returns ZERO in the VMware guest OS. I didn't check the CPUID (virtualized by VMWare) to detect PAT MSR availability, but I think it should be reported as "non-supported" to VMware guest OS. 

Issues when running XEN on top of VMware
There are two issues. The first one below might not be an issue because it is not a correct way to boot XEN on EFI machine with Grub.

1. Currently in legacy BIOS firmware, Grub can launch XEN with Multiboot1 boot standard/protocol, however, the Grub/UEFI cannot launch xen image directly with Multiboot1 (see my old post for Multiboot specification). The versions in my test environment are: XEN 4.4 + Ubuntu x64 + Kernel 3.13 + Grub/UEFI 2.02.
   
   In fact, Multiboot version 1 cannot be used to boot UEFI OS image, Multiboot version 2 can support it with some extension by adding an UEFI tag. However, even using Multiboot 2 protocol for boot XEN on Grub/EFI is also not a good option.
   
   So, now XEN community uses another solution to boot XEN on UEFI firmware platform. An UEFI application (e.g. xen-xxx.efi) is built as an "OS loader" that can be directly launched by UEFI Boot Manager (see above snapshot). Please refer to this link for details if you want to know more.
   .
2. Xen can only use the Shadow Page Tables (software MMU virtualization) for its guest OS. So we cannot boot a HVM (hardware virtualized machine) guest, instead we can only boot a PV guest (Paravirtualized guest).
   
   We know that VMware provides nested EPT (extended page tables) for the VMM software (XEN) that runs on top of VMware. However due to an issue in XEN, it cannot be used. See the code snippet below start_vmx() for the reasons.
   const struct hvm_function_table * __init start_vmx(void)
   {
       ...

        /*
        * Do not enable EPT when (!cpu_has_vmx_pat), to prevent 

        * security hole (refer to 

        * http://xenbits.xen.org/xsa/advisory-60.html).
        */
        if ( cpu_has_vmx_ept && cpu_has_vmx_pat )
        {
             vmx_function_table.hap_supported = 1;

             ...
     
             setup_ept_dump();
        }
 
        ...

        if ( cpu_has_vmx_ept
            && cpu_has_vmx_pat
            && cpu_has_vmx_msr_bitmap
            && cpu_has_vmx_secondary_exec_control )        

           vmx_function_table.pvh_supported = 1;

        ...
    } 

From this code, we can see that it requires that both EPT and PAT are available, unfortunately PAT is not supported by VMware as mentioned above.  

For the issue 2 above, maybe we can try to work around this issue by removing PAT check for enabling EPT. After all, this is just to set up the development environment for easily debugging XEN, not for a product.