Thoughts on VMXON and VMCS regions in VT-x (from security's point of view)

Previously when I was working on a Hypervisor based on Intel VT-x. I always did exactly as what I'm told to do in the VT-x specification when programing VMM software. 
But now I'm going to revisit this again starting by asking these questions: What if I won't do it as what we're told to do? Is Intel VT-x exploitable? Can we find vulnerabilities in hardware VT-x implementation? Is it possible to do "reverse-engineering" to get internals on VT-x CPU implementation? Can we take advantage of VT-x to attack other privileged resource? Can we bypass EPT layer?  However, those are just some open questions, I don't have answers either, but it has a lot of fun when you look into it.

VMXON Region

If you take a look at the Intel SDM (Software Development Manual), you can see some words about VMXON region like below:

"Before executing VMXON, software allocates a region of memory (called the VMXON region) that the logical processor uses to support VMX operation..... Software should use a separate region for each logical processor and should not access or modify the VMXON region of a logical processor between execution of VMXON and VMXOFF on that logical processor. Doing otherwise may lead to unpredictable behaviors"

"Before entering VMX operation, the host VMM allocates a VMXON region. A VMM can host several virtual machines and have many VMCSs active under its management. A unique VMCS region is required for each virtual machine; a VMXON region is required for the VMM itself."

When you are reading these texts, probably some questions will come out in your mind: What're unpredictable behaviors if we modify the content of VMXON region? How is the VMXON region layout organized? 

We know that basically in the IA32 specification, almost all the operations/instructions have the corresponding defined behaviors even if they are the exceptions. Why are there so many undefined behaviors for VT-x specification, can we inspect the "so-called undefined behaviors" if we modify VMXON region during the period of VMX operation?

According to the VT-x specification, we also are told that the format of VMXON region is not architecturally defined, meaning that its format varies from processor generations to generations. Therefore, can we assume that "Intel says theire behaviors are unpredictable just because the format is different between VT-capable processors"? In other words, for a particular processor, the behaviors of read/write access to the VMXON region might be well-defined to some extent. If this happens to be true, then the question now is how to explore the internal structure/format of VMXON region? What are we able to do if we can exactly know the format for a particular processor? Are we able to control the VMX operation behaviors by simply programming the VMXON region with ordinary read/write access operation instructions like MOV? 

I don't know the answers. But I will talk about a prototype later about how to inspect the internal format/layout of VMXON region and other VT-x regions with a software solution. 

VMCS Regions

Now let's take a look at VMCS structure. 

"Software should use the VMREAD and VMWRITE instructions to access the different fields in the current VMCS. Software should never access or modifythe VMCS data of an active VMCS using ordinary memory operations, in part because the format used to store the VMCS data is implementation-specific and not architecturally defined, and also because a logical processor may maintain some VMCS data of an active VMCS on the processor and not in the VMCS region (then where?). The following items detail some of the hazards of accessing VMCS data using ordinary memory operations:

• Any data read from a VMCS with an ordinary memory read does not reliably reflect the state of the VMCS. Results may vary from time to time or from logical processor to logical processor.

• Writing to a VMCS with an ordinary memory write is not guaranteed to have a deterministic effect on the VMCS. Doing so may cause the VMCS to become corrupted (see below).

(Software can avoid these hazards by removing any linear-address mappings to a VMCS region before executing a VMPTRLD for that region and by not remapping it until after executing VMCLEAR for that region.)
This section has identified operations that may cause a VMCS to become corrupted. These operations may cause the VMCS’s data to become undefined. Behavior may be unpredictable if that VMCS used subsequently on any logical processor. The following items detail some hazards of VMCS corruption: 

• VM entries may fail for unexplained reasons or may load undesired processor state. 

• The processor may not correctly support VMX non-root operation as documented and may generate unexpected VM exits. 

• VM exits may load undesired processor state, save incorrect state into the VMCS, or cause the logical processor to transition to a shutdown state."

So, we know that the behaviors of VMX operations (root mode and non-root mode) are also controlled by VMCS regions, and the format/layout of those regions are also undefined, and read/write access to them with ordinary memory operations are unpredictable too.

How to do 'reverse-engineering'-like things to look into the internal VMXON/VMCS region memory layout

Here is an idea (software solution, without any hardware device but a particular VT-capable processor):

• Implement a tiny VMM/Hypervisor and a tiny Guest VM software, both with only code running in ring 0 mode. No need complex things like thread, multiple processor, scheduling, interrupt/exceptions, etc.. just a piece of code that can execute in VMX root mode and non-root mode respectively. 
• Hypervisor allocates machine physical memory spaces for VMXON and VMCS region, and make these areas visible to guest VM software so that the guest software can read/write the VMXON/VMCS memory regions directly to inspect the content. 
• After environment is setup, we can do inspections instruction by instruction or event by event.. for example, if we want to know what memory bits are changed during a vmentry, then we can do it like this:
  (1)  In VMX-root mode, the VMM software logs all the contents of VMCS/VMXON regions right prior to calling VMRESUME; (2)  Just right after resuming back to guest, the VM software logs all the contents of VMCS/VMXON regions again, then compares the differences between this two points.
  (3)  The difference in some of memory areas might indicate something important that can control the behaviors of VMX operations. 
• Repeatedly do above things again and again for all kinds of VMX transition events. 
• Besides, for VMCS regions, we can read the contents before and after a legal VMCS write instruction, for example, VMCS_WRITE(GUEST_RIP) to get what has been changed for GUEST_RIP area.

I didn't do this, but I think this must be fun once you can get the VT-x internals. 


However, even though we could get the internal VMCS/VMXON data structure layout, what can we do? Imaging that there is a bug in a Hypervisor (XEN, KVM...), e.g. a buffer/stack overflow, that can lead to arbitrary memory overwriting... see below...:-)



One more question:

• How does the processor determine if the current processor execution mode is in VMX root mode or non-root mode? Does it look at some certain VMX mode state bit in VMXON region or VMCS regions before executing any instruction? This is unknown to us. But if my guess is true (hope not), then we can change that corresponding bit to control instruction behavior. For example, provided that there is a vulnerability in Hypervisor that can lead us overwrite arbitrary memory including VMCS/VMXON region, we can take advantage of it to change "such a bit", and then any malicious code that is executed in VMX non-root mode will be treated as execution in VMX root mode... this is horrible, since for example we can bypass EPT to directly write machine physical memory space even in VMX non-root mode.
  < Note that, however, for other process modes like protected mode and real mode, we can easily check CR0.PE bit, for different privilege modes (ring), we can check CPL bit of CS selector to determine ring 0~3 >

<The End>