Virtual 8086 mode


In the 80386 microprocessor and later, virtual 8086 mode allows the execution of real mode applications that are incapable of running directly in protected mode while the processor is running a protected mode operating system. It is a hardware virtualization technique that allowed multiple 8086 processors to be emulated by the 386 chip; it emerged from the painful experiences with the 80286 protected mode, which by itself was not suitable to run concurrent real mode applications well.
VM86 mode uses a segmentation scheme identical to that of real mode which creates 20-bit linear addresses in the same manner as 20-bit physical addresses are created in real mode, but are subject to protected mode's memory paging mechanism.

Overview

The virtual 8086 mode is a mode for a protected-mode task. Consequently, the processor can switch between VM86 and non-VM86 tasks, enabling multitasking legacy applications.
To use virtual 8086 mode, an operating system sets up a virtual 8086 mode monitor, which is a program that manages the real-mode program and emulates or filters access to system hardware and software resources. The monitor must run at privilege level 0 and in protected mode. Only the 8086 program runs in VM86 mode and at privilege level 3. When the real-mode program attempts to do things like access certain I/O ports to use hardware devices or access certain regions in its memory space, the CPU traps these events and calls the V86 monitor, which examines what the real mode program is trying to do and either acts as a proxy to interface with the hardware, emulates the intended function the real-mode program was trying to access, or terminates the real-mode program if it is trying to do something that cannot either be allowed or be adequately supported.
The V86 monitor can also deny permission gently by emulating the failure of a requested operation—for example, it can make a disk drive always appear not ready when in fact it has not even checked the drive but simply will not permit the real-mode program to access it. Also, the V86 monitor can do things like map memory pages, intercept calls and interrupts, and preempt the real-mode program, allowing real-mode programs to be multitasked like protected-mode programs. By intercepting the hardware and software I/O of the real-mode program and tracking the state that the V86 program expects, it can allow multiple programs to share the same hardware without interfering with each other. So V86 mode provides a way for real-mode programs designed for a single-tasking environment to run concurrently in a multitasking environment.

Usage

It is used to execute certain DOS programs in
FlexOS 386, Concurrent DOS 386, Windows/386 2.10, DESQview 386, Windows 3.x, Multiuser DOS, Windows for Workgroups 3.1x, OS/2 2.x, 4690 OS, REAL/32 running in 386 Enhanced Mode as well as in Windows 95, 98, 98 SE and ME through virtual DOS machines, in SCO UNIX through Merge, and in Linux through DOSEMU. NTVDM in x86 Windows NT-based operating systems also use VM86 mode, but with very limited direct hardware access.

Memory addressing and interrupts

The most common problem by running 8086 code from protected mode is memory addressing which is totally different between protected mode and real mode.
As mentioned, by working under VM86 mode the segmentation mechanism is reconfigured to work just like under real mode, but the paging mechanism is still active, and it is transparent to the real mode code; thus, memory protection is still applicable, and so is the isolation of the address space.
When interrupts occur, the processor switches off the VM86 mode and returns to work in full protected mode to handle the interrupt. Also, before servicing the interrupt, the DS, ES, FS, and GS registers are pushed on the new stack and zeroed.

Virtual 8086 mode enhancements (VME)

The Pentium architecture added a number of enhancements to the virtual 8086 mode. These were however documented by Intel only starting with the subsequent P6 ; their more recent formal name is Virtual 8086 Mode Enhancements, abbreviated VME; The enhancements address mainly the 8086 virtualization overhead, with a particular focus on interrupts. Before the extensions were publicly documented in the P6 documentation, the official documentation referred to the famed Appendix H, which was omitted from the public documentation and shared only with selected partners under NDA.
Activating VME is done by setting bit number 0 of CR4. Because the VME interrupt speed-up enhancements were found useful for non-VM86 protected tasks, they can also be enabled separately by setting only bit number 1, which is called PVI. Detecting whether a processor supports VME is done using the CPUID instruction, by testing the value of second bit in EDX register, which is set if VME is supported by the processor. In Linux, this latter bit is reported as the vme flag in the /proc/cpuinfo file, under the "flags" section.
In virtual 8086 mode, the basic idea is that when IOPL is less than 3, PUSHF/POPF/STI/CLI/INT/IRET instructions will treat the value of VIF in the real 32-bit EFLAGS register as the value of IF in the simulated 16-bit FLAGS register. VIP will cause a GP fault on the setting of simulated IF, directing the OS to process any pending interrupts. PVI is the same idea but only affects CLI/STI instructions.
First generation AMD Ryzen CPUs have been found to feature a broken VME implementation. The second generation Ryzen has fixed this issue.

64-bit and VMX support

Support for virtual 8086 mode is not available in x86-64 long mode, although it is still present on 64-bit capable processors running in legacy mode.
The addition of VT-x has added back the ability to run virtual 8086 mode from x86-64 long mode, but it has to be done by transitioning the processor to VMX root mode and launching a logical processor itself running in virtual 8086 mode.
Westmere and later Intel processors usually can start the logical processor directly in real mode using the "unrestricted guest" feature ; this method removes the need to resort to the nested virtual 8086 mode simply to run the legacy BIOS for booting.