TM

Progressive paravirtualization

Keir Fraser, XenSource HVM Architecture Domain 0 Domain N Guest VM (HVM) Guest VM (HVM) (32-bit) (64-bit) Linux xen64 ( x C M D m P o o e a / n v x d Unmodified OS Unmodified OS 3D 3P n t i e e r e c n l o l e s d l Linux xen64 ) F r o V n i r B t D t

e u a r n i c a v d k l e Guest BIOS Guest BIOS e d

r V n s r Native 0D i Native i d r 1/3P v t e Device Device u Virtual Platform Virtual Platform r a

Drivers Drivers l Callback / Hypercall VMExit VMExit PIC/APIC/IOAPIC Event channel emulation 0P

Control Interface Scheduler Event Channel Hypercalls Processor Memory I/O: PIT, APIC, PIC, IOAPIC Progressive paravirtualization

Hypercall API available to HVM guests Selectively add PV extensions to optimize . Net and Block IO . XenPIC (event channels) . MMU operations • multicast TLB flush • PTE updates (faster than page fault) . Time . CPU and memory hotplug PV Drivers Domain 0 Domain N Guest VM (HVM) Guest VM (HVM) (32-bit) (64-bit) Linux xen64 ( x C M D m P o o e a / n v x d Unmodified OS Unmodified OS 3D 3P n t i e e r e c n l o l e s d l Linux xen64 ) F F E E D D

r r V V i i v v F i i r r e e r t t r r o u u V s s n a a i r B l l t D t

e u a r n i c a v d k l e Guest BIOS Guest BIOS e d

r V n s r Native 0D i Native i d r 1/3P v t e Device Device u Virtual Platform Virtual Platform r a

Drivers Drivers l Callback / Hypercall VMExit VMExit PIC/APIC/IOAPIC Event channel emulation 0P

Control Interface Scheduler Event Channel Hypercalls Processor Memory I/O: PIT, APIC, PIC, IOAPIC Xen Hypervisor Hypercalls

HVM guest can detect hypervisor platform via CPUID instruction . New hypervisor leaves at 0x40000000 . Look for signature ‘XenVMMXenVMM’ . Space for future expansion and feature flags Hypercall page is filled in by writing its address to a special MSR . Location determined via CPUID . Currently always MSR 0x40000000 Hypercall page hides low-level details of transferring control to the VMM Building PV drivers for HVM

 PV drivers depend on architectural features of Xen . Grant tables for memory sharing . Event channels for asynchronous notifications  Encapsulate support in a ‘platform driver’ . Ioemu defines a dummy PCI device that triggers loading of the platform driver in the HVM guest  Xenbus, blkfront, netfront can be built as separate modules against a native Linux build . See unmodified_drivers/linux-2.6 in the xen-unstable tree Xen support for PV-on-HVM

 Event-channel notifications cause an interrupt to be delivered via the virtual APIC on a pre-registered vector . ‘Platform driver’ registers itself on that IRQ and demuxes pending events to registered drivers . Future: IRQ per-device or per-VCPU  Virtual CPUID and MSR addresses to allow hypervisor detection and hypercall setup  Hypercalls are being incrementally extended to support HVM guests . Also require support for 32-bit guests on 64-bit hypervisor: the 32-bit and 64-bit ABIs are different . This work overlaps strongly with PAE-on-64 PV guest support PV Driver performance

1000

900 rx tx 800

700

600 s / 500 b M

400

300

200

100

0 ioemMeasuredu P Vwith-on-H VttcpM , 1500 bytePV MTU