XenXen VTVT statusstatus andand TODOTODO listslists forfor XenXen--summitsummit
Arun Sharma, Asit Mallick, Jun Nakajima, Sunil Saxena
® R OutlineOutline • VMX Guests Status Summary • Status Domain0 restructuring – PCI/IOAPIC X86-64 VMX guests enhancements
à QEMU Device Models à Paravirtualized drivers Guest FW
• TODO list
2 R Copyright Intel Corp VMXVMX GuestsGuests StatusStatus SummarySummary
• VMX: Full support of 32-bit unmodified Linux guests without FW on IA-32 platforms
à Includes support of standard PC platform devices 32-bit unmodified Linux guest support without FW added to EM64T • Making Good progress on (Q2) Domain0 restructuring – PCI/IOAPIC X86-64 support (64-bit unmodified guests) VMX guests enhancements
à Device models, paravirtualized drivers Guest FW • In plan (Q3) SMP support for VMX guests, IPF VT guests, performance, installation
3 R Copyright Intel Corp Domain0Domain0 restructuringrestructuring –– PCI/IOAPICPCI/IOAPIC • Xen IO architecture today Xen enumerates PCI devices Xen partitions the devices among domains • Adds complexity to Xen Complex PCI configurations, need for ACPI Many workarounds for PCI hardware bugs needed • Proposed Solution Move all PCI device enumeration code to Domain0
à Including full support of ACPI Also requires enabling PCI, Local and I/O APIC
4 R Copyright Intel Corp Domain0 restructuring – PCI/IOAPIC (cont.) • Splitting of Resources and Device Enumeration Xen
à Owns Local APIC, IOAPIC including assignment of interrupt vectors à ACPI tables parsing only (no AML interpreter) Domain0
à Full ACPI support – Need to map ACPI tables into domain0 – Xen passes ACPI table pointer in shared_info_t à Local APIC – Stubs to satisfy MP parsing à IO APIC – Accessed by Domain0 using hypercalls • Primarily to set interrupt redirection entry à PCI Device Enumeration
5 R Copyright Intel Corp Domain0 restructuring – PCI/IOAPIC (Status) • A patch against xeno-unstable.bk exists and boots on some machines • Need to validate on a wider variety of machines Especially ones with complex PCI configurations • Other outstanding works MSI support
à Need to covert Domain0 IRQ to vector (hypercall) Partitioning of devices between domains PCI Hotplug support Hotplug of I/O APICs?
6 R Copyright Intel Corp X86X86--6464 StatusStatus
• It’s in unstable BK tree • Stable and usable Multi-user mode, e.g. login from remote and build the kernel • What’s left DomU support. Tested to build 32-bit VMX domain SMP support, 32-bit binary support, IOMMU, writable page tables, NUMA support? Performance tuning Running on big machines (# of CPUs, memory, I/O) • Need help from community Actively build, test, use. Finish what’s left
7 R Copyright Intel Corp X86X86--6464 –– RestructuringRestructuring • Need to have common chipset/platform code among {x86-32, x86-64} x {Linux, XenoLinux} for better code maintenance No compelling reasons for them to be slightly different: time.c, smp.c, smpboot.c, several headers Shared files with x86 as much as possible, but not enough
à Timer, PCI, microcode, etc. We can use the same code for x86-32 and x86-64 more for xenolinux
à Initialization à Chipset/platform support – PCI, APIC, ACPI, interrupt management, most of trap handling, timer management à Processor specific part – CPU detection, most of context switching à SMP support
8 R Copyright Intel Corp X86X86--6464 StatusStatus –– VMXVMX guestsguests • 32-bit VMX guests are available on both 32-bit and 64-bit VT-x Xen • Working on: 64-bit guests support
à 4-level/3-level shadow mode support Enhanced 32-bit support
à PAE, eXecute Disable (XD) support on 64-bit VT-x Xen Extend shadow code to support 4-level cleanly to support both types of guests SMP guest support
à Local and I/O APIC support Test more on MP systems
9 R Copyright Intel Corp VMXVMX GuestsGuests Enhancements:Enhancements: DeviceDevice ModelsModels
• What we have today Bochs device models providing
à PS/2 kbd/mouse, CMOS/RTC, Timer (8254), Generic PCI bridge, EIDE, VGA, Serial ports, NMI Ports, Floppy, NIC NE2K (ISA), Local APIC, IOAPIC, CDROM, USB à Gaps: SCSI, PCI NIC, IDE-DMA • What we are planning to do: switch to QEMU Get PCI NIC and IDE-DMA for free Code written in C Need to implement some functionality (for eg: APIC) either from – unstable or Bochs Status: Implementation complete, under regression/performance testing.
10 R Copyright Intel Corp VMXVMX GuestsGuests Enhancements:Enhancements: VBD/VNICVBD/VNIC
Domain 0 Domain N Guest VM (VMX domain)
Xenolinux (xm/xend) Control Models Device Panel 3P Req/Resp Unmodified 3D
Ring Buffer AppEvent Channel Linux PCI Device VBD FE Virtual Drivers (Virtual Event Driver Channel) Virtual driver VNIC Backend
Guest BIOS 0D 1P Native Device Virtual Platform Drivers Callback VMExit
Event channel 0P
Control Interface Scheduler Event Channel Hypercalls Processor Memory I/O: PIT, APIC, PIC PCI, ACPI Xen Hypervisor 11 R Copyright Intel Corp VMXVMX GuestsGuests Enhancements:Enhancements: VBD/VNICVBD/VNIC StatusStatus
• Basic implementation complete Not integrated with PCI virtual event channel device model Testing/performance analysis in progress • Work remaining Use of grant tables instead of mpa
12 R Copyright Intel Corp GuestGuest FWFW ArchitectureArchitecture Domain 0 VMX domain
Control Panel Unmodified Linux Boot Loader xm c xend a Web client Emulator/Execution Environment b ROM BIOS (Real Mode code @ 0xF000:0)
PICDevice ModelsDispatcher 5 VGA BIOS (Real Mode code @0xC000:0) IDE BIOS Data (Generated by ROM BIOS) SCSI Net 1 VMExit Physical Device Drivers 2 6 4 Arch -> request packet Send event Send event - Decode I/O - response packet - request packet - Creat request packet
7 3 Port x Resp ->Arch state VMexit -> Port n Control Interface Scheduler Event Channel Hypercalls Processor Memory I/O: PIT, APIC, PIC PCI, ACPI Hypervisor 13 R Copyright Intel Corp GuestGuest FWFW StatusStatus • Real Mode Emulator Implementation complete and integrated into Guest FW execution environment • ROM BIOS Using BOCHS/Plex86 ROM BIOS Extended to E820 memory map to be dynamic Adding ACPI support • VGA BIOS Using BOCHS/Plex86 VGA BIOS • Control Panel Enhancements Enhanced to load Execution environment, ROM BIOS and VGA BIOS in a VMX domain
14 R Copyright Intel Corp TODOTODO
• > 4 Gig memory support for 32-bit Xen and 32-bit domains PAE support • Discontigous memory support in Xen • 32-bit/64-bit MP Guests • Debugger / Crash dump • Performance monitoring tools • Scheduler enhancements for load balancing • Power-management
15 R Copyright Intel Corp