An Introduction to Intel GVT-g (with new architecture) Zhi Wang [email protected] WW24’17 Software & Services Group 1 GPU Virtualization VM VM … VM Hypervisor Intel Processor Graphics Software & Services Group 2 Existing Arts vs Intel GVT-g Legacy VGA API Direct Full Emulation Forwarding Pass-Through GPU Virtualization Qemu VGA BE FE vGPU GFX GFX GFX APIs GFX Hypervisor Hypervisor Hypervisor Hypervisor GPU GPU GPU GPU - 2D only +3D +3D/media/compute +3D/media/compute +Sharing +Performance +Performance - Media/compute - No sharing +Sharing - Compatibility Software & Services Group 3 Intel GVT-g Capabilities Performance Feature Sharing Near native performance Run native graphics stack Accelerate up to 8 VMs with direct GPU execution to sustain visual plus Dom0 experience Dom0 VM1 VM2 VM3 APPAPP APPAPP APPAPP APPAPP DirectX 11.1 Hypervisor OpenGL OpenCL 4.2 1.2 Software & Services Group 4 Intel Confidential GVT_g (KVMGT) Architecture Host Linux Guest OS Libvirt Qemu Mdev VFIO mgmt VFIO GFX Mdev Framework Driver … IOMMU Driver GFX Driver KVM Device Model IOMMU GPU Software & Services Group 5 Resource Management • vGPU resources I915 Resource Management – Graphics memory – Fence registers DOM0 Aperture Hidden Fence • Request GPU resource Request resource from host i915 from host resource GVT Resource Management allocator vGPU Type 0 vGPU Type 1 … PVINFO • GVT manages vGPU resource according to … vGPU types VM VM Software & Services Group 6 Interrupt Full GPU interrupt virtualization GPU Interrupts • Display Interrupts GVT Interrupt – VBlank Emulation Policy – GMBUS Framework – AUX Channel Interrupt • GPU command Interrupts Emulating – MI_USER_INTERRUPT Virtual – PIPE_CONTROL_NOTIFY Interrupt MPT Regs • Context Switch Interrupts Host VM VM VM Software & Services Group 7 Shadow GPU Page Table - GGTT • Global graphics memory space (GGTT) is partitioned - Dedicated resource for each VM High gfx memory space Low gfx memory space Host View VM1 VM2 VM3 Ballooned graphics memory space Available graphics memory space to VMs 8 Software & Services Group 8 Shadow GPU Page Table - PPGTT Features • 2/3/4 level page table • True per-process PPGTT • Page table cache • Reference counting • Out-of-Sync shadow Software & Services Group 9 Scheduling vGPU 1 vGPU 2 vGPU 3 … vExeclist vGuC Virtual HW Submission Interface vGPU workload Q Scheduling Policy GVT-g Workload Scheduler Scheduling Framework i915 scheduling Host i915 Submission Interface Software & Services Group 10 Scheduling I915 I915 … request request Host i915 Scheduling Dispatch i915 request Scheduling events GVT-g workload scheduling vGPU workload vGPU context switch dispatcher vGPU vGPU vGPU Workload vGPU Workload context context Software & Services Group 11 Scheduling MMIO Emulation Virtual HW submission – vExeclist / vGuC Schedule Workload Q intel_vgpu_workload Workload scheduler - Pick workload from Q Scheduling Policy - Notify virtual HW submission layer Framework - Shadow context & ring buffer - Submit shadow context & ring buffer to i915 - Wait workload finished - Complete workload Host i915 Submission System Software & Services Group 12 Display VM2 VM1 . GVT-g gets the guest frame buffer via virtual display . VFIO exposes guest frame buffer via dma-buf. Host GVT-g . User application attaches dma-buf Virtual into i915 GEM and gets i915 GEM Display Compositor handles DMA BUF GVT-g . User application shows guest screen DMA BUF VFIO context via i915 Render Engine texture window texture i915 Display Engine GEM OBJ plane pipe port GEM OBJ 13 Software & Services Group 13 Thanks Software & Services Group 14.