An Introduction to Intel GVT-g (with new architecture)
Zhi Wang [email protected] WW24’17
Software & Services Group
1 GPU Virtualization
VM VM … VM
Hypervisor
Intel Processor Graphics
Software & Services Group
2 Existing Arts vs Intel GVT-g
Legacy VGA API Direct Full Emulation Forwarding Pass-Through GPU Virtualization
Qemu VGA BE FE vGPU GFX GFX GFX APIs GFX
Hypervisor Hypervisor Hypervisor Hypervisor
GPU GPU GPU GPU
- 2D only +3D +3D/media/compute +3D/media/compute +Sharing +Performance +Performance - Media/compute - No sharing +Sharing - Compatibility
Software & Services Group
3 Intel GVT-g Capabilities
Performance Feature Sharing
Near native performance Run native graphics stack Accelerate up to 8 VMs with direct GPU execution to sustain visual plus Dom0 experience Dom0 VM1 VM2 VM3
APPAPP APPAPP APPAPP APPAPP DirectX 11.1
Hypervisor OpenGL OpenCL 4.2 1.2
Software & Services Group
4 Intel Confidential GVT_g (KVMGT) Architecture Host Linux Guest OS Libvirt Qemu Mdev VFIO mgmt
VFIO GFX Mdev Framework Driver
… IOMMU Driver GFX Driver KVM Device Model
IOMMU GPU
Software & Services Group
5 Resource Management
• vGPU resources I915 Resource Management – Graphics memory – Fence registers
DOM0 Aperture Hidden Fence
• Request GPU resource Request resource from host i915
from host resource GVT Resource Management allocator vGPU Type 0 vGPU Type 1 …
PVINFO • GVT manages vGPU resource according to … vGPU types VM VM
Software & Services Group
6 Interrupt
Full GPU interrupt virtualization GPU Interrupts
• Display Interrupts GVT Interrupt – VBlank Emulation Policy – GMBUS Framework – AUX Channel Interrupt • GPU command Interrupts Emulating – MI_USER_INTERRUPT Virtual – PIPE_CONTROL_NOTIFY Interrupt MPT Regs • Context Switch Interrupts
Host VM VM VM
Software & Services Group
7 Shadow GPU Page Table - GGTT
• Global graphics memory space (GGTT) is partitioned - Dedicated resource for each VM
High gfx memory space Low gfx memory space
Host View VM1 VM2 VM3
Ballooned graphics memory space
Available graphics memory space to VMs 8
Software & Services Group
8 Shadow GPU Page Table - PPGTT
Features
• 2/3/4 level page table • True per-process PPGTT • Page table cache • Reference counting • Out-of-Sync shadow
Software & Services Group
9 Scheduling
vGPU 1 vGPU 2 vGPU 3 …
vExeclist vGuC Virtual HW Submission Interface
vGPU workload Q
Scheduling Policy GVT-g Workload Scheduler Scheduling Framework
i915 scheduling Host i915 Submission Interface
Software & Services Group
10 Scheduling
I915 I915 … request request
Host i915 Scheduling
Dispatch i915 request Scheduling events
GVT-g workload scheduling vGPU workload vGPU context switch dispatcher
vGPU vGPU vGPU Workload vGPU Workload context context
Software & Services Group
11 Scheduling
MMIO Emulation
Virtual HW submission – vExeclist / vGuC
Schedule Workload Q intel_vgpu_workload
Workload scheduler
- Pick workload from Q Scheduling Policy - Notify virtual HW submission layer Framework - Shadow context & ring buffer - Submit shadow context & ring buffer to i915 - Wait workload finished - Complete workload
Host i915 Submission System
Software & Services Group
12 Display
VM2 VM1 . GVT-g gets the guest frame buffer via virtual display . VFIO exposes guest frame buffer via dma-buf. Host GVT-g . User application attaches dma-buf Virtual into i915 GEM and gets i915 GEM Display Compositor handles DMA BUF GVT-g . User application shows guest screen DMA BUF VFIO context via i915
Render Engine texture window texture i915 Display Engine GEM OBJ plane pipe port GEM OBJ
13
Software & Services Group
13 Thanks
Software & Services Group
14