An Introduction to GVT-g (with new architecture)

Zhi Wang [email protected] WW24’17

Software & Services Group

1 GPU Virtualization

VM VM … VM

Hypervisor

Intel Processor Graphics

Software & Services Group

2 Existing Arts vs Intel GVT-g

Legacy VGA API Direct Full Emulation Forwarding Pass-Through GPU Virtualization

Qemu VGA BE FE vGPU GFX GFX GFX GFX

Hypervisor Hypervisor Hypervisor Hypervisor

GPU GPU GPU GPU

- 2D only +3D +3D/media/compute +3D/media/compute +Sharing +Performance +Performance - Media/compute - No sharing +Sharing - Compatibility

Software & Services Group

3 Intel GVT-g Capabilities

Performance Feature Sharing

Near native performance Run native graphics stack Accelerate up to 8 VMs with direct GPU execution to sustain visual plus Dom0 experience Dom0 VM1 VM2 VM3

APPAPP APPAPP APPAPP APPAPP DirectX 11.1

Hypervisor OpenGL OpenCL 4.2 1.2

Software & Services Group

4 Intel Confidential GVT_g (KVMGT) Architecture Host Linux Guest OS Libvirt Qemu Mdev VFIO mgmt

VFIO GFX Mdev Framework Driver

… IOMMU Driver GFX Driver KVM Device Model

IOMMU GPU

Software & Services Group

5 Resource Management

• vGPU resources I915 Resource Management – Graphics memory – Fence registers

DOM0 Aperture Hidden Fence

• Request GPU resource Request resource from host i915

from host resource GVT Resource Management allocator vGPU Type 0 vGPU Type 1 …

PVINFO • GVT manages vGPU resource according to … vGPU types VM VM

Software & Services Group

6 Interrupt

Full GPU interrupt virtualization GPU Interrupts

• Display Interrupts GVT Interrupt – VBlank Emulation Policy – GMBUS Framework – AUX Channel Interrupt • GPU command Interrupts Emulating – MI_USER_INTERRUPT Virtual – PIPE_CONTROL_NOTIFY Interrupt MPT Regs • Context Switch Interrupts

Host VM VM VM

Software & Services Group

7 Shadow GPU Page Table - GGTT

• Global graphics memory space (GGTT) is partitioned - Dedicated resource for each VM

High gfx memory space Low gfx memory space

Host View VM1 VM2 VM3

Ballooned graphics memory space

Available graphics memory space to VMs 8

Software & Services Group

8 Shadow GPU Page Table - PPGTT

Features

• 2/3/4 level page table • True per-process PPGTT • Page table cache • Reference counting • Out-of-Sync shadow

Software & Services Group

9 Scheduling

vGPU 1 vGPU 2 vGPU 3 …

vExeclist vGuC Virtual HW Submission Interface

vGPU workload Q

Scheduling Policy GVT-g Workload Scheduler Scheduling Framework

i915 scheduling Host i915 Submission Interface

Software & Services Group

10 Scheduling

I915 I915 … request request

Host i915 Scheduling

Dispatch i915 request Scheduling events

GVT-g workload scheduling vGPU workload vGPU context switch dispatcher

vGPU vGPU vGPU Workload vGPU Workload context context

Software & Services Group

11 Scheduling

MMIO Emulation

Virtual HW submission – vExeclist / vGuC

Schedule Workload Q intel_vgpu_workload

Workload scheduler

- Pick workload from Q Scheduling Policy - Notify virtual HW submission layer Framework - Shadow context & ring buffer - Submit shadow context & ring buffer to i915 - Wait workload finished - Complete workload

Host i915 Submission System

Software & Services Group

12 Display

VM2 VM1 . GVT-g gets the guest frame buffer via virtual display . VFIO exposes guest frame buffer via dma-buf. Host GVT-g . User application attaches dma-buf Virtual into i915 GEM and gets i915 GEM Display Compositor handles DMA BUF GVT-g . User application shows guest screen DMA BUF VFIO context via i915

Render Engine texture window texture i915 Display Engine GEM OBJ plane pipe port GEM OBJ

13

Software & Services Group

13 Thanks

Software & Services Group

14