KVM PV DEVICES
1 Agenda
Introduction & brief history
VirtIO
Enhanced VirtIO with KVM support
Further implementation
2 General & history Fully virtualized devices performs bad 55 Mbps for RTL Lots of io-exits per packet
Decided to implement a modern e1000 Advantage: Only Qemu coding no guest tools involved Irq coalescing Only 2-3 io-exits per packet Can be the base of user-space PV
But then came Ingo…
3 PV driver architecture
VM VM VM HVM VMguest HVM guestR PV I R HVMfrontend guest PV N I driver frontend VM G HVMN guestR driver PV G I R frontend PV PV N I driver frontend backend PV Qemu G N driver driver backend Qemu G driver PV backend PV Qemu driver backend Qemu driver
Linux kernel & KVM
PV backend KVM driver
4 General & history V0 - leveraging Ingo Molnar’s PV code Make loadable module Add HVM support Add NAPI Add memory barriers and improved ring Keep running after performance & stability Merge to the kernel?
Alternatives Xen Polished drivers Xen specific VirtIO was just published.
5 PV driver requirements Close to native performance
Merge with the kernel
Leverage existing code
Have usermode implementation Block device for qcow, vmdk formats Ability to function without KVM (-no-kvm)
Ability to run HVM guests
Robust code Easily add drivers
6 VirtIO An API for virtual I/O Implements network & block driver logic Written by Rusty Russell
Motivation Many hypervisors of all types Hard to tune and maintain each one Code reuse – The KVM way ;)
Implementations Lguest KVM Possible (Xen, UML, Qemu, VMware?..)
7 VirtIO
GuestVM kernel VIRTIO VIRTIO BlK NET
Add_buf Get_buf Sync VM Restart Detach Hypervisor Specifics PV code
Kick Qemu
Linux kernel & KVM
PV backend KVM driver
scatterlist sg[] for skb/blk_req data 8 VirtIO – Hypervisor specifics The front end logic is implemented by VirtIO
GuestVM kernel The backend needs VIRTIO VIRTIO BlK NET
VM Probing & Bus services Hypervisor Specifics Enumeration PV code Qemu
Irq Linux kernel & KVM
Parameters (mac,..) KVM PV backend driver Shared memory with remote side Hypercalls Host driver/userspace backend
9 Enhanced VirtIO Motivation Increase re-use Allow operation with various bus types GuestVM kernel VIRTIO VIRTIO BlK Make new devices code smallest NET VM Hypervisor Specifics PV code
Qemu
Components Linux kernel & KVM
KVM PV backend Shared memory code driver With per hypervisor I/O hypercalls Bus (pci, virtual bus) Host backend
10 Enhanced VirtIO
VM Blk Net VIRTIO Net VIRTIO BlK Backend Backend
Guest Enhanced VIRTIO VIRTIO kernel
Shared Virtual Bus Hypercalls mem backend VM
Qemu
Linux kernel & KVM
PV KVM backend driver
11 Enhanced VirtIO Status: Interface needs polishing 162 LOC 195 LOC VM Blk Net VIRTIO Net VIRTIO BlK KVM support Backend Backend Guest Enhanced VIRTIO VIRTIO PCI bus support kernel Shared Virtual Bus Hypercalls mem backend VM Result: Qemu Makes backend driver tiny 620 Mbps throughput for Linux kernel & KVM PV network KVM backend driver HVM Linux guest Before optimization Userspace backend driver 12 Enhanced VirtIO – shared memory backend
VirtIO backend Implements VirtIO interface Callbacks to hypervisor and
Code consists of VM Blk Net VIRTIO Net VIRTIO BlK add_buf, get_buf, restart, detach_buf Backend Backend Guest Enhanced VIRTIO VIRTIO Only shared memory logic needed kernel Shared Virtual Bus Hypercalls mem sync backend VM Ring logic Qemu IO pending hypercall Linux kernel & KVM be_virtqueue_interrupt handler PV KVM backend driver
13 Enhanced VirtIO – Day in a life of packet
2.Kernel 1.User VM 3.start_xmit Tcp/ip write Blk Net VIRTIO Net VIRTIO BlK Backend Backend 4.add_buf 12. skb_recv_done Guest Enhanced VIRTIO VIRTIO kernel 6.sync 13. get_buf… Shared Virtual Bus Hypercalls mem 5.add_buf backend VM 11.irq 7.sync 8.Notify hcall Qemu
Linux kernel &10.Trigger KVM irq
PV KVM backend driver 9.The pkts travels to tap/backend host device 14 Enhanced VirtIO – shared memory details
Based on lguest
1-1 shared memory
Data structure Page of descriptors for rx, tx. Available pointers page controlled by guest Used pointers page controlled by host
SG list is currently internal to descriptors Descriptors are chained by next pointer
15 Enhanced VirtIO – network be driver Implements kvm_virtnet_probe for pci bus Creates tx,rx be_new_virtqueue VM Blk Net VIRTIO Net VIRTIO BlK Backend Backend
Guest Enhanced VIRTIO VIRTIO 195 LOC kernel
Shared Probes virtnet Virtual Bus Hypercalls mem VM backend
Qemu Request_irq Linux kernel & KVM
Irq# taken from bus PV KVM backend driver
Register hypercall - shared memory pfn Device key for enumeration taken from bus
16 Further work Basic Add readv/writev handlers to Qemu Complete the user-space block device
Update with VirtIO gso. VM Blk Net VIRTIO Net VIRTIO BlK Complete migration support Backend Backend
Guest Enhanced VIRTIO VIRTIO kernel
Shared Advanced (also simple) Virtual Bus Hypercalls mem VM Publish the enhanced interface backend Optimize and stabilize Qemu Add host back end drivers
Add virtual bus Linux kernel & KVM
HVM improvements PV KVM backend Test with PV kernel driver
17 Q&A
Thank you ;)
18