TCG Enhancements on Powerpc

TCG enhancements on PowerPC

Nikunj A. Dadhania KVM Forum nikunj@linux.vnet.ibm.com

Linux Technology Center, India, IBM 25th August 2016 About me

. Guest firmware(SLOF) developer . QEMU user/developer

2/27 Agenda

3/27 How is emulation done ?

PowerPC POWER ISA Machine Code addi r9,r9,127 Translate

x86 Intel ISA Machine Code add $0x7f,%rbp

LAPTOP(x86)

Credits: Alexander Graf’s QEMU’s Recompilation Engine 4/27 More architectures !

ARM PowerPC X86 Binary Binary Binary

X86 Instructions

X86 Hardware

Credits: Alexander Graf’s QEMU’s Recompilation Engine 5/27 N x N Support: Very complex

ARM PowerPC X86 Binary Binary Binary

PowerPC X86 ARM Instructions Instructions Instructions

PowerPC X86 ARM Hardware Hardware Hardware

Credits: Alexander Graf’s QEMU’s Recompilation Engine 6/27 QEMU TCG – Tiny Code Generator

ARM PowerPC x86 S390 Target Binary Binary Binary Binary

TCG

tcg micro ops Machine Independent Intermediate notation

ARM PowerPC x86 S390 Host Hardware Hardware Hardware Hardware

Credits: Alexander Graf’s QEMU’s Recompilation Engine 7/27 QEMU Avatar – linux-user m m a m r a a r r g g o g r o o r r p

p p

4 . Input: Target binary and libraries

… c 6 m c p r p p a .

p Provides Linux system call emulation . Emulates target ISA

Laptop(x86)

8/27 POWER ISA 3.0

9/27 POWER ISA 3.0

. POWER (Performance Optimization With Enhanced RISC) . Adds ~180 new instructions . Various instructions added in different classes  Atomic memory operations  Hashing support operations  String operations (character testing, string processing) © 2015 IBM Corporation  Arithmetic operations(multiply-add, modulo)  ……... . http://ibm.biz/power-isa3 (needs registration)

10/27 Status - POWER ISA 3.0

. 24 instructions queued in ppc-for-2.8  Modulo, Special compare  Vector absolute, compare, shift . 24 instructions posted under review  Load/Store vector/scalar  Vector insert, extract, count© 2015 IBM trailing Corporation zeros . 25 instructions under test

. https://github.com/nikunjad/qemu/commits/p9-tcg

11/27 Challenges: POWER ISA 3.0

. Testing and verifying the instructions  Correctness  Repeatability  Negative test cases . Can use:  kvm-unit-test © 2015 IBM Corporation  QEMU qtest . Anton Blanchard’s instruction fuzzer  Compares physical CPU to QEMU emulation

12/27 PowerNV Platform

13/27 QEMU Avatar – System Emulation

4 . Invoked as machines (-machine M 6 6 8 C R x P A

pseries) P - –

- 2

1 . Runs isolated in its own memory M M M V V V space . Can be used to debug firmware, e Emulated Devices r t

a DISK Network s kernel, etc. w TCG e

F SoftMMU

Laptop(x86)

14/27 pSeries Machine Emulation ) ) ) e e . Based on sPAPR standard l l 4 4 4 6 6 6 c c c p p p p . Guest Emulation ( p p

( (

n 1 2 . Hyper-Call based M M M V V V . Para-virtualized guest pSeries Machine . Has been supported since a while SLOF

Laptop(x86)

15/27 PowerNV Machine Emulation ) ) ) e e l l 4 4 4 6 6 6 c c c p p p p ( p p

( ( . Emulate Bare Metal POWER platform n 1 2 M M M

V . Model Board Management controller V V (BMC) PowerNV Machine . Supports Hypervisor mode skiboot TCG MMU OCC © 2015 IBM. CorporationCan run nested guest BMC INTC . Assists in early bringup . Support IPMI Laptop(x86)

16/27 Status: PowerNV

. Initiated by Benjamin Herrenschmidt . Cédric Le Goater developing and pushing patches upstream . PowerNV ~50 preparatory patches upstream  POWER8 Hypervisor SPRs  Split Instruction and Data caches

17/27 PowerPC support for Multi-threaded TCG

18/27 System emulation - runs vCPUs serially

. Emulates multi-processor VM: but vCPUn serially. ... . vCPUs run in round robin mode vCPU2 VNC . Can’t emulate concurrent behaviour vCPU1 IO

CPU 1 CPU 2 CPU 3 CPU 4 …. CPU 8

PowerPC Emulation on X86

Credits: Alex Bennée Towards Multithreaded TCG 19/27 QEMU Multi-threaded TCG

. QEMU for multi-core system bringup . Community effort in progress . Challenges: Atomics, Memory Barriers, TLB Flush, etc.

VNC vCPU1 vCPU2 vCPU3 vCPU4 IO

CPU 1 CPU 2 CPU 3 CPU 4 …. CPU 8

PowerPC Emulation on X86

Credits: Alex Bennée Towards Multithreaded TCG 20/27 Status – PPC support for MTTCG

. Based on MTTCG base patches and atomic cmpxchg . Take iothread locks during hcalls . Load with reservation(lwarx and family) . Store conditional(stwcx. and family) with atomic cmpxchg micro-ops . Booted VM with 4 vCPUs . Ebizzy performance (ebizzy -S 300 -t 16)

1514 records/s 3.5x 5415 records/s real 300.00 s real 300.00 s user 222.74 s user 420.01 s sys 976.80 s sys 778.93 s

21/27 Challenges: PPC support for MTTCG

. Still unstable  https://github.com/nikunjad/qemu/commits/pseries_mttcg_wip . pSeries uses hcall for page table update/invalidate. . Memory barriers . Supporting PowerNV platform

22/27 Misc TCG Improvements

. Load/Store improvements – Benjamin Herrenschmidt . Exception handling improvements – Benjamin Herrenschmidt . Load/Store consolidation – Nikunj

23/27 Future

. Complete POWER ISA 3.0 support . Upstreaming PowerNV in QEMU . Future - POWER9 PowerNV support . Stabilize MTTCG on POWER

24/27 Credits

. Benjamin Herrenschmidt . Cédric Le Goater . Alexander Graf – QEMU’s Recompilation Engine https://dl.dropboxusercontent.com/u/8976842/TCG.pdf . Alex Bennée – Towards Multithreaded TCG http://www.linux-kvm.org/images/c/cf/02x02-Alex_Benee-Towards_Multithreaded_TCG.pdf© 2015 IBM Corporation

25/27 Legal statement

. This work represents the views of the author, and does not necessarily represent the view of IBM . IBM and IBM (logo) are trademark of International Business Machines in the United States and/or other. . Linux is a registered trademark of Linus Torvalds . Other company, product, logos and service names may be trademarks or service marks of others © 2015 IBM Corporation . This document is provided “AS IS”, with no express or implied warranties. Use the information in this document at your own risk . Results mentioned in the presentation is for reference purposes only, and are not to be relied on in any manner.

26/27 धꅍयव픾द

27/27