IBM System z z/VM Basics Arwed Tschoeke Systems Architect [email protected] © 2009 IBM Corporation © 2008 IBM Corporation Introduction We'll explain basic concepts of System z: – Terminology – Processors – Memory – I/O – Networking We'll see that z/VM virtualizes a System z machine: – Virtual processors – Virtual memory – … and so on Where appropriate, we'll compare or contrast: – PR/SM or LPAR – z/OS – Linux 2 z/VM: The Very Basics z/VM: The Very Basics 1 IBM System z © 2008 IBM Corporation System z Parts Nomenclature x86, UNIX, etc. System z Memory Storage (though we are moving toward "memory") Disk, Storage DASD – Direct Access Storage Device Processor Processor, Engine, PU (processing unit) IOP (I/O processor) CPU (central processing unit) CP (central processor) SAP (system assist processor) Specialty engines –IFL (Integrated Facility for Linux) –zAAP (System z Application Assist Processor) –zIIP (System z9 Integrated Information Processor) Computer CEC (central electronics complex) Server 3 z/VM: The Very Basics © 2008 IBM Corporation IBM System z Virtualization Genetics Over 40 years of continuous innovation in virtualization – Refined to support modern business requirements System z10 . – Exploit hardware technology for economical growth , .. ity System z9 z/VM V5 bil – LPAR, Integrated Facility for Linux, HiperSockets xi 64-Bit Fle – System z Application Assist Processors s, zSeries es VM/ESA Virtual Switch – System z Information Integration stn 9672 bu ESA Guest LANs Set Observer Ro Processors y, 9x21 ilit VM/XA Virtual Machine Resource Manager ab eli 3090 31-Bit Virtual Disks in Storage Performance Toolkit , R ity VM/HPO CMS Pipelines QDIO Enhanced Buffer State Mgmt bil 308x ala 64 MB Real Accounting Facility Minidisk Cache HiperSockets Sc 303x e : Absolute | Relative SHARE SIE on SIE Automated Shutdown alu 4381 VM/SP V Discontiguous Saved Segments Named Saved Systems I/O Priority Queuing ss SMP ine Instruction TRACE Start Interpretive Execution (SIE) Host Page-Management Assist us VM/370 B Programmable Operator (PROP) LPAR Hypervisor Integrated Facility for Linux HyperSwap S/370 Inter-User Communication Vehicle (IUCV) VM Assist Microcode Adapter Interruption Pass-Through CP-67 Conversational Monitor System (CMS) Dedicated I/O Processors Multiple Logical Channel Subsystems (LCSS) S/360 Diagnose Hypervisor Interface Program Event Recording (PER) Open Systems Adapter (OSA) Network Switching Control Program Hypervisor Translation Look-Aside Buffer (TLB) REXX Interpreter Large SMP Dynamic Virtual Machine Timeout Dynamic Address Translation (DAT) Zone Relocation Expanded Storage Multiple Image Facility (MIF) N_Port ID Virtualization (NPIV) 1960s 1972 1980 1981 1988 1995 2007... IBM System z – a comprehensive and sophisticated suite of virtualization function 4 z/VM: The Very Basics z/VM: The Very Basics 2 IBM System z Virtual Machines © 2009 IBM Corporation © 2008 IBM Corporation Virtual Machines Virtual Machine Virtual Machine … Virtual Machine Hypervisor (z/VM Control Program) A virtual machine is an execution context that obeys the architecture The purpose of z/VM is to virtualize the real hardware: – Faithfully replicate the z/Architecture Principles of Operation – Permit any virtual configuration that could legitimately exist in real hardware – Let many virtual machines operate simultaneously – Allow over commitment of the real hardware (processors, for example) – Your limits will depend on the size of your physical zSeries computer Virtual machine aka VM user ID, VM logon, VM Guest, Virtual Server 6 z/VM: The Very Basics z/VM: The Very Basics 3 IBM System z © 2008 IBM Corporation Virtual Machines in Practice Linux Linux z/OS CMS z/VSE z/TPF Others 31-bit 64-bit Hypervisor (z/VM Control Program) Control Program Component – manages virtual machines that adhere to the S/390 architecture and the z/Architecture Extensions available through CP system services and features CMS is special single user system and part of z/VM Control Program interaction via console device 7 z/VM: The Very Basics © 2008 IBM Corporation Phrases associated with Virtual Machines In VM – Guest: a system that is operating in a virtual machine, also known as user or userid – Running under VM or Running on VM: running a system as a guest of VM – Running second level: running a system as a guest of VM which is itself a guest of another VM – A virtual machine may have multiple virtual processors – Sharing is very important In relationship to LPAR (partitioning) – Logical Partition: LPAR equivalent of a virtual machine – Logical Processor: LPAR equivalent of a virtual processor – Running native or Running in BASIC mode : running without LPAR • Note: Basic mode is not available on z890, z990 or z9 – Isolation is very important 8 z/VM: The Very Basics z/VM: The Very Basics 4 IBM System z © 2008 IBM Corporation Phrases Associated with Virtual Machines Linux Virtual Linux z/OS or z/VSE processor z/VM Control Program Virtual Virtual Virtual Virtual processor processor processor processor z/VM Control Program Logical Logical Logical Logical Logical Logical Logical processor processor processor processor processor processor processor Logical Partition Logical Partition PR/SM LPAR 9 z/VM: The Very Basics © 2008 IBM Corporation A Virtual Machine z/Architecture We permit any 512 MB of memory configuration that a 2 processors real zSeries machine Basic I/O devices: could have – A console In other words, we – A card reader completely Virtual – A card punch implement the Machine – A printer Some read-only z/Architecture disks Principles of Some read-write Operation disks There is no “standard Some networking virtual machine devices configuration” 10 z/VM: The Very Basics z/VM: The Very Basics 5 IBM System z © 2008 IBM Corporation VM User Directory Definitions of: USER LINUX01 MYPASS 512M 1024M G – memory MACHINE ESA 2 IPL 190 PARM AUTOCR – architecture CONSOLE 01F 3270 A – processors SPOOL 00C 2540 READER * – spool devices SPOOL 00D 2540 PUNCH A – network device SPOOL 00E 1403 A – disk devices SPECIAL 500 QDIO 3 SYSTEM MYLAN LINK MAINT 190 190 RR – other attributes LINK MAINT 19D 19D RR LINK MAINT 19E 19E RR MDISK 191 3390 012 001 ONEBIT MW MDISK 200 3390 050 100 TWOBIT MR 11 z/VM: The Very Basics © 2008 IBM Corporation CP Commands CP DEFINE – Adds to the virtual configuration somehow – CP DEFINE STORAGE – CP DEFINE PROC – CP DEFINE {device} {device_specific_attributes} CP ATTACH – Gives an entire real device to a virtual machine CP DETACH – Removes a device from the virtual configuration CP LINK – Lets one machine's disk device also belong to another's configuration CP SET – Change various characteristics of virtual machine Changing the virtual configuration after logon is considered normal Usually the guest operating system detects and responds to the change 12 z/VM: The Very Basics z/VM: The Very Basics 6 IBM System z © 2008 IBM Corporation Getting Started IML – Initial Machine Load or Initial Microcode Load – Power on and configure processor complex – VM equivalents are: • LOGON uses the MACHINE statement in the CP directory entry • The CP SET MACHINE command – Analogous to LPAR image activation IPL – Initial Program Load – Like booting a Linux system – zSeries hardware allows you to IPL a system – z/VM allows you to IPL a system in a virtual machine via the CP IPL command – Linux kernel is like VM nucleus – Analogous to the LPAR LOAD function 13 z/VM: The Very Basics Processors © 2009 IBM Corporation z/VM: The Very Basics 7 IBM System z © 2008 IBM Corporation Processors Configuration – Virtual 1- to 64-way • Defined in user directory, or • Defined by CP command – A real processor can be dedicated to a virtual machine Control and Limits – Scheduler selects virtual processors according to apparent CPU need – “Share” setting - prioritizes real CPU consumption • Absolute or relative • Target minimum and maximum values • Maximum values (limit shares) either hard or soft – “Share” for virtual machine is divided among its virtual processors 15 z/VM: The Very Basics © 2008 IBM Corporation Start Interpretive Execution (SIE) SIE = “Start Interpretive Execution”, an instruction z/VM (like the LPAR hypervisor) uses the SIE instruction to “run” virtual processors for a given virtual machine. SIE has access to: – A control block that describes the virtual processor state (registers, etc.) – The Dynamic Address Translation (DAT) tables for the virtual machine z/VM gets control back from SIE for various reasons: – Page faults – I/O channel program translation – Privileged instructions (including CP system service calls) – CPU timer expiration (dispatch slice) – Other, including CP asking to get control for special cases CP can also shoulder tap SIE from another processor to remove virtual processor from SIE (perhaps to reflect an interrupt) 16 z/VM: The Very Basics z/VM: The Very Basics 8 IBM System z © 2008 IBM Corporation Scheduling and Dispatching VM – Scheduler determines priorities based on share setting and other factors – Dispatcher runs a virtual processor on a real processor – Virtual processor runs for (up to) a minor time slice – Virtual processor keeps competing for (up to) an elapsed time slice LPAR hypervisor – Uses weight settings for partitions, similar to share settings for virtual machines – Dispatches logical processors on real engines Linux – Scheduler handles prioritization and dispatching processes for a time slice or quantum 17 z/VM: The Very Basics © 2008 IBM Corporation Anomalies of Time VM virtualizes