Virtualization options for hypervisors on Linux on IBM z15 and IBM LinuxONE III Richard Young IBM Executive IT Specialist IBM Z and LinuxONE IBM Systems Lab Services Architectural options for installations with Linux on IBM Z & LinuxONE Main Layers for Linux on Z Components - In Flight Security - At Rest - Key Management - SUSE Linux Distribution - Red Hat - Ubuntu - LPAR only - z/VM Virtualization - KVM - Containers, SSC, K8S, RH OCP - OSA options Network attachments - RoCE / ISM ~ TCP, SMC-R, SMC-D - Hipersockets - Virtual (MacvTap, Vswitch, bond) LPAR Management - DPM - PR / SM - FCP / SCSI Storage - FICON ECKD & Disk attachments - Internal NVMe - Spectrum Scale Agenda Ø Benefits of virtualization • Available options • Considerations for virtualization decisions • Virtualization options • Firmware hypervisors • Software hypervisors • Software Containers • Firmware hypervisor decision guide • Virtualization decision guide • Summary Why do we virtualize? What are the benefits of virtualization? § Simplification – use of standardized images, virtualized hardware, and automated configuration of virtual infrastructure § Migration – one of the first uses of virtualization, enable coexistence, phased upgrades and migrations. It can also simplify hardware upgrades by make changes transparent. § Efficiency – reduced hardware footprints, better utilization of available hardware resources, and reduced time to delivery. Reuse of deprovisioned or relinquished resources. § Resilience – run new versions and old versions in parallel, avoiding service downtime § Cost savings – having fewer machines translates to lower costs in server hardware, networking, floor space, electricity, administration (perceived) § To accommodate growth – virtualization allows the IT to be more responsive to business growth, hopefully avoiding interruption 4 © Copyright IBM Corporation 2020 Agenda • Benefits of virtualization Ø Available options • Considerations for virtualization decisions • Virtualization options • Firmware hypervisors • Software hypervisors • Software Containers • Firmware hypervisor decision guide • Virtualization decision guide • Summary What hypervisors and virtualization options on Linux on IBM Z & LinuxONE q IBM traditional PR/SM or via DPM (Dynamic Partition Manager) – Firmware based virtualization to securely share and partition hardware resources. DPM providing graphical interface & REST interfaces with simplified management, automation, and dynamic capability for LinuxONE. q IBM z/VM – IBM developed, software-based mainframe virtualization that can be traced back to the beginning of Virtualization in computing q Linux KVM – Open source software-based virtualization. Supports multiple hardware architectures. Kernel based virtual machines started in mid 2000’s. Available via Linux Distro’s. q Containers – System Containers and Application containers. Via Linux cgroups and namespaces, provide an isolated environment for applications to run. Containers share a single host kernel. qOCI based Containers - Standard for container with a toolset ( Docker, Podman, ..) image build process, an API & CLI, a registry. Clustering added with additional tools like Dock swarm, Kubernetes. q IBM Secure Service Container (SSC) – Special partition for fully encrypted workloads. Traditional system administrator access removed. Limited and encrypted network access. 6 © Copyright IBM Corporation 2020 Agenda • Benefits of virtualization • Available options Ø Considerations for virtualization decisions • Virtualization options • Firmware hypervisors • Software hypervisors • Software Containers • Firmware hypervisor decision guide • Virtualization decision guide • Summary Considerations for virtualization decisions q Current in house standards – Distros q Open vs proprietary q Software supported in combination with it q Outage avoidance – Live migration/relocation q Hardware support – i.e. NVMe, CTC, ISM q Feature/Function and requirements q Colocation requirements ( z/OS, x86) ØLive relocation requirements x, y ,z q Available skill set in house to manage q Dynamic by design – No outages to change q Ability to hire talent with needed skills q Performance / Scalability rd q Learning curve / duration to become q Ecosystem – Documentation, training, 3 fluent/expert – Simplicity vs complexity party solutions and support q Level of Isolation / security q Cost – Direct / Indirect for additional features qMonitoring , Security, Automation, Auditing, q Certifications & Multitenancy requirements Time to train rd q Automation capability – Rest APIs or 3 party q Integrity and Isolation tooling – i.e. Kickstart deployment, qSecure boot OpenStack, or Ansible qSecure Execution qAutomated / Manual encryption 8 © Copyright IBM Corporation 2020 Agenda • Benefits of virtualization • Available options • Considerations for virtualization decisions Ø Virtualization options • Firmware hypervisors • Software hypervisors • Software Containers • Firmware hypervisor decision guide • Virtualization decision guide • Summary IBM LinuxONE virtualization options overview Server virtualization. There are typically Application isolation. There are typically dozens or hundreds of Linux servers in a thousands of Containers in Linux on KVM or z/VM LPAR. IBM Z. IBM LinuxONE 2nd level virtualization Linux Linux for Test & QA only Linux SSC Linux 2 Linux Virtual Linux Secure Linux CPUs Linux ServiceLinux (cores) Linux Container KVM Linux z/VM Linux (SSC) Virtual CPUs Server (cores) virtualization KVM z/VM LPAR LPAR1 LPAR2 LPAR3 LPAR4 virtualization Logical (PR/SM or DPM) CPUs (cores) Real P1 P2 P3 P4 P5 P6 P7 P8 CPUs* (cores) P1 – P8 are Central Processor Units (CPU -> core) or Integrated Facility for Linux (IFL) Processors (IFL -> core) * - One shared Pool of cores per System only 10 2020 IBM Corporation Note: - LPARs can be managed by DPM or traditional PR/SM IBM Z virtualization options Server virtualization. Typically dozens - hundreds of Linux Application isolation. There are typically servers in a KVM or z/VM LPAR collocated with z/OS or thousands of Containers in Linux on others IBM Z. IBM Z 2nd level virtualization Linux z/OS for Test & QA only Linux SSC Linux 2 Linux Virtual Linux Secure Linux CPUs Linux z/OS or ServiceLinux (cores) Linux Container KVM Linux z/VM z/VSE or Linux (SSC) Virtual z/TPF CPUs Server (cores) virtualization KVM z/VM LPAR LPAR1 LPAR2 LPAR3 LPAR4 LPAR5 virtualization Logical with PR/SM or CPUs DPM (With (cores) supported operating Real systems) P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 CPUs* (cores) P1-P2 CPUs as general-purpose CPs, P3 – P10 are Central Processor Units (CPU -> core) or Integrated Facility for Linux (IFL) Processors (IFL -> core) * - One shared Pool of cores by type per System only 11 Note: - LPARs can be managed by DPM or traditional PR/SM 2020 IBM Corporation Architectural Options 1) Firmware hypervisor management • Traditional PR/SM • IBM Dynamic Partition Manager 12 © Copyright IBM Corporation 2020 2020 IBM Corporation Traditional PR/SM Management • Does not have to be implemented in a dynamic manner, but can be under the right conditions and process. You must take care to be dynamic capable. • You could build an assembler macro deck in a text file, but this is error prone and very labor intensive. • Typically you would build an IODF and populate an IOCDS from the IODF. • An IODF can be created and managed with HCD or HCM. • Both of these programs are available in z/OS and z/VM environments • In a z/VM environment, HCD manages an IODF and IOCDS, but does not have the panel system to build the IODF, that is where HCM is used. • z/VM can also manage the IO configuration via CP commands • While HCM graphically builds an IODF, it does not write an IOCDS or activate a new configuration. 13 © Copyright IBM Corporation 2020 What is IBM Dynamic Partition Manager? • Built on existing PR/SM technology capabilities • Simplified, consumable, enhanced, partition life-cycle and integrated dynamic I/O management capabilities LINUX LINUX LINUX SSC • Provides the technology foundation that enables APIs for IaaS and secure, private KVM Clouds PR/SM DPM IBM DPM Powerful and easy HMC 14 © Copyright IBM Corporation 2020 Architectural decisions for LPAR level virtualization management PR/SM - Processor Resource/Systems Manager • For Mixed workload (i.e. z/OS & Linux) with all features supported • For LinuxONE with all HW features supported • Needs specialized skill for new IBM Z & LinuxONE Admins • Requires use of HCD and optionally HCM to manage the IO configuration, which comes with z/VM or z/OS LINUX LINUX LINUX SSC DPM – Dynamic Partition Manager KVM • For Linux, z/VM, KVM, and SSC only, no z/OS, VSE, or others • Intuitive Graphical interface , all configuration from HMC PR/SM DPM • REST APIs for integration in SDD - Software Defined Datacenter HMC • Python and Ansible libraries for REST APIs • No support for z/VM SSI & LGR (CTC support – required) • No support yet for ISM and GDPS Appliance (NVMe is supported) • Requires FC0016 and two 1000BaseT adapters 15 © Copyright IBM Corporation 2020 Architectural Options 1) Firmware hypervisor management • Traditional PR/SM • IBM Dynamic Partition Manager 2) Optionally, one or more software hypervisor • IBM z/VM • KVM 16 © Copyright IBM Corporation 2020 2020 IBM Corporation Optionally a software Hypervisor, what if you choose none? Ø Limited to the number of partitions the Ø Could still utilize containers for enhanced isolation machine supports. Inhibits scale. vs a single Linux instance Ø Eliminates any hypervisor imposed limits