Development of Multicore Computing for a Cloud-Based Unikernel Operating System Maghsoud Morshedi Chinibolagh Master’s Thesis Spring 2016 Development of Multicore Computing for a Cloud-Based Unikernel Operating System Maghsoud Morshedi Chinibolagh 18th May 2016 Abstract The cloud computing is going to change the IT design methodologies. This infrastructure requires innovation in order to enhance efficiency and functionality. One of the key element in the could infrastructure is the operating system which manages application and services. Hence, the operating system customization specifically for the cloud computing can enhance efficiency, scalability, and functionality. The Unikernel operating system will be the next generation of cloud operating systems and introduces a generative mechanism for the information infrastructure in the cloud. Since the processor vendors produce new processors with more independents cores inside and the cloud infrastructure equipped with high-end hardware and massive computing power, therefore the Unikernel operating systems should use this high computing power. Hence, the multicore computing can be a requirement for Unikernel operating system in order to enhance the performance and efficiency. Indeed, this thesis developed multicore computing for a Unikernel operating system such that it utilized new approaches in order to deal with multicore computing challenges in the virtual environment. The development provides the Unikernel operating system with as many cores as the virtualization platform can allocate to a virtual machine. Therefore, the new solutions developed in order to deal with the race condition, shared data and task management in the multicore environment. Finally, the results demonstrated that the multicore computing for an IncludeOS Unikernel operating system achieved better performance than a regular operating systems such as Ubuntu. Hence, the multicore computing enhances the efficiency, performance, and functionality of the Unikernel operating systems. Consequently, Unikernel operating systems can compete with existing solutions as a cloud operating system. i Acknowledgements I would like to express my sincere appreciation and gratitude to the following people: • First of all, I would like to thanks my wife Raheleh for her encouragement and rich discussions during my study. Her support, quiet patience, and unwavering love were undeniably bedrock upon which the past four years of my life have been built. • Hårek Haugerund for his supervision, patience, support and encour- agement throughout this thesis. It is not often that one finds a super- visor that always finds the time for listening to little problems that unavoidably encountered during the research. His technical and ed- itorial recommendation was essential to the completion of this thesis. • Kyrre Begnum who has been a constant source of enthusiasm and encouragement during the two years of my Master program. The door of his office was always open whenever I had a question. • Alfred Bratterud for his technical feedback, support and encourage- ment during this thesis. • Paal E. Engelstad for his encouragement and support during my study. In addition, for giving the opportunity to be an assistant for his courses. • Anis Yazidi for his feedback and encouragement during my thesis. • I am deeply thankful to my family for their love, support, and sacrifices. Without them, I would not be able to achieve this. • My classmates and friends for all the worthwhile discussion and great times that we have shared. ii Contents 1 Introduction 1 1.1 Motivation . .1 1.2 Problem Statement . .6 1.3 Thesis Structure . .9 1.4 Summary . .9 2 Background 10 2.1 Cloud Computing . 10 2.2 Cloud Operating System . 12 2.3 Unikernel Operating System . 13 2.4 IncludeOS . 14 2.5 Multicore Computing . 16 2.5.1 Multicore Versus Multitasking . 17 2.5.2 History of Multicore Computing . 18 2.5.3 Multiprocessor (MP) Configuration Table . 19 2.5.4 Advanced Configuration and Power Interface (ACPI) 19 2.5.5 Model Specific Registers (MSRs) . 21 2.5.6 CPUID Instruction . 22 2.5.7 Advanced Programmable Interrupt Controller (APIC) 24 2.5.7.1 Discrete APIC . 24 2.5.7.2 Local APIC . 24 2.5.8 Processor Initialization . 25 2.5.9 Bootstrap and Application Processors . 26 2.5.10 Multicore Initialization Consideration . 26 2.5.11 Multicore Initialization Protocol . 27 2.5.12 Multicore Message Passing Technique . 27 2.5.13 Multicore Shared Memory Technique . 28 2.6 Related Works . 29 2.6.1 OSv Project . 29 2.6.2 MirageOS . 31 2.7 Summary . 31 3 Methodology 32 3.1 Research Method . 32 3.2 System Design . 34 3.2.1 Virtualization Platform . 35 3.2.2 Programming Languages . 36 iii 3.2.3 Compiler and Linker . 37 3.3 Solution Development . 37 3.3.1 Processor Basic Information . 38 3.3.1.1 MP Floating Pointer Structure . 38 3.3.1.2 MP Configuration Table Header . 40 3.3.1.3 Root System Description Pointer . 42 3.3.1.4 Root System Description Table . 43 3.3.1.5 IA32_APIC_BASE MSR . 45 3.3.1.6 Vendor Identification String . 46 3.3.1.7 Hyper-Threading Technology . 46 3.3.2 APIC ID . 48 3.3.3 Logical Processor ID . 49 3.3.4 Processor Core ID . 50 3.3.5 Package ID . 50 3.3.6 Hierarchy Mapping of Processor Topology . 50 3.3.7 Multicore Initialization . 51 3.3.7.1 Application Processor Initialization . 51 3.3.7.2 Interrupt Command Registers . 52 3.3.7.3 INIT Inter-Processor Interrupt . 54 3.3.7.4 STARTUP IPI . 55 3.3.7.5 Self-Configuration Code . 55 3.4 Solution Implementation . 56 3.5 Testing . 57 3.6 Evaluation . 57 3.6.1 Experiment . 58 3.7 Summary . 59 4 Implementation 60 4.1 Build Existing System . 60 4.2 Development . 61 4.2.1 Collecting Processor Basic Information . 61 4.2.1.1 MP Configuration Table . 61 4.2.1.2 ACPI Specification . 64 4.2.1.3 Model Specific Registers (MSRs) . 67 4.2.1.4 CPUID Instruction . 69 4.2.1.5 Vendor Identification String . 71 4.2.1.6 Hyper-Threading Technology . 72 4.2.2 Advanced Programmable Interrupt Controller . 72 4.2.3 Logical Processor Enumeration . 74 4.2.4 Processor Core Enumeration . 75 4.2.5 Processor Package Enumeration . 76 4.2.6 Processor Topology Hierarchy Mapping . 76 4.2.7 Multicore Initialization . 76 4.2.7.1 Interrupt Command Register . 78 4.2.7.2 INIT IPI . 80 4.2.7.3 STARTUP IPI . 81 4.2.7.4 Self-Configuration Code . 82 4.3 Testing . 85 iv 4.3.1 Build . 85 4.3.2 Development Testing . 87 4.4 Evaluation . 88 4.4.1 Building Experiments . 89 4.5 Summary . 91 5 Analysis 92 5.1 Multicore Unikernel Operating System . 92 5.2 Critical Section in Multicore Operating System . 100 5.3 Multicore Communication . 101 5.4 Non-Preemptive Multicore . 102 5.5 Development Evaluation . 103 5.6 Summary . 106 6 Discussion 107 6.1 Research Master Plan . 107 6.2 Identifying Processors . 110 6.3 Race Condition Among Processors . 111 6.4 Task Management . 111 6.5 Shared Data . 112 6.6 Multicore Versus Multiple Single Core Operating System . 113 6.7 Future Work . 114 6.8 Summary . 115 7 Conclusion 116 Bibliography 118 A Program codes 124 B Scripts 135 v List of Figures 2.1 Structure of ACPI description tables . 20 3.1 The research methodology diagram . 34 3.2 Conceptual system design . 36 3.3 MP Floating Pointer Structure . 39 3.4 MP Configuration Table Header . 41 3.5 Root System Description Pointer . 43 3.6 Root System Description Table . 44 3.7 IA32_APIC_BASE MSR . 46 3.8 Multicore system supporting Hyper-Threading Technology . 48 3.9 Processor topology enumeration from initial APIC ID . 49 3.10 Interrupt Command Register . 54 4.1 MP floating pointer table Results . 63 4.2 MP configuration table Results . 65 4.3 Root system description pointer (RSDP) table results . 66 4.4 Root system description table (RSDT) results . 68 4.5 IA32_APIC_BASE MSR value on the BSP . 69 4.6 The Intel® Core™ i7-3632QM processor supported feature on ECX inside the QEMU virtual machine . 73 4.7 The Intel® Core™ i7-3632QM processor supported feature on EDX inside the QEMU virtual machine . 74 4.8 The ICR value while sending the INIT IPI to all APs . 80 vi List of Tables 2.1 Web operating systems . 13 2.2 Unikernel operating systems . 14 2.3 The CPUID leaves and information which they can return . 23 3.1 Vendor identification string returned by CPUID instruction . 47 3.2 Initial APIC ID hierarchy for hypothetical system with 2 physical processors which each processor provide three cores and each core supports two logical processors . 51 5.1 Execution time of 500000th prime number computation within different operating system with 36 cores . 105 vii Listings 3.1 The Bash command to create a virtual machine . 36 3.2 The IncludeOS project in GitHub . 56 4.1 Multicore computing for IncludeOS project in GitHub . 60 4.2 Program code to retrieve the address of EBDA . 62 4.3 MP floating pointer data structure . 62 4.4 MP configuration table data structure . 64 4.5 RSDP data structure . 66 4.6 RSDT data structure . 67 4.7 Read the IA32_APIC_BASE MSR . 69 4.8 Determine whether CPUID instruction is supported . 70 4.9 CPUID instruction . 70 4.10 Intel vendor ID . 71 4.11 Program code in order to retrieve Vendor identification string 72 4.12 Program code to determine the Hyper-Threading Techno- logy support . 73 4.13 Determine mask width of SMT_ID and Core_ID . 75 4.14 Determine number of processor’s cores . 76 4.15 Processor Topology Hierarchy Mapping .