HERCULES: High-Performance Real-Time Architectures for Low-Power Embedded Systems Project title: High-Performance Real-Time Architectures for Low-Power Embedded Systems Acronym: HERCULES Project ID: 688860 Call identifier: H2020 – ICT 04-2015 – Customized and low power computing Project coordinator: Prof. Marko Bertogna, University of Modena and Reggio Emilia D4.5: Multi-OS Integration and Virtualization Document title: Multi-OS Integration and Virtualization Version: 1.3 Deliverable No.: D4.5 Lead task beneficiary: EVI Partners involved: EVI, UNIMORE, CTU Author: C. Scordino, L. Cuomo, M. Solieri, M. Sojka Status: Final Date: 2018-12-30 Nature: S Dissemination level: PU – Public D4.5: Multi-OS Integration and Virtualization Purpose & Scope Deliverable D4.5 is a package containing the software for the concurrent execution of multiple operating systems (i.e., the Real-Time Linux developed in Task 4.1 and the Lightweight Real-Time Operating System (RTOS) de- veloped in Task 4.2) on the same multi-core System-on-Chip (SoC). In addition, the software implements proper mechanisms (namely, cache colouring, memguarding and co-scheduling) to limit the interference on shared hardware resources between the different Operating Systems (OSs). This document aims at illustrating the code developed, providing information about how to configure and use the provided software. The developed source code is available in the package provided along with this document. A subset of the developed code has been also released as open-source software [JailEvi, JailTX1, JailTX2]. The remaining part will we be similarly published after the project conclusion. Revision History Version Date Author/Reviewer Comments 0.1 2018-06-21 EVI First version 0.2 2018-06-26 UNIMORE LaTeX conversion 0.3 2018-07-27 UNIMORE Revised structure, added description and guide for cache color- ing support, fixed acronyms 1.0 2018-07-29 CTU Added MemGuard description 1.1 2018-10-30 UNIMORE Extended PREM guide, general revision 1.2 2018-10-30 UNIMORE Included reviewers comments 1.3 2018-12-30 UNIMORE Final revision HERCULES project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement: 688860. This document reflects only the author’s view and the EU Commission is not responsible for any use that may be made of the information it contains. Ref: D4.5 HERCULES: Horizon 2020 project #688860 Page i D4.5: Multi-OS Integration and Virtualization Contents Acronyms 1 1 Introduction 2 1.1 Objectives .......................................... 2 2 Hypervisor 3 2.1 Introduction ......................................... 3 2.2 Hypervisor Selection ..................................... 3 2.3 Jailhouse ........................................... 4 3 Contributions 6 3.1 Porting on the Reference Platforms .............................. 6 3.2 Inter-Guest Communication Mechanism ........................... 6 3.3 Cache Coloring ........................................ 7 3.4 MemGuard .......................................... 7 3.5 Memory Co-Scheduling .................................... 8 4 User’s Guide 9 4.1 Hypervisor Installation .................................... 9 4.2 Cache Coloring Configuration ................................. 9 4.2.1 Hypervisor configuration ............................... 9 4.2.2 Preliminaries .................................... 9 4.2.3 Color Definition Configuration ............................ 10 4.2.4 Color Assignment Configuration ........................... 11 4.3 MemGuard Configuration ................................... 12 4.4 Memory Co-Scheduling Configuration ............................ 14 Ref: D4.5 HERCULES: Horizon 2020 project #688860 Page ii D4.5: Multi-OS Integration and Virtualization Acronyms API Application Program Interface . 6 ARXML AUTOSAR XML . 7 AUTOSAR AUTomotive Open System ARchitecture . 3 ECU Electronic Control Unit . 3 CPU Central Processing Unit . 4 DRAM Dynamic Random Access Memory . 14 GA Grant Agreement . 2 GPOS General-Purpose Operating System . 3 GNU GPL GNU General Public License . 4 I/O Input/Output . 5 LLC Last-Level Cache . 7 MPSoC MultiProcessor System-on-Chip . 10 PREM PReditable Execution Model . 14 OS Operating System . i OSEK/VDX Offene Systeme und deren Schnittstellen für die Elektronik in Kraftfahrzeugen / Vehicle Distributed eXecutive . 3 RAM Random-Access Memory . 4 RTE RunTime Environment . 7 RTOS Real-Time Operating System . i SDRAM Synchronous Dynamic Random-Access memory . 5 SoC System-on-Chip. .i US+ Xilinx Zinq UltraScale+. .7 Ref: D4.5 HERCULES: Horizon 2020 project #688860 Page 1 D4.5: Multi-OS Integration and Virtualization 1 Introduction 1.1 Objectives As illustrated in the Grant Agreement (GA), [Task 4.5 of HERCULES] aims at investigating, designing and developing efficient mechanisms for the concurrent execution of multiple OS on the same SoC. In particular, it focuses on supporting the concurrent execution of the Linux and ERIKA Enterprise OS on the reference platform selected in WP2. Besides the mere concurrent execution of different OSs on the same platform, the task also investigates and designs mechanisms for an efficient synchronization and communication of data between the OSs. As part of this task, several activities have been carried out by the consortium: • The investigation and selection of a suitable hypervisor for the envisioned scenario. • The porting of the selected hypervisor onto the reference platforms. • The design and implementation of the cache colouring mechanism (resulting from Task 5.1). • The design and implementation of the co-scheduling mechanism (resulting from Task 5.3). • The design and implementation of the memguarding mechanism (resulting from Task 5.3) This document will describe all these parallel activities, providing insights about the implementation and information about using the software. Ref: D4.5 HERCULES: Horizon 2020 project #688860 Page 2 D4.5: Multi-OS Integration and Virtualization Figure 1: Overall software infrastructure. 2 Hypervisor 2.1 Introduction Software running safety-critical tasks must be developed according to certified development processes, and to strict safety standards. This process aims at preventing possible injuries due to misbehaving or faulty software. In the case of automotive Electronic Control Units (ECUs), in particular, the Offene Systeme und deren Schnittstellen für die Elektronik in Kraftfahrzeugen / Vehicle Distributed eXecutive (OSEK/VDX) and the newer AUTomotive Open System ARchitecture (AUTOSAR) standards already established a set of rules and constraints for operating systems design, including memory and timing protection, and stack monitoring. These standards impose the usage of a RTOS implementing a set of well-known scheduling algorithms and techniques. Usually, the size of the RTOS is kept at a bare minimum to reduce the code complexity (hence the number of possible bugs) and the costs of the verification process. In addition to this, there is an ever growing interest from the industry for using the advanced features and potential of modern computing platforms, including networking and graphics subsystems, typically supported by a General-Purpose Operating System (GPOS) like Linux. Finally, there is an increased need to integrate together on the same SoC legacy automotive applications based on such OSEK/VDX or AUTOSAR standards, without radically changing the application source code. A hypervisor solution like the one devised in HERCULES is able to fulfill these three requirements, allowing the coexistence of different operating system, guaranteed real-time behavior and integration of legacy applications. In particular, the HERCULES project has chosen to implement a Multi-OS paradigm in which the Linux GPOS and the ERIKA Enterprise RTOS are executed concurrently under the supervision of a hypervisor for proper isolation of the safety-critical activities. The overall infrastructure is the one illustrated in Figure 1. 2.2 Hypervisor Selection Several interesting open-source technologies were available to be used as the basic brick for the hypervisor to be developed in HERCULES. Among the most interesting and actively maintained projects, we can cite: • Jailhouse [KSa18] Ref: D4.5 HERCULES: Horizon 2020 project #688860 Page 3 D4.5: Multi-OS Integration and Virtualization • KVM [KVM; Wik; Red] • Xen [Xen] • Xvisor [Xvia; Xvib] When selecting the most suitable hypervisor for HERCULES project’s scope, we focused on the following re- quirements: • Support for the reference hardware; i.e. ARM. • Real-time capabilities and, especially, the possibility of handling low-latency control). • Availability of the source code under some kind of open-source license (e.g. BSD license, GNU General Public License (GNU GPL), MIT licence). • Good project activity by the development and maintaining community. • Small codebase, which paves the way to a possible certification. These requirements have allowed to narrow the investigation, and eventually to choose Jailhouse. 2.3 Jailhouse Jailhouse [KSa18; Lin] is a project aiming at creating a small and lightweight hypervisor targeting industrial- grade applications. Despite its youth (born in 2013), the project is sponsored by a big company (i.e. Siemens), it received support into the official Linux kernel, and it is getting visibility and traction in the open-source community. The hypervisor supports both the x86-64 architecture, employed by both Intel and AMD, and the 32-bit and 64-bit variants of ARM architectures, provided that the instruction