A Lightweight Virtualization Layer with Hardware-Assistance for Embedded Systems
Total Page:16
File Type:pdf, Size:1020Kb
PONTIFICAL CATHOLIC UNIVERSITY OF RIO GRANDE DO SUL FACULTY OF INFORMATICS COMPUTER SCIENCE GRADUATE PROGRAM A LIGHTWEIGHT VIRTUALIZATION LAYER WITH HARDWARE-ASSISTANCE FOR EMBEDDED SYSTEMS CARLOS ROBERTO MORATELLI Dissertation submitted to the Pontifical Catholic University of Rio Grande do Sul in partial fullfillment of the requirements for the degree of Ph. D. in Computer Science. Advisor: Prof. Fabiano Hessel Porto Alegre 2016 To my family and friends. “I’m doing a (free) operating system (just a hobby, won’t be big and professional like gnu) for 386(486) AT clones.” (Linus Torvalds) ACKNOWLEDGMENTS I would like to express my sincere gratitude to those who helped me throughout all my Ph.D. years and made this dissertation possible. First of all, I would like to thank my advisor, Prof. Fabiano Passuelo Hessel, who has given me the opportunity to undertake a Ph.D. and provided me invaluable guidance and support in my Ph.D. and in my academic life in general. Thank you to all the Ph.D. committee members – Prof. Carlos Eduardo Pereira (dissertation proposal), Prof. Rodolfo Jardim de Azevedo, Prof. Rômulo Silva de Oliveira and Prof. Tiago Ferreto - for the time invested and for the valuable feedback provided.Thank you Dr. Luca Carloni and the other SLD team members at the Columbia University in the City of New York for receiving me and giving me the opportunity to work with them during my ‘sandwich’ research internship. Eu gostaria de agraceder minha esposa, Ana Claudia, por ter estado ao meu lado durante todo o meu período de doutorado. O seu apoio e compreensão foram e continuam sendo muito importantes para mim. Obrigado pela paciência, amor e carinho durante todos esses anos. Também gostaria de agradecer aos meus pais e irmão pelo apoio durante toda a minha vida. Agradeço aos colegas e amigos do laboratório GSE/PUCRS, especialmente ao amigo Sergio Johann Filho pelas importantes discusões durante a execução do trabalho. UMA CAMADA LEVE DE VIRTUALIZAÇÃO ASSISTIDA POR HARDWARE PARA SISTEMAS EMBARCADOS RESUMO O poder de processamento presente nos sistemas embarcados modernos permite a adoção de técnicas de virtualização. Juntamente com os ganhos em redução de custo e melhor utilização dos recursos, como por exemplo uma melhor utilização do processa- dor, a virtualização possibilita a co-execução de diferentes sistemas operacionais em um processador, sejam eles sistemas operacionais de tempo real (RTOS) e/ou de propósito geral (GPOS). A implementação da técnica de virtualização esta baseada em um módulo de software denominado hypervisor. Devido a complexidade de se desenvolver uma nova camada de virtualização es- pecialmente projetada para sistemas embarcados, muitos autores propuseram modifica- ções em sistemas de virtualização que são largamente empregados em servidores na nu- vem para melhor adapta-los às necessidades dos sistemas embarcados. Contudo, a utili- zação de memória e os requisitos temporais de alguns dispositivos embarcados requerem abordagens diferentes daquelas utilizadas em servidores. Além disso, a atual tendência de utilização de virtualização nos dispositivos projetados para a internet das coisas (do inglês Internet of Things - IoT) aumentou o desafio por hypervisors mais eficientes, em termos de memória e processamento. Estes fatores motivaram o surgimento de diversos hypervisors especialmente projetados para atender os requisitos dos atuais sistemas embarcados. Nesta tese, investigou-se como a virtualização embarcada pode ser melhorada a partir de seu estado atual de desenvolvimento para atender as necessidades dos sistemas embarcados atuais. Como resultado, propõe-se um modelo de virtualização capaz de agre- gar os diferentes aspectos exigidos pelos sistemas embarcados. O modelo combina virtu- alização completa e para-virtualização em uma camada de virtualização híbrida, além da utilização de virtualização assistida por hardware. Uma implementação baseada neste mo- delo é apresentada e avaliada. Os resultados mostram que o hypervisor resultante possui requisitos de memória compatíveis com os dipositivos projetados para IoT. Ainda, GPOSs and RTOS podem ser executados mantendo-se o isolamento temporal entre eles e com o baixo impacto no desempenho. Palavras-Chave: Virtualização, Sistemas Embarcados, Hypervisor, Tempo-real. A LIGHTWEIGHT VIRTUALIZATION LAYER WITH HARDWARE-ASSISTANCE FOR EMBEDDED SYSTEMS ABSTRACT The current processing power of modern embedded systems enable the adoption of virtualization techniques. In addition to the direct relationship with cost reduction and bet- ter resource utilization, virtualization permits the integration of real-time operating systems (RTOS) and general-purpose operating systems (GPOS) on the same hardware system. The resulting system may inherit deterministic time response from the RTOS and a large software base from the GPOS. However, the hypervisor must be carefully designed. Due to the complexity of developing a virtualization layer designed specially for em- bedded systems from scratch, many authors have proposed modifications of the widely used server virtualization software to better adapt it to the particular needs of embedded system. However, footprint and temporal requisites of some embedded devices require different ap- proaches than those used in server farms. Also, currently virtualization is being adapted for the field of the Internet of Things (IoT), which has increased the challenge for more efficient hypervisors. Thus, a generation of hypervisors focused on the needs of embedded systems have emerged . This dissertation investigated how embedded virtualization can be improved, start- ing from the current stage of its development. As a result, it is proposed a virtualization model to aggregate different aspects required by embedded systems. The model combines full and para-virtualization in a hybrid virtualization layer. In addition, it explores the newer features of embedded processors that have recently adopted hardware-assisted virtualiza- tion. A hypervisor implementation based on this model is presented and evaluated. The results show that the implemented hypervisor has memory requirements compatible with devices designed for IoT. Moreover, general-purpose operating systems and real-time tasks can be combined while keeping them temporally isolated. Finally, the overall virtualization overhead is for most part lower than in other embedded hypervisors. Keywords: Virtualization, Embedded Systems, Hypervisor, Real-time. LIST OF FIGURES 2.1 A taxonomy of Virtual Machines. Classic-system VMs are the subject of study of this dissertation. Adapted from [70]. 34 2.2 Types of hypervisors. 36 2.3 Illustrating Popek and Goldbergs’s Theorem. Adapted from [70]. 38 2.4 Relation between hypercalls on a virtualized system and syscalls on a typi- cal operating system. 39 2.5 Supervisor privileged level for hypervisor execution. 42 2.6 Memory virtualization approaches in non-virtualized versus virtualized sys- tems......................................................... 43 2.7 Comparison of the IOMMU to the MMU. 45 3.1 Xen’s Architecture. Adapted from [9]. 52 3.2 KVM’s Architecture. 59 3.3 SPUMONE’s single-core Architecture. Adapted from [39]. 66 3.4 SPUMONE’s multicore Architecture. Adapted from [51]. 66 3.5 XtratuM’s architecture. It is a type 1 hypervisor designed for safety critical systems. Adapted from [19]. 68 3.6 Summarized L4 microkernel family tree that resulted in embedded hypervi- sors. Adapted from [25]. 69 3.7 OKL4’s architecture. It was developed targeting mobile devices supporting several mobile OSs, and it powered the first commercial virtualized phone in2009....................................................... 70 3.8 Fiasco’s architecture. The L4/Fiasco maps the L4Linux kernel threads di- rectly to VCPUs. 71 3.9 PikeOS’s architecture. It is a proprietary microkernel with support for virtu- alization designed to attend safety-critical systems. 72 3.10 Xvisor’s architecture. The Xvisor hypervisor uses the third privileged level from ARM’s architecture. 74 3.11 RT-Linux is a kernel patch that adds a software layer with a real-time sched- uler and it makes the Linux kernel to share the processor with real-time tasks. ....................................................... 77 4.1 Overall view of the proposed virtualization model. 83 4.2 Flexible mapping for multiprocessor embedded systems with real-time sup- port. ........................................................ 85 4.3 Virtual Hellfire Hypervisor’s virtualization model. 89 5.1 MIPS32 memory map. 95 5.2 Hypervisor’s software block diagram. 95 5.3 Example of the scheduling strategy to combine the EDF and best-effort schedulers. 100 5.4 Virtual memory organization view of our hypervisor. 102 5.5 Quantum scheduler scheme for interrupt delivery. 104 5.6 Example of inter-VM communication involving two guests. 106 6.1 Performance overhead for user-land applications relative to non-virtualized performance. 118 6.2 Performance overhead for syscall applications relative to non-virtualized performance. 118 6.3 Iperf bandwidth results for TCP protocol comparing native versus virtualized execution with different hypervisor’s scheduler quantum. 119 6.4 Performance overhead for CPU intensive applications relative to non-virtualized performance. 120 6.5 Histogram for RTT of the messages for native execution of the Linux/MIPS. 123 6.6 Histogram for RTT of the messages without policy. 123 6.7 Histogram for RTT of the messages with fast interrupt policy and quantum recycling. 124 6.8 Histogram for RTT of the messages