Containers in HPC: Is It Worth It?

Universitat Politecnica` de Catalunya (UPC) - BarcelonaTech Facultat d'Informatica` de Barcelona (FIB) Master in Innovation and Research in Informatics High-Performance Computing Containers in HPC: Is it worth it? Author: Advisor: Kilian Peiro Filippo Mantovani Tutor: Co-advisor: Eduard Ayguade Marta Garcia-Gasulla Department: Company: Computer Architecture Barcelona Supercomputing Center June 2020 Abstract Container technologies usage has been growing in data centers and supercomputer facilities during the last years. More and more people are benefiting from the portability, flexibility and other traits of containers. Despite being some evaluation performed in the field of HPC, there is still a lot of research to be done on the matter. Also, the system-wide deployment and usage of a container infrastructure in an HPC system requires a non-negligible effort. Therefore a part of the study is also to analyze if the deployment effort is worth it. In this thesis I study the performance and energy consumption of Singularity containers with HPC workloads of real scientific applications (OpenFOAM, GROMACS) on emerging HPC ar- chitectures (Armv8). In addition, I make some suggestions on the Singularity deployment from a user and a System Admin- istrator perspective. Results show almost no performance nor energy differences between the executions of applications ran inside and outside of a container when tested on a small number of compute nodes. Containerization with Singularity seems overall a good tool to have in an HPC system. In appreciation for: My family, for the mutual affection and support through the years. My friends, for each and every one good moment. My coworkers, for unknowingly offering the best environment one could imagine. Music, for being and not being there as pleased. Contents 1 Introduction2 1.1 Competences............................3 2 Background5 2.1 HPC computing systems......................5 2.2 HPC cluster deployment......................6 2.3 HPC cluster usage.........................7 2.4 Environment modules.......................7 2.5 Virtualization............................7 2.6 Containerization..........................8 2.7 Benchmarks.............................9 2.8 Applications.............................9 2.9 State of the art........................... 10 3 Research questions 11 4 Implementation 13 4.1 Bare-metal Software Stack..................... 13 4.2 Singularity Image Setup...................... 13 4.3 Methodology............................ 14 4.4 Testing the OSU benchmarks................... 15 4.5 Testing OpenFOAM........................ 16 4.6 Testing GROMACS........................ 16 5 Evaluation 18 5.1 Startup time............................ 18 5.2 OSU Benchmarks.......................... 19 5.3 OpenFOAM............................. 20 5.4 GROMACS............................. 21 6 Conclusions 24 6.1 Further work............................ 25 Acronyms 26 Appendix A Reproducibility 27 1 1 Introduction This Master Thesis is the result of part of the work done in the Barcelona Supercomputing Center (BSC) 1 from February 2020 to June 2020. I work at BSC as a High-Performance Computing (HPC) System Administrator, and I am in charge of maintaining several Arm- based HPC clusters from the Mont-Blanc 2, Mont-Blanc 3 and Mont-Blanc 2020 European projects 2, as well as the hardware and software infrastructure for the European Processor Initiative (EPI) 3 project. Due to my experience with the Arm architecture, I have also collaborated in Mont-Blanc 3 Work Package 6, analyzing the performance of applications, mini-applications and benchmarking on the project test platforms. I have also formed part of the UPC-BSC ISC Student Cluster Competition 4 teams of 2018, 2019 and 2020. The first one as a student and the latter two as an advisor. These two environments have made me grow as a System Administrator but also as a researcher, searching for state-of-the-art methods in order to configure HPC clusters so that users can maximize the performance of their applications with minimal effort. Virtual Machines and Containers were developed many years ago for different purposes which include portability, Bring Your Own Environment (BYOE), composability and version control integration. Some of these traits are beneficial for the world of HPC, but both containers and Virtual Machines had a performance downside incompatible with the HPC vision. New HPC-oriented containers, such as Singularity, have been emerging recently, and it is our duty as researchers to test and measure if it is worth to use containers in the HPC world. It is also important to measure the energy consumption of these containers. In the thesis I try to tackle these challenges, by performing tests with two real scientific applications { OpenFOAM and GROMACS {, comparing bare-metal executions with executions inside a Singularity container. I selected OpenFOAM and GROMACS because they are complex applications that use a relevant fraction of computing time on several data centers around the globe, and also on the MareNostrum supercomputer at BSC. I also explain parts of the procedure of creating Docker and Singularity images, and how to run applications inside a container. All tests are performed in Arm-based clusters, since the Arm architecture grown its im- portance in HPC during the last months up to being top in the Top500 list in the recent edition of June 2020 with the Fugaku 5 supercomputer by Fujitsu, powered by Arm. My results show that the average setup time with Singularity is negligible (less than one second), and that in general the bare-metal version performs between 1% and 12% faster than the Singularity version. Also, the bare-metal version consumes a proportional fraction of energy less than the Singularity version. To conclude, the experiments show that Singularity is capable of running applications within the HPC standards, and obtaining results similar to the bare-metal version. From a point of view of a System Administrator, it can be a valuable software to install in an HPC machine, since users may benefit from the traits that Singularity offers. Nev- ertheless, throughout the thesis I have found several handicaps when using Singularity containers, so as a personal opinion, the software still needs more development before 1https://www.bsc.es/ 2https://www.montblanc-project.eu/project/presentation 3https://www.european-processor-initiative.eu/ 4https://www.isc-hpc.com/student-cluster-competition.html 5https://www.r-ccs.riken.jp/en/fugaku/project 2 becoming a production tool. The document is structured as follows: • Section1 introduces the thesis ideas and contents. • Section2 explains the background research done and the state of the art. • Section3 explains the ideas derived from the background and the research questions that guided my exploration. • Section4 explains all the setup and methodology done for the thesis experiments. • Section5 explains and analyzes the results obtained. • Section6 explains the final thoughts of the thesis, based on the results obtained. • Before the appendix, there is an acronyms section. • AppendixA is a guide on how to reproduce the work performed for this thesis. 1.1 Competences This thesis is part of the Master in Innovation and Research in Informatics (MIRI) 6 of the Facultat d’Informàticade Barcelona (FIB) 7 and Universitat Politècnicade Barcelona (UPC) 8, and it covers part of the list of competences of the Master. The list of competences covered is written below. Transversal Competences: • CTR3: Capacity of being able to work as a team member, either as a regular member or performing directive activities, in order to help the development of projects in a pragmatic manner and with sense of responsibility; capability to take into account the available resources. • CTR4: Capability to manage the acquisition, structuring, analysis and visualization of data and information in the area of informatics engineering, and critically assess the results of this effort. • CTR5: Capability to be motivated by professional achievement and to face new challenges, to have a broad vision of the possibilities of a career in the field of informatics engineering. Capability to be motivated by quality and continuous improvement, and to act strictly on professional development. Capability to adapt to technological or organizational changes. Capacity for working in absence of information and/or with time and/or resources constraints. • CTR6: Capacity for critical, logical and mathematical reasoning. Capability to solve problems in their area of study. Capacity for abstraction: the capability to create and use models that reflect real situations. Capability to design and implement simple experiments, and analyze and interpret their results. Capacity for analysis, synthesis and evaluation. 6https://www.fib.upc.edu/en/studies/masters 7https://www.fib.upc.edu/en/ 8https://www.upc.edu/en 3 • CB6: Ability to apply the acquired knowledge and capacity for solving problems in new or unknown environments within broader (or multidisciplinary) contexts related to their area of study. • CB8: Capability to communicate their conclusions, and the knowledge and rationale underpinning these, to both skilled and unskilled public in a clear and unambiguous way. • CB9: Possession of the learning skills that enable the students to continue studying in a way that will be mainly self-directed or autonomous. Generic Competences: • CG1: Capability to apply the scientific method to study and analyse of phenomena and systems in any

Load more