Proc. of 2016 3rd Int. Conf. on Information Tech., Computer, and Electrical Engineering (ICITACEE), Oct 19-21st, 2016, Semarang, Indonesia Commodity Cluster Using Single System Image Based on Linux/Kerrighed for High-Performance Computing Iwan Setiawan and Eko Murdyantoro Department of Electrical Engineering Universitas Jenderal Soedirman Purwokerto, Indonesia {stwn, eko.murdyantoro}@unsoed.ac.id Abstract—Commodity Information Technology (IT) applications with 16 PC-based homogeneous nodes. It has been infrastructure, including hardware, software, and networking, is inspiring academics, researchers, and practitioners to build commonly used in computing facilities, such as computer their own clusters for HPC with commodity infrastructure, laboratory. Unfortunately, the infrastructure in that particular including software that is commonly Free/Libre/Open Source place need to be updated and upgraded regularly to cater to the Software (FLOSS). For instance, Linux for the OS and MPICH demands of users and applications for more and more computing for the parallel library, one of the implementations of Message resources, especially in scientific computation. The existing Passing Interface (MPI). infrastructure can be utilized to tackle the issue by forming it into cluster computing. We report our exploration results in Many organizations, specifically universities and research constructing Vincster, a low-cost and scalable commodity cluster institutes, employ commodity IT infrastructure in their for high-performance computing (HPC) to utilize the existing IT computing facilities, e.g. computer laboratory. One of the infrastructure in a computer laboratory. The cluster uses one issues in computer laboratory is how to keep up with the additional computer to serve as a head node containing a Single increasingly requests of more computing resources for users System Image (SSI) Operating System (OS) based on Linux with and their applications, especially in scientific computation. Kerrighed as the kernel-level SSI support, along with Hence, the infrastructure in that site need to be updated and middleware and application softwares that build up the SSI upgraded regularly to meet the demands. The existing approach. Vincster has been constructed and implemented with 7 infrastructure can be utilized to overcome the issue by forming SSI key services/features utilizing 17 compute nodes from 20 it into scalable commodity cluster for HPC at minimal cost. available computers. We evaluated computational performance of the cluster by running parallel matrix multiplication program. However, there is a challenge in building cluster computing with traditional method, i.e., to administer the cluster manually Keywords—commodity cluster; high-performance computing; or with tools including installation and maintenance, since the single system image; linux; kerrighed; cots; computer laboratory compute nodes with their computing resources are distributed. It is not easy to make the nodes coherent and identical to one I. INTRODUCTION another, and disk replication technique that applies across Nowadays, the need of computing resources is increasingly nodes is not flexible. Thus, users and applications still see demanding because of the big amount of data to be handled cluster as a system with distributed resources, not as a single and more CPU cycles to be used. Barroso, Dean, and Hӧ lzle and unified system. [1] stated that “on average, a single query on Google reads Single System Image (SSI), “the property of a system that hundreds of megabytes of data and consumes tens of billions of hides the distributed and heterogeneous of the available CPU cycles.” This trend coupled with the high cost and low resources and presents them to users and applications as a accessibility of traditional supercomputers, along with current single unified computing resource” as described by Buyya, advancement of high-performance microprocessors, high-speed Cortes, and Jin [5], is an approach to tackle the issue networks, and standard tools for HPC, makes commodity aforestated. By abstracting the distributed resources, the cluster cluster, cluster computing that uses fully commodity will be easier to use, administer, and maintain. SSI can be components or commodity-off-the-self (COTS) [2] such as implemented at different levels of abstraction: hardware, Personal Computer (PC), appealing solution for cost-effective OS/kernel, middleware, and application [3] [5] [6]. Each parallel computing [3]. The increasing need of computing implementation level has its advantages and disadvantages. resources trend itself is not just for commercial applications but The more low level the implementation, the more difficult the also computational science applications [3]. implementation has to be constructed, but with more Beowulf is an example of commodity cluster [2], which transparency to the higher levels of the system layer. was first built by Sterling et al. [4] for Earth and space science Cooperation between levels is needed to build a good SSI [5]. 978-1-5090-0890-2/16/$31.00 c 2016 IEEE 367 There is a study of three implementations of SSI OS based the cluster should include important or key SSI on Linux kernel, namely openMosix, OpenSSI, and Kerrighed. services/features in [5]. We attempt to incorporate any This study was conducted by Lottiaux, Gallard, Vallée, Morin, accessible methods to include the features into the cluster. The and Boissinot [7]. According to the study, Kerrighed offers the second one is the cluster need to be scalable in terms of best performance, notably regarding its Inter-Process flexibility for nodes to join or disjoin to the cluster (loosely Communication (IPC) and filesystem. Kerrighed mainly coupled system). The third one is the cluster should be low-cost targeted in kernel-level SSI providing the most transparent SSI, by minimizing efforts in buying, changing, and reconfiguring compared to other patchset-based implementations [6]. It also existing commodity IT infrastructure. has interesting features, e.g. cluster-wide process management, support for cluster-wide shared memory, cluster filesystem, Furthermore, there are environment characteristics in transparent process checkpointing, high-availability for user computer laboratory that should be addressed, related to the applications, and customizable SSI features. It supports several construction of the cluster. These characteristics include hardware architectures, i386/x86-32/IA32 for version <= 2.3.0, number of compute nodes and the heterogeneity of their and x86-64 for version >=2.4.0 [8]. specifications, including the types and number of cores/processors on each node. As a case study, we conducted Armay, Zulfikar, and Simaremare [9] have built a this research in a computer laboratory of Department of commodity cluster with 9 PC-based homogeneous nodes in a Engineering (now Faculty of Engineering), Universitas computer laboratory with traditional method. There is no single Jenderal Soedirman (Unsoed). We started this research by and unified-view of the cluster, since they do not use SSI doing survey of the existing commodity infrastructure in that approach. Moreover, they demonstrate a test using MPICH1 to laboratory to formulate the design and develop system proof that parallel processing can be run on the cluster, but architecture of the cluster. there is no computational performance presented in the study. Sandhya and Raju [10] built a commodity cluster using SSI Lastly, we use “Vincster” as the name of the cluster that approach based on Linux/Kerrighed with four PC-based nodes. stands for “vintage commodity cluster.” It represents our hope These nodes have two different specifications in processor and of this cluster’s goal, that is bringing back to life outdated memory. They report evaluation results of computational computing resources to harvest more its values. performance for 10 to 100 procesess computing the value of pi on the cluster, but they do not explain how those processes run III. SYSTEM DESCRIPTION on each of the compute nodes. In the study, the relation In this section, we describe the system architecture of between performance of the cluster, number of compute nodes Vincster, and also the cost needed for its construction. It and their cores/processors, along with various problem sizes, is consists of a head node, and a number of compute nodes. The not described. head node provides network services for the compute nodes, We attempt to explore on constructing cluster that addresses including network booting via Preboot Execution Environment the issues aforementioned. To summarize, our contributions are (PXE), Dynamic Host Configuration Protocol (DHCP) for as follows. assigning IP addresses dynamically, Trivial File Transfer Protocol (TFTP) for transfering kernel/OS image to compute 1. We revisit Beowulf/commodity cluster for HPC combined nodes, Network File System (NFS) for provisioning the root with SSI approach to address the issues by designing and filesystem and sharing storage that are needed to run full- constructing Vincster, a low-cost and scalable cluster based fledged SSI OS on compute nodes. on Linux OS with Kerrighed as the kernel-level SSI support together with middleware and application softwares that We use network booting and centralized storage technique build up the SSI approach. As a case study, we implement in Vincster. All PCs in the laboratory are booted by loading the design and deploy the commodity cluster over existing PXE first from ROM/BIOS or flash drive.