Calhoun: The NPS Institutional Archive Theses and Dissertations Thesis Collection 2004-09 Establishing Linux Clusters for high-performance computing (HPC) at NPS Daillidis, Christos Monterey, California. Naval Postgraduate School http://hdl.handle.net/10945/1445 NAVAL POSTGRADUATE SCHOOL MONTEREY, CALIFORNIA THESIS ESTABLISHING LINUX CLUSTERS FOR HIGH-PERFORMANCE COMPUTING (HPC) AT NPS by Christos Daillidis September 2004 Thesis Advisor: Don Brutzman Thesis Co Advisor: Don McGregor Approved for public release; distribution is unlimited THIS PAGE INTENTIONALLY LEFT BLANK REPORT DOCUMENTATION PAGE Form Approved OMB No. 0704-0188 Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for review- ing instruction, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the col- lection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188) Washington DC 20503. 1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED September 2004 Master’s Thesis 4. TITLE AND SUBTITLE: Establishing Linux Clusters for High-performance 5. FUNDING NUMBERS Computing (HPC) at NPS 6. AUTHOR(S) Christos Daillidis 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION Naval Postgraduate School REPORT NUMBER Monterey, CA 93943-5000 9. SPONSORING /MONITORING AGENCY NAME(S) AND 10. SPONSORING/MONITORING ADDRESS(ES) AGENCY REPORT NUMBER N/A 11. SUPPLEMENTARY NOTES The views expressed in this thesis are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government. 12a. DISTRIBUTION / AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE Approved for public release; distribution is unlimited 13. ABSTRACT Modeling and simulation (M&S) needs high-performance computing resources, but conventional supercomputers are both expensive and not necessarily well suited to M&S tasks. Discrete Event Simulation (DES) often involves repeated, independent runs of the same models with different input parameters. A system which is able to run many replications quickly is more useful than one in which a single monolithic application runs quickly. A loosely coupled parallel system is indicated. Inexpensive commodity hardware, high speed local area networking, and open source software have created the potential to create just such loosely coupled parallel systems. These systems are constructed from Linux-based computers and are called Beowulf clusters. This thesis presents an analysis of clusters in high-performance computing and establishes a testbed implementation at the MOVES Institute. It describes the steps necessary to create a cluster, factors to consider in selecting hardware and software, and de- scribes the process of creating applications that can run on the cluster. Monitoring the running cluster and system administration are also addressed. 14. SUBJECT TERMS Clusters, Beowulf, HPC, Rocks, HPL, BPS 15. NUMBER OF PAGES 186 16. PRICE CODE 17. SECURITY 18. SECURITY 19. SECURITY 20. LIMITATION OF CLASSIFICATION OF CLASSIFICATION OF THIS CLASSIFICATION OF ABSTRACT REPORT PAGE ABSTRACT Unclassified Unclassified Unclassified UL NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89) Prescribed by ANSI Std. 239-18 i THIS PAGE INTENTIONALLY LEFT BLANK ii Approved for public release; distribution is unlimited ESTABLISHING LINUX CLUSTERS FOR HIGH-PERFORMANCE COMPUTING (HPC) AT NPS Christos Daillidis Major, Hellenic Army B.S., Hellenic Military Academy, 1989 Submitted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE IN COMPUTER SCIENCE from the NAVAL POSTGRADUATE SCHOOL September 2004 Author: Christos Daillidis Approved by: Don Brutzman Thesis Advisor Don McGregor Thesis Co-Advisor Peter J. Denning Chairman, Department of Computer Science iii THIS PAGE INTENTIONALLY LEFT BLANK iv ABSTRACT Modeling and simulation (M&S) needs high-performance computing resources, but conventional supercomputers are both expensive and not necessarily well suited to M&S tasks. Discrete Event Simulation (DES) often involves repeated, independent runs of the same models with different input parameters. A system which is able to run many replications quickly is more useful than one in which a single monolithic application runs quickly. A loosely coupled parallel system is indicated. Inexpensive commodity hardware, high speed local area networking, and open source software have created the potential to create just such loosely coupled parallel sys- tems. These systems are constructed from Linux-based computers and are called Beo- wulf clusters. This thesis presents an analysis of clusters in high-performance computing and es- tablishes a testbed implementation at the MOVES Institute. It describes the steps neces- sary to create a cluster, factors to consider in selecting hardware and software, and de- scribes the process of creating applications that can run on the cluster. Monitoring the running cluster and system administration are also addressed. v THIS PAGE INTENTIONALLY LEFT BLANK vi TABLE OF CONTENTS I. INTRODUCTION........................................................................................................1 A. PROBLEM OVERVIEW................................................................................1 B. MOTIVATION ................................................................................................1 C. PROBLEM STATEMENT .............................................................................1 D. THESIS ORGANIZATION............................................................................2 II. BACKGROUND AND RELATED WORK..............................................................3 A. BACKGROUND ..............................................................................................3 B. RELATED WORK ..........................................................................................4 III. LINUX CLUSTERS FOR HPC..................................................................................5 A. BACKGROUND ..............................................................................................5 1. Introduction..........................................................................................5 2. Clusters Defined...................................................................................5 3. Symmetric Multiprocessing and Clusters..........................................7 4. Primary Benefits ..................................................................................9 5. Applications..........................................................................................9 6. Process Scheduler...............................................................................10 7. Message Passing Interface (MPI) .....................................................11 B. CLUSTER CLASSIFICATIONS.................................................................13 1. Classification by Type of Hardware.................................................13 2. Classification by Network Technology ............................................15 3. Classification by Size .........................................................................15 4. Classification by Shared Resources..................................................16 5. Classification by Cluster Architecture.............................................18 C. CLUSTER OPERATING SYSTEM............................................................18 1. Failure Management..........................................................................18 2. Load Balancing...................................................................................19 3. Parallelizing Computation ................................................................19 a. Parallelizing Compiler............................................................19 b. Parallelized Application..........................................................19 c. Parametric Computing............................................................19 D. CLUSTER ARCHITECTURE.....................................................................19 E. DIFFERENT CLUSTER INSTALLATIONS.............................................21 1. Windows Cluster Service ..................................................................21 2. Sun Cluster .........................................................................................21 3. Beowulf and Linux Clusters..............................................................22 F. HARDWARE CONSIDERATIONS............................................................24 1. CPU .....................................................................................................25 2. Memory Capabilities, Bandwidth and Latency ..............................28 3. I/O Channels.......................................................................................29 a. PCI and PCI-X........................................................................29 b. AGP..........................................................................................29 c. Legacy Buses...........................................................................30