Cluster Computing: Architectures, Operating Systems, Parallel Processing & Programming Languages
Total Page:16
File Type:pdf, Size:1020Kb
Cluster Computing Architectures, Operating Systems, Parallel Processing & Programming Languages Author Name: Richard S. Morrison Revision Version 2.4, Monday, 28 April 2003 Copyright © Richard S. Morrison 1998 – 2003 This document is distributed under the GNU General Public Licence [39] Print date: Tuesday, 28 April 2003 Document owner: Richard S. Morrison, [email protected] ✈ +612-9928-6881 Document name: CLUSTER_COMPUTING_THEORY Stored: (\\RSM\FURTHER_RESEARCH\CLUSTER_COMPUTING) Revision Version 2.4 Copyright © 2003 Synopsis & Acknolegdements My interest in Supercomputing through the use of clusters has been long standing and was initially sparked by an article in Electronic Design [33] in August 1998 on the Avalon Beowulf Cluster [24]. Between August 1998 and August 1999 I gathered information from websites and parallel research groups. This culminated in September 1999 when I organised the collected material and wove a common thread through the subject matter producing two handbooks for my own use on cluster computing. Each handbook is of considerable length, which was governed by the wealth of information and research conducted in this area over the last 5 years. The cover the handbooks are shown in Figure 1-1 below. Figure 1-1 – Author Compiled Beowulf Class 1 Handbooks Through my experimentation using the Linux Operating system and the undertaking of the University of Technology, Sydney (UTS) undergraduate subject Operating Systems in Autumn Semester 1999 with Noel Carmody, a systems level focus was developed and is the core element of this material contained in this document. This led to my membership to the IEEE and the IEEE Technical Committee on Parallel Processing, where I am able to gather and contribute information and be kept up to date on the latest issues. My initial interest in the topic has developed into a very practical as well as detailed theoretical knowledge. In Autumn semester 2001 I undertook to tutor the UTS Operating Systems subject which included guest lectures. This enabled me to further develop my ability to articulate my knowledge to students with no previous experience of the internals of operating systems or systems level programming and facilitate their learning. Cluster Computing Page 3 Revision Version 2.4 Copyright © 2003 Focus This document reviews the current parallel systems theory with respect to Beowulf type clustering and experimentally investigates certain elements of parallel systems performance. The investigation work was carried out at the University of Technology, Sydney in Laboratory 1/2122E. A separate document could be written on developing applications for parallel systems and currently it would be largely dependant on the target system. I have however focused on the systems level as it has been apparent over the years that I have been researching the area that while Scientific Computing drove the HPCC to develop clustering, Corporations are now requiring the same level of advantages as possible with Clusters. As such, Operating system vendors are now taking the experience of the HPCC and integrating these features into their operating systems. Next generation tools are available now to develop parallel programs and it is envisioned that parallel systems will be the only model in the future. Before this can happen, standardisation must be reached (Formal or Pseudo), as this will be an important step to minimise the transitional cost to parallel software. [Refer to Section 3.4.2 for further detail] Page 4 Cluster Computing Revision Version 2.4 Copyright © 2003 Executive Summary Cluster Computing: Architectures, Operating Systems, Parallel Processing & Programming Languages The use of computers within our society has developed from the very first usage in 1945 when the modern computer era began, until about 1985 when computers were large and expensive. Two modern age technologies, the development of high-speed networking and the personal computer have allowed us to break-down these price barriers and construct cost effective clusters of PCs which provide comparable performance to super- computers at a fraction of the cost. As PC’s and networks are in common use, this allows most commercial organizations, governments, and educational institutions access to high performance super-computers. The major difference between a network of PC’s and a super-computer is the software which is loaded on each machine, and the way in which an application is processed, namely in parallel. Parallel processing is the method of breaking down problems or work into smaller components to be processed in parallel thus taking only a fraction of the time it would take to run on a stand-alone PC. The only drawback to this cost-effective way of computing is how can we effectively design these systems to meet our performance needs? Can widely used operating systems be used such as Windows? What software is available for users on this type of machine and how do we run this software on other machines built using the same technology? Can we use existing applications or do we need to develop new ones, if so how? How can we ensure that each PC is doing its fair share of work, or is not overloaded? This document explores these issues from theory to practice, details a design methodology and shows by experimental investigation that from a structured design the speedup obtained with many PC’s can be known within bounds prior to implementation. To achieve our initial cost-effectiveness the Linux Operating system is used, however Windows NT can be used if desired while still maintaining a competitive edge over traditional super-computers. Additionally programming languages are available that abstract from the system and free the programmer up from worrying about system details. Richard S. Morrison B.E. (Computer Systems) Hons MIEEE, MIEAust February 2003 Cluster Computing Page 5 Revision Version 2.4 Copyright © 2003 Contents 1. INTRODUCTION.......................................................................................................................................12 1.1. BRIEF HISTORY OF COMPUTING AND NETWORKING ..............................................................................12 1.2. PARALLEL PROCESSING .........................................................................................................................12 1.3. MOTIVATION..........................................................................................................................................13 1.3.1. Applications of Parallel Processing..............................................................................................14 2. ARCHITECTURES....................................................................................................................................17 2.1. COMPUTER CLASSIFICATION SCHEMES..................................................................................................17 2.2. CLUSTER COMPUTING CLASSIFICATION SCHEMES.................................................................................21 2.3. BEOWULF...............................................................................................................................................22 2.3.1. History...........................................................................................................................................22 2.3.2. Overview .......................................................................................................................................23 2.3.3. Classification.................................................................................................................................24 2.4. NOW/COW...........................................................................................................................................25 2.5. DISTRIBUTED VS. CENTRALIZED SYSTEMS ............................................................................................26 3. SYSTEM DESIGN......................................................................................................................................28 3.1. PERFORMANCE REQUIREMENTS.............................................................................................................29 3.1.1. The Need for Performance Evaluation..........................................................................................29 3.1.2. Performance Indices of Parallel Computation..............................................................................30 3.1.3. Theoretical Performance of Parallel Computers ..........................................................................31 3.1.4. Performance Analysis and Measurement......................................................................................36 3.1.5. Practical Performance of Parallel Computers..............................................................................36 3.2. HARDWARE PLATFORMS........................................................................................................................38 3.2.1. CPU...............................................................................................................................................38 3.2.2. Symmetric Multiprocessing ...........................................................................................................38 3.2.3. Basic Network Architectures.........................................................................................................40 3.2.3.1. Network Channel Bonding ........................................................................................................................42