View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by University of Huddersfield Repository University of Huddersfield Repository Liang, Shuo, Holmes, Violeta and Kureshi, Ibad Hybrid Computer Cluster with High Flexibility Original Citation Liang, Shuo, Holmes, Violeta and Kureshi, Ibad (2012) Hybrid Computer Cluster with High Flexibility. In: IEEE Cluster 2012, 24-28 September 2012, Beijing, China. This version is available at http://eprints.hud.ac.uk/15840/ The University Repository is a digital collection of the research output of the University, available on Open Access. Copyright and Moral Rights for the items on this site are retained by the individual author and/or other copyright owners. Users may access full items free of charge; copies of full text items generally can be reproduced, displayed or performed and given to third parties in any format or medium for personal research or study, educational or not-for-profit purposes without prior permission or charge, provided: • The authors, title and full bibliographic details is credited in any copy; • A hyperlink and/or URL is included for the original metadata page; and • The content is not changed in any way. For more information, including our policy and submission procedure, please contact the Repository Team at: [email protected]. http://eprints.hud.ac.uk/ Hybrid Computer Cluster with High Flexibility Shuo Liang Dr Violeta Holmes Ibad Kureshi School of Computing and Engineering School of Computing and Engineering School of Computing and Engineering University of Huddersfield University of Huddersfield University of Huddersfield Huddersfield, United Kingdom Huddersfield, United Kingdom Huddersfield, United Kingdom Email: [email protected] Email: [email protected] Email: [email protected] Abstract—In this paper we present a cluster middleware, the others support multi-platform, supplying different user designed to implement a Linux-Windows Hybrid HPC Cluster, interfaces and document support. which not only holds the characteristics of both operating sys- One way of running these applications on different operat- tem, but also accepts and schedules jobs in both environments. Beowulf Clusters have become an economical and practical ing systems is to divide a computer cluster into smaller sub- choice for small-and-medium-sized institutions to provide High clusters for each platform, which would lead to a duplication Performance Computing (HPC)resources. The HPC resources and poor utilisation of the resources. Another solution is to are required for running simulations, image rendering and implement virtualised systems. Although the latest virtuali- other calculations, and to support the software requiring a sation technology could be a better alternative, it cannot be specific operating system. To support the software, small- scale computer clusters would have to be divided in two implemented effectively on legacy hardware. Hence, the aim or more clusters if they are to run on a single operating of our research work is to devise and implement a system system. The x86 virtualisation technology would help running that will enable better utilisation of the HPC resources and multiple operating systems on one computer, but only with support multi-platform applications used by our research the latest hardware which many legacy Beowulf clusters do community. not have. To aid the institutions, who rely on legacy non- virtualisation-supported facilities rather than high-end HPC resources, we have developed and deployed a bi-stable hybrid Software Name Description OS Abaqus Finite Element Analysis L system built around Linux CentOS 5.5 with the improved Amber Assisted Model Building with Energy L OSCAR middleware; and Windows Server 2008 and Windows Refinement aimed at biological sys- HPC 2008 R2. This hybrid cluster is utilised as part of the tems University of Huddersfield campus grid. Backburner Rendering software for 3ds Max W Blender Open Source 3D Modeller and Ren- L Keywords-cluster middleware, computer cluster, job sched- derer uler, resource manager CASTEP CAmbridge Sequential Total Energy L Package I. INTRODUCTION COMSOL Multiphysics Modelling, Finite Ele- W&L ment Analysis, Engineering Simulation This paper is motivated by a growing need for High- Software DL POLY General purpose classical molecular L Performance Computing (HPC) resources in Higher Educa- dynamics (MD) simulation software tion Institutions and the development of computer clusters ANSYS FLUENT Computational Fluid Dynamics (CFD) W&L built from Commercial off-the-shelf components (COTS) GAMESS-UK Molecular QM code L GULP General Utility Lattice Program L using open source cluster middleware. Building computer LAMMPS Large-scale Atomic/Molecular Mas- L clusters with legacy x86 hardware is an effective and practi- sively Parallel Simulator cal choice for providing affordable HPC resources. In order MATLAB Numerical Computing Environment W&L METADISE Minimum Energy Techniques Applied L to cater for the variety of user demands, those Beowulf to Defects, Interfaces and Surface En- clusters are expected to support applications that require ergies different hardware and software platforms. NWChem Multi-purpose QM and MM code L Opera Finite Element Analysis for Electro- W Many universities in the UK started to establish their magnetics departmental and campus clusters in the last decade, e.g. a Table I computer cluster at the University of Nottingham was built APPLICATIONS ON THE HUDDERSFIELD CAMPUS CLUSTER in 2004/5, and it had hit the June 2005 TOP500. [1] (W:WINDOWS, L:LINUX) In the University of Huddersfield there are a number of research groups requiring HPC resources for molecular and fluid dynamics simulation, 3D rendering, etc. using software Our initial objective was to devise a simple dual-boot such as DL-POLY, ANSYS, and Backburner. Some packages system to support both Windows and Linux platforms for can be run on Linux or Windows operating systems while the software in table I. A number of case studies were conducted to identify the efficiency and ease of use of these However, hardware support was not provided for their entire software from the scientific and administrative users point range of products. of view. Based on the results from our case studies and A multi-boot computer could be implemented by various the initial dual-boot system evaluation, the improved hybrid approaches, such as: system design had been developed. This system provides 1) Changing active partition more effective multi-platform middleware, which utilises 2) Multi-system bootloader, e.g. GRUB (for Linux as existing hardware to provide the maximum performance in main system), GRUB4DOS (for Windows as main both Linux and Windows based environments. system) The rest of the paper is organised as follows: Section II 3) PXE also can enable multi-boot function introduces our motivation; Section III describes the initial Pros: Multi-boot solution has wide range compatibility simple dual-boot system design and implementation; Section of hardware. It does not require new technology, and there IV describes our proposed hybrid system design. Section V is no performance reduction. summarises the results and outlines possible future develop- Cons: Reboot takes time, normally about 5 mins. ment. A Beowulf cluster at the University of Huddersfield was II. MOTIVATION built from re-used laboratory computers with Intel CoreTM 2 Quad-core Q8200 processor that have no virtualisation As a medium-sized Higher Education institution, the support. Hence, we had to consider other ways to enable University of Huddersfield is still finding its place within the the multi-platform cluster. research community. The HPC Computing Resources centre A multi-boot approach is in our opinion, better suited for was built to aid new research efforts requiring powerful com- the legacy machines that have no hardware virtualisation putational resources. As part of the University Queensgate support. This system could sit alongside High-end HPC campus grid project, a number of computer clusters were resources making an efficient use of legacy machines. built with Windows and Linux operating systems to cover the requirements of Linux Scientific Computing Software III. THE INITIAL DUAL-BOOT OSCAR SYSTEM and Windows exclusive software [2]. While maintaining the Because of native incompatibility between Windows sys- Queensgate Clusters, we had identified that the Clusters tem and UNIX-like system, and the above-mentioned so- need automated software to handle the specialisation for lutions limitations, the multi-boot computer cluster system dual-boot, re-imaging, etc. The need for multiple operat- could be a viable solution for legacy hardware. ing systems is driven by both user needs and application requirements. A. System architecture A lot of closed-source non-Java software, especially some In our initial approach, the dual-boot cluster was imple- large-scale Windows software, that use a large number of mented on a small-scale cluster with 16 compute nodes system exclusive API, have weaknesses in system platform (COTS computers) and 64 processors. Our dual-boot system migration. Since their source code is not published, the is illustrated in Figure 1. user can only run the program on the platform which the developer has offered. Although some of the companies offer WIN QUEUE multi-platform supports, others would not change a platform Users Users JOB1__CPU_NEEDED WINHEAD JOB2__CPU_NEEDED JOB3__CPU_NEEDED policy for the benefit of a small group