Liquid Cooling Guidelines for Datacom Equipment Centers | Second Edition Centers Equipment Liquid Cooling Guidelines for Datacom The Guide Every Datacom Professional Needs Liquid Cooling Data center rack heat loads are steadily climbing, and the ability for many data centers to deliver either adequate airflow rates or chilled air is now being stretched to the limit to avoid decreased equipment availability, wasted floor space, and inefficient cooling system operation. This situation is creating a Guidelines need for liquid cooling solutions to reduce the volume of airflow needed by the racks and for lower processor temperatures for better computer perfor- mance. This book provides cooling system designers, data center operators, equipment manufacturers, chief information officers (CIOs), and IT specialists for Datacom with best practice guidance for implementing liquid cooling systems in data centers. This second edition includes updated references and further information Equipment Centers on approach temperatures and liquid immersion cooling, plus guidance on water quality problems and wetted material requirements. It also includes definitions for liquid and air cooling as they apply to the IT equipment, along with an overview of chilled-water and condenser water systems and other datacom equipment cooling options. The book also bridges the liquid cooling systems by providing guidelines on interface requirements between the chilled-water system and the technology cooling system and on the Second Edition requirements of liquid-cooled systems that attach to a datacom electronics rack to aid in data center thermal management. This book is the fourth in the ASHRAE Datacom Series, authored by ASHRAE Technical Committee 9.9, Mission Critical Facilities, Data Centers, Technology Spaces and Electronic Equipment. This series provides com- prehensive treatment of datacom cooling and related subjects. ISBN 978-1-936504-67-1 ASHRAE Datacom Series, Book 4 4 ASHRAE Datacom Series 9 781936 50467 1 1791 Tullie Circle, NE 4 Atlanta, GA 30329-2305 Product code: 90564 3/14 www.ashrae.org/bookstore LiquidCoolingCover_2ndEd_tb.indd 1 2/6/2014 3:01:27 PM © 2014 ASHRAE (www.ashrae.org). For personal use only. Additional reproduction, distribution, or transmission in either print or digital form is not permitted without ASHRAE's prior written permission. Liquid Cooling Guidelines for Datacom Equipment Centers Second Edition © 2014 ASHRAE (www.ashrae.org). For personal use only. Additional reproduction, distribution, or transmission in either print or digital form is not permitted without ASHRAE's prior written permission. Liquid Cooling Guidelines for Datacom Equipment Centers is authored byASHRAE Tech- nical Committee (TC) 9.9, Mission Critical Facilities, Data Centers, Technology Spaces, and Electronic Equipment.ASHRAE TC 9.9 is composed of a wide range of industry repre- sentatives, including but not limited to equipment manufacturers, consulting engineers, data center operators, academia, testing laboratories, and government officials who are all committed to increasing and sharing the body of knowledge related to data centers. Liquid Cooling Guidelines for Datacom Equipment Centers is not anASHRAE Guideline and has not been developed in accordance with ASHRAE’s consensus process. For more information on the ASHRAE Datacom Series, visit www.ashrae.org/datacenterefficiency. For more information on ASHRAE TC 9.9, visit http://tc99.ashraetcs.org/ Any updates/errata to this publication will be posted on the ASHRAE Web site at www.ashrae.org/publicationupdates. © 2014 ASHRAE (www.ashrae.org). For personal use only. Additional reproduction, distribution, or transmission in either print or digital form is not permitted without ASHRAE's prior written permission. Liquid Cooling Guidelines for DatacomEquipment Centers Second Edition ASHRAE Datacom Series Book 4 Atlanta © 2014 ASHRAE (www.ashrae.org). For personal use only. Additional reproduction, distribution, or transmission in either print or digital form is not permitted without ASHRAE's prior written permission. ISBN 978-1-936504-67-1 © 2006 and 2014 ASHRAE 1791 Tullie Circle, NE Atlanta, GA 30329 www.ashrae.org All rights reserved. Printed in the United States of America Cover image by Joe Lombardo of DLB Associates. ASHRAE is a registered trademark in the U.S. Patent and Trademark Office, owned by the American Society of Heating, Refrigerating, and Air-Conditioning Engineers, Inc. ASHRAE has compiled this publication with care, but ASHRAE has not investigated, and ASHRAE expressly disclaims any duty to investigate, any product, service, process, procedure, design, or the like that may be described herein. The appearance of any technical data or editorial material in this publi- cation does not constitute endorsement, warranty, or guaranty by ASHRAE of any product, service, process, procedure, design, or the like. ASHRAE does not warrant that the information in the publica- tion is free of errors, and ASHRAE does not necessarily agree with any statement or opinion in this publication. The entire risk of the use of any information in this publication is assumed by the user. No part of this publication may be reproduced without permission in writing from ASHRAE, except by a reviewer who may quote brief passages or reproduce illustrations in a review with appropriate credit, nor may any part of this publication be reproduced, stored in a retrieval system, or transmitted in any way or by any means—electronic, photocopying, recording, or other—without permission in writing from ASHRAE. Requests for permission should be submitted at www.ashrae.org/permissions. Library of Congress Cataloging-in-Publication Data Liquid cooling guidelines for datacom equipment centers. -- Second edition. pages cm -- (Datacom series ; 4) Includes bibliographical references and index. Summary: "Provides information on liquid cooling for datacom equipment centers. Concerned with energy efficiency"-- Provided by publisher. ISBN 978-1-936504-67-1 (softcover) 1. Data processing service centers--Cooling. 2. Electronic data processing departments--Equipment and supplies-- Protection. 3. Hydronics. 4. Electronic apparatus and appliances--Cooling. 5. Electronic digital computers-- Cooling. 6. Buildings--Environmental engineering. I. American Society of Heating, Refrigerating and Air- Conditioning Engineers. TH7688.C64L57 2014 697.9'316--dc23 2014000819 ASHRAE STAFF SPECIAL PUBLICATIONS Mark S. Owen, Editor/Group Manager of Handbook and Special Publications Cindy Sheffield Michaels, Managing Editor James Madison Walker, Associate Editor Roberta Hirschbuehler, Assistant Editor Sarah Boyle, Assistant Editor Michshell Phillips, Editorial Coordinator PUBLISHING SERVICES David Soltis, Group Manager of Publishing Services and Electronic Communications Jayne Jackson, Publication Traffic Administrator Tracy Becker, Graphics Specialist PUBLISHER W. Stephen Comstock © 2014 ASHRAE (www.ashrae.org). For personal use only. Additional reproduction, distribution, or transmission in either print or digital form is not permitted without ASHRAE's prior written permission. Contents Acknowledgments.........................................vii Chapter 1 Introduction .....................................1 1.1 Definitions ........................................3 1.2 Liquid Cooling Systems.............................5 Chapter 2 Facility Cooling Systems ..........................9 2.1 Introduction.......................................9 2.2 Equipment .......................................14 Chapter 3 Facility Piping Design ............................25 3.1 General .........................................25 3.2 Spatial Considerations.............................26 3.3 Basic Piping Architecture ..........................27 3.4 Piping Arrangements for the Cooling Plant ............36 3.5 Water Treatment Issues ............................39 3.6 Seismic Protection ................................39 Chapter 4 Liquid Cooling Implementation for Datacom Equipment.............................41 4.1 Overview of Liquid-Cooled Racks and Cabinets ........41 4.2 Overview of Air- and Liquid-Cooled Datacom Equipment 46 4.3 Overview of Immersion Cooling of Datacom Equipment . 50 4.4 Overview of Coolant Distribution Unit (CDU)...........52 © 2014 ASHRAE (www.ashrae.org). For personal use only. Additional reproduction, distribution, or transmission in either print or digital form is not permitted without ASHRAE's prior written permission. vi Contents Chapter 5 Liquid Cooling Infrastructure Requirements for Facility Water Systems ........................59 5.1 Facility Water Systems (FWS)........................59 5.2 Non-Water Facility Systems .........................80 5.3 Liquid Cooling Deployments in NEBS Compliant Space. .82 Chapter 6 Liquid Cooling Infrastructure Requirements for Technology Cooling Systems ...................87 6.1 Water-Based Technology Cooling System .............87 6.2 Non-Water-Based Technology Cooling System .........92 References and Bibliography ................................93 Glossary of Terms .........................................95 Appendix ................................................101 Index ...................................................103 © 2014 ASHRAE (www.ashrae.org). For personal use only. Additional reproduction, distribution, or transmission in either print or digital form is not permitted without ASHRAE's prior written permission. Acknowledgments Representatives from the following companies participated in