Data Centers 16 No Power Is No Option 22 What’S Hot in Cooling 53 the Corporate Technical Journal

W ABB 4 |13 review en The unsung heroes of the Internet 6 Direct current – a perfect fit for data centers 16 No power is no option 22 What’s hot in cooling 53 The corporate technical journal

Data centers requiring less space, and reducing equipment, installation, real estate and maintenance costs. In early 2013, the facility earned the prestigious Watt d’Or award for the scale of the energy In 2012, ABB supplied the world’s most powerful savings achieved. Later in the year, ABB installed direct-current (DC) power distribution system its Decathlon® DCIM, an advanced data center at the greenDatacenter Zurich-West facility in infrastructure management system that ensures Switzerland. This 1 MW installation demon- maximum reliability, energy efficiency and optimal strates that DC systems are less complex than utilization of all data center assets (see also AC systems, making fewer power conversions, page 16).

2 ABB review 4|13 Contents

7 Data center defined Data center The infrastructure behind a digital world

11 Designed for uptime primer Defining data center availability via a tier classification system

16 DC for efficiency Data center Low-voltage DC power infrastructure in data centers

22 Backing up performance power supply ABB emergency power systems for data centers

29 Power guarantee Uninterruptible power supply for data centers

34 Continuous power Digital static transfer switches for increased data center reliability

41 Automated excellence Data center New concepts in the management of data center infrastructure design and 48 Design decisions operation What does ABB contribute to the design of data centers? 53 Keeping it cool Optimal cooling systems design and management

58 In the crystal ball Looking ahead at data center design optimization

64 Taking charge Transportation Flash charging is just the ticket for clean transportation

70 In control ABB’s dredger drives control unit provides a more reliable and integrated control platform for dredging motor systems

74 Robust radio Communication Meshed Wi-Fi wireless communication for industry 79 The right fit and partnerships ABB partners with a family-owned company to power floating flow pumps

81 Index 2013 Index 2013 The year at a glance

Contents 3 Editorial Data centers and critical technologies

Dear Reader, You may be surprised to learn how deeply power supply and its control (including such involved ABB is in the dynamic and con functions as cooling) are equally vital. In fact, tinually expanding sector of data center with the global power consumption of data technology – and has been from its very centers rapidly approaching that of countries beginning. like Argentina or the Netherlands, the effective use and management of this energy (while Data centers began to develop in earnest upholding extremely high levels of reliability) is around the time of the so-called dot-com becoming a topic of ever-increasing societal bubble in the 1990s when demand for fast relevance. and continuous Internet connectivity began its steep growth, and in-house resources Building on its background in supplying Claes Rytoft of individual companies could no longer mission-critical power and automation keep pace. Large facilities called Internet technologies, ABB has similarly become a data centers (IDCs) were created to handle player in the supply of key components and increasingly large-scale computing. In his systems to the IT industry. While other book “The Big Switch,” Nicholas Carr suppliers are assembling data centers from describes seeing a data center for the first components designed for commercial and time in 2004. He observed that a data center office use, ABB offers inherently reliable, was much like a power plant – a computing robustly designed and energy-efficient plant that would power the information age products and systems. The value of ABB’s much as power plants had powered the contribution to data centers is evident not industrial age. only in the quality of individual products but also in the company’s ability to develop and While accurate, Carr’s analogy seems so implement entire systems, covering both the vastly understated today: The data center power delivery chain as well as automated has become the most crucial IT asset for monitoring and control. nearly any 21st century enterprise. The path of increasing digitalization is rendering the Beyond the articles related to data centers, uninterrupted flow of data absolutely essential this issue of ABB Review also looks at an for day-to-day (even fraction-of-a-second electric bus that recharges in 15 s, automa- to fraction-of-a-second) operations. The tion on board a dredger and a robust wireless IT industry analyst 451 Research predicts communications system for industry. that global data traffic will reach 11 zetta bytes/month by 2017 (zetta means 1021). Enjoy your reading. Data centers are becoming ever larger, more complex and more costly to run. This edition of ABB Review looks at these trends, explores how data centers operate and – importantly – how their reliability can be maintained. Claes Rytoft While the layperson may associate data Chief Technology Officer and centers foremost with arrays of servers Group Senior Vice President processing information, the associated ABB Group

4 ABB review 4|13 Editorial 5 6 ABB review 4|13 Data center defined

The infrastructure behind a digital world

MIETEK GLINKOWSKI – Today’s mobile society means that people are consuming and creating data at unprecedented levels – the Internet, search engines, mobile apps, smart phones – all are omnipresent, yet their existence is basically taken for granted. The reality is that all of today’s mobile gadgets, and more and more of all business enterprises, depend on the storage, networking and processing of digital data, nearly all of it via or inside a data center. Without question, data centers are the backbone and unsung heroes of the Internet boom, and have become a vital industry for organizations to run mission-critical applications. ABB provides a wide range of products, integrated solutions and expertise that ensure data centers operate safely, reliably and efficiently.

Title picture In today’s world of unprecedented amounts of data use and storage, ABB is helping organizations run mission critical applications.

Data center defined 7 1 Segments of the data center industry

There are a variety of distinct industry segments from the Internal Revenue Service to the in which data centers are needed. Department of Defense and Social Security Administration. For government agencies data Colocation/hosting centers are a cost. Many small- and medium-size businesses do not want or cannot afford their own IT infrastructure Healthcare such as data centers and so they outsource This segment is expected to grow rapidly their IT needs to colocation companies. These with the emerging trend of digitalization of companies provide IT services, from web hosting, patient records and all medical data from to enterprise IT hosting, to other businesses. private doctor’s visits to hospitalization and This segment of the data center market is clearly major surgeries. For the healthcare industry focused on revenues from IT; for them the data data centers are a cost. centers are the primary business offering. Corporations, retail, manufacturing, utilities Financials This includes a large group of private and Banks and other financial institutions such publicly traded companies in a variety of as the New York Stock Exchange (NYSE), industries such as oil and gas plastics, retail NASDAQ, Tokyo Stock Exchange (TSE), etc. store chains, and power, gas and water utilities. need data centers and their high availability to Although many small and midsize corporations perform financial transactions but data centers would choose collocation services the larger per se are not their source of income. companies own and operate their dedicated data centers. For example, in Singapore, BP Telecom operates its Most of the World (MoW) Mega From landline digital services to the mobile and Data Centre, one of four mega data centers smartphone, telecom providers play a major from which BP runs its global IT operations. role in the data center industry. Today, virtually all phone services are digital and many of them Cloud computing is not considered a segment, use VoIP, utilizing the connectivity of the Internet. but rather a service, within the database indus- Major players such as NTT, AT&T, T-Mobile, all try. It is a means of distributing IT applications own, build and operate data centers. over a number of physical servers and even physical data centers. There is no longer a urrent state-of-the-art data IT services direct relationship between an application and centers are highly special- Companies such as Google, Amazon, eBay, a physical device or even physical data center. Facebook and others debuted with the Internet A good example of this is Apple’s iTunes ized industrial facilities, full boom approximately 15 years ago. Although application where data – eg, music, videos, C of intricate and interrelated these companies rely on data centers as their movies – is distributed over a combination of equipment and systems with particu- primary assets, their revenue stream varies servers and separate Apple data centers. lar mission-critical needs ➔ 1. Some from advertising to online shopping. They are This distribution is dynamic, ie, it depends on innovative in their way of building data centers, resources, availability of IT (as well as power, may be small buildings of 200 m2, others providing services and serving customers. cooling and several other factors), Internet the size of 15 soccer fields (about traffic, etc. 140,000 m2). Some require 500 kW of Government power, others 100 MW. In 1999 the US Federal government operated Footnote 432 data centers; in 2013 this number had * Government Accountability Office of the risen to about 7,000*. This includes everything US Government, 2013, www.gao.gov The field is expanding at a tremendous rate. For example, globally, the number of IT racks in 2012 reached 7.7 million – an increase of 15 percent compared Data centers consume large quantities What is a data center? to 20111. Estimated growth for data of electrical energy. Current estimates Data centers can be defined as three centers this year in the United States are that up to 2 percent of global energy side-by-side infrastructures – IT, power was 25 percent with some countries, for is consumed by data center enterpris- and cooling ➔ 3. The three infrastructures instance Turkey, reporting a 60 percent es.3 With the global installed electricity have to be perfectly compatible, matched, growth. The expansion of the corporate and optimized to data center industry was well captured provide seamless in a report by Digital Realty.2 ➔ 2 shows A large variety of software, operation of the the most important performance factors mission-critical fa- and features fueling the expansion of the databases, operating cility ➔ 4. industry. Energy efficiency and security were viewed as extremely important, systems and clouds run The IT infrastruc- whereas consolidation, connectivity and in data centers. ture contains pri- redundancy were rated as very impor- marily the IT equip- tant to somewhat important. ABB pro- ment with its as- vides cost-effective solutions to meet capacity of about 5,000 GW 4 this means sociated software. The equipment is typ- the needs of today’s data centers. data centers consume about 120 GW, ically grouped into three categories: almost twice as much as the electricity servers, network switches and storage capacity of Mexico, and more than the (memory). Each group has its unique countries of Spain or Italy. function; however in many cases servers

8 ABB review 4|13 2 Where do professionals see priorities in data center expansion? 4 Data center infrastructure

80 Very important/somewhat important Extremely important

60 Power

Percent Cooling 30

IT 10

DCIM Security Connectivity Redundancy Virtualization Green issues Green Consolidation Internet cloud Cooling issues Power capacity Energy efficiency Possible regulation Application/services More square footage square More

Disaster recovery/SoX Data center infrastructure includes the interplay Source: Digital Realty, 2011 of IT, power and cooling with DCIM.

3 Physical layout of a generic data center

Power IT

Cooling

contain storage. This infrastructure is conditioned and distributed to the serv- where the main functions of the data ers in the IT racks. centers are implemented and the IT services are delivered. A large variety of IT equipment generates a lot of heat. The software, virtualization, databases, web power infrastructure accounts for 60 per- hosting, operating systems, and clouds cent and cooling accounts for 40 per- run in data centers. cent of the energy consumed in a generic data center ➔ 5. The power usage Power and cooling are the two infra- effectiveness (PUE) factor is equal to structures necessary to operate the IT PUE = 100/55 = 1.82, which is better than equipment. Power is primarily in the form the industry average of 1.9. The power of grid electricity (although there are infrastructure can be broken down into some exceptions, such as fuel cells). four components, eventually leading to Footnotes Power is delivered to the IT equipment the IT processes (IT equipment) consum- 1 Data Center Dynamics Converged – Media Pack 2012 via complex topologies of transformers, ing about 44 percent of the total. Nearly 2 What is Driving the US Market? 2011, Digital switchgear, gensets (rotating engine all of the electricity flowing through the Realty Trust generator sets), uninterruptible power power infrastructure and used in cooling 3 The estimates vary from 1.1% to 2.5 %; see supplies (UPSs), busways and automatic is lost as heat. multiple sources: http://www.analyticspress. com/data centers.html, www.greenpeace.org, transfer switches. The raw power from www.forbes.com the utility is transformed, converted, 4 Data as of 2010 EIA.gov

Data center defined 9 5 Simplified energy balance for a generic data center

The electric energy is consumed 60 percent Server IT by the power infrastructure and 40 percent consumption process by cooling. 44%

The power usage effectiviness (PUE) factor would be equal to PUE=100/55=1.82*), which is better Electric power PSU 10% load 60% than the industry average. UPS losses and all Almost cooling power are counted as overhead power. 100% of UPS 5% total energy is wasted The power infrastructure can be broken down as heat into four components, eventually leading to the Electric power Power distribution 1% 100% IT processes (IT equipment) consuming about 44 percent.

Footnote 25% heat transfer Cooling * The alert reader may be confused that the 40% P figure is 55 rather than 60. The difference Cooling losses load 15% of 5 percent is accounted for by the the UPS losses shown.

This heat has to be removed to assure 6 Performance indicators that the operating temperatures of the equipment stay within the specifications Power usage effectiveness (PUE) is the most cooling system the PUE will actually increase and that the environment around the common key performance indicator (KPI) for since the denominator of the PUE equation equipment can be accessed by person- data centers today. It is defined as decreases. nel. Data centers employ very sophisti- P PUE = Total In some cases this can be a disincentive to cated and diverse cooling systems to P IT Load modernize facilities. In other cases in an effort control this environment, including liquid where PTotal is the total power consumed by the to improve the PUE factor, data centers switch cooling, air cooling, immerse cooling, data center, PIT Load is the power consumed by to more water-intensive cooling and therefore hot-aisle containment, cold-aisle con- the IT load. By definition PUE is always greater consume more water. This is why a new set of tainment, computer room air condition- than 1.0; everything above 1.0 is overhead KPIs have been introduced, including water power consumed by other non-IT loads, such usage effectiveness (WUE). Carbon usage ers (CRACs) and computer room air han- as cooling, lighting, security systems. effectiveness (CUE) is a data center measure- dler (CRAH) units. Cooling is the primary ment that takes the total CO2 and carbon component of the energy consumption The average PUE reported by the environmen- equivalent emissions produced as a result of responsible for the overhead power, ie, tal protection agency (EPA) in the United States the data center energy used and divides it by in 2007 was 1.9 (90 percent overhead power the energy of the IT equipment housed in the PUE factors above 1.0 ➔ 6. consumed). In 2012 Digital Realty reported that data center; the value is in kgCO2e/kWh. the average PUE for non-IT companies was Another component of the infrastructure, even worse, approximately equal to 2.9. data center infrastructure management However there is more to the energy consump- (DCIM), is becoming increasingly more tion than PUE. For example, if a data center important. DCIM is a platform to collect, owner improves the energy consumption of the control, integrate, monitor and manage IT load – ie, replacing older servers with the all the systems of the data center. Ensur- newer technology – and keeps the same ing that the temperature sensors of the cooling CRAC units are set properly to match the temperature requirements that servers read on their own motherboards data (eg, temperature, voltage, current, is not a trivial task, nor is making sure air flow, alarms), process it, display it and that the power distributed to the racks of enable an operator to make informed de- the IT equipment loads the individual cisions. DCIM is referred to as the glue feeders in a uniform fashion and does that holds all the components of a data not overload individual cables and circuit center together – an all-encompassing breakers. Keeping track of where the IT umbrella for the data center business. equipment is located, what purpose it Mietek Glinkowski serves, when it needs to be replaced, or ABB Data Centers who owns it (in the case of a colocation Raleigh, NC, United States company) is also necessary. All of these [email protected] functions and more can be handled by a DCIM platform consisting often of both Further reading hardware and software to collect the www.abb.com/datacenters

10 ABB review 4|13 Designed for uptime

MIETEK GLINKOWSKI – All systems can fail – this is a simple Defining data center fact that every industry must deal with. The paramount availability via a tier concern for the data center industry is the unbroken continuity of systems operations. Industry analysts esti- classification system mate that a one-hour outage in a data center costs on average $350,000. And the cost is expected to only go up as more and more business enterprises depend on the storage, networking and processing of digital data, nearly all of it through or inside a data center. Since loss of service for a data center is so costly, even if only for an extremely short time, availability is still the most critical driver for data centers design, operation and maintenance.

Designed for uptime 11 The paramount 1 Reliability and availability concern for the Reliability and availability are often misinter- which determines how critical these compo- preted and confused with the quality of a nents are to the mission critical function of the data center system or a product. Reliability is defined as data center. Therefore, the reliability has to be a function of time: evaluated at different points of the system industry is the R(t) = e-λ1 where power is to be delivered to the IT load. where R(t) is reliability, t is time, and λ=Tf/Tp is a failure rate. Tf is the total number of failed As mentioned, reliability and availability are unbroken continu- occurrences during the total period of Tp. The not the same as quality. Quality refers to the longer the system is operating the lower the condition of the new equipment when delivered ity of a systems reliability. The parameter λ is a reciprocal of to the customer – ie “out of the box.” Reliability MTBF (mean time between failures). Mean time and therefore availability is measured over a operation. to repair (MTTR), which is the time needed to period of time. This, besides quality, includes repair a failed system or device, is another the effects of aging and stress level of the important parameter. Used in combination, equipment within the system. MTBF and MTTR determine the inherent availability (Ai) of a system or device: Increased reliability can be accomplished by Ai = MTBF/(MFTBF + MTTR). redundancy (of equipment and delivery paths). However, the more equipment the greater the If one expands the concept of availability to likelihood for one or more of the components include the scheduled maintenance downtime to fail. For any system design there is a the availability changes to the operational balance between the level of redundancy and availability, Ao. associated complexity and reliability gains. Good system designs need to get the most out Reliability and availability are not fixed numbers. of the equipment, utilize their full potential and They are both functions of specific components provide a sufficient level of redundancy and of the system as well as the system topology, back up for reliable energy supply.

vailability of the data center nual IT downtime. The different tier de- refers to meeting the uptime signs are also capable of accommodat- expectations of the users. ing different power load densities, from A The current high availability 200 W/m2) to 1,500 W/m2. For power of data centers has been achieved most- engineers it is important to realize that ly through redundancy in design, equip- the higher the tier the higher the utility ment (both IT equipment and power voltage supplied to the facility. This is devices), electricity delivery paths and predominantly related to the fact that the software ➔ 1. Several classification sys- availability of power within a power systems exist in the industry to define data tem is generally increasing from low-volt- center availability. Rapidly changing tech- age (LV) area distribution to medium- nologies, desire to differentiate among voltage (MV) distribution to high-voltage themselves, environmental awareness (HV) transmission systems. The closer and foremost cost pressures often dictate one is to the infinite bus of a large power designs that either fall in between differ- system the less the likelihood of a distur- ent tier structures or even seek more bance or blackout. radical departures. The tier structure from the Uptime Institute, though not Tier I always followed, is considered an impor- This architecture is the simplest and tant industry guideline and thus is the therefore offers the lowest availability classification referenced in this article. and lowest IT load power density. This The Uptime Institute defines a four-tier design concept is called N, reflecting the system, where each level describes the fact that “n” IT loads need “n” sets of availability as a guideline for designing UPS units and gensets. ➔ 3 identifies the data center infrastructure ➔ 2. The higher basic components of a data center, as the tier, the greater the availability. described below.

The lowest cost and the lowest perfor- Utility source mance data center, Tier I, has a target The utility source component in a Tier I availability of 99.671 percent, which classification feeds an input transformer translates to 28.8 hours of annual IT stepping down from MV to LV. downtime. The highest level data center Title picture What sort of designs do data centers follow in order design, Tier IV, has a target of 99.995 to meet the high demands for availability? percent availability, or 24 minutes of an-

12 ABB review 4|13 2 Tier similarities and differences The current high Tier I Tier II Tier III Tier IV availability of data Number of delivery paths Only 1 Only 1 1 active 2 active 1 passive centers has been Redundant components N N +1 N +1 2 (N +1) or S + S Utility voltage 208, 480 208, 480 12-15kV 12-15kV achieved mostly Annual IT downtime due 28.8 hours 22.0 hours 1.6 hours 0.4 hours through redundan- to site Site availability 99.671% 99.749% 99.982% 99.995% cy in design, equip © The Uptime Institute ment, electricity delivery paths and 3 Tier I design N 4 Tier II design (N+1) software.

Utility feed Utility feed

GEN GEN GEN N N +1

Main switchgear Main switchgear GEN switchgear

UPS Switchgear UPS UPS Switchgear N N +1

Mechanical load Mechanical load LV switchgear LV switchgear (cooling) (cooling)

PDU PDU

IT equipment IT equipment (critical load) (critical load)

Genset gensets this time can increase to up to A genset is an emergency power genera- a minute. tor, typically with a diesel engine, that provides a long-term power backup in Uninterruptible power supplies the event of a utility outage. Long-term There are primarily three types of uninter- is defined by the amount of fuel stored ruptible power supply (UPS) technolo- in the tank and can vary from 24 to gies – standby, line interactive and dou- 72 hours. Having a high-priority fuel ble conversion. By far the most popular delivery contract can extend the time. is double conversion, where all the pow- The generator is in the form of a synchro- er flowing through the UPS is rectified nous machine with power ratings of few from AC to DC, inverted back to AC and hundred kW to 2 to 3 MW. therefore fully conditioned and cleaned from all utility-side disturbances, tran- Automatic transfer switching sients, voltage sags and swells, and other Using a specialized automatic transfer power quality (PQ) effects. The DC bus in switchgear (ATS) with control and pro- the middle is also connected to the bat- tection logic allows for a seamless tery bank, which, in the event of power switch from the source between the util- loss, provides short-term power. The ity and the genset under a number of switch between the utility AC power and different conditions. Most of the time the internal battery power is seamless the switch from the utility to the genera- and instantaneous. Short-term power is tor is open-before-close – ie, when the determined by the size of the battery utility power is lost the utility breaker is bank and typically varies from 2 to 3 min open and the genset is closed only after to 7 to 10 min. the genset has started properly, reached the desired rpm and excitation, and is synchronized. The starting of the genset can take a few seconds. With multiple

Designed for uptime 13 The addition of the 5 Tier III active-passive design; no UPS in the passive path passive delivery path significantly Active Utility feed Utility feed Passive raises the cost of GEN GEN the entire system Main switchgear GEN switchgear GEN switchgear Main switchgear and also compli- UPS UPS Switchgear Switchgear N +1 cates the control, LV switchgear LV switchgear Mechanical load coordination and (cooling) PDU PDU maintenance. IT equipment (critical load)

Switchgear additional genset and UPS. This pro- A variety of switchgear is needed in data vides some degree of device redundancy centers to distribute the power to the many of the most critical components of the different rows of IT equipment (critical system for short-term and long-term loads) as well as cooling equipment backup. All other components of the (pumps, fans, valves, compressors, etc.) system are basically the same. Even with and other auxiliary loads. The circuit break- this redundancy there are still several dif- ers in the switchgear also provide protec- ferent single points of failures in the path tion against faults and other abnormal to deliver power to the IT load. conditions. In the Tier I facility all of the switchgear is low voltage (less than ~1 kV). Tier III Tier III is referred to as an active-passive Power distribution unit system ➔ 5. In a Tier III classification the Power distribution units (PDUs) are com- power delivery path has to be doubled. prised of circuit breakers, metering units Besides the redundant critical compo- and, in North America, LV transformers, nents there has to be a second path par- to further distribute the power to the IT allel to the critical IT load in case the pri- racks as well as provide protection and mary path has failed. This second path measure the power (voltage and current) could be passive, ie, used only in case of to the individual loads. emergency. A Tier III classification also requires a second utility connection. The Power supply units addition of the passive delivery path sig- Power supply units (PSUs) are part of the nificantly raises the cost of the entire IT equipment. Similar to the power sup- system and also complicates the control, ply of a desktop computer these units coordination, maintenance, etc. There is transform the 220 V or 110 V input power also an additional switchgear and motor to the DC voltage distributed to the control center (MCC), which should allow various IT equipment: servers, network the full operation of the data center from and storage systems. The most popular the passive path. The IT equipment can PSUs are transformer-less switched now take full advantage of the dual sup- mode power supply (SMPS). Due to the ply paths and therefore utilize dual PSUs redundancy of the power distribution for for each server, for example. As a result Tier III and IV more and more PSUs are the number of single points of failure is now provided with dual AC inputs and significantly reduced. However, the pas- can function from either of the two. sive delivery path does not require UPS so during the emergency conditions the Tier II system is vulnerable to utility conditions, This design is known as N+1 ➔ 4. The therefore potentially exposed to utility primary difference between a Tier I and power quality issues or even power out- Tier II classification is the presence of an ages.

14 ABB review 4|13 6 Tier IV design 2N+1; two simultaneously active paths For any system design there is a Utility feed A Utility feed B balance between GEN GEN GEN GEN N +1 N +1 the level of redun-

Main switchgear GEN switchgear GEN switchgear Main switchgear dancy and associ-

UPS UPS Switchgear Switchgear UPS UPS ated complexity N +1 N +1 and reliability LV switchgear LV switchgear Mechanical load (cooling) gains. PDU PDU A B IT equipment (critical load)

Tier IV For example, during one year, 10 short Referred to as a 2N+1 system, the Tier power interruptions at the server power IV classification is also considered the supply lasting 50 ms each will have a Cadillac of data center design ➔ 6. A much more detrimental impact on the relatively small number of data centers in operation of the servers than one longer interruption of 500 ms during Tier IV designs are fully the same period of time. Although redundant, complete dual both will result in the same annual systems running actively availability (total of in parellel. 0.5 s of lost power) the first one will cause the servers the world are certified as Tier IV designs. to reboot and possibly lose some data They are fully redundant, complete dual 10 times during the year; the second one systems running actively in parallel. By will result in only one reboot a year. virtue of the redundancy the rating of each path has to be 100 percent of the Highly skilled engineering resources are load and therefore the maximum utiliza- needed to design, implement, and opti- tion of the two paths under normal oper- mize the entire data center ecosystem ating conditions is at maximum 50 perfor their availability and reliability. The tra- cent. In addition, some Tier IV designs ditional way of thinking about availability will have N+1 of UPSs and gensets in and reliability is changing rapidly. In- each path, further increasing the com- creased system voltages, more sophisti- plexity and cost but at the same time cated switching schemes, wider operat- gaining the valuable fraction of a percent ing regimes for IT equipment, and (0.01 percent to be exact) for availability. foremost the advent of failure-resilient The target for Tier IV availability is to software and cloud computing introduce allow a maximum of 24 min per year of new dimensions to data center reliability. the annual site-caused end-user down- So, stay tuned. time (representing one failure every five years).

Changes to come Tier structure availability and downtime Mietek Glinkowski are not the only factors to consider. Im- ABB Data Centers pact of the interruptions on the operation Raleigh, NC, United States of the mission critical facility can vary. [email protected]

Designed for uptime 15 DC for efficiency

ANDRÉ SCHÄRER – Looking at all data centers worldwide, around 80 million Low voltage DC MWh of energy are consumed each year, corresponding to about 2 percent of global CO emissions. Before long, these values will be equivalent to the power infrastructure 2 electrical consumption of Argentina or the Netherlands. With the addition in data centers of more than 5.75 million new servers worldwide annually, global carbon emissions from data centers will quadruple by 2020 – if the electricity mix does not fundamentally change and no measures are taken to increase energy efficiency. The thirst for power of a single medium-sized data center corre- sponds to that of approximately 25,000 private households in the United States (or almost twice as many in Europe). What can be done to make data centers more frugal energetically? ABB recognizes DC as an important tool in achieving this goal. DC offers several advantages, most notably lower losses by eliminating conversion and transformation steps in the power delivery chain. Losses between infeed and server can be reduced by 10 percent.

16 ABB review 4|13 With DC, there are two less conversion steps in total.

he call for energy efficiency tem has dominated the transmission With the growing role of DC in the fields and for the comprehensive and distribution of electricity for more of generation, transmission, storage and use of renewable energies is than 100 years. consumption, more and more electricity T becoming louder and loud- takes the form of DC at least once er ➔ 1. One important solution being So that means DC is dead? Far from somewhere along its supply chain. Some promoted by ABB is the use of DC in it. In today’s digital age, more and more conversion steps are necessary, but in data centers. devices are operated with DC – consumer some cases, the voltage and frequency electronics, industrial information tech- levels used are justified by historical Direct current technology nology, communication technologies and reasons only, and yet the associated The struggle between the proponents of electrical vehicles, to name just a few. At conversion steps cause avoidable energy AC (Nicola Tesla and George Westing- the other end of the energy supply chain losses. Supported by advances in power house) and the advocate of DC (Thomas are photovoltaic systems and fuel cells electronics, ABB is reconsidering the A. Edison) toward the end of the 19th (and some wind parks) that generate DC. incontestability of AC transmission and century, also known as the “War of Cur- In transmission too, there is a notable seeking to advance DC into fields where rents,” was finally won by AC. This sys- exception facing AC’s predominance: high- it can deliver energy savings. voltage direct current (HVDC) provides large transmission capacity at low losses World’s most powerful direct current over long distances. ABB has played, data center Title picture and continues to play, a leading role as Data centers are particularly suited for In 2012, ABB supplied the world’s most powerful DC power distribution system, installed at the supplier and developer of the technology a DC supply. The reason is that there greenDatacenter Zurich-West facility in Switzerland. over its almost 60 year history. are a large number of identical, or at

DC for efficiency 17 With regard to the choice of DC voltage, 1 The road to efficiency This pilot project an open-circuit voltage of 400 V was There are various approaches to making data selected. On the one hand, it is neces- is a one-time so- centers more ecological; DC (direct current) sary to keep the voltage as high as pos- lution specifically technology is not the only tool in the arsenal. sible to minimize losses and the amount Other approaches include the location and of copper needed. On the other hand, design of the data center, technical advances developed, in- in server technologies and cooling, better staff safety and equipment compatibility utilization and operational philosophies. were taken into consideration (there are stalled and start- also indications that 380 V could devel- It is important to recognize that optimization op into a standard in DC supply and dis- ed up in record restricted to individual components will lead to a less-than-optimal overall system. The key to tribution: Committees such as the IEC, 1 time for ABB’s success lies in considering the overall system NEMA and Emerge Alliance have including the interaction between the owners/ already addressed this topic). customer, Green operators of data centers and their hardware suppliers. Proven and industry-tested ABB tech- Datacenter AG. nology was selected for the entire DC supply chain to ensure high reliability and availability. While the central rectifier unit was developed specifically for this project, its core contains the latest modular power electronics known from a multitude of other applications.

From grid to chip The redundant infeed by the local utility least similar, consumers (servers, net- uses 16 kV (medium voltage) from two work components, storage, etc.) thus independent substations. limiting the multitude of voltage levels needing to be provided. This infeed, together with the emergency power of a diesel generator, is first In 2011, Green Datacenter AG, the fed to a gas-insulated medium-voltage operator of the data center business switchgear of type ABB ZX0. An ABB for the Internet provider green.ch, de- Tanomat-type control system automati- cided to operate a 1,100 m² extension cally ensures that the switches are set to (of a 3,300 m² data center) in Zurich- the appropriate positions for the operat- West ➔ title picture using DC technology ing mode (normal operation, emergency and chose ABB as its partner. power operation, test operation, back- feed to utility). This article explores the concept of DC distribution supplied specifically for this Rectification data center. This is a customer- and The output of the medium-voltage project-specific solution and does not in switchgear connects directly to the cen- this form represent a standard product. tral rectifier unit. Within this unit, there is first a medium-voltage switch discon- Technical solution nector, followed by a highly efficient To demonstrate the efficiency gains on ABB 1,100 kVA three-winding dry-type a large scale, it was decided to design transformer that converts the 16 kV to the direct current supply system with low voltage. Two parallel, thyristor- a capacity of almost 1 MW ➔ 3. A few based, 6-pulse ABB DCS800-type recti- smaller and similar systems are already fier modules then carry out the actual in use around the world. They are how- rectification – this step is performed ever used primarily for research and once for the energy supply of the serv- development purposes. ers (main supply) and once for charging the batteries (these guarantee an auton- omy of around 10 min at full load).

On the output side, the rectifier modules Footnote are connected in series. They thus en- 1 An open industry association leading the rapid able a center tap, which can be ground- adoption of safe DC power distribution in commercial buildings through the development ed. The resultant three-conductor of EMerge Alliance standards. system provides L+ (+200 V), M and

18 ABB review 4|13 2 A DC supply for data centers involves fewer components and lower losses than AC

MV/LV Switchgear UPS PDU Server power supply unit Server transformer 4 5

AC architecture 16 400 400 400 400 380 12 kVAC VAC VAC VAC VAC VDC VDC

Battery

MV/LV AC/DC PDU Server power Server Fewer conversions transformer central supply unit – Better efficiency rectifier Fewer components 1 – Lower cost DC – Smaller footprint architecture 16 380 380 12 kVAC VDC VDC VDC

L– (–200 V), whereas the consumers are maximum rated short-circuit withstand of connected between L+ and L–. 65 kA was certified, taking into account Rigorous tests the particular conditions of this project The subsequent ABB MNS® low-voltage (contribution of batteries to short circuit, were performed to switchgear has two functions: On the etc.). ensure the safety one hand it serves as an interface to the batteries. On the other, it distributes MNS iS power distribution units of people and Two redundant MNS iS 400 V DC PDUs distribute the energy within the IT equipment in both Proven and indus- rooms and ultimately feed the servers. Depending on customer requirements, normal operation try-tested ABB the newly launched ABB intelligent and in the event of remote power panels MNS iRPP may technology was additionally be used for this task, allow- a short circuit. selected for the ing more precise distribution. The MNS iS PDUs are based on the same low- entire DC supply voltage switchgear system (MNS) as the main distribution described above and chain to ensure have the same performance data, ex- high reliability and cept that their rated current is 1,600 A availability. (each). Each output contains a high-precision the energy to the MNS iS PDU (power measurement based on the shunt mea- distribution units), which are directly suring principle. This not only makes in- adjacent to the IT rooms and constitute dividual energy measurement possible, a type of sub-distribution unit. but also enables predictive maintenance to be carried out, for example by The MNS switchgear is designed for an measuring and recording the tempera- operating voltage of 400 V DC and can ture in each conductor (L+ and L–) in convey a maximum constant current of real time. If the superordinate control 3,000 A. To ensure the safety of people system detects an abnormal state or and equipment in normal operation and negative trend, proactive intervention in the event of a short circuit, the switch- can be triggered therefore preventing a gear was also rigorously tested and cer- dangerous operating condition or mal- tified by an independent laboratory – a function.

DC for efficiency 19 The discussions 3 DC power supply for greenDatacenter about the advan- Generator Utility tages of DC sup-

Central ply in data centers rectifier assembly are often reduced Battery charger – / – to energy efficien- Charger bus cy, but DC has Battery MNS switchgear

6 Strings total many other Discharge bus DC bus DC advantages. cable/ busduct PDU

MNS iRPP MNS iRPP

DC/DC PSU DC/DC PSU – / – – / – 12 V 12 V

Load Load

Server and more as compared with a state-of- The energy supply chain concludes with the-art AC PSU according to informa- a rack containing various industry stan- tion from Power-One). Apart from the dard servers. A setup with one HP connection, there are no visible differ- X1800 G2 network storage system, four ences on the exterior (identical form HP ProLiant DL385 G7 servers, one blade factor).

System comparison There is a wide- Comparison of the circuit topology implemented in this project against con- spread yet errone- ventional AC (as also used at green- ous view that IT Datacenter), shows that with DC, there are two less conversion steps in hardware supplied total ➔ 2. First, there is no traditional uninterruptable power supply (UPS) with with DC power rectifier and inverter. The rectification on differs from that the input of the server power supply unit is also omitted. supplied with AC. An AC data center for North America (fulfilling the ANSI standard) would have system c3000 with three HP BL465c G7 an additional transformer within the CTO blades and one HP 5500-24G DC PDU to transform 480 / 277 V to 208 / EI switch is used for demonstration pur- 120 V – primarily for reasons of personal poses, with ABB running some applica- safety. In this case, the DC solution also tions to make use of the capacity. has one transformation less.

There is a widespread yet erroneous Results view that IT hardware supplied with DC The energy efficiency of the power power differs from that supplied with infeed through to the server (including AC. This is not so: The server is identi- the server power supply unit) can be cal. The only difference is in the power improved by up to 10 percent when supply unit (PSU). For DC, the unit is using DC compared with AC (depending simplified (eg, omission of the rectifier). on load). This is thanks to the smaller This has a positive effect on energy number of conversions and additional efficiency (an improvement of 3 percent effects.

20 ABB review 4|13 4 Advantages of new DC solution (to be launched in 2015) With the growing role of DC in the

– Active front-end AC/DC power conversion fields of generation, transmission, for minimal harmonic distortion – Stable, regulated 380 VDC output with low storage and consumption, more and ripple enables use of so-called narrow- band power supply units with highest more electricity takes the form of efficiencies – Superior rectifier efficiency over wide DC at least once somewhere along power range – System cost significantly lower than its supply chain. state-of-the-art AC UPS system – Smallest footprint and ease of access – Truly integrated and modular platform, scalable in increments – Microgrid enabled (ease of integration of batteries and alternative power sources without paralleling / synchronizing controls) – Connection of AC legacy equipment – Short-circuit-proof design – Type-tested assembly

Beyond this, the cooling needs in the A balanced, facts-based evaluation of renovations and small extensions to IT room are decreased, which further DC and AC systems should take ac- existing AC facilities. reduces the energy required. count of all factors, from planning and construction costs to operating and However, the technology sees an addi- The discussions about the advantages maintenance costs. tional boost when the data center is of DC supply in data centers are often considered a DC microgrid – ie, it shifts reduced to energy efficiency. DC’s fur- New generation from being a pure consumer (energy as ther advantages are only rarely men- As mentioned above, this pilot project is an expense) to a generator (energy as a tioned. In this project, the following a one-time solution specifically devel- source for revenue) through “on-site results could be achieved based on oped, installed and started up in record generation.” In this scenario, energy can comparison measurements and real time for ABB’s customer, Green Data- flow in both directions. As numerous data: center AG. power conversions are eliminated, inter- – 10 percent improvement in energy connection and compatibility for all on- efficiency (not counting the reduced Presently ABB is developing a new DC site equipment is simplified. This can need for cooling in the IT room). data center solution that will further rev- include on-site alternative energy sourc- – 15 percent lower investment costs olutionize the power supply architec- es (photovoltaics, wind, fuel cell, etc.), related to the electrical components ture. The standard product will be energy storage (eg, batteries) and con- for the data center power supply. launched on the market at the latest in sumers in the data center. – 25 percent less space required for 2015 and will boast the advantages laid the electrical components for the out in ➔ 4. The idea of the data center as a mi- data center power supply. crogrid is not a long-term vision – there Use of direct current and are already initiatives and projects being Using fewer components also increases DC microgrid pursued in this area. reliability and decreases the likelihood DC is not the be-all and end-all for data of human error. centers. There are applications for which alternating current is more suitable. For The costs for installation, operation and optimum results, data centers must be maintenance also dropped thanks to considered in their entirety and planned simpler architecture and less equip- in an integrated manner – from the grid ment. The savings in installation costs infeed through to the server. In smaller amount to around 20 percent. This val- data centers, savings may not be high ue is based on the experiences gath- enough in absolute terms to justify DC. ered in the project. Qualified statements André Schärer on operating and maintenance costs DC technology should preferably be ABB Low Voltage Systems cannot be made at this time. used in new and large data centers. Its Lenzburg, Switzerland advantages diminish when it comes to [email protected]

DC for efficiency 21 Backing up performance

ABB emergency power systems for data centers

MANFRED FAHR, RALPH SCHMIDHAUSER, JOHN RABER – essential aspect of their operation. Despite all the Data centers are one of the least visible but most crucial precautions taken during the design and operation of parts of our modern infrastructure. The data they contain data centers, situations can arise in which external – bank details, medical histories, company data, pension power is totally lost for a significant period. Such records, tax returns, social media treasures (Facebook blackouts result in data loss, nonavailability of essential receives over 300 million new photos each day) and a services, risk to hardware and, potentially, financial plethora of other data – are, to different degrees, impor- losses of millions of dollars. For these reasons, highly tant to modern life. So reliant has society become on dependable emergency power systems are increasingly data centers that 100 percent uptime is now often an mission-critical for the data center industry.

22 ABB review 4|13 1 ABB gensets provide reliable backup power for data centers.

are underestimated or even overlooked tems: Control and power systems have altogether. Critically, nonstandardized to grow seamlessly with increasing en- control systems and nonmatching or ergy demand and adapt to changing low-quality system components can customer needs and priorities. This has introduce a single point of failure, thus to be achieved without compromising xternal threats to the power increasing the risk of malfunction exactly quality or reliability, or introducing the grid are difficult, or impossible, when reliable power is needed most. need for system downtime. to control. Every year, storms Inferior installation practices can be E and adverse weather condi- costly too: One global Internet-based Data center business cases often allow tions – for example, the recent superstorm supplier was recently fined over half-a- for expansion in several stages over Sandy in the United States – cause major million dollars for installing and repeat- time. A modern emergency power sys- power interruptions and stretch many edly running diesel generators without tem has to be designed to provide full emergency power systems beyond the obtaining the required standard environ- functionality from the initial operation limits of their capabilities. Construction- mental permits on a site in the state of levels right up to the final data center related incidents are another major cause Virginia, in the United States [1]. Poorly expansion stage. This requires thorough of utility outages. Even without such installed gensets are generally becoming design of the supply concept, communi- events, utilities have to cope with power a matter of concern. cation structure, control systems and grids that are aging, increasingly decen- building infrastructure. Standardized com tralized and unpredictable. For a data In short, the performance, functionality ponents with upstream and downstream center, therefore, a highly dependable and reliability of any emergency power compatibility and long-term availability emergency power system is a must. system are highly dependent on, and Quality is paramount determined by, the At the heart of the ABB Most data centers employ uninterrupt- capabilities of the able power supplies (UPSs) combined control system, the emergency power concept with diesel generator sets (“gensets”) to quality of all sys- safeguard against power interruptions or tem components lies the programmable logic total loss. However, design and installa- and the profession- controller (PLC). tion of gensets and emergency power alism with which control systems are often oversimplified the system instal- and only poorly executed. This results in lation is carried out. Further, when devel- allow for changes and extensions over a internal and “homemade” threats which oping world-class emergency power period of many years without the need to system concepts, all needs and benefits replace entire systems. must be considered, not just the techni- Title picture cal features ➔ 1. ABB system concepts are designed to Data centers that aim for 100 percent uptime need allow for step-by-step extensions or a highly reliable diesel generator backup for the Scalability changes without the need for system eventuality that the external power fails for a length of time. Just what are the characteristics of such Scalability is absolutely essential when downtime and they accommodate inde- an emergency backup system? designing modern backup power sys- pendent testing of new stages without

Backing up performance 23 2 ABB control cabinets lie at the heart of the emergency power concept.

risk to the ongoing data center opera- The PLC is a vital part of any critical The performance, tion. power concept and represents a single point of failure – a failure that could have functionality and New criticality paradigms potentially catastrophic consequences. Power criticality concepts and philoso- To mitigate this risk, ABB control sys- reliability of any phies vary widely between industries tems are based on standardized com emergency power and, in many cases, are unique to indi- ponents and offer compatibility with all vidual customers. Further, consumer other relevant ABB products. This allows system are highly groups can no longer be simply catego- conceptual changes, functionality up- rized according to whether they are grades and capacity expansions to be dependent on the merely UPS-supported or require emer- made at any time without interruptions, capabilities of the gency power or are supplied by the grid and without system availability and reli- only. Rather, it is now essential to distin- ability being compromised. control system, guish between consumers who can tol- erate medium-length, short or no power Reliability and availability component quality interruptions. This changes the emer- ABB designs and supplies fully integrat- and the profession- gency power system concept, and se- ed emergency and backup power prod- lection and sizing of system compo- ucts and complete turnkey systems. alism with which nents. Reliability can be further increased Having one port of call for planning, en- by reducing or removing less-critical gineering and installation of the complete the system installa- consumers while providing power to system, including auxiliaries, allows for tion is carried out. essential servers only. seamless integration, easy future expansion, simplified service and maintenance, Controlling emergency power while reducing the number of interfaces ABB’s emergency power activities include and thus increasing reliability. Bundling entirely new installations and moderniza- electrical system components such as tions of complete control systems that low-voltage and medium-voltage switch- manage both emergency power groups gear, transformers and control systems and main distribution systems. At the with auxiliaries like fuel systems, exhaust heart of the ABB emergency power con- systems, ventilation and cooling under cept lies the programmable logic control- one contract offers peace of mind for ler (PLC) ➔ 2 – 4. The task of the PLC is to supply, integration, commissioning, control the diesel engines and generators maintenance and service. belonging to the emergency power groups and communicate with other con- High-quality standardized products also trol systems, individual consumers, UPSs, significantly reduce intervention time switchgear and the process control sys- during maintenance or in the event of tems. The performance and reliability of failure – components can be changed a power supply system is highly depen- quickly and easily, service is simplified dent on, and, more importantly, limited by, and some modules can even be hot- the quality and capability of the control swapped. system and its components.

24 ABB review 4|13 3 Control cabinets can be easily modified and extended to accommodate data center expansion. For a data center, a highly dependable emergency power system is a must.

Advanced technology tribution in data centers. DC systems are ABB is able to design emergency power also less complex and require less space concepts based on a range of technolo- – reducing equipment, installation, real gies. A highly capable and scalable con- estate and maintenance costs. This can trol system allows for the use of tech- result in savings of up to 30 percent on nologies such as diesel rotary uninterruptable power Financial flexibility can be systems (DRUPSs) or even the inte- nearly as important as tech gration of com- pressed-air power nical specifications. For in- storage solutions. stance, leasing and full-ser-

The most modern vice models allow for accurate data center power technologies are operational expense planning based on direct and maintain the highest level current (DC). One of the top informa- of reliability. tion and communications technology (ICT) service providers in Switzerland, the total facility costs. The green.ch data green.ch, has chosen ABB to design and center uses ABB emergency power install an advanced, DC power distribu- gensets ➔ 5. tion system in a new state-of-the-art data center (see also pages 16–21 of The advanced AC500 PLC at the heart of this edition of ABB Review). DC technol- the ABB Master control system provides ogy trims power conversion losses and is an interface to ABB’s data center infra- 10 to 20 percent more energy efficient structure management (DCIM) system, than traditional alternating current (AC) Decathlon. Integrated fiber-optic com- technology when used for electrical dis- munication rings enable the emergency

Backing up performance 25 High-quality standardized products 4 Control cabinet interior also significantly reduce intervention time during maintenance or in the event of failure – components can be changed quickly and easily, service is simplified and some modules can even be hot-swapped.

power controller to continuously communicate with upstream and downstream systems and components. Cus- tomers have the ability to monitor, analyze and control emergency power systems locally, and to increase supply security and optimize operations remote- ly, as required.

Remote monitoring and notification services have been developed to relay critical information to mobile devices including mobile phones. This allows an immediate response to threats and facili- tates the planning of preventative mea- The IBC has been widely adopted in sures to ensure that 100 percent avail- North America and ABB has already ability is not compromised. Furthermore, implemented many of its standards into remote access ability allows utility opera- its products. tors to access and purchase additional power during peak periods. All ABB industrial gaseous liquid-cooled (IGLC) and industrial diesel liquid-cooled High-quality diesel engines (IDLC) stationary gensets meet the IBC ABB utilizes only high-quality diesel en- wind resistance requirements. These re- gines from well-regarded original equip- quirements vary depending on exposure ment manufacturers (OEMs). This en- category and occupancy category – for ables ABB to meet and exceed the most example, a life-critical building such as a stringent environmental requirements. hospital requires a higher safety factor Diesel exhaust systems can be designed than a manufacturing plant or mall. to further reduce emissions and noise Mathematical modeling of various sce- pollution. narios and the stresses inherent in those scenarios has been performed on the ABB gensets comply with the stringent gensets to determine their ability to with- structural integrity obligations laid out by stand the wind under different situations. the International Building Code (IBC). The IBC is a broad collection of struc- ABB generators also comply with the tural building requirements that help pre- Underwriters Laboratory (UL) UL 2200 vent injury and damage from earthquakes standard for safety. and other such phenomena. The IBC and other building codes are now written UL 2200 is the most widely adopted so that, in the event of a catastrophe, safety certification in the United States. mission-critical systems will be able to If the genset operates at 600 V or less withstand the same forces the building and is intended for installation and use in housing them can. A unit that complies ordinary locations in accordance with the with IBC seismic standards will have National Electrical Code NFPA-70, it can been certified through seismic analysis be designed to meet UL 2200 standard. and tri-axial shake table testing. This means that the unit has gone

26 ABB review 4|13 5 Genset and master control cabinets (in the background) as supplied to green.ch. A redundant fiber-optic bus system is included.

Technical and financial concepts also Scalability is cater for interim solutions: Additional demand can easily be met with the addition absolutely essential of temporary power units and containerized systems can comfortably bridge the when designing gap during extension phases without the modern backup need of risky and costly shutdowns and power systems. compromised availability. As data centers increase in number and size, the emergency power systems that through rigorous testing to ensure it has support them will grow in sophistication a longer uptime, meets higher safety and capability. ABB will continue to de- standards and will be less likely to fail velop this technology to ensure that data Manfred Fahr than an equivalent noncertified unit. centers continue to conform to regula- Ralph Schmidhauser tions and that its customers can contin- ABB Low Voltage Products Business models ue to operate with 100 percent uptime. Lenzburg, Switzerland Data center emergency power systems [email protected] are significant investments so delivery [email protected] and financial flexibility can be nearly as important as technical specifications. For John Raber instance, leasing and full-service models Baldor Electric Company, allow for accurate operational expense a member of the ABB Group planning and the avoidance of unexpect- Oshkosh, WI, United States ed costs, while maintaining the highest [email protected] level of reliability. Other financial models accommodate upgrades, extensions and Reference new technology platforms. Rental mod- [1] New York Times (2012), “Power, Pollution els avoid large capital expenditure, aid and the Internet,” retrieved from http://www.nytimes.com/2012/09/23/ swift project execution, leave flexibility technology/data-centers-waste-vast-amounts- for future growth and provide clear and of-energy-belying-industry-image. easy control of finances. html?pagewanted=all&_r=0 (2013, August 1).

Backing up performance 27 28 ABB review 4|13 Power guarantee

Uninterruptible power supply for data centers

JUHA LANTTA – The articles in this issue ower disturbances come in many ment). The former topology is appropriate, of ABB Review underline just how much guises: On top of total power out- in most cases, for large data centers and modern society depends on data ages and blackouts, the voltage the latter is usually found in smaller data centers. It is of critical importance that Pmay sag or swell over short peri- centers. there is zero downtime in data center ods; it may also do so over longer periods operations, so a continuous supply of – so-called brownouts or overvoltages; Servers are not the only elements of a data clean power must be guaranteed. The there can be electrical noise on the line, center that require UPS protection: Auxil- key component in ensuring this is the or frequency variation; or harmonics may iary devices and systems that manage uninterruptible power supply (UPS). appear in the voltage. cooling and safety, often called “mechani- Because reliability is so crucial, it has cal loads,” are also critical for the smooth been made a cornerstone of the ABB A UPS remediates all of these operation of the data center and ABB pro- UPS design philosophy. In addition, as A UPS will condition incoming pow- vides reliable backup power solutions for data centers are major consumers of er ➔ 1. Spikes, swells, sags, noise and these, too. electrical power, the high energy harmonics will all efficiency of ABB UPS systems brings a be eliminated. In welcome reduction in the power bills the case of total landing on the doormat. Although data power failure, pow- A UPS will condition incoming centers vary in their power protection er will be supplied needs, the combination of required from batteries or power. Spikes, swells, sags, availability and reasonable costs of other energy stor- noise and harmonics will all ownership (initial investment and age systems. A running costs) need not necessarily backup generator be eliminated. entail compromises if the appropriate will kick in for lon- insight is employed in optimizing the ger power outag- solution for each case. es. This ensures that data center Data center designs and ratings operation is available around the clock The detailed design of a data center and that no data corruption or loss will depends on its size, power density and occur. criticality. The power scheme is part of

Title picture data center site’s infrastructure and the Disruptions in the power flowing to a data center Applications in data centers Uptime Institute’s Tier ratings (I–IV) give can happen at any time and can jeopardize the In a data center, the principal mission of guidelines and help in understanding the integrity of the continuous operation of the data the UPS is to protect the servers. The UPS levels of power protection that may be center. The problem can be avoided by choosing the correct UPS type and configuration. Shown can be located centrally or beside each applicable ➔ 2: here is the ABB Conceptpower DPA 500 UPS. row of server racks (“end of row” place-

Power guarantee 29 1 Power disturbances 2 Characteristics of 4 tiers of the power infrastructure

Double-conversion UPS Tier I Tier II Tier III Tier IV

Number of Only 1 Only 1 1 active 2 active delivery path 1 passive

Redundancy N N+1 N+1 S+S or 2 (N+1)

Concurrently No No Yes Yes maintainable

Fault tolerant None None None Yes worst event Input Output

Site availability (%) 99,670 99,750 99,980 99,990

− Tier I: basic site infrastructure Availability, a measure of how good the In a Tier IV data (nonredundant) system is, is formally defined as: − Tier II: redundant-components site center, a “system + infrastructure (redundant) MTBF / (MTBF + MTTR) × 100% system” configura- − Tier III: concurrently maintainable site infrastructure where MTBF is mean time between failures tion, namely two − Tier IV: fault-tolerant site infrastructure and MTTR is mean time to repair (in hours). These are common parameters in the UPS separate UPS sys- Power availability increases with tier industry and both impact system avail tems, each with ranking. ability. Modular UPS designs minimize the The “dual-cord” IT load innovation en- system’s MTTR. N + 1 redundancy, abled the development of the dual bus concept, now used in Tier IV applica- ABB’s Conceptpower DPA 500 UPS, for enables infrastruc- tions. Today, the fault-tolerant Tier IV example, ensures availability and reliability ture work to be power infrastructure is very commonly by employing a so-called decentralized used in critical data centers, even if the parallel architecture (DPA) ➔ 4. In this, each undertaken without data center itself is not necessarily Tier UPS module contains all the hardware and IV certified. This is due to the importance software required for full system operation. disrupting the criti- of the protected power relative to its The modules share no common compo- costs. This design is able to withstand a nents – each UPS module has its own in- cal load. disastrous failure on either side of the dependent static bypass, rectifier, inverter, supply, it allows concurrent maintenance logic control, control panel, battery char- and it is even possible to undertake infra- ger and batteries. structure work on it without disrupting the critical load. This is achieved by im- With all the critical components duplicated plementing a “system plus system” con- and distributed between individual units, figuration, namely, two separate UPS potential single points of failure are elimi- systems, each with N + 1 redundancy – nated. In the unlikely event of one UPS ie, with enough UPS elements to meet module failing, the overall system will con- the maximum expected demand, plus tinue to operate normally, but with one one ➔ 3. module fewer of capacity. The failed module will be fully disconnected and will have Reliability and availability no impact on the operating modules. UPSs play a vital role in ensuring IT reliability and, thus, data availability. As a result, The ABB Conceptpower DPA modules the reliability of the UPS itself is a major can be removed or inserted without risk consideration. Any time a UPS fails and to the critical load and without the need becomes unavailable, mission-critical to power down or transfer to raw mains electrical loads are put at risk. The surest supply ➔ 5. This unique feature directly way to increase availability of power is to addresses continuous uptime require- optimize the redundancy of the UPS sys- ments, significantly reduces mean time tem and to minimize its maintenance and to repair (MTTR), reduces inventory levels repair time. of specialist spare parts and simplifies system upgrades.

30 ABB review 4|13 3 Tier IV power system with 6 + 6 UPS units UPSs play a vital UPS System A 4x300 kW UPS System B 4x300 kW role in ensuring IT G G G G reliability and, thus, data availability. As a result, the reli- PDUs A Load PDUs A ability of the UPS UPS System A 6x500 kW UPS System B 6x500 kW itself is a major G G G G consideration.

PDUs A Load PDUs A

4 ABB’s Conceptpower DPA 500 is scalable up to a maximum power of 3 MW.

Vertical scalability: one to five modules in one single cabinet

Horizontal scalability: cabinets in parallel configuration up to 3MW

This online swap technology, along with battery, charger and inverter power blocks The most widely used, in both the power significant reductions in repair time, can are utilized in the same manner as in the rating (500 W to 5 MW) and application also achieve so-called six-nines (99.9999 offline system, but due to the added regu- senses, UPS topology is the double-con- percent) availability – highly desirable for lation circuits in the bypass line, a voltage- version online topology. As its name sug- data centers in pursuit of zero downtime. regulating tap-changer transformer is often gests, the incoming alternating current used to handle any small undervoltages (AC) is continuously converted by rectifi- UPS topologies and overvoltages that may occur. Thus, the er to direct current (DC) and then back to Broadly speaking, UPS designs fall into load is transferred to the battery-fed invert- AC via an inverter. In this way, a perfectly one of three operational architectures: er supply less often. The line voltage is ac- clean waveform can be produced under standby, line-interactive and double-con- tively monitored and when the input voltage any mains or generator supply condi- version online. or frequency goes out of range, an inverter tions. and battery maintain power to the load. Standby (also known as offline) systems This UPS design offers the highest degree are usually low-power (up to 5 kVA) and Line-interactive UPS topologies are usually of critical supply integrity. The load is sup- supply the critical load directly from the used for low power ratings (up to 10 kVA), plied with processed power at all times. mains without performing any active volt- where they often compete with standby age conversion ➔ 6. They transfer the load UPSs. They are more costly but able to Double-conversion topology is used for to the inverter in the event of a bypass protect the load against long duration critical applications like data centers. Its supply failure. A battery is charged from brownouts. ability to run in load-sharing parallel con- the mains and is used to provide stable figurations provides the redundancy that is power in the event of a mains failure. There are also larger systems in the market desired in such applications. where the tap-changer transformer is re- Like standby models, line-interactive UPSs placed with an active automatic voltage UPS classification normally supply the critical load from the regulator (AVR). These line-interactive UPS To standardize UPS characteristics, the mains and transfer it to the inverter in the systems are capable of supplying hun- IEC introduced (in IEC 62040-3) a three- event of a bypass supply failure ➔ 7. The dreds of kVA. step UPS classification code based on the

Power guarantee 31 5 DPA 500 modules can be swapped without powering down.

operational behavior of the UPS output performance. Only when this part of Hydrogen fuel cells exploit the fact that voltage: the designator is “111” can the user when hydrogen and oxygen chemically − Step 1: dependency of UPS output on be assured that critical loads will be combine to produce water, electrical ener- the input power supply optimally protected. This expression gy is also produced. They are significantly − Step 2: the voltage waveform of the signifies the quality of output voltage more expensive than batteries. Also, hy- UPS output under all operational conditions. drogen is an explosive gas, so great care − Step 3: the dynamic tolerance curves has to be taken with its storage. However, of the UPS output Energy storage systems though in its infancy hydrogen fuel cell Batteries are employed by almost all technology holds a promise as a power These steps are summarized in an AA-BB- (around 99 percent) UPS manufacturers reserve for UPS systems. CCC-type designator. ABB’s UPSs have to store energy to be used when the pow- the top ratings in each and are thus certi- er fails or goes out of range. Flywheels, Low total cost of ownership fied as “VFI-SS-111.” The designator ele- which store energy as kinetic energy, are ABB UPSs have a very low cost of owner- ments have the following meanings: an alternative to batteries. They are unaf- ship, partly because of the modularity − VFI (voltage and frequency indepen- and scalability de- dent): The output voltage is indepen- scribed above, but dent of all power line voltage and Each UPS module in ABB’s also because of frequency fluctuations and remains their best-in-class regulated within the tolerances set Conceptpower DPA 500 UPS energy efficiency. forth by IEC 61000-2-4. Usually, only ABB’s Concept- double-conversion UPSs meet the VFI has all the hardware and soft- power DPA 500, for criteria, while, for example, standby ware required for full system example, operates UPSs receive the lowest rating – VFD with an efficiency of (voltage and frequency dependent). operation. This ensures full up to 96 percent. − SS: total harmonics factor of the Its efficiency curve output voltage is less than 0.08 availability and reliability in the is very flat so there (IEC 61000-2-2) under all linear and event of a failure. are significant sav- under reference nonlinear loads. ings in every work- − 111: refers to three tolerance curves ing regime. This that describe the output voltage limits fected by cycling, require little cooling, gives this particular product the lowest versus duration in dynamic situations. can operate in a broad temperature total cost of ownership of any comparable The first digit shows the performance range. The initial costs of a flywheel sys- UPS system. at change of operating mode, eg, nor- tem are, however, significantly higher than mal mode – stored energy mode – those of a battery-based system and the The power usage effectiveness (PUE) ratio bypass mode; the second digit the load can only be supported for seconds is a measure used by the data center step linear load performance; and the rather than the minutes that a battery industry to characterize power efficiency. third digit the step nonlinear load system can manage.

32 ABB review 4|13 However, the total cost of ownership and 6 Standby UPS sustainability will drive development toward even more energy-efficient tech- Normal operation Bypass – common bypass configuration nologies.

Transformer-free UPSs will continue to dominate the market. The footprint of the Charger Output to critical UPS can be squeezed further, but the load copper needed to carry high current Mains supply cannot. Therefore, alternative or comple- Mechanical switch mentary UPS solutions that run at medi- Inverter um voltage (MV) levels will certainly show up. Due to the relatively smaller currents Battery involved, MV UPSs can be built that cater for tens of megawatts. These can

7 Line-interactive UPS then accommodate very large load blocks, or even entire data centers. Buck/boost transformer Normal operation

Alternative energy sources, smart grids, Bypass data center infrastructure management (DCIM) tools, etc., will set new stan-

Output to dards. Of course, other concepts as yet Charger Inverter critical unthought-of will arise too – after all, load data centers represent one of the fast- Mains supply est-growing and fastest-moving indus- Mechanical or static switch tries on the planet and, as such, are fertile areas for inspiration.

Battery

8 Double-conversion UPS

Normal operation

Bypass – split bypass configuration Mains supply Output to Rectifier Inverter critical load Mains supply

Static switch

Battery

The PUE is derived by dividing the total UPS developments power used by the facility, by the power Data centers are set to increase in size, used by the equipment related to data number and complexity, upping the storage. Data centers strive for a PUE ratio challenge to UPS products. Also, in- that is as close to unity as possible and creasingly sophisticated modular and high UPS efficiency helps achieve this. containerized data centers will require more versatile power protection schemes. Further, cooling costs in data centers are But, because continuous availability of substantial. Because they consume less power is the sole reason for the exis- power, high-efficiency UPSs require less tence of UPSs, reliability and maintain- cooling effort, creating further savings. ability will remain as cornerstones of ABB UPS solutions also have a very small UPS design. Juha Lantta footprint – ideal for data centers, where Newave SA, a member of the ABB Group real estate can be restricted and expen- Quartino, Switzerland sive. [email protected]

Power guarantee 33 Continuous power

Digital static transfer switches for increased data center reliability

CHRISTOPHER BELCASTRO, HANS PFITZER – The informa- disappears this fact must be instantly recognized and tion flowing through data centers is, in many cases, the backup power must be brought in so quickly that essential to the smooth running of modern society. For the changeover is invisible to the data center. Static this reason, it is vital that a data center is available at transfer switches provide an ideal way to do this and all times. The power grid cannot always be relied upon, these sophisticated products have become an estab- and, consequently, every data center has a backup lished component of all mission-critical data center power scheme. When the grid power degrades or architectures.

34 ABB review 4|13 (“preferred” and “alternate”) that remain 1 An ABB DSTS isolated from each another in all operating modes.

The power quality (PQ) on each source is continuously monitored in terms of its voltage, phase and waveform. If a source’s PQ falls outside user-defined limits for a set period of time, the DSTS makes the decision to transfer to the other source. Typically, the switching time from the detection of an anomaly to com- pletion of the transfer is one-quarter of a voltage cycle, or about four milliseconds. The switching technique employed is an open transition or “break before make” transfer. In this way, a data center load can be protected from even very short interruptions, or from any surges or sags in the primary power source.

The ABB DSTSs discussed in the subsequent sections are three-phase units operating between 100 and 4,000 A, at 208 to 600 V ➔ 1. − Infrared ports allow thermal monitor- To make the device maintainable without ing of critical load connections, transfer switch is an electrical causing downtime, the design of the ABB without introducing risk by removing device that switches a load DSTS includes plug-in style molded case equipment panels. between two power sources switches (MCSs) that provide isolation for − Redundant power supplies prevent A either manually or automati- regular maintenance and guided bypass. logic failures. cally. Thirty years ago, Cyberex, a mem- The MCS provides short-circuit interrupt − Redundant cooling fans with failure ber of the ABB Group, revolutionized capability, while eliminating nuisance trip- sensing avoid overheating or load loss power distribution with its invention of the ping arising from the lack of an overload due to fan failure. digital static transfer switch (DSTS). Since trip element. A traditional two-source − Shorted SCR detection prevents load then, Cyberex has installed more units DSTS incorporates six MCSs: two for loss should an outage occur. than any other manufacturer. The ABB source inputs (isolated), two for bypass − Downstream fault detection and DSTS uses power semiconductors, spe- (maintenance) and two parallel MCSs isolation prevents the propagation of cifically silicon-controlled rectifiers (SCRs), at the output to ensure no single point of high-current faults to other upstream as high-speed, open-transition switching failure through the switching elements distribution systems. devices to deliver quality power to a cus- and to electrically tomer’s critical load. “Digital” refers to the isolate the SCRs technologies implemented – namely, digi- when maintenance The STS is fed by two inde- tal signal processing (DSP) hardware and is required ➔ 2. patented software that performs real-time pendent power sources that analysis of the source waveforms and Reliability logic control of the DSTS. The features de- remain isolated from each scribed above are other in all operating modes Basic STS characteristics not the only as- ABB’s two-source DSTSs are designed to pects that enhance and each source’s voltage, power mission-critical loads where con- ABB’s DSTS reli- tinuous conditioned power and zero ability: phase and waveform is downtime are required [1,2]. The DSTS is − Type II rated continuously monitored. fed by two independent power sources SCRs provide optimal fault clearing capability that coordinates In addition, since 2004 an availability of Title picture with upstream protection. 99.9999 percent, or “six nines,” has been Discreetly, the ABB digital static transfer switch can − Redundant output switches prevent a observed for the DSTS. Further, it dis- instantaneously transfer power sources when the single point of failure. plays an operating efficiency of 99.60 per- preferred source falters in any way. The end result is continuous conditioned power to a data center’s cent at half load and 99.73 percent at full critical load. load.

Continuous power 35 2 Single-line diagram of a typical six-MCS With dynamic inrush restraint enabled, STS peak inrush current can be limited Source 1 Source 2 to less than 120 percent of the peak

S1 S2 full-load current of the transformer. input input MCS MCS Thyristors

S1 S2 bypass bypass MCS MCS

output MCS

Output

Data center availability should facility requirements increase. The In today’s business environment, data configuration does, however, have some centers are required to operate at disadvantages: extremely high reliability and efficiency − Single point of failure with common levels. Data center availability, a metric load bus and single-corded loads known as “nines” ➔ 3, is generally ex- − Faults will propagate through each pressed as: parallel redundant module Availability = MTBF/(MTBF+MTTR) − Low efficiency due to light loading on where: the UPSs MTBF = mean time between failures = uptime − UPS modules must be the same MTTR = mean time to repair = downtime. rating

Thus, as reliability and maintainability in- Distributed redundant design crease, so does availability. The need for A distributed redundant, or “catcher,” a common standard to classify data cen- design boasts independent input and ters’ reliability and maintainability became output feeds from three or more UPS apparent in the mid-1990s. To address modules that are coupled with two or this, the Uptime Institute developed a more STSs ➔ 4b. Advantages compared four-tiered classification benchmark that with parallel redundant (N+1) architec- has been utilized since 1995 ➔ 3. tures are: − High availability at a lower cost Data center architecture – − Higher efficiency than parallel redun- DSTS relevance dant and 2(N+1) designs Some simple configurations seen in data − Increased number of points of centers can highlight the importance and conditioned power, through UPS and flexibility of the DSTS. DSTS − Faults will propagate through one Parallel redundant (N+1) design UPS module only In general, an N+1 redundant design con- − Reduces single points of failure sists of paralleled UPS modules of the same capacity and configuration con- The disadvantage is: nected to a common output bus ➔ 4a. − DSTS cannot support multiple, The configuration is considered N+1 re- concurrent UPS failures. dundant if a system (N) has at least one additional autonomous backup element System plus system redundant with no STS (2N) (+1). The extra UPS module gives better System plus system redundant (2N) to- availability than the N configuration and pologies are the most reliable, and most the structure makes expansion easy expensive designs in the data center

36 ABB review 4|13 3 The four-tier classification of data centers The ABB DSTS can be applied Tier level Availability (%) Downtime (hr/yr) Average Common Requirements downtime names over 20 years as a two- or three- Tier I 99.671 28.82 96.07 N Nonredundant capacity components and source utility switch single, nonredundant distribution path to server loads for higher-avail

Tier II 99.741 22.69 75.63 Parallel Redundant capacity redundant components and sin- ability applications. N+1 gle, nonredundant distribution path to server loads

Tier III 99.982 1.58 5.26 Distributed Redundant capacity redundant components and redundant distribution paths to server loads

Tier IV 99.995 0.44 1.46 System plus Multiple isolated sys- system tems containing re- multiple parallel dundant capacity bus components and mul- 2N, 2N+1, 2N+2 tiple, active distribution paths to server loads

world ➔ 5a. Typically, dual-corded loads − Ability to service upstream equipment, are implemented. Advantages are: like switchgear, without going into − Separate power sources and paths bypass mode eliminate single points of failure − The STS provides redundancy for throughout the architecture dual-cord loads and protects against − Redundancy throughout the entire either source failing system − Effectively removes power quality − Ability to service upstream equipment issues upstream without causing a like switchgear without going into disturbance downstream bypass mode − Continuous conditioned power The disadvantages are: − High cost and large footprint The disadvantages are: − Low efficiency due to light loading on − High cost and large footprint the UPSs − Less efficient due to being lightly loaded Upstream comparisons − Does not maintain power to both Upstream, there will typically be a utility inputs of a dual-corded load in the and backup generator, which are event of UPS failure switched by an automatic transfer switch (ATS) ➔ 6a. Though low-cost, this solu- System plus system redundant with STS tion involves longer contact transfer By definition, Tier III and Tier IV systems times, delayed power generation startup supply continuous power to redundant and unpredictable generator perfor- dual-corded loads. However, they do not mance. provide redundant power availability to dual-corded loads that require quality The ABB DSTS can be applied as a two- power to not just one, but both cords con- or three-source utility switch for higher- tinuously. One way to provide this supple- availability applications ➔ 6b. The proba- mentary reliability is by applying STSs ➔ 5b. bility of a simultaneous power outage on a fully redundant, dual-feed system is rel- The advantages of this approach are: atively low. By implementing two indepen- − Highest level of availability dent feeds from separate substations, an − Continuous, multiple points of ABB DSTS can provide protection, conditioned power switching power and speeds, and plant- − Separate power sources and paths wide distribution efficiencies superior to eliminate single points of failure ATS. Cyberex has installed numerous throughout the architecture (redun- large DSTSs at power entry points in data dant throughout) centers and industrial facilities. Though

Continuous power 37 Digital signal pro- 4 Parallel redundant (N+1) design with 4 loads vs. distributed redundant “catcher” design cessing hardware UPS-1 Load PDU Load PDU STS UPS-1 UPS-2 and patented soft- Load PDU STS UPS-2 UPS-3 Load PDU Utility ware performs UPS-4 Utility UPS-C UPS-5 real-time analysis Load PDU UPS-6 of the waveforms Load PDU STS UPS-3 UPS-7 Load PDU and STS logic UPS-8 Load PDU STS UPS-4

control. Parallel redundant (N+1) Distributed redundant catcher UPS – single-corded load with STS – single-corded load

Availability (%) 99.976 (three nines) Availability (%) 99.976 (three nines)

Downtime (h/yr) 2.08 Downtime (h/yr) 2.10

Power interrup- 6.95 Power interrup- 6.99 tions/20 yr tions/20 yr

Cost ($) 1.7 million Cost ($) 1.28 million

4a (N+1) design 4b Distributed redundant “catcher” design

5 System plus system redundant with no STS vs. system plus system redundant with STS

PDU UPS-1 PDU STS UPS-1

Utility Utility

Load Load

PDU UPS-2 PDU STS UPS-2

Dual UPS – Dual UPS with STS – dual-corded load dual-corded load

Availability (%) 99.987 (three nines) Availability (%) 99.99999 (seven nines)

Downtime (h/yr) 1.12 Downtime (h/yr) 0.0005

Power interrup- 3.73 Power interrup- 0.0017 tions/20 yr tions/20 yr

Cost ($) 460,000 Cost ($) 540,000

5a System plus system redundant with no STS (2N) 5b With STS (2N)

UPS = uninterruptible power supply / PDU = power distribution unit

more expensive than the ATS approach, − Flexibility to add a third source and requiring two utility sources, the (eg, backup generator) DSTS approach has many advantages, − Lower cost than UPS including: − Highest level of upstream availability Digital STS advanced features − The DSTS removes all power anoma- Apart from the advantages described lies propagated from the utilities and above, the DSTS has further features distributes continuous power to all worth noting. downstream components − Ability to service one utility source Dynamic inrush restraint (DIR) while providing continuous condi- DIR limits downstream transformer inrush tioned power from a second utility current when switching between two source sources that are out of phase. This is − Extremely high electrical distribution done by continuously monitoring the efficiency levels transformer flux and precise timing of

38 ABB review 4|13 4) Once the SCR naturally commutates 6 Upstream comparisons off, the gate signal is enabled on the reciprocal device to complete the transfer. Utility Utility

Reliability delivers availability The ABB DSTS can effectively remove

Downstream ATS Downstream STS upstream power quality issues without causing a disturbance downstream. It Generator Utility can be a cost-effective replacement for an upstream ATS or even a facility-wide UPS system – generating improved levels of reliability while drastically reducing One source, Two sources - one generator - ATS upstream STS upstream footprint, managing higher electrical effi-

Availability (%) 99.994 (four nines) Availability (%) 99.9998 (five nines) ciencies, and reducing overall cost.

Downtime (h/yr) 0.49 Downtime (h/yr) 0.013

Power interrup- 1.64 Power interrup- 0.044 In system plus system redundant configu- tions/20 yr tions/20 yr rations, the highest level of availability can be achieved by providing mutual, dual- bus feeds to a DSTS. This architecture 6a Utility and generator with ATS 6b Dual-utility source with STS provides multiple layers of redundancy that eliminate single points of failure, down to and including dual-cord load 7 Transformer inrush current (can be up to 7,200 A for a full-load Ampere value of 600 A) power supplies. Finally, a DSTS also pro- when not using the DIR algorithm. vides superior fault isolation and increased protection during maintenance, Full-load amperes ensuring continuous conditioned power Inrush current is delivered to a customer’s critical load. Current (arbitrary units) Current

Time (arbitrary units)

the transfer so the flux does not exceed Smooth transfer the saturation point of the transformer’s The DSTS source transfer algorithm trans- core. Energizing a transformer results in a fers from an active set of SCRs to an inac- potential peak inrush current of 5 to 12 tive set by removing a gate signal from times full-load ampacity (FLA); transfer- two parallel-connected, opposite-sense, ring between out-of-phase sources re- current-carrying SCRs that, in combina- sults in a peak inrush current of up to 20 tion, carry AC in either direction. The trans- times FLA ➔ 7. fer process is simple: Christopher Belcastro 1) Removal of a gating signal on the active Hans Pfitzer With DIR enabled, peak inrush current source, due to the detection of poor PQ ABB Low Voltage Products can be limited to less than 1.2 times full- or a manual transfer request. Richmond, VA, United States load current of the transformer. 2) Current is sensed through the two active [email protected] SCRs to determine the current-carrying [email protected] PQ/sensing algorithms state of each device over a specific Two DSPs sample the sources 10,000 period. times per second and utilize patented al- 3) Once both states are determined, a gate References gorithms to detect source disruptions and signal is applied to the corresponding [1] IEEE Gold Book Std 493–1990, “Design of failures in less than 2 ms, thus enabling SCR in the inactive set. This enables Reliable Industrial and Commercial Power Systems,” New York, NY, 1991. transfers within a quarter cycle. current flow through this device while [2] T. A. Short, Distribution Reliability and Power simultaneously preventing current from Quality. 1st edition, Boca Raton, passing between the sources. FL: CRC Press, 2006.

Continuous power 39 40 ABB review 4|13 Automated excellence

New concepts in the management of data center infrastructure

JIM SHANAHAN – As data centers grew out of server closets to become the computing titans that now consume over 2 percent of grid power in many countries, they brought with them a legacy of automation systems that they had outgrown but to which they continued to cling. The industry has finally realized that modern data center infrastructure management (DCIM) tools need to provide scalable solutions that bring advanced technologies into play, enabling those who best leverage them to leapfrog their competitors. ABB is helping those customers differentiate themselves in a very fast-moving industry.

Title picture Sophisticated tools that allow all aspects of a data center to be managed in an integrated way are essential if an operator is to differentiate and survive in the very competitive data center world.

Automated excellence 41 1 Decathlon architecture

User interfaces Command center Portable client Web portal

TM Asset and Power Building capacity DecathlonTM management management planning secure cloud Application modules ® Global energy Energy Maintenance Other apps management management intelligence

Asset library Alarm Remote History and Control and Core management monitoring reporting automation functionality Decathlon aspect directory

Monitoring and secure control

External Mechanical Electrical IT and O/S Application Other interface management

− DCIM analyzes this data and provides actionable information about data center management. ata centers usually operate − DCIM is not a standalone solution, along lines that mirror their but a component of a comprehensive makeup. As a consequence, data center management strategy. D facility operations (mechanical and electrical systems) tend to run in iso- To the IT engineer, DCIM can be a tool lation from IT and server operations. This to manage server location, configura- silo approach makes it difficult to get an tion and application load; for the facili- overview of what is happening in the ties manager, it can be a system to control and monitor electrical and ABB has brought its best mechanical equipment; to a senior practice solutions from other manager, it can be a way to compare industries and merged them data centers and with new data-center-specific leverage business intelligence. ABB’s libraries and applications to DCIM product, DecathlonTM, is one form Decathlon. of the most advanced DCIM solu- data center as a whole, even though tions on the market today. Delivered via most critical decisions need to take hardware and software, the Decathlon account of the entire picture. system provides the tools to manage a flexible network of power, cooling and IT Initially, DCIM may seem confusing be- equipment. The information is present- cause the term is used so broadly. How- ed in a single operational environment ever, the definitions of DCIM published and via a single data source, which by leading industry research firms concur helps overcome information barriers. that: Both IT and facility personnel can work − DCIM requires instrumentation in together more effectively – sharing a order to gather and normalize data “single truth” from which they can index center metrics. and report their data center improvements.

42 ABB review 4|13 2 Decathlon workflow integration – CMMS work order Decathlon takes automation lessons learned from process industries and applies them to data centers.

Essentially, a data way to achieve such control is to look Decathlon also offers ad- center converts not just at the environmental tempera- power to transac- ture sensors around the racks, but to vanced control, maintenance tions and gener- look at onboard server temperatures too. ates a lot of data This means reading CPU temperatures management, strategic energy and heat (that has from each server via a simple network procurement and the ability to to be removed) in management protocol (SNMP), then aver- the process. It is aging this across each rack of, typically, shift computing loads between instructive to look 30 to 40 servers. By controlling the envi- at this entire chain ronment based on CPU temperature – data centers based on the of events in a little the hottest part of the data center – higher cost or availability of energy. more detail to un- efficiency can be achieved and problems derstand some of with individual servers can be detected the mechanisms early. (See article on data center cooling More recently, fully featured converged involved, some newer ideas around how on page 52 of this issue.) DCIM solutions have emerged that offer they can be managed and the value of a end-to-end visibility. Whoever pays the converged DCIM solution. Building management power bill can now measure data center A building management system (BMS) efficiency in terms of workload-per-kWh Keeping cool monitors and controls the environmental – for example, the number of SAP trans- The starting point for a DCIM project is and safety systems – such as those for actions per MW or the number of e-mails often a need to control or monitor the lighting, ventilation and fire – in a large processed per dollar. This visibility pro- physical environment around the servers. building. As concerns about energy con- vides new leverage for data center own- In recent years, it has become popular to servation gained critical mass, BMS fea- ers and operators, and drives efficiency raise server inlet temperatures to achieve ture enhancements evolved to become in the data center organization. ABB has higher efficiency because less cooling is more aligned to energy efficiency. Howev- brought its best practice solutions from then required. It is not uncommon now to er, a BMS cannot cope with the rapid and other industries and merged them with find “cold aisle” temperatures at server dynamic expansion (and consolidation) of new data-center-specific libraries and inlets in excess of 27 °C. This means the data center operations where data from applications to form Decathlon ➔ 1. As “hot aisle” at the server outlet can ex- onboard sensors in thousands of servers well as “normal” DCIM functions, Decath- ceed 40 °C. ABB robots are being con- at multiple sites are factored into uptime lon also offers advanced control, mainte- sidered for some duties, such as moving and optimization strategy and tactics. nance management, strategic energy servers or cables, in the hot aisle, where Decathlon, which is built on the ABB procurement and, through a concept humans cannot comfortably operate. Extended Automation System 800xA plat- known as software defined power, the form, collects, normalizes, records and ability to shift computing loads between In these extreme environments, tight analyzes the large amounts of data from data centers based on the cost or avail- control of temperature is critical to en- both IT and facility systems. Furthermore, ability of energy. sure the server does not overheat. One Decathlon exploits its rich history in control

Automated excellence 43 3 Chiller overview graphic

technologies and automation, such as tion of the entire power tree from the grid advanced process control, autotune and connection right down to each server advanced alarm handling, to optimizing motherboard. the data center. For the purpose of data center performance monitoring and opti- Capacity management mization, a traditional BMS is more prob- From the time a server enters the data lematic and expensive because it is not center in a box to the time it is decommis- designed for broad and granular data cap- sioned three years later, it goes through ture, analysis and user configuration. many stages of racking, imaging, burn-in, power and network allocation, live de- Decathlon tracks server loca- ployment and so on. All these stag- tion to automatically allocate es need to be tracked and man- a new server to an optimal aged. To accom- rack position to make best plish this, an asset management and use of available power, cool- capacity planning application is em- ing and network connections. ployed. Decathlon uses Nlyte or other Power monitoring technology partner solutions and syn- On the electrical side of the facility, the chronizes the server location information power chain from pylon to processor with its internal database. This application provides a myriad of opportunities to can automatically allocate a new server to monitor and optimize. Decathlon does an optimal rack and position within that not just measure and report on power rack to make best use of available power, from installed meters, breaking data cooling and network connections. This down by user, area and source, it also can extend the life of the entire data cen- analyzes power quality events such as ter by ensuring that all available capacity spikes, manages breakers for load shed- is used and that there is no “stranded” ding or alarming, and provides visualiza- power, cooling or network capacity. The

44 ABB review 4|13 4 Generation and trading chart

system also issues work orders to manage the entire process for server addi- By controlling tions, moves or changes, and can track which virtual machines, operating sys- the environment tems and applications run on each physical “server metal.” By combining the based on CPU asset management system’s knowledge temperature, of server physical location and connections with the real-time information on the higher efficiency server’s environment and onboard parameters, Decathlon can close the control can be achieved loop to provide tight control and ad- and problems vanced reporting across the traditional silos of facilities and IT operations. with individual

Asset health servers can be Apart from IT assets like servers and net- detected early. work switches, a normal data center has standby generators, UPSs, batteries, switchgear, chillers, pumps, computer operating correctly. Should they start to room air handlers or conditioners (CRAHs drift outside acceptable limits, a mainte- or CRACs), fire detection and suppres- nance work order can be raised even sion systems, access control systems, before the equipment itself goes into an leak detection systems, etc., all of which alarm state. This condition-based main- have regular maintenance requirements. tenance is further enhanced by Decath- Decathlon can be interfaced with some lon’s remote operations center (ROC) industry-standard computerized mainte- service where data center subject matter nance management systems (CMMSs) experts (SMEs) are on hand to prevent such as SAP or Maximo or it can be bun- an incident from escalating to an outage dled with an ABB CMMS such as Servi- by assisting the responding technician. cePro or Ventyx Ellipse ➔ 2. Decathlon can deploy asset monitors on critical equipment items to ensure they are

Automated excellence 45 Compute load can 5 A typical data center power one-line diagram in Decathlon be shifted from one bank of servers to another, or from one data center to another, to save energy or for reasons of cost or availability of power.

Moving loads instance, rather than perform monthly Decathlon can monitor CPU utilization generator tests where the power is dis- across all of the servers in a data center, sipated into a load bank, the generators or across multiple data centers. In a pro- are run when needed by the grid and the cess known as run book automation, owner can earn significant revenue. This and through integration with virtualiza- bidirectional grid connection also signifi- tion solutions, compute load can be cantly improves the resilience of the data shifted from one bank of servers to an- center over a conventional automatic other, or from one data center to another. transfer switch (ATS) arrangement. This can be done to save energy, where the unused servers are put into a sleep Server efficiency can also be increased mode, or for reasons of cost or availabil- by using server “power capping,” where ity of power. Global energy intelligence a limit is imposed at certain times on the (GEI) provides a data center owner with power that can be drawn by CPUs per- a single interface to all of the world’s forming noncritical functions. Increased energy markets so that IT loads can be utilization can be achieved without in- shifted between data centers based creased risk by balancing compute infra- on power market or demand-response structure with actual demand. Decathlon opportunities. ABB’s investment in Power determines the optimal capacity required Assure, a company based in Santa Clara for a given IT load and dynamically ad- (United States), delivers GEI, run book justs server availability in real time along automation and power capping to with required cooling and facility resourc- Decathlon. Energy market pricing and es. This, in turn, results in significantly trading facilities can also be provided to increased operational efficiency and Decathlon through the Ventyx suite of decreased energy costs. (Please refer to products. the article on data center design optimization on page 48 of this issue of ABB Clever energy Review.) Decathlon uses the features of Energy Manager, a solution successfully used in High visibility pulp and paper and other industries, to- Decathlon presents all of this informa- gether with GEI to help data centers min- tion to the user through a “single pane imize their peak demand, or to make rev- of glass” ➔ 3 – 5. Basic measures of data enue from their grid connection – for center facility efficiencies like power example by using their standby genera- usage effectiveness (PUE) are supple- tors to sell power back to the grid under mented in Decathlon’s configurable a demand-response program. In this dashboards and reports by more com-

46 ABB review 4|13 6 The data center maturity model

Investing in capabilities for rapid response – IT & facilities automation to compete and win in (controls) the market place Investing in capabilities – More flexibility to for availability, achieve an agile data sustainability analysis Performance center and growth optimization – Global energy Blind faith intelligence – Operational Uptime honeymoon – Rapid service savings Low efficiency Resource responsiveness consolidation – Energy efficiency – Dynamic IT – – Visibility and automation of Backup and – Hardware savings better control facilities supporting IT over-provisioning – Data center – Lower risk; better load shifting in any savings availability data center – Managed chaos – Environmental environment – Run to fail impact Strategic value of the data center to enterprise

Stage 1 Stage 2 Stage 3 Stage 4

Reactive, tactical, discrete improvements Proactive, strategic, operational improvements Time

Backup andover-provisioning Resource consolidation

Performance optimization IT & facilities automation (controls)

prehensive metrics like corporate aver- center. This means that as a data center The underlying trend in data centers age data center efficiency (CADE) that starts to deploy a DCIM solution, it can today is that over-provisioning of equip- calculate efficiencies by taking server progressively move up the data center ment is being supplanted by software utilization into account. The jury is still maturity model in manageable steps, resilience. The future – where entire data out as to which metric will replace PUE rather than have to deploy everything centers go on a standby mode and con- as a more comprehensive data center at once. Most owners starting a DCIM sume no power or where an entire com- efficiency indicator. However, with its deployment will be at stages one or two pute load can be seamlessly shifted from end-to-end visibility, Decathlon offers of the model ➔ 6. one data center to another based on energy availability or cost – is today’s An existing facility emerging reality. And it is all enabled by Decathlon helps minimize peak operator may have DCIM. had a couple of demand or helps generate years of “uptime honeymoon” with revenue by using standby a new facility be- generators to sell power back fore gradually real- izing that more to the grid under a demand- attention to real- time monitoring response program. and maintenance is required to avoid the data center owner or operator a incidents and outages. In this instance, a unique and comprehensive view into power management solution can im- their systems with the possibility to con- prove uptime by providing early warning figure custom performance indicators. of issues like breaker trips or power spikes before they cause outages, while Apps and modules asset monitors can prevent outages on Decathlon is a modular system, meaning critical equipment through condition- that once the core system is installed, based maintenance. A more mature additional application modules can be operator can turn his grid connection added easily. In practice, each applica- from a cost item to a source of revenue tion enhances and leverages the core while increasing uptime by installing a database so that as mechanical, electri- bidirectional grid connection and partici- cal or IT equipment and systems are pating in automated demand-response Jim Shanahan added, the visualization, reporting and programs. ABB Process Automation, Control Technologies analytics applications can provide a Dublin, Ireland more comprehensive picture of the data [email protected]

Automated excellence 47 Design decisions

What does ABB PATRICK KOMISCHKE – Design should always be driven by the purpose of the end product, and this should be reflected in the requirements of contribute to the the customer or end-user. These requirements, including codes and industrial standards, are combined with the capabilities and compe- design of data tences of the supplier to create the product. The design of the electrifi- centers? cation of data centers occurs in a very dynamic environment. It is not a completely new field of engineering, but the range of design approaches and the rapid development of technologies and customer requests create numerous challenges. This is reflected in the fact that various standards for data center design exist.

48 ABB review 4|13 1 Advantages of ABB’s systems approach

Savings potential compared with year prior to kickoff (as percentage) 45

0 What is Review of Review siting Finalize Develop value- Project Cost savings Ongoing problem that functional options conceptual engineered kickoff developed service to needs to be options design design during reduce O&M solved execution costs and risks

– Lower project risks. ABB provides a mix of global equipment – ABB systems project awarded on a firm technology, project execution expertise, lump-sum basis (ABB paid for results, project/discipline/product engineering along not effort). with the expertise of integrating third-party – One integrated project schedule coordinated contractors to deliver the optimum mix of with customer and owner engineers. technology, project engineering, project – ABB system model reduces the number management, and local expertise for the of companies involved in the project, most cost effective solution. hence fewer interfaces to coordinate. – ABB experts support the project – Lower customer project management costs. organization in all disciplines. – Lower project execution costs. – ABB projects feature direct involvement – Lower customer warranty, operations and t the beginning of the design of ABB factory personnel. maintenance costs. process of a data center are – An ABB Manager at each ABB factory is – ABB has expertise in a broad range of fields. responsible for equipment to be delivered. – ABB has global expertise in the data center the identification of the load – ABB system approach reduces delivery industry. A requirements that the center challenges by securing priority production – Technical experts from different disciplines will need to handle and the required reli- slots from its factories. and factories. ensure the best solution from ability. The reliability definition is inter- – ABB system projects reduce emergent a single source. technology risks by accessing factory – Lower cost of ownership (see inset graphic). preted in the context of the Tier concept. experts on a real-time basis. Additional parameters to taken into – Lower awarded costs. account are geographical and physical locations as well as security aspects and required compatibility with other systems. ensure a smooth integration and cooper- turer) systems approach. Here, the full ation, several control systems based on strength of the company’s wide product A typical design starts at the high-voltage different software platforms are used to portfolio is paired with the competence (HV) connection where the power is combine these components. of an OEM system integrator. This means drawn from independent sources. Power that the products do not only come from sources can be utilities or independent The handling of this huge span of disci- a single company, but are integrated into energy suppliers. From here, the power plines and technologies requires an or- one system and supplied to customers passes through the medium- and low- ganization covering a broad palette of from a single source. voltage (MV and LV) distribution, which engineering resources and the associat- connects and combines different sources ed expertise under one umbrella. It is fur- The acquisitions ABB made over recent while feeding and supplying the energy to thermore important to work closely with years have expanded the company’s the points where it is required: primarily the customer in the selection of the opti- product portfolio further, meaning the the server racks but also all the auxiliary mal design. company can now cover almost the full systems supporting the reliable and safe spectrum of data center electrification. operation of a data center. These are Why is the ABB approach different? Indeed if there is a gap, the systems ap- mainly hardware implementations, but to ABB draws on a comprehensive and proach ensures that a third party product long experience as a product supplier for can be selected and seamlessly integrat- data center applications. In recent years ed with ABB’s offering. Title picture The design of a data center is not only about increasing efforts were made to package choosing between myriad competing suppliers and these products and to offer customers a The systems approach, which is equal to their products, but also between different design broader product portfolio from a single an EPC (engineering, procurement, con- and operational philosophies. Decisions made at an supplier. The real potential and advan- struction), offers significant advantages early stage will have implications and repercussions throughout the life of the data center. So what is the tage of ABB’s offering, however, lies is in to customers or investors in the data best way through this labyrinth of decisions? the OEM (original equipment manufac- center industry ➔ 1 – 2.

Design decisions 49 2 Traditional outsourcing options

Design-bid-build Engineer-procure-construct (DBB) (EPC)

Customer project Customer project manager manager

EPC Engineering Procurement Construction project department department department manager

Engineering Equipment Construction Engineering Equipment Contractors contractors suppliers contractors

Multiple contracts awarded One contract awarded to and managed by multiple to and managed by one organizations organization – Low awarded cost – Lowest total cost – Most customer resources – Fewer customer resources – High customer project risk – Low customer project risk – Longer project schedule – Shorter project schedule

Internal ABB project Design approach in detail ABB can now To test and cement the systems ap- Starting on the HV side, the design had proach, ABB began an internal project in to consider different solutions such as cover almost 2012 with the goal of designing a 20 MW air- and gas-insulated switchgear (AIS the full spectrum Tier III data center design with maximum and GIS) ➔ 5, different transformer types, ABB content, while remaining as close and control systems – to ensure reliabili- of data center as possible to market typical solutions. ty but also correct connection and grid integration on the electrification. utility side. The AIS Technical experts from differ- vs. GIS comparison is a widely dis- ent disciplines and factories. cussed topic and a very good example ensure the best solution out for demonstrating of one hand. the advantage of a system approach. Before committing The target was to ensure the systems to a decision, ABB is able to look at the approach by using ABB products as well grid where the data center should be in- as products from the recently acquired tegrated and make a recommendation in companies, Baldor and Thomas & Betts, collaboration with the customer and the and integrate them into an ABB data related utility. An example of the evalua- center solution. tion process is shown in ➔ 3. Conclu- sions of this evaluation are shown in ➔ 4. The project’s specified deliverables were single line diagrams, physical layouts, In addition to the system/grid analysis, specifications and other supporting ma- other factors such as physical location terial that could be used as a basis to and safety requirements can play a role. deliver both data center equipment and An example of a physical AIS vs. GIS integration out of one hand (the system comparison (showing significant space approach) while being closely attuned to savings) is shown in ➔ 5. customer and market requirements. This internal project led to a successful mar- The same design steps and analysis are ket introduction of the defined system also applicable for the MV side, but in ad- approach. dition, the integration of loads and subsystems such as generator sets, needed to be

50 ABB review 4|13 3 Substation optimization process

C Functional Methodology major pillars requirements Selecting u Technical & Ranking 1 Collecting S/S functional requirements 4 Economic analysis Substation optimal s economical substation 2 Identifying S/S alternatives 5 Ranking S/S alternatives alternatives substation 3 Reliability analysis 6 Selecting optimal S/S solution t analysis alternatives configuration o Customer m preferences 65 e 60 r TurboSpec SubRel SubRank 55 Functional specification Functional specification Offering 50

Failure (MUSD) 40 outages frequency 35 0 – 0.02 30 GIS AIS AIS-DCB

0.02 – 0.04 25 0.04 – 0.06 0 5 10 15 20 25 30 35 40 45 50 55

Cost of interupption ($/kW) 0.06 – 0.08

> 0.08

Configuration Total OF Total OD Failure OF Failure OD Maint. OF Maint. OD Type /yr hr/yr /yr hr/yr /yr hr/yr AIS 0.94748 14.57 0.09748 3.17 0.85 11.40

AIS-DCB 0.54136 13.01 0.09136 3.21 0.45 9.80

GIS 0.36770 12.69 0.05100 3.16 0.32 9.53

4 Assessment process tools 5 ABB high-voltage gas-insulated switchgear (actual AIS footprint vs. GIS footprint)

Captures customer objectives Ranks power system alternatives

Life cyle cost – Initial capital costs – Reliability cost – O&M cost

Performance – Flexibility – Safety – Automation level – Technology vintage

Environmental factors – Ecological impact – Air pollution tolerance – Appearance/aesthetics – Audible noise generated – EMF fenerated – Radio/television – Interference generated – Disposal concerns GIS substations cover approximately 15 percent of comparable conventional substation footprint while delivering increased reliability.

considered as options. Decisions such as systems approach displays its value as Looking forward indoor vs. outdoor, physically together with ABB can support decisions on integrat- Facing the constraints in the electrical in- the data center or not, noise, safety etc. ing a third party product or launch an infrastructure sector, such as limited quali- must be reviewed and answered. ABB ternal development effort. fied in-house resources, customers are also provides the full expertise to integrate increasingly seeing the value offered by this part into the optimal solution. Beyond the traditional HV/MV/LV disci- ABB’s systems approach. Opportunities, plines, ABB’s portfolio includes other however, will still remain for customers The numerically most significant selec- products and software solutions that fit in interested in ABB’s products and seek- tion of ABB products and variants con- the data center landscape and can be ing to combine them with solutions in- sidered in this internal project were those used to combine, connect or extend the house. on the LV level and the interface to the above solutions. A notable part of this server structure (where ABB offers vast category is ABB’s systems integration experience in the data center industry). expertise that can combine products to a There remain, however some gaps where system. By focusing on the systems ap- Patrick Komischke ABB has no products to cover a given proach and drawing on the full knowledge ABB Power Systems function or a product exists that would from across the company, an optimal so- Raleigh, NC, United States be difficult to integrate. Here again, the lution can be delivered to every customer. [email protected]

Design decisions 51 52 ABB review 4|13 Keeping it cool

Optimal cooling systems design and management

SHRIKANT BHAT, CARSTEN FRANKE, LENNART MERKERT, NAVEEN BHUTANI – Heat generation is a cause for major concern in data centers. Indeed, up to 45 percent of the total energy used in a data center can go to just cooling the server racks [1]. This figure is set to rise as servers become ever more compact and, as a result, power densities increase. Cooling technologies, power management and associated control systems are rapidly evolving to combat this escalating heat problem. A modern cooling system that can rise to the challenge of this situation must adopt a radical approach and focus on improved energy efficiency, integrated management and high reliability for the entire data center. ABB’s experience in managing critical power systems and complex industrial processes stands it in good stead to take up the cooling challenges a data center presents.

Title picture A large part of the energy consumed by a data center ends up as waste heat. Dealing with such a large heat load in such a small volume requires sophisticated cooling technology and techniques. Photo courtesy: © 2013 Michelle Kiener

Keeping it cool 53 1 Heat flow in data centers

Waste heat

Intermediate Source Heat removed from medium to transfer Sink (IT equipment) the source by fans, heat (chiller) plus (environment) pumps, etc. economizer where fitted

Electric power

Novel cooling designs Liquid cooling, absorption cooling and There are various cooling technologies at evaporation-based cooling have already different stages of commercial maturity been practiced in other industries. How- and some of these show promising re- ever, data centers pose unique challenges sults ➔ 3. Aisle containment, for instance, in terms of the nonhomogeneous heat ntil recently, heat management is practiced commercially and can improve generation associated with highly dynamic techniques in data centers were system efficiency by up to 30 percent[3] . load behavior and the requirements for based on the methods used to On-chip cooling is at a preliminary research high reliability. ABB has expertise in ensur- U cool buildings. Thermally, a serv- phase and has been reported to achieve ing high reliability for critical power system er was treated as an “equivalent human” cooling of up to 15 °C for heat fluxes as components along with extensive experi- and this assumption worked fairly well. high as 1,300 W/cm2 [4]. Liquid cooling is ence in integrated process management. However, the heat flux from commercial expected to reduce cooling energy con- This capability can help address the chal- microprocessors has increased from sumption by as much as 50 percent com- lenges posed by integration of novel cool- around 1 W/cm2 to 100 W/cm2 over the pared with conventional air-cooled sys- ing technologies with data centers. last decade and this is expected to rise tems and is being commercialized now. further [2]. This represent a massive in- Membrane air drying and evaporative Monitoring and sensing crease on the demands faced by any cool- cooling is reported to reduce energy The first step in managing and controlling ing system. requirements by up to 86.2 percent com- cooling is to monitor the thermal behavior pared with conven- Cooling in data centers involves the trans- tional mechanical fer of heat generated from IT equipment vapor compression The clear target of cooling (source) to the environment (sink) in a two- systems [5]. step process: The heat is first transported efficiency measures, then, is by a medium (air or liquid) out of the server The waste heat racks and then it is rejected to the environ- from a data center to reduce the energy required ment ➔ 1. Both these steps consume elec- can be augmented to remove the heat and trical energy. The target of cooling efficien- by solar thermal cy measures, then, is to reduce the energy energy to drive an recover and reuse as much required to remove the heat and recover absorption chiller, and reuse as much of it as possible. This thus reducing pow- of it as possible. can be achieved through innovations in the er usage effective- design of the cooling system itself as well ness to less than as by inventive operating strategies – eg, one (absorption chillers use the hot water of the data center. Hot spots are a major smart sensing and monitoring, and inte- from the primary cooling loop, and solar cause of concern and these can be de- grated system management. heat on occasion, to drive an additional tected using infrared sensing or wireless chiller loop). sensors. Soft sensors that combine data Cooling system design and management already available with detailed computa- has several important areas and it is worth- tional fluid dynamics models, or empirical while to examine each of these ➔ 2. models, are another important tool.

54 ABB review 4|13 2 Focus areas for cooling system management IT load management is often de- Sensing/ monitoring coupled from the cooling and power systems – so IT Novel cooling Cooling technologies control jobs are started Cooling system with no regard for management the cooling or power required. IT/power/ cooling Reliability integrated To avoid this, management coordination of all three subsystems is required. It is also important to benchmark emerging Optimizing such a cooling system in an technologies: integrated way involves minimizing the net − What are the current cooling tech- cost of power while ensuring that cooling nologies and their limitations? requirements for a given IT load are met. − What advanced solutions can be This often results in a complex demand- integrated with the cooling system? response problem that involves inputs of − Up to what level is integration or weather forecast, energy prices and load- adaptation feasible and what are the versus-efficiency curves for all the equip- system limitations? ment involved. An integrated cooling − What is the impact of a new solution approach involving only economizer inte- on the reliability of the overall cooling gration, along with model predictive con- and IT system? trol strategies for temperature control, has − What will be the value (cost benefits, been shown to reduce cooling manage- return on investment, etc.) of the ment costs by up to 30 percent [6]. This newly added resource? situation could be further improved by the use of additional storage and demand- ABB has demonstrated the use of con- response management to exploit energy cepts such as infrared sensing, wireless price variation. communication, soft sensing and finger- printing across different application areas A modular approach in the power and automation domain. This Modular cooling units allow data centers to know-how can be extended, with suitable expand their capacities incrementally. So adaptations, toward data center perfor- popular have such units become that they mance monitoring. now constitute a de facto design standard. However, they present a challenge to inte- Cooling control grated cooling control as there is an inter- A data center cooling unit has a chiller, action between them and related common cooling tower, pumps and thermal stor- facilities such as the chiller, evaporator and age ➔ 4. It often also has an economizer, economizer. This poses additional con- which provides a form of “free cooling.” straints on the integrated cooling control Economizers complement the existing problem described above. cooling by drawing in colder outside air and using it to reduce chiller energy con- ABB’s cpmPlus Energy Manager has the sumption. The external air passes through ability to handle such integrated demand one or more sets of filters to catch particu- response management problems to help lates that might harm the hardware. It is customers realize additional benefits. also conditioned to an appropriate relative humidity.

Keeping it cool 55 Optimizing a cooling system 3 Drivers for novel cooling design and representative cases Driver Representative Comments in an integrated way involves cases Thermodynamic Aisle containment Efforts targeted toward minimizing minimizing the net cost of efficiency On-chip cooling energy and exergy loss by localized heat removal and avoidance of mixing different power while ensuring that temperature streams. Materials Liquid cooling Novel materials are offering cooling IT requirements are higher efficiency and more Membrane air rapid heat removal. met. This often results in a drying and cooling complex demand response Waste Absorption cooling Cooling with waste heat recovered heat recovery from data centers is one of the most problem. promising options.

Renewable Solar cooling Solar cooling is one of the most integration promising options for using renewables for data center cooling.

Integrated management of power, consumption. This technique, called dy- IT and cooling namic voltage and frequency scaling In almost all existing data centers the IT (DVFS), is performed in such a way so as load management is not coupled with the to ensure IT jobs do not violate their given cooling management or the power supply. service level agreements (SLAs). IT jobs That means the IT load management soft- can be migrated to other servers, too, to ware makes an independent decision save power or cooling. In the past, such when to start new IT jobs, or when to migration was limited to a very few appli- migrate running jobs, without any consid- cations that supported check-pointing, but eration for the cooling or power required. increased use of virtualized servers now This “selfish” behavior can reduce the makes migration easier. power used by the IT equipment, but at the expense of a higher cooling energy Coordinated management can be extend- consumption. ed to incorporate resources not just from one data center but from several, geo- To avoid such scenarios, coordination of all graphically distributed data centers. This three subsystems is required. Further- can lead to further energy savings of 5 to more, it is also necessary to have a dy- 10 percent. The main advantage of spread- namic and predictive IT load management ing IT loads between data centers is that tool so that the data center location and the installed capacity per data center can corresponding time-varying energy provi- be smaller than the maximum that would sioning can be taken into account. Such be needed were the data center to operate an advanced load management, which in isolation, as some resources can be could be integrated with ABB’s data center shared. This indirectly also leads to a bet- infrastructure management (DCIM) sys- ter energy usage. An IT load-sharing strat- tem, DecathlonTM, can lead to energy sav- egy exploits time-of-day energy price vari- ings of 20 to 40 percent [7]. ations and ambient temperature differences between locations. Energy price predic- Such a solution can also directly accom- tions and cooling forecasts can easily be modate maintenance aspects – for exam- extracted from Decathlon, leaving only the ple, by load sharing among servers and information flow to the IT load manage- their related cooling devices to equalize ment to be established. component aging. It can help with power management too: Should cooling require- IT load management across data centers ments or energy prices reach critical val- provides benefits but is also subject to ues, Decathlon, for example, can dynami- legal and logistical constraints. For exam- cally lower the voltage supplied to ple, data may be bound to a certain juris- components or reduce the clock frequen- diction, thus limiting migration options. In cy to reduce cooling needs and energy addition, security aspects and data pro-

56 ABB review 4|13 4 Schematic of data center showing cooling, IT and power components Advanced load Cooling system components management, which could be

Cooling Free Thermal Chiller Pumps integrated with tower cooling storage ABB’s Decathlon

Power DCIM system, can (conventional and/or renewable) lead to energy savings of 20 to

Power conversion/distribution units, UPS 40 percent.

IT load

Shrikant Bhat Naveen Bhutani ABB Corporate Research tection become important if the data cen- upcoming failures, so monitoring the oper- Bangalore, India ters belong to different legal entities. Fur- ating conditions of critical components [email protected] thermore, the additional energy demand can allow better planning of maintenance [email protected] and communication costs involved in and replacement actions. migration must be considered. Carsten Franke For example, the voltages across several ABB Corporate Research Reliability capacitors of a power converter show Baden-Dättwil, Switzerland Fluctuating humidity, poor air quality and massive voltage drops and unusual oscilla- [email protected] temperature variations are the main phe- tions shortly before the power adapter nomena related to the use of an econo- fails. If such deviations are monitored and Lennart Merkert mizer that impact reliability. To improve reli- automatically tracked, preventive actions ABB Corporate Research ability, the intake air quality can be like repair or replacement can be initiated Ladenburg, Germany monitored and if it drops below certain just when they are needed. Equipment [email protected] standards preventive actions can be tak- downtime is thus decreased as failures are en. For example, Decathlon can automati- predicted and corrected before equipment cally close external air intake vents and drops out. Consequently, reliability and References [1] J. B. Marcinichen et al., “A review of on-chip switch to another means of cooling when availability of the data center are increased. micro-evaporation: Experimental evaluation of air quality standards are threatened. In addition, unnecessary maintenance and liquid pumping and vapor compression driven replacement costs are eliminated. cooling systems and control,” Applied Energy, Hot spots also detrimentally affect reliabili- vol. 92, issue C, pp. 147–161, 2012. [2] J. B. Marcinichen et al., “On-chip two-phase ty. Effective monitoring and control can ABB has demonstrated its capability to cooling of data centers: Cooling system and deal with these without overprovision of monitor and ensure system reliability in a energy recovery evaluation,” Applied Thermal cooling for the entire data center. This wide range of mission-critical applications Engineering, vol. 41, pp. 36–51, 2012. directly reduces energy costs. in industrial power and automation set- [3] Subzero Engineering Inc., (2013, August) Hot aisle containment. [Online]. Available: tings. This experience puts ABB in a per- http://www.subzeroeng.com/containment/ Another approach used to increase reli- fect position to manage mission-critical hot-aisle-containment ability is to regularly maintain or replace data centers for customers, especially [4] C. Ihtesham et al., “On-chip cooling by critical equipment before failure occurs. when tools like Decathlon are available. superlattice-based thin-film thermoelectrics,” Nature Nanotechnology, Vol. 4, Issue 4, The intervention can occur after a fixed pe- pp. 235–238, 2009. riod defined by the mean time between [5] El-Dessouky et al., “A novel air conditioning failures or the manufacturer’s warranty. system: Membrane air drying and evaporative However, a fixed period approach is not cooling,” Trans. IChemE, Vol. 78, Part A, pp. 999–1009, 2000. ideal. Load profiles, and environmental and [6] R. Zhou et al., “Optimization and control operating conditions, might vary from the of cooling microgrids for data centers,” average values indicated by the manufac- HP Technical Report, 2012. turer so it is better to tailor maintenance [7] W. Nebel et al., “Untersuchung des Potentials von rechenzentrenübergreifendem Lastmanage- and replacement for each piece of equipment zur Reduzierung des Energieverbrauchs in ment individually. A loss of performance or der KIT,” Technical report, OFFIS Institut für unusual equipment behavior can indicate Informatik, 2009.

Keeping it cool 57 In the crystal ball

Looking ahead at data center design optimization

ZHENYUAN WANG, ALEXANDRE OUDALOV, FRANCISCO CANALES, ERNST SCHOLTZ – “Data center design optimization” is a phrase that rolls easily off the tongue, but actually optimizing the design of a data center is a good deal more difficult than it sounds because the owners, architects and engineers who have a say in the design may all have different priorities. The ability to reconcile the desires of these parties as well as accommodate current and future trends in the industry is a core skill in the art of data center optimization. Energy efficiency is one particularly significant and dynamic trend, and the DC-only, energy self-sufficient feature is one aspect of this trend that is attracting major attention worldwide.

58 ABB review 4|13 For data centers, a DC-only world would be perfect, especially as DC is native to most renewable energy sources.

onventional power genera- wind sources, zero-net-energy buildings huge financial impact on a data center tors are usually alternating (ZEBs) can become a self-sufficient alter- owner/operator. A self-healing function current (AC) based and be- native to conventional, externally pow- in the power supply network can improve C tween the generator and the ered buildings. Data centers are a major reliability and this is becoming increas- direct current (DC) electronic loads in, application area of this vision. ingly popular in data centers. On the say, a data center, there can be many other hand, reliability and availability im- wasteful AC/DC/AC/DC conversion stag- Other considerations provement often incurs more cost. es. A DC-only world would be perfect, For data center optimization, however, especially as DC is native to most renew- there are considerations other than Protection and safety able energy sources. This DC vision has energy efficiency. Appropriate protection and safety mea- inspired, for example, the DC microgrid- sures have to be rigorously implemented. enabled “enernet” ideas of the EMerge Capex and opex Alliance – a not-for-profit, open industry Many factors impact the ultimate cost of Scalability association that is promoting the rapid a particular architecture – for example, To meet growing requirements, some adoption of safe DC power distribution in mitigating harmonic currents injected to data center owners plan to incrementally commercial buildings through the devel- the AC network may require filtering expand server capability and power ca- opment of appropriate standards [1]. By equipment to be inserted between the pacity. The latter may involve backup reducing the number of AC/DC conver- utility grid and the data center, thus generator type and number consider- sion stages in typical AC-powered elec- increasing capex. ations, modular UPS converter/battery tronics, a DC building can be typically five configuration, etc. to fifteen percent more efficient. Further, Reliability and availability by producing electrical energy locally Conventional AC data center designs are Footprint from biofuel, solar photovoltaic (PV) and classified into different tiers and each tier A smaller footprint is advantageous where has its own reliability and availability re- real estate is costly. However, this neces- quirements (see pages 11–15 of this edi- sitates higher power density in server Title picture Server performance is just one of many factors to tion of ABB Review). Apart from public racks, UPS converters, etc. and trans- be considered when designing a data center. image damage, outages can also have a lates into higher cooling system costs.

In the crystal ball 59 1 Finding the minimum TCO for a single performance target (eg, reliability, efficiency, environment impact)

Capex

Opex

Cost Total cost

Low Performance High

Renewables gives the minimum total cost of owner- By producing elec- Renewable energy sources, especially ship (TCO) for a given performance tar- PV and wind, should be easily accom- get ➔ 1. For others, it is the one with the trical energy locally modated. Use of renewables polishes least environmental impact, or with the the data center’s public image and addi- smallest footprint, or the highest efficien- from biofuel, pho- tional capex can often be recouped in cy, and so on. For a greenfield developer tovoltaic and wind renewable-resource-rich locations. Glob- with a strong sense of environmental ally, the “green” data center is a growing responsibility and a strong capital posi- sources, zero-net- trend. tion, “optimal” would most likely mean “greenest”; for a small developer who energy buildings Zero-net-energy (ZNE) wants a quick return on investment, the can become a ZNE data centers are usually smaller smallest initial capex may be his “opti- than average and often have access to mal” and he may not be interested in self-sufficient alter- renewable energy resources. A reliable costly renewable technologies now. utility backup and service agreement as native to conven- well as energy storage will be needed in An optimal data center architecture tional, externally- most cases. design is always possible for a given data center developer with clear objec- powered buildings. Cooling tives in mind. But some fundamental Modern data centers have rack power assumptions and requirements must be densities over 10 kW/rack and this will discussed: continue to increase. Liquid cooling ap- − The number of years the data center plies over 20 kW/rack – this will translate should function before a major into higher initial capex. makeover. − The geographical location of the Location intended data center, as this deter- The geographical location of the data mines the cost of real estate and center is a consideration when there are energy, alternative energy supply multiple options. The location impacts the potential, weather (cooling costs) and real estate cost, cost of electricity, the factors such as contracts with utilities initial and operating cost of cooling, etc. to provide ancillary services, or with other building owners to provide The question is: Given so many intercon- centralized heating services (this can nected factors affecting the final archi- help to offset expenses). tecture decision, how can one determine − Average server rack power density for an optimal architecture? the planned functional lifetime of the data center. Is an optimal design even possible? − The reliability and availability targets The definition of an “optimal design” is or, alternatively, the annual outage important. For some, it is the design that penalty that can be tolerated.

60 ABB review 4|13 2 Data center architecture design optimization process. Arrows show dependencies; engineering analysis items are in the light blue blocks.

Overall architecture design assessment (Weighted summation)

Capex Opex Data center estimation estimation design attribute weighting factors PDU/STS/PSU Reliability/ analysis availability analysis

Server room power Emergency/backup Safety analysis distribution analysis Scalability analysis power supply analysis System efficiency analysis (System and people (Server rack power density (Future retrofit (Scalability, footprint and (At different loading levels) protection) and server room cooling potential) power quality requirements) power requirement)

Site location and associated characteristics, total IT load, average server rack power density, cooling technology preferences, architecture designs

Text colors: Data center owner Architect Engineers

− Site constraints (available space, Most importantly, the owner gives major utility supplies and connection input to the process due to the fact that Scalability is impor- requirements). it is he who will weight the different attri- − Long-term plan for the site and the butes in the overall assessment. tant as, to meet data center. The architect’s role growing require- Given the definition of “optimal” and the Based on the owner’s inputs, the data ments, some data fundamental assumptions and require- center architect will come up with several ments, multiple data center architecture designs. These designs can be based on center owners plan designs can be developed and analyzed DC, conventional AC or a mixture of the to determine the best candidate. This two. A design can also incorporate mul- to incrementally process, however, requires the involve- tiple emergency/backup energy sources, expand server ment of all parties: the owner, the archi- protection schemes, etc. In principle, the tect and the engineers (for IT, network, architecture will roughly determine the capability and electrical, cooling, etc.) ➔ 2. cost and performance attributes of a data center – more exact figures will be power capacity. The data center owner’s role determined later by rigorous engineering The data center owner (or recipient of calculations and evaluations. the optimized architecture solution) plays a pivotal role in the optimization process The engineer’s role as he is well acquainted with many as- Engineering analysis takes center stage pects relevant to the data center design. after the owner’s requirements and the These include but are not limited to: architecture have been clarified. Provi- − The geographical location, with the sion of the power supply alone involves associated information mentioned numerous analyses ➔ 2: above. − Power distribution unit (PDU), static − Planned load capacity (in MW) in the transfer switch (STS) and power short-term and in the future. This supply unit (PSU) analysis. Depending impacts the oversizing and reliability on the architecture and total IT load, considerations of the power equipment. the type, rating, footprint, power − Average server rack power density density, efficiency, reliability, cost and (kW/rack). This will influence the number required must be determined. cooling system design and the − Server room power distribution dimensioning of the power equip- analysis. Depending on the server ment. rack power density and the selected − Preferred cooling technologies. cooling technology, this analysis

In the crystal ball 61 3 A ZNE data center must consume no net energy from the utility grid over a specified time Cost minimization period. and energy effi- Utility power is bought Fuel cell units are on (generation Maximum PV output ciency maximiza- at lower tariff cost is lower than utility price) tion can be treated 140 as the two most 120 100 important data 80 center design ob- 60 40 Net zero energy jectives – reliability exchange with 20 a utility is a given require- 0 2:00 4:00 6:00 8:00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 -20 ment and cannot Power unit (100 is maximum load) -40 be compromised. -60

-80

-100

PV ramp control is done Battery is absorbing Power fed back at by energy storage excessive solar power higher tariff

IT load HVAC load Total load PV

Fuel cell Energy storage Utility power

determines the size, length and safety − Safety analysis determines appropri- grounding for the power distribution ate protection devices and grounding bus and feeder. practices – including the type, rating − Emergency/backup power supply and number of the protection devices, analysis. Emergency power supply and the size/length of the grounding refers to uninterruptable power supply conductors. The fault-current limiting (UPS) systems, which can be based function of converters is considered in on batteries, ultracapacitors or the protection device dimensioning. flywheels; backup power supply refers − System efficiency analysis will be to diesel generator sets or other types done for at least three loading levels: of generation devices that can provide 20, 50 and 100 percent. Efficiency curves of PDU/ STS/PSU and Multiple data center architec- UPS converters are the main ture designs can be devel- inputs to this oped and analyzed to deter- analysis. Server room distribution mine the best candidate. feeders are quite short and their efficiency can be power for hours to days. The architect assumed to be 100 percent, when may have considered the type and they are considered. redundancy, but this analysis details − The system efficiency analysis result the ratings, auxiliaries (protection and is the major input for the opex control), footprint, efficiency, reliability, estimation, as are data center cost and number of these power emergency/backup operation cost supplies. Construction cost differ- and outage revenue loss or penalty. ences between alternative technolo- Capex is estimated based on data gies are considered in the layout of center IT power supply/distribution the emergency/backup power supply equipment and protection equipment rooms (eg, converter/battery rooms). costs. Other types of opex and capex

62 ABB review 4|13 are considered to be the same for all − Progress in energy-efficient building architecture designs and are not construction technologies (in the case The owner gives considered in the optimization of data centers, more efficient process. architectures, too) major input to the − Reliability/availability analysis is important to ensure the architecture By definition, a ZNE data center must process due to the meets certain requirements [2]. A DC consume no net energy from the utility fact that it is he data center design may be on a par grid over a specified time period ➔ 3. with a higher-tier AC design in this Since data centers are characterized by who will weight the respect due to savings in power a very high consumption density (100 conversion stages. times that of an average office building) different attributes − Scalability analysis looks at potential with relatively low daily/seasonal varia- in the overall power equipment modularity and tion, several key factors must be consid- hot-swapping benefits, or the inte- ered in ZNE data center architecture assessment. grated data center. designs: − Availability of energy supply for local Overall architecture design assessment generation The overall assessment is usually straight- − Type, operation mode and size of forward – it can be a simple weighted local generation sum calculation. However, as explained − IT load balancing earlier, the data center owner bears the − Near-term IT load and local genera- ultimate responsibility in assigning ranking tion forecasting weights to different design attributes. The design can be AC or DC. However, a Trends DC design will be more efficient, making In general, cost minimization (both capex it easier to achieve ZNE operation. and opex) and energy efficiency maximi- zation can be treated as the two most For fuel-cell and PV-powered ZNE data important data center design objectives centers, microgrid operation, ie, self- (reliability is a given requirement and powered and isolated from the grid, is a cannot be compromised). In addition, real possibility. However, design optimi- data center design optimization has to zation to accommodate microgrid opera- consider the major industry trends: tion as well as all the other requirements − Greener: Designs using renewable or mentioned above is a whole other story. reduced carbon energy resources are Zhenyuan Wang of growing interest. A zero net energy Ernst Scholtz (ZNE) data center is a goal. ABB Corporate Research − Modular: Data centers can be quickly Raleigh-Durham, NC, United States constructed and maintained by using [email protected] standardized and plug-and-play- [email protected] capable server racks, power modules, battery packs, cooling equipment and Alexandre Oudalov generator modules. Francisco Canales − Cloudier: Economies of scale can be ABB Corporate Research exploited by colocating the IT services Baden-Dättwil, Switzerland of several organizations, especially [email protected] cloud service providers, in one data [email protected] center. − Hotter: With the advent of blade servers, the power density of server References [1] B. T. Patterson, “DC, Come Home,” racks has increased significantly, IEEE Power & Energy Magazine, November/ posing cooling challenges. December, 2012. Further, the ZNE building concept is at- [2] F. Bodi, “DC-grade” reliability for UPS in tracting interest due to several drivers [3] telecommunications data centers, in 29th International Telecommunications Energy that are also relevant to data centers: Conference, Rome, Italy, 2007, pp. 595–602. − Rapid price drop of local generation [3] S. Pless, P. Torcellini, “Net-Zero Energy technologies (mainly PV panels) Buildings: A Classification System Based on − Controllable loads – heating, ventila- Renewable Energy Supply Options,” National Renewable Energy Laboratory, Golden, tion, air conditioning and lighting; the CO, Technical Report TP-550-44586, 2010, IT load can be shifted, especially in Available: http://www.nrel.gov/sustainable_nrel/ cloud computing data centers pdfs/44586.pdf

In the crystal ball 63 Taking charge

Flash charging BRUCE WARNER, OLIVIER AUGÉ, ANDREAS MOGLESTUE – If you thought that charging electric vehicles was all about fiddling with charger cables followed by long and unpro- is just the ductive waits, then think again. ABB has (together with partners) developed an electric bus that not only automatically charges in 15 s, but also provides high transportation ticket for clean capacity and energy efficiency. The bus connects to an overhead high-power charging transportation contact when it pulls into a stop and tops up its batteries during the time its passengers are embarking and disembarking. Besides being an attractive means of transportation, the TOSA bus that is presently running in the Swiss city of Geneva also has numerous environmental bonuses. It is silent, entirely emissions free, uses long-life and small batteries while the visual clutter of overhead lines and pylons that is often a barrier to trolleybus acceptance is made a thing of the past. The system is inherently safe because the overhead connectors are only energized when they are engaged, and the electromagnetic fields associated with inductive charging concepts are avoided. Such has been the success of the demonstrator that the concept is now being developed for series production. Let the future begin.

64 ABB review 4|13 he world is becoming increas- tions aside, this transmission takes one reduced acceptability of noise and pollu- ingly urban. In 2008, for the first of two forms: Power is either stored tion have led manufacturers and opera- time in the history of humanity, on the vehicle (usually in the form of tors to think about alternatives to diesel T more than half the planet’s diesel fuel, as on a population lived in cities. Cities bring bus) or transmitted with them many challenges, not least of electrically (requir- Recovering energy when which is the efficient organization of ing a continuous transportation. To avert gridlock and contact system as the bus brakes helps reduce reduce pollution, planners across the on metros, trams globe are encouraging the use of public and trolleybuses). wastage. transportation. The latter forms of transportation are typically seen on heav- for powering buses ➔ 1. Solutions imple- Public transportation in cities can take ily used corridors where the significant mented to varying degrees include less numerous forms, but what they all have infrastructural investment is easier to conventional fuels (such as natural gas) in common is that they require energy to justify. The former solution is typical for and adopting alternative propulsion con- be transmitted from a fixed supply to a more lightly patronized corridors where cepts, for example hybrid buses, battery moving vehicle. Some particular solu- lower startup costs permit routes to be buses and trolleybuses. A feature shared created or modified more flexibly. by the latter three is that they use electric motors, permitting energy to be recov- This status quo has held its own for ered when the bus brakes, creating an Title picture The TOSA demonstration bus is presently running in many decades, but how much longer opportunity to reduce energy wastage. public service in Geneva. can it do so? Rising fuel prices and the Recovering energy is not, however,

Taking charge 65 1 Signs of change 2 Comparison of modes of operation

Before service At stop Braking At stop – The International Energy Agency Accelerating predicts that oil prices will remain above $100 / barrel in the foreseeable future. Diesel bus – A McKinsey study predicts the price of Li-Ion batteries for cars will fall by almost Braking energy dissipated 75 percent (from present level) by 2025. – Carbon duties are being introduced across the world and will rise further. Trolleybus – There is global pressure to reduce emissions from road transportation (eg, EURO 6 emissions standards) – Progress in power electronics (higher Battery switching frequencies, lower losses, more bus compact converters) are increasing the viability of all-electric solutions.

TOSA

Diesel engine Electric motor

Diesel fuel Battery Energy flow

3 Alternatives to overhead lines

The idea of seeking to transmit power to be used to recharge other road vehicles, vehicles by means other than overhead lines including buses. However, the system retains is far from new. In the early part of the 20th several disadvantages, including energy losses century, some tram systems used a so-called during charging and the high cost of burying “conduit”, in which a conductor was embed- the charging infrastructure. ded in a narrow groove in the road. However, the groove was vulnerable to blockage by ABB’s flash charging system is inherently debris, while the risk of electric shock to other safe because the charging points are only road users could not be excluded. Several energized when the bus is actually connect- manufacturers have revisited the idea in recent ed ➔ 5. Because it uses a direct electrical years, with the conduit being replaced by a connection, concerns over electromagnetic safer and more sophisticated contact – or fields can be mitigated. Furthermore, not induction-based transmission. These can be requiring the installation of heavy equipment combined with batteries avoiding the need under the roadway simplifies the installation to embed the costly equipment along the full process and reduces the associated route. The induction-based version can also disruption.

strictly the same as re-using it. Hybrid teries (and recharging them more often) and battery vehicles use batteries to but such additional visits to the charging bridge the mismatch between supply station have a time and productivity and demand, whereas in the case of trol- penalty. leybuses this can be handled by the substations and grid ➔ 2. The trolleybus trumps these disadvantages. The absence of a larger on-board Battery buses have limitations. Despite energy storage system reduces vehicle considerable progress in battery technol- weight and permits better acceleration ogy, their energy density is orders of using less energy. The disadvantage, magnitude lower than that of diesel fuel 1. however, lies (or rather hangs) in the The extra weight that batteries add to overhead lines. These are costly to install the bus reflects negatively on its energy and maintain and are not always wel- footprint, and the space they require can come due to their visual impact ➔ 3. reduce passenger-carrying capacity. This can be countered by using fewer bat Is there a way to keep a battery bus on the road without resorting to either large, heavy and space-consuming energy Footnote storage or to frequently having to take 1 Diesel fuel has an energy density of about 46 MJ/kg, whereas rechargeable batteries still the bus out of service for a deep and full have less than 1 MJ/kg. recharge?

66 ABB review 4|13 4 Partners in the TOSA project Overhead lines for (hence the name TOSA, which also stands for “trolleybus optimisation système alimentation” trolleybuses are or optimized charging system for trolleybus) costly to install Further partners of the project include: – Palexpo (trade fair center) and Geneva airport and maintain and – Hepia (University of applied science), architecture design of the bus stops are not always – HESS, manufacturer of the bus The following four companies initiated the – Canton of Geneva, Federal Office for Energy, welcome due to TOSA project: Federal Office for Highways – TPG, Geneva’s public transportation operator – EPFL and HeArc Universities their visual impact – OPI, Lake Geneva area office for the promotion of industry – SIG, Geneva’s utility – ABB (ABB Sécheron Ltd.)

5 The bus recharges at stops using its roof-mounted contacts that engage using laser guidance.

Transport passengers not batteries Flash charging and smart grid One fundamental difference between The high-power (400 kW, 15 s) charging buses and automobiles is that buses fol- of the high-power density batteries on low fixed routes. The question of “range the bus can result in load peaks affecting of operation” which is of significance the local grid. The flash charger station, to electric cars is reduced to the more however, flattens out the demand by manageable “distance to next recharging charging super capacitors over a period opportunity” for a bus. With buses pre- of a few minutes while drawing a lower dictably stopping at regular intervals, current from the grid. As this current is charging points can be located at the up to 10 times less than would be the stops. With the bus being able to top case without storage, the connection up its charge at these points, the need can be made with a cheaper and more for large and heavy batteries is avoided readily available low-power supply. Addi- and the vehicle becomes lighter, more tionally, recharging of the supercapacitors agile, more energy efficient as well as is timed so that they are left discharged providing more space for passengers for longer periods when the bus service inside. Furthermore, if charging time can is running at lower frequency. As super- be limited to the time that the bus needs capacitors are aged by higher voltages to stop anyway, negative effects on the this “smart” functionality allows the life of schedule can be avoided. Together with the supercaptitors to be doubled. partners ➔ 4, ABB has created the TOSA bus to present a solution based on this approach.

Taking charge 67 The TOSA demon- 6 Short top-up charges help maintain the battery level. strator bus will be 510 34 running in public 490 32 service on a short 470 30 line in Geneva, 450 28 Switzerland until 430 Altitude (m) 26 April 2014. 410 Battery energy level (kWh) 390 Altitude 24

Battery energy level

370 22 Bus stop

350 20 0 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 9,000

Distance (m)

With limited time being available at stops It was this requirement that led to the (passengers typically embark and disem- creation of two types of feeding stations bark in 15 to 25 s), as little time as possi- along the route: The flash station and the ble should be lost in establishing the elec- terminal station. As described, the flash trical connection. The electrical connec- stations provide a short high power tion is established in under a second. As boost of energy. However, drawing the bus approaches a stop it is the driv- 400 kW for 15 s is not sufficient to fully er’s responsibility to oversee the safety of recharge the batteries ➔ 6. More pro- the passengers and pedestrians and keep longed charges of three to five minutes at 200 kW are thus delivered at the ter- The bus draws a 400 kW minus where buses are scheduled to charge for 15 seconds while stop for longer periods (in order to at a stop. permit the driver to take a break and an eye on surrounding traffic. To avoid to provide some recovery buffer in case placing additional demands on the driver, the bus is running late). The time required the connection system is automatic. A for recharging at the terminus should laser aligns the moving equipment on the thus not risk causing the bus to fall bus roof with the static overhead recep- behind its schedule or to be unable to tacle ➔ 5. The connection is made as catch up when it running late. soon as the brakes are applied. The terminal charger consists of a simple By virtue of the receptacle’s height above 12 pole diode rectifier. This converts the the road surface, and furthermore by incoming AC supply to DC in a similar being energized only when a bus is pres- way as is done for DC railways, trams or ent, this is an inherently safe solution. trolleybuses. The voltage used in this case is 500 V. This solution is simple and The timetable defines the service and cost effective as well as being extremely the economics reliable. Operating costs for a bus service are highly dependent on driver wages, The flash charger has a more complicat- schedule frequency and fleet size. ed but more flexible power electronic Therefore, the change from diesel to configuration. It uses a controlled rectifi- electric supply should not reduce the er to charge the supercapacitors. This is commercial average speed nor require able to regulate the amount of charging an increase of the fleet size to provide current. When the bus connects the con- the same service. troller closes a contact on the output

68 ABB review 4|13 7 The technology will be introduced on a 8 Flash charging is already a competitive solution. full-length bus route in Geneva. Its competitiveness will increase further in the future.

– 11 articulated buses (18 m) 140 – Two powered axles (out of three) 120 – Batteries per bus equivalent to circa two 100 electric cars (38 kWh) 80 – Every bus has capacity for 134 passengers 60 – Charging draws energy from 400 VAC 40 low-voltage network 20

Comparative costs (percent) 0 Diesel Hybrid Trolleybus Battery TOSA TOSA TOSA, (Diesel) swapping terminal in-route in-route only charge charge Current annualized TCO Future annualized TCO w/o storage with storage

Costs shown exclude costs common to all modes (such as the driver).

side of the supercapacitors to discharge Renewable energy them into the bus. The TOSA bus is inherently suitable for using renewable energy. In contrast to During operation, the batteries receive classical electric vehicles, which typically further top-up charges as the bus recharge when they arrive home in the brakes. Rather than using a friction- evening, the bus recharges during the based system that converts all the kinet- day and can thus make direct use of ic energy to heat, the bus’ motors can solar energy as it is produced. The ability switch to generator mode and return of flash charging stations to store energy much of this energy to the batteries ➔ 2. for short periods and flatten out charging peaks can also protect the system The battery charge for a typical trip is against short-term fluctuations in solar mapped in ➔ 6. The graph shows how generation. the batteries are topped up at stops but a far larger charge is received at the ter- Demonstration in Geneva minus stop. The TOSA demonstrator bus is currently running in public service on a short line in the Geneva, Switzerland (between The TOSA bus is the airport and the convention center PALEXPO). The demonstration will con- inherently suitable tinue until April 2014. The bus has so far performed flawlessly. As a next step, the for using renew- technology will be introduced on a full- able energy. length bus line in Geneva ➔ 7.

A competitive solution There is a third type of charger, for the ABB’s flash charging system for buses depot, where a longer charge is applied is already competitive today, and will to compensate the energy required be- become even more competitive in the tween the operating line and depot loca- future. An economic comparison of tion. As there is more time for charging flash charging to other modes is shown at the depot a flash charging station is in ➔ 8. The future scenario predicted is not required. The bus is plugged into a based on assumptions of rising fuel dedicated supply using a cable. A total costs and CO2 duties and the diminish- Bruce Warner of four buses can be connected to a ing costs of batteries. Olivier Augé depot charger which charges them ABB Sécheron S.A. sequentially. The electrical configuration With diesel buses becoming increasingly Geneva, Switzerland is the same as that of the terminal, a less attractive, both financially and from [email protected] 12-pulse diode rectifier, however the and emissions point of view, and opera- [email protected] power rating of the depot is 50 kW as tors seeking an attractive modern form opposed to the 200 kW of the terminal. of transportation without having to hang Andreas Moglestue wires in the street, flash charging is well ABB Review situated to replace both existing trolley- Zurich, Switzerland bus routes and urban diesel routes. [email protected]

Taking charge 69 In control

DAVID-BINGHUI LI, EVAN-FEI E, VISTA-HAO FENG, WEIWEI LONG – Program- ABB’s dredger drives mable logic controllers (PLCs) are the backbone of automating control unit provides a electromechanical processes. Designed for multiple I/O arrangements, extended temperature ranges, immunity to electrical noise, and more reliable and inte- resistance to vibration and impact, they are well adapted to a wide range of automation tasks. Almost any production line, process or grated control platform machine function can be greatly enhanced by using this type of control. for dredging motor ABB has taken its own PLC system to a new level, having developed an advanced PLC to control, protect and supervise all dredging consumers systems and systems working within trailing suction hopper dredger vessels. Its uniqueness lies in the fact the new dredger drive control unit (DreDCU) simultaneously controls multiple drive chains. ABB has successfully installed the unit in three vessels.

70 ABB review 4|13 1 Typical configuration of a hopper dredger

ACS800 Seal water pump motor 400 V Seal water pump motor

6.6 kV ACS800LC Shaft diesel generator (SDG) Jet pump motor Dredger pump Drag head

Underwater ACS6000LC pump

trailing suction hopper dredg- These consumers include motors for the complicated. For instance, an additional er (TSHD) has large, powerful dredger mud, jet water, underwater and changeover function between a mud pumps and engines that en- seal water pumps. Each drive chain in- pump and an underwater pump must be A able it to suck up sediments cludes a drive transformer, drive and controlled, or a master/follower function from the ocean or riverbeds. One or two motor. between two dredger pumps must be suction pipes run from the vessel to the overseen. However this need for in- sediment floor. A drag head is attached to Adjusting needs creased cooperation between different the end of the pipe and lowered to just Even as recently as five years ago only a drive chains and protection for each above the sediment floor, making it pos- few of the dredging consumers, such as chain from the system level does not sible to regulate the mixture of sand and the jet water pump, were controlled by a become a problem due to the sophisti- water that it takes in. A TSHD generally frequency convertor, with simple control cation of the DreDCU. stores the dredged material in its own and protection based on the product hopper and discharges the leftover water level. The other large dredging consum- Development process overboard. The material can be dis- ers were still driven by diesel engines ABB already offered a sophisticated PLC charged through hatches in the bottom of with separate control systems. Therefore unit for single-drive systems used for pro- the ship or by pumping for land reclama- pulsion and thrust- tion or beach nourishment. ers. Yet in order to The DreDCU easily handles accommodate the Because a TSHD is used in a wide range complicated and of applications, and can dredge and the need for increased multiple drives spe- transport material over long distances, it cific to dredger ap- is often referred to as the workhorse of cooperation between different plications, a new the dredging industry. drive chains. control unit needed to be developed. A typical TSHD electric drive system has The company uti- a number of diesel engines running gen- the drive control was simple and easily lized one of its existing controllers as a erators to supply electrical power to the handled by the drives firmware. base for the DreDCU. main frequency convertors that drive all the relevant dredging consumers ➔ 1. Yet it became clear that significant fuel ABB’s Extended Automation System efficiency could be gained by having fur- 800xA family of controllers, communi- ther dredging consumers controlled by cation interfaces and I/O modules have Title picture frequency convertors. Adding additional been meeting the needs of today’s most ABB’s highly sophisticated dredger drives control unit brings significant fuel efficiency to trailing consumers to the drive control clearly sophisticated plant automation sys- suction hopper dredgers. makes the control process much more tems. The flagship controller of System

In control 71 2 ABB’s new dredger drive control unit 3 Main features of DreDCU

– One unit controls up to 11 drives Dredger control room Process and power workplaces – Interface to remote control and integrated automation system (supports PROFIBUS and Modbus) Dredger control system – Optional local panel – Optional interface to remote diagnostics support Control network Process control server (Non-ABB scope) – Optional interface to advisory system – Meets main class society requirements Dredger drives control unit 24 VDC Controller

I/O cards

The hardware is located in the DreDCU Drive-bus/profibus cabinet (alternatively a part of the converter cabinets), which is placed in the MV MV LV LV MV LV/MV frequency convertor room on the vessel. drive drive drive drive multidrive drive

As the DreDCU is designed for a specific vessel type with specific functions and

Dredger Underwater Jet Seal water Cutter Mooring Traverse Propulsion relationship with ABB drives, it is avail- pump pump pump pump motor winch able as part of an entire electric system motor motor motor motor Hopper dredger drives control Cutter dredger drives control package ➔ 3. The multidrive control aspects have been tweaked to handle dredger applications. The current DreDCU application software is a standardized and scalable software package that is 800xA, the AC 800M, is a modular pro- based partly on the existing software The current cess PLC with communication func- library of the AC 800M. The existing tions as well as full redundancy and library was adjusted to shift from a pro- DreDCU applica- support for a large range of I/O sys- pulsion application to a dredging appli- tems. It can integrate various networks, cation. A changeover function between a tion software is a fieldbuses, serial protocols and I/Os, mud pump and an underwater pump had standardized and providing seamless execution of ad- to be developed, special mud pump in- vanced and unhindered process control terfaces added and the start/stop proce- scalable software strategies as well as functional safety, dure changed according to the dredger electrical, quality control, and power application. A process panel software package based management applications. It can deliver was also developed for the mud pumps partly on the exist- automation solutions for small and large on which information for all drive chains ing software library applications. can be checked. The DreDCU is made up of an AC 800M The new control software is adapted of the AC 800M. hardware platform with embedded appli- to the project-specific configuration by cation software, communication mod- means of parameterization due to differ- ules and modems, I/O modules and ent dredging applications. The DreDCU power supplies ➔ 2. An optional local software offers standard control for dredger consumers such as se- Adding additional consumers quence start/stop control, emergen- to the drive control makes the cy stop and ramp accelerate. Option- control process much more al control types complicated. include master/follower, duty overload running and panel provides an alarm list and displays changeover. The software monitors and detailed information for each activated protects all relevant dredger drive chains, alarm. sends alarms to an integrated automation system and implements auxiliary control for main dredger consumers. In

72 ABB review 4|13 4 Trailing suction hopper dredgers equipped with the DreDCU

TongTu No9 XinHaihu No8 XinHaihu Builder: Guangzhou WenChong shipyard Builder: Guangzhou WenChong shipyard Builder: Zhenhua Changxin shipyard Owner: CCCC Tianjin Dredging Co. Ltd. Owner: CCCC SDC Waterway Construction Co., Ltd. Owner: CCCC Shanghai Dredging Co., Ltd. Designed by 708 Designed Institute, Designed by 708 Designed Institute, Designed by 708 Designed Institute, classified by CCS, delivered in 2011 classified by CCS, delivered in 2012 classified by CCS, delivered in 2012

ABB scope of supply: ABB scope of supply: ABB scope of supply: – 2x 10,000 kVA shaft generator – 2x 7,200 kVA / 1,500 rpm / 6.6 kV generators – 2x 7,200 kVA / 1,500 rpm / 6.6 kV generators – 1x MSB (Main switchboard) – 2x earthing resistors – 2x earthing resistors – 4x 1,150 kVA jet pump transformer – 1x 6.6 kV / 50 HZ MV switchboard – 1x 6.6 kV / 50 HZ MV switchboard – 4x 3,450 kVA mud pump transformer – 2x 1,725 V / 5,200 kVA, mud pump transformers – 2x 1725 V / 5,200 kVA mud pump transformers – 2x 2,000 kVA distribution transformer – 2x 710 V / 1,650 kVA, jet pump transformers – 2x 710 V / 1,650 kVA jet pump transformers – 2x 110 V / 30 A DC-UPS – 2x 400 V / 1,600 kVA, distribution transformers – 2x 400 V / 1,600 kVA distribution transformers – 2x ACS 6000 mud pump converter – 2x 3.3 kV ACS 1000 drive – 2x 3.3 kV ACS 1000 drive – 2x ACS 800LC jet pump converter – 2x 690 V ACS 800 drive – 2x 690 V ACS 800 drive – 2x ACS 800 sealed pump converter – 2x 4,500 kW / 1,500 rpm mud pump motors – 2x 4,500 kW / 1,500 rpm mud pump motors – 2x 6,000 kW mud pump – 2x 1,000 KW / 1,500 rpm jet pump motors – 2x 1,000 kW / 1,500 rpm jet pump motors – 2x 2,000 kW jet pump – 2x softstarters – 2x softstarters – 1x DCU cabinet – 2x DC UPS – 2x DC UPS

the dredger drive control unit on a sec- Simultaneous ond type of dredging vessel, the cutter suction dredger. The end goal is to install monitoring of the the drive control unit on a wide range of special vessels including supply ves- status of all dredg- sels, heavy lift vessels, crane vessels and ing operation installation support vessels. equipment allows for more efficient operations.

2012 ABB installed the unit in three vessels ➔ 4.

Making headway The benefits of the DreDCU solution are multifold. It increases the reliability of dredging operations by monitoring dredger consumers’ conditions and harsh working environments, thus reducing the David-Binghui Li risk of downtime due to power loss. Evan-Fei E Simultaneous monitoring of the status of Vista-Hao Feng all dredging operation equipment allows Weiwei Long for more efficient operations. Implement- ABB Marine and Crane ing a standard platform enables easy Shanghai, China interfacing with other ABB products. [email protected] Smaller cabinets provide flexibility for [email protected] equipment location. Development con- [email protected] tinues with the next goal being to install [email protected]

In control 73 Robust radio

Meshed Wi-Fi wireless communication for industry

PETER BILL, MATHIAS KRANICH, NARASIMHA CHARI – Communi- technology with the acquisition of the Silicon-Valley-based cation is an enabler of key applications in many sectors of company, Tropos. The Tropos mesh technology has a very industry – and wireless is often the most cost-effective and robust technical foundation and is already being applied in practical means of providing it. Recognizing this, ABB has major implementations in different industrial fields. extended its portfolio to include mesh 802.11 wireless

74 ABB review 4|13 1 Tropos allows logical separation of the multiple applications running over the common infrastructure.

AMI & demand management Billing/DSM

Distribution Distribution automation management system

Mobile GIS/ Mobile workforce workforce applications

Substation automation Security

Substation security Separate VLANs

Mesh routing intelligence The foundation of the Tropos mesh By combining patented RF resource architecture is the Predictive Wireless management algorithms with standards- Routing ProtocolTM (PWRP), which is based radio technologies operating in based on patented routing algorithms unlicensed frequency bands, the Tropos that maximize the performance and resil- architecture provides a highly reliable, ience of wireless mesh networks. PWRP BB’s communication net- scalable, fault-tolerant network infra- is a dynamic, wireless-aware routing pro- works business now offers structure that is capable of quickly and tocol that allows mesh routers to perform a market-leading, IP-based seamlessly routing around interference end-to-end measurements of path qual- A outdoor wireless broadband and congestion bottlenecks. ity and use these measurements to make infrastructure that can be cost-effectively routing decisions that result in the high- deployed for one or multiple applica- Unlike network architectures that are est end-to-end throughput. tions. ABB’s Tropos solution has many dependent on a central controller, the advantages over competing technolo- Tropos mesh architecture, because of its Flexible dual-radio routers gies [1] and is The IEEE 802.11 standard set provides designed to de- support for two frequency bands of op- liver high broad- Tropos offers a market-leading, eration and Tropos is unique in enabling band speed, re- both radios of a dual-radio router to be siliency, security IP-based outdoor wireless used for either mesh connections or client and scalability. access, thereby significantly increasing The mesh archi- broadband infrastructure the reliability and the capacity of multi- tecture is de- that can be cost-effectively band networks. Dual-mode routers in- centralized and crease mesh capacity by opportunistically highly flexible. deployed for one or multiple exploiting less congested 4.9/5.8 GHz links whenever possible. In areas where The strength of applications. 4.9/5.8 GHz use is restricted due to ABB’s Tropos lack of line-of-sight, the routers auto solution is founded on six cornerstones: distributed networking capabilities, can matically fall back to using 2.4 GHz radios, mesh routing intelligence, radio frequen- easily recover from the loss of any net- which provide a reliable long-range con- cy (RF) resource management, multilayer work component. Each router continually nection. security, outdoor optimized router hard- monitors its environment for potential ware, open standards and advanced ways to optimize the network, so if a Seamless mobility control and analysis software. problem occurs with either a gateway or The fixed infrastructure Tropos mesh node router, the mesh automatically networks can be quickly extended with adapts its topology to keep the network mobile routers from the same product up and running. When the router is line, for use by emergency services, for Title picture brought back online, the network quickly example. Each mobile node extends Many industrial applications depend on resilient, re-establishes an optimal configuration. connectivity to client devices in the vehi- secure and scalable wireless broadband communications. How does ABB’s Tropos mesh 802.11 cle vicinity, creating a tactical response wireless technology provide this? zone in almost any location.

Robust radio 75 2 Tropos configuration

Voltage controller

Utility data center Sensors

Relay Substation Substation

Video camera Tropos 1410 Tropos 7320 Capacitor bank Substation premises

Recloser Feeder Mobile data AMI collector

Energy Feeder Tropos 1410 storage device Voltage regulator

Fiber/Licensed PTP Tropos 1410

2.4/5.8 GHz Advanced 900 MHz Home metering area network infrastructure Tropos 1410 HAN (ZigBee) (HAN) (AMI)

RF resource management scrutiny by the security community, The distributed PWRP uses patented algorithms to con- such as IPSec, IEEE 802.1x, IEEE tinuously and dynamically optimize the 802.11i, AES encryption, SSL/TLS, Tropos mesh use of the available spectrum: FIPS 140-2, etc. − PowerCurveTM: This distributed − Robust security at every layer, from architecture is algorithm dynamically increases or the physical hardware (eg, tamper- capable of quick- decreases transmission power levels resistant, ruggedized hardware) right and continuously adapts the link data up to application-level traffic protocols ly and seamlessly rates to maintain the reliability of each (eg, HTTPS-based security). wireless link and also maximize the − A security approach that allows routing around number of concurrent links. This granular, operator-specified policies to interference and stops, for example, “loud” routers ensure the logical separation of the from drowning out nearby “conversa- multiple applications running on the congestion tions.” common infrastructure ➔ 1. − Airtime Congestion ControlTM (ACC): − Software that adapts to the evolving bottlenecks. ACC is designed to provide consis- threat landscape and that encom- tent performance for a large number passes the latest security standards of users, especially under heavily and requirements. loaded network conditions, thus overcoming a well-known shortcom- Outdoor optimized router ing of 802.11 MAC. The Tropos router hardware has a bat- − Adaptive noise immunity (ANI): ANI tery backup and is ruggedized for opera- adjusts chip-level packet detection tion in the most challenging operating parameters in real time to minimize environments. The radios are designed false detection events and maximize for optimal outdoor performance: They receiver sensitivity. can transmit up to the maximum allowed transmission power level and they offer Multilayer security the industry’s best receiver sensitivity. Security-wise, wireless networks are more vulnerable than traditional wired infrastruc- Open standards tures, so Tropos’ comprehensive security The Tropos solution set/technology aims approach is based on: to provide maximum interoperability and − Open-standard security mechanisms investment protection through support that have undergone extensive of all relevant open standards at every

76 ABB review 4|13 3 Tropos router at EOG Resources exploration site PWRP is a dynamic, wireless-aware routing protocol that allows mesh routers to perform end-to-end measurements of path quality and use these measurements to make routing decisions that result in the highest throughput.

layer of the protocol stack including In the coming years, additional applica- IEEE 802.3 Ethernet, IEEE 802.11 Wi-Fi, tions for smart grids related to distribu- IEEE 802.1x access control, TCP/IP, etc. tion automation, distributed generation, electric vehicles and video security will Advanced control and analysis create a new appetite for high-bandwidth Tropos Control is a software application and low-latency communications that that provides comprehensive network only a scalable broadband network like management to streamline the deploy- Tropos can provide. ment, optimization, maintenance and control of large-scale networks. Burbank Water and Power (BWP) in the United States is using Tropos for AMI, Mesh applications demand response and distribution auto- There is a huge number of possible mation. With a smart grid, BWP seeks to applications for ABB’s mesh 802.11 flatten demand peaks (to avoid having to solutions. build new generating plants) and accommodate the growth in electric vehicle Smart grid applications numbers. The utility also plans to seg- An advanced metering infrastructure ment data traffic across different user (AMI) is just one of many applications groups and applications and share the that are required to fulfill the vision of network with other city departments. the smart grid. However, demand management and response, distribution auto Open-pit mining applications mation and control, outage manage- Safe and efficient operation of open-pit ment, and mobile workforce applications mines requires precise coordination of are also needed to make the vision a some of the world’s largest and most reality ➔ 1. Deploying and managing expensive machines in settings charac- separate networks for each application terized by punishing heat or cold as well is not cost-effective. A single, stan- as extreme shock and vibration. Maxi- dards-based, high-performance network, mizing productivity in operations and such as Tropos, that aggregates com- maintenance can yield substantial im- munications for multiple applications is provements in profitability and safety. not only simpler to manage, but also yields an attractive return on investment ➔ 2.

Robust radio 77 Container port applications The Tropos solution Busy container ports, with large, con- stantly moving metal objects, present a includes a suite of particularly challenging wireless network algorithms for effi- environment. In one of the largest Mexi- can ports, for example, Tropos is suc- cient RF spectrum cessfully being used for tracking and real-time location of shipping containers management and both outdoors and in warehouses.

optimal spatial Smart city applications frequency reuse. In smart cities, multiple city agencies can benefit from a Tropos wireless communications network. For example, Oklahoma Wireless communications can signifi- City’s network of Tropos fixed and mobile cantly enhance the efficiency, productiv- wireless routers, covering 1,600 km2, is ity, safety and security of open-pit mines. used by more than 180 city applications, A wireless network enables truck and including: heavy equipment telemetry data, opera- − Mobile broadband in police vehicles, tional and surveillance video feeds, safe- allowing 1,500 officers to spend ty and security system information, high- 100,000 more hours per year in the wall scans and field data that drive mine field. management software all to be transmit- − Several hundred IP video cameras for ted to a central location where the data monitoring and surveillance. is monitored, analyzed and acted on in − The building inspection agency, real time. allowing inspectors to be more productive in the field and reduce The PotashCorp-Aurora phosphate mine application turnaround time. in Aurora, North Carolina, has deployed − Traffic signal controllers in the a Tropos network with fixed and mobile downtown area. nodes that provides equipment telemetry, real-time vehicle monitoring (speed, Simple and safe temperature, tire pressure, etc.), manu- ABB’s patented algorithms and software facturing process data and voice over IP in the Tropos product line, along with its (VoIP) communications. industrial-grade hardware, put the company ahead of competitive alternatives in Oil and gas applications terms of reliability, performance and ease Measurement, logging and adjustment of maintenance, while providing easy duties at remote rigs and wellheads are access for thousands of different Wi-Fi often performed by well tenders who standards-compatible endpoint devices. travel long distances to site. However, wireless communication enables remote Many applications require a wireless monitoring, in real time. This makes broadband solution with high resilience, better use of skilled resources, speeds security and scalability. ABB provides problem resolution and reduces travel these features, allowing customers to Peter Bill time. In addition, a wireless network build and operate high-performance com- Mathias Kranich can provide cost-effective voice and munications networks enabling their appli- ABB Power Systems high-speed data services to field facili- cations to operate in multiple industries. Baden, Switzerland ties even in areas that lack cellular [email protected] coverage. [email protected]

EOG Resources, an oil and gas company Narasimha Chari operating in North America, owns very ABB Communication Networks remote sites where cellular coverage is Sunnyvale, CA, United States absent. The Tropos networks they have [email protected] implemented provide their workforce with connectivity between these sites and the operational control center. This Reference leads to improved operational perfor- [1] P. Bill, M. Kranich, N. Chari, “Fine mesh: mance as well as increased workforce Mesh 802.11 wireless network connectivity,” security ➔ 3. ABB Review 1/2013, pp. 42–44.

78 ABB review 4|13 The right fit

ABB partners with OSCAR AVELLA – Partnership can be defined as a collaborative agreement between two or more parties in which all participants agree to a family-owned work together to achieve a common purpose or undertake a specific task and to share risks, resources and competencies. ABB has a strong company to power history of successfully forming partnerships with companies, big and floating flow pumps small. A recent example is the story of how a small, family-owned Colombian engineering company is transforming traditional irrigation systems in the Middle East, and the key is a floating flow pump powered by ABB process performance motors.

The right fit 79 1 Floating pump powered by ABB motors

tors. Last year, ETEC placed an order 2 Breakdown of ordered ABB process with ABB for 38 process performance performance squirrel cage induction motors squirrel cage induction motors ➔ 2 to run Quantity Output Description the floating pumps destined for irrigation 8 371 kW – M3BP projects in the Middle East. “We decided – Frame size 355 to partner with ABB Motors and Genera- 15 336 kW – 380 V, 50 Hz tors, because they have a worldwide – 1,500 rpm technical reputation that would guaran- – IP55 tee the product that we were about to 15 485 kW – IM2001 offer,” says Thiriez. ounded by Eric Thiriez in Cart- agena, Colombia, ETEC’s initial The floating pump design requires a mo- combustion motor – there are fewer fail- goal was to build stationary tor with a small frame that is also totally ure points, less preventive maintenance F flow pumps for government self-cooled with an enclosed fan and has and lower operating costs. companies. Because, in some loca- high efficiency and low temperature rise. tions, the shore would be too soft for Thanks to its comprehensive portfolio, For ABB the partnership means long- the weight of the stationary pumps, ABB was able to meet these requirements term customer relations, an increase in ETEC had to find an alternative to heavy exactly: high energy efficiency (IE2 and the process performance motor busi- construction. A pump floating in water IE3)1, combination rated power Vs frame, ness, and an important order intake. was the solution. After the initial order, ABB received an additional order of 55 motors for the The floating pumps are complete, inte- ABB process Middle East irrigation project. ABB con- grated units, designed for continuous tinues to work with ETEC to develop a operation and capable of handling more performance squir- general performance portfolio for serial than 5,000 l of water per second ➔ 1. They pumps, also increasing participation with can be installed and placed in operation rel cage induction process performance motors together in a short period of time, without the motors run the with ABB softstarters to offer a complete need for the civil construction work typi- pump solution. cally required for other types of pumps floating pumps. with similar or lower flow rates. The floating solution is applicable to a wide range and thermal margins that allow motor of high capacity pumps, from axial flow operation in outside environments up to to mixed flow and multistage pumps, and 55°C at an altitude of 0 m above sea level. are used to move water in aqueducts, agricultural and aquacultural farms, flood By partnering with ABB, ETEC was able Oscar Avella control and irrigation systems. to ensure a cost-effective solution with ABB Discrete Automation and Motion the floating pumps by using ABB high- Bogota, Colombia Running the pumps efficiency process performance motors. [email protected] To power their floating pumps ETEC ETEC offers the pump design with ABB’s chose ABB’s process performance mo- electric motors when a power source is available either by running cables into an Footnote 1 IE2 refers to high-efficiency motors according to electric network or through a local gen- Title picture IEC 60034-30 (2008); IE3 refers to premium- ABB process performance motors are designed for erator. For the floating pumps an electric efficiency motors according to IEC 60034-30 demanding applications and energy savings. motor has distinct advantages over a (2008).

80 ABB review 4|13 ABB review 1|13 ABB review 2|13 Innovation Breakthrough technology

ABB 1 |13 ABB 2 |13 The corporate The corporate review technical journal review technical journal DC breaker and other innovation highlights 6 Hybrid breaker for HVDC 6 Powering up data centers 13 Breathing under the ground 35 Cloud-based vehicle charging 24 Staying ahead in maintenance 64 Interfaces designed for people 70 100 years in power electronics 70 Innovation Breakthrough technology

6 Innovation highlights 6 Breakthrough! ABB’s top innovations for 2013 ABB’s hybrid HVDC breaker, an innovation breakthrough 13 Power packed enabling reliable HVDC grids Smart modular UPS designs 14 Breaking new ground 16 Power factors A circuit breaker with the capacity to switch 15 large power Power quality – problems and solutions plants 20 Guaranteed power 19 The two-in-one chip Smart modular UPS designs provide flexibility and increase The bimode insulated gate transistor (BIGT) availability 24 Easy admittance 24 Cloud-controlled charging The ultimate earth-fault protection function for compen- ABB’s connectivity solutions are changing the sated networks electric vehicle charging industry 29 Clean contact 29 Intelligent workload Contactor technology for power switching and motor A new circuit breaker that reduces breaks by control managing loads 35 Deep breaths 36 Making the switch Optimizing airflow for underground mines ABB’s new multiservice multiplexer, FOX615, meets the new 42 Top gear challenges faced by operational communication networks Technology to improve mining productivity 42 Fine mesh 48 Mine of information Mesh 802.11 wireless network connectivity Integration of mobile equipment in underground mining 45 Cast and calculation 52 OCTOPUS-Onboard eRAMZES – Breakthrough in advanced computer simulations ABB’s motion-monitoring, response-prediction and 52 Net gain heavy-weather decision-support system Keep track of your control system via the Web with ABB’s 54 Control room convergence My Control System Merging industrial monitoring and control systems with 59 Conservation of energy data center operations A paper machine fingerprint cuts energy usage 59 Virtually speaking 64 More Power DCS-to-subsystem interface emulation using SoftCI Understanding your user 64 CRIM 70 Understanding your user Identifying the best maintenance strategy for complex Ethnography helps deliver better operator interface displays process plants 76 Reactor reaction 70 From mercury arc to hybrid breaker ABB batch management with 800xA comes to 100 years in power electronics Colombia for the first time

Index 81 ABB review 3|13 ABB review 4|13 Simulation Data centers

W ABB 3 |13 ABB 4 |13 The corporate en review technical journal review Applied mathematics rationalizes processes 11 The unsung heroes of the Internet 6 The ultrafast disconnector 27 Direct current – a perfect fit for data centers 16 Staying ahead in robotics 61 No power is no option 22 Surviving earthquakes 77 What’s hot in cooling 53 The corporate technical journal

Simulation Data centers

6 Reality predicted 7 Data center defined Simulation power for a better world The infrastructure behind a digital world 11 Reordering chaos 11 Designed for uptime Applied mathematics improves products, industrial Defining data center availability via a processes and operations tier classification system 16 Simulation Toolbox 16 DC for efficiency Dielectric and thermal design of power devices Low-voltage DC power infrastructure in data centers 22 Resisting obsolescence 22 Backing up performance The changing face of engineering simulation ABB emergency power systems for data centers 27 Opening move 29 Power guarantee 30 times faster than the blink of an eye, simulating the Uninterruptible power supply for data centers extreme in HVDC switchgear 34 Continuous power 34 Switching analysis Digital static transfer switches for increased data center Simulation of electric arcs in circuit breakers reliability 39 Picture perfect 41 Automated excellence Electromagnetic simulations of transformers New concepts in the management of data center 44 Head smart infrastructure Strengthening smart grids through real-world pilot 48 Design decisions collaboration What does ABB contribute to the design of data centers? 47 Making sense 53 Keeping it cool Designing more accurate and robust sensors through Optimal cooling systems design and management system and multiphysics simulation 58 In the crystal ball 54 Feeling the pressure Looking ahead at data center design optimization Simulating pressure rise in switchgear installation rooms 64 Taking charge 61 Robot design Flash charging is just the ticket for clean transportation Virtual prototyping and commissioning are enhancing robot 70 In control manipulators and automation systems development ABB’s dredger drives control unit provides a more 65 Integrated ingenuity reliable and integrated control platform for dredging New simulation algorithms for cost-effective design of motor systems highly integrated and reliable power electronic frequency 74 Robust radio converters Meshed Wi-Fi wireless communication for industry 72 Molding the future 79 The right fit Polymers processing enhanced by advanced computer ABB partners with a family-owned company to power simulations floating flow pumps 77 Shake, rattle and roll 81 Index 2013 Helping equipment to withstand earthquakes and reduce The year at a glance noise with design simulations

82 ABB review 4|13 Editorial Board

Claes Rytoft Chief Technology Officer Group R&D and Technology

Clarissa Haller Head of Corporate Communications

Ron Popper Head of Corporate Responsibility

Eero Jaaskela Head of Group Account Management

Andreas Moglestue Chief Editor, ABB Review

Publisher ABB Review is published by ABB Group R&D and Technology.

ABB Technology Ltd. ABB Review Affolternstrasse 44 CH-8050 Zurich Switzerland [email protected] Preview 1|14 ABB Review is published four times a year in English, French, German, Spanish and Chinese. ABB Review is free of charge to those with an interest in ABB’s technology and objectives. For a subscription, please contact your nearest Innovation ABB representative or subscribe online at www.abb.com/abbreview

Partial reprints or reproductions are permitted The opening issue of ABB Review for 2014 will be dedicated to subject to full acknowledgement. Complete reprints require the publisher’s written consent. innovations. Contributions being prepared include a look at some of the challenges around giant wind turbines, how to economize Publisher and copyright ©2013 ABB Technology Ltd. fuel with the latest breakthroughs in turbocharging and how Zurich/Switzerland advanced physics is being used to measure current with previ- ously unknown levels of accuracy. The issue will also explore how Printer Vorarlberger Verlagsanstalt GmbH tomorrow’s drinking water can reach consumers more efficiently, AT-6850 Dornbirn/Austria offer a taste of what is new in simulation, and much more.

Layout DAVILLA AG Zurich/Switzerland

Disclaimer The information contained herein reflects the views of the authors and is for informational purposes only. Readers should not act upon the information contained herein without seeking professional advice. We make publications available with the understanding that the authors are not rendering technical or other professional advice or opinions on specific facts or matters and assume no liability whatsoever in connection with their use. ABB Review on the tablet The companies of the ABB Group do not make any warranty or guarantee, or promise, expressed or implied, concerning the content or accuracy of the A tablet version of ABB Review is now available. To access this, views expressed herein. please visit http://www.abb.com/abbreview ISSN: 1013-3119 www.abb.com/abbreview Stay informed . . .

Have you ever missed a copy of ABB Review? There is now an easy way to be informed every time a new edition of ABB Review (or special report) is published. You can sign up for the e-mail alert at www.abb.com/abbreview

Preview 83 Affix address label here

CanCan one one supplier supplier provide provide your your most most critical critical systems? systems? Certainly.

Think differently about your data center. Rather than integrating products and systems from many different sources, consider a partnership with ABB for comprehensive, intelligent data center packages to power, monitor and automate Think differently about your data center. Rather than integrating products and key elements of your infrastructure. From AC and DC power distribution systems to systems from many different sources, consider a partnership with ABB for grid connections, DCIM and modular UPS solutions, combined with local project comprehensive, intelligent data center packages to power, monitor and automate management and service, ABB is transferring decades of success in mission-critical key elements of your infrastructure. From AC and DC power distribution systems to facilities to the decades ahead for high-performance, reliable data centers. grid connections, DCIM and modular UPS solutions, combined with local project www.abb.com/datacenters management and service, ABB is transferring decades of success in mission-critical Certainly. facilities to the decades ahead for high-performance, reliable data centers. www.abb.com/datacenters