Investigation of Separated Two-Phase Thermosiphon Loop for Relieving the Air-Conditioning Loading in Datacenter

energies

Article Investigation of Separated Two-Phase Thermosiphon Loop for Relieving the Air-Conditioning Loading in Datacenter

Haﬁz M. Daraghmeh 1 , Mohammed W. Sulaiman 1 , Kai-Shing Yang 2 and Chi-Chuan Wang 1,*

1 Department of Mechanical Engineering, National Chiao Tung University, Hsinchu 300, Taiwan; haﬁ[email protected] (H.M.D.); [email protected] (M.W.S.) 2 Green Energy & Environment Research Laboratories, Industrial Technology Research Institute, Hsinchu 300, Taiwan; [email protected] * Correspondence: [email protected]; Tel.: +886-3-5712121

Received: 28 November 2018; Accepted: 24 December 2018; Published: 29 December 2018

Abstract: This study investigates the feasibility of using R-134a filled separated two-phase thermosiphon loop (STPTL) as a free cooling technique in datacenters. Two data center racks one of them is attached with fin and tube thermosiphon were cooled by CRAC unit (computer room air conditioning unit) individually. Thermosiphon can help to partially eliminate the compressor loading of the CRAC; thus, energy saving potential of thermosiphon loop was investigated. The condenser is a water-cooled design and perfluoroalkoxy pipes were used as adiabatic riser/downcomer for easier installation and mobile capability. Tests were conducted with filling ratio ranging from 0 to 90%. The test results indicate that the energy saving increases with the rise of filling ratio and an optimum energy savings of 38.7% can be achieved at filling ratios of 70%, a further increase of filling ratio leads to a reduction in energy saving. At a low filling ratio like 10%, the evaporator starves for refrigerant and a very uneven air temperature distribution occurring at the exit of data rack. The uneven temperature distribution is relieved considerably when the evaporator is fully flooded. It is also found that the energy saving is in line with the rise of system pressure. Overfilling of the evaporator may lead to a decline of system pressure. A lower thermal resistance occurs at high filling ratios and higher ambient temperature.

Keywords: datacenter; free cooling; two-phase ﬂow; thermosiphon; separated loop; PFA pipe

1. Introduction Effective thermal management in datacenter has become more and more important. This is because nowadays the electrical energy consumption of datacenters represents 1.1~1.5% of the world’s total electricity consumption. Besides, traditional cooling systems of datacenter consume nearly 50% of the total energy. Moreover, according to industry predictions, data centers’ annual power demand can reach up to 20% [1]. In this regard, recent researches had focused on implementing new efﬁcient cooling techniques that might help to minimize the electrical power usage and then minimize the costs spent on thermal energy management. Thus, many recent studies had employed free cooling techniques such as direct/indirect airside and waterside free cooling [2]. By introducing these techniques to datacenter, the compressor load can be partially or completely relieved. However, mechanical piping system of such techniques requires long distance transportation of air or water. In other words, they consume a lot of electrical energy through fans or pumps. One alternative for energy-saving technique is based on heat pipe for its passive feature. Using heat pipe for data center cooling such as thermosiphon is being

Energies 2019, 12, 105; doi:10.3390/en12010105 www.mdpi.com/journal/energies Energies 2019, 12, 105 2 of 18 considered as a promising technology for effective heat removal with short distance transportation. Moreover, thermosiphon features unique characteristics such as the simple structure, low-cost and high effective thermal conductivity. For the best energy savings, integrated system of mechanical refrigeration and thermosiphon (ISMT) has been considered as an ideal economization system; that is because it is a gravity driven device which does not require pumping power and auxiliary refrigeration system, thereby reducing considerable greenhouse gas emissions. Furthermore, thermosiphon can work even when the temperature difference between indoor and outdoor is small. ISMT has been used widely for energy saving purposes. Han et al. [3] developed a new ISMT in high heat density mobile phone stations. The integrated system which combines the traditional vapor compression cycle and thermosiphon cycle, consumed about 34.3–36.9% less energy than traditional cooling systems. Zhang et al. [4] proposed an ISMT consists of two circulation loops: a mechanical refrigeration loop and a thermosiphon loop. The experimental investigation results of the performance of the ISMT showed an annual energy-saving up to 47.3%. Zhang et al. [5] studied the effect of air flow rate and geometric parameters on the performance of the ISMT. The results showed that cooling capacity and circulation flow rate increased with the riser diameter of thermosiphon and air flow rate but decreased with the tube length. Generally, thermosiphon can be classified into two main categories. The first one is similar to a single heat pipe with close ends and it comprises of three sections; evaporator, adiabatic section and condenser. The other kind of thermosiphon is the separated two-phase thermosiphon loop (STPTL), sometimes called loop thermosiphon. It is called separated because the evaporator and the condenser are separated and located in different levels, and they are connected to each other by two adiabatic pipes; vapor line (riser), and liquid line (downcomer). Amongst these two kinds of thermosiphon systems, the loop thermosiphon is more reliable in datacenter environment because of its higher capability to transfer heat, the ability to transfer heat at small temperature difference, and since evaporator and condenser are separated, they can be located comparatively far away from each other, which enables them to transfer heat to a longer distance [6]. Many researchers had adopted the STPTL for thermal energy management of datacenters [7–10]. They studied the heat transfer performance of the STPTL and the factors that influence the heat transfer capacity of the loop, such as the thermosiphon structure, working fluid, operating conditions and the like. Among these factors, selecting the working fluid is the most crucial factor. For example, some of the aforementioned studies had investigated the usage of carbon dioxide (R-744) as a working fluid for free cooling in datacenters. R-744 was selected because of its excellent thermodynamic properties (high heat transfer rate and low pressure drop). However, R-744 is not workable in some conditions since it has low critical temperature and comparatively high pressure. Therefore, it can only be used for two-phase thermosiphon systems when the working temperature of thermosiphon is less than 31.1 ◦C[11], which makes it infeasible for high density datacenter racks. Recently, electrical power density of datacenter racks is dramatically increasing. Hence, testing thermosiphon heat exchangers that uses refrigerants with higher critical temperature is more beneficial. Generally, two kinds of experimental researches had been conducted; the first kind of experiment examined different kind of refrigerants in order to decide which among them gives the best heat transfer capability for specific operating conditions. The other focused on finding the optimal filling ratio for specific kind of refrigerant that gives the best heat transfer capability. In this paper, an experimental study has been conducted to compare the power consumption of two identical datacenter racks; one of them is combined with thermosiphon (evaporator) as a part of the STPTL. Also, each rack’s cooling system was tested separately under the same operating and environmental conditions. The average airside outlet temperature of the traditional rack was measured at constant heating load in the range of 28.7~38.9 ◦C. Thus, R-134a has been selected as a working fluid, since its critical temperature is 101.06 ◦C[12]. It is worth mentioning that the STPTL’s adiabatic section that connects the evaporator (thermosiphon) and the condenser is a Teflon pipe made from fluoropolymers called perfluoroalkoxy (PFA). The soft pipe design offers a Energies 2019, 12, 105 3 of 18

ﬂexible and mobile arrangement of condenser. A summary of literatures related to STPTL is shown in Table1.

Table 1. Related literature in association with STPTL.

Source STPTL’s Description Working Fluid Operating Conditions Results ISMT that has three The optimal filling working modes: T.S R-744 The indoor temperature ratios for CO , R22 and Zhang et al. [1] mode, vapor R-22 = 27 ◦C and outdoor 2 R134a are 120%, 100% compression mode and R-134a temperature = (7–22) ◦C and 90%, respectively dual mode Integrated air Heating load = 2112 W. conditioner with The AC rated cooling Energy savings up to Han et al. [3] thermosyphon R-22 capacity was 3200 W. 34.3–36.9% (dual mode direct Indoor temperature = expansion evaporator) 22 ◦C A three-fluid heat Indoor temperature = exchanger is used to The annual energy 27 ◦C Zhang et al. [4] connect refrigeration Unknown saving rate is Outdoor Temperature = loop and a 5.4–47.3% (7–22) ◦C thermosyphon loop Separated heat pipe Heat transfer ability of system. Heat source’s R-22 R-22 is 4.8% higher The evaporator was temperature (32~40) ◦C Ding et al. [6] R-134a than R134a, while placed inside the Condenser’s R-410A R-410A is 8.7% higher datacenter rack to cover temperature (10~14) ◦C than R134a all rack surface area T.S was built to use Heating load = 1.1 kW ambient energy to cool a the annual energy T = 25 ◦C Feng et al. [7] typical data center, the Unknown a consumption could be T.S cooling capacity = volume of T.S = (0.75 × reduced by 35.4% 2 kW 0.32 × 0.29) mm3 Results showed that increasing the liquid and vapor lines section Loop T.S that uses an diameters could be one evaporator that is of the vital keys to n-pentane composed of two improve the system HFC-365mfc Heating load = compensation chambers thermal efficiency. Chehade et al. [9] SES-36 (100~700) W for the liquid and vapor Also, increasing HFE-7000 T = 25 ◦C working fluid separated a ambient temperature FC-3284 by a increases the mass flow microchannels block. rate in the loop and decreases the evaporator pressure drop Finned-tube heat Heat transfer rate = The total thermal exchangers R-744 (1–4) kW resistance of R-744 is Tong et al. [10] The heat transfer area is R-22 Outdoor temperature = (22–25) % lower than about 16.9 m2 18 ◦C that of the R22 Maximum heat The inner surface areas transfer ability is of the evaporator and T was controlled achieved at the fill condenser is (0.42 m2) c,i between 13 and 15 ◦C. ratio around 100%. Z. Tong et al. [11] each. The inner R-744 Heating load range Lowest driving diameters of the riser (200~1700) W temperature difference and downcomer are is achieved at the fill both 6 mm. ratio around 62% Free cooling Heating load = 4.5 kW. percentages of ISMT system consists of T = 24 ◦C approximately 30–70%. Zhang et al. [13] two circulation loops: an R-22 a Filling quantity of T.S The annual energy AC loop and a T.S loop. loop = 3.8 kg saving rates are 16–49% Energies 2019, 12, 105 4 of 18

Table 1. Cont.

Source STPTL’s Description Working Fluid Operating Conditions Results Best cooling capacity A tube-fin three-fluid The indoor and outdoor of thermosiphon mode Zhang et al. [14] heat exchanger and a R-22 temperature differences is 7.1 kW at 20 ◦C parallel flow evaporator are (10, 15 and 20) ◦C temperature difference Integrated air The proposed system ◦ conditioner combines Ta = 24 C can save 33% of the Zhang et al. [15] thermosyphon and Water Cooling capacity was up energy consumed for vapor compression into to 49.2 kW cooling, compared one device with traditional system The optimal refrigerant filling ratio was 88–101%. 1. Dry/wet bulb A micro channel Cooling capacity was Temperature = separated heat pipe, increased by 138% Ling et al. [16] R-22 (13/7.3) ◦C evaporator’s volume when the temperature 2. Dry/wet bulb (750 × 780 × 25) mm3 difference between Temperature = (8/3.4) ◦C indoor and the outdoor increased from 5 ◦C to 10◦C The optimal filling CO loop thermosyphon ratio of R-744 is 150%. 2 The indoor and outdoor with microchannel R-744 The heat transfer rate Zhang et al. [17] temperature difference is parallel-flow R-22 and circulation flow 10 ◦C heat exchangers rate increased with height difference

With reference to the aforementioned related literatures, it is obvious that most of recent studies had focused on the usage of the separated two-phase thermosiphon loop in association with the outdoor environments for cooling the data centers, lacking detailed quantitatively energy saving for the system. Moreover, the idea of placing the thermosiphon on the rear of the data center rack was rarely discussed. This idea was experimentally investigated by Ding et al. [6]. However, they focused on investigating heat transfer capability and the factors that may inﬂuence the heat transfer performance of the loop such as ﬁlling ratio and no quantitatatively energy savings were reported. As a consequence, the objective of this experimental study is to bridge the realization and the actual energy saving when a separated thermosiphon loop as a free cooling technique is adopted, and to investigate the feasibility of using PFA pipe as an adiabatic connection as far as mobility is concerned. Moreover, as depicted in Table1, very rare studies were available concerning the use of R-134a in STPTL even though it is widely used in air-conditioning and refrigeration industry.

2. Experimental Setup and Approach

2.1. STPTL’s Working Principle The thermosiphon attached to the back of data center rack transfers energy from a heat source (data center rack) to a heat rejection section (shell and coil condenser) via natural circulation of the R-134a as the working fluid. The generated vapor moves from the evaporator through riser toward the condenser where it condenses thereafter and circulates back to evaporator through downcomer to complete circulation. The geometric configuration of the STPTL is arranged such that the condensed liquid can flow back to the evaporator by gravity as can be seen in Figure 2a.

2.2. Introduction to the Experimental Setup Two identical datacenter racks have been installed in a small room. The size of the room is nearly (3 × 3 × 3) m3. Two electrical heaters were installed inside each rack. Both heaters reject a maximum heating load of 4 kW to the ambient. The size of each rack is (0.6 × 0.6 × 1.95) m3. The two racks were placed to face each other as shown in Figure1a. An air conditioner (AC) having 14 kW cooling capacity Energies 2018, 11, x FOR PEER REVIEW 5 of 18 Energies 2019, 12, 105 5 of 18 heating load of 4 kW to the ambient. The size of each rack is (0.6 × 0. 6× 1.95) m³. The two racks were was usedplaced as theto face main each cooler other for as the shown racks. in TheFigure AC 1a. was An connected air conditioner from the(AC) top having with rectangular14 kW cooling duct to capacity was used as the main cooler for the racks. The AC was connected from the top with transfer the cold air to a diffuser that was placed between the top of the two racks as shown in Figure1. rectangular duct to transfer the cold air to a diffuser that was placed between the top of the two racks The cross-sectional area of the rectangular duct is (0.3 × 0.3) m2. Also, each rack has an identical set of as shown in Figure 1. The cross‐sectional area of the rectangular duct is (0.3 × 0.3) m². Also, each rack fans (15has fans an identical each). set of fans (15 fans each).

(a) (b)

(c)

FigureFigure 1. Datacenter 1. Datacenter racks’ racks’ conﬁguration: configuration: ( a(a)) A A generalgeneral view of of datacenter datacenter racks racks arrangement arrangement with with duct system;duct system; (b) 3D-modeling(b) 3D‐modeling of of the the datacenter datacenter rack,rack, it shows shows the the position position of ofheaters heaters that that simulate simulate networknetwork servers; servers; (c) Dimensions (c) Dimensions of of the the thermosiphon thermosiphon inin cm.

2.3. Description2.3. Description of the of STPTLthe STPTL A fin-and-tubeA fin‐and‐tube thermosiphon thermosiphon heat heat exchanger exchanger made of of copper copper pipes pipes and and aluminum aluminum fins finswas was attachedattached on the on backthe back of oneof one of theof the racks. racks. The The surface surface area of of the the thermosiphon thermosiphon is 1.11 is 1.11 × 0.69× 0.69m² m2 including collector and header, and it consists of 48 vertical copper tubes with 9.52 mm nominal including collector and header, and it consists of 48 vertical copper tubes with 9.52 mm nominal diameter (10.3 mm collar diameter after expansion). The header of the thermosiphon consists of five diametercopper (10.3 pipes mm (1 collar‐inch diameter diameter each) after which expansion). are connected The header to another of the 2 thermosiphon inches diameter consists extended of five copperheader pipes as (1-inch shown diameterin Figure 2a each) to allow which better are connectedvapor distribution to another and easier 2 inches transmission diameter to extended the headercondenser. as shown The in thermosiphon Figure2a to represents allow better the evaporator vapor distribution of the STPTL, and which easier was connected transmission with the to the condenser.condenser The by thermosiphon an adiabatic fluorothermo represents‐plastic the evaporator material pipes of the (PFA). STPTL, The whichPFA serves was as connected vapor line with the condenserand liquid byline. an Two adiabatic vapor lines fluorothermo-plastic of 2 inches diameter material and about pipes 2.2 m (PFA).length were The PFAused for serves lowering as vapor line and liquid line. Two vapor lines of 2 inches diameter and about 2.2 m length were used for lowering transmission impedance to the condenser, whereas one liquid line of 2 inches diameter, 1.65 m length was installed, which is large enough for the condensed liquid to flow back via gravity to the evaporator. The condenser was placed at the highest point close to the ceiling, it should be higher than the thermosiphon to provide enough space for the condensed liquid to accumulate in the Energies 2018, 11, x FOR PEER REVIEW 6 of 18

transmission impedance to the condenser, whereas one liquid line of 2 inches diameter, 1.65 m length was installed, which is large enough for the condensed liquid to flow back via gravity to the Energies 2019, 12, 105 6 of 18 evaporator. The condenser was placed at the highest point close to the ceiling, it should be higher than the thermosiphon to provide enough space for the condensed liquid to accumulate in the liquid liquidline, line, this this is isquite quite essential essential especially especially at high at high filling ﬁlling ratios ratios (e.g., (e.g., 70% 70%and and90%) 90%) because because the level the levelof of liquid is comparatively high (close to the condenser’s outlet). The condenser is a simple water‐cooled liquid is comparatively high (close to the condenser’s outlet). The condenser is a simple water-cooled shell and coil heat exchanger, and it was designed to reject heating load up to 6 kW. It consists of a shell and coil heat exchanger, and it was designed to reject heating load up to 6 kW. It consists of a 0.75‐inch diameter, 3.5 m length coiled copper pipe as shown schematically in Figure 2a. Chilled 0.75-inchwater diameter, is circulated 3.5 in m the length coiled coiled pipe, copper while the pipe R‐134a as shown vapor schematicallyis circulated in the in Figure annulus2a. around Chilled the water is circulatedtube. in the coiled pipe, while the R-134a vapor is circulated in the annulus around the tube.

(a) (b) FigureFigure 2. The 2. The schematic schematic diagram diagram of the the experimental experimental setup: setup: (a) The (a) components The components of the STPTL; of the ( STPTL;b) (b) MeasurementsMeasurements apparatus.

2.4. Experimental2.4. Experimental Approach Approach and and Operating Operating Conditions Conditions In orderIn order to obtain to obtain the the desired desired results, results, the the following following parameters parameters have have been been studied: studied:

 Ambient air temperature (Ta): the experiment was conducted at three different ambient • Ambient air temperature (Ta): the experiment was conducted at three different ambient temperatures (21 °C, 23 °C, and 25 °C). temperatures (21 ◦C, 23 ◦C, and 25 ◦C).  Chilled water’s temperature (Tc): the experiment was conducted twice at two different • Chilledtemperatures, water’s temperature(15 °C and 20 °C). (Tc ): the experiment was conducted twice at two different ◦ ◦ temperatures, Chilled water’s (15 flowC and rate: 20 waterC). flow rate was constant for all cases (15 L/min). • Chilled Filling water’s ratio of flow R‐134a: rate: the water experiment flow rate was was conducted constant at for five all different cases (15 filling L/min). ratios (10%, 30%, • Filling50%, ratio 70%, of and R-134a: 90%). theThe experimentfilling ratio is wasbased conducted on the volume at five of the different evaporator. filling ratios (10%, 30%, 50%, Heating 70%, and Load: 90%). The The experiment filling ratiowas conducted is based onat a the constant volume heating of the load evaporator. (4 kW) with two heaters (2 kW each) being turned on for all experiment cases. See Figure 1b. • Heating Load: The experiment was conducted at a constant heating load (4 kW) with two heaters (2 kWThe each) experiment being turned was conducted on for all for experiment all cases under cases. the See same Figure conditions,1b. and the total power consumption of cooling systems was compared and recorded then plotted for all the cases of the Theaforementioned experiment parameters. was conducted More fordetails all cases about under the experimental the same conditions, apparatus and and the operating total power consumptionconditions of are cooling explained systems in the following was compared section. and recorded then plotted for all the cases of the aforementioned parameters. More details about the experimental apparatus and operating conditions are explained2.5. Experimental in the Apparatus following and section. Measurements Fans were installed on the front door of each rack. Air velocity was measured by thermo‐ 2.5. Experimentalanemometer Apparatus(LM‐8000, Tecpel, and Measurements Taipei, Taiwan), the measuring range of the device is (0.2~30) m/s Fanswith precision were installed of 0.1 m/s. onAverage the air front velocity door for of each each set of rack. fans was Air 2.5 m/s, velocity PWM controllers was measured were by thermo-anemometerused to adjust the (LM-8000,air velocity Tecpel,to be identical Taipei, for Taiwan), both racks the and measuring it was kept range the same of the for device all cases. is (0.2~30)To find the power consumed by each set of fans, digital multimeter was used to measure voltage and m/s with precision of 0.1 m/s. Average air velocity for each set of fans was 2.5 m/s, PWM controllers current, thus, the calculated power was 66.48 W. The inlet and outlet temperature of the rack were were used to adjust the air velocity to be identical for both racks and it was kept the same for all cases. To find the power consumed by each set of fans, digital multimeter was used to measure voltage and current, thus, the calculated power was 66.48 W. The inlet and outlet temperature of the rack were measured by using 45 T-type thermocouples (OMEGA, Singapore, Singapore), they were fixed and distributed evenly on a mesh as shown in Figure2b. The mesh was fabricated to cover the whole fans area at the inlet and outlet (0.7 × 0.55) m2. The obtained temperature values were used to calculate the average temperature of the evaporator (Te,avg). In order to measure the average temperature of the condenser (Tc,avg), more T-type thermocouples were installed at the inlet and the outlet of the pipe Energies 2018, 11, x FOR PEER REVIEW 7 of 18

171 measured by using 45 T-type thermocouples (OMEGA, Singapore, Singapore), they were fixed and 172 distributed evenly on a mesh as shown in Figure 2b. The mesh was fabricated to cover the whole fans 173 area at the inlet and outlet (0.7 × 0.55) m². The obtained temperature values were used to calculate the 174 average temperature of the evaporator (Te,avg). In order to measure the average temperature of the 175Energies condenser2019, 12, 105 (Tc,avg), more T-type thermocouples were installed at the inlet and the outlet of7 th ofe 18 pipe 176 wall of the chill water as shown in Figure 2(b). A rotameter flow meter was used to measure water 177 flowrate in condenser, and its accuracy is ±0.2 l/min. All thermocouples were calibrated by RTD with wall of the chill water as shown in Figure2b. A rotameter flow meter was used to measure water 178 an accuracy of ±0.1°C. The values of temperatures were obtained by using digital recorder (LR8400- flowrate in condenser, and its accuracy is ±0.2 l/min. All thermocouples were calibrated by RTD with 179 20, HIOKI E. E., Nagano, Japan). The same data logger was used to measure the working pressure of an accuracy of ±0.1◦C. The values of temperatures were obtained by using digital recorder (LR8400-20, 180 R-134a at different filling ratios, the values were recorded by using (Yokogawa, Tokyo, Japan) type HIOKI E. E., Nagano, Japan). The same data logger was used to measure the working pressure of 181 absolute pressure transducer which was connected with the header of the thermosiphon as shown in R-134a at different filling ratios, the values were recorded by using (Yokogawa, Tokyo, Japan) type 182 Figure 2b, the range of the device is 0~20 kgf/cm2, and the measurement uncertainty for voltage is absolute pressure transducer which was connected with the header of the thermosiphon as shown in 183 ±0.001 V. The electrical power consumption was recorded for every case using WT210 digital power Figure2b, the range of the device is 0~20 kgf/cm 2, and the measurement uncertainty for voltage is 184 meter (Yokogawa, Tokyo, Japan), model number 760401, and its uncertainty is ±0.1 W. Experimental ±0.001 V. The electrical power consumption was recorded for every case using WT210 digital power 185 apparatus with the accuracy of each device are tabulated in Table 2. meter (Yokogawa, Tokyo, Japan), model number 760401, and its uncertainty is ±0.1 W. Experimental apparatus with the accuracy of each device are tabulated in Table2. 186 Table 2. Measurements devices and their accuracy.

Apparatus Table 2. Measurements devicesAccuracy and their accuracy. Unit Anemometer (LM-8000), ±0.1 m/s Apparatus Accuracy Unit Digital multimeter ±0.01 V, A Anemometer (LM-8000) ±0.1 m/s Thermocouple (T-Type)Digital multimeter ±0.1±0.01 V, A °C Rotameter flow meterThermocouple (T-Type) ±0.2±0.1 ◦C l/min Pressure transducerRotameter ﬂow meter ±0.001±0.2 l/min V Pressure transducer ±0.001 V Digital power meterDigital power meter ±0.1±0.1 W W

187 3. Data Reduction 3. Data Reduction 188 A comparative experiment was performed to validate the effectiveness of the STPTL. The A comparative experiment was performed to validate the effectiveness of the STPTL. 189 experiment started with testing the traditional datacenter rack (with no thermosiphon). Electrical The experiment started with testing the traditional datacenter rack (with no thermosiphon). Electrical 190 power consumed by AC (Pac), the inlet air temperature (Tai), and outlet air temperature (Tao) of the power consumed by AC (Pac), the inlet air temperature (Tai), and outlet air temperature (Tao) of the 191 datacenter rack were recorded then compared with the results of the other rack that is combined with datacenter rack were recorded then compared with the results of the other rack that is combined with 192 the STPTL. The performance of STPTL was investigated at different filling ratios of the working fluid the STPTL. The performance of STPTL was investigated at different filling ratios of the working fluid 193 (10%, 30%, 50%, 70%, and 90%). The filling ratio is defined by the following Equation: (10%, 30%, 50%, 70%, and 90%). The filling ratio is defined by the following Equation: Ѵ FR% = l Ѵ (1) FR% = el (1) e 194 where Ѵ퐥 is the R-134a liquid volume and Ѵ퐞 is the volume of the evaporator. Best performance of 195 the loop can be obtained through the longest relief of compressor running time of the air-conditioning where l is the R-134a liquid volume and e is the volume of the evaporator. Best performance of the 196loop system can be where obtained maximum through energy the longest savings relief can of be compressor obtained. running time of the air-conditioning system where maximum energy savings can be obtained. 197 3.1. Power Consumed by the Rack with no Thermosiphon. 1983.1. PowerWhile Consumed measuring by the Rackthe inlet with and No Thermosiphonoutlet air temperature at front and back door of the rack, power 199 meterWhile recording measuring the the power inlet andconsumption outlet air temperatureof the AC within at front specific and backperiod door of oftime the (30 rack, minutes) power was 200meter adopted recording for theall cases. power Power consumption of fans was of the calculated AC within by specific multiplying period the of supply time (30 voltage min) was and adopted the current 201for allwhich cases. were Power measured of fans by was the calculated multi-meter, by multiplying namely the supply voltage and the current which were measured by the multi-meter, namely Pf = V  I (2)

202 where Pf is the total power consumed P byf = theV × (15I fans). Thus, total power consumption (2) of the 203 traditional rack (Ptr) is the summation of power consumed by fans and AC, i.e., where Pf is the total power consumed by the (15 fans). Thus, total power consumption of the traditional Ptr = Pac + Pf (3) rack (Ptr) is the summation of power consumed by fans and AC, i.e.,

204 3.2. Power Consumed by the Rack with the PThermosiphontr = Pac + Pf Loop STPTL. (3)

3.2. Power Consumed by the Rack with the Thermosiphon Loop STPTL In addition to the power consumed by fans and AC, power consumed by condenser’s chilled water should be added. Accordingly, total power consumed by the rack with thermosiphon can be calculated such that, Energies 2018, 11, x FOR PEER REVIEW 7 of 18

186 where, Pch is the powerTable consumed 2. Measurements by chiller devices and Pp andis the their power accuracy. consumed by the pump. In order to calculate Pch, the amount of heat removed at the condenser has to be known, it can be determined by theAp energyparatus balance equation: Accuracy Unit Anemometer (LM-8000), Qc = m˙ wa±0.1× Cpwa (Tc,o − Tc,i)m/s (5) Digital multimeter ±0.01 V, A where Tc,i and Tc,o are the inlet and outlet temperatures of the condenser, respectively. Then, Pch can Thermocouplebe calculated (T-Type) by dividing chilled water’s±0.1 cooling capacity on the coefficient of performance°C of the Rotameterchiller asflow follows: meter ±0.2 l/min Q Pressure transducer ±0.001P = c V (6) ch COP Digital power meter ±0.1 W where COP being chiller’s coefficient of performance and it is equal to 4.2. The liquid pump’s electrical consumption at a specific flow rate (15 l/min) is approximately equal 187 3. Data Reduction to 60 W. The saving of energy subject to saving’s percentage was calculated such that, 188 A comparative experiment was performed to validate the effectiveness of the STPTL. The P 189 experiment started with testing the traditionalE.S = datacenter1 − t rack× 100% (with no thermosiphon). Electrical (7) Ptr 190 power consumed by AC (Pac), the inlet air temperature (Tai), and outlet air temperature (Tao) of the 191 datacenter rackUncertainty were recorded analysis then regarding compared power with consumption the results calculations of the other are rack tabulated that is in combined Table3. with 192 the STPTL. The performance of STPTL was investigated at different filling ratios of the working fluid Table 3. Uncertainty analysis regarding power consumption calculations. 193 (10%, 30%, 50%, 70%, and 90%). The filling ratio is defined by the following Equation: Equation Uncertainty Ѵ s l FR% = 2 (1) δ(P ) δ(Ѵ)e δ(I) 2 Pf f = + 0.136 W Pf I 194 where Ѵ is the R-134a liquid volume and Ѵ qis the volume of the evaporator. Best performance of 퐥 퐞 2 2 Ptr δ( ) = (δ ) + (δ ) 0.169 W 195 the loop can be obtained through the longestPtr relief Poff compressorPac running time of the air-conditioning s 2 δ(Q ) δ( ) δ(T) 2 196 system where maximum energyQc savings canc = be obtained.+ (11.49~24.15) W Qc T q P 2 2 2 (25.51~55.88) W 197 3.1. Power Consumed by the Rt ack with noδ(P Tt)hermosiphon.= (δPtr) + (δP f) + (δPac)

198 While4. measuring Results and the Discussion inlet and outlet air temperature at front and back door of the rack, power 199 meter recording the power consumption of the AC within specific period of time (30 minutes) was 200 adopted for4.1. all Power cases. Consumption Power of fans and Energywas calculated Savings Results by multiplying the supply voltage and the current 201 which were measuredThe experiment by the wasmulti conducted-meter, namely subject to three ambient temperatures (21 ◦C, 23 ◦C, and 25 ◦C, respectively) in association with ﬁlling ratio ranging from 10~90%. The power consumptions for Pac, Pf = V  I ◦ ◦ (2) Pf,Pch, and Pp, are tabulated in Tables4 and5 for T c,i = 15 C and 20 C, respectively. Figure3a,b show 202 where Pf isthe the measured total power power consumption consumed by at steady the (15 state fans). as a function Thus, total of ﬁlling power ratios. consumption Yet the typical of outlet the

203 traditionaltemperature rack Energies(Ptr) is 2018 ofthe, the 11 ,summation x data FOR PEER rack REVIEW is schematicallyof power consumed shown in by Figure fans4 .and AC, i.e., 9 of 18

5000 5000 Ptr = Pac + Pf (3) P.C at Ta = 21 °C P.C at Ta = 21 °C 4500 4500 P.C at T = 23 °C P.C at Ta = 23 °C

a (W)

(W)

P.C at Ta = 25 °C P.C at Ta = 25 °C 4000 204 3.2. Power Consumed4000 by the Rack with the Thermosiphon Loop STPTL. 3500 3500 Consumption Consumption

3000 3000

Power 2500 Power 2500

Total Total 2000 2000

1500 1500 (No T. S) 10% 30% 50% 70% 90% (No T. S) 10% 30% 50% 70% 90% FR (%) FR (%) (a) (b) 1000 0.020 Figure 3. Cont.0.018 980 0.016 960 (kg/s)

0.014 (kPa)

940 0.012

flowrate 0.010 920 Pressure 0.008

900 Mass

0.006 T = 21 °C 880 Ta = 21 °C 134a a ‐ Working 0.004 R T = 23 °C Ta = 23 °C a 860 0.002 T = 25 °C Ta = 25 °C a 840 0.000 10% 30% 50% 70% 90% 10% 30% 50% 70% 90% FR (%) FR (%) (c) (d)

Figure 3. Total power consumed by the cooling system and the corresponding working pressure and mass flow rate of working fluid at different filling ratios: (a) Case (1), total power consumption when

Tc,i = 15 °C; (b) Case (2), total power consumption when Tc,i = 20 °C; (c) Working pressure at different

filling ratios when Tc,i = 15 °C; (d) Mass flowrate of R‐134a at different filling ratios when Tc,i = 15 °C.

FR = 10 % FR = 30 % FR = 50 % FR = 70 % FR = 90 % Ta = 21 °C Ta = 21 °C Ta = 21°C Ta = 21°C Ta = 21°C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C SD = 5.24 °C SD = 5.10 °C SD = 4.38 °C SD = 1.72 °C SD = 2.87 °C

FR = 10 % FR = 30 % FR = 50 % FR = 70 % FR = 90 % Ta = 23°C Ta = 23 °C Ta = 23 °C Ta = 23 °C Ta = 23 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C

EnergiesEnergies2019, 12 2018, 105, 11, x FOR PEER REVIEW 9 of 189 of 18

(a) (b) 1000 0.020

0.018 980 0.016 960 (kg/s)

0.014 (kPa)

940 0.012

flowrate 0.010 920 Pressure 0.008

900 Mass

0.006 T = 21 °C 880 Ta = 21 °C 134a a ‐ Working 0.004 Ta = 23 °C R Ta = 23 °C 860 0.002 Ta = 25 °C Ta = 25 °C 840 0.000 10% 30% 50% 70% 90% 10% 30% 50% 70% 90% FR (%) FR (%) (c) (d)

FigureF 3.igureTotal 3. Total power power consumed consumed by by the the cooling cooling system system andand the corresponding corresponding working working pressure pressure and and mass flowmass rate flow of rate working of working fluid fluid at different at different filling filling ratios: ratios: ((aa)) Case (1), (1), total total power power consumption consumption when when Tc,i◦ = 15 °C; (b) Case (2), total power consumption when Tc,i = 20 °C;◦ (c) Working pressure at different Tc,i = 15 C; (b) Case (2), total power consumption when Tc,i = 20 C; (c) Working pressure at different filling ratios when Tc,i = ◦15 °C; (d) Mass flowrate of R‐134a at different filling ratios when Tc,i = 15 °C. ◦ Energiesfilling 2018 ratios, 11, x whenFOR PEER Tc,i REVIEW= 15 C; (d) Mass flowrate of R-134a at different filling ratios when Tc,i9 =of15 18 C.

FR = 10 % FR = 30 % FR = 50 % FR = 70 % FR = 90 % FR = 10 % FR = 30 % FR = 50 % FR = 70 % FR = 90 % Ta = 21 °C Ta = 21 °C Ta = 21°C Ta = 21°C Ta = 21°C Ta = 21 °C Ta = 21 °C Ta = 21°C Ta = 21°C Ta = 21°C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C SD = 5.24 °C SD = 5.10 °C SD = 4.38 °C SD = 1.72 °C SD = 2.87 °C SD = 5.24 °C SD = 5.10 °C SD = 4.38 °C SD = 1.72 °C SD = 2.87 °C

FR = 10 % FR = 30 % FR = 50 % FR = 70 % FR = 90 % FR = 10 % FR = 30 % FR = 50 % FR = 70 % FR = 90 % Ta = 23°C Ta = 23 °C Ta = 23 °C Ta = 23 °C Ta = 23 °C Ta = 23°C Ta = 23 °C Ta = 23 °C Ta = 23 °C Ta = 23 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C SD = 5.52 °C SD = 5.22 °C SD = 4.43 °C SD = 1.65 °C SD = 2.37 °C SD = 5.52 °C SD = 5.22 °C SD = 4.43 °C SD = 1.65 °C SD = 2.37 °C

FR = 10 % FR = 30 % FR = 50 % FR = 70 % FR = 90 % FR = 10 % FR = 30 % FR = 50 % FR = 70 % FR = 90 % Ta = 25 °C Ta = 25 °C Ta = 25 °C Ta = 25 °C Ta = 25 °C Ta = 25 °C Ta = 25 °C Ta = 25 °C Ta = 25 °C Ta = 25 °C

Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C Tc,i = 20 °C SD = 5.49 °C SD = 5.27 °C SD = 4.55 °C SD = 1.63 °C SD = 1.64 °C ◦ FigureFigure 4. Racks’Racks’ outlet outlet temperature temperature distribution distribution (Tc,i = (T 20c,i °C).= 20 C).

In Figure 3, the test results showed that the performance of the STPTL is improved with the rise of filling ratios and the maximum energy saving occurs at a filling ratio of 70%, and a further increase of the filling ratio to 90% leads to a moderate decline of energy saving. At a low filling ratio such as 10% and 30%, despite some energy saving is still achievable for such low filling ratios, the uneven high outlet temperatures as shown in Figure 4 implicates that the low filling ratios cannot effectively provide a temperature uniformity, the standard deviation of the temperature variation of airflow at the outlet of thermosiphon heat exchanger is as high as 5.52 C. Details about energy saving percentage are explained in Table 4 and Table 5. Notice that percentage of energy savings can reach up to 33.2% in one of 30% filling ratio cases.

Table 4. Energy consumption and energy savings details at Tc,i = 15 °C.

Ta (°C) All Cases Pac (W) Pch (W) Pf (W) Pp(W) Pt(W) E.S (%) 21 (No T.S) 4493.7 0.0 66.48 0 4560.2 0.0

Energies 2019, 12, 105 10 of 18

In Figure3, the test results showed that the performance of the STPTL is improved with the rise of filling ratios and the maximum energy saving occurs at a filling ratio of 70%, and a further increase of the filling ratio to 90% leads to a moderate decline of energy saving. At a low filling ratio such as 10% and 30%, despite some energy saving is still achievable for such low filling ratios, the uneven high outlet temperatures as shown in Figure4 implicates that the low filling ratios cannot effectively provide a temperature uniformity, the standard deviation of the temperature variation of airflow at the outlet of thermosiphon heat exchanger is as high as 5.52 ◦C. Details about energy saving percentage are explained in Tables4 and5. Notice that percentage of energy savings can reach up to 33.2% in one of 30% filling ratio cases.

◦ Table 4. Energy consumption and energy savings details at Tc,i = 15 C.

◦ Ta ( C) All Cases Pac (W) Pch (W) Pf (W) Pp (W) Pt (W) E.S (%) (No T.S) 4493.7 0.0 66.48 0 4560.2 0.0 FR (10%) 3338.0 230.0 66.48 60 3694.5 19.0 FR (30%) 3085.8 417.7 66.48 60 3630.0 20.4 21 FR (50%) 3025.3 526.6 66.48 60 3678.4 19.3 FR (70%) 2520.8 592.9 66.48 60 3240.2 28.9 FR (90%) 2478.8 684.5 66.48 60 3289.8 27.9 (No T.S) 3901.0 0.0 66.48 0 3967.4 0.0 FR (10%) 2606.0 142.9 66.48 60 2875.3 27.5 FR (30%) 2483.1 171.4 66.48 60 2781.0 29.9 23 FR (50%) 2317.1 276.2 66.48 60 2719.8 31.4 FR (70%) 1933.4 385.7 66.48 60 2445.6 38.4 FR (90%) 1873.6 447.6 66.48 60 2447.7 38.3 (No T.S) 2604.6 0.0 66.48 0 2671.0 0.0 FR (10%) 2068.6 99.2 66.48 60 2294.3 14.1 FR (30%) 1913.7 205.5 66.48 60 2245.7 15.9 25 FR (50%) 1733.6 241.9 66.48 60 2102.0 21.3 FR (70%) 1373.6 211.2 66.48 60 1711.3 35.9 FR (90%) 1371.2 285.7 66.48 60 1783.4 33.2

◦ Table 5. Energy consumption and energy savings details at Tc,i = 20 C.

◦ Ta ( C) All Cases Pac (W) Pch (W) Pf (W) Pp (W) Pt (W) E.S (%) (No T.S) 4493.7 0.0 66.48 0 4560.2 0.0 FR (10%) 3830.7 166.7 66.48 60 4123.8 9.6 FR (30%) 3153.4 369.0 66.48 60 3648.9 20.0 21 FR (50%) 3147.9 381.0 66.48 60 3655.3 19.8 FR (70%) 2601.8 409.8 66.48 60 3138.0 31.2 FR (90%) 2559.8 473.8 66.48 60 3160.1 30.7 (No T.S) 3901.0 0.0 66.48 0 3967.4 0.0 FR (10%) 3005.7 119.0 66.48 60 3251.2 18.1 FR (30%) 2358.3 166.7 66.48 60 2651.5 33.2 23 FR (50%) 2450.0 190.5 66.48 60 2766.9 30.3 FR (70%) 2066.2 238.1 66.48 60 2430.8 38.7 FR (90%) 2039.3 285.7 66.48 60 2451.5 38.2 (No T.S) 2604.6 0.0 66.48 0 2671.0 0.0 FR (10%) 2148.5 81.0 66.48 60 2355.9 11.8 FR (30%) 1981.3 125.2 66.48 60 2233.0 16.4 25 FR (50%) 1964.9 156.2 66.48 60 2247.6 15.9 FR (70%) 1604.9 166.7 66.48 60 1898.1 28.9 FR (90%) 1617.4 202.4 66.48 60 1946.2 27.1

As shown in Tables4 and5, the best power consumption was obtained at filling ratio of 70% where the energy savings is as high as 38.7%. This is because the evaporator is fully flooded with such filling ratio, and the evaporator can fulfill effective evaporation. Yet the yielded higher pressure with such high filling ratios also ensure circulating more working fluid across the circuitry as shown in Figure3c, since the flowrate also increases with the rising pressure. Figure3d depicts the working Energies 2019, 12, 105 11 of 18

fluid’s flowrate at different filling ratios. This can also make clear from the temperature contour at the outlet of data rack as seen in Figure4 where the temperature uniformity is almost attainable when the filling ratio exceeds 70%. However, a further increase of filling ratio to 90%, result in a slightly reduction of energy saving as compared to the filling ratio of 70%. The results indicate that there is no extra energy saving benefit when the filling ratio exceeds the fully flooded condition of the evaporator. Hence, increasing filling ratio beyond fully flooding is actually not helping as far as energy saving is concerned, indicating overfilling might lead to degradation in heat transfer rate. This can be elaborated from the pressure measurement as depicted in Figure3c where the system pressure for 90% filling ratio is slightly lower (about 10 kPa less) than a filling ratio of 70%. The energy saving is actually in line with the system pressure, and the maximum energy saving (70%) occurs at the maximum system pressure. Note that the system pressure is gradually increased from about 855 kPa to 990 kPa while the filling ratio ranges from 10~70%. The results reveal a plateau at an optimum filling ratio near 70%. The system pressure then drops about 10 kPa when the filling ratio is further raised to 90%. To explain this phenomenon, one would need to understand the phase change process of the thermosiphon at the present constant heating load condition. For a low filling ratio of 10–30%, the evaporator can be divided into two portions, the lower part of thermosiphon is the boiling zone while the upper part of the thermosiphon is simply a superheated zone, and the schematic of the low filling ratio can be depicted in Figure5a. The level of the working fluid in these cases could not exceed 0.22 m height in the STPTL. The under-filling situation starves the evaporator, thereby leading to a higher uneven temperature distribution especially at the upper part of the thermosiphon heat exchanger as shown in Figure4. Thereby, the cooling capacity and circulation flowrate will be decreased. With continuous rise of filling ratio from 10% to 70%, the system pressure also increases accordingly and the flow inside the evaporator may become fully flooded at a filling ratio near 70% with the lower part being the boiling zone while the upper part being evaporation zone as schematically seen in Figure5b. Boiling of the refrigerant at the lower part generates appreciable amount of vapor flow that brings the liquid entrainment toward the upper part of the thermosiphon that eventually hits and retains on the upper periphery of the thermosiphon by vapor flow, thus resulting in a film like structure, hence the heat transfer mode at the upper part is via evaporation rather than the boiling mode at the lower portion. The simultaneous presence of evaporation regime and boiling zone in a typical closed thermosiphon was first reported by [18] who experimentally reported geyser boiling in a vertical annular two-phase closed thermosiphon. A schematic about the geyser boiling is shown in Figure6 by Khazaee et al. [ 19]. As seen in the figure, small bubbles are generated from the nucleate boiling zone, followed by growing size to the diameter of the thermosiphon tube. The vapor plugs then pushes the liquid slug upwards and eventually breaks up to spread around the tube periphery to yield annular flow zone as depicted in Figure5b. This phenomenon was also numerically verified by Jouhara et al. [20]. In this range with filling ratios from 10% to 70%, a continuous rise of vapor fraction through phase change via boiling or evaporation can effectively expand the vapor volume pronouncedly, therefore leading to a rise of system pressure. On the other hand, when the filling ratio is increased to 90% where the evaporator is in a state of overfilling as seen in Figure5c. In this case, the evaporator is simply contained with a single boiling zone. The overfilling into the evaporator will not induce more vapor volume since the total heat input still maintains the same. Instead, it will reduce the mean vapor quality at the evaporator. Figure7 shows the levels of liquid in evaporator and condenser at different filling ratios subject to no heat inputs, it is obvious that the liquid level of 90% FR exceeds the height of the rack. As a consequence, part of the liquid does not expose to the hot air. Note that the boiling heat transfer coefficient is normally higher than the evaporation heat transfer coefficient. This is normally the case in the present thermosiphon design for the vapor velocity is comparatively low. Notice that the present thermosiphon design is a passive device which differs greatly from active designs (forced convective boiling via compressor or pump). In essence, effect of nucleate boiling may surpass the influence of convective evaporation. The results indicate the effective heat transfer coefficient for the 90% filling ratio is higher than that of filling ratio of 70%. Based on a simple relation Energies 2018, 11, x FOR PEER REVIEW 12 of 18

Energies 2018, 11, x FOR PEER REVIEW 12 of 18 In this range with filling ratios from 10% to 70%, a continuous rise of vapor fraction through phaseIn thischange range via with boiling filling or ratiosevaporation from 10% can to effectively 70%, a continuous expand the rise vapor of vapor volume fraction pronouncedly, through phasetherefore change leading via boiling to a rise or of evaporation system pressure. can effectively On the other expand hand, the when vapor the volume filling ratiopronouncedly, is increased thereforeto 90% where leading the to evaporator a rise of system is in a pressure.state of overfilling On the other as seen hand, in Figure when 5c.the In filling this case, ratio the is increasedevaporator tois 90% simply where contained the evaporator with a is single in a state boiling of overfilling zone. The as overfilling seen in Figure into 5c.the In evaporator this case, the will evaporator not induce is moresimply vapor contained volume with since a thesingle total boiling heat input zone. still The maintains overfilling the intosame. the Instead, evaporator it will will reduce not theinduce mean morevapor vapor quality volume at the since evaporator. the total heatFigure input 7 shows still maintains the levels the of same. liquid Instead, in evaporator it will reduce and condenser the mean at vapordifferent quality filling at the ratios evaporator. subject to Figure no heat 7 showsinputs, the it is levels obvious of liquid that the in evaporatorliquid level andof 90% condenser FR exceeds at differentthe height filling of the ratios rack. subject As a consequence,to no heat inputs, part itof is the obvious liquid thatdoes the not liquid expose level to the of 90%hot air.FR exceedsNote that thethe height boiling of heatthe rack. transfer As acoefficient consequence, is normally part of thehigher liquid than does the notevaporation expose to heat the transferhot air. Notecoefficient. that theThis boiling is normally heat transfer the case coefficient in the present is normally thermosiphon higher than design the forevaporation the vapor heat velocity transfer is comparatively coefficient. Thislow. is Noticenormally that the the case present in the thermosiphon present thermosiphon design is a design passive for device the vapor which velocity differs greatlyis comparatively from active low.designs Notice (forced that the convective present thermosiphon boiling via compressor design is a passiveor pump). device In essence,which differs effect greatly of nucleate from activeboiling Energiesdesignsmay2019 surpass, 12(forced, 105 the convective influence boiling of convective via compressor evaporation. or pump). The results In essence, indicate effect the effective of nucleate heat boiling transfer 12 of 18 maycoefficient surpass for the the influence 90% filling of convective ratio is higher evaporation. than that Theof filling results ratio indicate of 70%. the Based effective on a simpleheat transfer relation coefficientof Newton’s for the cooling 90% fillinglaw where ratio isQ higher = hA (T thanw – thatTsat), of where filling h ratio is the of heat 70%. transfer Based on coefficient, a simple relation Tw is the surface temperature and Tsat is the saturation temperature. The corresponding saturation temperature of Newton’sof Newton’s cooling cooling law law where where Q Q = = hA hA (T(Tw –− TTsatsat), where), where h is h the is the heat heat transfer transfer coefficient, coefficient, Tw is the Tw is the must be reduced due to a higher boiling heat transfer coefficient for a filling ratio of 90%. In this surfacesurface temperature temperature and and T satTsatis is thethe saturation saturation temperature. temperature. The The corresponding corresponding saturation saturation temperature temperature mustmustregard, be reduced be reduced the corresponding due due to a to higher a higher system boiling boiling pressure heat heat transferis slightlytransfer coefficient decreasedcoefficient for with for a fillinga filling ratio ratioratio of ofof 90%. 90%. 90%. In In this this regard, the correspondingregard, the corresponding system pressure system pressure is slightly is slightly decreased decreased with awith filling a filling ratio ratio of 90%. of 90%.

Figure 5. Schematic of the pool boiling process subject to different filling ratios. FigureFigure 5. 5.Schematic Schematic ofof the pool boiling boiling process process subject subject to different to different filling ﬁlling ratios. ratios.

Energies 2018, 11, x FOR PEER REVIEW 13 of 18 FigureFigure 6. A 6. schematic A schematic presentation presentation about about geyser geyser boiling in in a aclosed closed thermosiphon thermosiphon [19]. [ 19]. Figure 6. A schematic presentation about geyser boiling in a closed thermosiphon [19].

FigureFigure 7. Liquid7. Liquid levels levels in in the the STPTLSTPTL for for different different filling ﬁlling ratios. ratios.

4.2. Thermal Resistance of the STPTL Calculations and Results For further elaboration of the performance of thermosiphon heat exchanger, performance was investigated in terms of thermal resistance of the STPTL. The heat dissipation from the heater is regarded as Q and 45 thermocouples were used to measure the average air inlet temperature (Tai) and outlet temperature (Tao), and some other thermocouples were used to measure the average temperature (Tc, avg) of the condenser. As mentioned earlier, the experiment was conducted at constant heating load (4 kW). However, the actual heat absorbed by the thermosiphon varies with operating conditions, such as rack inlet temperature (ambient temperature) and filling ratio. As depicted in Figure 8, there is no direct contact between heaters and thermosiphon, yet; the amount of heat transferred to the evaporator depends on the temperature in‐between the heaters and thermosiphon (Th). Hence, heaters’ temperature can be calculated as:

QH Th Tai (8) ṁa1cpa

Where QH represents the heaters’ power into the data rack. As can be seen in Figure 8, air temperature and air flowrate changed after exiting the thermosiphon. Accordingly, the amount of heat absorbed by the thermosiphon can be calculated as,

Q ṁ CPT T (9)

Energies 2019, 12, 105 13 of 18

4.2. Thermal Resistance of the STPTL Calculations and Results For further elaboration of the performance of thermosiphon heat exchanger, performance was investigated in terms of thermal resistance of the STPTL. The heat dissipation from the heater is regarded as Q and 45 thermocouples were used to measure the average air inlet temperature (Tai) and outlet temperature (Tao), and some other thermocouples were used to measure the average temperature (Tc,avg) of the condenser. As mentioned earlier, the experiment was conducted at constant heating load (4 kW). However, the actual heat absorbed by the thermosiphon varies with operating conditions, such as rack inlet temperature (ambient temperature) and ﬁlling ratio. As depicted in Figure8, there is no direct contact between heaters and thermosiphon, yet; the amount of heat transferred to the evaporator depends on the temperature in-between the heaters and thermosiphon (Th). Hence, heaters’ temperature can be calculated as: QH Th = + Tai (8) m˙ a1cpa where QH represents the heaters’ power into the data rack. As can be seen in Figure8, air temperature and air ﬂowrate changed after exiting the thermosiphon. Accordingly, the amount of heat absorbed by the thermosiphon can be calculated as,

Q = m˙ a2 Cpa(Tao − Th) (9) Energies 2018, 11, x FOR PEER REVIEW 14 of 18

FigureFigure 8. 8.Changes Changes ofof air temperature and and flow ﬂow rate rate during during experiment. experiment.

TheThe average average temperature temperature of of evaporator evaporator was was measured measured at atthe the surface, surface, then then experimental experimental overall overall thermalthermal resistance resistance can can be be calculated calculated as, as,

T, , R Te,avg − Tc,avg (10) ,R = Q (10) th,exp Q The overall thermal resistance is plotted in Figure 9 as a function of filling ratios. The results showedThe overall that lower thermal thermal resistance resistance is plotted occurs at in high Figure filling9 as ratios a function (70% and of filling90%) and ratios. at a Thehigher results showedambient that temperature. lower thermal Correspondingly, resistance occurs the boiling at high heat filling transfer ratios coefficient (70% is and higher 90%) at high and filling at a higher ambientratios. temperature. This is because Correspondingly, the corresponding boiling the boiling heat transfer heat transfer coefficient coefficient is increased is higher with the at system high filling ratios.pressure. This is Conversely, because the the corresponding condensation heat boiling transfer heat coefficient transfer coefficient decreases with is increased the rise of with pressure. the system pressure.In a typical Conversely, thermosiphon, the condensation the increase heat of boiling transfer heat coefficient transfer coefficient decreases is with always the higher rise of than pressure. the In a typicaldecline thermosiphon, in condensation the increase heat transfer of boiling coefficient, heat transfer this can coefficient be made isclear always through higher some than typical the decline correlations applicable for boiling and condensation, e.g., Equations (11) and (12). In essence, a slight in condensation heat transfer coefficient, this can be made clear through some typical correlations reduction in overall thermal resistance is seen with the rise of ambient temperature. To validate the applicable for boiling and condensation, e.g., Equations (11) and (12). In essence, a slight reduction in experimental data of overall thermal resistance, some empirical correlations can be used to predict overallpool thermal boiling resistanceand film condensation is seen with heat the risetransfer of ambient coefficients temperature. are made. To For validate current thestudy, experimental The datacorrelations of overall thermalfor estimating resistance, the boiling some heat empirical transfer correlations coefficient in can a thermosphion be used to predict by Imura pool et al. boiling [21] and filmhas condensation been chosen heat to predict transfer pool coefficients boiling heat are transfer made. Forcoefficient current (h study,b), while The Nusselt’s correlations theory for of estimating film the boilingcondensation heat transferon a vertical coefficient flat plate in can a thermosphion be applied to predict by Imura the average et al. [ 21heat] has transfer been coefficient chosen to of predict the nucleate liquid film boiling (hco) [22], these correlations are as follows:

. . . . . ρ k C g P h 0.32 q. . . . (11) ρ h μ P

ρρ ρ g k h (12) h 0.707 μT Tl Subsequently, the predicted overall thermal resistance can be calculated such that: 1 1 R, (13) hA hA The results of the predicted overall thermal resistance are shown in Figure 9. It is obvious that predicted values of thermal resistance are in line with the experimental data, especially at high filling ratios. This is because the thermal resistance can be well described by the boiling heat transfer correlation since the evaporator is fully flooded. Yet the influence of the ambient temperature on the overall thermal resistance is rather small as explained earlier. Note that a higher ambient temperature into the data rack leads to a higher heating load of the thermosiphon, and a lower thermal resistance accordingly. Analogous results were obtained by Fadhl et al. [23] who also concluded that the thermal

Energies 2019, 12, 105 14 of 18

pool boiling heat transfer coefficient (hb), while Nusselt’s theory of film condensation on a vertical flat plate can be applied to predict the average heat transfer coefficient of the nucleate liquid film boiling (hco)[22], these correlations are as follows: ! ρ 0.65k 0.3C 0.7g0.2 P 0.3 = l l pl v 0.4 hb 0.32 0.25 0.4 0.1 q (11) ρv hfg µl Patm

1 3 ! 4 ρl(ρl − ρv) g kl hfg hco = 0.707 (12) µl(Tsat − Tw)lc Subsequently, the predicted overall thermal resistance can be calculated such that:

1 1 Rth,pre = + (13) heAe hcAc

The results of the predicted overall thermal resistance are shown in Figure9. It is obvious that predicted values of thermal resistance are in line with the experimental data, especially at high filling ratios. This is because the thermal resistance can be well described by the boiling heat transfer correlation since the evaporator is fully flooded. Yet the influence of the ambient temperature on the overall thermal resistance is rather small as explained earlier. Note that a higher ambient temperature into theEnergies data 2018 rack, 11, leads x FOR PEER to a REVIEW higher heating load of the thermosiphon, and a lower thermal15 resistance of 18 accordingly. Analogous results were obtained by Fadhl et al. [23] who also concluded that the thermal resistanceresistance of the of thermosiphonthe thermosiphon tends tends to to increase increase atat lowlow heating input, input, while while it was it was appreciably appreciably decreased at high heating inputs. Their results showed that thermal resistance becomes relatively decreased at high heating inputs. Their results showed that thermal resistance becomes relatively stable at a specific high heating input. However, as shown in Figure 9, the measured overall thermal stableresistance at a specific of the high thermosiphon heating input. at lower However, filling ratios as shown (e.g., 10~30%) in Figure are9, thedetectably measured higher overall than those thermal resistanceof calculations. of the thermosiphon The departure at loweramid the filling predictions ratios (e.g., and the 10~30%) experimental are detectably measurements higher is than because those of calculations.of the starvation The departure of evaporator amid the at predictions such low filling and the ratios experimental where the measurementsheat transfer process is because in the of the starvationevaporator of evaporator is appreciably at such offset low by filling the upper ratios part where of the the evaporator heat transfer where process only single in the‐phase evaporator heat is appreciablytransfer offset for vapor by the prevails. upper partTo sum of the up, evaporator thermal performance where only of single-phase the thermosiphon heat transfer tends to for get vapor prevails.improved To sum by up, raising thermal the filling performance ratios. However, of the thermosiphon the performance tends may to reach get improved a plateau bywhen raising the the fillingevaporator ratios. However, is fully flooded, the performance and a further may overfilling reach amay plateau impair when the overall the evaporator performance. is fully flooded, and a further overfilling may impair the overall performance.

0.12

0.10 Rth, exp at Ta = 21 °C

/W) R at T = 23 °C

th, exp a 0.08 R at T = 25 °C (°C th, exp a

Rth, pre at Ta = 21 °C 0.06 Rth, pre at Ta = 23 °C

Rth, pre at Ta = 25 °C

Resistance 0.04

0.02 Thermal

0.00

10% 30% 50% 70% 90%

FR (%)

FigureFigure 9. Experimental 9. Experimental and and predicted predicted thermal resistance resistance of the of theSTPTL STPTL for different for different filling ratios, ﬁlling (T ratios,c,i ◦ (Tc,i == 15 15 C).C).

5. Conclusions This experimental study aims to investigate the energy saving potential by using a separated two‐phase thermosiphon loop (STPTL) as a free cooling technique in datacenter. The thermosiphon heat exchanger takes the form as fin‐and‐tube configuration and is installed at the exit of a data rack. R‐134a is used as the working fluid, and a water‐cooled shell and tube heat exchanger is used as the condenser. The connected adiabatic piping, i.e., riser and downcomer, are made via PFA for easier installation and mobility consideration. The fin‐and‐tube thermosiphon heat exchanger is made of copper pipes and aluminum fins. The surface area of the thermosiphon is 1.11 × 0.69 m² including collector and header, and it consists of 48 vertical copper tubes with 9.52 mm nominal diameter (10.3 mm collar diameter after expansion). Tests are conducted with filling ratio ranging from 0 to 90% and the ambient temperatures varies from 21 C to 25 C. Based on the foregoing discussions, the following conclusions can be derived: 1. The experimental results indicate that the energy saving increases with the rise of filling ratio. However, the optimum energy savings can be achieved at a filling ratio of 70%, a further increase of filling ratio leads to a reduction in energy saving. A substantial energy savings of 38.7% can be achieved at the filling ratio of 70%. 2. At a low filling ratio like 10%, the evaporator starves for refrigerant and a very uneven air temperature distribution occurring at the exit of data rack. The uneven temperature distribution is relieved considerably when the evaporator is fully flooded.

Energies 2019, 12, 105 15 of 18

5. Conclusions This experimental study aims to investigate the energy saving potential by using a separated Energies 2018, 11, x FOR PEER REVIEW 7 of 18 two-phase thermosiphon loop (STPTL) as a free cooling technique in datacenter. The thermosiphon heat exchanger takes the form as fin-and-tube configuration and is installed at the exit of a data rack. 171 measured by using 45 T-type thermocouples (OMEGAR-134a is, Singapore used as the working, Singapore fluid,) and, they a water-cooled were fixed shell and and tube heat exchanger is used as the 172 distributed evenly on a mesh as shown in Figure 2b.condenser. The mesh The was connected fabricated adiabatic to piping, cover i.e.,the riserwhole and fans downcomer, are made via PFA for easier 173 area at the inlet and outlet (0.7 × 0.55) m². The obtainedinstallation temperature and mobility values consideration. were used The to fin-and-tube calculate thermosiphon the heat exchanger is made of copper pipes and aluminum fins. The surface area of the thermosiphon is 1.11 × 0.69 m2 including 174 average temperature of the evaporator (Te,avg). Incollector order to and measure header, and the it average consists of temperature 48 vertical copper of the tubes with 9.52 mm nominal diameter 175 condenser (Tc,avg), more T-type thermocouples were(10.3 installed mm collar at diameter the inlet after and expansion). the outlet Tests of are th conductede pipe with filling ratio ranging from 0 to 176 wall of the chill water as shown in Figure 2(b). A90% rotameter and the ambient flow meter temperatures was used varies fromto measure 21 ◦C to 25 water◦C. Based on the foregoing discussions, the 177 flowrate in condenser, and its accuracy is ±0.2 l/minfollowing. All thermocouples conclusions can be were derived: calibrated by RTD with 178 an accuracy of ±0.1°C. The values of temperatures1. wereThe obtained experimental by resultsusing indicate digital that recorder the energy (LR8400 saving- increases with the rise of filling ratio. 179 20, HIOKI E. E., Nagano, Japan). The same data loggerHowever, was used the optimumto measure energy the savings working can be pressure achieved at of a filling ratio of 70%, a further increase of filling ratio leads to a reduction in energy saving. A substantial energy savings of 38.7% can be 180 R-134a at different filling ratios, the values were recorded by using (Yokogawa, Tokyo, Japan) type achieved at the filling ratio of 70%. 181 absolute pressure transducer which was connected2. withAt athe low header filling ratioof the like thermosiphon 10%, the evaporator as shown starves in for refrigerant and a very uneven air 182 Figure 2b, the range of the device is 0~20 kgf/cm2, andtemperature the measurement distribution occurring uncertainty at the exitfor ofvoltage data rack. is The uneven temperature distribution 183 ±0.001 V. The electrical power consumption was recordedis relieved for every considerably case whenusing the WT210 evaporator digital is fully power flooded. 184 meter (Yokogawa, Tokyo, Japan), model number 760401,3. It is alsoand found its uncertainty that the energy is saving±0.1 W. is inExperimental line with the rise of system pressure. Overfilling of the evaporator may lead to a decline of system pressure. This is because the evaporator is fully 185 apparatus with the accuracy of each device are tabulatedflooded in Table and only 2. nucleate boiling prevails. 4. A lower overall thermal resistance occurs at high filling ratios and a higher ambient temperature. 186 Table 2. Measurements devicesHowever, and their the effectaccuracy. of ambient temperature on the overall thermal resistance is negligible at high filling ratios. Yet appreciable increase in overall thermal resistance is encountered at low filling Apparatus Accuracyratios due to the starvation of workingUnit fluid. Anemometer (LM-8000), ±0.15. It is found that the overall thermal resistancem/s is in line with the predictions especially at the high Digital multimeter ±0.01 filling ratios. V, A Thermocouple (T-Type) ±0.1 °C Author Contributions: All the authors have contributed their efforts to complete the paper. H.M.D. conducted Rotameter flow meter ±0.2the experiments and wrote the first draft, M.W.S.l/min provided technical assistance, K.-S.Y. supervised the experiment, Pressure transducer ±0.001and C.-C.W. supervised the work and review andV editing the manuscript. Funding: The authors are indebted to the financial support from the Ministry of Economics Affairs (Department Digital power meter ±0.1of Industrial Technology), Taiwan, under grantW number H301AR6100 and Ministry of Science and Technology, Taiwan under contracts 107-2622-E-009-002-CC2 and 107-2221-E-009-142. 187 3. Data Reduction Conflicts of Interest: The authors declare no conflict of interest.

188 A comparative experiment was performedNomenclature to validate the effectiveness of the STPTL. The 189 experiment started with testing the traditional datacenterT rack temperature (with (◦ noC) thermosiphon). Electrical V voltage (V) 190 power consumed by AC (Pac), the inlet air temperature (Tai), and outlet air temperature (Tao) of the I electrical current (A) 191 datacenter rack were recorded then compared withP the results electrical of the powerother consumption rack that (W)is combined with 192 the STPTL. The performance of STPTL was investigatedQ at different heat transfer filling rate (W)ratios of the working fluid 193 (10%, 30%, 50%, 70%, and 90%). The filling ratio ism˙ defined by mass the flowfollowing rate (kg/s) Equation: Cp specific heat (kJ/kg·◦C) gѴl gravity acceleration (m/s2) FR% = 3 (1) Ѵe volume (m ) R thermal resistance (◦C/kW) 194 where Ѵ퐥 is the R-134a liquid volume and Ѵ퐞 is hthe volume heatof the transfer evaporator. coefficient (W/m Best2· ◦performanceC) of 195 the loop can be obtained through the longest relief of compressor running time of the air-conditioning 196 system where maximum energy savings can be obtained.

197 3.1. Power Consumed by the Rack with no Thermosiphon. 198 While measuring the inlet and outlet air temperature at front and back door of the rack, power 199 meter recording the power consumption of the AC within specific period of time (30 minutes) was 200 adopted for all cases. Power of fans was calculated by multiplying the supply voltage and the current 201 which were measured by the multi-meter, namely

Pf = V  I (2)

202 where Pf is the total power consumed by the (15 fans). Thus, total power consumption of the 203 traditional rack (Ptr) is the summation of power consumed by fans and AC, i.e.,

Ptr = Pac + Pf (3)

204 3.2. Power Consumed by the Rack with the Thermosiphon Loop STPTL.

Energies 2019, 12, 105 16 of 18

A area (m2) P pressure (kPa) q heat ﬂux (W/m2) hfg latent heat (kJ/kg) l length (m)

Greek symbols ρ density (kg/m3) k thermal conductivity (W/m·◦C) µ Dynamic viscosity (Pa.s)

Subscripts a ambient c condenser i inlet o outlet e evaporator avg average ai air inlet ao air outlet l liquid ac air conditioner f fans tr traditional t total ch chiller wa water p pump w wall sat saturation h hot air H heater th thermal b pool boiling co ﬁlm condensation v vapor atm atmospheric exp experimental pre predicted

Abbreviations STPTL separated two-phase thermosiphon loop CRAC computer room air-conditioning ISMT integrated system of mechanical refrigeration and thermosiphon PFA perfluoroalkoxy T.S thermosiphon AC air conditioner T.C thermocouple PWM pulse-width modulation RTD resistance temperature detector FR filling ratio COP coefficient of performance P.C power consumption E.S energy savings SD standard deviation Energies 2019, 12, 105 17 of 18

References

1. Zhang, H.; Shao, S.; Jin, T.; Tian, C. Numerical investigation of a CO2 loop thermosyphon in an integrated air conditioning system for free cooling of data centers. Appl. Therm. Eng. 2017, 126, 1134–1140. [CrossRef] 2. Daraghmeh, H.M.; Wang, C.-C. A review of current status of free cooling in datacenters. Appl. Therm. Eng. 2017, 114, 1224–1239. [CrossRef] 3. Han, L.; Shi, W.; Wang, B.; Zhang, P.; Li, X. Development of an integrated air conditioner with thermosyphon and the application in mobile phone base station. Int. J. Refrig. 2013, 36, 58–69. [CrossRef] 4. Zhang, H.; Shao, S.; Xu, H.; Zou, H.; Tian, C. Integrated system of mechanical refrigeration and thermosyphon for free cooling of data centers. Appl. Therm. Eng. 2015, 75, 185–192. [CrossRef] 5. Zhang, H.; Shao, S.; Tian, C. Simulation of the Thermosyphon Free Cooling Mode in an Integrated System of Mechanical Refrigeration and Thermosyphon for Data Centers. Energy Procedia 2015, 75, 1458–1463. [CrossRef] 6. Ding, T.; Cao, H.W.; He, Z.G.; Li, Z. Experiment research on influence factors of the separated heat pipe system, especially the filling ratio and Freon types. Appl. Therm. Eng. 2017, 118, 357–364. [CrossRef] 7. Zhou, F.; Tian, X.; Ma, G. Investigation into the energy consumption of a data center with a thermosyphon heat exchanger. Chin. Sci. Bull. 2011, 56, 2185–2190. [CrossRef] 8. Zheng, Y.; Li, Z.; Liu, X.; Tong, Z.; Tu, R. Retrofit of air-conditioning system in data center using separate heat pipe system. In Proceedings of the 8th International Symposium on Heating, Ventilation and Air Conditioning; Springer: Berlin/Heidelberg, Germany, 2013; pp. 685–694. 9. Chehade, A.; Louahlia-Gualous, H.; Le Masson, S.; Lépinasse, E. Experimental investigations and modeling of a loop thermosyphon for cooling with zero electrical consumption. Appl. Therm. Eng. 2015, 87, 559–573. [CrossRef] 10. Tong, Z.; Ding, T.; Li, Z.; Liu, X.-H. An experimental investigation of an R744 two-phase thermosyphon loop used to cool a data center. Appl. Therm. Eng. 2015, 90, 362–365. [CrossRef] 11. Tong, Z.; Liu, X.-H.; Li, Z.; Jiang, Y. Experimental study on the effect of fill ratio on an R744 two-phase thermosyphon loop. Appl. Therm. Eng. 2016, 99, 302–312. [CrossRef] 12. Ong, K.S.; Haider-E-Alahi, M. Performance of a R-134a-filled thermosyphon. Appl. Therm. Eng. 2003, 23, 2373–2381. [CrossRef] 13. Zhang, H.; Shao, S.; Xu, H.; Zou, H.; Tang, M.; Tian, C. Simulation on the performance and free cooling potential of the thermosyphon mode in an integrated system of mechanical refrigeration and thermosyphon. Appl. Energy 2017, 185, 1604–1612. [CrossRef] 14. Zhang, H.; Shao, S.; Xu, H.; Zou, H.; Tang, M.; Tian, C. Numerical investigation on integrated system of mechanical refrigeration and thermosyphon for free cooling of data centers. Int. J. Refrig. 2015, 60, 9–18. [CrossRef] 15. Zhang, P.; Wang, B.; Wu, W.; Shi, W.; Li, X. Heat recovery from Internet data centers for space heating based on an integrated air conditioner with thermosyphon. Renew. Energy 2015, 80, 396–406. [CrossRef] 16. Ling, L.; Zhang, Q.; Yu, Y.; Liao, S.; Sha, Z. Experimental study on the thermal characteristics of micro channel separate heat pipe respect to different filling ratio. Appl. Therm. Eng. 2016, 102, 375–382. [CrossRef]

17. Zhang, H.; Shi, Z.; Liu, K.; Shao, S.; Jin, T.; Tian, C. Experimental and numerical investigation on a CO2 loop thermosyphon for free cooling of data centers. Appl. Therm. Eng. 2017, 111, 1083–1090. [CrossRef] 18. Lin, T.F.; Lin, W.T.; Tsay, Y.L.; Wu, J.C.; Shyu, R.J. Experimental investigation of geyser boiling in an annular two-phase closed thermosyphon. Int. J. Heat Mass Transf. 1995, 38, 295–307. [CrossRef] 19. Khazaee, I.; Hosseini, R.; Noie, S.H. Experimental investigation of effective parameters and correlation of geyser boiling in a two-phase closed thermosyphon. Appl. Therm. Eng. 2010, 30, 406–412. [CrossRef] 20. Jouhara, H.; Fadhl, B.; Wrobel, L.C. Three-dimensional CFD simulation of geyser boiling in a two-phase closed thermosyphon. Int. J. Hydrogen Energy 2016, 41, 16463–16476. [CrossRef] 21. Imura, H.; Kusuda, H.; Ogata, J.-I.; Miyazaki, T. Sakamoto, Heat transfer in two-phase closed-type thermosyphons. Nasa ADS 1979, 45, 712–722. Energies 2019, 12, 105 18 of 18

22. Jouhara, H.; Robinson, A.J. Experimental investigation of small diameter two-phase closed thermosyphons charged with water, FC-84, FC-77 and FC-3283. Appl. Therm. Eng. 2010, 30, 201–211. [CrossRef] 23. Fadhl, B.; Wrobel, L.C.; Jouhara, H. Numerical modelling of the temperature distribution in a two-phase closed thermosyphon. Appl. Therm. Eng. 2013, 60, 122–131. [CrossRef]

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).