water Article A Practical Protocol for the Experimental Design of Comparative Studies on Water Treatment Long Ho 1,* , Olivier Thas 2,3,4, Wout Van Echelpoel 1 and Peter Goethals 1 1 Department of Animal Sciences and Aquatic Ecology, Ghent University, Ghent 9000, Belgium; [email protected] (W.V.E.); [email protected] (P.G.) 2 Department of Mathematical Modelling, Statistics and Bioinformatics, Ghent University, Ghent 9000, Belgium; [email protected] 3 National Institute for Applied Statistics Research Australia, University of Wollongong, Wollongong 2522, NSW, Australia 4 Interuniversity Institute for Biostatistics and statistical Bioinformatics, Hasselt University, Hasselt 3500, Belgium * Correspondence: [email protected]; Tel.: +32-926-438-95 Received: 16 December 2018; Accepted: 15 January 2019; Published: 17 January 2019 Abstract: The design and execution of effective and informative experiments in comparative studies on water treatment is challenging due to their complexity and multidisciplinarity. Often, environmental engineers and researchers carefully set up their experiments based on literature information, available equipment and time, analytical methods and experimental operations. However, because of time constraints but mainly missing insight, they overlook the value of preliminary experiments, as well as statistical and modeling techniques in experimental design. In this paper, the crucial roles of these overlooked techniques are highlighted in a practical protocol with a focus on comparative studies on water treatment optimization. By integrating a detailed experimental design, lab experiment execution, and advanced data analysis, more relevant conclusions and recommendations are likely to be delivered, hence, we can maximize the outputs of these precious and numerous experiments. The protocol underlines the crucial role of three key steps, including preliminary study, predictive modeling, and statistical analysis, which are strongly recommended to avoid suboptimal designs and even the failure of experiments, leading to wasted resources and disappointing results. The applicability and relevance of this protocol is demonstrated in a case study comparing the performance of conventional activated sludge and waste stabilization ponds in a shock load scenario. From that, it is advised that in the experimental design, the aim is to make best possible use of the statistical and modeling tools but not lose sight of a scientific understanding of the water treatment processes and practical feasibility. Keywords: experimental design; comparative studies; wastewater treatment; preliminary studies; virtual experiments; power analysis 1. Introduction Water treatment is a crucial technology in water reuse and management. In the last century, technologies for drinking water purification and wastewater treatment have evolved rapidly to deal with the significantly increasing amount of water discharge as a result of the industrial revolution and the growth of urbanization [1]. To increase the removal and energy efficiency, environmental engineers have paid a great deal of attention to optimizing technology via comparing or contrasting different treatments. In particular, water treatments can be improved by changing five elements: (1) treatment technologies, (2) system configurations, (3) experimental methodology, (4) system inputs, and (5) Water 2019, 11, 162; doi:10.3390/w11010162 www.mdpi.com/journal/water Water 2019, 11, 162 2 of 17 Water 2019, 11, x FOR PEER REVIEW 2 of 17 operationaland (5) conditions operational (Figure conditions1). The (Figure improvement 1). The improvement of the treatments of the can treatments then be demonstrated can then be via threedemonstrated sets of indicators, via three which sets represent of indicators, the balanced which represent sustainability the balanced of water sustainability technologies, of water including environmentaltechnologies, performance, including economicenvironmental performance, performa andnce, societaleconomic sustainability performance, [2]. and societal sustainability [2]. FigureFigure 1. Overview 1. Overview of of different different types types ofof comparativecomparative studies studies on on water water treatment treatment and andthe use the of use of comparativecomparative indicators. indicators. IncludingIncluding multidisciplinary multidisciplinary aspects, aspects, suchsuch asas physics, chemistry, chemistry, microbiology, microbiology, and and bioprocess bioprocess engineering, studies on water treatment heavily depend on experimental observations [3]. In these engineering, studies on water treatment heavily depend on experimental observations [3]. In these experiments, three essential elements should be carefully considered: (1) analytical techniques, (2) experiments,experimental three setup, essential and (3) elements statistical shouldanalysis. be Th carefullye first two considered:elements have (1) been analytical well covered techniques, by (2) experimentalmany popular setup, textbooks and (3)or statisticalguidelines, analysis.such as ‘Standard The first Methods two elements for the Examination have been of well Water covered and by manyWater popular’ of APHA textbooks [4], and or guidelines,‘Experimental such Methods as ‘ Standardin Water Treatment Methods’ forof van the ExaminationLoosdrecht, Nielsen, of Water Lopez- and Water’ of APHAVazquez [4], andand Brdjanovic ‘Experimental [3]. MethodsOn the other in Water hand, Treatment to the authors’’ of van knowledge, Loosdrecht, there Nielsen, is no standardized Lopez-Vazquez and Brdjanovicprocedure for [3]. statistical On the otheranalysis hand, in comparative to the authors’ studies knowledge, on water treatment. there is no Interestingly, standardized in proceduremany for statisticalother fields, analysis such inas comparativeecology, evolution, studies biology, on water or treatment.behavioral Interestingly,science, researchers in many agree other that fields, such asexperimental ecology, evolution, design and biology, statistical or analysis behavioral have an science, intimate researchers link with each agree other. that As experimental such, it is crucial design to carefully think about the exact formulation of the research questions, and explicitly translate them and statistical analysis have an intimate link with each other. As such, it is crucial to carefully think into statistical hypotheses before conducting experiments and collecting data [5]. about the exact formulation of the research questions, and explicitly translate them into statistical This paper aims at providing a practical protocol for experimental design within the scope of hypothesescomparative before studies conducting on water experiments treatment. andThroughout collecting this data protocol [5]. , we highlight key steps in the Thisexperimental paper aims design at where providing the connection a practical to statistics protocol is formost experimental critical. Moreover, design the withinrole of modeling the scope of comparativeand simulation studies as onthe watercounterpart treatment. of real experiments Throughout in this the virtual protocol, world we is highlight also emphasized key steps in the in the experimentalprotocol since design these where tools the can connection be very effective to statistics but have is rarely most critical.been used Moreover, in experimental the role design. of modeling To and simulationkeep the size as of the this counterpart paper manageable of real experiments and avoid getting in the lost virtual in the world intricacies is also of emphasized statistics and in the protocolmodeling, since thesewe keep tools the canprotocol be very as simple effective as possi butble, have hence, rarely details been on used some in basic experimental principles design.of To keepexperimental the size ofdesign, this papersuch as manageable blocking, randomization, and avoid gettingreplication, lost and in thefactorial intricacies designs, of which statistics can and be found in many textbooks, are not included. Finally, to ensure that experimenters find this protocol modeling, we keep the protocol as simple as possible, hence, details on some basic principles of straightforward, easy to use, and highly applicable, its application is demonstrated via a case study experimental design, such as blocking, randomization, replication, and factorial designs, which can be on the performance comparison between conventional activated sludge (CAS) and waste foundstabilization in many textbooks, pond (WSP) are in nota peak included. load scenario. Finally, to ensure that experimenters find this protocol straightforward, easy to use, and highly applicable, its application is demonstrated via a case study on the performance comparison between conventional activated sludge (CAS) and waste stabilization pond (WSP) in a peak load scenario. Water 2019, 11, 162 3 of 17 2. The ProtocolWater 2019, 11, x FOR PEER REVIEW 3 of 17 The main2. The structure Protocol of the protocol is composed of four main stages with 10 modules and two feedback loops (FigureThe main2 ).structure Each ofof thesethe protocol stages is composed will be discussedof four main instages the with following 10 modules sections, and two with specific attention to thefeedback section-specific loops (Figure 2). modules. Each of these stages will be discussed in the following sections, with specific attention to the section-specific modules. Figure 2. Main structure of the protocol. The protocol includes two feedback loops that allow for new Figure