The Analysis of Residuals Variation and Outliers to Obtain Robust Response Surface
Total Page:16
File Type:pdf, Size:1020Kb
A Service of Leibniz-Informationszentrum econstor Wirtschaft Leibniz Information Centre Make Your Publications Visible. zbw for Economics Bashiri, Mahdi; Moslemi, Amir Article The analysis of residuals variation and outliers to obtain robust response surface Journal of Industrial Engineering International Provided in Cooperation with: Islamic Azad University (IAU), Tehran Suggested Citation: Bashiri, Mahdi; Moslemi, Amir (2013) : The analysis of residuals variation and outliers to obtain robust response surface, Journal of Industrial Engineering International, ISSN 2251-712X, Springer, Heidelberg, Vol. 9, pp. 1-10, http://dx.doi.org/10.1186/2251-712X-9-2 This Version is available at: http://hdl.handle.net/10419/147149 Standard-Nutzungsbedingungen: Terms of use: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Documents in EconStor may be saved and copied for your Zwecken und zum Privatgebrauch gespeichert und kopiert werden. personal and scholarly purposes. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle You are not to copy documents for public or commercial Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich purposes, to exhibit the documents publicly, to make them machen, vertreiben oder anderweitig nutzen. publicly available on the internet, or to distribute or otherwise use the documents in public. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, If the documents have been made available under an Open gelten abweichend von diesen Nutzungsbedingungen die in der dort Content Licence (especially Creative Commons Licences), you genannten Lizenz gewährten Nutzungsrechte. may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by/2.0/ www.econstor.eu Bashiri and Moslemi Journal of Industrial Engineering International 2013, 9:2 http://www.jiei-tsb.com/content/9/1/2 ORIGINAL RESEARCH Open Access The analysis of residuals variation and outliers to obtain robust response surface Mahdi Bashiri* and Amir Moslemi Abstract In this paper, the main idea is to compute the robust regression model, derived by experimentation, in order to achieve a model with minimum effects of outliers and fixed variation among different experimental runs. Both outliers and nonequality of residual variation can affect the response surface parameter estimation. The common way to estimate the regression model coefficients is the ordinary least squares method. The weakness of this method is its sensitivity to outliers and specific residual behavior, so we pursue the modified robust method to solve this problem. Many papers have proposed different robust methods to decrease the effect of outliers, but trends in residual behaviors pose another important issue that should be taken into account. The trends in residuals can cause faulty estimations and thus faulty future decisions and outcomes, so in this paper, an iterative weighting method is used to modify both the outliers and the residuals that follow abnormal trends in variation, like descending or ascending trends, so they will have less effect on the coefficient estimation. Finally, a numerical example illustrates the proposed approach. Keywords: Coefficient estimation, Response surface, Ordinary least squares, Outliers, Robust model Introduction squares (OLS) method. But basic OLS is very sensitive to In many cases, especially in experimental results, some of outliers, and they may have an inordinate effect on the ul- the data are wrong and should be treated as outliers. timate conclusion. So a robust method or a modified OLS These points, which may occur because of operator read- should be used for decreasing the outliers’ sensitivity. ing faults and the like, may have a confusing effect on the Our goal in this study is to decrease the destructive total interpretation of the results. A common method of effect of outliers. In order to do so, at the first stage, the explaining and analyzing the results of experiments is by robust regression or modified regression model should be response surface design. This term is used for a regression computed. Then the trend of residuals in response surface equation that shows the whole behavior of the control var- design is another aspect which should be considered. The iables, the nuisance factors, and the response or responses. trend behaviors among residuals, both descending and as- We can use the estimated function to predict the response cending trends, can cause faulty interpretations. However, to changes in the values of specific controllable factors. there are some assumptions in estimating the regression After determining an experimental design and performing coefficients that should not be violated. It seems that by experiments, the next steps are generally statistical ana- decreasing the effects of some residuals in the coefficient lysis and then the selection of values for the input estimation stage, the initial assumptions can be satisfied, variables so as to optimize the output. This can be done and moreover, because this decreases the overall varia- by fitting a regression model between the controllable bility, the robustness of the model will be increased. The factors and the response variables. Future interpretations main purpose of this paper is to decrease the effects of are based on this regression model, so the exact model is outliers that violate the variance equality test of residuals. very important and may affect the optimization stage. This An example of such trends and abnormal behavior is model is generally constructed by the ordinary least shown in Figure 1. As mentioned before, the OLS method is very sensitive * Correspondence: [email protected] to outliers. To diminish the effect of these points, some al- Department of Industrial Engineering, Faculty of Engineering, Shahed ternative methods of model fitting, such as least absolute University, Tehran, Iran © 2013 Bashiri and Moslemi; licensee Springer. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Bashiri and Moslemi Journal of Industrial Engineering International 2013, 9:2 Page 2 of 10 http://www.jiei-tsb.com/content/9/1/2 a. Descending trend b. Ascending trend c. oscillation trend Figure 1 Trends in residual behavior. (a) Descending trend. (b) Ascending trend. (c) Oscillation trend. deviations and other robust approaches that simplify the defined by minimizing a robust measure of the scatter of task of outlier identification by weighting the large resi- the residuals. Generalizing this, Rousseeuw and Yohai duals, are used instead of OLS. Response surfaces have (1984) introduced S-estimators which are significantly been studied by many researchers, and many approaches more efficient than the previous estimators. A more re- have been proposed either to obtain efficient response sur- cent suggestion is the constrained M-estimates, or CM, faces or optimize the response surface by different models. proposed by Mendes and Tyler (1995), which combines Hejazi et al. (2010) proposed a novel approach based on the good local properties of the M-estimates and the good goal programming to find the best combination of factors global robustness properties of the S-estimates. A ‘partial’ to optimize multi-response-multi-covariate surfaces by version of the M-estimator based on the ‘fair’ ψ function considering location and dispersion effects. Kazemzadeh and an appropriate weighting scheme was recently et al. (2008) proposed a method to optimize multi- proposed by Serneels et al. (2005). The authors believed response surfaces based on a goal programming method. that the partial robust M-regression outperforms existing Robust regression approaches have also been surveyed by methods for robust partial least square regression. many researchers. Huber (Bertsimas and Shioda 2007) Bertsimas and Shioda (2007) presented mixed integer pro- proposed M-estimator methods to obtain robust regres- gramming or MIP models for the classification and robust sion. Morgenthaler and Schumacher (1999) discussed regression problems. Zioutas and Avramidis (2005) exam- robust response surfaces in chemistry based on design of ined the effect of deleting outliers in the regression model experiment. Because of the weakness of the previous obtained by mixed integer programming and compared methods in compensating for outliers, the redescending the performance of this model with that of least squares, M-estimators (also named GM-estimators) were proposed or LS, and LMS. Another new method in robust regres- by Andrews et al. (1972), which are able to reject extreme sion is the mixed linear model surveyed by Dornheim and outliers entirely. Hund et al. (2002) presented various Brazauskas (2011). (Pop and Sârbu 1996) proposed a new methods of outlier detection and evaluated robustness fuzzy regression algorithm to obtain robust models. tests with different experimental designs. Bickela and Maronna et al. (2006) proposed many M-estimators using Frühwirthb (2006) compared different robust estimators robust regression methods in both single response and with their applications. The M- and GM-estimators work multiple responses. Shahriari et al. (2011) proposed a by an iterative procedure. As a consequence, several au- novel two-step