Journal of the Korean Institute of Industrial Engineers Vol. 34, No. 2, pp. 122-139, June 2008.
Performance and Robustness of Control Charting Methods for Autocorrelated Data
Chang-Ho Chin1 Daniel W. Apley2
1School of Mechanical and Industrial Systems Engineering, Kyung Hee University, 446-701, Republic of Korea 2Department of Industrial Engineering and Management Sciences, Northwestern University, USA
With the proliferation of in-process measurement technology, autocorrelated data are increasingly common in industrial SPC applications. A number of high performance control charting techniques that take into account the specific characteristics of the autocorrelation through time series modeling have been proposed over the past decade. We present a survey of such methods and analyze and compare their performances for a range of typical autocorrelated process models. One practical concern with these methods is that their performances are often strongly affected by errors in the time series models used to represent the autocorrelation. We also provide some analytical results comparing the robustness of the various methods with respect to time series modeling errors. * Keywords: Control Charts, Autocorrelation, Robustness, Average Run Length, Sensitivity Measure
1. Introduction time. With significant advances in measurement and data collection technology, however, measurements are taken at increasingly higher rates and are more Statistical process control (SPC) has been used to ach- likely to be autocorrelated (Montgomery and Woodall ieve and maintain control of various processes in in- 1997; Woodall and Montgomery 1999). This leads to dustry (Stoumbos, Reynolds, Ryan, and Woodall a significant deterioration of traditional control chart 2000). The control chart is a primary SPC tool to mon- performance, a phenomenon that has been discussed itor process variability and promote quality improve- by Johnson and Bagshaw (1974), Bagshaw and Johnson ment by means of detecting process shifts requiring cor- (1975), Harris and Ross (1991), Alwan (1992), Woodall rective actions. As a graphical monitor, control charts and Faltin (1993), and many others. Positive autocorre- generally contain a centerline and two other horizontal lation typically increases the variance of the charted lines called control limits, the width of which is often statistic so that the control limits determined under the proportional to the standard deviation of the charted independence assumption are too narrow, giving a statistic. If a point plots outside the control limits, the higher-than-expected number of false alarms. Gold- process is declared not to be in a state of control. smith and Whitefield (1961) revealed this relation be- Since the advent of Shewhart charts, many control tween the nature of autocorrelation and the false alarm charts have been developed to monitor, control, and rate for CUSUM charts. improve processes. Traditional control charts such as There are two primary classes of approaches for con- x-bar charts, CUSUM (cumulative sum) charts, and trol charting in the presence of autocorrelation: Apply- exponentially weighted moving average (EWMA) ing traditional control charts to the original autocorre- charts assume the independence of observations over lated data with the control limits adjusted to account
This research is supported by the Kyung Hee University Research Fund in 2005. (KHU-20051078). Corresponding author : Chang-Ho Chin, School of Mechanical and Industrial Systems Engineering, Kyung Hee University, Yongin-si, Gyeonggi-do 446-701, Republic of Korea. Fax. +82-31-203-4004, E-mail: [email protected] Received March 2007; revision received April 2008; accepted May 2008. Performance and Robustness of Control Charting Methods for Autocorrelated Data 123 for the autocorrelation (Johnson and Bagshaw 1974; (Box, Jenkins, and Reinsel 1994) Vasilopoulos and Stamboulis 1978; Yashchin 1993; Wardell, Moskowitz, and Plante 1994; VanBrackle and Reynolds 1997; Zhang 1998) and fitting a time- series model to the process data and applying control where B is the time-series backward shift operator de- charts to the uncorrelated residuals of the model with fined such that ; is an independently normal control limits (Alwan and Roberts 1988 and identically distributed (i.i.d.) Gaussian process Wardell, Moskowitz, and Plante 1992; Runger, Wille- main, and Prabhu 1995; Lin and Adams 1996; Apley with mean 0 and variance denoted ~ NID(0, ); and Shi 1999). Moreover, many control charting tech- ⋯ and niques in the second category are designed to take into ⋯ are the AR account the specific characteristics of the autocorre- and MA polynomials of order p and q, respectively. lation through time series modeling (e.g., Box and for all t for the in-control process and ≠ Ramírez 1992; Luceño 1999; Apley and Shi 1999; for the out-of-control process. We are assuming, with- Chin and Apley 2006; Apley and Chin 2007). out loss of generality, that the in-control mean is zero. In light of the fact that effective methods for control The model residuals (i.e., the one-step-ahead pre- charting autocorrelated processes are of increasing im- diction errors) are generated via the linear filtering op- portance as data-rich environments such as in manu- eration (Apley and Shi 1999). facturing and service industries proliferate, this paper surveys various methods and investigates their relative (1) performance and robustness. Here, robustness is with respect to errors in the fitted time series models that are used to represent the autocorrelation. Many of the papers just cited have noted that lack of robustness to modeling errors is one of the most serious shortcom- where is a filtered version of the ings of control charts for autocorrelated data. Although deterministic mean shift . The residuals are un- for the performance comparison we primarily rely on correlated under the assumption that the fitted model simulation, for the robustness comparison we develop used to generate the residuals is a perfect representa- analytical results that provide insight into why some tion of reality. Because this is never the case in prac- charts are robust but others are not. tice, a later section of this paper is devoted to quantify- The format of the remainder of the paper is as ing the effect of modeling errors on the performance follows. Section 2 presents a survey of control chart- of the charts. For the time being, however, we assume ing techniques for autocorrelated data. In Section 3, that there are no modeling errors. the performances of such methods are compared for a variety of autocorrelated processes that can be repre- 2.1 Conventional Methods Modified for Auto- sented as autoregressive moving average (ARMA) correlated Data time series models. We derive some analytical robust- ness results in Section 4 and verify these with simu- In this section, we review Shewhart, CUSUM (cum- lation in Section 4. Section 5 concludes the paper. ulative sum), and EWMA charts applied either to the autocorrelated data with control limits modified to take into account the autocorrelation or to the residuals et. Vasilopoulos and Stamboulis (1978) proposed modi- 2. Survey of Control Charting Methods fied control limits for an x-bar chart and an s chart for for Autocorrelated Data autocorrelated data xt that follow a second-order autor- egressive [AR(2)] model with a constant mean shift
Through this paper, the process data x (t is a time in- t dexorobservationnumber)isassumedtofollowan ARMA process model, plus (potentially) an additive deterministic mean shift, , the form of which is The control limits on are given by 124 Chang-Ho Chin Daniel W. Apley
± (2) tistic is the EWMA statistic introduced by Roberts (1959) where n is the subgroup size and is a correction factor that widens/narrows the control limits ,(3) to take into account the autocorrelation (see the Appendix of Vasilopoulos and Stamboulis (1978) for where z0 =0,and isaconstant(≺≤). The specific values). The constant L is chosen to provide a difference between the EWMAST and the conven- desired in-control average run length (ARL) or a false tional EWMA chart is that the variance of EWMA sta- alarm rate. In the case of no serial correlation so that tistic zt, upon which the control limits are based, is cal- and ,the ,andthe culated using the autocorrelation function of xt (denoted by ρ(k)forlagk =1,2,3,… ): control limits reduce to the traditional ±Lσ control limits. For example, control limits on uncorrelated da- (4) ta give an in-control ARL of 370. As an example of auto- correlated data, suppose that and . In this case, the adjusted control limits for sub- groups of size n =5are± while the traditional This reduces to the conventional EWMA variance control limits ± are much narrower and (see Hunter, 1986), when no autocorrelation exists in would result in many false alarms. thedata(i.e.,ρ(k) = 0). As for the aforementioned Johnson and Bagshaw (1974) established a theoretical Shewhart and CUSUM charts, the control limits for basis for obtaining approximate thresholds h of one- the EWMAST chart are also established based on the sided CUSUM charts to provide desired performances variance of chart statistic, taking into account the auto- for autocorrelated data. They considered the one-sided correlation. The objective is to provide a desired false alarm rate of in-control ARL. CUSUM chart proposed by Page (1955) with a test Traditional control charts also can be applied di- statistic C =max[0,x － K + C ] for observations x , i i i-1 i rectly to the model residuals in Equation (1) without where E(x )=0,Var(x )= <,and∞ K is a reference i i any modification of the control limits, as long as the value. For autocorrelated AR(1) and MA(1) processes, process model is accurate, in which case the residuals 2 the run length distribution and its ARL (≈ h / )were are uncorrelated. One common example of residual- approximated by establishing the convergence of the based control charts is an EWMA control chart on the normalized partial sums to a Wiener process and using residuals, which is of the form the usual Wiener process approximation. The threshold with control limits ± σ ,where h were defined with the estimate of the standard equals the steady-state (large t) version of in deviation and a threshold hI which is based on the Equation (4) when there is no autocorrelation. This assumption of zero correlation and unit variance for control charting scheme provides the same ARL as an the observations. If the estimate of the standard devia- EWMA control chart on independent observations tion, is used for AR(1) autocorrelated with control limits ± . Another ver- data x1, x2, x3,,… xn, the threshold h becomes sion of residual-based control charting scheme pro- posed by Jiang et al. (2002) is monitoring residuals and the corresponding ARL obtained by subtracting the proportional integral de- ≐ . Johnson and rivative (PID) predictor from the original process da- ta: . Residual-based control charts have Bagshaw (1974) derived some approximate formula for been broadly investigated (Berthouex, Hunter, and modifying (usually widening) the control limits to take Pallesen 1978, Alwan and Roberts 1988, Montgomery into account the autocorrelation. The correction factor and Mastrangelo 1991, Superville and Adams 1994, is a function of the parameters of the ARMA model Wardell, Moskowitz, and Plante 1994, Runger, Wille- used to represent the autocorrelation. main, and Prabhu 1995, Lin and Adams, 1996, Vander Zhang (1998) proposed the EWMAST chart, an EWMA Wiel 1996, Apley and Shi 1999, Lu and Reynolds chart for stationary processes, in which the charted sta- 1999a, English, Lee, Martin, and Tilmon 2000). Performance and Robustness of Control Charting Methods for Autocorrelated Data 125
2.2 Methods Developed for Autocorrelated the CUSCORE (Fisher 1925, Bagshaw and Johnson Data 1977, Box and Ramírez 1992, Box and Luceño 1997, Luceño 1999), the GLRT (Apley and Shi 1999), the Equation (1) implies that et is composed of random optimal general linear filter (OGLF, Apley and Chin shock at and the filtered version of the deterministic 2007), and the optimal second-order linear filter mean shift .The component experiences certain dynamics that depend on the ARMA process model, (OSLF, Chin and Apley 2006). after which it settles down to a steady-state value if the One of the various CUSCORE Charts based on the efficient score statistics of Fisher (1925) was proposed ARMA model is stable and invertible and is a step mean shift in the original process. To illustrate how a by Luceño (1999) and analyzed by Shu, Apley, and step mean shift in the original process can result in a Tsung (2002), Runger and Testik (2003), and Luceño time-varying mean shift in the residuals, consider the (2004). The upper one-sided CUSCORE is calculated mechanical vibratory system of Pandit and Wu (1983), recursively via an ARMA(2,1) model for which is given by (Jiang et S = max[{S + (e − μ~ / 2)μ~ };0] al. 2000), t t−1 t t t (5)
and sounds an alarm when St exceeds a pre-specified threshold. The GLRT statistic of Apley and Shi (1999) based on a likelihood ratio test also uses a feared re- where . For a step mean shift defined as sidual mean shift as the amplifier for t > 0 and 0, otherwise, the residual mean oscillates about zero as shown in
-3 which do not make use of the valuable information in
-4 the dynamics. We can view the Equations (5) and (6) for generat- 0 5 10 t ing the charted statistics as multiplying the residuals Figure 1. Residual Mean by a sequence of detector coefficients that are pre- cisely the elements of . The conceptual effect of this In order to improve the charting performance in the is to amplify the presence of in the residuals fol- face of forecast recovery, a number of residual-based lowing a mean shift. Note that is sometimes called control charts have been proposed that specifically the feared signal, and the CUSCORE and GLRT are look for the presence of the dynamics or “patterns” viewed as matched filters. The OGLF and OSLF have represented by in the residuals. Such charts include a similar intent, except that their coefficients are only 126 Chang-Ho Chin Daniel W. Apley
µt
x e at Θ(B) t Φ(B) t yt Prefilter : Linear Filter : H(B) Φ(B) ⊕ Θ(B)
Figure 2. Block Diagram Representation of a Control Chart Statistic Generated via a Linear Filtering peration.
based on and do not necessarily coincide with the stricted to the class of all second-order linear filters, which considerably simplifies its design and imple- elements of . The OGLF was developed recognizing that many mentation. Specifically, the OSLF charted statistic is common control charts can be viewed as charting the of the form output of a linear filter applied to process data (Apley and Chin 2007). The statistics of many control charts ,(8) such as the Shewhart and the EWMA charts applied to xt can be represented in the form of , where , , ,andγ are the OSLF design parame- where ⋯ is some linear ters to be determined and the control limits are ±1 due filter in impulse response form and {hj : j =0,1,2,… } are the impulse response coefficients of the filter (see to the scaling constant γ. The design procedure of the Box, Jenkins, and Reinsel 1994 for basic background OSLF uses the same ARL1 optimization criterion and ARL0 constraint as the OGLF. For certain autocorre- on linear filtering). For the EWMA in Equation (3), zt -1 lated processes with prominent residual mean dynam- = H(B)xt =(λ (1－－ (1 λ)B) )xt =(λ + λ(1－λ) B + 2 2 j ics, the OGLF and OSLF coefficients tends to mimic λ(1－ λ) B +)… xt and hj = λ(1－λ) .The the shape characteristic of the residual mean. In many Shewhart individual chart with statistic has the identity filter, H(B) = 1 (see Apley and Chin (2007) examples (Chin and Apley 2006), the OSLF performs for more examples). This generalization also applies to substantially better than an optimized EWMA and al- residual- based charts, where one can view the residual most as good as the OGLF. generation as a linear pre-filter applied to xt, and then the filter is applied to .
Table 1. Zero-state ARL Comparison for the OGLF, the OSLF, the CUSCORE, and the GLRT Tim Series Shift OGLF OSLF OPtimal CUSCORE GLRT Model EWMA
No. Type Size ARL0zs ARL1zs ARL0zs ARL1zs ARL0zs ARL1zs ARL0zs ARL1zs ARL0zs ARL1zs 1 0 0 Step 0.5 499.24 28.38 499.24 28.38 499.24 28.38 499.63 31.27 500.60 48.34 (3.05) (0.10) (3.05) (0.10) (3.05) (0.10) (3.18) (0.11) (3.11) (0.26) 2 0 0 Step 1.5 500.81 5.45 500.81 5.45 500.81 5.45 500.74 5.45 500.12 5.66 (3.15) (0.02) (3.15) (0.02) (3.15) (0.02) (3.17) (0.02) (3.14) (0.02) 3 0 0 Step 3 500.02 1.87 500.02 1.87 500.02 1.87 500.77 1.79 500.83 1.92 (3.14) (0.01) (3.14) (0.01) (3.14) (0.01) (3.16) (0.01) (3.18) (0.01) 4 0 0 Step 4 500.43 1.21 500.43 1.21 500.43 1.21 499.28 1.20 500.78 1.31 (3.16) (0.00) (3.16) (0.00) (3.16) (0.00) (3.15) (0.00) (3.13) (0.00) 5 0.9 0 Step 0.5 500.16 353.79 500.16 353.79 500.16 353.79 500.46 375.76 500.36 473.82 (2.71) (1.79) (2.71) (1.79) (2.71) (1.79) (2.62) (1.88) (3.14) (2.96) 6 0.9 0 Step 1.5 500.37 129.60 500.37 129.60 500.37 129.60 499.01 138.02 500.48 324.75 (2.86) (0.57) (2.86) (0.57) (2.86) (0.57) (3.01) (0.78) (3.13) (2.16) 7 0.9 0 Step 3 499.24 47.32 500.75 48.00 500.76 49.73 499.81 30.99 500.64 78.77 (3.03) (0.31) (3.07) (0.32) (3.00) (0.22) (3.11) (0.28) (3.15) (0.85) 8 0.9 0 Step 4 499.96 14.01 500.63 14.00 500.77 29.44 499.17 8.04 500.64 14.92 (3.11) (0.18) (3.10) (0.18) (3.06) (0.14) (3.08) (0.13) (3.15) (0.29) 9 0.9 0 Spike 0.5 499.14 490.17 500.43 493.43 500.55 500.13 500.18 491.94 (3.15) (3.12) (3.14) (3.11) (3.17) (3.15) (3.18) (3.11) 10 0.9 0 Spike 1.5 499.29 418.03 499.20 424.69 499.34 454.13 499.95 433.78 (3.14) (3.11) (3.12) (3.13) (3.15) (3.14) (3.15) (3.14) 11 0.9 0 Spike 3 499.98 84.12 500.13 88.45 499.01 176.54 499.23 97.98 (3.17) (1.76) (3.14) (1.81) (3.15) (2.38) (3.12) (1.87) 12 0.9 0 Spike 4 500.66 6.25 499.62 7.00 500.76 27.85 500.70 8.41 (3.13) (0.43) (3.19) (0.45) (3.14) (0.98) (3.14) (0.54)
13 0 0 Sinusoid S1 499.22 15.78 499.22 15.78 500.56 122.56 499.62 15.82 499.33 19.64 (3.09) (0.05) (3.09) (0.05) (3.14) (1.31) (3.15) (0.06) (3.14) (0.09)
14 0 0 Sinusoid S2 499.51 30.74 499.51 30.74 500.63 225.55 500.12 31.04 500.15 46.59 (3.08) (0.11) (3.08) (0.11) (3.18) (2.13) (3.10) (0.13) (3.14) (0.29)
15 0 0 Sinusoid S3 499.12 33.34 500.17 43.47 499.44 177.91 499.50 33.86 500.84 48.45 (3.07) (0.14) (3.13) (0.25) (3.17) (1.78) (3.05) (0.16) (3.10) (0.31)
16 0 0 Sinusoid S4 500.91 10.64 499.37 11.40 499.45 26.23 499.92 10.58 499.80 10.54 (3.10) (0.04) (3.11) (0.04) (3.16) (0.16) (3.13) (0.04) (3.16) (0.04) 17 0.9 -0.9 Step 0.5 499.80 446.77 499.80 446.77 499.80 446.77 499.61 343.69 499.14 475.79 (2.72) (2.37) (2.72) (2.37) (2.72) (2.37) (4.80) (3.70) (3.14) (3.10) 18 0.9 -0.9 Step 1.5 499.63 142.06 500.91 162.26 499.74 257.46 500.74 86.56 499.37 193.75 (2.93) (1.73) (3.10) (2.21) (2.73) (1.24) (3.97) (1.43) (3.13) (2.38) 19 0.9 -0.9 Step 2 500.86 41.78 500.03 42.21 500.93 194.06 500.01 29.02 499.19 55.60 (3.09) (1.12) (3.06) (1.14) (3.16) (0.91) (3.39) (0.77) (3.12) (1.36) 20 0.9 -0.9 Step 3 499.05 3.16 500.37 3.33 499.27 74.70 500.01 3.33 499.61 2.43 (3.09) (0.08) (3.17) (0.13) (3.14) (1.53) (3.14) (0.19) (3.15) (0.08) 21 0.9 0.5 Step 0.5 499.02 206.73 500.13 206.91 500.13 206.91 500.25 235.32 500.13 398.64 (2.73) (0.97) (2.81) (0.96) (2.81) (0.96) (2.75) (1.12) (3.09) (2.52) 22 0.9 0.5 Step 1.5 499.72 50.82 499.72 50.82 499.72 50.82 499.15 54.34 500.95 113.86 (3.00) (0.22) (3.00) (0.22) (3.00) (0.22) (3.07) (0.29) (3.16) (0.79) 23 0.9 0.5 Step 3 500.08 10.73 500.08 10.73 499.27 10.69 500.45 6.88 500.11 7.96 (3.12) (0.08) (3.12) (0.08) (3.11) (0.08) (3.11) (0.07) (3.15) (0.09) 24 0.9 0.5 Step 4 500.62 2.77 500.40 2.85 500.36 2.83 499.21 1.85 499.56 1.68 (3.15) (0.03) (3.12) (0.04) (3.14) (0.04) (3.16) (0.03) (3.15) (0.01) 25 0.9 0.5 Spike 0.5 499.70 497.13 499.46 497.88 500.83 499.58 499.30 496.52 (3.11) (3.15) (3.13) (3.14) (3.19) (3.17) (3.15) (3.16) 26 0.9 0.5 Spike 1.5 499.55 457.83 500.84 466.37 499.25 467.66 499.40 463.13 (3.10) (3.08) (3.17) (3.14) (3.17) (3.13) (3.15) (3.13) 27 0.9 0.5 Spike 3 500.99 207.74 500.57 258.83 499.10 260.10 499.41 213.92 (3.13) (2.52) (3.15) (2.75) (3.15) (2.78) (3.12) (2.56) 28 0.9 0.5 Spike 4 500.47 50.40 499.38 83.39 500.16 84.60 500.15 54.04 (3.16) (1.30) (3.17) (1.75) (3.19) (1.76) (3.14) (1.45) 29 0 0 Ramp 0.5 499.18 33.28 499.18 33.28 499.83 33.33 500.09 35.94 499.88 59.15 (3.08) (0.11) (3.08) (0.11) (3.07) (0.11) (3.08) (0.12) (3.14) (0.31) 30 0 0 Ramp 1.5 500.09 10.50 500.09 10.50 500.04 10.51 499.56 10.96 500.61 11.17 (3.12) (0.02) (3.12) (0.02) (3.14) (0.02) (3.11) (0.02) (3.11) (0.02) 31 0 0 Ramp 3 500.82 6.60 500.82 6.60 499.78 6.60 500.94 6.82 499.50 6.88 (3.13) (0.01) (3.13) (0.01) (3.17) (0.01) (3.16) (0.01) (3.14) (0.01) 32 0 0 Ramp 4 499.16 5.49 499.16 5.49 500.84 5.50 500.54 5.64 500.12 5.74 (3.14) (0.01) (3.14) (0.01) (3.13) (0.01) (3.17) (0.01) (3.15) (0.01) 128 Chang-Ho Chin Daniel W. Apley pends strongly on the characteristics of the residual amplitude 0.75 and periods of 2, 4, and 8 timesteps, re- mean, which depend on the characteristics of the proc- spectively. S4 has amplitude 1.5 and period 8 time- ess as represented by the ARMA model and the form steps. The ramp mean shift is defined as for t < and magnitude of the mean shift. Thus, we consider a 1, for 1 ≤ t <10and for t ≥ 10. broad combination of scenarios in the 32 examples 0 15 30 4 0 (a) Ex. 4 -4 4 0 (b) Ex. 8 -4 4 0 (c) Ex. 12 -4 4 0 (d) Ex. 16 -4 μ t 4 0 (e) Ex. 20 -4 Residual mean Residual 4 0 (f) Ex. 24 -4 4 0 (g) Ex. 28 -4 4 0 (h) Ex. 32 -4 0 15 30 t Figure 3. Illustration of residual mean shifts for ARMA(1, 1) processes : (a) i.i.d. process ( = =0)withastep mean shift of size 4; (b) AR(1) process ( = 0.9) with a step mean shift of size 4; (c) AR(1) process ( = 0.9) with a spike mean shift of size 4 (d) i.i.d. process ( = = 0) with a sinusoidal mean shift of period 8 and amplitude 1.5; (e) ARMA(1,1) process ( =0.9, = -0.9) with a step mean shift of size 3; (f) ARMA(1,1) process ( =0.9, = 0.5) with a step mean shift of size 4; (g) ARMA(1,1) process ( = 0.9, = 0.5) with a spike mean shift of size 4; (h) i.i.d. process ( = = 0) with a ramp mean shift of size 4. Performance and Robustness of Control Charting Methods for Autocorrelated Data 129 forms the CUSCORE or performs quite comparably to en proportional to the feared signal (e.g., )as it (e.g., the common, practical scenarios of Examples shown in Equation (5). For the GLRT chart, we used 1~ 4 and 29~30, in which there is a sustained step and the same window length N (= 20) as recommended in a drifting ramp shift in i.i.d. data). The PID chart is not Apley and Shi (1999). Lucen˜o (1999) proposed the two included into the comparison. We had compared the versions of the CUSCORE chart, depending on whether PID chart to the OGLF and OSLF charts in Chin and or not the in Equation (5) is reinitialized whenever Apley (2006) and Apley and Chin (2007) and found the statistic reaches its zero limit. The CUSCORE with- that when one used the design guidelines suggested by out reinitialization is excluded from comparison be- Jiang et al. (2002), the performance of the PID chart cause of its significant ineffectiveness in the steady was not competitive with the other charts. state (Runger and Testik 2003; Chin and Apley 2006; We calculated the zero-state ARL for each example Apley and Chin 2007). based on Monte Carlo simulation with 25,000 repli- cates. The zero-state ARL refers to the ARL of which 3.2 Performance Comparison based on the the evaluation starts with the initial observation. Because zero-state ARL the EWMA, OGLF, and OSLF charts are inherently two-sided and the absolute value in Equation (6) makes The EWMA and OSLF charts are special cases of the GLRT chart two-sided, the two-sided versions of the OGLF and thus, cannot perform better than the the CUSCORE chart is considered for comparison. The OGLF. However, the EWMA is included to represent lower-sided CUSCORE statistic would be constructed control charts which do not take advantage of the in- substituting - for in Equation (5), and the two-sid- formation on the residual mean dynamics and show ed version consists of the two one-sided versions the performance difference from control charts devel- together. The CUSUM chart is not included in this oped for a time-varying mean shift. As expected, the comparison because the EWMA chart performance is OGLF, CUSCORE, and GLRT charts outperform the virtually identical (Vander Wiel 1996, Yang and Makis EWMA chart by a wide margin, except for the i.i.d. 1997, Montgomery 2005). The residual- based EWMA processes with a step mean shift and other processes chart for comparison is defined as with a mean shift of size 0.5 which do not have prom- inent residual mean dynamics. (9) For the 32 examples under analysis, Time-reversed Residual Mean (•) OGLF () 3 0 -3 -8 -4 0 4 8 12 16 20 24 28 32 t Figure 4. The OGLF for Example 20 applied to the Residual Mean Three Timesteps after the Occurrence of the Shift mized EWMAs. For the others, they perform com- which are reminiscent of the matched filter of the cor- parably. As a simplified version of the OGLF, the responding GLRT chart, as illustrated in the following. OSLF combines the design and implementation effi- For processes with pronounced dynamic patterns of re- ciency with performance that is substantially better sidual mean shifts as in Examples 9~16 and 18~20, the than an optimized EWMA and as good as the OGLF matched filters of the OGLF, OSLF, CUSCORE, and in a number of examples. GLRT charts are used to increase the detection proba- Note that the CUSOCRE charts cannot be set up for bilities by fostering the correlation with the residual Examples 9~12 and 25~28. Chin (2008) showed that means. The chart statistics in Equations (5), (6), (7), the CUSCORE chart statistic of Luceño (1999) might and (8) include the summation of the product of the re- totally lose the detection capability. It was illustrated siduals and the matched filter coefficients. The sum- with Examples 9~12 and 25~28. Examples 9~12 have mation can be viewed as a measure of the correlation spike mean shifts which have only two non-zero co- between the residuals and the matched filter. Hence, efficients at the first two timesteps and the remaining the higher the correlation between two signals, the zero coefficients as shown in 1999b; Reynolds and Stoumbos 2001), the only differ- popular in SPC applications, but have also suffered the ence being that the latter simultaneously runs two sepa- criticism of lacking robustness to inevitable errors in rate chart statistics, while the former combines them in- fitting an ARMA model to process data. Since the con- to one statistic. Combined Shewhart-EWMA schemes trol charts are designed based on the assumption of no arewidelyknowntoworkwellforprocesseswherethe modeling errors, any modeling error affects the control residual mean has a pronounced initial spike and then chart performance represented by the in-control ARL, converges to a small steady-state value. The Shewhart the out-of-control ARL, the false alarm rate, and so component of combined Shewhart-EWMA schemes is effective at detecting the initial spike and its EWMA forth. Hence, the robustness to modeling errors is a component is effective at detecting the lasting small very critical element of control charts required to en- steady state shift, which makes them outperform opti- sure they perform well in practice. mized EWMA charts. Suppose that the true parameter differs from the 4. Robustness Analysis estimate . The residuals generated by Equation (1) are not independent, but actually follows the ARMA ARMA model-based approaches have been very (1,1) model 0.3 0.3 0.2 0.2 0.1 0.1 hj hj 0 0 -0.1 -0.1 0 25 50 tt0 25 50 Figure 5. Impulse response coefficients of the OGLF charts : (a) Example 8 and (b) Example 28 132 Chang-Ho Chin Daniel W. Apley shown in Equations (11) and (12), instead, the stand- (10) ardized variance changes with respect to modeling er- rors are proposed as the indirect measures for their corresponding ARL changes of control charts. Using Equations (11) and (12), we derive the fol- lowing sensitivity measures for the GLRT, OGLF, and OSLF charts applied to . See Appendix I for the de- tailed derivation. These measures reduce to relatively simple expressions for the ARMA(1, 1) processes con- where the “^” symbol denotes an estimate of a quan- sidered in Section 3. For each (t)oftheGLRTap- tity. With underestimated, the residual autocorrela- plied to the residuals defined in Equation (1), tion would be positive and the resulting standard devi- ation of would be substantially larger than that un- ⎛ ξ − k −1 ⎞ ⎜ μ~ μ~ ⎟ der the assumption of no modeling error. The in- ∞ ξ − 2 ∑ j j + k +1 k ⎜ j =1 ⎟ S (φ ,ξ ) = 2 P ρ =2 φ creased variance inflates the false alarm rate, which e ,GLRT 1 ∑ k 1+ k ∑ 1 ⎜ ξ ⎟ k = 0 k = 0 ~ 2 ⎜ ∑ μ ⎟ leads to the decrease in the in-control ARL. That is, ⎝ j =1 ⎠ the effect of modeling errors on the ARL corresponds closely to the effect of modeling errors on the variance ξ −k−1 ⎛ μ~ μ~ ⎞ ∞ ξ −2 ⎜ ∑ j j+k+1 ⎟ of the control chart statistic. Apley and Lee (2008) de- k ⎜ j=1 ⎟ Se,GLRT (θ1,ξ) = 2∑Qk ρ1+k = −2 ∑θ1 ξ fined the sensitivity as the partial derivative of the var- k=0 k=0 ⎜ ~2 ⎟ ⎜ ∑ μ ⎟ . iance of the control chart statistic (which we generi- ⎝ j=1 ⎠ cally denote by ) with respect to the ARMA parame- ters, scaled by the nominal variance: For the OGLF defined in Equation (7), Tr−2 Tr−k−1 2 2 k ⎛ ⎞ ∂σ y S (φ ) = φ h h e,OGLF 1 Tr ∑ 1 ⎜ ∑ j j+k+1 ⎟ h2 k=0 ⎝ j=0 ⎠ ∂φi ∑ j S()φ = ω=ωˆ j=0 i 2 : i =1,2,… ,p, and (11) σˆ y Tr−2 Tr−k−1 2 k ⎛ ⎞ Se,OGLF (θ1 ) = − Tr ∑ θ1 ⎜ ∑ h j h j+k+1 ⎟ 2 k=0 ⎝ j=0 ⎠ 2 ∑ h j ∂σ j=0 . y ∂θi S()θ = ω=ωˆ i 2 : i =1,2,… ,q, (12) For the OSLF defined in Equation (8), σˆ y 2 ⎛ a1λ1 + a2λ2 ⎞ where ⋯ ⋯ is the vector of S (φ ) = ⎜α φ + ⎟ e,OSLF 1 1−α φ −α φ 2 ⎜ 2 1 a + a ⎟ 1 1 2 1 ⎝ 1 2 ⎠ ARMA parameters and is the variance of the control 2 ⎛ a λ + a λ ⎞ chart statistic when . When multiplied by a pa- S (θ ) = − ⎜α θ + 1 1 2 2 ⎟ e,OSLF 1 1−α θ −α θ 2 ⎜ 2 1 a + a ⎟ rameter error (denoted by ∆ or ∆), Equations 1 1 2 1 ⎝ 1 2 ⎠ (11) and (12) represent the percentage change in the where variance, due to modeling errors. Hence, the sensi- tivity measures can be viewed as the analytical per- (λ1 − β )(1− βλ1 ) a1 = − 2 , centage variance changes (PVC) for modeling errors, (1− λ1 ) which are compared with the empirical PVCs in (λ2 − β )(1− βλ2 ) a2 = 2 , Table 2. Sensitivities for the OGLF, the OSLF, the optimal EWMA, and the GLRT when the one of the true parame- ters of an ARMA(1,1) process is incorrectly estimated ( =0.89vs. =0.9and = -0.89 vs. =-0.9) Time Series Optimal Model Shift OGLF OSLF EWMA GLRT No. Type Size Se( ) Se( ) Se( ) Se( ) Se( ) Se( ) Se( ) Se( ) 1 0 0 Step 0.5 1.906 -1.906 1.906 -1.906 1.906 -1.906 1.800 -1.800 200Step1.51.516 -1.516 1.516 -1.516 1.516 -1.516 1.800 -1.800 300Step30.648 -0.648 0.648 -0.648 0.648 -0.648 1.800 -1.800 400Step40.226 -0.226 0.226 -0.226 0.226 -0.226 1.800 -1.800 5 0.9 0 Step 0.5 19.607 -1.996 19.607 -1.996 19.607 -1.996 1.651 -0.330 6 0.9 0 Step 1.5 18.683 -1.986 18.683 -1.986 18.683 -1.986 1.651 -0.330 7 0.9 0 Step 3 6.914 -0.782 6.717 -0.742 16.468 -1.958 1.651 -0.330 8 0.9 0 Step 4 3.958 -0.360 3.960 -0.360 14.337 -1.924 1.651 -0.330 9 0.9 0 Spike 0.5 -1.014 1.197 -0.999 1.107 00-0.994 0.994 10 0.9 0 Spike 1.5 -1.006 1.069 -0.999 1.099 00-0.994 0.994 11 0.9 0 Spike 3 -1.003 1.044 -0.999 1.093 00-0.994 0.994 12 0.9 0 Spike 4 -1.001 1.033 -0.999 1.096 00-0.994 0.994 13 0 0 Sinusoid S1 -1.815 1.815 -1.815 1.815 00-1.800 1.800 14 0 0 Sinusoid S2 0.018 -0.018 0.018 -0.018 00 00 15 0 0 Sinusoid S3 1.430 -1.430 1.443 -1.443 0.784 -0.784 1.286 -1.286 16 0 0 Sinusoid S4 1.217 -1.217 1.387 -1.387 0.768 -0.768 1.286 -1.286 17 0.9 -0.9 Step 0.5 19.607 -1.052 19.607 -1.052 19.607 -1.052 -0.887 6.298 18 0.9 -0.9 Step 1.5 1.683 11.641 -1.006 11.408 19.416 -1.051 -0.887 6.298 19 0.9 -0.9 Step 2 -1.006 11.410 -1.006 11.408 19.228 -1.050 -0.887 6.298 20 0.9 -0.9 Step 3 -0.720 6.715 -0.922 5.745 00-0.8876.298 21 0.9 0.5 Step 0.5 19.200 -3.967 19.228 -3.969 19.228 -3.969 4.506 -2.171 22 0.9 0.5 Step 1.5 16.468 -3.836 16.468 -3.836 16.468 -3.836 4.506 -2.171 23 0.9 0.5 Step 3 8.460 -3.143 8.469 -3.154 8.462 -3.143 4.506 -2.171 24 0.9 0.5 Step 4 3.921 -1.609 3.726 -2.135 3.726 -2.135 4.506 -2.171 25 0.9 0.5 Spike 0.5 -0.876 0.654 -0.334 0.361 00-0.879 0.645 26 0.9 0.5 Spike 1.5 -0.876 0.654 -0.105 0.110 00-0.879 0.645 27 0.9 0.5 Spike 3 -0.876 0.654 -0.068 0.069 00-0.879 0.645 28 0.9 0.5 Spike 4 -0.876 0.654 -0.129 0.137 00-0.879 0.645 29 0 0 Ramp 0.5 1.954 -1.954 1.954 -1.954 1.992 -1.992 1.714 -1.714 30 0 0 Ramp 1.5 1.790 -1.790 1.790 -1.790 1.954 -1.954 1.714 -1.714 31 0 0 Ramp 3 1.511 -1.511 1.511 -1.511 1.838 -1.838 1.714 -1.714 32 0 0 Ramp 4 1.345 -1.345 1.345 -1.345 1.734 -1.734 1.714 -1.714 2(1 − λ ) Se,Shewhart (φ1 ) = Se,Shewhart (θ1 ) = 0 S e ,EWMA (φ1 ) = . 1 − φ1 (1 − λ ) 2(1− λ ) 21~22. While the moving window length of the GLRT rameters are =0.89and = -0.9 in one situation chart is fixed (i.e., N = 20), those of the OGLF, OSLF, and =0.9and = -0.89 in the second situation. and EWMA chart are optimally determined according For these errors in estimating and , respectively, to the magnitude of the feared signal through the de- Equation (10) implies that the residuals with no mean sign procedures. For the same type of feared signal, shift obey the models Performance and Robustness of Control Charting Methods for Autocorrelated Data 135 (1− 0.9B) (1+ 0.89B) tual PVC, the first-order approximation of the PVC e = a e = a t (1− 0.89B) t and t (1+ 0.9B) t , based on sensitivity measures, and corresponding in- control ARLs for the incorrectly estimated parameters respectively. The Monte Carlo simulations with are shown in Table 3. Comparison of the Analytical Results of Sensitivities with Empirical Results for Examples 17~20 Estimated True Optimized parameter Shift parameter OGLF OSLF EWMA GLRT Type Size PVC() PVC() PVC() PVC() ARL0, ARL0, ARL0, ARL0, (std. err.) (std. err.) (std. err.) (std. err.) Se()Δ Se()Δ Se()Δ Se()Δ PVC() ARL PVC() ARL PVC() ARL PVC() ARL No. 0, 0, 0, 0, (std. err.) (std. err.) (std. err.) (std. err.) Se()Δ Se()Δ Se()Δ Se()Δ 17 .9 -.9 Step .5 .89 -.9 -17.043 611.73 -17.036 611.48 -17.036 611.89 0.902 482.03 -19.607 (1.67) -19.607 (1.67) -19.607 (1.67) 0.887 (1.51) .9 -.89 -1.050 511.06 -1.050 510.78 -1.050 510.34 6.487 434.67 -1.052 (1.39) -1.052 (1.39) -1.052 (1.38) 6.298 (1.37) .87 -.9 -40.198 901.34 -40.201 898.77 -40.201 906.22 2.793 448.08 -58.821 (2.50) -58.821 (2.50) -58.821 (2.52) 2.661 (1.40) .9 -.87 -3.135 522.56 -3.122 521.35 -3.122 522.20 20.555 323.31 -3.156 (1.42) -3.156 (1.41) -3.156 (1.42) 18.894 (1.01) 18 .9 -.9 Step 1.5 .89 -.9 -1.452 531.18 1.033 484.28 -16.895 621.38 0.902 480.71 -1.683 (1.58) 1.006 (1.49) -19.416 (1.72) 0.887 (1.50) .9 -.89 12.211 382.26 12.103 373.76 -1.062 509.25 6.487 436.40 11.641 (1.10) 11.408 (1.15) -1.051 (1.40) 6.298 (1.38) .87 -.9 -2.904 570.44 3.173 453.30 -39.892 953.12 2.793 455.10 -5.049 (1.69) 3.018 (1.40) -58.248 (2.69) 2.661 (1.43) .9 -.87 40.330 248.08 39.705 230.39 -3.128 522.19 20.555 323.86 34.923 (0.68) 34.224 (0.69) -3.153 (1.44) 18.894 (1.01) 19 .9 -.9 Step 2 .89 -.9 1.036 476.16 1.033 483.16 -16.737 629.58 0.902 480.58 1.006 (1.47) 1.006 (1..49) -19.228 (1.77) 0.887 (1.51) .9 -.89 11.991 368.10 12.103 371.35 -1.053 510.63 6.487 433.45 11.410 (1.12) 11.408 (1.14) -1.050 (1.42) 6.298 (1.36) .87 -.9 3.109 446.96 3.173 451.21 -39.607 1011.6 2.793 450.01 3.018 (1.38) 3.018 (1.39) -57.684 (2.93) 2.661 (1.42) .9 -.87 39.600 228.89 39.705 230.94 -3.137 527.98 20.555 323.57 34.230 (0.69) 34.224 (0.70) -3.150 (1.48) 18.894 (1.00) 20 .9 -.9 Step 3 .89 -.9 0.783 487.67 0.965 480.10 0.096 501.69 0.902 479.44 0.720 (1.53) 0.922 (1.50) 0.000 (1.59) 0.887 (1.50) .9 -.89 7.137 396.28 6.140 402.12 0.096 499.33 6.487 434.46 6.715 (1.24) 5.745 (1.26) 0.000 (1.57) 6.298 (1.35) .87 -.9 2.437 460.42 2.895 445.65 0.382 493.23 2.793 446.10 2.160 (1.43) 2.766 (1.40) 0.000 (1.56) 2.661 (1.40) .9 -.87 23.499 261.02 20.263 272.64 0.478 488.59 20.555 321.63 20.145 (0.80) 17.235 (0.85) 0.000 (1.53) 18.894 (1.00) 136 Chang-Ho Chin Daniel W. Apley changes in ARLs implies that the sensitivity measures they may actually have a substantial effect on the are appropriate as the indirect measures for the ARL performance. For example, the first row of With underestimated, the actual variance for the for the EWMA chart with underestimated is also OSLF chart is 0.1519 which is 12.10% larger than the consistent with the analytical result -3.15%. For the assumed one. This PVC 12.10% is also consistent with EWMA chart with overestimated, the difference the analytical result that the approximate percentage between the analytical approximation and actual PVC variance increase is 11.41%, decreasing the in-control gets to be larger from 14.9% (for the parameter error ARL from 500 to 374. For and of the EWMA of +0.01) to 46.0% (for the parameter error of +0.03). chart, the analytical percentage variance decreases are Similar augmentations are found in other examples. 19.42% and 1.06%, which agree reasonably well with For all of Examples 17~20 under analysis, however, the actual percentage variance decreases 16.90% and this inevitable augmentation does not have any influ- 1.05%, respectively. These increases (decreases) in the ence on selecting the most robust chart and does not variance of the chart statistic result in higher (lower) blur distinct difference between sensitivities of control false alarm rates, which decrease (increase) the in-con- charts. trol ARL. The in-control ARLs for the OSLF chart de- crease by 3.14% and 25.25% for the true and . Those for the EWMA chart increase by 24.27% and 5. Conclusions 1.85%, respectively. The difference between the changes of the variance and performance can be explained in that performance change of a control charting scheme In this paper, we have surveyed control charting is affected by its autocorrelation level as well as its var- schemes for autocorrelated data, with a focus on per- iance with the control limits fixed (Jiang and Tsui formance and robustness of residual-based control 2001). However, the sensitivities provide fairly reason- charts. In the ARL comparison using Monte Carlo able comparison for the robustness to modeling error. simulations, the OGLF, OSLF, CUSOCRE, and In addition, the sign of the sensitivity indicates whether GLRT charts substantially outperform the optimized the variance and ARL will increase or decrease for a EWMA chart which does not take advantage of the particular type of modeling error. The negative sign of information hidden in the dynamic characteristics of the Se,OSLF( ) for Example 18 imply that the under- the residual mean. For the i.i.d. processes with a step estimation of results in an increase in variance and a mean shift, for which there are no prominent dynam- decrease in ARL. ics in the residuals, the first three control charts per- While the ±0.01 parameter errors may seem small, form comparably to the EWMA chart. Generally, the Performance and Robustness of Control Charting Methods for Autocorrelated Data 137 OGLF or CUSCORE chart performs the best. The ∞ : i =1,2,… ,p,and (A.1) OGLF chart perform better for processes in which the residual mean settles down to zero or a very small ∞ : i =1,2,… ,q,(A.2) steady-state value after experiencing initial prominent dynamics. The CUSCORE chart is more effective at detecting a residual mean decaying slowly and con- where denotes the autocorrelation function of verging to a relatively significant steady-state value. under the assumption of no modeling errors at lag j One drawback of the CUSCORE chart with re- and { : j =0,1,2,… }and{: j =0,1,2,… }de- initialization is that it may totally lose the ability to note the impulse response coefficients of detect a shift when the feared signal converges to zero ∞ ∞ and ,respec- after few initial spikes. The GLRT performs worse than the EWMA for processes with a short-lived mi- tively. We derive the sensitivities for the GLRT, nor residual mean shift such as Example 5 and 21. OGLF, and OSLF charts applied to the residuals of The robustness to modeling errors is a critical char- ARMA(1, 1) processes with parameter and un- acteristic that should be considered when selecting der assumption that ~ NID(0, 1). and designing a control chart. To investigate the ro- For the GLRT chart defined in Equation (6) (Apley bustness of the control charts under consideration, we and Shi 1999), the variance and covariance functions derived analytical expressions for their sensitivity to for the Generalized Likelihood Ratio (GLR) statistic ARMA modeling errors and demonstrated their val- with a moving window of length ζ are defined as idity by comparing the analytical results with the ac- and tual PVCs and the ARLs calculated using Monte −1 ⎛ ξ ⎞ ξ −k Carlo simulations. The sensitivity of a chart depends ⎜ μ~2 ⎟ μ~ μ~ ⎜∑ j ⎟ ∑ j j+k . primarily on the autocorrelation of the chart statistic ⎝ j=1 ⎠ j=1 and AR and MA polynomials of the original process. The autocorrelation function is obtained as The more slowly decaying the autocorrelation, the ξ −k ~ ~ higher the sensitivities in general. The GLRT is the ∑ μ j μ j+k γ k,ξ j=1 most robust among the control charts developed for ρk,ξ = = ξ . 2 γ 0,r ~ detecting a pre-specified time-varying mean shift. ∑ μ j j=1 No control chart is consistently superior to others in terms of performance and robustness. However, if it is Based on the general expressions of the sensitivity possible to quantify the performance and robustness of measures (Apley and Lee 2008), the sensitivity meas- a control chart based on the information known a pri- ures for ARMA(1,1) processes are approximated by ori about the original process and the nature of the ξ −k−1 mean shift of interest, it would aid in selecting a con- ⎛ μ~ μ~ ⎞ ∞ ξ −2 ⎜ ∑ j j+k+1 ⎟ k ⎜ j=1 ⎟ trol chart more appropriately. The survey and empiri- Se,GLRT (φ1,ξ ) = 2 ∑ Pk ρ1+k = 2 ∑φ1 ξ and k=0 k=0 ⎜ μ~2 ⎟ cal and analytical results in this paper were provided ⎜ ∑ j ⎟ ⎝ j=1 ⎠ for this purpose. ξ −k−1 ⎛ μ~ μ~ ⎞ ∞ ξ −2 ⎜ ∑ j j+k+1 ⎟ k ⎜ j=1 ⎟ Se,GLRT (θ1,ξ ) = 2 ∑Qk ρ1+k = −2 ∑θ1 ξ . k=0 k=0 ⎜ ~2 ⎟ ⎜ ∑ μ j ⎟ For the output of a general linear filter tions of the OGLF chart (Apley and Chin 2007) are appliedtoanARMA(p, q) process , Apley and Lee given by (2008) derived the following expressions for the sensi- Tr 2 tivity measures defined in Equations (11) and (12): γ 0 = Cov(yt , yt ) =Var(yt ) = ∑ h j j=0 , 138 Chang-Ho Chin Daniel W. Apley Tr−k of Model Uncertainty, ASME Journal of Manufacturing ∑ h j h j+k γ k j=0 Science and Engineering, 124(4), 891-898. Tr−k ρ k = = Tr 2 Apley, D. W. and Chin, C. (2007), An Optimal Filter Design γ k = Cov(yt , yt+k ) = ∑ h j h j+k γ 0 h j=0 ,and ∑ j . j=0 Approach to Statistical Process Control, Journal of Quality Technology, 39(2), 93-117. The sensitivity measures for ARMA(1,1) processes Apley, D. W. and Lee, H. C. (2003), Design of Exponentially Weighted Moving Average Control Charts for Autocorrelated are defined as Processes with Model Uncertainty, Technometrics, 45(3), 187-198. Tr−2 Tr−k −1 2 k ⎛ ⎞ Apley, D. W. and Lee, H. C. (2008), Robustness Comparison of Se,OGLF (φ1 ) = Tr ∑ φ1 ⎜ ∑ h j h j+k +1 ⎟ 2 k =0 ⎝ j=0 ⎠ Exponentially Weighted Moving Average Charts on ∑ h j j=0 and Autocorrelated Data and on Residuals, Journal of Quality Technology, to appear. 2 Tr − 2 ⎛ Tr − k −1 ⎞ S (θ ) = − θ k ⎜ h h ⎟ Apley, D. W. and Shi, J. (1999), GLRT for Statistical Process e,OGLF 1 Tr ∑∑1 ⎜ j j + k +1 ⎟ h 2 k = 0 ⎝ j = 0 ⎠ Control of Autocorrelated Processes, IIE Transactions, ∑ j . j = 0 31(12), 1123-1134. Bagshaw, M. and Johnson, R. A. (1975), The Effect of Serial For the OSLF chart defined as Equation (8), the au- Correlation on the Performance of CUSUM Tests II, tocorrelation function is defined as (Pandit and Wu Technometrics, 17(1), 73-80. 1983) Bagshaw, M. and Johnson R. A. (1977), Sequential Procedures for Detecting Parameter Changes in a Time-Series Model, Journal of the American Statistical Association, 72(359), 1 k k 593-597. ρ k = ()a1λ1 + a2 λ2 a1 + a2 Berthouex, P. M., Hunter, E. and Pallesen, L. (1978), Monitor- ing Sewage Treatment Plants: Some Quality Control Aspects, where, Journal of Quality Technology, 10(4), 139-149 Box, G. E. P., Jenkins, G., and Reinsel, G. (1994), Time Series Analysis, Forecasting, and Control, 3rd ed., Prentice-Hall, (λ1 − β )(1− βλ1 ) (λ2 − β )(1 − βλ 2 ) a1 = − a2 = Englewood Cliffs, NJ. (1− λ 2 ) , (1 − λ 2 ) ,and 1 2 Box, G. E. P. and Ramírez, J. (1992), Cumulative Score Charts, 1 2 Quality and Reliability Engineering International, 8(1), λ1 ,λ2 = (α1 ± α1 + 4α 2 ) 2 . 1727. Box, G. E. P. and Luceño, A. (1997), Statistical Control by The sensitivity measures for ARMA(1,1) processes Monitoring and Feedback Adjustment, Wiley, New York, are calculated by NY. Chin, C. (2008), Modified Cuscore Charts for Autocorrelated Processes, to be submitted. 2 ⎛ a λ + a λ ⎞ S (φ ) = ⎜α φ + 1 1 2 2 ⎟ Chin, C. and Apley, D. W. (2006), Optimal Design of Second- e,OSLF 1 2 ⎜ 2 ⎟ and 1 − α1φ − α 2φ ⎝ a1 + a2 ⎠ Order Linear Filters for Control Charting. Technometrics, 48(3) 337-348. 2 ⎛ a λ + a λ ⎞ S (θ ) = − ⎜α θ + 1 1 2 2 ⎟ English,J.R.,Lee,S-c.,Martin,T.W.,andTilmon,C.(2000), e,OSLF 1 2 ⎜ 2 ⎟ . 1− α1θ − α 2θ ⎝ a1 + a2 ⎠ Detecting Changes in Autoregressive Processes with X-bar and EWMA charts, IIE Transactions, 32(12) 1103-1113. Fisher, R. A. (1925), Theory of Statistical Estimation, Proceed- ing of the Cambridge Philosophical Society, 22, 700-725. References Goldsmith, P. L. and Whitefield, H. (1961), Average Run Lengths in Cumulative Chart Quality Control Schemes, Technometrics, 3(1), 11-20. Adams, B. M. and Tseng, I. T. (1998), Robustness of Harris, T. J. and Ross, W. H. (1991), Statistical Process Control Forecast-Based Monitoring Schemes, Journal of Quality Procedures for Correlated Observations, Canadian Journal Technology, 30(4), 328-339. of Chemical Engineering, 69, 48-57. Alwan, L. C. (1992), Effects of Autocorrelation on Control Hunter, J. S. (1986), The Exponentially Weighted Moving Chart Performance, Communications in Statistics: Theory Average, Journal of Quality Technology, 18(4), 203-210. and Method, 21(4), 1025-1049. Jiang W. and Tsui, K. (2001), Some Properties of the ARMA Alwan, L. C. and Roberts, H. V. (1988), Time-Series Modeling Control Chart, Nonlinear Analysis, 47, 2073-2088. for Statistical Process Control, Journal of Business & Jiang, W., Tsui, K. and Woodall, W. H. (2000), A New SPC Economic Statistics, 6(1), 87-95. Monitoring Method: The ARMA Chart, Technometrics, 42 Apley, D. W. (2002), Time Series Control Charts in the Presence (4), 399-410. Performance and Robustness of Control Charting Methods for Autocorrelated Data 139 Johnson, R. A. and Bagshaw, M. (1974), The Effect of Serial toring Fault Signatures: Cuscore versus GLR, Quality and Correlation on the Performance of CUSUM Tests, Techno- Reliability Engineering International 19, 387-396. metrics, 16(1), 103-112. Runger, G. C., Willemain, T. R., and Prabhu, S. (1995), Average Lin, S. W., and Adams, B. M. (1996), Combined Control Charts Run Lengths for CUSUM Control Charts Applied to Resid- for Forecast-Based Monitoring Schemes, Journal of Quality uals, Communications in Statistics - Theory and Methods, Technology, 28(3), 289-301. 24(1), 273-282. Luceño, A. (1999), Average Run Lengths and Run Length Shu, L., Apley, D. W., and Tsung, F. (2002), Autocorrelated Probability Distributions for Cuscore Charts to Control Process Monitoring Using Triggered Cuscore Charts, Quality Normal Mean, Computational Statistics and Data Analysis, and Reliability Engineering International, 18, 411-421. 32(2), 177-195. Stoumbos, Z. G., Reynolds, M. R., Ryan, T. P., and Woodall, W. Lu, C. and Reynolds, M. R., Jr. (1999a), EWMA Control Charts H. (2000), The State of Statistical Process Control as We for Monitoring the Mean of Autocorrelated Processes, Journal Proceed into the 21st Century, Journal of the American of Quality Technology, 31(2), 166-188. Statistical Association, 95(451), 992-998. Lu, C. and Reynolds, M. R., Jr. (1999b), Control Charts for Superville, C. R. and Adams, B. M. (1994), An Evaluation of Monitoring the Mean and Variance of Autocorrelated Forecast-Based Quality Control Schemes, Communications in Processes. Journal of Quality Technology, 31(3), 259-274. Statistics: Simulation and Computation, 23(3), 645-661. Lucas, J. M. and Saccucci, M. S. (1990), Exponentially Weighted VanBrackle, L. N. and Reynolds, M. R., Jr. (1997), EWMA and Moving Average Control Schemes: Properties and CUSUM Control Charts in the Presence of Correlation, Enhancements, Technometrics, 32(1), 1-12. Communications in Statistics:Simulation and Computation, 26(3), 979-1008. Luceño, A. (2004), Cuscore Charts to Detect Level Shifts in Autocorrelated Noise, Quality Technology and Quantitative Vander Wiel, S. A. (1996), Monitoring Processes That Wander Using Integrated Moving Average Models, Technometrics, Management, 1(1), 27-45. 38(2), 139-151. Montgomery, D. C. (2005), Introduction to Statistical Quality Vasilopoulos, A. V. and Stamboulis, A. P. (1978), Modification Control, 5th ed., Wiley, New York, NY. of Control Chart Limits in the Presence of Data Correlation, Montgomery, D. C. and Mastrangelo, C. M. (1991), Some Journal of Quality Technology, 10(1), 20-30. Statistical Process Control Methods for Autocorrelated Data, Wardell, D. G., Moskowitz, H., and Plante, R. D. (1992), Control Journal of Quality Technology, 23(3), 179-193. Charts In the Presence of Data Correlation, Management Montgomery, D. C. and Woodall, W. H. (1997), A Discussion of Science, 38(8), 1084-1105. Statistically-Based Process Monitoring and Control, Journal Wardell, D. G., Moskowitz, H., and Plante, R. D. (1994), of Quality Technology, 29(2), 121-162. Run-Length Distributions of Special-Cause Control Charts for Moustakides, G. (1986), Optimal Stopping Times for Detecting Correlated Observations, Technometrics, 36(1), 3-27. Changes in Distributions. Annals of Statistics, 14(4), 1379- Woodall, W. H. and Faltin, F. (1993), Autocorrelated Data and 1387. SPC, ASQC Statistics Division Newsletter, 13(4), 1821. Page, E. S. (1955), A Test for a Change in a Parameter Occurring Woodall, W. H. and Montgomery, D. C. (1999), Research Issues at an Unknown Point, Biometrica, 42(3/4), 523-527. and Ideas in Statistical Process Control, Journal of Quality Pandit, S. M., and Wu, S. M. (1983), Time Series and System Technology, 31(4), 376-386. Analysis, With Applications, New York: John Wiley. Yang J. and Makis V. (1997), On the Performance of Classical Reynolds, M. R., Jr., and Stoumbos, Z. (2001), Monitoring the Control Charts Applied to Process Residuals, Computers and Process Mean and Variance Using Individual Observations Industrial Engineering, 33, 121-124. and Variable Sampling Intervals, Journal of Quality Yashchin, E. (1993), Performance of CUSUM Control Schemes Technology, 33(2), 181-205. for Serially Correlated Observation, Technometrics, 35(1), Roberts, S. W. (1959),Control Chart Tests Based on Geometric 37-52. Moving Averages, Technometrics, 1(3), 239-250. Zhang, N. F. (1998), A Statistical Control Chart for Stationary Runger, G. C. and Testik, M. C. (2003), Control Charts for Moni- Process Data, Technometrics, 40(1), 24-38.. All processes for comparison are 12, 16, 20, 24, 28, and 32. For different magnitude modeled as ARMA(1,1) plus possibly a deterministic mean shifts, the residual means have exactly the same mean shift shapes but are scaled differently. We compare EWMA, OGLF, OSLF, CUSCORE, and GLRT charts in terms of their ARL performance. , The EWMA chart is one of the most popular control charting methods and is often recommended for mon- which includes, as special cases, AR(1) when =0, itoring the residuals of autocorrelated data. In fact, the MA(1) when = 0, and i.i.d. when = =0. EWMA chart is revealed to be optimal in several sit- Without loss of generality, is assumed to be 1 for uations under consideration in this paper. Note that the the remainder of the paper. Step, spike, sinusoidal, and EWMA reduces to a Shewhart individual chart in the ramp mean shifts are considered with a wide range of limit (as λ approaches 1.0) such as in Examples 9~12 magnitudes from 0 to 4. The step mean shift is de- and 25~28, which is known to be effective at detecting fined as =0fort <1and = for t ≥ 1andthe spikes. Although it will be shown that the CUSCORE spike mean shift is defined as = and =0for outperforms the EWMA for the examples in which the ≠. The sinusoidal shifts are denoted S1－S4 in residual mean has pronounced dynamics, there are
. S1,S2,andS3 are sinusoidal functions with quite a few examples in which the EWMA outper-
shows the zero-state ARLs denoted by ARL0zs and ARL1zs, where ≺≤is the EWMA parameter and ζ is a respectively. The standard errors are in parentheses. scaling constant. The Shewhart individual chart is in- The minimum out-of-control ARL for each example is directly considered for comparison because it is a spe- indicatedbyboldfontineachrowof
. The superiority of mize the zero-state out-of-control ARL while constrain- one over the other chart is determined depending on ing the zero-state in-control ARL to be 500. The out-of- whether the initial dynamics plays a larger role than control ARL minimization is for an assumed mean shift the lasting effects of the residual mean. The OGLF shape and magnitude and a shift time-of-occurrence performs better than the CUSCORE in Examples 5, 6, that coincides with the initial observation. Chin and 20, 21, and 22 where the residual mean has prominent Apley (2006) and Apley and Chin (2007) plot the im- dynamics and rapidly converges to zero or a very small pulse response coefficients for all of the examples that steady state shift and the control charts are expected to we consider here. For the EWMA chart, the value of rely primarily on the initial dynamics. The opposite is is chosen using the same constrained optimization true for Examples 7, 8, 17, 18, 19, 23 and 24 where criterion. The thresholds of the CUSCORE and GLRT the residual mean converges to zero very slowly or has charts are determined to provide the desired in-control a small but considerable steady state shift compared ARL using Monte Carlo simulations and for the design to the significance of initial dynamics. The OGLF parameters of two, the values that Luceño(1999) and chart performs slightly better than the CUSCORE Apley and Shi (1999) recommended are respectively and GLRT for Examples 29~32 with ramp mean shifts, taken. The handicap for the CUSCORE chart was chos- but nearly identically to the OSLF charts and the opti- 130 Chang-Ho Chin Daniel W. Apley
. The analytical PVCs agree reasonably well (1− λ2 ) with the empirical PVCs and are also fairly consistent 1 2 λ1,λ2 = (α1 ± α1 + 4α 2 ) with the percentage ARL changes, in the sense that 2 . larger PVC values almost always correspond to larger ARL percentage changes. Although it would be more For the EWMA chart defined in Equation (9), as a spe- desirable to directly consider the sensitivity of the cial case of the OSLF chart when , ARL, it is unfortunately too analytically intractable. As ,and , Performance and Robustness of Control Charting Methods for Autocorrelated Data 133
shows the sensitivities for the 32 Exam- S e ,EWMA (θ1 ) = − 1− θ1 (1− λ ) . ples considered in Section 3. The minimum sensitivity for each example is indicated by bold font. Several in- For the Shewhart chart as a special case of the EWMA teresting characteristics are revealed. As shown in chart when , Equations (A.1) and (A.2), the sensitivities are the 134 Chang-Ho Chin Daniel W. Apley weighted sum of the impulse response coefficients of the sensitivities tend to get smaller as the magnitude of the AR and MA polynomials of the original process, the feared signal increases. In order to increase the de- where the weights are given by the autocorrelation tection probability in this case, a control chart is de- function of the control chart statistic. Specifically, the signed to place more weight on recent observations to sensitivities mainly rely on the type (positive or neg- improve the detection of mean shifts of large magni- ative) and decay rate of autocorrelation functions of tude, which leads to a control chart with fast-decaying the control chart statistic and AR and MA polynomials autocorrelation function. For instance, the λ’s of the of the original process. The more slowly decaying the EWMA charts for Examples 1~ 4 are 0.047, 0.242, autocorrelation functions, typically the higher the sen- 0.676, and 0.887, respectively. Since the impulse re- sitivities. Hence, the sensitivity of a control chart may sponse coefficients {gj} of the EWMA are , change for different original processes. The EWMA the EWMA chart has the largest λ for the largest charts for 11 of 32 examples reduce to a Shewhart feared signal and has the smallest sensitivities. This is chart, which has zero sensitivities. Consider the EWMA consistent with the sensitivity expressions in Equations charts for the remaining 21 examples, for which the (A.1) and (A.2). This fact implies that EWMA charts chart statistics have positive autocorrelation. The mag- with other larger non-optimal λ’swouldbemorero- nitudes of their sensitivities can be explained in the bust, even though their performance are a little bit light of the autocorrelation of the original processes. worsened. Apley and Lee (2003) also found that larger For Examples 1~4, 15~16, and 29~32 in which the values of λ do indeed result in better robustness. original processes are i.i.d., the sensitivities for both However, the performance for small shifts may be so and are low. For Examples 5~8, 17~19, and 21~24, much worse for larger values of λ that a more attrac- the AR polynomials have highly positive autocorrela- tive alternative is to use a smaller value of λ but use tion and thus, the sensitivities for are high. On the control limits that are wider than normal to prevent an other hand, the sensitivities for are relatively low excessive number of false alarms. Apley and Lee due to the MA polynomials of Examples 5~8 with no (2003) and Apley (2002) also presented methods for autocorrelation, those of Examples 17~19 with neg- suitably widening the control limits for this purpose. ative autocorrelation, and those of Examples 21~24 The Se,GLRT( ,)andSe,GLRT( ,)oftheGLRTare with low positive autocorrelation. identical for the same type of mean shift regardless of For i.i.d. processes ( ), both sensitivities are magnitude, respectively, because the same unit magni- the same because the autocorrelations of the tude feared signal is used as a matched filter. Note that and are identical. Except for processes such as the sinusoidal means for Examples 13 and 14 have dif- Examples 2~ 4, 9~20, 25~28, 31, and 32 in which the ferent periods, which results in different sensitivities, optimally tuned EWMA charts turn out to have large whereas the sensitivities are the same for Example 15 λ’s resulting in very small moving window lengths and 16 of the same period. We calculate the sensitiv- (e.g., it becomes a Shewhart chart with for ities of the GLRT for only one of the GLRT statistics Examples 9~14 and 25~28) or when the MA poly- (e.g., for a mean shift occurring at only one of the N nomial of the original process has negative autocorre- timesteps within the window). We chose in the lation, the GLRT chart has the lowest sensitivities (i.e., following examples, because it is in the mid range of the most robust) to the modeling error due to the small all theζ values in Equation (6) and most reasonably number of impulse response coefficients constituting a represents the sensitivity of the GLRT. moving window, and the OGLF are at least as robust We consider two situations that and are in- as the EWMA is. Note that the GLRT chart consistently correctly estimated for Examples 17~20 (let the pa- has small sensitivities for all examples, but the EWMA rameter values listed in
represent the esti- chart does not such as for Examples 5~8, 17~19, and mated parameters). Suppose that the values of true pa-
. The actual PVCs agree rea- 100,000 replicates are used to compare the analytical sonably well with the approximations, and the close results of sensitivities with empirical results. The ac- correspondence between the PVCs and the percentage
change of control charts with respect to modeling shows that an error of 0.01 in the AR parameter causes errors. a 17% decrease in the variance of the EWMA and For Example 18, the OSLF and EWMA chart statistics OGLF statistics, which is substantial. The surprisingly are recursively calculated by large sensitivity of the charts underscores the need for and , analytical sensitivity expressions that explain the respectively (Chin and Apley 2006). The respective mechanisms behind the lack of robustness of certain variances for both charts under the assumption of no charts. Because larger parameter errors are also of in- modeling error are 0.1355 and 0.5179. To illustrate the terest, we extend this analysis to the parameter errors effects of modeling errors, we calculate the variances of ±0.03, and the results are shown in
. The for both charts experiencing the preceding modeling difference among the analytical results of sensitivities error in which only one of the two parameters is in- and the changes of the variances and performances be- correctly estimated. With overestimated, the actual comes larger, because the sensitivity is based on a first- variance for the OSLF chart is 0.1369 which is 1.03% order partial derivative. Note that the sensitivities still larger than the assumed one. This PVC 1.03% is con- provide a reasonable basis for comparison, however. sistent with the analytical result that the approximate For the OSLF with incorrectly estimated and in percentage variance increase in the OSLF variance is Example 18 aforementioned, the PVCs 3.17% and Se,OSLF( )( ) = 1.01%. The actual PVC leads to 39.71% are still fairly consistent with the analytical re- the decrease in the in-control ARL from 500 to 484. sult 3.02% and 34.22%, respectively. The PVC -3.13%