Maximum Likelihood Estimator (MLE) for the Process Change Point Using the Step Change Likelihood Function for a Binomial Random Variable

A robust change point estimator for binomial CUSUM control charts

Advisor: Takeshi Emura ( 江村剛志 ) 博士 Presenter : Yi Ting Ho

2014/6/24 1 Outline

 Introduction

 Background 1. Binomial CUSUM control chart 2. Maximum likelihood estimator 3. Page’s last zero estimator

 Method Combine CUSUM estimator and MLE

2014/6/24 2 Outline

 Simulations Design closely follows those of Perry and Pignatiello (2005)

 Data analysis Jewelry manufacturing data by Burr (1979)

 Conclusion

2014/6/24 3 Outline

 Introduction

 Background 1. Binomial CUSUM control chart 2. Maximum likelihood estimator 3. Page’s last zero estimator

 Method Combine CUSUM estimator and MLE

2014/6/24 4 Introduction

• What is the change point?

• We consider observations come from binomial distribution with the same fraction nonconforming p . CUSUM Chart

15 Out-of-control

h=12.043

CumulativeSum 5

Change

0 point

0 10 20 30 40 50 60 2014/6/24 Subgroup number 5 Introduction

In SPC, the np-chart is most famous charts used to monitor the number of nonconforming items for industrial manufactures.

np-chart

When sample size is large and defective 10

upper control limit rate is not too small, the np-chart works 8

well. 6 defectives

4 center line

lower control limit

2014/6/24 0 5 10 15 20 25 30 Subgroup number 6 Introduction

Page (1954, 1955) first suggested the CUSUM chart to estimate the change point.

The binomial CUSUM control chart is a good alternative when small changes are important.

2014/6/24 7 Introduction

Samuel and Pignatiello (2001) proposed maximum likelihood estimator (MLE) for the process change point using the step change Likelihood function for a binomial random variable.

Perry and Pignatiello (2005) shows that the performance of the MLE is often better than Page’s last zero estimator.

2014/6/24 8 Introduction

The MLE method outperforms CUSUM method when magnitudes of change is large.

In order to construct more robust in different parameter setting, this thesis combines CUSUM estimator and MLE. Furthermore, compares the new method with two estimators.

2014/6/24 9 Outline

 Introduction

 Background 1. Binomial CUSUM control chart 2. Maximum likelihood estimator 3. Page’s last zero estimator

 Method Combine CUSUM estimator and MLE

2014/6/24 10 Binomial CUSUM control chart

i.i.d i.i.d True X1, X 2 ,, X ~ Bin( n, p0 ), X 1, X 2 ,, XT ~ Bin( n, pa )

True True  { ( , pa ) |  {1, 2 ,, T }, p0  pa 1} for a fixed value

T {1, 2,}.

 n x nx    p0 (1 p0 ) , if i 1, 2,, ,   x P( X i  x )   n   ( pTrue )x (1 pTrue )nx , if i  1,  2,, T.    a a   x

p 0 : in-control fraction nonconforming. True pa : out-of-control fraction nonconforming.  : the true change point . 2014/6/24 11 Binomial CUSUM control chart

S  0 and Si  max{ 0, Xi  nk  Si1 }, i 1, 2, 0  1 pas   ln a   1 p  k   0  , where  pas(1 p )  ln a 0  as   p0 (1 pa ) 

k : The reference value. as p a : out-of-control fraction nonconforming for which to design the CUSUM chart.

2014/6/24 12

Binomial CUSUM control chart

When S i exceeds decision interval h  0 the chart signals that an increase in the process fraction nonconforming has occurred.

2014/6/24 13 Maximum likelihood estimator (MLE)

i.i.d i.i.d True X1, X 2 ,, X ~ Bin( n, p0 ), X 1, X 2 ,, XT ~ Bin( n, pa )

True True  { ( , pa ) | {1, 2 ,, T }, p0  pa 1} for fixed value

T {1, 2,}.

The likelihood function is given by

 T n  x n  True   i nxi   True xi True nxi L(, pa | p0 ,  )   p0 (1 p0 )  ( pa ) (1 pa ) i1  xi  i 1 xi 

where   ( x1,x2,, xT ).

2014/6/24 14 Maximum likelihood estimator (MLE)

The log-likelihood is   True log L( , pa | p0 ,  )  C  log( p0 ) xi  log( 1 p0 )( n  xi ) i1 i1 T T True True  log( pa )  xi  log( 1 pa ) ( n  xi ), i 1 i 1 True where C is a constant. We can rewrite log L(, pa | p 0 ,  ) as

 pTrue  T  1 pTrue  T log L(, pTrue | p ,  )  C*  log a  x  log a  ( n  x ), a 0  p   i  1 p   i  0 i 1  0 i 1

T T * where C  C  log( p0 ) xi  log( 1  p0 )( n  xi ) is a constant. i1 i1 2014/6/24 15 Maximum likelihood estimator (MLE)

If  is given, the log-likelihood equation becomes

T T x ( n  x )   i  i log L( , pTrue | p ,  )  i 1  i 1 , pTrue a 0 pTrue 1 pTrue a a a

T T T x  xi ( n  xi )  i i 1  i 1  pˆ True( )  i 1 . pTrue 1 pTrue a T a a n i 1

2014/6/24 16 Maximum likelihood estimator (MLE)

T T 2  xi ( n  xi )  i 1 i 1 2 log L( , pa | p0 ,  )   2  True 2  0. True True (1 p ) p p a a a

pˆ True( ) True Putting a into log L (, p | p ,  ), we have a profile a 0 log-likelihood for  as

pˆ True( ) T 1 pˆ True( ) T log L(, pˆ True( ) | p ,  )  C*  log( a ) x  log( a ) ( n  x ). a 0  i  i p0 i 1 1 p0 i 1

2014/6/24 17

Maximum likelihood estimator (MLE)

Therefore, the change point estimator  ˆ MLE is

True True   pˆ ( )  T  1 pˆ ( )  T  ˆ a a  MLE  arg max log   xi  log  ( n  xi ) . {1, 2,, T } p 1 p   0 i 1  0 i 1 

2014/6/24 18 Page’s last zero estimator

Under CUSUM control chart we have cumulative sum

S0  0 and Si  max{ 0, Xi  nk  Si1 }, i 1, 2,

An estimate of the change point is given by ˆCUSUM  max{ i : Si  0}.

2014/6/24 19 Outline

 Introduction

 Background 1. Binomial CUSUM control chart 2. Maximum likelihood estimator 3. Page’s last zero estimator

 Method Combine CUSUM estimator and MLE

2014/6/24 20 Proposed method

We propose a new change point estimator that combines ˆMLE and ˆCUSUM.

ˆNEW ( w )  wˆCUSUM(1 w )ˆMLE , 0  w 1,

 True    pa     pTrue  p  p0   a 0  True as  , if pa  p0  pa  p0 ,  pas  p  True as  a 0  w  w( p | pa )  a   True   pa   as    p  p  p0   a 0  , if pTrue  p  pas  p .  pTrue  p  a 0 a 0  a 0 

2014/6/24 21 Proposed method

True First, we estimate unknown pa by MLE. T  xi iˆMLE 1 pˆ a (ˆMLE )  T , n iˆMLE 1

The estimator of weight function as

 pˆ a (ˆMLE )      pˆ (ˆ )  p  p0   a MLE 0  , if pˆ (ˆ )  p  pas  p ,  pas  p  a MLE 0 a 0 as  a 0  wˆ  w( pˆ a (ˆMLE ) | pa )    pˆ (ˆ )   a MLE   as    p  p  p0   a 0  , if pˆ (ˆ )  p  pas  p .  pˆ (ˆ )  p  a MLE 0 a 0 2014/6/24  a MLE 0  22 Proposed method

Hence, we obtain the proposed estimator of the change point as

ˆ ( wˆ )  wˆˆ (1 wˆ )ˆ , 0  wˆ 1. NEW CUSUM MLE

2014/6/24 23 Outline

 Simulations Design closely follows those of Perry and Pignatiello (2005)

 Data analysis Jewelry manufacturing data by Burr (1979)

 Conclusion

2014/6/24 24 Simulations

i.i.d i.i.d True X1, X 2 ,, X ~ Bin( n, p0 ), X 1, X 2 ,, XT ~ Bin( n, pa )

• We assume the true change point  100.

• The in-control fraction nonconforming p0  0.1.

• The out-of-control fraction nonconforming

pTrue { 0.11, 0.12,, 0.25, 0.30 }. a

25 2014/6/24

Simulations

• Consider CUSUM charts designed to detect 30% increase in as by setting pa 1.3 p0  0.13.

• Choose h  6. 57 or h  11 . 42 such that in-control process average run lengths (ARL) is close to 150 or 370.

2014/6/24 26

as p0  0.1, pa  0.13,

Simulations True pa  0.1, 0.11, 0.15, 0.20, 0.25, 0.30

Cusum Chart Cusum Chart Cusum Chart

12 12 12

h=6.57 h=6.57 h=6.57

10 10 10

8 8 8

6 6 6

4 4 4

Cumulative Sum Cumulative Sum Cumulative Sum Cumulative

2 2 2

0 0 0

0 50 100 150 200 250 300 0 20 40 60 80 100 120 0 20 40 60 80 100

i i i

Cusum Chart Cusum Chart Cusum Chart

12 12 12

h=6.57 h=6.57 h=6.57

10 10 10

8 8 8

6 6 6

4 4 4

Cumulative Sum Cumulative Sum Cumulative Sum Cumulative

2 2 2

0 0 0

0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100

i i i 27 2014/6/24 as p0  0.1, pa  0.13,

Simulations True pa  0.1, 0.11, 0.15, 0.20, 0.25, 0.30

Maximum likelihood estimator Maximum likelihood estimator Maximum likelihood estimator

20 20 20

15 15 15

h=6.57 h=6.57 h=6.57

10 10 10

Log - Likelihood Log - Likelihood Log - Likelihood Log

5 5 5

0 0 0

0 50 100 150 200 250 300 0 20 40 60 80 100 0 20 40 60 80 100

i i i

Maximum likelihood estimator Maximum likelihood estimator Maximum likelihood estimator

20 20 20

15 15 15

h=6.57 h=6.57 h=6.57

10 10 10

Log - Likelihood Log - Likelihood Log - Likelihood Log

5 5 5

0 0 0

0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 i i i 28 2014/6/24 as p0  0.1, pa  0.13,

Simulations True pa  0.1, 0.11, 0.15, 0.20, 0.25, 0.30

Cusum Chart Cusum Chart Cusum Chart

20 20 20

h=11.42 h=11.42 h=11.42

15 15

10 10 10

Cumulative Sum Cumulative Sum Cumulative Sum Cumulative

5 5 5

0 0 0

0 50 100 150 200 250 300 0 50 100 150 0 20 40 60 80 100

i i i

Cusum Chart Cusum Chart Cusum Chart

20 20 20

h=11.42 h=11.42 h=11.42

15 15 15

10 10 10

Cumulative Sum Cumulative Sum Cumulative Sum Cumulative

5 5 5

0 0 0

0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100

i i i 29 2014/6/24 as p0  0.1, pa  0.13,

Simulations True pa  0.1, 0.11, 0.15, 0.20, 0.25, 0.30

Maximum likelihood estimator Maximum likelihood estimator Maximum likelihood estimator

20 20 20

h=11.42 h=11.42 h=11.42

15 15

10 10 10

Log - Likelihood Log - Likelihood Log - Likelihood Log

5 5 5

0 0 0

0 50 100 150 200 250 300 0 20 40 60 80 100 0 20 40 60 80 100

i i i

Maximum likelihood estimator Maximum likelihood estimator Maximum likelihood estimator

20 20 20

h=11.42 h=11.42 h=11.42

15 15 15

10 10 10

Log - Likelihood Log - Likelihood Log - Likelihood Log

5 5 5

0 0 0

0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100

i i i 30 2014/6/24 Simulations

Table 4 Simulation results for estimating the change point  under n  50, p0  0.1 as and pa  0 . 13 based on 1000 simulation runs. The true change point  100.

2014/6/24 31 Simulations

2014/6/24 32

Simulations

Table 5 Simulation results for estimating the mean squared error the change point  as Under n  50, p0  0 . 1 and pa  0 . 13 based on 1000 simulation runs. The true change point  100.

2014/6/24 33 Simulations

2014/6/24 34 Simulations

• In most cases, the outperforms other estimators.

• Despite some of the mean squared error of  ˆ NEW ( wˆ ) in all cases are not the smallest, provides very precise estimator of change point.

2014/6/24 35 Outline

 Simulations Design closely follows those of Perry and Pignatiello (2005)

 Data analysis Jewelry manufacturing data by Burr (1979)

 Conclusion

2014/6/24 36 Data Analysis

Jewelry manufacturing process

2471 yellow beads (good pieces)

229 229 red beads We set the in-control fraction nonconforming p0   0.085. (defective pieces) 2700 Each subgroup contains n  50 beads so the number of T  2700  54. subgroups is 50

Xi  the number of defective pieces in 50 beads, for i 1, 2,, 54. 2014/6/24 37 38 2014/6/24 Data Analysis

• Center line  np  500.085  4.25.

• Upper control limit  np 3 n p0 (1 p0 )  4.25 3 500.0850.915 10.165.

• Lower control limit  np 3 n p0 (1 p0 )

 4.25 3 500.0850.915  1.665.

39 2014/6/24 Data Analysis

np-chart 10

defectives 4

47th sample

0 10 20 30 40 50 60 Subgroup number Although np-chart do not detect the process out-of-control, the data slightly increase after the subgroup 40.

2014/6/24 40 Data Analysis

CUSUM Chart Maximum likelihood estimator

8 15 Out-of-control

^τ=50

6 10

h=12.043

Log - Likelihood - Log

CumulativeSum

0 0

0 10 20 30 40 50 60 0 10 20 30 40 50 60

i Subgroup number Change 2014/6/24 point 41 Data Analysis

ˆ ˆ • We obtain ˆCUSUM  43, ˆMLE  50 and  NEW ( w )  47.70983.

• ˆ CUSUM always underestimate the true change point while ˆMLE slightly True as overestimate  when pa  Pa .

• Our proposed method may provide more unbiased estimator.

2014/6/24 42 Outline

 Simulations Design closely follows those of Perry and Pignatiello (2005)

 Data analysis Jewelry manufacturing data by Burr (1979)

 Conclusion

2014/6/24 43 Conclusion

• The estimator ˆNEW ( wˆ ) contains advantage of  ˆ CUSUM and ˆMLE.

• Our proposed method are unbiasedness for estimating the true change point.

• The estimator is more robust than and  ˆ MLE under different parameter setting.

2014/6/24 44 References

• Assareh H, Mengersen K, Change point Estimation in Monitoring Survival Time. PloS ONE 2012; 7(3): e33630. Doi:10.1371/journal.pone.0033630. • Assareh H, Mengersen K, Change point detection in risk adjusted control charts. Statistical Methods in Medical Reasearch 2011; 1-22, doi: 10.1177/0962280211426356. • Burr WI, Elementary Statistical Quality Control (1st edn) Milwaukee: New York and Basel, 1979. • Duran RI, Albin SL. Monitoring a fraction with easy and reliable settings of the false alarm rate. Quality and Reliability Engineering International 2009; 25(8): 1026-1043. • Emura T, Lin YS, A comparison of normal approximation rules for attribute control charts. Quality and Reliability Enginerring International 2013, doi: 10.1002/qre.1601. • Fuh CD, Mei Y (2008) Optimal stationary binary quantizer for decentralized quickest change detecction in hidden Markov models, 11th International Conference on IEEE Information Fusion. • Hawkins DM, Olwell DH, Cumulative sum charts and charting for quality improvement ( 1st edn ). Wiley: New York, 1998. • Khan RA, A note on estimating the mean of a normal distribution with known coefficient of variation. Journal of the American Statistical Association 1968; 1039-1041, dol:10.2307/2283896. 45 2014/6/24 References

• Montgomery DC. Introduction to Statistical Quality Control. Wiley: New York, NY, 2009. • Page ES, Continuous inspection schemes. Biometrika 1954; 41:100-114. • Page ES, A test for a change in a parameter occurring at an unknown point. Biometrika 1955; 523-527. • Pignatiello JJ Jr, Samuel TR, Identifying the time of a step change in the process fraction nonconforming. Quality Engineering 2001; 13(3): 357-365. • Pignatiello JJ Jr, Perry MB, Estimation of the change point of the process fraction nonconforming in SPC applications. Interational Journal of Reliability Quality and Safety Engineering 2005; 12: 95-110. • Pignatiello JJ Jr, Simpson JR, Perry MB, Estimating the change point of the process fraction nonconforming with a monotonic change disturbance in SPC. Quality and Reliability Engineering Inernational 2007; 23: 327-339. • Rossi G, Sarto SD, Marchi M, A new risk-adjusted Bernoulli cumulative sum chart for monitoring binary health data. Statistical Methods in Medical Reasearch 2014; 1-10, dol: 10.1177/0962280214530883.

2014/6/24 46 References

• Wetherill GB, Brown DB, Statistical Process Control. Theory and Practice, 1991. • Wang H, Comparison of p control charts for low defective rate. Computational Statistics & Data Analysis 2009; 53:4210-4220. • Wang YH, Economic design of CUSUM chart with variable sampling. Master thesis, NTHU library, 2008. • Wald A, Sequential Analysis (1st edn). Wiley: New York, 1947. • Yang SF, Cheng TC, Hung YC, Cheng SW. A new chart for monitoring service process mean. Quality and Reliability Engineering International 2012; 28: 377-386.

2014/6/24 47

Thanks for your attention .

2014/6/24 48