Setting Specifications Statistical considerations

Enda Moran – Senior Director, Development, Pfizer Melvyn Perry – Manager, , Pfizer Bas ic Sta tis tics

Population distribution 1.5

(usually unknown). True bat ch assay described by μ and σ. 1.0 Distribut ion of possible values 0.5

0.0 98 99 100 We infer the population from samples by calculating x and s.

1.5 1.5 1.5 True bat ch assay True bat ch assay Sample 3 True bat ch assay Sample 1 Sample 2 Average 99.0 1.0 Average 1.0 Average 98.7 1.0 98.6 Dist ribut ion of Distribut ion of Dist ribut ion of possible values possible values possible values 0.5 0.5 0.5

0.0 0.0 0.0 98 99 100 98 99 100 98 99 100 2 ItInterval s

30

28

26

24 Population average 22

20

18

16 0 102030405060708090100

•100 samples of size 5 taken from a population with an average of 23.0 and a of 2.0. •The highlighted intervals do not include the population average (there are 6 of them). •For a 95% confidence level expect 5 in 100 intervals to NOT include the population average. •Usually we calculate just one interval and then act as if the population falls within this interval. 3 ItInterval s

Point Estimation The best estimate; eg MEAN

Interval Estimation A which contains the true population parameter or a future observation to a certain degree of confidence.

Confidence Interval - The interval to estimate the true population parameter (e.g. the population mean).

Prediction Interval - The interval containing the next single response.

Tolerance Interval - The interval which contains at least a given proportion of the population.

4 FlfItlFormulae for Intervals

Intervals are defined as: x ± ks AiAssuming a norma lditibtil distribution • Confidence (1-α) interval 0.5 ⎛ 1 ⎞ CI = x ± ⎜ ⎟ t α s 1− ,n−1 ⎝ n ⎠ 2 • Prediction (1-α) interval for m future observations

0.5 ⎛ 1 ⎞ PI = x ± ⎜1+ ⎟ t α s 1− ,n−1 ⎝ n ⎠ 2m • Tolerance interval for confidence (1-α) that proportion ()i(p) is covere d

⎛ 1 ⎞ 2 ()n − 1 ⎜1+ ⎟z(1−p) ⎝ n ⎠ TI = x ± 2 s χ 2 α,n−1 5 Process C apa bility

Process capability is a measure of the 3s 3s risk of failing specification. The spread of the are compared with the width of the specifi cati ons.

The distance from the mean to the nearest specification relative to half the process width (3s).

The index measures actual performance. Which may or may not be on target i.e., centred. LSL x USL x − LSL USL − x ⎧USL − x x − LSL⎫ Ppk = min⎨ , ⎬ ⎩ 3s 3s ⎭

6 Process C apa bility – PkPpk and dCk Cpk

• Ppk should be used as this is the actual risk of failing specification. • Cpk is the potential capability for the process when free of shifts and drifts. Random data of mean 10 and SD 1, thus natural span 7 to 13. Added shifts to simulate trends around common average. With specs at 7 and 13 process capability should be unity. When data is with trend Ppk

Process Capability of Shifted less than Cpk due to method of I Chart of Shifted 15 1 LSL USL calculation of std dev. Process Data Within 14 LS L 7 Overall UCL=13.589 Target * Potential (Within) C apability 13 USL 13 Sample Mean 10.0917 Cp 0.86 CPL 0.88 Ppk uses sample SD. Ppk less Sample N 30 12 StDev (Within) 1.16561 CPU 0.83 StDev (O v erall) 1.95585 Cpk 0.83 11 O v erall C apability than 1 at 0.5. _ Pp 0.51 10 X=10.092 PPL 0.53 PPU 0.50 ividual Value

dd Ppk 0.50 In 9 Cpm * CkCpk uses average moving 8 range SD (same as for control 7 LCL=6.595 6 8 10 12 14 6 O bserv ed P erformance Exp. Within Performance Exp. O v erall Performance chart limits). 1 4 7 10 13 16 19 22 25 28 PPM < LSL 0.00 PPM < LSL 3995.30 PPM < LSL 56966.34 PPM > USL 100000.00 PPM > USL 6296.59 PPM > USL 68512.70 Observation PPM Total 100000.00 PPM Total 10291.88 PPM Total 125479.03 Cpk is close to 1 at 0.83.

Process Capability of Raw I Chart of Raw

13 UCL=13.038 LSL USL Process Data Within LS L 7 When data is without trend Overall Target * 12 USL 13 Potential (Within) C apability Sample Mean 10.0917 Cp 1.02 Ppk is same as Cpk. Only Sample N 30 CPL 1.05 11 StDev (Within) 0.982187 CPU 0.99 StDev (O v erall) 1.14225 Cpk 0.99 _ O v erall C apability small differences are seen. 10 X=10.092 Pp 0.88 PPL 0.90 PPU 0.85 Ppk 0.85 ndividual Value II 9 C pm * Cpk effectively 1 at 0.99.

8 7 8 9 10 11 12 13 Ppk close to 1 at 0.85. LCL=7.145 7 O bserv ed P erformance Exp. Within Performance Exp. O v erall Performance 1 4 7 10 13 16 19 22 25 28 PPM < LSL 0.00 PPM < LSL 822.51 PPM < LSL 3397.73 7 PPM > USL 0.00 PPM > USL 1533.13 PPM > USL 5446.84 Observation PPM Total 0.00 PPM Total 2355.65 PPM Total 8844.57 MtUtitMeasurement Uncertainty

LSL Good USL parts σ almost measurement always passed Bad parts σtotal almost Bad parts always almost rejtdjected always rejected

±3σmeasurement ±3σmeasurement

The grey areas highlighted represent those parts of the curve with the potential for wrong decisions, or mis-classification. 8 Misclassification with less varibliable process

σtotal LSL USL σmeasurement

±3σmeasurement ±3σmeasurement

9 MtUtitMeasurement Uncertainty

Precision to Tolerance Ratio: 6×σ P /T = MS Tolerance How much of the tolerance is taken up by measurement error. Tolerance = USL − LSL This estimate may be appropriate for evaluating USL = Upper Spec Limit how well the measurement system can perform LSL = Lower Spec Limit with respect to specifications. σ MS = Std. Dev. of Measurement Sys.

σ MS Gage R&R (or GRR%) %R & R = ×100 σ Total What percent of the total variation is taken up by measurement error (as SD and thus not additive).

Use Measurement Systems Analysis to assess if the assay method is fit for purpose. It is unwise to have a method where the specification interval is consumed by the measurement variation alone. 10 Specification example – TlTolerance ItInterval

Data from three sites used to set specifications. 16.5 16.0 Tolerance interval found from pooled data of 253 15.5 batches. 15.0

14.5 Tolerance interval chosen as 95% probability that 14.0 mean ± 3 standard deviations are contained. 13.5

13.0 Sample size: Mean ± 3s Tolerance 1 23 n=253 interval Code

Mean = 14.77 (%/(95% / N k multiplier Table of values for 95% 99.7%) 56.60 probability of interval s=0.58 10 4.44 containing 99% of Range 13. 03 -16. 51 12. 89 -16. 65 15 3893.89 population values. 30 3.35 ∞ 2.58 Note at ∞ value is 2.58 which is the z value for 99% coverage of a normal If sample size was smaller, difference between population these calculations increases.

As the sample size approaches infinity the TI approaches mean ± 3s. 11 Specification Example - Stabilit y

Batch history Total SD (process and measurement)

Stability batch with SD (measurement)

Shelf life set from 95% CI on slope from three clinical batches.

Need to find release criteria for high probability of production batches meeting shelf life based on individual results being less than specification.

RiReview of b at ch hi st ory will lea d to a process capa bility s ta temen t against release limit. 12 Specification Example - Stabilit y

Release limit is calculated from the rate of change and includes the uncertainty in the 8 slope estimate (rate of change of parameter) 7 and the measurement variation).

6

⎛ s 2 ⎞ 5 RL = Spec - ⎜Tm + t T 2s 2 + a ⎟ ⎜ m n ⎟ ⎝ ⎠ 4

Specification = 8 3 Shelf life (()T) = 24 Parameter slope (m) = 0.04726 2

Variation in slope (sm) = 0.00722 1 Variation around line (sa) = 0.86812 t on 126 df approx. = 2 0 N= 1 for single determination 0 6 12 18 24 30 36 MONTHS Release limit = 5.1

A similar approach can be taken in more complex situations; e.g,, after reconstitution of lyophilised product. Product might slowly change during cold storage and then rapidly change on reconstitution. Allen et al., (1991) Pharmaceutical Research, 8, 9, p1210-1213 13 Cop ing w ithliitddtith limited data

Problems with n<30. • Difficult to calculate a reliable estimate of the SD. • Difficult to assess distribution. • Difficult to assess process stability.

With very limited data the specifications will be wide due to the large multiplier. • This reflects the uncertainty in the estimates of the mean and SD.

Is there a small scale model that matches full production scale?

How variable is the measurement system?

Alternatives are: Example with 4 values • Min / max values based on N(20, 4) – 19. 9, 16. 9, 22. 1, 21. 3 scientific rationalisation • Mean ± 3s Mean ± 3s 13.2 27.0 • Mean ± 4s or minimum Ppk = 1.33. Mean ± 4s 10.9 29.3 • Tolerance interval approach TI (95/99.7) -1.5 41.7 14 Uppgpdating Specifications When, why, and a scenario for consideration

• Specifications for CQAs might require revision during the product lifecycle. For example, new clinical data and in telli gence on con tinue d use o f a pro duc t m ig ht dr ive a c hange to a CQA spec ifica tion • Scenario: What if a process producing a new product approaching regulatory approval is ‘incapable’ of meeting the clinically- tested CQA in the long-term? For example......

Proposed Spec. Proposed Spec. If spec. Process required Process capability capability CQA tbto be set CQA estimate estimate at the Clinical Clinical experience experience / New spec. clinically- tested level......

Clinical / Validation Lots Clinical / Validation Lots An ‘incapable’ process • How could we manage this situation? Complex. Many factors need to be considered if setting specifications at clinically tested level ()(will be discussed through this workshop) • IF it must be done for risk reduction purposes, and if there is sufficient uncertainty that the Proposed Spec. for the CQA could cause harm, a possible approach may be to move towards the clinically-tested level in stages. - Accrue approx. n=25-30 batches/lots to consider estimates of future pppyrocess capability acceptable. Adjust spec. to a revised process capability estimate. - Accrue further lots of the stabilised process to approx. n=85. Make FINAL spec. adjustment to a revised process capability estimate. 15 Summary

• Process capability & measurement uncertainty • Setting spec.s - large number of datapoints • Setting spec.s – small number of datapoints

16