Estimating the Survival Function
Total Page:16
File Type:pdf, Size:1020Kb
S-Plus Commands for Survival Estimation Estimating the Survival Function > t_c(1,1,2,2,3,4,4,5,5,8,8,8,8,11,11,12,12,15,17,22,23) One-sample nonparametric methods: > surv.fit(t,status=rep(1,21)) 95 percent confidence interval is of type "log" We will consider three methods for estimating a survivorship time n.risk n.event survival std.dev lower 95% CI upper 95% CI function 1 21 2 0.90476190 0.06405645 0.78753505 1.0000000 2 19 2 0.80952381 0.08568909 0.65785306 0.9961629 S(t)=Pr(T ≥ t) 3 17 1 0.76190476 0.09294286 0.59988048 0.9676909 4 16 2 0.66666667 0.10286890 0.49268063 0.9020944 5 14 2 0.57142857 0.10798985 0.39454812 0.8276066 without resorting to parametric methods: 8 12 4 0.38095238 0.10597117 0.22084536 0.6571327 11 8 2 0.28571429 0.09858079 0.14529127 0.5618552 12 6 2 0.19047619 0.08568909 0.07887014 0.4600116 15 4 1 0.14285714 0.07636035 0.05010898 0.4072755 17 3 1 0.09523810 0.06405645 0.02548583 0.3558956 22 2 1 0.04761905 0.04647143 0.00703223 0.3224544 (1) Kaplan-Meier 23 1 1 0.00000000 NA NA NA (2) Life-table (Actuarial Estimator) (3) via the Cumulative hazard estimator 1 2 (1) The Kaplan-Meier Estimator Motivation: First, consider an example where there is no censoring. The Kaplan-Meier (or KM) estimator is probably The following are times of remission (weeks) for 21 leukemia the most popular approach. It can be justified patients receiving control treatment (Table 1.1 of Cox & from several perspectives: Oakes): 1, 1, 2, 2, 3, 4, 4, 5, 5, 8, 8, 8, 8, 11, 11, 12, 12, 15, 17, 22, 23 • product limit estimator • likelihood justification How would we estimate S(10), the probability that an indi- vidual survives to time 10 or later? • redistribute to the right estimator What about S˜(8)? Is it 12 or 8 ? We will start with an intuitive motivation based 21 21 on conditional probabilities, then review some of Let’s construct a table of S˜(t): the other justifications. Values of t Sˆ(t) t ≤ 1 21/21=1.000 1 <t≤ 2 19/21=0.905 2 <t≤ 3 17/21=0.809 3 <t≤ 4 4 <t≤ 5 5 <t≤ 8 8 <t≤ 11 11 <t≤ 12 12 <t≤ 15 15 <t≤ 17 17 <t≤ 22 22 <t≤ 23 3 4 Empirical Survival Function: What if there is censoring? When there is no censoring, the general formula is: Consider the treated group from Table 1.1 of Cox and Oakes: # individuals with T ≥ t S˜(t)= total sample size 6+, 6, 6, 6, 7, 9+, 10+, 10, 11+, 13, 16, 17+ 19+, 20+, 22, 23, 25+, 32+, 32+, 34+, 35+ Example for leukemia data (control arm): [Note: times with + are right censored] We know S(6)= 21/21, because everyone survived at least until time 6 or greater. But, we can’t say S(7) = 17/21, because we don’t know the status of the person who was censored at time 6. In a 1958 paper in the Journal of the American Statistical Association, Kaplan and Meier proposed a way to nonpara- metrically estimate S(t), even in the presence of censoring. The method is based on the ideas of conditional proba- bility. 5 6 A quick review of conditional probability: Now, let’s apply these ideas to estimate S(t): Suppose ak <t≤ ak+1. Then Conditional Probability: Suppose A and B are two S(t)=P (T ≥ a ) events. Then, k+1 P (A ∩ B) P (A|B)= = P (T ≥ a1,T ≥ a2,...,T ≥ ak+1) P (B) k = P (T ≥ a1) × P (T ≥ aj+1|T ≥ aj) Multiplication law of probability: can be obtained j=1 from the above relationship, by multiplying both sides by k = [1 − P (T = aj|T ≥ aj)] P (B): j=1 P (A ∩ B)=P (A|B) P (B) k = [1 − λj] j=1 k d ˆ ∼ j Extension to more than 2 events: so S(t) = 1 − j=1 rj Suppose A1,A2...Ak are k different events. Then, the prob- dj ability of all k events happening together can be written as = 1 − j:a <t r a product of conditional probabilities: j j dj is the number of deaths at aj P (A1 ∩ A2... ∩ Ak)=P (Ak|Ak−1 ∩ ... ∩ A1) × rj is the number at risk at aj ×P (Ak−1|Ak−2 ∩ ... ∩ A1) ... ×P (A2|A1) ×P (A1) 7 8 Intuition behind the Kaplan-Meier Estimator 4 possibilities for each interval: Think of dividing the observed timespan of the study into a (1) No events (death or censoring) -conditional prob- series of fine intervals so that there is a separate interval for ability of surviving the interval is 1 each time of death or censoring: (2) Censoring -assume they survive to the end of the in- terval, so that the conditional probability of surviving D C C D D D the interval is 1 Using the law of conditional probability, (3) Death, but no censoring -conditional probability of not surviving the interval is # deaths (d) divided by # Pr(T ≥ t)= Pr(survive j-th interval Ij | survived to start of Ij) j ‘at risk’ (r) at the beginning of the interval. So the con- ditional probability of surviving the interval is 1−(d/r). where the product is taken over all the intervals including or preceding time t. (4) Tied deaths and censoring -assume censorings last to the end of the interval, so that conditional probability of surviving the interval is still 1 − (d/r) General Formula for jth interval: It turns out we can write a general formula for the conditional probability of surviving the j-th interval that holds for all 4 cases: d 1 − j rj 9 10 We could use the same approach by grouping the event times The Kaplan-Meier estimator of the survivorship into intervals (say, one interval for each month), and then function (or survival probability) S(t)=Pr(T ≥ t) counting up the number of deaths (events) in each to esti- is: mate the probability of surviving the interval (this is called the lifetable estimate). ˆ rj−dj S(t) = j:τ <t j rj However, the assumption that those censored last until the dj = j:τ <t 1 − end of the interval wouldn’t be quite accurate, so we would j rj end up with a cruder approximation. where As the intervals get finer and finer, the approximations made • τ1, ...τK is the set of K distinct death times observed in in estimating the probabilities of getting through each inter- the sample val become smaller and smaller, so that the estimator con- • d is the number of deaths at τ verges to the true S(t). j j • rj is the number of individuals “at risk” right before the j-th death time (everyone dead or censored at or after This intuition clarifies why an alternative name for the KM that time). is the product limit estimator. • cj is the number of censored observations between the j-th and (j + 1)-st death times. Censorings tied at τj are included in cj Note: two useful formulas are: (1) rj = rj−1 − dj−1 − cj−1 (2) rj = (cl + dl) l≥j 11 12 Calculating the KM - Cox and Oakes example KM plot for treated leukemia patients Make a table with a row for every death or censoring time: ˆ + τj dj cj rj 1 − (dj/rj) S(τj ) 18 63 1 21 21 = 0.857 71 0 17 90 1 16 10 11 13 16 17 19 20 22 23 Note: most statistical software packages summa- + rize the KM survival function at τj , i.e., just af- ter the time of the j-th failure. Note that: ˆ + In other words, they provide S(τj ). • Sˆ(t+) only changes at death (failure) times ˆ + • S(t ) is 1 up to the first death time When there is no censoring, the empirical survival estimate • Sˆ(t+) only goes to 0 if the last event is a death would then be: # individuals with T > t S˜(t+)= total sample size 13 14 Output from STATA KM Estimator: Two Other Justifications for KM Estimator failure time: weeks failure/censor: remiss I. Likelihood-based derivation (Cox and Oakes) Beg. Net Survivor Std. For a discrete failure time variable, define: Time Total Fail Lost Function Error [95% Conf. Int.] ------------------------------------------------------------------- 6 21 3 1 0.8571 0.0764 0.6197 0.9516 dj number of failures at aj 7 17 1 0 0.8067 0.0869 0.5631 0.9228 9 16 0 1 0.8067 0.0869 0.5631 0.9228 rj number of individuals at risk at aj 10 15 1 1 0.7529 0.0963 0.5032 0.8894 (including those censored at aj). 11 13 0 1 0.7529 0.0963 0.5032 0.8894 13 12 1 0 0.6902 0.1068 0.4316 0.8491 λj Pr(death) in j-th interval 16 11 1 0 0.6275 0.1141 0.3675 0.8049 (conditional on survival to start of interval) 17 10 0 1 0.6275 0.1141 0.3675 0.8049 19 9 0 1 0.6275 0.1141 0.3675 0.8049 20 8 0 1 0.6275 0.1141 0.3675 0.8049 22 7 1 0 0.5378 0.1282 0.2678 0.7468 The likelihood is that of g independent binomials: 23 6 1 0 0.4482 0.1346 0.1881 0.6801 g d 25 5 0 1 0.4482 0.1346 0.1881 0.6801 j rj−dj L(λ)= λj (1 − λj) 32 4 0 2 0.4482 0.1346 0.1881 0.6801 j=1 34 2 0 1 0.4482 0.1346 0.1881 0.6801 35 1 0 1 0.4482 0.1346 0.1881 0.6801 Therefore, the maximum likelihood estimator of λj is: ˆ λj = dj/rj Now we plug in the MLE’s of λ to estimate S(t): ˆ ˆ S(t)= (1 − λj) j:aj<t dj = 1 − j:aj<t rj 15 16 II.