Stats Moment: Confidence Interval

 Quantifies the uncertainty in measurement.  Is a measure of the precision of a sample estimate.  For data of normal distribution, it is calculated by calculating the standard error of the mean… . SE = SD/ √n (n=sample size) and then multiplying that number by 1.96  The value of 1.96 is based on the fact that 95% of the area of a normal distribution is within 1.96 standard deviations of the mean

 A confidence interval is always qualified by a particular confidence level, usually expressed as a percentage (ie. 95% confidence interval).  The end points of the confidence interval are referred to as confidence limits. For a given estimation procedure in a given situation, the higher the confidence level, the wider the confidence interval will be  When reported as 95% CI: If this point estimate in this study is the true value, and the study were repeated many times, in 95% of those repetitions, the result would fall within those confidence intervals. . Example: For an absolute risk reduction of 10% with a 95% confidence interval of 5%-15%, we would know that if we repeated the study many times, 95% of the time, the true ARR would be between 5% and 15%.

 A major factor determining the width of a confidence interval is the size of the sample used in the estimation procedure . “Was the trial large enough to answer our clinical question?”

“Positive” Trials:

In trials that report a positive difference between the treatment group and the control group, we need to examine the lower-limit confidence interval. If this value lies above the point that we deem to be clinically significant (ie: a 1% absolute risk reduction in mortality), then we would deem the study to be large enough to definitively answer our question.

Conversely, if the lower-limit confidence interval lies below the point that we deem to be clinically significant, then we would interpret the study as having a sample size that is too small to adequately answer our question.

“Negative” Trials:

In trials that report no difference between the treatment and the control group, we need to examine the upper-limit confidence interval. If this value lies below the point that we deem to be clinically significant (ie:excludes any important clinical benefit), then we can conclude that the trial is definitively negative. If the upper-limit confidence interval lies above a value that is still of clinical significance/benefit, then the study sample is not large enough to rule out the possibility of benefit with treatment. In the above figure, assume that a 1% ARR is the smallest benefit that patients would consider important enough to warrant undergoing treatment. The uppermost point of the bell curve is the observed treatment effect (the point estimate), and the tails of the bell curve represent the boundaries of the 95% CI.

Given the information in Fig A: a) Would you recommend this treatment to your patient if the point estimate represented the truth? b) What about if the upper boundary of the CI represented the truth? c) What if the lower boundary of the CI represented the truth? In Fig B: a) Would your patients choose to undergo the treatment if either the point estimate represented the true effect? b) What about if the upper boundary or the lower boundary of the CI represented the truth? In Fig C: a) In trial 3, is the sample size adequate and is the trial truly “negative”? b) How about in trial 4?

Estimating Confidence Intervals: Extreme Proportions Often, journal articles will report results with numerators of ‘0’ (eg: outcome of interest 0/20) but omit confidence intervals. The upper-limit confidence interval can be estimated by the formula: 3/n (where n is the sample size).

Conversely, when the numerator is very close to the denominator (eg: outcome of interest 20/20) and there is no confidence interval, the lower-limit confidence interval can be calculated by using 3/n to calculate the complement of the lower-limit confidence interval. Then, you can obtain the lower-limit confidence interval by: 100% - 3/n (where n is the sample size).

Back to our studies: Are the articles we reviewed truly “negative” trials?

In the Drendel et al. study a) What is the upper limit of the confidence interval for the primary outcome? b) Would this fall below the smallest difference that patients might consider important? c) What % reduction was thought to be clinically important prior to beginning the study d) Is this a definitely negative study?

In the Abu-Laban et al. study a) What is the upper limit of the confidence interval for the primary outcome? b) Would this fall below the smallest difference that patients might consider important? c) Is this a definitively negative study?