Understanding Statistics 3

Home , Confidence interval, Standard error, Statistical significance, Statistics

research/toolboxtoolbox

Understanding statistics 3

Elliot Abt Department of Dentistry, Illinois Masonic Medical Center, Chicago, Illinois, USA

The last article in this series described the dif- Table 1 Type I and Type II errors ferent types and nature of data used in quan- Null hypothesis (h ) is true alternative hypothesis (h ) is true titative research. This article will define and 0 1 examine the relationships between P-values, Fail to reject Right decision Wrong decision Null hypothesis Type II Error confidence intervals and sample size. False Negative reject Wrong decision Right decision P-values Null hypothesis Type I Error False Positive Central to the theme of hypothesis testing is the P-value, which is often misunderstood or misinterpreted. While it is often stated as have argued against this all or none phenome- variability in a sample, and generally does the probability of the data having arisen by non of statistical significance.2 At the very least, not change as the sample size increases chance, the P-value is actually the probabili- a study should report a specific P-value (i.e. n = sample size ty of obtaining the observed effect (or a more p=0.03) as this will provide a better estimate CI = confidence interval extreme value), given the null hypothesis is of the strength of the evidence against the null = mean or point estimate true.1 Traditionally, the level of significance hypothesis. Unfortunately, what P-values don’t is set at 5% (0.05), as at this level we are no tell us is the direction of the effect (is drug A This is the equation for a 95% CI which longer 'comfortable' that an observed effect better than B, or vice versa) and little about the corresponds to setting the level of statisti- is due to chance, but this is arbitrary, and it magnitude of the effect. For that, confidence cal significance at 5%. Other CIs can be is occasionally set at 1% or even 10%. intervals (CIs) are required. calculated for different levels of statisti- If P<0.05, this tells us that either two cal significance. These are basic equations groups are different, or perhaps that there is Intervals and sample size which change with increasing complexity of a 1 in 20 chance we are making an error. This Many are familiar with descriptive statistics; statistical tests. is referred to as a type I error, or a 'false posi- the presentation, organization, and tive', where we falsely reject the null hypoth- summarization of data such as simple P-values, confidence intervals, and esis. Conversely, if P>0.05, either there are percentages. Inferential statistics involve the null value no differences between groups, or we have extrapolating data from a sample to a A relationship exists between P-values, made a type II error, or falsely accepted the population, and CIs give readers a range confidence intervals and the null value. null hypothesis (table 1). This is often due to of values where we are confident the true If the 95% CI includes the null value, by a study which is underpowered, or lacking population lies.3 CIs are derived from the definition, then P>0.05. If the 95% CI does enough participants to show a difference standard error (SE), which, in turn is derived not include the null value, then P<0.05. It between groups if one actually exists. One from the sample size. That is, the larger the should be noted that the null value is not could question the ethics of conducting a study sample, the smaller the SE, and the always “0”. It varies depending on what type trial which does not have enough subjects to narrower the CI. This may be somewhat of data are used in a study. show differences (or associations) between intuitive, that the more participants, the The next article in this series will begin to groups. In studies with excessive numbers of more confident we are in the study results. explore basic statistical tests and give examples participants, very small differences between Thus, as sample size increases, CIs narrow, of dental studies using different data types. It groups (around 1%) are likely to be statisti- providing a more precise estimate of effect. will provide assistance with statistical analysis, cally significant, although not necessar- The relationship between sample size and CI s including interpretation of CIs and their effect ily clinically relevant. Thus, readers of the can be illustrated in the following two (basic) on clinical relevance. literature should look for a power (sample equations: size) calculation to estimate an appropriate SE = s ÷ √n 1. Altman DG. Practical Statistics for Medical Research. Section 8.5; Chapman and Hall/CRC. number of study participants. 95% CI = ± 1.96(SE) 2. Guyatt G, Jaeschke R, Heddle N, Cook D, Shannon Many practitioners have been trained to SE = standard error - which measures the H, and Walter S. Basic statistics for clinicians: 1. Hypothesis testing. CMAJ 1995;152: 27-32. see if the P-value is greater or less than 0.05. amount of variability in the sample mean 3. Kirkwood BR, and Sterne JAC. Essential Medical Statistics. 2nd Edition. Ch. 8; Blackwell Science Technically, P-values of 0.049 and 0.051 would and indicates how accurately the sample be on opposite sides of statistical significance, mean represents the mean of a population Evidence-Based Dentistry (2010) 11, 118. when, in fact, they are equivalent, and some s = standard deviation - describes the doi:10.1038/sj.ebd.6400761