[Updated 2012] Statistics Guide for Research Grant Applicants Authors (in alphabetical order) Bland JM, Butland BK, Peacock JL, Poloniecki J, Reid F, Sedgwick P Department of Public Health Sciences St George's Hospital Medical School Cranmer Terrace London SW17 0RE This handbook was funded by the South East Regional Office (SERO) Research and Knowledge Management Directorate and is targeted at those applying for research funding, from any source. Table of Contents Preface Background Content Using the handbook Statistical review Development A Describing the study design Introduction A-1.........................................................Type of study A-1.1......................................................Type of study: Observational or experimental A-1.2......................................................Combinations and sequences of studies A-1.3......................................................Cohort studies A-1.4......................................................Case-control studies A-1.5......................................................Cross-sectional studies A-1.5a....................................................Prevalence studies A-1.5b....................................................The estimation of sensitivity and specificity A-1.5c....................................................When to calculate sensitivity and specificity A-1.5d....................................................Cross-sectional ecological studies A-1.5e....................................................Studies of measurement validity, reliability and agreement A-1.6......................................................Confounding A-1.6a....................................................Confounding or interaction A-1.7......................................................Experiments and trials A-1.8......................................................Randomised controlled clinical trials A-1.9......................................................Pilot and Exploratory Studies A-2.........................................................Follow-up A-3.........................................................Study subjects A-4.........................................................Types of variables A-4.1......................................................Scales of measurement A-4.2......................................................Types of data A-4.3......................................................Methods of data collection A-4.4......................................................Validity and reliability A-4.4a....................................................Validity A-4.4b....................................................Repeatability (test-re-test reliability) A-4.4c....................................................Inter-rater reliability (inter-rater agreement) B Clinical trials B-1.........................................................Describing the patient group / eligibility criteria B-2.........................................................Describing the intervention (s) / treatment (s) B-3.........................................................Choice of control treatment / need for control group B-4.........................................................Blindness B-4.1......................................................Double blind and single blind designs B-4.2......................................................Placebos B-5.........................................................Randomisation B-5.1......................................................What is randomisation? B-5.2......................................................When might we use randomisation? B-5.3......................................................Why randomise? B-5.4......................................................What is not randomisation? B-5.5......................................................How do we randomise? 2 B-5.6......................................................Randomisation in blocks B-5.7......................................................Randomisation in strata B-5.8......................................................Minimisation B-5.9......................................................Clusters B-5.10....................................................Trial designs B-5.10a..................................................Parallel groups B-5.10b..................................................Crossover design B-5.10c..................................................Within group comparisons B-5.10d..................................................Sequential design B-5.10e..................................................Factorial design B-6.........................................................Outcome variables B-7.........................................................Data monitoring B-7.1......................................................Data monitoring committee B-7.2......................................................When should a trial be stopped early? B-8.........................................................Informed consent B-8.1......................................................Consent B-8.2......................................................Emergencies B-8.3......................................................Children B-8.4......................................................Mentally incompetent subjects B-8.5......................................................Cluster randomised designs B-8.6......................................................Randomised consent designs B-9.........................................................Protocol violation and non-compliance B-10.......................................................Achieving the sample size B-11.......................................................After the trial is over C Observational studies C-1.........................................................Case-control studies C-1.1......................................................Choice of control group in case-control studies C-1.2......................................................Matching in case-control studies C-2.........................................................Assessment bias C-3.........................................................Recall bias C-4.........................................................Sample survey: selecting a representative sample C-5.........................................................Generalisability and extrapolation of results C-6.........................................................Maximising response rates to questionnaire surveys D Sample size calculation D-1.........................................................When should sample size calculations be provided? D-2.........................................................Why is it important to consider sample size? D-3.........................................................Information required to calculate a sample size D-4.........................................................Explanation of statistical terms D-4.1......................................................Null and alternative hypothesis D-4.2......................................................Probability value (p-value) D-4.3......................................................Significance level D-4.4......................................................Power D-4.5......................................................Effect size of clinical importance D-4.6......................................................One-sided and two-sided tests of significance D-5.........................................................Which variables should be included in the sample size calculation? D-6.........................................................Allowing for response rates and other losses to the sample D-7.........................................................Consistency with study aims and statistical analysis 3 D-8.........................................................Three specific examples of sample size calculations & statements D-8.1......................................................Estimating a single proportion D-8.2......................................................Comparing two proportions D-8.3......................................................Comparing two means D-9.........................................................Sample size statements likely to be rejected E Describing the statistical methods E-1.........................................................Introduction E-1.1......................................................Terminology E-1.2......................................................Level of detail required E-2.........................................................Is the proposed method appropriate for the data? E-2.1......................................................'Ordinal' scores E-3.........................................................Paired and unpaired comparison E-4.........................................................Assumptions E-4.1......................................................Transformations E-5.........................................................Adjustment for confounding E-6.........................................................Hierarchical or multilevel data? E-6.1......................................................Analysing hierarchical data E-7.........................................................Multiple testing E-7.1......................................................Multiple testing: when does it arise? E-7.2......................................................Multiple