This thesis has been submitted in fulfilment of the requirements for a postgraduate degree (e.g. PhD, MPhil, DClinPsychol) at the University of Edinburgh. Please note the following terms and conditions of use: This work is protected by copyright and other intellectual property rights, which are retained by the thesis author, unless otherwise stated. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This thesis cannot be reproduced or quoted extensively from without first obtaining permission in writing from the author. The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the author. When referring to this work, full bibliographic details including the author, title, awarding institution and date of the thesis must be given. Systematic review and meta-analysis of experimental multiple sclerosis studies Hanna Mikaela Vesterinen BSc (Hons) PhD by Research 2013 Abstract Background: Multiple sclerosis (MS) is the most common cause of disability in young people and yet there are no interventions available which reliably alter disease progression. This is despite several decades of research using the most common animal model of multiple sclerosis, experimental autoimmune encephalomyelitis (EAE). There is now emerging evidence across the neurosciences to suggest that limited internal validity (measures to reduce bias) and external validity (e.g. using a clinically relevant animal model) may influence the translational success. Aim and objectives: To provide an unbiased summary of the scope of the literature on candidate drugs for MS tested in EAE to identify potential reasons for the failures to translate efficacy to clinical trials. My objectives were, across all of the identified publications, to: (1) describe the reporting of measures to reduce bias and to assess their impact on measures of drug efficacy; (2) assess the relationship between treatment related effects measured using different outcome measures; (3) assess the prevalence and impact of any publication bias; (4) compare findings from the above with another disease with limited translational success (Parkinson’s disease; PD). Methods: I used systematic searches of three online databases to identify relevant publications. Estimates of efficacy were extracted for neurobehavioural scores, inflammation, demyelination and axon loss. For PD experiments, we searched for dopamine agonists tested in animal models of PD with outcome assessed as change in neurobehavioural scores. I calculated normalised mean difference or standardised mean difference effect sizes and combined these in a meta-analysis using a random effects model. I used stratified meta-analysis or meta-regression to assess the extent to which different study design characteristics explained differences in reported efficacies. These characteristics included: measures to reduce bias (random allocation to group and blinded assessment of outcome), the animal species, sex, time of drug administration, route of drug administration and the number of animals per group. Publication bias was assessed using funnel plotting, Egger regression and “trim and fill”. Results: I identified 1464 publications reporting drugs tested in EAE. Reported study quality was poor: 11% reported random allocation to group, 17% reported blinded i assessment of neurobehavioural outcomes, 28% reported blinded assessment of histological outcomes, and <1% reported a sample size calculation. Estimates of efficacy measured as the reduction in inflammation were substantially higher in un- blinded studies (47.1% reduction (95% CI 41.8-52.4)) versus blinded studies (33.1% (25.8-40.4). Moreover, the same finding was identified for 121 publications on dopamine agonists tested in experimental PD models where efficacy was measured as change in neurobehavioural outcomes. For EAE studies we were unable to include data from 631 publications describing original research. Usually this was because the publication did not include basic details such as the number of animals in each group (115 publications), the observed variance (592) or suitable control data (49). For each category of outcome I found evidence of a substantial publication bias. Interventions were most commonly administered on or before the induction of EAE with shorter times to treatment associated with higher estimates of efficacy for the reduction in mean severity scores (a neurobehavioural outcome). Treatment related effects were found to vary across different outcome measures with the largest effect being for the reduction in axon loss. Where neurobehavioural scores and axon loss were measured in the same cohort of animals, the concordance between efficacies in these increased with later times to treatment. Conclusions: In this, the largest systematic review and meta-analysis of animal studies in any domain, I have found that a large number of publications present incomplete data. In addition, measures to reduce bias are seldom reported, the lack of which is associated with overstatements of efficacy for both a measure of drug efficacy in EAE and experimental PD studies. Translational success may have also been affected by the majority of studies administering drugs on or before EAE induction which is of limited relevance in the clinical setting where patients do not present at that stage of disease. Moreover, my analysis of the relationship between outcome measures provides empirical evidence from systematically identified studies to suggest that targeting axon loss as later time points is most strongly associated with improvements in neurobehavioural scores. Therefore drugs which are successfully able to target axon loss at these time points may offer substantial hope for clinical success. Overall, improvements in the conduct and reporting of preclinical studies are likely to improve their utility, and the prospects for translational success. While my findings relate predominately to the animal modelling of MS and PD it is likely that they also hold for other animal research. ii Declaration I hereby declare that all of the material presented in this thesis is my own work. Contributions made by others are stated in the relevant sections of this thesis. This thesis has not been submitted for any other degree or qualification. Signed.............................................................................................. Date…………………………………………. iii iv Publications arising from the work presented in this thesis ROOKE, E.D.M., VESTERINEN, H.M., SENA, E.S., EGAN, K. J. & MACLEOD, M.R. 2011. Dopamine agonists in animal models of Parkinson's disease: A systematic review and meta-analysis. Parkinsonism & Related Disorders, 17, 313-320. VESTERINEN, H.M., SENA, E.S., FFRENCH-CONSTANT, C., WILLIAMS, A., CHANDRAN, S. & MACLEOD, M.R. 2010. Improving the translational hit of experimental treatments in multiple sclerosis. Multiple Sclerosis Journal, 16, 1044-1055. VESTERINEN, H.M., EGAN, K., DEISTER, A., SCHLATTMANN, P., MACLEOD, M.R. & DIRNAGL, U. 2011. Systematic survey of the design, statistical analysis, and reporting of studies published in the 2008 volume of the Journal of Cerebral Blood Flow and Metabolism. Journal of Cerebral Blood Flow and Metabolism, 31, 1064-1072. VESTERINEN, H.M., SENA, E.S., EGAN, K.J., HIRST, T., CURRIE, G., ANTONIC, A., HOWELLS, D.W. & MACLEOD, M.R. Meta-analysis of data from animal studies: a practical guide. Under review. VESTERINEN, H.M., SENA, E.S. & MACLEOD M.R. The prevalence and impact of study quality, publication bias and incomplete reporting in experimental autoimmune encephalomyelitis. Under review. VESTERINEN, H.M., SENA, E.S. & MACLEOD M.R. The relationship between treatment related effects on neurobehavioural and histological outcomes in EAE: a systematic approach to regression and path analyses. Manuscript in preparation. v Acknowledgements I am extremely grateful to Malcolm Macleod for giving me this opportunity which has been so rewarding, for excellent supervision, for always having an answer for my most difficult questions, and for continually reminding me that data are plural. Emily Sena, I am grateful for all of the following and more: word of the day, semi- colons, correcting my split infinitives, and for hosting such good parties. Thank you especially for reading and re-reading my manuscripts, proof reading this thesis and for all of the many hours spent teaching me how to use the database, run analyses, etc... I couldn’t have got through this PhD without this support. The benchmark of publishing 18 manuscripts during a PhD was too much to match, but was inspirational nonetheless. This PhD was funded by the Centre for Clinical Brain Sciences PhD scholarship, and part of the research was funded by the Multiple Sclerosis Society of Great Britain. The financial support is much appreciated. In addition I am grateful to the following: my second supervisor, Anna Williams for sharing her expertise on EAE and MS; Kieren Egan for sharing the highs and lows of doing a PhD and for the fun we had at two conferences in Barcelona; Siddharthan Chandran and Charles ffrench-Constant for helpful discussions; Aidan Hutchison for teaching me to use Microsoft Access and for fixing it every time I broke it; Deirdre Beecher and Filippino Graziella from MS Cochrane; Professor Martino for kindly letting me observe the EAE experiments in his lab and to all the people that made my stay in Milan so enjoyable; Helen Cullion, Rachel Burrow, Judi Clarke,