Systematic Reviews of Experimental Animal Studies
Total Page:16
File Type:pdf, Size:1020Kb
mily Sena, Gillian Currie, Hanna Vesterinen, Kieren Egan, Nicki Sherratt, Cristina Fonseca, Zsannet Bahor, Jing Liao, heo Hirst, Kim Wever, Hugo Pedder, Katerina Kyriacopoulou, Julija Baginskaite, Ye Ru, Stelios Serghiou, Aaron cLean, Catherine Dick, Tracey Woodruff, Patrice Sutton, Andrew Thomson, Aparna Polturu, Sarah MaCann, Gillian ead, Joanna Wardlaw, Rustam Salman, Joseph Frantzias, Robin Grant, Paul Brennan, Ian Whittle, Andrew Rice, osie Moreland, Nathalie Percie du Sert, Paul Garner, Lauralyn McIntyre, Gregers Wegener, Lindsay Thomson, David owells, Ana Antonic, Tori O’Collins, Uli Dirnagl, H Bart van der Worp, Philip Bath, Mharie McRae, Stuart Allan, Ian arshall, Xenios Mildonis, Konstantinos Tsilidis, Orestis Panagiotou, John Ioannidis, Peter Batchelor, David Howells, anne Jansen of Lorkeers, Geoff Donnan, Peter Sandercock, Emily Sena, Gillian Currie, Hanna Vesterinen, Kieren gan, Nicki Sherratt, Cristina Fonseca, Zsannet Bahor, Jing Liao, Theo Hirst, Kim Wever, Hugo Pedder, Katerina yriacopoulou, Julija Baginskaite, Ye Ru, Stelios Serghiou, Aaron McLean, Catherine Dick, Tracey Woodruff, Patrice utton, Andrew Thomson, Aparna Polturu, Sarah MaCann, Gillian Mead, Joanna Wardlaw, Rustam Salman, Joseph antzias, Robin Grant, Paul Brennan, Ian Whittle, Andrew Rice, Rosie Moreland, Nathalie Percie du Sert, Paul Garner, auralyn McIntyre, Gregers Wegener, Lindsay Thomson, David Howells, Ana Antonic, Tori O’Collins, Uli Dirnagl, H Bart n der Worp,Systematic Philip Bath, Mharie McRae, reviewsStuart Allan, Ian Marshall, of Xeniosexperimental Mildonis, Konstantinos Tsilidis, Orestis anagiotou, John Ioannidis, Peter Batchelor, David Howells, Sanne Jansen of Lorkeers, Geoff Donnan, Peter andercock, Emily Sena, Gillian Currie, Hanna Vesterinen, Kieren Egan, Nicki Sherratt, Cristina Fonseca, Zsannet ahor, Jing Liao, Theo Hirst, Kim Wever, Hugo Pedder, Katerina Kyriacopoulou, Julija Baginskaite, Ye Ru, Stelios animal studies erghiou, Aaron McLean, Catherine Dick, Tracey Woodruff, Patrice Sutton, Andrew Thomson, Aparna Polturu, Sarah aCann, Gillian Mead, Joanna Wardlaw, Rustam Salman, Joseph Frantzias, Robin Grant, Paul Brennan, Ian Whittle, ndrew Rice, Rosie Moreland, Nathalie Percie Malcolmdu Sert, Paul Macleod Garner, Lauralyn McIntyre, Gregers Wegener, Lindsay homson, David Howells, Ana Antonic, Tori O’Collins, Uli Dirnagl, H Bart van der Worp, Philip Bath, Mharie McRae, uart Allan, Ian Marshall,Collaborative Xenios Mildonis, Approach Konstantinos to Tsilidis, Meta -OrestisAnalysis Panagiotou, and Review John Ioannidis, of Peter Batchelor, avid Howells, Sanne Jansen of Lorkeers, Geoff Donnan, Peter Sandercock, Emily Sena, Gillian Currie, Hanna esterinen, Kieren Egan, Nicki Sherratt,Animal Cristina Data Fonseca, from Experimental Zsannet Bahor, StudiesJing Liao, Theo Hirst, Kim Wever, Hugo edder, Katerina Kyriacopoulou, Julija Baginskaite, Ye Ru,and Stelios Serghiou, Aaron McLean, Catherine Dick, Tracey oodruff, Patrice Sutton, Andrew Thomson, Aparna Polturu, Sarah MaCann, Gillian Mead, Joanna Wardlaw, Rustam alman, Joseph Frantzias, Robin Grant, PaulUniversity Brennan, Ian of Whittle, Edinburgh Andrew Rice, Rosie Moreland, Nathalie Percie du ert, Paul Garner, Lauralyn McIntyre, Gregers Wegener, Lindsay Thomson, David Howells, Ana Antonic, Tori O’Collins, i Dirnagl, H Bart van der Worp, Philip Bath, Mharie McRae, Stuart Allan, Ian Marshall, Xenios Mildonis, Konstantinos ilidis, Orestis Panagiotou, John Ioannidis, Peter Batchelor, David Howells, Sanne Jansen of Lorkeers, Geoff Donnan, eter Sandercock, Emily Sena, Gillian Currie, Hanna Vesterinen, Kieren Egan, Nicki Sherratt, Cristina Fonseca, sannet Bahor, Jing Liao, Theo Hirst, Kim Wever, Hugo Pedder, Katerina Kyriacopoulou, Julija Baginskaite, Ye Ru, elios Serghiou, Aaron McLean, CatherineCAMARADES: Dick, Tracey BringingWoodruff, Patrice evidence Sutton, toAndrew translational Thomson, Aparna medicine Polturu , Disclosures • Member of UK Home Office Animals in Science Committee (Convener of 3Rs subgroup) • Member, UK Commission for Human Medicines, MHRA • Have sought and will seek funding for work in this area CAMARADES: Bringing evidence to translational medicine I am not in the office at the moment. Send any work to be translated. CAMARADES: Bringing evidence to translational medicine Translational Medicine 101 Definition: using information from one research domain to guide research in a different research domain Context: Many proposals for clinical trials claim some justification from supporting animal data CAMARADES: Bringing evidence to translational medicine Winner of the 2012 Ignoble Prize for Neuroscience CAMARADES: Bringing evidence to translational medicine Systematic review • A review article where criteria for identifying and considering information are determined in advance and are transparent • Contrasts with, – and is less biased than – narrative reviews • Provides additional insights to assessments of “biological truth” CAMARADES: Bringing evidence to translational medicine Systematic reviews v narrative reviews • House dust mites and asthma – 63 of 70 review articles claimed efficacy for physical eradication measures (vacuum cleaning, bed covers, freezing …) – Most frequently cited study had 7 patients per group – Systematic review (Cochrane) identified 28 trials involving 939 patients • Found no effect of physical measures in improving outcome Schmidt and Gotzsche, 2005 J Fam Practice “Authors often use non randomised studies to create a false impression of consensus” CAMARADES: Bringing evidence to translational medicine CAMARADES: Bringing evidence to translational medicine Meta-analysis • A statistical technique to combine data from separate experiments – To give an overall “best estimate” of a biological effect – To understand the impact of other things on that effect • Related to the exposure (dose, time …) • Related to the animal (age, sex, comorbidity …) • Related to experimental design CAMARADES: Bringing evidence to translational medicine CAMARADES: Bringing evidence to translational medicine You can usually find what you’re looking for … • 12 graduate psychology students • 5 day experiment: rats in T maze with dark arm alternating at random, and the dark arm always reinforced • 2 groups – “Maze Bright” and “Maze dull” Group Day Day Day Day Day 1 2 3 4 5 “Maze 1.33 1.60 2.60 2.83 3.26 bright” “Maze 0.72 1.10 2.23 1.83 1.83 dull” Δ +0.60 +0.50 +0.37 +1.00 +1.43 Rosenthal and Fode (1963), Behav Sci 8, 183-9 CAMARADES: Bringing evidence to translational medicine Bias in ischaemia studies • Infarct Volume – 11 publications, 29 experiments, 408 animals – Improved outcome by 44% (35-53%) Efficacy Randomisation Blinded conduct Blinded of experiment assessment of outcome Macleod et al, 2008 CAMARADES: Bringing evidence to translational medicine Lessons from other neuroscience domains Multiple Sclerosis Alzheimer´s disease 1.2 AD 1.0 0.8 0.6 0.4 0.2 (Standardised Effect Size) Improvement in behavioural outcome 0.0 No Yes Blinded assessment of behavioural outcome Parkinson´s disease CAMARADES: Bringing evidence to translational medicine Associations between quality of reporting and observed effect sizes FK506 Hypothermia Number of study quality checklist items scored CAMARADES: Bringing evidence to translational medicine Risk of Bias Slide from Tracey Woodruff/ Navigating the Science Sequence generation Low risk Allocation concealment Probably low risk Blinding Probably high risk Incomplete outcome data High risk Selective reporting Other bias Conflict of interest 0% 20% 40% 60% 80% 100% CAMARADES: Bringing evidence to translational medicine The scale of the problem RAE 1173 “an outstanding contribution to the internationally excellent position of the UK in biomedical science and clinical/translational research.” “impressed by the strength within the basic neurosciences that were returned …particular in the areas of behavioural, cellular and molecular neuroscience” 1173 publications using non human animals, published in 2009 or 2010, from 5 leading UK universities Rand Blind I/E SSC CAMARADES: Bringing evidence to translational medicine Publication bias in toxicology studies CAMARADES: Bringing evidence to translational medicine Different patterns of publication bias in different fields outcome observed corrected Disease improvement 40% 30% Less models improvement Toxicology harm 0.32 0.56 More harm model Harm Benefit CAMARADES: Bringing evidence to translational medicine How big a hole does this make in published research? Risk of Bias Publication Bias Underpowered experiments CAMARADES: Bringing evidence to translational medicine Technical reflections • The most important thing we will find out is about risk of bias in this research domain • Fixed effects v random effects • Weighted mean difference v standardised mean difference • Stratified meta-analysis v meta-regression • Univariate v multivariate meta-regression • Tau estimation CAMARADES: Bringing evidence to translational medicine Approaches to meta-analysis • If you expect studies to have roughly the same result, weight according to inverse variance – Fixed effects meta-analysis (1/sd2) • If you expect studies to have varying results, blunt FE weighting according to the extent of differences – Random effects meta-analysis (1/(sd2 + tau2)) • Explore differences between studies by exploring variablity – Overall – Within studies with shared characteristics – Between groups of studies with different characteristics CAMARADES: Bringing evidence to translational medicine Partitioning heterogeneity Total