What is ...? For further titles in the series, visit: www.whatisseries.co.uk What is meta-analysis? Robert Hodgson PhD Research Consultant, York Health Economics Consortium, University of York G Meta-analysis is a set of statistical G Meta-analysis of trials provides more techniques for combining data from precise estimates of treatment effect, by independent studies to produce a single making use of all available data. estimate of effect. G Meta-analysis is often part of the systematic G Meta-analysis is often used review process, many systematic reviews within healthcare, but is also applied include one or more meta-analyses. in other disciplines including psychology and the social sciences. G The validity of any meta-analysis depends on the studies on which it is based. G Within healthcare, meta-analysis is often used to assess the clinical G Well-conducted meta-analyses aim for effectiveness of interventions; it does complete coverage of all relevant studies, this by combining data from two or look for the presence of heterogeneity more studies (usually randomised among studies, and explore the robustness of controlled trials). the main findings using sensitivity analysis. EVIDENCE-BASED HEALTHCARE • theory • knowledge • practice Copyright © Hayward Medical Communications 2014. All rights reserved. No unauthorised reproduction or distribution. For reprints or permissions, contact [email protected] What is ...? What is meta-analysis? Systematic reviews difficulties that anyone trying to make sense and meta-analysis of effectiveness research might encounter. It is common in medical research to find that several clinical trials have attempted to Small effects and trends answer similar questions about clinical Because meta-analysis may combine data effectiveness; for example: Does the new from a number of studies, it potentially treatment confer significant benefits includes a large number of patients and compared with the conventional may be better placed to examine rare treatment? Often, it is the case that these events, such as safety issues and side effects individual studies will not be able to associated with interventions. The larger conclusively demonstrate whether a sample size achieved by combining a treatment confers significant clinical benefit. number of studies also allows investigators Meta-analysis allows the data from all to examine any differences in intervention relevant trials to be combined and can help effects between specific patient groups, to establish with greater certainty the something that is often not possible within relative clinical benefit of the treatment individual studies. options. A good example of this is a retrospective review of the evidence on the Inconsistency effectiveness of thrombolytic therapy for the The results of studies evaluating a particular prevention of myocardial infarction.1 The intervention can often appear to be study showed that, had meta-analysis been inconsistent or contradictory. Meta-analysis conducted at an early stage, after only a few provides a way of not only estimating trials had been completed, it would have the average likely effect, but also a way demonstrated the benefits of thrombolytic to quantify and explain inconsistencies. therapy. Instead, experts remained unaware This can be achieved through the of its benefits for many years and patients examination of subgroups, as described were not given an effective therapy. Meta- above, or through use of the technique of analyses are now a fundamental part of meta-regression (see below). evidence-based medicine. Overcoming bias When can you When meta-analysis is undertaken as part of do a meta-analysis? a rigorously conducted systematic review, it Meta-analysis can be used whenever there can also help to overcome bias. A well- is more than one study that has estimated conducted systematic review allows data the effect of an intervention or risk factor, from all relevant studies to be included in a and the studies are sufficiently similar in meta-analysis. A meta-analysis carried out as terms of the participants, interventions, part of such a review therefore allows for outcome measurements and settings, so unbiased synthesis of the empirical data. that it is reasonable to combine the results Unsystematic reviews are less likely to of these studies. Meta-analysis need not identify all available evidence and may be only be used to combine the results of influenced by the prior beliefs of the randomised controlled trials (RCTs), but can reviewer. Meta-analysis carried out as part of also be used to combine data from other an unsystematic review is therefore more types of studies, such as case-control and likely to be based on only a proportion of cohort studies to give a relative risk, or the available evidence and also more likely diagnostic test accuracy studies to provide to produce biased estimates of effect. summary estimates of the sensitivity and specificity of a new test compared with an Transparency existing test. As well as potentially reducing bias, meta- analyses conducted as part of a rigorous Benefits of meta-analyses systematic review have the further Meta-analysis offers a rational and helpful advantage of transparency. An important way of dealing with a number of practical aspect of the systematic review approach is 2 www.whatisseries.co.uk Copyright © Hayward Medical Communications 2014. All rights reserved. No unauthorised reproduction or distribution. For reprints or permissions, contact [email protected] What is ...? What is meta-analysis? that all the decisions that have been taken (such as funnel plots) and a greater throughout the process of achieving the understanding of the pitfalls of meta- final aggregate effect sizes are made clear. analysis.5 This allows readers to determine for themselves the reasonableness of the Meta-analysis and the decisions taken and their likely impact on systematic review process the final estimate of effect size. It also allows The main requirement for a worthwhile other researchers to replicate their results, meta-analysis is a well-conducted systematic which is an important part of the scientific review.6 Such a review should include an process. extensive search, which interrogates several electronic databases and other information Criticisms of meta-analysis sources to identify as many relevant studies A common criticism of meta-analysis is that as possible. The systematic review should it seeks to combine different kinds of studies also use explicit and objective criteria for the in the same analysis, thereby ignoring inclusion or rejection of studies.7 However important differences between studies. competent the meta-analysis, if the original Careful consideration is necessary when systematic review was partial, flawed or planning a meta-analysis to ensure that the otherwise unsystematic, then the meta- studies included in the analysis are analysis may provide a precise quantitative sufficiently similar so that that any resulting estimate that is simply wrong. The main summary estimate has a clinically requirement of systematic review is easier to meaningful interpretation; for example, it state than to execute: a complete, unbiased makes no sense to combine studies that collection of all the original studies of compare different interventions. It is, acceptable quality that examine the same however, worth emphasising that one of the therapeutic question. Among the many strengths of meta-analysis is its ability to checklists for the assessment of the quality answer broader questions and allow of systematic reviews, the PRISMA statement inconsistencies to be investigated. (Preferred Reporting Items for Systematic Therefore, combining dissimilar studies may Reviews and Meta-Analyses) is particularly be justified when attempting to explain recommended and widely adopted by variations in clinical effectiveness. journal editors.8 Another criticism levelled at meta-analysis is that the results of meta-analyses can Conducting meta-analyses appear to contradict those of subsequent Forest plots RCTs. A number of studies have, for The usual way of displaying data from a example, compared the results of meta- meta-analysis is by a pictorial representation analyses with subsequent findings from (often known as a Forest plot). An example large-scale, well-conducted, RCTs (so-called is shown in Figure 1.9 This displays the ‘mega trials’). The results of such findings from each individual study as a comparisons have, so far, been mixed – blob or square, with squares towards the left there has been good agreement in the side indicating that the new treatment is majority of cases but some discrepancies in better, whereas blobs on the right indicate others.2,3 For example, one such exercise led that the new treatment is less effective than to publication of a paper subtitled ‘Lessons the comparator. The size of the blob or from an “effective, safe, simple intervention” square is proportional to the precision of the that wasn’t’ (use of intravenous magnesium study (roughly speaking, the sample size). A after heart attacks).4 In many of these cases, horizontal line (usually the 95% confidence it emerged that the different analyses were interval) is drawn around each of the studies’ either asking different questions or differed blobs to represent the uncertainty of the in some other way. In some cases, these estimate of the treatment effect. The differences have been explained by flaws in aggregate effect size obtained by the original meta-analyses and have led