Lidia R. Arends Multivariate meta-analysis: modelling the heterogeneity Multivariate meta-analysis: modelling the heterogeneity ISBN 90 90 20786 4 Mixing apples and oranges: dangerous or delicious? Lidia R. Arends omslag Lidia Arends.indd 1 16-05-2006 08:41:52 MULTIVARIATE META-ANALYSIS: MODELLING THE HETEROGENEITY Mixing apples and oranges: dangerous or delicious? Lidia R. Arends Acknowledgements The publication of this thesis was financially supported by: The Department of Epidemiology & Biostatistics of the Erasmus MC, Erasmus University Rotterdam, GlaxoSmithKline, Serono Benelux BV, Boehringer Ingelheim BV, Pfizer BV and the Dutch Cochrane Centre. Cover design: Bureau Stijlzorg, Utrecht Layout: EM Osseweijer, Etten-Leur Printed by: Haveka BV, Alblasserdam ISBN 90-9020786-4 © LR Arends, 2006 No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means, without permission of the author, or, when appropriate, of the scientific journal in which parts of this book have been published. Multivariate Meta-analysis: Modelling the Heterogeneity Mixing apples and oranges: dangerous or delicious? Multivariate meta-analyse: het modelleren van de heterogeniteit Het mengen van appels en peren: gevaarlijk of heerlijk? Proefschrift ter verkrijging van de graad van doctor aan de Erasmus Universiteit Rotterdam op gezag van de rector magnificus Prof.dr. S.W.J. Lamberts en volgens besluit van het College voor Promoties. De openbare verdediging zal plaatsvinden op woensdag 28 juni 2006 om 15.45 uur door Lidia Roelfina Arends geboren te Eelde Promotiecommissie Promotor: Prof.dr. Th. Stijnen Overige leden: Prof.dr. M.G.M. Hunink Prof.dr. J.C. van Houwelingen Prof.dr. J.D.F. Habbema Contents Chapter 1 Introduction 9 Chapter 2 Baseline risk as predictor of treatment benefit 17 Chapter 3 Advanced methods in meta-analysis: multivariate approach and meta-regression 47 Chapter 4 Combining multiple outcome measures in a meta-analysis: an application 93 Chapter 5 Multivariate random-effects meta-analysis of ROC curves 119 Chapter 6 Meta-analysis of summary survival curve data 157 Chapter 7 Discussion 181 Summary 187 Samenvatting 193 Dankwoord 201 About the author 203 List of publications 205 Manuscripts based on studies described in this thesis Chapter 2 Arends LR, Hoes AW, Lubsen J, Grobbee DE, Stijnen T. Baseline risk as predictor of treatment benefit: three clinical meta-re-analyses. Statistics in Medicine 2000; 19(24): 3497-3518. Chapter 3 van Houwelingen HC, Arends LR, Stijnen T. Advanced methods in meta-analysis: multivariate approach and meta-regression. Statistics in Medicine 2002; 21(4): 589-624. Chapter 4 Arends LR, Voko Z, Stijnen T. Combining multiple outcome measures in a meta-analysis: an application. Statistics in Medicine 2003; 22(8): 1335-1353. Chapter 5 Arends LR, Hamza TH, van Houwelingen JC, Heijenbrok-Kal MH, Hunink MGM, Stijnen T. Multivariate random-effects meta-analysis of ROC curves. Medical Decision Making. Provisionally accepted. Chapter 6 Arends LR, Hunink MGM, Stijnen T. Meta-analysis of summary survival curve data. To be submitted. "Of course it mixes apples and oranges; in the study of fruit nothing else is sensible; comparing apples and oranges is the only endeavor worthy of true scientists; comparing apples to apples is trivial." Gene V. Glass, 2000 1 Introduction Introduction 1 Introduction This thesis is about multivariate random effects meta-analysis and meta-regression. In this introduction these terms will be explained and an outline of the thesis will be given. 1.1 What is meta-analysis? Meta-analysis may be broadly defined as the quantitative review and synthesis of the results of related but independent studies[1]. These studies usually originate from the published literature. For the purpose of critically evaluating a clinical hypothesis based on published clinical trials, meta-analysis is an efficient tool for summarizing the results in the literature in a quantitative way. In most of the cases it results in a combined estimate and a confidence interval[2]. Meta-analysis allows for an objective appraisal of the evidence, which may lead to resolution of uncertainty and disagreement. It can reduce the probability of false-negative results and thus prevent undue delays in the introduction of effective treatments into clinical practice. A priori hypotheses regarding treatment effects in subgroups of patients may be tested with meta-analysis[3] as well. It may also explore and sometimes explain the heterogeneity between study results, see the section on meta-regression below. Since the introduction in 1976[4] of the term 'meta-analysis', it has become an increasingly important technique in medical research. This is illustrated in Figure 1, where the number of studies found in Medline containing the keyword 'meta- analysis' is plotted against the year of publication. 1200 1000 800 600 400 200 0 Number of 'meta-analysis' publications 1975 1980 1985 1990 1995 2000 Year of Publication Figure 1. The meta-analysis trend 11 Chapter 1 With the increasing popularity of meta-analysis, also the field of application of statistical meta-analysis methods is growing. In earlier days the main and often only interest was to statistically pool the results of independent but 'combinable' studies[5] to increase power, resulting in an overall estimate of one specific outcome measure and a confidence interval. For this situation most meta-analysts know how they can analyse the collected data. Nowadays we are often faced with meta-analysis of more complex medical data, and there are many practical situations where appropriate statistical meta-analytic methods are still lacking or underdeveloped. New statistical methods are needed to meta-analyse these complex data types. 1.2 Fixed and random effects meta-analysis In every meta-analysis the point estimates of the effect size will differ between the different studies in the meta-analysis, at least to some degree. One cause of these differences is sampling error, which is present in every estimate. When observed effect sizes differ only due to sampling error, the true underlying study specific effects are called homogeneous. In this case the differences between the estimates are just random variation, and not due to systematic differences between studies. In other words, the true underlying effect size is exactly the same in each study. In that case, if every study would be infinitely large, all studies would yield an identical result. The case of homogeneity can be accommodated in meta-analysis by using a what is called the 'fixed effects model'[6]. In the early days of meta-analysis statistical modelling was always done under the assumption that the true effect measure was homogeneous across all studies, thus with a fixed effects model. However, often the variability in the effect size estimates exceeds that expected from sampling error alone, i.e. there is not just one and the same true effect for each study, but 'real' differences exist between studies. In this case we say that there is heterogeneity between the treatment effects in the different studies. In a famous paper DerSimonian and Laird (1986)[7] introduced a statistical model that allows heterogeneity in the true treatment effects. In that model the different true study specific effects are assumed to have a distribution. This distribution is characterized by two parameters, the mean and the standard deviation, and both have to be estimated from the data. The first is the parameter of main interest, and is interpreted as the average effect. The other parameter is called the between studies standard deviation and describes the heterogeneity between the true effects. This model is called the 'random effects model'. This model is tending to become the standard method for the simple case where the meta-analysis is focused on a single (univariate) effect measure, e.g. one treatment effect. See for instance the review article of Normand (2000)[1]. 12 Introduction The fixed effects method to estimate a common treatment effect yields a narrower confidence interval than the random effects estimation of an average treatment effect when there is heterogeneity observed between the results of the different trials[8]. This explains why the fixed effects method is still often used. The simplistic assumption of a common treatment effect in all trials used in a fixed effects analysis ignores the potential between-trial component of variability and can lead to over dogmatic interpretation[9]. Since the trials in a meta-analysis are almost always clinically heterogeneous, it is to be anticipated that to some extent their quantitative results will be statistically heterogeneous[10]. Hence a random effects model appears more justified than a fixed effects model. 1.3 Meta-regression: Is there an explanation for heterogeneity? In the previous section we discussed the term heterogeneity, i.e. the part of the variability in the outcome measure across studies not due to within study sampling variability. If there is much heterogeneity between the studies, one could question whether it is wise to combine the studies at all. However, heterogeneity can be regarded as an asset rather than a problem. It allows clinically and scientifically more useful approaches attempting to investigate how potential sources of heterogeneity impact on the overall treatment effect[10]. For example, the treatment effect could be higher in trials that included a lot of old males, whereas the treatment effect could be lower in studies with a lot of young female patients. The dependence of the treatment effect on one or more characteristics like mean age and sex of the trials in the meta- analysis can be explored via meta-regression. In meta-regression the trial characteristics are put as covariates in a regression analysis with the estimated treatment effect of the trial as dependent variable. Ideally the covariates used in such analyses should be specified in advance to reduce the risk of post hoc conclusions prompted by inspecting the available data[8].