This article was downloaded by: 10.3.98.104 On: 27 Sep 2021 Access details: subscription number Publisher: CRC Press Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: 5 Howick Place, London SW1P 1WG, UK

Handbook of Meta-Analysis

Christopher H. Schmid, Theo Stijnen, Ian R. White

Choice of Effect Measure and Issues in Extracting Outcome Data

Publication details https://www.routledgehandbooks.com/doi/10.1201/9781315119403-3 Ian R. White, Christopher H. Schmid, Theo Stijnen Published online on: 08 Sep 2020

How to cite :- Ian R. White, Christopher H. Schmid, Theo Stijnen. 08 Sep 2020, Choice of Effect Measure and Issues in Extracting Outcome Data from: Handbook of Meta-Analysis CRC Press Accessed on: 27 Sep 2021 https://www.routledgehandbooks.com/doi/10.1201/9781315119403-3

PLEASE SCROLL DOWN FOR DOCUMENT

Full terms and conditions of use: https://www.routledgehandbooks.com/legal-notices/terms

This Document PDF may be used for research, teaching and private study purposes. Any substantial or systematic reproductions, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The publisher shall not be liable for an loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material. Handbook of Meta-Analysis of Handbook

Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 3.3 Data Outcome in Extracting Choice of Effect Measure and Issues 3 effect measure could be the mean of a quantitative mean could the outcome be measure effect proportion experiencing or the For meta-analysis. example, of the a descriptive focus study, in main the is studies the parametertheand defines parameters)(or quantified are across combination whose aims outcome and measure choice data of meta-analysis: extraction. effect the must that related two precede issues We a meta-analysis. to included be in discuss studies identifiedset of a has and chapter, performed been asystematic that has we review assume this In data structured. be might ways meta-analysis which different and in meta-analysis basic the concepts of systematic reviews,Chapter Chapter 1described 2introduced and 3.1 3.2 ...... 3.1 Introduction CONTENTS Stijnen Theo and Schmid, H. Christopher White, R. Ian References...... 3.4 3.5 The “effect measure” or “metric” describes the particular way in which the meta-analysis meta-analysis way the which in “effectThe particular the or measure” “metric” describes Introduction 3.3.2 3.3.1 How to Perform Outcome...... Data Extraction 3.3.3 3.2.5 Transformation...... 3.2.4 3.2.3 3.2.2 3.2.1 ...... Measure Effect the Choosing . Group-Level Study-Level from Calculating Summaries Summaries 3.5.7 3.5.6 3.5.5 3.5.4 3.5.3 3.5.2 3.5.1 ...... Results Published Study-Level from Calculating Summaries Extracting Study-LevelExtracting Summaries ...... Group-LevelExtracting Summaries Which Summaries Are Appropriate? Are Summaries ...... Which Avoiding...... Measures Effect ...... Studies Association General Comparative...... Studies ...... Studies Descriptive What Next?What ...... Measures Effect Different Report Studies Comparative Two More with Than Studies Groups...... Trials ...... Cross-Over Trials...... Cluster-Randomized Before-and-After Data...... Information Missing ...... 34 34 33 38 38 38 38 36 36 36 28 28 28 27 27 32 32 32 37 37 31 31 Choosing Effect Measure and Extracting Outcome Data Extracting and Measure Effect Choosing Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 formula formula tricks are discussed in Section 3.5. Section in discussed are tricks and some study-level pitfalls published and reports, from extracting in summaries arise to convert to study-level group-level Finally, summaries complications summaries. often of study-level flexibleand popular is (Chapter summaries 4), how3.4Section so describes how outcome the to perform at data group extraction level study and level. Meta-analysis error). its standard 3.3 with we measure Section explain (an effect of In the estimate mary informative), and reported commonly ideally both are that or a study-levelstatistics sum data at levels: various data (IPD), participant individual (summary group-level summaries We studies. may the extract primary the from extracted are measure effect of the estimation effect measure. the 3.2, how to choose Section groups. we odds ratio or In the discuss between means outcome; acomparativea binary in of study, difference could the be measure effect the 28 the formula in Table in formula the 3.1 survival, mean for the follow-up, follow-up to refers alimited survival period. mean Note that whereas restricted (Chapter instead multiple may unlimited points used 15). be time requires survival Mean the data; use of full understood make but easily is usually point not time does a specific data. survival event called often data are continuous data, time-to- and called of Quantitative often number data individuals. are would aknown data agroup with which count events or ordinal within of binary maries of follow-up, person-time total known a group with of eventsnumbers within not to sum prevalence. the is available By only measure the count data effect we then refer todisease, for measure: example, effect outcome the the prevalence if the of is a determines directly of data type the In condition. cases, most five-yearin a particular or predicting survival in apopulation, educational test,attainment diagnostic describing of a or specificity tivity sensi the prevalence estimating the for of adisease, example,descriptive aim: estimating a with studies usually are notation. These statistical and possible measure choices of effect studies, Table for of which single-group We by meta-analysis considering start 3.1 lists Studies Descriptive 3.2.1 for meta-analysis. constructed are that any summaries of outcome data. By “outcome data”, not and studies individual the data we in the mean type the and meta-analysis of the driven aim by the usually is measure choice ofThe effect 3.2 a shorthand for “experimental treatment” and “standard for and “experimentala shorthand treatment” treatment”. observational an In groups “treatment” “control”, randomized and the call we usually trial, randomized as a In of comparativeWe to trials. meta-analysis randomized now as studies, such turn Studies Comparative 3.2.2 “Outcome data extraction” is the process by which summary outcome“Outcome data suitable summary by for process which data the extraction” is Time-to-event data can be summarized in a number of anumber ways. at in probability Survival summarized be Time-to-event data can Choosing the Effect Measure Effect the Choosing ò 0 ¥ tf i () td t where ft ii () =( - St ¢ ) is the density function. density the is ò 0 ¥ St i () dt , is equivalent to the more standard equivalent more, is to the standard adok f Meta-Analysis of Handbook - - - Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 geneity and is hence less desirable hence less general, ratio is odds ratio butgeneity and risk and in the have that to tends have hetero larger difference show risk the that of meta-analyses many Studies 0.5 tend to down-weighted up-weighted be and difference odds by ratio. risk by the the ferent overall outcome proportions: for example, outcome with studies proportion near However,proportions. may they behave have when studies very differently dif markedly ratio,risk odds ratio) and overall outcome have when behave studies similarly all similar deviation (Lu standard al., et the 2014). sensitive increases that to is ence noise “medium”, 0.8 and “large” is (Cohen, 1988). However, differ mean standardized of the use absolute related be to an can that scale, for sizes example, effect yields 0.2 “small”, is 0.5 is difference mean groups. Use standardized control of group the the or pooledfrom within deviation, may population by estimated which standard be the difference mean the ing ways different allows to outcomes compared be by in divid difference measured mean would scales not comparable. be on different changes standardized one-unit Instead, the case, it would since this inappropriate be differences, In mean to combine construct. same the to assess scales rating for different example, use psychology often studies different ways— different concept in same the to measure studies for different sciences social in It scale. outcome very same common the is on to the report measure trials all requires of mercury.reduces blood by 10 pressure However, millimeters difference mean the using results: for example, interpreted easily yields atreatment that well understood,is this or (occasionally) difference outcome the mean ratio the When ally the variable of means. example, possibility of bias or large in the meta-analysis. the in example, of or variance bias large possibility properties, forSecond, one would poor statistical to avoid with wish measure effect an heterogeneous. to less be likely is that one would measure studies, then effect prefer an of heterogeneity between degree their in differ might measures First, effect if important. relevant also form. are scientifically interpretable and criteria an into Two more technical relevant, It scientifically studies. interpretable and be or convertible should also primary Table 3.2 of comparative for meta-analysis possible measure choices of effect lists studies. more choice is complex. the and more groups, measures pare are possible there effect study, “exposed” be groups might the “unexposed”. and to com here is aim the Because  Data Outcome Extracting and Measure Effect Choosing With a binary outcome, the three common choices of effect measure (risk measure difference, outcome, choices of common effect With abinary three the With a quantitative outcome, the preferred effect measure in biomedical studies is usu With is biomedical aquantitative studies outcome, in measure effect preferred the it the that from is should estimable be measure effect the choosing A key in criterion Quantitative Type data of Studies of Descriptive for Meta-Analysis Measure ofChoice Effect 3.1 TABLE Time toevent Count Categorical Binary Mean Restricted meansurvivaltotimet Survival atfixedtimet Rate Ordered categoricaldatamayberegarded asquantitative Often dichotomizedandhandledasbinary Proportion (risk,prevalence) Median survival Mean survival Effect measure 0 0 Notation in study i study in Notation S ò ò 0 i - 0 t ¥ 0 1 S St St (0.5) i μ π λ (t i i () () i i i 0 ) dt dt 29 ------Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 Count Type data of of Comparative for Meta-Analysis Studies Measure ofChoice Effect 3.2 TABLE 30 reduce the expected count by one, and equals the reciprocal of the risk difference. It difference. has risk of the reciprocal count by the one,reduce expected equals and the would that of to number to treated individuals be need the is medicine, in to treat, used outcome (Deeks, 2002).ofharmful whether toor consider beneficial the levels of choice 2002). heterogeneity (Deeks, similar further the ratio is With risk there the b a Time to Categorical Binary Quantitative event Not estimablefrom group-level summaries,unlessaconstanthazard isassumed. Binary measures are riskdifference, riskratio,andoddsratio. Two outcomes needed number mention. deserve for The binary measures effect other Hazard ratio Rate difference Odds ratiosformultinomial Odds ratioforordinal Mean difference, etc. Comparisons ofother Risk ratio Standardized mean Ratio ofmeans Binary measures Binary measures Rate ratio Binary measures Odds ratio Risk difference Mean difference occurrence atanytime before afixedtimet outcome outcome dichotomizing outcome survival mean survivalormedian measures, e.g.,restricted difference Effect measure b a a a forevent forevent after 0 h () λ π μ λ π μ St 11 Definition in Definition mm i2 i2 i2 etc. i2 i2 i2 i2 ii -- 20 p /λ −λ ii (t)/h /π −π /μ −μ () 21 i p 2 study i study - i1 i i1 i1 i1 2 i1 i1 - i1 St (t) / 10 p () s i p 1 i i 1 , Inappropriately ignores censoringand Estimated from multinomial logitmodel; Estimated from proportional oddsmodel if Treats outcomeasquantitative; maybe Makes incompleteuseofdata Appropriate butlessoftenused Assumed constantovertime;allowsfor Not usuallyappropriate: tendstobeaffected May beappropriate whendifferent studies Appropriate whenstudiesreport the sameor Appropriate whenallstudiesreport the Appropriate, butignores dataafterthefixed Usually appropriate: lesslikelytobeaffected Not usuallyappropriate: tendstobeaffected Usually appropriate: lesslikelytobeaffected Usually appropriate: lesslikelytobeaffected approximation andriskingbias length offollow-up,thusinvolving multiple parameters categories are ordered appropriate ifcategoriesare ordered censoring andlengthoffollow-up (Deeks, 2002) hence more heterogeneous thanalternatives by theoveralllevelofriskinstudy, and report different outcomes;σ similar proportionate changesinoutcome different outcomesandare likelytohave same outcome(e.g.,systolicbloodpressure) period by theoverallrate by theoverallrate by theoverallrisk by theoverallrisk adok f Meta-Analysis of Handbook Comment i isasuitable Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 main advantage is that its asymptotic variance advantage variance its that asymptotic main is p-values, yielding just a summary p-value.p-values, For just asummary example, yielding given p-values p of ameta-analysis It possible by is performing to avoid measure effect of an use the 3.2.4 18) or correlation coefficients (Chapter 16). from a suitable coefficients modelregression regression here are measures (e.g.,Chapter variables. Suitable of between study explores effect more type general associations A third 3.2.3 3.5.2. Section in cussed levelthe theoutcome of 21.dis covariatebaseline is are the issues baseline when Specific Chapter in alternative approaches discussed shown are be to similar; be can confounding control of ent their levels of covariate unless combined adjustment should not directly be differ with studies and to control for confounding, covariate required adjustment often is since different, is studies of practice. observational case in The tend to small be differences however, measure: effect same do the should they not since not estimate combined be the of covariate degrees adjustment different with analyzed trials randomized that argued be (oncovariate accompany adjustment can increase average) an It errors. standard could in Jewell, odds ratio and unadjusted (Robinson 1991). power in due to aresult, increase As the the one than from further is covariate-adjusted trial true odds a randomized ratiothe in forever, example, estimated: being measure effect covariate the adjustment change does of covariatedegrees reasonably adjustment combined. For be can outcomes, other how different reduction (on with analyzed average) trials randomized errors, and standard in powerdataset). in due to increase covariate The to adjustment a corresponds therefore any given in differ (although estimates difference mean their unadjusted the as same the is trial arandomized in for example, covariate-adjustedestimated: difference true mean the being measure effect the not does change come, trials covariate randomized adjustment in Table in measures effect covariates. adjusted 3.2 be For for can baseline a quantitative out measure. sample However, size. effect an its as wider use limited has its of interpretation lack easy study (either all one-tailed or all two-tailed),study or (either all Fisher’s statistic one-tailed the all uses method Furukawa et al., et (Furukawa as 2002). defined is difference arcsine The measure effect of another derived meta-analysis but be meta-analysis, can after the in measure effect the as should and not therefore used be properties poor statistical  Data Outcome Extracting and Measure Effect Choosing) (Chapter 13). (Chapter genetics “fail-safe ofin the part 17) N” for forms publication method and bias used patient. of on the burden p-values Meta-analysis sometimes is its increased justify example,nificant—for treatment reducesenoughamountrisk large by a towhether sig scientifically neity, are results whether to tell p-values because insufficient and are of heteroge no assessment is there of because effect, magnitude the to quantify it fails å c 2 2 Adjustment for baseline covariates in randomized trials can increase power. increase can Any of the trials randomized covariatesAdjustment in for baseline I distribution (Fisher, distribution 1934). of not p-values desirable Meta-analysis usually is because i I =1

- Avoiding Effect Measures Avoiding Effect General Association Studies 2 lo g p i : if the null hypothesis is true for all studies then this statistic follows statistic the this then studies for all true is hypothesis null the : if 4 nn 1 ii 12 + 4 1 depends only on the study on the only depends arcsin pp ii 21 - arcsin i in the i the in . Its Its . 31 th th ------Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 tion coefficient might be transformed by the Fisher by z-transformation. transformed mightbe coefficient tion correla the onto and logit the scale transformed for be proportion might example, abinary later more chapters in plausible: described assumptions modeling the to make used be can Table 3.2) transformations Other onto before log analysis. would the scale transformed be odds ratio, (including the aboveMany of the measures ratio, ratio risk effect in hazard and 3.2.5 32 Levels of outcome data extraction and levels of data analysis in ameta-analysis. in analysis of data levels and extraction of data outcome Levels FIGURE 3.1 of number eventsdata a proportion, hence the as to the although treat and and at any time Hozo (2005), al. et (2019), al. et Higgins Wan and (2014). al. et (1992), al. et given by are Follmann case this SD and in mean the SD. for inferring Methods and mean not range and the wanted—for interquartile not and example, those median the are presented statistics group-level the deviation. summary Often standard and mean the should ofdata, individuals exclude numbers the who any individuals did not contribute to of presence publications.missing the in In confused be error, can standard these the since publications. from summaries such extracting in Table of data in for type each 3.3. marized may We that several arise pitfalls now discuss raw of the data, sum to simple be as summaries taken usually are Group-level summaries 3.3.1 section. topic the is of which this reports, suitable to extract need outcome meta-analysts Chapter 8. published datain Other from further discussed is This analyses. required the to perform data required the assemble approach to negotiate study needs with investigators this who to meta-analyst chooses errors). advantages IPD. Chapter the 2introduced disadvantages and of collecting The standard associated tistics), with measure effect of or study the level estimates (yielding ticipant level IPD), (yielding sta group level summary simple informative (yielding and Chapter levels 3.1): 2, noted in As (Figure data at may extracted three be par individual 3.3 Survival data also present challenges. It is common to extract the number of participants of number participants It the to common extract is present challenges. data also Survival deviation from standard the For quantitative to distinguish data, must taken be care How Outcome Perform to Data Extraction

Transformation Extracting Group-Level Summaries Group-Level Extracting analysis data Level of extraction data Level of Participant Data (Chapter 8) Individual Individual (IPD) Group-level (Chapter 5) summaries Group adok f Meta-Analysis of Handbook (estimates &SE) Study-level summaries (Chapter 4) Study - - - - Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 group-level summaries to estimate a risk ratio arisk of 0.86 to estimate (95%group-level summaries CI, 0.76 to 0.98). not is This study the authors’of rate of using the ratio, estimate authors the used meta-analysis the arate to estimate ratio. used be Instead cannot group-level the MATRIX summaries trial, 0.85|/2.16 =0.075. could have deduced of 2.16 az-statistic of error approximately hence astandard and |log authorsrate the the ratio had reported instead If of 0.85 a p-value with of 0.03, we then log 0.85 of 0.85 analysis (95%survival CI, 0.74 to 0.99). We deduce can alog rate ratio of approximately above, authors arate the discussed For reported ratio example, from MATRIX the trial in to work appropriatecase, on the it to error derive necessary backwards is standard the scale. meta-analyst. modelerror, or by computed either authors astatistical reported is that using by the its standard measure, with together effect of the estimate A study-level an is summary 3.3.2 al., et (Parmar 1998;curves Tierney al., et 2007). Kaplan–Meier group-specific IPD from the case, it to attempt best is to reconstruct this In model. ratio aCox under of hazards ahazard proportional estimation as such analysis survival to reproduce enough a full information contain no group-levelally summaries (as for count data). summary may usu non-constant, be events hazards auseful are When of number total and person-time total the roughly then constant, is hazard data. the If Capodanno, 2015).and outcome outcome (And time-to-event a binary the with approximating statistics, mary sum four these 429 used of to femoral 4207 in access.and Ameta-analysis randomized outcome 369 in of 4197 event primary The occurred access to radial patients randomized undergoing invasive al., et (Valgimigli for syndromes management 2015). acute coronary patients femoral in with access radial comparing trial was arandomized MATRIX trial not does account follow-up for different Forthis example, censoring. potential and the  Data Outcome Extracting and Measure Effect Choosing Group-level summaries and study-level not and Group-level need For agree. example, the summaries in summaries Often the authors report a contrast such as a rate ratio with a ; in this authors aratethis as the acontrast such in report ratiointerval; aconfidenceOften with of the nature Alternative time-to-event group-level to the approaches more are faithful

Extracting Study-Level Summaries Study-Level Extracting = −0.163 of error approximately astandard and Quantitative data Type outcome of Types of Outcome Data for Different Group-Level Summaries 3.3 TABLE Time toevent Count Ordinal Binary Mean μ Survival S Rate λ Probability ofoutcomelevellπ Probability ofeventπ Standard deviationσ ij ij Parameters ij (t) orhazard h ij ij ij (t) ijl Number ofperson-yearsT Number ofeventsy Number y Number ofindividualsn Number ofeventsy Number ofindividualsn Standard deviations Mean m Number ofindividualsn Can bedifficult: seetext ( lo Group-level summary in in summary Group-level gl 0.99 i study jof group ij ijl - atoutcomelevell og 0.74) ij ij ij /2 ij ij ij ij z 0.975 = 0.074 33 ò - - .

Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 analysis (requiring a normal approximation in analysis) without gaining its flexibility. approximation analysis) anormal without in gaining (requiring analysis suitable because cost of rarelyused, data available two-stage the are it because brings and is rarely it Otherwise point. time to afixed relates to survival measure data effect when the for time-to-event may useful be log This errors. the odds) standard their with together data group-level at the group (e.g., level parameters above by estimating to summarize is al., et 2019). for evaluations systematic (Higgins reviews oflevel randomized summaries Collaboration to prefer tends Cochrane group- the reasons, For them. these between choosing for to formulate needs rules meta-analyst the then reported, multipleresult. are If analyses the of adjustedreport for significance acovariate doing improved so because statistical the (Chapter for 13): reporting selective tial for example, a study that concerned onebe might summaries, though IPD are preferable IPD though are (Chaptersummaries, 21). group-level of study-level use rather the implies than for usually confounders summaries epidemiology, data. observational In ofappropriate missing to adjust need the handling an through point estimate the of in reduce bias might or risk they the trial; randomized foravoid a complex by error allowing standard a cluster- the study as such design in bias improve might power trials for by covariates; adjusting randomized in might yses they have can advantages: various This to used. be for example, allows these tion complex anal ple, proportion y observed the level analysis. forces atwo-stage data at study the extracting hand, other the On required. if analysis for two-stage maries poor. Further, at group data level extracted the later converted be can study-level into sum where approximation the studies involved is for small analysis two-stage useful in cially data at group level the (ChapterExtracting enables aone-stage analysis espe 5). is This 3.3.3 period, we would author-reported the prefer to rate use ratio here. were followed participants all that up same for wrongly the assumes and for censoring was very lowportion (under 10%). ratio not risk does of account the estimation Because event the because rate pro reported identical ratio to the numerically but very similar is 34 in Tablein (1987) 3.4. Greenland are general references (2000). Useful Wilson Lipsey and and for computing study-level where appropriate) (after transformation summaries given are study-level always is using possible 3.1). analysis (Figure two-stage summaries Formulae avoiding of study-level possible, use aone-stage analysis is the ries, then but a summaries provide studies doing so. group-level all in If pitfalls summa and some tricks describes section This published or reports. from ) group-level from their and summaries study-level measures to calculate (estimated effect it necessary is summaries Sometimes 3.4 1, log but observed odds, the An intermediate option between group-level option between study-level described and intermediate as An summaries The flexibility of complex statistical analyses in study publications brings with the poten in studyit publications brings analyses of complex flexibility The statistical In some cases, study some cases, Study-level authorsIn complex analyses. report statistical data extrac Some of the study-level summaries in Table study-levelSome of the in bias. 3.4 summaries For small-sample from exam suffer Calculating Study-Level Summaries from Group-Level Summaries from Study-Level Summaries Calculating Which Summaries Are Appropriate? Are Which Summaries lo i 1 / g n ny i 1 ii is an unbiased estimate of the true probability in group in probability true of the estimate unbiased an is 11 y - i 1 , is not an unbiased estimate of the true log odds, true of the estimate unbiased not, is an adok f Meta-Analysis of Handbook ------Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 (Jackson and White, 2018). and (Jackson solutions have Specific remove correlation to this proposed been meta-analysis the to in bias leading point estimates, correlated be with can studies variance. the avoidsalso problems estimating in count, reduction correction even bias would the though continuity apply The studies. all in zero cell with a is only in studies applied In correction practice, theundefined. continuity Type data of Group-Level Summaries from Study-Level Summaries Extracting 3.4 TABLE a Count Binary Quantitative studies Comparative Time toevent in the case of events zero case or the non-events ( in by (2004). al. et Sweeting detail in important It discussed particularly is is and correction is, by adding 0.5 to all cell counts in the 2 the in counts is, by cell adding 0.5 to all Quantitative Single-group studies Binary All sumsare overj Notation isdefinedin Table 3.3,withsubscriptsiforstudyand(inthecaseofcomparativestudies) j forgroup. Count  Data Outcome Extracting and Measure Effect Choosing (1989) largely be removed can bias showed estimate by the this that proportion. Cox observed of the Snell and transformation log anon-linear the odds is since Two quantitative studies Association Time toevent variables degrees offreedom, and Standardized mean difference. Here s and Thomas(2005). Another problem with the summaries in Table 3.4 is that estimated variances in small Table in small in problem variances 3.4 summaries Another the estimated with that is = 1,2. Log rateratio Log oddsratio Log riskratio Risk difference Mean difference (See text) Mean Proportion SMD Rate Fisher’s transformation Fisher’s (see text) coefficient r of thecorrelation Effect measure J a () nn »- 13 i /( i isanestimateofthepopulation standard deviation, 41 - ) and × () m m Estimate y y lo lo lo yn 2 1 y mm i i ii /T /n K i2 i 22 gl gl gl lo i 2 table. The quantity 0.5 is called the continuity continuity the 0.5 called 2 table. is quantity The 1 ii // −m 21 () () () or =0 nn ny g yT yn i i - ii ii ii 22 1 1 22 22 y »- i1 // // - + - - i 2 1/ r r yn i i Js ii 11 () 2( n y - - - ii i 1 / og og =n og Study-level summaries Study-level 1.94) () () ny yT yn i 1 ii ii ii 11 11 11 ), since otherwise the log the odds is ), otherwise since y - i withexactexpressions givenbyWhite 1 1/(n å å å å å å sn yT yn ii 2 ii ii () / / Estimated variance Estimated j j j j j j 1/ yn 1/ () è ç ç æ sn i −3) - 11 ij ij 2 2 yn lo // / () 11 y yn nS ij yn ij ii ij g ij ij + ij / + - - ny yn ij ii n MD 11 3 ij y - ii -+ / =2 i ij y 1 ij 2 nn + K ij 3 ø ÷ ÷ ö 12 () 0.5 n +- i 0.5 i : that : that isits 35 Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 intervention, and standard deviations s deviations standard intervention, and j atextracted group level al., et (Riley 2013). the in group level the that summaries Assume are data before-and-after when arises difficulty some intervention. Aparticular after groups two before and in variable observed same is when the data arise Before-and-after Data Before-and-After 3.5.2 way. asimilar in handled study, another equal. in are sample observed errors sizes unless done. to standard It equal are would errors standard not appropriate be missing to assume not usually model, is but this ahierarchical via studies between varying deviations dard Ideally, characteristics. to haveone would allowance chosen similar made be stan for the study—preferably another in observed deviations to standard equal are they assuming asimple in way validly. errors may deviations tackled be standard by standard Missing to estimate of but analysis, it focus required is primary not deviation is the standard The In a randomized trial, where before-intervention means m where before-intervention means trial, arandomized In group j with pared formulae below. the in for group used j be effect intervention the also Then can we pool the variances across groups to give across we variances pool the for example, deviation s standard the may missing: for be avalid analysis required some information that is theme A common Information Missing 3.5.1 summaries. study-level in outcomethat arise of extraction data may difficulties some describes section final This 3.5 al., et (Emerson 1993; R 36 solution best the (Chapter usually is 5).analysis lowest ρ Thus, if variance. the has but adjusted have the expectation, difference same the three tion, all

th group i of the th Other examples of missing information arise in the next three subsections and may and be subsections next three the in arise information examples of missing Other 2. 3. 1. Calculating Study-Level Summaries from Published from Results Study-Level Summaries Calculating (2 after-intervention valuesafter-intervention on before-intervention values. A difference of after-intervention means, means, of after-intervention A difference A difference of change scores, of change A difference An adjusted An difference, groups. across values, equal after-intervention before-intervention and assumed sn sn ss iA iA 22 2 . . iA 2 . (1 (1 -+ /1 -+ r r ii ii 12 ii )(1 + .. Ai ss /1 th study are presented as the count n the as presented study are th /) Bi = n 12 1 may be estimated in three different ways: different three in 1 may estimated be ü 2 . . Bi cker et al., et cker aproblem 2009) correlation is aone-stage then but this if )(1 /) n i // is reported then the adjusted difference should computed. be adjusted the difference then reported is i nn 12 , where + 1) i , where ρ lr ij () of aquantitative outcome may not reported. be ijB mm ii = before and s before and iA 21 () -- ss mm s iA .. i , iA B 22 / and and iA i is the within-group correlation of the within-group the is -- iB is the within-group regression of regression within-group the is iB s l i ii ijA , () A mm ij , though group-specific variances variances group-specific , though after intervention. For simplicity, after , means m , means mm 21 () i 1 Bi mm B iA 21 and m and - iA 11 - adok f Meta-Analysis of Handbook - B iA with variance variance with i iB ijB 2 B , with variance variance , with are equal in expecta in equal are before and m before and with variance variance with com = 2 ijA after after - - - Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 treatment If If  Data Outcome Extracting and Measure Effect Choosing treatment effect isestimatedas receiving C.To eliminatetheeffect ofanytrend inoutcomeovertime(aperiodeffect), the example, supposeinstudyithatgroup j the meanofgroup 1inperiodreceiving T, andm by the “design effect” 1 this adjustment(DonnerandKlar, 2002).Therelevant adjustmentmultipliesthevariance do notadjustfortheclustering,andtherefore theoutcomedataextraction mustestimate tering, then there is no problem. However, commonly reported study-level summaries correlation (ICC) Bland, and 1998). intra-class by (Kerry the measured is lation (“clustering”) must allowed be Clustering errors. order for standard in to give correct corre resulting clusters, the and in grouped are participants trial, acluster-randomized In 3.5.3 ways. different in have estimated whereies been effects intervention stud it rarely and reasonable them, is to meta-analyze between to choose should used be ways. different in have estimated where been effects intervention made, are studies decisions it reasonable to meta-analyze is these scores. Once change or after-intervention means, of difference the efficientestimate, statistically aless to use 3.5.1. available Section study in as value another it necessary be in may instead Otherwise, be reported astheoutcomemeansm prove needed. often also acorrelation is level inadequate and summaries group- Again, treatment. each outcomes of after and treatments, sequences observed are different to groups representing randomized are participants trial, across-over In Trials Cross-Over 3.5.4 apply and to study-levelvert them statistics effect. design the summary approach Acommon to con is trials. cluster-randomized in errors standard to give correct may becombinedwithothertrialdesignsinatwo-stagemeta-analysis. value from another study as in Section 3.5.1. Once this is done, cluster-randomized trials is sometimesreported; ifnot,thenitmaybenecessarytoassumeequalanobserved The formulacanbeextendedforunequalclustersizes(KerryandBland,2001).ICC in atwo-stage meta-analysis(Curtinetal., 2002). 3.5.1. Oncethisisdone,however, cross-over trialsmaybecombinedwithothertrialdesigns be necessarytoassumeit is equaltoanobservedvaluefrom anotherstudyasinSection comes ontreatment andcontrol inthejthgroup oftheithstudyisnotreported. Itmaythen standard deviation of change scores as well as s well as as scores deviation of change standard ss iT 2 If study-level summary statistics are reported using ananalysis that allows for the clus- Subject-matter arguments study, different. are observational estimators an In three the Assuming atwo-treatment two-perioddesignwithaquantitativeoutcome,thedatamay Table in described not sufficient statistics 3.3 also simple are The group-level summary ρ 1 i is not reported, it may be possible to infer it from other reported results (e.g., not results it is reported, reported it may other possible be from to infer the from +- lse-admzd Trials Cluster-Randomized iC 2 1 k 4 = 2 n rs i T,C withcorresponding samplesizesn 1 iT 11 Ci Ti ss 1 C + + )C, where (m −1)ICC, iT 2 2 +- sr iC 2 2 4 mm ijk 2 n iT forgroup j 11 i 2 = iT 22 - 22 1 represents thesequenceTthenC.Thenm Ci ss m is the cluster size (Kerry and Bland, 1998). iC Ti + ijB 2 mm and s and C = iT i1C ijk . Oftenacorrelation ρ 22 1,2 oftheithstudywhenreceiving =n isthemeanofgroup 1inperiod2 - ijA ij andstandard deviationss iC ) or to assume it is equal to an to an it equal ) or to is assume , withestimatedvariance ijTC betweenout- ijk . For i1T 37 is - - - Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 and in Chapter 8, in IPD.and we meta-analyze study-level Chapteranalyze 5, in group-level we summaries; meta-analyze summaries; Chapter 4, chapters: in we covered be meta- future in will This meta-analysis. the perform data have and we extracted, ready chosen been are been to has measure effect an Once Next? What 3.5.7 quantity. same the are hazard and odds ratio Yu, ratio and to risk (Zhang 1998). rate that to know data, For it survival useful is ratio, rate and ratio. For commoner events, some conversions available, are for example, ratio, risk the are odds odds, risk, also hence so rate and the and very similar, then are rare 2000). event data, (Chinn, the if For is difference binary mean ratio astandardized and some studies, ittitative possible in odds is to outcome convert an dichotomized between is Table formulae in the using 3.4. difference mean aquan Where standardized and ference (2000)Wilson (2001). Wilson and For quantitative dif data, convert mean one can between we give only some examples without given formulae; by is Lipsey and more information to able be conversions. to make it cases, useful is Here these In measures. effect different study-level report for meta-analysis the included in summaries studies different Often, 3.5.6 10. Chapters 9and in discussed those as such for needed multivariate many is and analyses estimated easily correlation is The correlated due control common are group. to their measures effect estimated the then since relative control to group. the when Acomplication study-level used, are arises summaries as above defined all are acontrolinclude is group, one group which measures effect the so usually multiple groups, two are studies measures;With such there more effect than 3.5.5 38 Fisher RA,1934.StatisticalMethodsforResearch Workers, 5thedition.OliverandBoyd. Emerson JD,HoaglinDCand MostellerF, 1993. A modifiedrandom-effect procedure forcombining Donner A andKlarN,2002.Issuesinthemeta-analysis ofclusterrandomizedtrials.Statisticsin Deeks JJ,2002.Issuesintheselectionofasummarystatistic formeta-analysisofclinicaltrialswith Cox DRandSnellEJ,1989.AnalysisofBinaryData.Chapman&Hall/CRC. Cohen J,1988.StatisticalPowerAnalysisfortheBehavioralSciences . Lawrence Erlbaum Associates, Inc. Chinn S,Nov2000. A simplemethodforconvertingan odds ratiotoeffect size foruseinmeta- Andò GandCapodannoD,2015.Radialversusfemoralaccessin invasivelymanagedpatientswith References 269–290. risk difference in setsof2×tablesfrom clinical trials. JournaloftheItalianStatisticalSociety Medicine 21(19):2971–2980. binary outcomes.StatisticsinMedicine21(11): 1575–1600. analysis. StatisticsinMedicine19(22):3127–3131. acute coronary syndrome. AnnalsofInternalMedicine163(12):932. Studies Report Different Effect Measures Effect Different Report Studies Comparative Studies with More Than Two Than More Comparative with Studies Groups adok f Meta-Analysis of Handbook 2(3): - - Downloaded By: 10.3.98.104 At: 07:38 27 Sep 2021; For: 9781315119403, chapter3, 10.1201/9781315119403-3 Data Outcome Extracting and Measure Effect Choosing  Zhang Jand Yu http:​//www KF, 1998.What’sthe relative risk? A methodofcorrecting calculator. theoddsratioincohort size effect meta-analysis Practical 2001. DB, Wilson White IRandThomasJ,2005. Standardised meandifferences inindividually-randomisedandclus- Wan X,Wang W, LiuJandTong T, 2014.Estimatingthesamplemeanandstandard deviationfrom Valgimigli M,Gagnor A, Calabro P, FrigoliE,Leonardi S,Zaro T, RubartelliP, BriguoriC, Ando Tierney JF, StewartLA,GhersiD, Burdett SandSydesMR,2007.Practicalmethodsforincorporating Sweeting MJ, Sutton AJ and Lambert PC, 2004. What to add to nothing? Use and avoidance of conti- Rucker G,Schwarzer G, Carpenter J andOlkin I, 2009.Whyaddanythingtonothing?Thearcsine Robinson LDandJewellNP, 1991.Somesurprisingresults aboutcovariateadjustmentinlogistic Riley RD,KauserI,BlandM,ThijsL,StaessenJA,Wang J,Gueyffier FandDeeksJJ,2013.Meta- Parmar MKB, Torri V and Stewart L, 1998. Extracting summary statistics to perform meta-analyses of Lu G,KounaliDand Ades AE, 2014.Simultaneousmultioutcomesynthesis andmappingoftreat- Lipsey MWandWilson DB,2000.PracticalMetaAnalysis.ThousandOaks,CA:Sage. Kerry SMandBlandJM,2001.UnequalclustersizesfortrialsinEnglishWelsh generalpractice: Kerry SMandBlandJM,1998.Statisticsnotes:theintraclustercorrelation coefficient inclusterran- Jackson DandWhiteIR,2018.Whenshouldmeta-analysisavoidmakinghiddennormalityassump- Hozo SP, DjulbegovicBandHozoI,2005.Estimatingthemeanvariancefrom themedian,range, Higgins JPT, ThomasJ,ChandlerCumpstonM,LiT, PageMJ,Welch VA (editors),2019.Cochrane Greenland S,1987.Quantitativemethodsinthereview ofepidemiologicliterature. Epidemiologic Furukawa TA, GuyattGHandGriffith LE,2002.Canweindividualizethe‘numberneededtotreat’? Follmann D,ElliottP, SuhIandCutlerJ,1992.Variance imputationforoverviewsofclinicaltrials studies ofcommonoutcomes. JAMA280(19):1690. or ter-randomised trials,withapplicationstometaanalysis. ClinicalTrials 2(2):141–151. 14: 135. the samplesize,median,rangeand/orinterquartile range.BMCMedicalResearch Methodology Lancet 385:2465–2476. acute coronary syndromes undergoing invasivemanagement: A randomisedmulticentre trial. Rothenbuhler M,Vranckx P andJüniP, 2015.Radialversusfemoralaccessinpatientswith M, Zingarelli A, deCesare N,RigattieriS,Tosi P, PalmieriC,Brugaletta S,RaoSV, HegD, Ausiello A, Presbitero P, Sardella G,Varbella F, EspositoG,Santarelli A, Tresoldi S,Nazzaro G, Repetto A, Limbruno U,CorteseB, SganzerlaP, Lupi A, GalliM,ColangeloS,Ierna summary time-to-eventdataintometa-analysis.Trials 8(1):16. nuity corrections inmeta-analysis ofsparsedata.StatisticsinMedicine23(9):1351–1375. 28(5): 721–738. difference as a measure of treatment effect in meta-analysis with zero cells. regression models.InternationalStatisticalReview/RevueInternationaledeStatistique59(2):227. availability ofindividualparticipantdata.StatisticsinMedicine32(16):2747–2766. analysis ofrandomisedtrialswithacontinuousoutcomeaccording tobaselineimbalanceand the publishedliterature forsurvivalendpoints. StatisticsinMedicine17(24):2815–2834. ment effects toacommon scale.Value inHealth17(2):280–287. Implications forsamplesizecalculations.StatisticsinMedicine20(3):377–390. domisation. BritishMedicalJournal,316(7142):1455–1460. tions? BiometricalJournal60(6):1040–1058. and thesizeofasample.BMCMedicalResearch Methodology 5(1):13. handbook. Sons. Also online version 6.0 (updated July 2019) available from www Handbook forSystematicReviewsofInterventions.2ndEdition.Chichester, UK:JohnWiley & Reviews 9:1–30. Epidemiology 31(1):72–76. An empiricalstudyofsummaryeffect measures inmeta-analyses.InternationalJournalof with continuousresponse. JournalofClinicalEpidemiology45(7):769–773. ​​g​/escalc/html/Effect​​SizeCalcul​​​ator-​​R3​.php. ​​.campbellcollaboration​. ​.training Statistics in Medicine .cochrane​ .org​ 39 /