Open Access

Volume: 41, Article ID: e2019013, 10 pages https://doi.org/10.4178/epih.e2019013

METHODS

Network meta-analysis: application and practice using R software

Sung Ryul Shim1,2, Seong-Jang Kim3,4, Jonghoo Lee5, Gerta Rücker6 1Department of Preventive Medicine, Korea University College of Medicine, Seoul, Korea; 2Urological Biomedicine Research Institute, Soonchunhyang University Hospital, Seoul, Korea; 3Department of Nuclear Medicine, Pusan National University Yangsan Hospital, Pusan National University School of Medicine, Yangsan, Korea; 4Biomedical Research Institute for Convergence of Biomedical Science and Technology, Pusan National University Yangsan Hospital, Yangsan, Korea; 5Department of Internal Medicine, Jeju National University Hospital, Jeju National University School of Medicine, Jeju, Korea; 6Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany

The objective of this study is to describe the general approaches to network meta-analysis that are available for quantitative data synthesis using R software. We conducted a network meta-analysis using two approaches: Bayesian and frequentist methods. The corresponding R packages were gemtc“ ” for the Bayesian approach and “netmeta” for the frequentist approach. In estimating a network meta-analysis model using a Bayesian framework, the “rjags” package is a common tool. “rjags” implements Markov chain Monte Carlo simulation with a graphical output. The estimated overall effect sizes, test for heterogeneity, moderator effects, and publication bias were reported using R software. The authors focus on two flexible models, Bayesian and frequentist, to deter- mine overall effect sizes in network meta-analysis. This study focused on the practical methods of network meta-analysis rather than theoretical concepts, making the material easy to understand for Korean researchers who did not major in statistics. The authors hope that this study will help many Korean researchers to perform network meta-analyses and conduct related research more easily with R software.

KEY WORDS: Network meta-analysis, Multiple treatments meta-analysis, Mixed treatment comparison, Consistency, Transi- tivity, Bayes’ theorem

INTRODUCTION In the conventional pairwise meta-analysis, the researchers col- lect studies that evaluate the same treatment, create pairs of the Network meta-analysis (NMA), also called multiple treatment treatment group and control group, and directly calculate the ef- meta-analysis, or mixed treatment comparison, aims to synthesize fect size (direct treatment comparison). However, NMA can cal- the effect sizes of several studies that evaluate multiple interven- culate the effect size between treatment groups through indirect tions or treatments [1-4]. treatment comparison, even if there is no direct comparison study, or if the treatments are different between the treatment groups. Correspondence: Sung Ryul Shim In the present study, the previous meta-analysis studies [1-3] Department of Preventive Medicine, Korea University College of are reviewed using R software. This study focuses on the technical Medicine, 145 Anam-ro, Seongbuk-gu, Seoul 02841, Korea implementation of Bayesian NMA and frequentist NMA using R. E-mail: [email protected] Thus, it requires understanding of the direct treatment compari- Received: Mar 14, 2019 / Accepted: Apr 8, 2019 / Published: Apr 8, 2019 son (which is the basic principle of NMA), indirect treatment com- This article is available from: http://e-epih.org/ parison through common comparators, and mixed treatment com- This is an open-access article distributed under the terms of the Creative parison that combines direct and indirect treatment comparisons, Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, as well as prior learning about the assumptions of NMA. These provided the original work is properly cited. concepts are described in previous studies [1-3]. 2019, Korean Society of Epidemiology

www.e-epih.org | 1 relevant disease), updated posterior information can be inferred by adding the prior

information to the present data. This is much more logical and persuasive than the

frequentist assumption that the given data is repeated infinitely.

Second, it is free from the large sample assumption, because the parameters are

considered as random variables. For example, the frequentist meta-analysis assumes

that the overall effect size follows a normal distribution. In other words, the normality

assumption of the normal distribution is satisfactory for a large sample, but most

meta-analysis studies have a small number of studies, and the overall effect size may

be biased. However, the Bayesian method calculates the posterior information by

adding prior information to the likelihood of the given data, and the parameters are

probability concepts that can change continuously. Thus, it is free from the effect of

a large sample [2,5].

1) Prior and posterior distribution in Bayesian inference When the parameter of interest is θ, the prior information follows the prior

Epidemiol Health 2019;41:e2019013 distribution P(θ). When event χ is observed in the present study, the likelihood is

P(χ|θ).

STATISTICAL APPROACH OF NETWORKTherefore, thelows posterior the prior distrib distributionution function P(θ). Whenof the updatedevent χ is p arameterobserved ofin interestthe θ, META-ANALYSIS present study, the likelihood is P(χ|θ). becomes P(θ|χ)Therefore, by multiplying the posterior the prior distribution distribution function by the likofelihood the updated function, as The NMA methods are largely divided into Bayesian methods parameter of interest θ, becomes P(θ|χ) by multiplying the prior follows: and frequentist methods. These two statistical methods follows:have dif - distribution by the likelihood function, as follows: ferent basic concepts for approaching the statistical model, but P(θ|χ) = ------(1) produce the same results if the sample size is large. P(θ|χ) = ------(1) ୔ሺ஘ሻכ୔ሺ஧ȁ஘ሻ

The Bayesian method calculates the posterior probability that ∝ ௉ሺ஧ሻ P(θ|χ) ∝ P(χ|θ) *P(θ) ------(2) the research hypothesis is true by adding the information given in the present data (likelihood) to previously known information In the conditional probability equation (1), P(χ) is a fixed con- (prior probability or external information). Therefore, it can be stant and omitted in general; thus, the equation can be expressed said that the Bayesian method is a probabilistic approach, where as (2), where ∝ means “proportional”. If the sample size is large, the probability that the research hypothesis is true can be changed the influence of prior information P(θ) is weak, and a similar re- 5 depending on the prior information [1,2]. sult as that of the frequentist method is obtained. However, if the

In contrast, the frequentist method calculates the probability of sample size is small and the amount of prior information P(θ) is significance (in general, p-value is 0.05) or the 95% confidence -in large, the posterior distribution will produce a different result. There- terval (CI) for rejecting or accepting the research hypothesis when fore, the Bayesian method generally shows the sensitivity analysis the present data is repeated infinitely based on a general statistical according to the prior distribution. theory. Therefore, the frequentist method is unrelated to external Prior distributions are largely divided into subjective and objec- information, and the probability that the research hypothesis is tive. The subjective prior distribution can reflect the results of pre- true within the present data (likelihood) is already specified, and it vious studies or empirical knowledge of the disease. In contrast, only determines whether or not to accept or reject it based on the the objective prior distribution is an informationless prior distri- significance level [1,2]. bution, which must be objectively quantified and input to the pri- or distribution. Bayesian method The prior distribution that is the same as the posterior distribu- The frequentist method considers the parameters that represent tion is called a conjugate prior distribution. The posterior distri- the characteristics of the population as fixed constants and infers bution is inferred using the normal distribution if the parameter is them using the likelihood of the given data. However, the Bayesi- average, beta distribution if it is a ratio, or inverse gamma distri- an method expresses the degree of uncertainty with a probability bution if it is variance. model by applying the probability concept to the parameters. For non-conjugate prior distribution, it is not easy to integrate The most important characteristics of the Bayesian method are the probability of the derived posterior distribution, because it is as follows. not generally used in statistical models [5]. First, it can use prior information. For example, if the prior in- formation of the parameter of interest exists (from previous re- Markov chain Monte Carlo simulation search or empirical knowledge of the relevant disease), updated For distributions commonly used in statistics, the area under posterior information can be inferred by adding the prior infor- the distribution curve can be simply obtained with an integral for- mation to the present data. This is much more logical and persua- mula. However, with the Bayesian method, it is difficult to calcu- sive than the frequentist assumption that the given data is repeat- late it if the posterior distribution does not follow a commonly used ed infinitely. distribution. In this case, the Markov chain Monte Carlo (MCMC) Second, it is free from the large sample assumption, because the simulation can be used to calculate it reversely. In this study, we parameters are considered as random variables. For example, the will briefly review the concepts of MCMC as a tool for Bayesian frequentist meta-analysis assumes that the overall effect size fol- inference. lows a normal distribution. In other words, the normality assump- tion of the normal distribution is satisfactory for a large sample, Markov chain but most meta-analysis studies have a small number of studies, In a Markov chain, the probability that a random variable will and the overall effect size may be biased. However, the Bayesian reach a certain state depends only on the previous state. method calculates the posterior information by adding prior in- formation to the likelihood of the given data, and the parameters Next state= current state * transition probability are probability concepts that can change continuously. Thus, it is free from the effect of a large sample [2,5]. Therefore, the next state is determined by the likelihood ratio of the current state and the transition probability, which is prior in- Prior and posterior distribution in Bayesian inference formation. In the initial simulation, the value of the next state is When the parameter of interest is θ, the prior information fol- significantly different from the current state, but when this calcu-

2 | www.e-epih.org Shim SR et al. : NMA: application and practice using R software

Figure 1. Monte Carlo simulation. lation is repeated, the difference becomes very small at a certain time, and it reaches a stable steady-state distribution. Present data, To summarize again, the Markov chain uses an algorithm that Prior- likelihood calculates the probability of the next state knowing the current probability state and transition probability, and there is no change in the prob- ability after a certain number of repeated calculations.

Monte Carlo simulation In Figure 1, let us assume that we want to find the area of a 1/4 circle with a radius of 1. This can be calculated easily by 1/4 * πr2 = 1/4*3.142*12, which is 0.7855. In Monte Carlo simulation, a square around the 1/4 circle is created, and many dots are printed randomly in the square. Then, the desired area can be determined by comparing the number of dots within a distance of 1 from the center of the circle with the total number of dots. For example, if a simulation is performed in which a total of 100 dots is printed, the number of dots within a distance of 1 from the center of the circle will be approximately Posterior- probability 78.55. Certainly, the difference between the simulated values and the measured values will decrease as the number of simulations Figure 2. Overall concept of the Bayesian approach using a Markov increases. chain Monte Carlo (MCMC) simulation.

Bayesian hierarchical model to the prior distribution. In turn, μ follows a hyperprior distribu- 2 In the fixed effect model, which only considers the within-study tion, which is a normal distribution with μ0 as mean and η0 as variations, the average and variance of the standard normal distri- variance, and τ2 follows a hyperprior distribution with p as mean 2 bution are input to the prior distribution as follows: and q as variance. These parameters μ0, η0 , p, and q of the prior distributions μ and τ2 are hyperparameters.

Ti ~ N(θ, vi), i= 1, …, k 2 θ ~ N(μ0, η0 ) Ti ~ N(θ, vi), i= 1, …, k 2 θi ~ N(μ, τ ) 2 2 where Ti is the actual observation effect of thei th study, and vi μ ~ N(μ0, η0 ), τ ~ IG(p, q) is the variance of the ith study. Here, θ is the true value of the treat- ment effect and a common effect size to be inferred by the fixed In this way, in the random effect model, the treatments effect of effect model. each study are connected by the hyperprior distributions from θi

In the random effect model, which considers both within-study to θk. Thus, it is no longer an independent model but a hierarchi- and between-study variations, the total average treatment effect of cal model [5]. the population (μ) and the between-study variance (τ2) are input

www.e-epih.org | 3 Epidemiol Health 2019;41:e2019013

1.2. Frequentist method

Summary of the Bayesian method The frequentistwhere method is A not denotes related to external the referenceinformation, and treatment, the probability Jthat denotes the com- With Figure 2, the Bayesian method can be summarized as follows. parative treatment (J= B,C,…), d denotes the study design, and i the research hypothesis is true within the present data (likelihood) is already specified First, a prior distribution (prior probability) is selected. For a denotes1.2. Frequentist the ith study method in the dth study design. (test for a p-value of 0.05 or a 95% CI). Thus, it only determines whether to accept or conjugate prior distribution, normal distribution, beta distribu- The design by treatment interaction model is a frequentist NMA reject the researchThe hypothesis frequentist by method the significance is not related level. to external information, and the probability that tion, and inverse gamma distribution are generally used. model that considers1.2. both Frequentist heterogeneity method between studies and in-

Second, the likelihood is calculated from the present data andThe afollowing consistency showsthe research the design betweenhypothesis by treatment is study true interaction within designs themodel present [4]. for inconsistency: data (likelihood) is already specified The frequentist method is not related to external information, and the probability that Bayesian hierarchical model is created – in NMA, the likelihood Here,(test for a p-value represents= of +0.05 or +the a 95% effect +CI). Thus, size it onlyof Jdetermi treatmentnes whether in contrastto accept or to �� �� �� �� �� is mainly expressed as the treatment effect θ. the A reference�� treatmentthe�� research �in hypothesis the�� ith isstudy true within of the the presentdth study data (likeli design,hood) is already specified where A denotes thereject reference the𝓎𝓎 treresearchatment,1.2.δ FrequentistJ hypothesisdenotesβ the comparat method ωby theive εsignificance treatment (J=B,C,…), level. d denotes the study design, and i denotes the ith study in the dth study design. Third, the prior distribution and likelihood are input to the MC­ considering heterogeneity between studies and inconsistency be- (test for a p-value of 0.05 or a 95% CI). Thus, it only determines whether to accept or The following shows the design by treatment interaction model for inconsistency: MC simulation, and a distribution that best converges the posteri- tween study Thedesigns. frequentist It method is called is not meanrelated to difference, external information, log and risk-ratio, the probability or that The design by treatment interaction modelreject is a thefrequentist research NMA hypothesis model that by theconsiders significance level. = + + + or distribution is set. The probability of stable distribution and the log odds ratiothe (OR). research Next,hypothesis is truerepresents within the present the treatment data (likelihood) contrast is already specified as both heterogeneity between studies andThe inconsistency following�� �� showsbetween ��the st udydesign designs�� by treatment [4].�� interaction model for inconsistency: area under the posterior distribution function can be determined a primary index of interest,�� and ��the effect� size�� of J treatment in where A denotes(test the forreference a p-value𝓎𝓎 treatment, of 0.05δ J ordenotes a β95% the CI). comparat ωThus, itive onlyε treatment determi (J=B,C,…),nes whether d to accept or through the MCMC simulation. contrastdenotes the to study the desig An, referenceand i denotes the treatment. ith study in the d th=Then, study +design. +represents + the 1.2. Frequentist1.2. Frequentistmethod method reject the research hypothesis by the significance�� �� level.�� �� �� Lastly, statistical reasoning for the treatment effect is performedHere, representsheterogeneity the effect between size of J treatment studies in contrastin the tosame �� the A d reference study�� design.� It�� is the �� where A denotes the reference𝓎𝓎 treatment,δ J denotesβ the comparatω iveε treatment (J=B,C,…), d �� The followingdenotes shows the studythe design design, by and treatment i denotes theinteraction ith2 study inmodel the d thfor study inconsistency: design. with the determined posterior distribution. Therefore, the Bayesitreatment-𝓎𝓎 insame theThe ith studydesignas calculating of bythe treatment dth study the design,interaction interstudy considering model heterogeneity isvariation a frequentist between τ NMA with model a random that considers ef- The frequentistThe method frequentist is methodnot related is not to related external to external information, information, and andthe theprobability probability that that 1.2. Frequentist method = + + + an NMA can analyze the posterior distributionthe research hypothesis even isif true it withinis notstudies the a present andfect inconsistencydataboth model (likeli heterogeneityhood) between in is pairwisealready studybetween specified designs. meta-analysis. studies It is andcalled inconsistency mean Then, difference, between log represents study designs incon [4]. - the research hypothesis is true within the present data (likelihood) is already specifiedThe design by treatment�� �� interaction�� model�� is a�� frequentist NMA model that considers standard distribution generally used in statistics. sistency. Thewhere loop A denotes inconsistency the reference𝓎𝓎 tre��atment, isδ J denotesbasedβ�� the comparatonω � Luive ε treatment��& Ades (J=B,C,…), [6], and d (test for a p-value of 0.05 or a 95% CI).risk-ratio, Thus, itor only log determi OR (oddsnes whetherratio). Next, to accept represents or the treatment contrast as a denotes the studyboth�� desigheterogeneityn, and i denotes between the ith study studies in the and dth studyinconsistency design. between study designs [4]. (test for a p-value of 0.05 or a 95% CI). Thus, it only determithe designnes whether inconsistency to acceptδ or was approached through meta-regres- The frequentist method is not related to externalreject information, the research hypothesis and the by the probabilityprimary significance index level. ofHere, that interest, representsand the effect the size effect of J size treatment of J treatment in contrast in to contrast the A to the A reference

Frequentist method sion analysis�� of the difference between study designs. Obviously, reject the research hypothesis by the significancereference level. treatment. Then,�� The represents design by the treatment heterogeneity interaction between model studies is a frequentist in the NMA model that considers The following shows the design by treatment interactiontreatment model𝓎𝓎 in f orthe inconsistency: ith study of the dth study design, considering heterogeneity between The frequentist method is not related to external information, in the consistency�� modelHere, that represents satisfies the effect the sizeconsistency of J treatment assumption, in contrast to the A reference the research hypothesis is true within the present data (likelihood) is already specified �� The following shows the design by treatmentsame interaction d study design. model It fisor the βinconsistency:both same heterogeneity as calculating �� between the interstu studiesdy variationand inconsistency τ2 with a between study designs [4]. = + + + �� and the probability that the research hypothesis is true within the studies= 0 forand allinconsistency studytreatment designs between𝓎𝓎 in the(d) study ith and study designs. contrast of the It d this studycalledtreatments design, mean consideringdifference, (J). heterogeneitylog between �� �� �� �� �� �� random�� effect �model ��in pairwise meta-analysis. Then, represents inconsistency. (test for a p-valuepresent ofdata 0.05 (likelihood) or a 95% isCI). already Thus, specifiedit whereonly A denotesdetermi (test the reference fornes= 𝓎𝓎 atrewhether atment,p-value + δ J denotes + toβof theaccept comparat +ω ive orεis treatment a within-study (J=B,C,…), d error. Generally,�� it is assumed normal dis- risk-ratio, or log OR (oddsstudies ratio). and inconsistencyNext, represents between thestudy treatment designs. contrastIt is called as ma ean difference, log denotes the study desig�� n, and i denotes the�� ith study�� in the dth�� study design. Here, represents the ω effect� size of J treatment in contrast to the A reference 0.05 or a 95% CI). Thus, it only determines whether to�� acceptThe loopor inconsistencytribution. is based on Lu & Ades [6], and the desig�� n inconsistency was �� �� � �� �� δ where A denotes the reference𝓎𝓎 treatment,δ J denotesβ the comparatω iveεprimary treatment index (J=B,C,…), of interest,𝓎𝓎�� d risk-ratio, and or the log effect OR (odds size of ratio). J treatment Next, in represents contrast tothe thetreatment A contrast as a reject the researchreject the hypothesis research hypothesis by the denotessignificance by the the study significance desig level.n, and i denotes level. the ith studyapproached in the dth study through design. meta-regression treatment in analysis the ith study of the of differencethe dth study between design, considering study heterogeneity between The design by treatment interaction model is a frequentist NMA model that considers �� reference treatment. Then,primary index represents of interest, the heterogeneity and the effect δbetween size of Jstudies treatment in the in contrast to the A The following shows the design by treatment interaction modeldesigns. Obviously, in the consistencystudies modeland inconsistency that satisfies betweenthe consistency study designs.assumption, It is called mean difference, log both heterogeneity between studies and inconsistencyBAYESIAN between study designsNMA [4]. USING�� R “gemtc” PACKAGE The following shows the design by treatment interaction model for inconsistency: β�� for inconsistency: The design by treatment interaction model is a frequentistsame NMA d modelstudyrisk-ratio, design.that considers orItreference islog the OR same treatment.(odds as ratio). calculating Then, Next, the represents represents interstu dy thethe variation heterogeneitytreatment τ2 contrast with between a as a studies in the �� �� �� Figure 3 shows thesame flowchart d study design. for It is usingtheβ δ same the as calculatingR package the interstu “gemtcdy” variation τ2 with a = both + heterogeneity + between + studies and inconsistency betweenrandom steffectudyprimary designsmodel index in [4]. pairwise of interest, meta-analysis. and the effect Then, size of J represents treatment10 ininconsistency. contrast to the A Here, represents the effect size of J treatmentfor NMA in contrast using to the the A reference Bayesian method. When ��coding the data first, �� reference treatment.random effect Then, model represents in pairwise the meta-analysis.heterogeneityω� betweenThen, studies represents in the inconsistency. �� �� �� ��𝓎𝓎�� �� The loop inconsistency is based on Lu & Ades [6], and the design inconsistency was treatment in the ith study of the dth study design, considering heterogeneity between �� �� �� �� � �� �� � 𝓎𝓎 δ β ω ε same d study design. It is theβ same as calculating the interstudy variationω τ2 with a where A denotes the reference treatment, J denotes the comparative treatment (J=B,C,…), d approached through The meta-regression loop inconsistency analysis is based of theon Lu difference & Ades [6], b etweenand the desig studyn inconsistency was denotes the study design, and i denotes the ithHere, study in represents thestudies dth and study the inconsistency effect design. size between of J treatment study designs. in contrastIt is called to m theean Adifference, reference log �� random effectapproached model in throughpairwise meta-analysis. meta-regression Then, analysis represents of the inconsistency. difference between study �� designs. Obviously, in the consistency model that satisfies the consistency assumption, treatment𝓎𝓎 in risk-ratio,the th study or log of OR the (odds th studyratio). design,Next, considering represents the heterogeneity treatment contrast between as a �� i d ω� �� The loop inconsistencydesigns. Obviously, is based in onthe Lu consistency & Ades [6], model and the that desig satisfiesn inconsistency the consistency was assumption, primary index of interest, and the effectδ size of J treatment in contrast to the A studies and inconsistency between study designs. It is called mean difference, log The design by treatment interaction model is a frequentist NMA model that considers approached through meta-regression analysis of the difference between study reference treatment. Then, represents the heterogeneity between studies in the 10 risk-ratio, or log OR (odds ratio). Next,�� represents the treatmentdesigns. contrast Obviously, as a in the consistency model that satisfies the consistency assumption, β�� 10 both heterogeneity between studies and inconsistencysame d study between design. It is thest udysame�� asdesigns calculating [4]. the interstudy variation τ2 with a primary index of interest, and the effectδ size of J treatment in contrast to the A random effect model in pairwise meta-analysis. Then, represents inconsistency. �� 10 Fixed or � reference treatment.The loop inconsistencyThen, represents is based on theLu &heterogeneity Ades [6], and ωthe between design inconsistencystudies in the was random�� effect �� same d studyapproached design. It throughis theβ model same meta-regression as calculating analysis the ofinterstu the differencedy variation between τ2 with study a Here, represents the effect size of J treatment in contrast to the A reference �� random effectdesigns. model Obviously, in pairwise in the meta-analysis.consistency model Then, that satisfies represents the consistency inconsistency. assumption, 𝓎𝓎�� �� treatment in the ith study of the dth study design, considering heterogeneity between� The loop inconsistency is based on Lu & Ades [6], and ωthe design inconsistency was

10 studies and inconsistency between studyapproached designs. through It is meta-regression called mean analysis difference, of the differencelog between study

designs. Obviously, in the consistency model that satisfies the consistency assumption, risk-ratio, or log OR (odds ratio). Next, represents the treatment contrast as a �� primary index of interest, and the effectδ size of J treatment in contrast to the A 10

reference treatment. Then, represents the heterogeneity between studies in the �� �� same d study design. It is theβ same as calculating the interstudy variation τ2 with a Treatment ranking random effect model in pairwise meta-analysis. Then, represents inconsistency. �� � The loop inconsistencyFigure 3. Flow is chartbased of on network Lu & Ades meta-analysis [6], and ωthe using desig then “inconsistencygemtc” R package. was MCMC, Markov chain Monte Carlo; DIC, deviance infor- mation criterion. approached through meta-regression analysis of the difference between study

designs. Obviously,4 | www.e-epih.org in the consistency model that satisfies the consistency assumption,

10

Shim SR et al. : NMA: application and practice using R software you must set the variable names in accordance with the relevant description= “Bayesian NMA binary data”) function. The process is as follows: network setup -> select a net- work model (fixed or random) -> select the MCMC convergence The “mtc.network” function performs network setup with the optimal model -> statistical reasoning in the final model (Figure 3). previously set “data_b_bin” and declares it as “network_b_bin”. There are two R packages for NMA: gemtc“ ” for Bayesian NMA · plot(network_b_bin) and “netmeta” for frequentist NMA. Before starting the analysis, you must install the packages with the following commands. For The plot function graphically shows the direct comparison be- a more detailed explanation, you can refer to the detailed code, tween the treatment groups comprising the network (Figure 4). data, and references for each package [7]. The thickness of the edge for connecting nodes means the · install.packages(“netmeta”) amount of data. · install.packages(“gemtc”) · summary(network_b_bin)

When running Bayesian NMA, the MCMC simulation is used, You can see the overall status of network setup. You can also see and the application required for this is Just Another Gibbs Sam- the number of 2-arm or 3-arm studies and number of responses pler; Gibbs Sampler is a representative method of MCMC (JAGS). to individual treatment. Thus, it numerically describes the above You can download the latest version 4.0 or higher from Google network plot. and install it. In addition, you must install the “rjags” package to use JAGS in R as follows: Network model · install.packages(“rjags”) Once the network setup is completed, you must set a network model of fixed effect model or random effect model. Although it We mark R commands with a dot (‘· ’) in front of them, to dis- is generally recommended to select a random effect model con- tinguish them from the main text. When long commands are ex- sidering the between-study variation, this study will explain with tended to the next line, there is no dot at the beginning of the next a fixed effect model for convenience. line. Thus, when you enter the command in the R software, you · model_b_bin_fe < - mtc.model(network_b_bin, linearModel= must type them without the dot (‘· ’) in front of them. ‘fixed’, n.chain= 4)

Data coding and loading With the “mtc.model” function, you can load the network set- First, load the “gemtc” package to perform Bayesian NMA. up data “network_b_bin” and set the fixed effect model and ran- · library(gemtc) dom effect model as “model_b_bin_fe”. “n.chain” indicates the number of chains to be performed in the following MCMC simu- Next, load the binary example file from the working directory lation. into the memory of R with the following command. Thus, you should save Supplementary Material 1 as “bin_dn.csv” in the specified working folder. · data_b_bin= read.csv(“bin_dn.csv”, header= TRUE)

This loaded file is saved as “data_b_bin” in the R memory. The “gemtc” package has many sub-functions. Among them, the “mtc.network” function can be run only if the data function name is a specific name. In the binary data, it must be “study,” “re- sponders,” “sampleSize,” or “treatment.” In this example, the varia- ble name is different; thus, you must change the variable name with the “colnames” command as follows: · colnames(data_b_bin) < - c(“study”, “responders”, “sampleSize”, “treatment”) “colnames” specifies the variable name (column), and you must sequentially enter the variable names of data_b_bin data.

Network setup For a network analysis of the prepared “data_b_bin” data, you must set up the network using the “mtc.network” function as fol- Figure 4. Network plot using the “gemtc” package. A: placebo; B: IV lows: (single); C: IV (double); D: topical; E: combination. IV, intravenous · network_b_bin < - mtc.network(data.ab= data_b_bin, injection.

www.e-epih.org | 5 Epidemiol Health 2019;41:e2019013

Markov chain Monte Carlo (MCMC) simulation and · summary(mcmc_b_bin_fe) convergence diagnosis Running MCMC simulation MCMC simulation and convergence status Once the network model is set up, you can perform a MCMC To verify if the MCMC simulation converged well, you can simulation. The overall process is to set and run an appropriate check the following items in combination. number of simulations, and then check whether the results con- verge. MCMC error First, an example of the fixed effect model will be explained. A smaller MCMC error indicates a higher accuracy, which means · mcmc_b_bin_fe < - mtc.run(model_b_bin_fe, n.adapt= 5000, a good convergence. Therefore, a sufficient sample size should be n.iter= 10000, thin= 20) achieved by performing many simulations and the burn-in pro- cess to remove the effect of initial values, and the data extraction In the “mtc.run” function, enter the name of the fixed effect mod- interval “thin” should be adjusted appropriately. el that has been set. “n.adapt= 5000” means to discard no.1-5,000 of the iterations. This is called burn-in, which is to remove a cer- Deviance information criterion tain part of the beginning of the created random numbers to ex- The deviance information criterion (DIC) is expressed as DIC = - - clude the effect of the initial values of the algorithm. “n.iter= 10000” D+pD, where D is the sum of residual deviances and pD is an esti- means to perform 10,000 simulations and “thin= 20” means to mated value of the parameter. Thus, the DIC considers both the extract every 20th value. fitness and complexity of the model, and the smaller the DIC is, To summarize the above explanation, the first to 5,000th data the better the model. are discarded (to reduce the effect of initial values in simulation), simulations are performed 10,000 times with 5,001st to 15,000th Trace plot and density plot values, and every 20th value is extracted (e.g., 5,020, 5,040….). If the trace plot (a graph visually showing the simulation result) In the Bayesian analysis, prior distribution considering multi- has no specific pattern and the chains are entangled, it is consid- chain is input to determine the posterior distribution. The multi- ered that the convergence is good. The density plot is a posterior chain simulation is performed by setting multiple initial values for distribution (posterior density function) and if the shapes are sig- the prior parameter of prior distribution, that is, the hyperparam- nificantly different for the same number of simulations, it means eter d (e.g., 4 values of -1, 0, 1, and 2). Therefore, because every the data did not converge well. 20th data are extracted among 10,000 simulations, 500 data points Figure 5A is the case when 100 and 500 were input as the total are extracted in each chain. number of iterations with no burn-in. In the first 100 iterations You can see a more detailed explanation by using the summary (Figure 5A left), the four chains have severe variations and are not command: even, but at approximately 500th iteration (Figure 5A right), the

Trace of d.A.B Trace of d.A.B

-1.0 -0.5

-1.8 -2.0 0 100 200 300 400 500 0 20 40 60 80 100 A Iterations Iterations

-1.0 0.5

-2.0 0.0 6,000 8,000 10,000 12,000 14,000 -2.0 -1.5 -1.0 B Iterations n=500, bandwidth=0.04565

-1.0 0.5

-2.0 0.0 -2.0 -1.5 -1.0 -0.5 6,000 8,000 10,000 12,000 14,000 C Iterations n=1,000, bandwidth=0.03867

Figure 5. Trace and density plots: (A) iterations=100 (left) vs. 500 (right); (B) iterations=10,000 & thin=20; (C) iterations=10,000 & thin=10.

6 | www.e-epih.org Shim SR et al. : NMA: application and practice using R software graph becomes even, with no specific pattern. Therefore, it is de- · summary(mcmc_b_bin_fe) sirable to perform at least 500 simulations for each channel. For · gelman.diag(mcmc_b_bin_fe) burn-in to exclude the effect of initial values, 1-100th data points · gelman.plot(mcmc_b_bin_fe) must be discarded without question because they are too uneven, and the number of discarded values must be at least 500. In this Consistency test example, the values were discarded for up to 5,000 times to mini- Consistency test in the assumptions of NMA is a critical tool mize the effect of the algorithm. that determines the applicability of NMA results. In Figure 5B, when the simulations are performed 10,000 times · nodesplit_b_bin_fe < - mtc.nodesplit(network_b_bin, linear­ with an extraction interval “thin” of 20, 500 data points are ex- Model= ‘fixed’, n.adapt= 5000, n.iter= 10000, thin= 10) tracted from each channel. If 10 is input to thin (Figure 5C), 1,000 · plot(nodesplit_b_bin_fe) values are extracted per channel, which increases the sample size · plot(summary(nodesplit_b_bin_fe)) and results in a more even distribution of the trace plot. Further- more, 1,000 samples of post density function were selected be- The fixed effect model “nodesplit_b_bin_fe” is created for con- cause it looks more similar to the normal distribution. sistency test by entering the network set-up data in the “mtc.node- The finally selected model performed burn-in for 5,000 values split” function. The MCMC simulation is also performed. and 10,000 simulations, and extracted every 10th data point for The variations between treatments and the consistency test re- 1,000 samples per channel. sults of all individual treatments can be easily seen. As a result of · mcmc_b_bin_fe < - mtc.run(model_b_bin_fe, n.adapt= 5000, the consistency test, the p-value of treatments E versus D was 0.043, n.iter= 10000, thin= 10) indicating inconsistency. However, no statistical significance was observed in all the other treatments. Therefore, it is desirable to Compared to the extraction interval thin of 20, the total sample set up a random effect model for a more robust analysis. size increased and the MCMC standard error of the treatment group AB decreased from 0.004 to 0.003, thus increasing the ac- Forest plot curacy. However, the DIC decreased from 56.24 to 56.63, which is Forest plot allows graphical comparison of the effect sizes by practically insignificant. treatment group through NMA. · forest(relative.effect(mcmc_b_bin_fe, t1= “A”), digits= 3) Gelman-Rubin statistics and plot · gelman.diag(mcmc_b_bin_fe) When you enter the final model through MCMC simulation in · gelman.plot(mcmc_b_bin_fe) the forest function, a forest plot with A reference treatment is cre- The “gelman.diag” command displays the Gelman-Rubin sta- ated (Figure 6). tistics on the console, and the “gelman.plot” command draws the The effect size (OR, blood transfusion rate) of every treatment Gelman-Rubin plot. As the number of simulations increases, it was lower than the placebo, and the 95% credible intervals did approaches 1, and the variations must be stabilized so that it can not overlap. be said to have converged well. In particular, the blood transfusion rate of the combination treatment method (E) was statistically significantly lower com- Selecting the final model for MCMC simulation pared to those of all the other treatments including single intrave- For MCMC simulation, a model that converges best should be nous injection (IV) injection [B, IV(single)], double IV injection selected by adjusting the number of chains appropriate for multi- [C, IV(double)], and topical application method [D, topical]. chain, the number of data for removal of initial effect (burn-in), the number of iterations, and the extraction interval (thin). Treatment ranking For the fixed effect model of this example, 4 chains, 5,000 burn- One of the most important functions of NMA is that the com- ins, 10,000 iterations, and an interval of 10 were selected, to suffi- ciently remove the effect of initial values, increase the iterations OR (95% CrI) and extraction interval, and minimize the MCMC error and DIC variation with almost no variations and stability of various plots. However, you should adjust the iterations appropriately because it can take significant time depending on the computer specifica- tions.

Burn-in 5,000, iteration 10,000, thin 10 · mcmc_b_bin_fe < - mtc.run(model_b_bin_fe, n.adapt= 5000, Figure 6. Forest plot_reference A. A: placebo; B: IV (single); C: IV n.iter= 10000, thin= 10) (double); D: topical; E: combination. IV, intravenous; OR, odds ratio; · plot(mcmc_b_bin_fe) CrI, credible interval.

www.e-epih.org | 7 Epidemiol Health 2019;41:e2019013 parative advantages of treatments can be determined. In other follows: effect size data format -> select a network model (fixed words, the cumulative probability that the top priority to the low- or random) -> statistical reasoning in the final model. est priority treatments are selected can be calculated. · ranks_b_bin_fe < - rank.probability(mcmc_b_bin_fe, pre- Data coding and loading ferredDirection= -1) Load the “netmeta” package to perform frequentist NMA. · print(ranks_b_bin_fe) · library(netmeta) Next, load the binary example file from the working directory Enter the final MCMC model in the “rank.probability” func- into the memory of R with the following command (Supplemen- tion. Set the “preferredDirection” to ‘-1’ or ‘1’, depending on wheth- tary Material 1). er a smaller effect size indicates a better treatment, or vice versa. · data_f_bin= read.csv(“bin_dn.csv”, header= TRUE) In this example, it is set to ‘-1’ because an effect size smaller than that of the reference treatment means a better treatment. The “netmeta” package can be run only if the variable name of As can be seen in the probability table, the best treatment is E the effect size data type is “studlab,” “TE,” “seTE,” “treat1,” or “treat2.” (combination) at 99.8%, followed by C (IV double) at 68.2%, B (IV Because the data in this example is raw data, it must be converted single), D (topical), and A (placebo). to effect size data type, and the variable names must be matched as well. FREQUENTIST NMA USING R “netmeta” · data_f_bin < - pairwise(trt, event= d, n= n, studlab= study, PACKAGE data= data_f_bin, sm = “OR” )

Figure 7 shows the flowchart for using the R package “netmeta” Enter the first treatment variable “trt” in the pairwise function, for NMA using the frequentist method. First, you must change and the other variables are matched: frequency (event= d), sample the data format to the effect size data format, and set the variable size (n= n), and study name (studlab= study). Lastly, select wheth- names in accordance with the relevant function. The process is as er to use OR or relative risk (RR) data.

Effect size format

Fixed or random effect model

If consistent (p-value of inconsistency model>0.05)

If consistent (side-split test between direct and indirect effect sizes)

Forest plot Treatment ranking League table Network plot

Figure 7. Flow chart of network meta-analysis using the “netmeta” R package.

8 | www.e-epih.org Shim SR et al. : NMA: application and practice using R software

Network model Enter the network model in the “netsplit” function and perform If the conversion to the effect size data type has been complet- a consistency test for each treatment. ed, select the fixed effect model or random effect model for the In all comparisons of treatments, the p-value was statistically network model. insignificant, showing no inconsistency. Therefore, the consisten- · network_f_bin_fe < - netmeta(TE, seTE, treat1, treat2, stud- cy model is supported once again. lab, data= data_f_bin, sm= “OR”, reference= “A”, comb.fixed= TRUE, comb.random= FALSE) Forest plot Set the fixed effect model to network_f_bin_fe by loading effect A forest plot allows a graphical comparison of the effect sizes size (TE), standard error (seTE), treatment 1 (treat1), treatment 2 by treatment group through NMA. (treat2), study name (studlab), and data_f_bin in the netmeta · forest(network_f_bin_fe, ref= “A”, digits= 3, xlab= “Odds Ratio”) function. Enter the set network model and “A” for reference treatment in Network plot the forest function. · tname_f_bin < - c(“Placebo”, “IV(single)”, “IV(double)”, “Top- The effect size (OR, blood transfusion rate) of every treatment ical”, “Combination”) was lower than that of the placebo, and the 95% CIs did not overlap. · netgraph(network_f_bin_fe, labels= tname_f_bin) In particular, the transfusion rate (OR, 0.033; 95% CI, 0.006 to 0.175) of the combination treatment (E) was statistically signifi- To enter the treatment name in the network plot, set “tname_f_ cantly lower than those of all the other treatments including sin- bin” as the treatment name. gle IV injection [B, IV (single)] (OR, 0.273; 95% CI, 0.186, to 0.399), You can enter “network_f_bin_fe” in the “netgraph” function to double IV injection [C, IV (double)] (OR, 0.229; 95% CI, 0.146, graphically show a direct comparison between the treatment 0.360), and the topical application method [D, topical] (OR, 0.329; groups comprising the network. The thickness of the edge for 95% CI, 0.197 to 0.550). connecting nodes means the amount of data. Treatment ranking Network model summary estimates One of the most important functions of NMA is that the com- Once the network model is set up, you can use “summary” to parative advantages of treatments can be determined. In other see the overall summary of the model. words, the cumulative probability that the top priority to the low- · summary(network_f_bin_fe) est priority treatments are selected can be calculated. · ranks_f_bin_fe < - netrank(network_f_bin_fe, small.values= The total number of studies is 21, the number of treatments is 5, ”good”) the number of paired direct comparisons is 25, and the number · print(ranks_f_bin_fe, sort= FALSE) of study designs is 6 (AB, AC, AD, AE, BDE, and ABE). Furthermore, the reference variable is A and the effect size is OR. Enter the network model in the “netrank” function. Enter “good” In this example, the fixed and random effect models have the in “small.values” if a smaller effect size indicates a better treatment, same effect size because the between-study variance (tau) is zero. or “bad” otherwise. As can be seen in the probability table, the best treatment is E Consistency test (combination) at 99.38%, followed by C (IV double) at 65.43%, B Consistency test in the assumptions of NMA is a critical tool (IV single), D (topical), and A (placebo). that determines the applicability of NMA results. COMPARISON OF NMA RESULTS: BAYESIAN VS. Global approach FREQUENTIST METHOD AND R VS. STATA This approach calculates the regression coefficient of the incon- SOFTWARE sistency model for each study design and then tests the linearity of the regression coefficients for all models by using the Wald test. Table 1 outlines the Bayesian and frequentist NMA. The NMA The consistency test was performed for all models in the same way used in STATA is a design by treatment interaction model based as for the STATA NMA model [4]. on regression analysis, which considers both heterogeneity be- · decomp.design(network_f_bin_re) tween studies and inconsistence between study designs [4]. How- As a result of the consistency test for every model, the p-value ever, the R “netmeta” package uses an electrical network model, was 0.9942. As this supports consistency, which is the null hypo­ which changed it slightly [8,9]. thesis, this network model is acceptable. The Bayesian method also shows similar values between the fixed and random effect models, which is an almost identical re- Local approach sult to that of the frequentist method. · print(netsplit(network_f_bin_re), digits= 3)

www.e-epih.org | 9 Epidemiol Health 2019;41:e2019013

Table 1. Comparison effect sizes between frequentist and Bayesian method in network meta-analysis Frequentist approach Bayesian approach1 Data type Treatment R "nemeta" package R "gemtc" package STATA2 Fixed Random Fixed Random Binary Placebo 1.000 (reference) 1.000 (reference) 1.000 (reference) 1.000 (reference) 1.000 (reference) IV (single) 0.273 (0.186, 0.399) 0.273 (0.186, 0.399) 0.273 (0.186, 0.399) 0.263 (0.181, 0.379) 0.264 (0.173, 0.399) IV (double) 0.229 (0.146, 0.360) 0.229 (0.146, 0.360) 0.229 (0.146, 0.360) 0.220 (0.138, 0.346) 0.220 (0.138, 0.357) Topical 0.329 (0.197, 0.550) 0.329 (0.197, 0.550) 0.329 (0.197, 0.550) 0.324 (0.193, 0.534) 0.322 (0.180, 0.551) Combination 0.033 (0.006, 0.175) 0.033 (0.006, 0.175) 0.033 (0.006, 0.175) 0.015 (0.001, 0.089) 0.014 (0.000, 0.083) Values are presented as odds ratio (95% confidence interval). IV, intravenous injection. 1Effect size (95% credible interval). 2Design-by-treatment interaction model.

CONCLUSION ORCID

This paper presented as little statistical theory as possible and Sung Ryul Shim: http://orcid.org/0000-0003-4143-7383; Seong- focused instead on the actual performance of meta-analysis meth- Jang Kim: http://orcid.org/0000-0001-5803-9934; Jonghoo Lee: ods, so that researchers who have not majored in statistics could http://orcid.org/0000-0003-2626-7099; Gerta Rucker: http://orcid. easily understand it. In other words, this study aimed to provide org/0000-0002-2192-2560 researchers from different fields an overview of how to adequately use already developed statistical methods in their field of study REFERENCES and interpret the results. This study compared the Bayesian network meta-analysis using 1. Hwang SD, Shim SR. Meta-analysis: from forest plot to network the “gemtc” package and the frequentist network meta-analysis meta-analysis. Seoul: Hannarae; 2018, p. 180-223 (Korean). using the “netmeta” package. We found that these two methods 2. Shim SR. Network meta-analysis using R software: Bayesian & produced the same results. Refer to the references for detailed de- frequentist approach. Gwacheon: SDB Lab; 2019 (Korean). scriptions for continuous data, besides the binary data presented 3. Shim S, Yoon BH, Shin IS, Bae JM. Network meta-analysis: appli- in the examples in this study [2]. cation and practice using Stata. Epidemiol Health 2017;39:e2017047. We hope that this study will help researchers to perform meta- 4. White IR. Network meta-analysis. Stata J 2015;15:951-985. analysis and conduct related research more easily. 5. Kim DH, Jang EJ, Hwang JS. Meta-analysis using R & WinBUGS. Paju: Free Academy; 2014, p. 255-337 (Korean). SUPPLEMENTARY MATERIALS 6. Lu G, Ades AE. Assessing evidence inconsistency in mixed treat- ment comparisons. J Am Stat Assoc 2006;101:447-459. Supplementary materials are available at http://www.e-epih.org/. 7. GitHub. R software “gemtc” & “netmeta” packages [cited 2018 Korean version is available at http://www.e-epih.org/. May 10]. Available from: https://github.com/gertvv/gemtc. 8. Rücker G. Network meta-analysis, electrical networks and graph CONFLICT OF INTEREST theory. Res Synth Methods 2012;3:312-324. 9. Rücker G, Schwarzer G. Reduce dimension or reduce weights? The authors have no conflicts of interest to declare for this study. Comparing two approaches to multi-arm studies in network me- ta-analysis. Stat Med 2014;33:4353-4369. ACKNOWLEDGEMENTS

None.

10 | www.e-epih.org