View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Elsevier - Publisher Connector VALUE IN HEALTH 17 (2014) 629– 633 Available online at www.sciencedirect.com journal homepage: www.elsevier.com/locate/jval An Updated Method for Risk Adjustment in Outcomes Research Suhail A.R. Doi, FRCP, PhD*, Jan J. Barendregt, PhD, Chalapati Rao, PhD School of Population Health, University of Queensland, Brisbane, Australia ABSTRACT Objective: To demonstrate why meta-analytic methods need modifi- with this modified method but cannot be done using standard meta- cation before they can be used to aggregate rates or effect sizes in analysis. Conclusions: We conclude that this method allows risk outcomes research, under the constraint of no common underlying adjustment to be performed in situations in which currently the effect or rate. Methods: Studies are presented that require different fixed- or random-effects methods of meta-analysis are inappropri- types of risk adjustment. First, we demonstrate using rates that ately used. The latter should be avoided when the underlying aim is external risk adjustment through standardization can be achieved risk adjustment rather than meta-analysis. fi using modi ed meta-analytic methods, but only with a model that Keywords: age standardization, burden of disease, fixed-effects model, fi allows input of user-de ned weights. Next, we extend these observa- meta-analysis, population standardization, quality-effects model, tions to internal risk adjustment of comparative effect sizes. Results: random-effects model. We show that this procedure produces identical results to conven- tional age standardization if a rate is being standardized for age. We Copyright & 2014, International Society for Pharmacoeconomics and also demonstrate that risk adjustment of effect sizes can be achieved Outcomes Research (ISPOR). Published by Elsevier Inc. Introduction completely. This is of particular importance to studies of quality improvement, in which many estimates of disease burden or risk The term standardization refers to the process of facilitating are strongly dependent on subpopulation, with rates of incidence comparison of summary measures of burden or risk of disease or mortality being much higher or lower between different across populations. Such standardization can be done in two subpopulations. In this situation, the differences between pop- ways. Internal standardization refers to the process of ensuring ulations independent of the confounding by sizes of subpopula- that summary measures adequately reflect the distribution of tions with different risks can be determined by standardizing burden or risk of disease within subpopulations of the same against a common external standard. In this sort of standardiza- overall population. If the subpopulations are investigated inde- tion, the standardized rate is in itself useful only for comparison pendently of each other (e.g., in separate studies), the overall and has no intrinsic interpretation. summary measure for the total population needs to take account The process of risk adjustment encompasses both standardiza- of the actual size of each subpopulation within the overall tion and other procedures for accounting for the effects of population structure so that the overall summary reflects the subpopulations with different risks. In this article, risk adjust- actual population meaningfully. For example, a summary meas- ment and standardization will both refer to methods of adjust- ure of mortality reported in different subpopulations by age can ment based on weighted averages in which the weights are be combined into a summary measure for the total population chosen to provide an “appropriate” basis for the comparison after standardizing against the actual distribution of ages. When (i.e., a “standard”). The latter is generally either the subpopula- such standardized rates are compared across different popula- tion sizes from each of the populations in the comparison or from tions, they can be interpreted as the mortality rate for an average a relevant external population. A common method in epidemi- member of each specific population. Thus, mortality rates in ology for this purpose has been direct standardization because it Australia versus India, obtained from different subpopulations, can be applied on the basis of any subpopulation distribution, for need to be internally standardized against the actual population example, on the basis of age, geographical clusters, and cancer structure in Australia and India, respectively, if a summary for incidence. Direct standardization is simply a process of weighted each country is to be compared. averaging of the subpopulation-specific rates to arrive at a stand- External standardization can also be done by replacing the ardized estimate that reflects a given subpopulation structure. internal standard described above with a common external The distribution of the “standard” provides the weights and standard against which subpopulations from different popula- usually represents the current or most common subpopulation tions are standardized, thus removing the subpopulation effect structure for internal and external standardization, respectively, * Address correspondence to: Suhail A.R. Doi, Clinical Epidemiology Unit, School of Population Health, University of Queensland, Herston Road, QLD 4006, Brisbane, Australia. E-mail: [email protected]; [email protected]. 1098-3015$36.00 – see front matter Copyright & 2014, International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. http://dx.doi.org/10.1016/j.jval.2014.05.003 630 VALUE IN HEALTH 17 (2014) 629– 633 and could represent subpopulation sizes based on age or geo- demonstrate equivalence. In the second example, this is then graphic cluster or any other distribution of whatever standard is extended to internal risk adjustment of a relative risk (RR) to be applied. This provides, for each population, one risk- measure using the incidence rates of cancer in each subpopu- adjusted or standardized rate that reflects the appropriate con- lation as the weights and demonstrates how this may be tribution of the subpopulation-specific risk or rates to the stand- extended beyond risk adjustment for rates. ard. In this article, we demonstrate, using two examples, which A simulation was also run (forÀÁ example 1) under sampling risk adjustment through direct standardization can be achieved variability by allowing [OjPoisson Oj after replacing any Oj ¼0 using modified meta-analytic methods, but only with our model with 1. Thus, within each of the 18 age-group subpopulations, Oj [1] that allows input of user-defined weights. The advantage here was now generated from a Poisson distribution with mean Oj.A is that this method can now be extended to any standard and any thousand iterations of each set of rates were run using Ersatz effect size (ES) other than rates. version 1.3 (Epigear International Pty Ltd., Brisbane, Australia). Coverage of the confidence interval and percent bias was then computed as described by Burton et al. [4]. Methods A modification was undertaken of the quality-effects model [1] of meta-analysis that allows moving the model from meta-analysis to risk adjustment. This model uses a risk of bias weighting Results—Some Examples of Risk Adjustment scheme in addition to inverse variance weighting and the modification entailed removing inverse variance weights and replacing bias weights with normalized subpopulation weights Example 1: External Risk Adjustment Across Age Groups and from a standard population. The subpopulation weights are a Simulation Study applied using a modification of our bias adjustment procedure in meta-analysis [1,2] for each subpopulation to come up with a Individual death records with multiple cause of death were the weighted average that represents the single risk-adjusted or primary source of data and accessed through the Australian standardized estimate across the subpopulations. This weighted Bureau of Statistics for the period 1999 and 2006. Deaths were averaging procedure does not use inverse variance weights and coded according to the International Statistical Classification thus is not a meta-analysis. Therefore, if subpopulation rates are of Diseases, 10th Revision by using the automated Mortality Medical being combined, it would give an equivalent result to direct Data System and results have been reported previously [5]. standardization used in epidemiology. We do not use log- The Australian population age distribution in 2006 was used as transformed rates because back transformation would result in the external standard population for the purpose of risk adjust- pooled estimates that depart from those computed using the ment. To examine mortality trends and differentials across time, standard method. The standard method used for the computa- we had created three estimates of a risk-adjusted mortality rate tion of the directly standardized rate (DSR) is given by: from renal failure due to diabetes using standard methods k w o as follows [5]: 1) Risk-adjusted rates (underlying cause rate) ¼ 1 Â ∑ j i ð Þ DSR k n 1 for diabetic renal disease based on deaths coded to diabetic j ¼ 1 j ∑ Wj nephropathy; 2) Risk-adjusted rates (multiple cause rate 1) for i ¼ 1 diabetic renal disease based on 1) above and additional deaths where Oj is the observed number of events in subpopulation (age coded to diabetes without complications but with renal failure group) j, nj is the number