Improving surgical outcomes through adoption of evidence-based process measures: Intervention specific or associated with overall hospital quality? Benjamin S. Brooke, MD,a Robert A. Meguid, MD, MPH,a Martin A. Makary, MD, MPH,a,c Bruce A. Perler, MD, MBA,a Peter J. Pronovost, MD, PhD,b,c and Timothy M. Pawlik, MD, MPH,a Baltimore, MD Background. The Leapfrog Group aims to improve surgical outcomes through promoting hospital adoption of procedure-specific process measures, although it is unclear whether compliance reflects a hospital’s overall quality. The purpose of this study was to evaluate whether implementation of Leapfrog’s standard for routine b-blockade was associated with reductions in mortality after open abdominal aortic aneurysm (AAA) repair alone versus other high-risk operations. Methods. Using a 2:1 matched case-control study design, hospitals that had not adopted the b-blockade standard (n = 72) were compared with hospitals that had implemented this Leapfrog standard (n = 36). Leapfrog survey data were linked to patient outcomes in the California OSHPD database from 2000 to 2005. Random-effects Poisson regression models were used to evaluate in-hospital mortality over time for patients undergoing AAA repair versus esophagectomy, hepatectomy, pancreatectomy, colectomy, gas- trectomy, and pulmonary lobectomy. Results. A total of 6,199 AAA repairs, 2,780 esophagectomies, 2,544 hepatectomies, 2,909 pancrea- tectomies, 57,795 colectomies, 6,267 gastrectomies, and 10,210 lobectomies were analyzed. AAA- associated mortality significantly declined in hospitals that adopted the b-blocker standard (relative risk [RR]: 0.49; 95% confidence interval [CI]: 0.24--0.97; P < .05). Implementation of this Leapfrog standard had no effect on reducing adjusted mortality rates for other high-risk operations, including esophagectomy (RR: 0.70; 95% CI: 0.25--1.89), hepatectomy (RR: 1.16; 95% CI: 0.32--4.29), pan- createctomy (RR: 0.76; 95% CI: 0.28--2.02), colectomy (RR: 1.12; 95% CI: 0.86--1.44), gastrectomy (RR: 1.17; 95% CI: 0.57--2.43), and lobectomy (RR: 0.98; 95% CI: 0.46--2.08) (all P > .05). Conclusion. Compliance with peri-operative b-blockade resulted in a significant reduction in mortality after open AAA repair over time, but it had no crossover effect on mortality associated with other high-risk operations in the same hospital. These data suggest that improvements in outcomes resulting from the adoption of evidence-based process measures are procedure specific and do not necessarily reflect overall hospital quality. (Surgery 2010;147:481-90.) From the Departments of Surgerya and Anesthesiology/Critical Care Medicine,b Johns Hopkins University School of Medicine, Baltimore; and Department of Health Policy and Management,c Johns Hopkins University School of Public Health, Baltimore, MD AN ESTIMATED 50,000 IN-HOSPITAL DEATHS occur annu- peri-operative adverse events are known to be pre- ally in the United States among patients who have ventable with the reliable application of evidence- undergone operative procedures.1 Many of these based medicine (EBM) practices by hospitals and health care providers. This fact has been high- Supported by Grant 1KL2RR025006-01 from the National lighted by several reports released by the Institute Center for Research Resources (NCRR), a component of the of Medicine over the past decade, which have pro- National Institutes of Health (NIH), and the NIH Roadmap for Medical Research. jected that thousands of patient lives might be Accepted for publication October 7, 2009. saved if patients underwent operations at a hospi- tal that adheres to EBM.2,3 Reprint requests: Timothy M. Pawlik, MD, MPH, Department of Surgery, Johns Hopkins Hospital, 600 North Wolfe Street, Based on these findings, several nationwide Halsted 614, Baltimore, MD 22187-6681. E-mail: tpawlik1@ initiatives have been formed that aim to refer jhmi.edu. surgical patients selectively to hospitals that adhere 0039-6060/$ - see front matter to EBM practices. One of the largest such pro- Ó 2010 Mosby, Inc. All rights reserved. grams is the Leapfrog Group Hospital Quality and doi:10.1016/j.surg.2009.10.037 Patient Safety Initiative, which was started in 2000 SURGERY 481 482 Brooke et al Surgery April 2010 by a group of business and health care leaders.4 A number of hospitals surveyed. The Leapfrog data cornerstone of the Leapfrog program has been the are based on surveys sent to the Corporate Exec- promotion of evidence-based standards for high- utive Officer or head administrator of 337 acute- risk operations, including abdominal aortic care hospitals in urban regions of California, with aneurysm (AAA) repair, esophagectomy, pancre- the collected data representing self-reported infor- atic resection, coronary artery bypass grafting mation regarding hospital demographics and an- (CABG), aortic valve replacement, and bariatric nual compliance with each of the Leapfrog Group surgery. The Leapfrog standards include encourag- hospital quality and safety standards. A total of 212 ing patient referral to hospitals that meet defined targeted hospitals returned Leapfrog surveys and case volume thresholds, as well as giving incentives were available for review. for hospitals to promote implementation of evi- In all, 36 California hospitals were identified in dence-based process measures for specific proce- which a hospital policy for routine peri-operative b- dures.5 Although the benefits of meeting EBM blocker use during AAA repair was in place starting standards on surgical outcomes have been noted in 2003. To meet Leapfrog criteria for this process by several recent analyses, most hospitals do not measure, at least 80% of patients who underwent meet case volume standards and overall compli- elective AAA repair at a given hospital must have ance with the adoption of evidence-based process been on b-blocker therapy during their hospitali- measures remains low.5,6 zation as well as at the time of discharge. California It is well recognized that numerous barriers may hospitals that met this evidence-based standard limit or prevent health care organizations from were matched by total hospital admission volume adopting new evidence-based practices into hospi- (1:2) to 72 control hospitals that returned Leap- tal policy.7-9 Hospitals that overcome these obsta- frog surveys declaring noncompliance with routine cles may therefore have unique characteristics or b-blocker use. To maintain homogeneity, only institution-wide programs dedicated toward quality hospitals that used open AAA repair were identi- improvement. Indeed, the adoption of evidence- fied and used for the purpose of analyses. To based process measures may serve as a surrogate determine whether improvements in outcomes marker for a hospital’s commitment toward EBM among hospitals willing to adopt evidence-based and/or improving operative processes of care. process measures were procedure specific (eg, Prior studies have suggested that hospitals with b-blockade for AAA peri-operative outcomes) or low mortality rates for 1 operation tend to have reflective of hospital-wide quality improvements lower mortality rates for other operations based over time, the peri-operative mortality outcomes of on shared evidence-based processes of care.10 As 6 other high-risk operations were determined. such, we sought to determine whether improve- Specifically, the peri-operative outcomes of pa- ments in outcomes among hospitals willing to tients who underwent elective esophagectomy, adopt evidence-based process measures were spe- pancreatectomy, hepatectomy, colectomy, gastrec- cific for the procedures to which they apply or tomy, and pulmonary lobectomy in the same whether the adoption of such evidence-based mea- California hospitals were assessed. All data on sures are reflective of more global hospital-wide patients who underwent any of these elective improvements over time. To address this question, operative procedures at the identified hospitals we evaluated whether hospitals that adopted the were obtained from the California Office of Leapfrog evidence-based process measure for rou- Statewide Health Planning and Development tine b-blocker use during AAA repair also experi- (OSHPD) database for the years between 2000 enced improved outcomes over time for other and 2005. The data were then linked by OSHPD high-risk operations. Such information is impor- identification number to the Leapfrog Group tant for determining markers of hospital quality survey results. International Classification of and may help ensure that operative patients are Diseases, 9th Revision procedure codes were referred to centers with the best outcomes. used to identify open AAA repair (38.34, 38.36, 38.44, 38.64, 39.25, and 39.52), esopha- METHODS gectomy (42.40, 42.41, 42.42, and 43.99), pan- Hospital and patient data. Response data ob- createctomy (52.70, 52.51, 52.52, 52.53, 52.59, tained from the Leapfrog Group Hospital Quality and 55.26), hepatectomy (50.22 and 50.30), and Safety Surveys sent to California hospitals colectomy (45.7, 45.71, 45.72, 45.73, 45.74, annually between 2001 and 2005 were reviewed. 45.75, 45.76, 45.79, and 45.8), gastrectomy California was the first state-wide region to be (43.5, 43.6, 43.7, 43.8, 43.89, 43.9, and 43.99), targeted by Leapfrog, and it contains the largest and pulmonary lobectomy (32.3 and 32.4) Surgery Brooke et al 483 Volume 147, Number 4 procedures from the OSHPD database. The pro- cedure codes for endovascular AAA repair (39.71) and nonelective operations were excluded from analyses. The Johns Hopkins University School of Medicine Institutional Review Board approved this study. Study design. Hospital characteristics and in- hospital mortality for all operative procedures were compared over consecutive time periods as follows: (1) the 3-year period (2000--2002) prior to the release of the Leapfrog Group process measure standard for routine b-blocker use; and (2) the 3-year period (2003--2005) after California hospitals were either compliant (treatment group) or non- b compliant (control group) with the -blocker Fig 1.