Research Report Measures and Methodology for International Comparisons of Health Care System Performance Final Report Laurie T. Martin, Carl Berdahl, Rachel M. Burns, Emily Hoch, Evan D. Peet, Peter S. Hussey C O R P O R A T I O N For more information on this publication, visit www.rand.org/t/RRA171-1 Published by the RAND Corporation, Santa Monica, Calif. © Copyright 2021 RAND Corporation R® is a registered trademark. Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. www.rand.org Preface Compared with other Organisation for Economic Co-operation and Development (OECD) member countries, the United States ranks lower on several notable health-related outcomes, such as infant mortality, that tend to be overemphasized by stakeholders and used as a marker of health care system performance. However, the driving factors contributing to many of these rankings could be attributed to factors outside of the health care system, such as greater inequality, less social protection, population density, and societal preferences. Concerns have also been raised around the variability in data availability and quality within the United States and abroad, which can introduce bias or make international comparisons more difficult. Given the increasingly important question of what the United States is buying for its health expenditures and interest in understanding what factors of the health system contribute to better health outcomes, in 2020, the Immediate Office of the Secretary of the U.S. Department of Health and Human Services (HHS) sought to identify measures that could improve the ability of the United States and other countries to learn from international comparisons of health system performance specifically. To inform the identification of measures for international comparison that could eventually be proposed to OECD, RAND Corporation researchers worked with a diverse group of 15 experts in quality measurement, clinical care, and health economics to generate and prioritize potential measure constructs that align with HHS priorities and are particularly promising for international comparisons. With few exceptions, the measure constructs identified as most promising will require additional development work to establish their operational definitions and specifications to ensure that any measure developed is valid and feasible for international comparisons of health system performance. This report is designed to lay a strong foundation for these future refinements by noting the degree of consensus among experts about the importance, scientific acceptability, feasibility, and usability of measure constructs; summarizing the strengths and limitations of the measure constructs; and providing additional context that can be useful for informing the selection of measure constructs that might ultimately be developed into measures and proposed to OECD for consideration. This research was funded by the Office of the Assistant Secretary for Planning and Evaluation, HHS (Contract No. HHSP233201500038I), and carried out within the Payment, Cost, and Coverage Program in RAND Health Care. RAND Health Care, a division of the RAND Corporation, promotes healthier societies by improving health care systems in the United States and other countries. We do this by providing health care decisionmakers, practitioners, and consumers with actionable, rigorous, objective evidence to support their most complex decisions. For more information, see www.rand.org/health-care, or contact iii RAND Health Care Communications 1776 Main Street P.O. Box 2138 Santa Monica, CA 90407-2138 (310) 393-0411, ext. 7775 [email protected] iv Abstract In 2020, the Immediate Office of the Secretary of the U.S. Department of Health and Human Services (HHS) sought to identify measures that could improve the ability of the United States and other countries to learn from international comparisons of health system performance. To inform the identification of measures for international comparison that could eventually be proposed to the Organisation for Economic Co-operation and Development (OECD), RAND Corporation researchers worked with a diverse group of 15 experts in quality measurement, clinical care, and health economics to generate and prioritize potential measure constructs that align with HHS priorities and are particularly promising for international comparisons. The authors implemented a modified Delphi panel approach using RAND’s ExpertLens platform. This process occurred in three phases. Phase 1 involved the identification of measure constructs that should be considered for inclusion in the expert rating process. Phase 2 was the initial assessment of each measure construct by experts against prespecified rating criteria. Phase 3 was a virtual convening of experts to discuss the ratings, explore areas of disagreement, and allow experts to adjust ratings, if desired. Eight measure constructs were identified as having the most promise for international comparison, but they will require additional development work to establish their operational definitions and specifications to ensure that any measure developed is valid and feasible for international comparisons of health system performance. This report is designed to lay a strong foundation for these future refinements by noting the degree of consensus among experts about the importance, scientific acceptability, perceived feasibility, and usability of measure constructs; summarizing the strengths and limitations of the measure constructs; and providing additional context that can be useful for informing the selection of measure constructs that might ultimately be developed into measures and proposed to OECD for consideration. v Contents Preface ........................................................................................................................................... iii Abstract ............................................................................................................................................ v Figures ............................................................................................................................................. x Tables ............................................................................................................................................. xi Summary ...................................................................................................................................... xiii Acknowledgments ........................................................................................................................ xix Abbreviations ................................................................................................................................ xx 1. Introduction ................................................................................................................................. 1 Methods ..................................................................................................................................................... 2 Phase 1: Identification of Measure Constructs That Should Be Considered ........................................ 2 Phase 2: Evaluation of Each Measure Construct Against Prespecified Criteria ................................... 3 Phase 3: Convene, Discuss, and Adjust Ratings ................................................................................... 4 Organization of This Report ...................................................................................................................... 6 2. Measure Constructs That Hold the Most Promise for International Comparisons ..................... 7 Treatment and Control of Hypertension ................................................................................................... 7 Description Provided in Evidence Table .............................................................................................. 7 Strengths and Limitations ..................................................................................................................... 8 What Is Needed to Move This Measure Forward ................................................................................. 9 Access to and Coverage for Telehealth ..................................................................................................... 9 Description Provided in Evidence Table .............................................................................................. 9 Strengths and Limitations ..................................................................................................................... 9