Programs Supporting the Use of Comparative Effectiveness Research and Patient-Centered Outcomes Research by State Policymakers Iv

Home , Health services research

Pr o g r a m s Su pp o r t i n g Th e Us e Of Co m p a r a t i v e Ef f e c t i v e n e s s Re s e a r c h An d Pa t i e n t - Ce n t e r e d Ou t c o m e s Re s e a r c h By St a t e Po l i c y m a k e r s

Barbara Wirth and Felicia Heider

u g u s t A 2014

h i s p r o j e c t w a s s u p p o r t e d t h r o u g h a a t i e n t e n t e r e d u t c o m e s e s e a r c h n s t i t u t e T u g e n e aP s h i n g t o-C n n g a g e mO e n t w a r d R I (PCORI) E W E A ii

b o u t t h e a t i o na l c a d e m y f o r t a t e e a l t h o l i c y A N A S H P The National Academy for State Health Policy We work across a broad range of health policy (NASHP) is an independent academy of state topics including: health policymakers. We are dedicated to • Affordable Care Act and State Health helping states achieve excellence in health Care Reform policy and practice. A non-profit and non- • Coverage and Access partisan organization, NASHP provides a forum • Medicaid for constructive work across branches and • Quality, Cost, and Health System agencies of state government on critical health Performance issues. • Long Term and Chronic Care To accomplish our mission we: • Quality and Patient Safety • Population and Public Health • Convene state leaders to solve problems • Insurance Coverage and Cost and share solutions Containment • Conduct policy analyses and research • Disseminate information on state policies Our strengths and capabilities include: and programs • Active participation by a large number • Provide technical assistance to states of volunteer state officials • Developing consensus reports through The responsibility for health care and health active involvement in discussions among care policy does not reside in a single state people with disparate political views agency or department. At NASHP, we provide a • Planning and executing large and unique forum for productive interchange across small conferences and meetings with all lines of authority, including executive offices substantial user input in defining the and the legislative branch. agenda • Distilling the literature in language useable and useful for practitioners • Identifying and describing emerging and promising practices • Developing leadership capacity within states by enabling communication within and across states

For more information about NASHP and its work, visit www.nashp.org Portland, Maine Office: Washington, DC Office: 10 Free Street, 2nd Floor 1233 20th Street, NW, Suite 303 Portland, ME 04101 Washington, DC 20036 Phone: [207] 874-6524 Phone: [202] 903-0101 National Academy for State Health Policy Follow us @nashphealth on Twitter iii

Table of Contents

Acknowledgments iv Executive Summary 1 Introduction 2 Methodology Background on CER and PCOR 3 What are CER and PCOR? 3 How many state policymakers surveyed use CER and PCOR findings? 3 How can CER and PCOR be used in policymaking? 4 Who funds and conducts CER and PCOR? 5 Programs Based in State Agencies 6 Commissions Serving Multiple Agencies 6 Agency-Specific Advisory Groups/Committees 7 Programs Existing Outside of State Agencies 9 Multi-state Collaboratives 9 Federal Government Programs 10 Academic Institutions 10 Non-profit organizations 11 For-profit organizations 12 Summary 15 Appendix 17 Survey Results 17 Semi-Structured Interviews with Policymakers 18 Endnotes 19

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers iv

Ac knowledgments T he authors would like to thank the state health policymakers who participated in the survey and calls including Deidre Gifford, Michelle Harper, Leah Hole-Marshall, and Judy Zerzan. We would also like to thank the state health policymakers who reviewed this report including Chris Atchison, Laura Nasuti and Emily Parento. We would like to acknowledge staff from the National Academy for State Health Policy, including Mary Takach, Senior Program Director, for her input and review of this report. Finally, the authors are grateful for the support from Greg Martin and others at the Patient-Centered Outcomes Research Institute.

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 1

Ex e c u t i v e Su m m a r y I n the American Recovery and Reinvestment Act of 2009, Congress invested $1.1 billion to support 1 the development and dissemination of comparative effectiveness research (CER), which is designed to examine the benefits and harms of alternative interventions related to health care. In 2010, under the Patient Protection and Affordable Care Act, Congress built on this major investment, by authorizing the creation of the private, non-profit Patient-Centered Outcomes Research Institute (PCORI) to fund 2 comparative clinical effectiveness research guided by patients, caregivers and the broader healthcare community, also known as Patient-Centered Outcomes Research (PCOR). CER and PCOR provide state policymakers with evidence comparing the effectiveness and safety of different interventions being considered for use within their state. In addition to supplying background information on CER and PCOR, this report provides examples of various programs supporting the generation, synthesis, analysis, and implementation of this research in policy decision making. State agency programs supporting the use of CER and PCOR vary from commissions charged with reviewing evidence on multiple issues to single-agency advisory groups focused on a specific health-related topic. Bureaus or commissions may use their reviews of research to make evidence-based recommendations for multiple state agencies. The Oregon Health Evidence Review Commission (HERC), for 3 example, conducts CER on health technologies and maintains a list of the comparative benefits of various health services for use by state agencies. Existing standing agency-specific advisory groups, such as Medicaid Medical Advisory Committees, are specifically tasked with using research to advise the state on medical issues impacting state Medicaid policies. Numerous CER and PCOR programs are also based outside of state agencies and include multistate collaboratives, federal government initiatives, programs housed in academic institutions, and both private 4 not-for-profit and for-profit organizations. Multi-state collaborations such as the New England Comparative Effectiveness Public Advisory Council (CEPAC) serve a unique role both as a source of CER and as a support for state agencies to collaborate, better understand, and use the evidence to inform their work. Multiple CER and PCOR initiatives are supported by the federal government and often serve as a driving force for funding, conducting, and disseminating CER and PCOR, including the support of research programs housed in academic institutions. State health policymakers have access to a range of programs with the potential to support the use of CER and PCOR in decision making. The examples of programs described in this report provide support for the use of CER and PCOR either by funding and conducting the research, reviewing and synthesizing available research, and/or supporting the translation of research findings into information that policymakers can use when making program and policy decisions. Knowledge of these existing programs will aid state policymakers in leveraging CER and PCOR to make informed policy decisions.

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 2

In t r o d u c t i o n I n the American Recovery and Reinvestment Act of 2009, Congress invested $1.1 billion to support 5 the development and dissemination of comparative effectiveness research (CER) designed to examine the benefits and harms of alternative interventions related to health care. In 2010, under the Patient Protection and Affordable Care Act, Congress built on this major investment, by authorizing the creation of the Patient-Centered Outcomes Research Institute (PCORI) to fund comparative clinical 6 effectiveness research guided by patients, caregivers and the broader healthcare community, also known as Patient-Centered Outcomes Research (PCOR). CER and PCOR provide state policymakers with evidence comparing the effectiveness and safety of different interventions being considered for use within their state. The purpose of this review is to provide information on various state programs currently funding, conducting or supporting state policymakers in using CER and PCOR to inform policy decision making. Multiple entities exist, including programs housed within specific state agencies and others established outside of agencies, including, for example, programs operating as multi-state collaboratives, programs based in academic institutions or within non-profit or for-profit organizations. These programs may serve several different roles to support the use of CER and PCOR in policy decision making including funding or conducting the research, reviewing and synthesizing available research for use by others, or providing supports to translate the evidence into policy. The information presented in this review will provide state policymakers with examples of the resources and programs available to use CER and PCOR and their potential to support the development of similar programs in other states. e t h o d o l o g y M The findings referenced in this review are not intended to provide a comprehensive report on state CER and PCOR activity, but rather a landscape review of various programs currently in place at the state level providing support for the use of this research by state policymakers. The content in this report was informed largely by an online review of programs and from information obtained from state policymakers who participated in a National Academy for State Health Policy (NASHP) project. Funded by the Patient Centered Outcomes Research Institute (PCORI), this project included a survey distributed to 494 state policymakers examining their knowledge and use of research, including CER and PCOR. Responses were received from 130 state policymakers (26 percent) representing multiple agencies across 48 states and DC. In addition, NASHP convened group and individual calls to explore the use of CER and PCOR among state policymakers (see Appendix for summary of survey and group calls). As part of this project, NASHP also completed a guide to support state policymakers in using CER and PCOR entitled A Roadmap for 7 State Policymakers to Use Comparative Effectiveness and Patient-Centered Outcomes Research to Inform Decision Making. Consistent with restrictions in PCORI’s research funding announcements, the use of research by policymakers to examine cost-effectiveness specifically was not a focus of the study.

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 3

Ba c k g r o u n d o n CER a n d PCOR

E 8 vidence-based practice in health care requires the use of results from the best available research to aid in making the most informed decisions. Two forms of research, Comparative Effectiveness Research (CER) and Patient-Centered Outcomes Research (PCOR), provide evidence on the risks and benefits of different interventions and support evidence-based decision making by policymakers. h a t a r e a n d W CER PCOR? The Institute of Medicine identifies CER as research designed to compare different e y efinitions interventions for their effectiveness, examining Evidence-Based Practice their risks and benefits, supporting consistent K D is the application of the and rational decision making, and improving best available evidence when making health care 9 the delivery of care at both the individual decisions. and population level. CER may include a Comparative Effectiveness Research (CER) is single study using a methodological approach designed to compare different interventions for their designed to compare several interventions effectiveness, examining their risks and benefits, for their clinical, safety or cost differences, or supporting consistent and rational decision making, may be completed through a systematic review and improving the delivery of care at both the examining and comparing a number of individual individual and population level. research studies. Hypothetical examples of CER Patient-Centered Outcomes Research (PCOR) include the comparison of two programs for helps people and their caregivers communicate childhood obesity and their impact on weight and make informed healthcare decisions, allowing gain, or different programs supporting hospital their voices to be heard in assessing the value of transitions for high-needs, high-cost patients healthcare options. Specifically, PCOR highlights and their impact on hospital readmission rates. comparisons and outcomes of value to individuals The emphasis on comparison of interventions and includes considerations of their preferences and distinguishes this type of research from studies differences. utilizing control groups or placebos as the comparison population. Within this review, the discussion of CER will include information on what works best clinically or improves the delivery of care and will not explore the use of studies examining cost-effectiveness. PCOR, a term defined by the Patient-Centered Outcomes Research Institute (PCORI), is research 10 assessing the benefits and harms of interventions while also including an individual’s preferences and needs, and focuses on those outcomes of most value to the patient. Established in 2010, PCORI was 11 created as a non-profit entity under the Patient Protection and Affordable Care Act (ACA) to fund PCOR and to date has made more than 313 awards totaling nearly $550 million in research projects. Examples of funded projects include the use of telehealth self-management for older adults with health failure, 12 improvements in transitional care for individuals with serious mental illnesses, and the engagement of communities in preventing preterm birth. o w m a n y s t a t e policymakers s u r v e y e d u s e a n d f i n d i n g s H CER PCOR ? Respondents to NASHP’s national survey provided information on the current use of CER and PCOR by state policymakers. Nearly half (43 percent) of the respondents completing the national survey reported National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 4

their state uses CER or PCOR to inform decision making by health policymakers; 42 percent were unsure. The use of CER and PCOR also varied by agency; for example, Medicaid respondents were more likely to respond that their state uses CER or PCOR (69 percent) compared to those responding on behalf of a worker’s compensation program (20 percent). Although the majority (64 percent) of respondents reported using research evidence “almost always” or “often” to inform their work, this number dropped to one-third both when asked if they used CER and when asked if they used PCOR. Although reported use of PCOR was low, respondents agreed that research including a focus on outcomes identified by patients was considered important in their work on state health programs and policy (82 percent). (See Appendix for additional information on survey and interviews). o w c a n a n d b e u s e d in policymaking H CER PCOR ? CER and PCOR provide policymakers with a better understanding of the relative risks and benefits of various options and support the crafting of evidence-based programs and policies. Current use of PCOR, as strictly defined by PCORI, is limited given the relatively recent funding for this form of research. Available PCOR or, more broadly, research including the patient experience and priorities, may be used to inform policy and programs that aim to achieve desired health outcomes. For example, examining the patient’s priorities when implementing a wellness or substance abuse treatment program may be a critical component for the success of an intervention. PCOR can also be used for improving adherence with selected treatment options by helping identify the option with the fewest negative side effects, as identified by patients, relative to its effectiveness. State policymakers have access to multiple programs to gather CER and PCOR findings and influence policy and program decision making. For example, some states have allocated significant resources to create specific programs to conduct and/or promote the use of CER within state agencies. The Washington Health Technology Assessment program, for example, evaluates medical technologies for their effectiveness and safety to make coverage decisions for multiple Washington State agencies (see page 7). Though an established program is often ideal, state resources or priorities may limit the feasibility of establishing new entities. Policymakers may promote the use of CER and PCOR within existing groups such as advisory committees responsible for a specific issue or may seek to collaborate or learn from other organizations that aim to put research into practice, such as a university-affiliated research organizations or multi-state collaboratives (see page 9). h o f u n d s a n d c o n d u c t s a n d W CER PCOR? Funding for CER comes from federal, state, and private entities either performing independent research within their organizations or providing funding to academic and other organizations to conduct research. 13 As noted earlier, funding for PCOR, is primarily through federal funds provided to and distributed by PCORI, a private, non-profit organization. Specifically, the ACA created a PCOR Trust Fund that, in 14 addition to sustaining PCORI, also supports PCOR initiatives in the U.S. Department of Health and Human Services. Public entities, including federal government and public universities, are often responsible for conducting CER or PCOR and disseminating research findings to state agencies and the general public. Several federal agencies supporting this research include the Agency for Healthcare Research and Quality (AHRQ), the National Institutes of Health (NIH), the Food and Drug Administration (FDA), and the Centers for Disease Control (CDC). AHRQ, a driver of CER, funds Evidence-based Practice Centers (EPCs) housed in universities and other research institutions; EPCs have generated more than 200 public CER reports National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 5

15 to date. With access to the PCOR Trust Fund, AHRQ is charged with the dissemination of findings from PCORI-funded research, as well as building CER capacity through training of researchers. CDC’s Prevention Research Centers received funding through the American Recovery and Reinvestment Act to 16 conduct CER on the benefits and harms associated with various public health interventions in community settings. Various state agencies also finance and conduct CER for selected high priority issues for their state or agency. 17 Examples of private entities conducting CER and PCOR include for profit and not-for-profit organizations 18 including pharmaceutical companies and organizations such as RTI International and the Cochrane Collaboration.

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 6

Pr o g r a m s Ba s e d i n St a t e Ag e n c i e s S tates use various programs based within state agencies to support the use of CER and PCOR within their decision making process. Bureaus or commissions may be charged with reviewing available evidence on a range of health-related issues and making evidence-based recommendations for use by multiple state agencies. States may also use existing standing agency-specific committees or advisory bodies to review CER and incorporate research findings into programs and policies. Several examples of programs within state agencies are presented below. o m m i s s i o n s e r v i n g u l t i p l e g e n c i e s C S M A

A number of state governments have established entities to review evidence and report their findings to guide decision making across multiple state agencies. Tasked with serving as a state resource, these groups may be administered through a specific state agency or directed by state legislature. Membership within the groups varies. For example, membership may include solely state appointees selected by the governor’s office, on the other hand they may be comprised primarily of volunteers from the health care community. The information generated within these groups is made available to an array of decision makers including clinicians, consumers, and purchasers of health care including t a t e o c u s Program Conducting CER for Policymakers: Medicaid and State Employee Benefits. The Oregon HealthS Evidence F Review Commission textbox to the right highlights an example of a 19 commission specifically tasked with conducting The Oregon Health Evidence Review Commission CER for use by health policy makers. Several (HERC) is administered through the Office other examples of these programs are described for Oregon Health Policy and Research and below. overseen by the Oregon Health Authority. HERC The • Washington State Health is responsible for conducting CER on health Technology Assessment Program technologies or treatments and identifying is overseen by the WA Health Care evidence-based guidelines to be used by clinicians, Authority and serves as a resource for consumers and purchasers of health care in state agencies purchasing health care Oregon. The Commission is made up of volunteer including Medicaid, Veterans Affairs, members of the health-care community, including 20 Department of Corrections, and Labor physicians, nurses, pharmacists and consumer and Industries. The Washington representatives, and emphasizes a transparent Health Technology Clinical Committee process with input from the public and those (HTCC), composed of practicing impacted by the decisions. HERC maintains a clinicians and health professionals, list of health services showing the comparative annually selects approximately 10 clinical and cost benefits of each service to inform technologies to be evaluated by a decisions made by different agencies. Using HERC contracted research organization. The findings, the Oregon State Employee Benefits technology assessment reports are Board chose to build additional charges into their then reviewed by the HTCC, which benefit design for those services lacking strong makes coverage decisions based on HERC evidence to support their effectiveness. safety, effectiveness and efficiency. The National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 7

HTCC’s process includes public meetings and their final coverage decisions are published online. Examples of reviews included cochlear implants, cervical spinal fusion and hip resurfacing therapy. 21 To encourage the development of consistent coverage decisions across state agencies, state agencies are required to implement the coverage decisions from the HTA program. • The Minnesota Health Services Advisory Council, under the direction of the Minnesota Legislature, reviews research on the effectiveness of different services and advises the 22 Commissioner of Human Services on benefit and coverage policies for the state’s public health care programs. A recent report evaluating evidence-based treatment options for autism 23 spectrum disorders (ASD) used CER studies from AHRQ and other sources to help inform the passage of legislation for a treatment benefit for children with ASD. Members of the council are appointed by the Commissioner of Human Services and include 11 clinicians and a consumer representative. The Health Services Advisory Council holds public meetings for public commentary during their process of evaluating coverage decisions. 24 • The Bree Collaborative is a public/private consortium established by the 2011 Washington Legislature to examine health care issues with high utilization or variation in treatment patterns. The 24 members of the collaborative are appointed by the Governor and include purchasers, employers, plans, clinicians and other organizations. Each year, the members convene an expert workgroup to review three issues and provide information to the state to guide the type of health

care provided to Medicaid enrollees, state employees and others. A Bree review of obstetrics25 care was used to inform the state on the adoption of the recommended strategies by state-purchased health plans and contributed to changes in the state’s payment policies for deliveries. • The Washington State Institute for Public Policy (WSIPP) was created by the Washington State Legislature as a non- t a t e o c u s partisan research center. Governed in Program Reviewing Evidence: part by the state legislature, the Institute WashingtonS Pharmacy F and Therapeutics supports policymakers, particularly Committee legislators, by answering relevant policy questions and creating reports The Washington State Health Care Authority examining research on the effectiveness maintains a State Pharmacy and Therapeutic of different interventions. A recent report Committee to advise the state’s Preferred Drug included an inventory of evidence-based List (PDL) and meet a federal mandate to use 26 and promising practices for services evidence when making coverage decisions about addressing adult behavioral health. prescription drugs. The committee reviews the list of drugs studied by the Drug Evaluation Research g e n c y pecific d v i s o r y r o u p s Program (DERP), comparing the drugs within a Ao m m i tt-Se e s A G / C therapeutic class for comparative safety, efficacy States convene various committees and advisory and effectiveness. Recommendations are then boards to review and synthesize available used by various state programs, including the evidence related to the needs of a specific Public Employees Benefits Board, Medicaid and population or issue. These standing committees the Worker’s Compensation Administration. provide opportunities to promote evidence- based practices for specific health policy decisions and, as existing bodies, may require minimal additional state resources to incorporate CER and National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 8

PCOR into the decision-making process. Committees may serve either solely an advisory role or may have final decision-making capacity. Members may include predominately clinicians or a range of stakeholders, including legal advisors, epidemiologists, consumers and experts, all with variations in their level of understanding of evidence-based practice, CER and PCOR. Public input may be restricted with closed meetings for members only or may be obtained through open meetings and public comment periods. 27 Several examples of such committees are presented below and included in the accompanying text box on Washington’s pharmacy and therapeutic committee. • California Worker’s Compensation Medical Evidence Evaluation Committee was established following legislative ruling in 2007 enabling the Division of Worker’s Compensation to develop its own guidelines. The closed, multi-disciplinary committee is composed of members of the medical community and is responsible for ranking the evidence and advising the Medical Director on 28 updating and adopting medical treatment guidelines. Contractors perform the studies to inform decisions on guidelines. Health care providers are then required to use the adopted guidelines, and, if another approach is requested, must present a higher level of evidence for approval.

• Rhode 29 Island’s Medicaid Medical Advisory Committee is a federally mandated committee designed to review medical issues impacting state Medicaid programs and provide recommendations to the state. In Rhode Island, the Medicaid advisory committee reviews research reports from the New

England Comparative Effectiveness Public Advisory30 Council (CEPAC) and has used these reports to inform their Medicaid policy on issues including use of community health workers and attention deficit hyperactivity disorder (ADHD) treatment.

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 9

Pr o g r a m s Ex i s t i ng Ou t s i d e o f St a t e Ag e n c i e s T he programs described in this section are entities operating largely outside state government oversight and assist state policymakers in accessing and translating CER and PCOR for use in programs and policies. Programs may have specific roles, such as funding novel CER and PCOR (see Table 1), or may be tasked with several roles including evaluating and synthesizing available research and translating the findings into accessible and understandable formats for different audiences. Different organizations housing programs are summarized below and include several examples of existing programs used by state policymakers. u l t i s t a t e o l l a b o r a t i v e s M - C Multi-state collaborations serve a unique role both as a source of CER through systematic reviews of available evidence and as a support for state agencies to understand and use the evidence to inform their work. Though several require membership fees, these multi-state collaborations also have the financial advantage of combining resources to generate needed information and the ability to bring state policymakers together to discuss and compare programs and policy decisions. The comparative effectiveness reports that are generated may be made available for public review or restricted to those states with membership in the collaborative. The Oregon Center for Evidence-based Policy oversees two multi-state collaboratives - the Drug 31 Evaluation Research Program (DERP) and Medicaid Evidence-based Decisions Project (MED) to help policymakers from member states make informed decisions. DERP, in collaboration with the Pacific 32 Northwest Evidence-based Practice Center, produces proprietary systematic reviews examining the effectiveness and safety of different drug classes. Currently, nine state Medicaid programs and other partners pay a fee for membership, and as members, determine the drug classes to review, the key questions that will guide the review, and have access to the final reports. For example, Colorado, a member 33 of DERP, recently included DERP findings in their Pharmacy and Therapeutic Committee evaluation when making recommendations on the effectiveness and safety of different insulin groups. Members also may decide to make a report available outside the group when findings on a particular issue are determined 34 to be of significant importance. Findings from a joint DERP-MED review of a new Hepatitis C drug, for example, will be made available to the National Association of Medicaid Directors. The Medicaid Evidence-based Decisions Project (MED), a self-governing collaboration of 13 state Medicaid agencies, provides participating Medicaid agencies with access to independent and objective 35 evaluations of clinical evidence, group collaborative problem-solving opportunities, and tools and resources to translate the evidence into policy. Texas Medicaid has utilized MED resources, for example, 36 in discussions of health homes, payment reform, telemedicine and chronic conditions and has participated in workgroups with other states on behavioral and oral health. 37 The New England Comparative Effectiveness Public Advisory Council (CEPAC) is an independent group 38 of physicians, experts, and patient/public members with participants from six states. Managed by the Institute for Clinical and Economic Review (ICER) , CEPAC is tasked with reviewing comparative effectiveness findings from ICER with a particular focus on research relevant to these states. Reports have included reviews on treatments for ADHD, atrial fibrillation, sleep apnea and depression. A recent review 39 of community health workers included an evaluation of the evidence on their effectiveness and an action guide for use by the Community Health Worker workforce. National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 10

e d e r a l o v e r n m e n t r o g r a m s F G P Multiple federal programs support the use of comparative effectiveness or patient-centered outcomes research by supporting a variety of different resources and programs. Federal agencies are a significant source of funding for research, reviews and tools to aid in decision making. Examples of agencies include the Agency for Healthcare Research and Quality (AHRQ), National Library of Medicine (NLM), Food and Drug Administration (FDA), and Centers for Disease Control and Prevention (CDC) and National Institutes of Health (NIH). Examples of activities within these organizations supporting the use of CER and PCOR are provided below. • AHRQ, for example, through its Effective Health Care Program, funds individual researchers, research centers, and academic organizations to work together with the AHRQ to produce effectiveness and comparative effectiveness research for clinicians, consumers, and policymakers. 40 To date, AHRQ’s Effective Health Care Program has produced more than 400 comparative effectiveness research projects. With access to the PCOR Trust Fund, AHRQ is charged to develop supporting the dissemination of findings from PCORI-funded research. • The U.S. National Library of Medicine (NLM) serves as an extensive resource for research, 41 including CER, and can provide state policymakers access to databases, grant initiatives, publications, key organizations, and relevant legislation. NLM, in collaboration with the American College of Physicians, also oversees the project “Information RX”, a web-based information site 42 that provides patients with clear and accurate medical information on different treatments and interventions. • The FDA, working with the National Cancer Institute Center for Biomedical Informatics and 43 Information Technology, has formed the Janus Clinical Trials Repository. The Repository serves as a resource to “support regulatory review and patient centered outcomes research (PCOR)”. 44 • The CDC in collaboration with the NIH has created Prevention Impacts Simulation Models (PRISM) for use by health policy planners. These models use multiple sources of evidence and data and allow the user to simulate trajectories for selected interventions and aid in their 45 understanding of the comparative effects of the interventions on various chronic diseases. The models can be adapted to local, state or larger populations. c a d e m i c n s t i t u t i o n s A I Academic institutions house various programs supporting the use of CER and PCOR by state policymakers. The Georgia Health Policy Center provides an example of a public health institute housed within the academic center, Georgia State University. The center is closely involved with translating research evidence for use by state policy makers. (see text box, page 11). Several examples of other academic centers actively engaged in CER and supporting the use of evidence areprovided below. • The RTI –University of North Carolina Evidence-based Practice Center is one of AHRQ’s Evidence-based Practice Centers described above and serves as a resource to the state’s health care community. In collaboration with RTI International, the University produces reviews and 46 creates materials geared towards aiding patients, clinicians and others in using evidence in their health care decisions. CER reports for North Carolina have included an examination of the 47 impact of Medicaid funding cuts for a maternity care coordination project run through a local public health department.

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 11

• The University of Washington School of Medicine houses the Evidence-based Practice Institute (EBPI) established by t a t e o c u s the state legislature in 2007. Working Program Translating Evidence into Policy: in collaboration with the Division of GeorgiaS Health F Policy Center Behavioral Health, EBPI identifies, The Georgia Health Policy Center has developed evaluates and promotes the use of best tools and educational programs to distill research practices and provides training to support 48 and provide the findings in an understandable the implementation of evidence-based fashion to legislators and agency leads. Using a practices within the community. Legislative Health Policy Certificate Program, for • The University of Alabama at Birmingham example, the Center provides state lawmakers conducts specific health services research with training on how to review policy issues with at the request of the Alabama Department a “systems thinking-focused” approach. Per diem of Public Health. Examples of special funding for legislators attending the training is studies include whether successive years provided by the state. A model using research of insurance coverage decrease asthma- and epidemiological data on childhood obesity related emergency use or hospitalizations helped lawmakers understand the impact of their and how access to more preventive dental 49 different policy decisions. Able to compare the visits may impact subsequent dental visits consequences of different actions in a number and costs. of policy areas, such as school food options and o n p r o f i t o r g a n i z a t i o n s improving school physical education, legislators N - noted the model informed their deliberations. In addition to academic institutions, multiple non- and contributed to the passage of a bill requiring profit entities have established programs that may fitness testing and the enforcement of physical serve as valuable resources to support the use of 50 education requirements in the school system CER and PCOR. These organizations may emphasize a specific role, such as funding research (see text 51 52 box on Patient-Centered Outcomes Research Institute ), or conducting systematic reviews (e.g., the Cochrane Collaboration and RTI International ). Other organizations, such as public health institutes, state public/private forums, advocacy groups and professional organizations, may also emphasize engaging the public or providing specific supports to translate the findings into best practices and policy decisions. Materials produced by these organizations are largely available for public review. Public health institutes (PHIs) are independent, non-profit organizations linked with state public health agencies and use research to improve public health across different sectors of the health care delivery 53 54 system. The National Network of Public Health Institutes maintains a website with information on PHIs 55 56 located in multiple states, including the Colorado Health Institute , Kansas Health Institute , the Missouri Foundation for Health and the Georgia Health Policy Center (see earlier reference under Academic Institutions). The California Technology Assessment Forum, a non-profit, open public forum, convenes broad stakeholder groups to provide objective and transparent information on the safety and effectiveness of different treatments, medical technologies, and models of care. Members come from a range of disciplines, including medical ethics, consumer advocacy, practicing clinicians and researchers. Supported by the Institute for Clinical and Economic Review (ICER), the Forum examines both clinical and economic

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 12

data as well as reviews topics related to public 57 health, care management and delivery system innovation. A review of treatment options for e y a c t chronic Hepatitis C resulted in a final report Program FundingK F CER and PCOR: and several action guides for use by clinicians, Patient-Centered Outcomes Research Institute 58 patients and policymakers when making decisions Established in 2010 under the Affordable Care on the different available medications. Act, the Patient-Centered Outcomes Research Professional organizations have resources to Institute (PCORI) is a non-profit, private disseminate findings and translate available organization using federal funds to fund more research into formats to aid in evidence-based than 310 patient-centered comparative clinical decision making. The National Association effectiveness research projects. Funding priorities of Medicaid Directors, for example, informs include research on improving healthcare systems, members of promising practices and current addressing disparities, and the assessment of research findings on topics of interest. Working prevention, diagnosis and treatment options. with 70 national organizations, the American Since PCORI is still a relatively new entity, all the Board of Internal Medicine formed the research findings are not yet available; however, 59 educational campaign “Choosing Wisely” (see information on the studies funded by PCORI can text box below). The American College of be found on their website. Physicians also collaborated with the National 60 Library of Medicine to create the web resource “Information RX,” providing reliable information to patients on their diseases and treatment options. Clinical practice guidelines created by organizations such as American Academy of Pediatrics, the 61,62 American Academy of Family Physicians and American College of Physicians also provide professional recommendations based on reviews of research evidence for multiple diseases and conditions. National advocacy groups for specific populations provide state or regional chapters with region-specific information needed to address the populations, as well as research on issues of interest. The consumer advocacy organization, Center for Science in the Public Interest was cited by a e y a c t public health interviewee as a resource used to Program Translating CER and PCOR: 63 K F inform their state policies on health, nutrition, Choosing Wisely and food safety. Disease or condition-specific The American Board of Internal Medicine advocacy groups may also be purveyors of Foundation, in collaboration with other information about specific treatments and organizations, has created the Choosing Wisely provide a valuable source for the patient initiative to aid in health care decision making. perspective within the decision-making process. Lists entitled Things Physicians and Providers Should Use of the evidence obtained through some Question are provided with specific evidence- advocacy groups may require a close evaluation based recommendations for providers and of the source and the material provided to ensure patients to learn about health care procedures the reports provide an unbiased view and have that may be ineffective or unnecessary and could thoroughly reviewed all available options. actually cause harm. The recommendations o r p r o f i t o r g a n i z a t i o n s provided are to be used to generate conversations F - on what treatments are actually necessary and Private entities are available both as sources appropriate. of evidence-based reviews and as supportNational for Academy for State Health Policy Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 13

incorporating evidence-based practices into policy decisions. In general, information or services obtained from these organizations involve a fee and the materials may not be available to the general public. 64 For example, Hayes, Inc., an international consulting and research firm, evaluates medical technologies and works to integrate the evidence from their work into policy development and decision making. Clients with subscriptions for their services include hospitals, health plans, government programs, and 65 employers. Truven Health Analytics both conducts CER and supports the development of data analytic tools and clinical guidelines using CER. Truven tools were used by the Alabama Bureau of Children’s 66 Health Insurance when considering coverage for HPV vaccine resulting in a recommendation to the State Health Officer to cover the vaccine. Other private entities such as The Work Loss Data Institute produce 67,68 materials including the Official Disability Guidelines (ODG) available for purchase by state Workers’ Compensation agencies to inform their policy and coverage decisions.

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 14

Table 1: Examples of Programs Using Research to Inform State Decision Making

CER/PCOR Example Type of Program Programs Program Description Primary Role(s)* Washington The Washington HTA Program reviews State Health • Reviews and Commissions evidence for the safety, effectiveness, and Technology synthesizes evidence serving multiple efficiency of medical technologies. The HTA Assessment agencies then makes coverage decisions implemented • Translates evidence (HTA) Based across state agencies. into policy Program in State Agencies Rhode Island Rhode Island’s MCAC is a federally Medicaid mandated committee advising Rhode Island • Reviews and Agency-specific Medical Medicaid on the medical services they cover. synthesizes evidence advisory groups/ Advisory Rhode Island’s MCAC regularly reviews committees • Translates evidence Committee reports from CEPAC to inform and support into policy (MCAC) their work.

Drug DERP, a program overseen by the Oregon • Reviews and Evaluation Center for Evidence-based Policy, conducts Multi-state synthesizes evidence Research systematic reviews of drug classes to help collaboratives Program policymakers from member states make • Translates evidence (DERP) informed drug coverage decisions. into policy The Effective Health Care Program is an • Funds and conducts initiative of the Agency for Healthcare original research Research and Quality (AHRQ) that funds, Effective Federal government conducts, and disseminates research • Reviews and Health Care programs including CER. Within the Effective Health synthesizes evidence Program Care Program, 11 Evidence-based Practice • Creates research Centers generate evidence reports to inform summaries various practices and policies. The Georgia Health Policy Center is a • Conducts original Existing program run out of Georgia State University research Outside that assists local, state, and national entities Georgia of State Academic implement evidence-based practices. • Reviews and Health Policy institutions In addition to conducting research, the synthesizes evidence Agencies Center program also translates evidence through • Translates evidence providing tools and educational programs to into policy support state policymakers. Patient- PCORI is a private not-for-profit Centered organization created under the Affordable Non-profit Outcomes • Funds and conducts Care Act and provides funding for organizations Research original research comparative effectiveness and patient Institute centered research. (PCORI)

The Work Loss Data Institute produces • Reviews and The Work For-profit the Official Disability Guidelines that can synthesizes evidence Loss Data organizations be used to inform various states’ Workers’ Institute • Translates evidence Compensation programs. into policy

*Though the organizations described in this table often serve multiple purposes, this table draws attention to their National Academy for State Health Policy primary roles as they pertain to the use of evidence. Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 15

Su m m a r y V arious resources and programs are available to help state health policymakers use comparative effectiveness research (CER) and patient-centered outcomes research (PCOR) to better inform their decision making. CER provides policymakers with research comparing the safety and effectiveness of different interventions and PCOR utilizes a comparative effectiveness approach focusing on outcomes and priorities that patients have identified as most significant. States may use research within programs overseen by a state agency including, for example, standing commissions and agency- specific advisory boards. State policymakers also may access programs independent of state government including multi-state collaboratives, academic institutions and non-profit or for profit organizations. These public and private programs serve various roles, including funding or conducting CER and PCOR, synthesizing available research into evidence-based reports, and/or supporting the translation of evidence into policy. As policymakers increasingly have access to CER and PCOR, knowledge of these existing programs will aid them in leveraging this research to make informed policy decisions.

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers App e n d i c e s 17

App e n d i x

Executive Summary of Survey and Semi-Structured Interviews

As part of the project that informed this document, NASHP conducted an online survey to better understand how state officials view and use research (CER and PCOR in particular) to inform their work. The survey was sent via email to 494 health policymakers representing all 50 states, the District of Columbia, and select US territories. Recipients represented a wide breadth of state offices and agencies, including governors’ health policy advisors, legislators, public health officials, Medicaid/CHIP directors, state employee/retiree health benefits administrators, workers’ compensation directors, and state insurance commissioners. Following the survey, NASHP conducted a series of semi-structured interviews with individuals and groups of state policymakers. u r v e y e s u l t s S R In total, 130 state officials representing 48 states and the District of Columbia completed at least the first set of questions (26 percent response rate), and 101 of those 130 (78 percent) completed the entire survey. The majority of respondents (55 percent) represented Medicaid, CHIP, or Public Health. Although the number of respondents was too low to state statistically significant findings, the results of the survey identified interesting trends. Overall, state officials responding to the survey tended to report at least moderate familiarity with the concept of research evidence (92 percent), although they were relatively less familiar with the specific concepts of CER (73 percent) and PCOR (69 percent). Respondents were generally positive about the use of research to informing policymaking: • 93 percent agreed with a statement that state health policymakers should use research to inform their work; • 89 percent agreed with a statement that research should be used to determine health benefits coverage; and • 89 percent also agreed with a statement that research should be used to address the needs of patients with complex health issues.

The majority of respondents also agreed that research including a focus on outcomes identified by patients was considered important in their work on state health programs and policy (82 percent), making health benefits coverage decisions (71 percent), and addressing needs of patients with complex health issues (71percent).

Most respondents also agreed with statements that they would like to use research more often in their work (87 percent), to determine benefits coverage (74 percent), or address the needs of patients with complex health issues (76 percent). However, the most commonly reported barriers to using research in their work included difficulty in finding CER (54 percent), difficulty translating CER to inform programs and policies (49 percent), and significant concerns that CER would be used to restrict patients’ freedom of choice (31 percent).

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 18

Group #Sent #Respondents* Response Rate Governor’s Offices 42 8 19% Legislators 109 9 8% Public Health 57 27 47% Medicaid/CHIP 136 50 37% Insurance 46 9 20% Commissioners’ Offices State Employee/Retiree 51 20 39% Health Benefits

Workers’ Compensation 53 18 34% Offices *Some respondents self-identified as representing more than one office or agency, thus the breakdown of respondents by group adds up to greater than the total number of respondents (130). Furthermore, some respondents self-identified with agencies other than the category we had expected when we sent the survey. e m i t r u c t u r e d n t e r v i e w s w i t h olicymakers S -S I P Following the survey, NASHP conducted a series of semi-structured interviews with individuals and groups of state policymakers. Participants were selected from those who had indicated an interest in follow-up calls when completing the survey and others who were identified through NASHP’s database. Over the course of seven calls, 24 state policymakers were interviewed, and several provided additional information through follow-up emails and individual calls. The common themes identified through these calls, with guidance from our advisory group, were used to develop the action steps and considerations included in Roadmap that follows. Key differences across agencies were also identified such as the unique considerations that arise from being both a policymaker and a health care purchaser (e.g., Medicaid).

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 19

En d n o t e s

1 Institute of Medicine, Initial National Priorities for Comparative Effectiveness Research (Washington, DC: National Academies Press, 2009), 1. http://www.iom.edu/~/media/Files/Report%20Files/2009/ ComparativeEffectivenessResearchPriorities/CER%20report%20brief%2008-13-09.pdf. 2 Patient Protection and Affordable Care Act, Pub. L. No. 111-48 §6301(a). 3 Oregon Health Authority. “Health Evidence Review Commission.” Retrieved July 7, 2014. http://www. oregon.gov/oha/herc/Pages/index.aspx. 4 The New England Comparative Effectiveness Public Advisory Council. “About CEPAC.” Retrieved July 7, 2014. http://cepac.icer-review.org/http://cepac.icer-review.org/?page_id=18. 5 Institute of Medicine, Initial National Priorities for Comparative Effectiveness Research (Washington, DC: National Academies Press, 2009), 1. http://www.iom.edu/~/media/Files/Report%20Files/2009/ ComparativeEffectivenessResearchPriorities/CER%20report%20brief%2008-13-09.pdf. 6 Patient Protection and Affordable Care Act, Pub. L. No. 111-48 §6301(a) 7 Forthcoming Barbara Wirth, Charles Townley, and Mary Takach, A Roadmap for State Policymakers to Use Comparative Effectiveness Research and Patient-Centered Outcomes Research to Inform Decision Making (Portland, ME: National Academy for State Health Policy, 2014). 8 Agency for Healthcare Research and Quality. “Effective Healthcare Program: Glossary of Terms.” Retrieved July 7, 2014. http://effectivehealthcare.ahrq.gov/index.cfm/glossary-of-terms/?pageaction=showterm&termid=24. 9 Institute of Medicine, Initial National Priorities for Comparative Effectiveness Research. http://www.iom.edu/~/ media/Files/Report%20Files/2009/ComparativeEffectivenessResearchPriorities/CER%20report%20brief%2008- 13-09.pdf. 10 Patient-Centered Outcomes Research Institute. “Research We Support.” Retrieved July 7, 2014. http://www. pcori.org/research-we-support/pcor/. 11 Patient-Centered Outcomes Research Institute. “PCORI Funding Awards.” Retrieved July 7, 2014. http:// pfaawards.pcori.org/. 12 Ibid. 13 National Pharmaceutical Council and Social & Scientific Systems, Inc.,Comparative Effectiveness Research and the Environment for Health Care Decision-Making (Washington, DC: National Pharmaceutical Council, 2013). http://www.npcnow.org/publication/state-comparative-effectiveness-research-and-environment-health-care- decision-making-0. 14 Patient-Centered Outcomes Research Institute. “How We’re Funded.” Retrieved July 7, 2014. http://www. pcori.org/about-us/how-were-funded/. 15 Agency for Healthcare Research and Quality. “Evidence-based Practice Centers (EPC) Program Overview.” Retrieved July 7, 2014. http://www.ahrq.gov/research/findings/evidence-based-reports/overview/index.html . 16 Centers for Disease Control and Prevention. “Prevention Research Centers Comparative Effectiveness Research Program.” Retrieved July 7, 2014. http://www.cdc.gov/prc/newsroom/comparative-effectiveness-research- program.htm. 17 RTI International. “Comparative Effectiveness Research.” Retrieved July 7, 2014. http://www.rti.org/page. cfm/Comparative_Effectiveness_Research. 18 The Cochrane Collaboration. Retrieved July 7, 2014. http://www.cochrane.org/.

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 20

19 Oregon Health Authority. “Health Evidence Review Commission.” Retrieved July 7, 2014. http://www. oregon.gov/oha/herc/Pages/index.aspx. 20 Washington State Health Care Authority. “Health Technology Assessment: About the Program.” Retrieved July 7, 2014. http://www.hca.wa.gov/hta/Pages/about.aspx. 21 Washington State Health Care Authority. “HTA Process.” Retrieved July 7, 2014. http://www.hca.wa.gov/ hta/Pages/tech_process.aspx. 22 2013 Minnesota Statutes, sec. 256B.0625; Subd. 3c. https://www.revisor.mn.gov/statutes/?id=256B.0625 &year=2005&keyword_type=exact&keyword=health+services+policy+committee. 23 Minnesota Department of Human Services. Autism Spectrum Disorders: Report to the Minnesota Commissioner of Human Services by the Health Services Advisory Council: DHS-6181-ENG 2-13. (St. Paul, MN: Minnesota Department of Human Services, 2013). https://edocs.dhs.state.mn.us/lfserver/Public/DHS-6181-ENG. 24 Dr. Robert Bree Collaborative. “Our Background.” Retrieved July 7, 2014. http://www.breecollaborative.org/ about/background/. 25 Leah Hole-Marshall. Phone interview with authors. April 9, 2014. 26 Washington State Institute for Public Policy. “Reports.” Retrieved July 7, 2014. http://www.wsipp.wa.gov/ Reports. 27 Washington State Health Care Authority. “Pharmacy and Therapeutics Committee and Medicaid Drug Utilization Review Board.” Retrieved July 7, 2014. http://www.hca.wa.gov/pdp/Pages/pt.aspx. 28 State of California Department of Industrial Relations. “Commission on Health and Safety and Workers’ Compensation.” Retrieved July 7, 2014. www.dir.ca.gov/chswc/StudiesProjects.htm. 29 42 CFR §431.12 30 Deidre Gifford. Phone interview with authors. April 7, 2014. 31 Oregon Health & Science University. “Center for Evidence-based Policy.” Retrieved July 7, 2014. http://www. ohsu.edu/xd/research/centers-institutes/evidence-based-policy-center/. 32 Oregon Health & Science University. “Pacific Northwest Evidence-based Practice Center.” Retrieved July 7, 2014. http://www.ohsu.edu/xd/research/centers-institutes/evidence-based-practice-center/. 33 Judy Zerzan. Phone interview with authors. April 19, 2014. 34 Ibid. 35 Oregon Health & Science University. “Medicaid Evidence Based Decisions Project (MED).” Retrieved July 7, 2014. http://www.ohsu.edu/xd/research/centers-institutes/evidence-based-policy-center/med/index.cfm. 36 Michelle Harper. Phone interview with authors. April 9, 2014. 37 The New England Comparative Effectiveness Public Advisory Council. “CEPAC Members.” Retrieved July 7, 2014. http://cepac.icer-review.org/?page_id=22. 38 The New England Comparative Effectiveness Public Advisory Council. “About CEPAC.” Retrieved July 7, 2014. http://cepac.icer-review.org/http://cepac.icer-review.org/?page_id=18. 39 The New England Comparative Effectiveness Public Advisory Council, Community Health Workers: A Review of Program Evolution, Evidence on Effectiveness and Value, and Status of Workforce Development in New England. (Boston, MA: Institute for Clinical & Economic Review, 2013). http://cepac.icer-review.org/?page_id=1066. 40 The Agency for Healthcare Research and Quality. ““Search for Resource Summaries, Reviews, and Reports” Retrieved July 7, 2014. http://www.effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/ 41 U.S. National Library of Medicine. “National Information Center on Health Services Research and Health Care Technology.” Retrieved July 7, 2014. https://www.nlm.nih.gov/hsrinfo/cer.html. National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 21

42 National Networks of Libraries of Medicine. “Information RX Tool Kit.” Retrieved July 7, 2014. http://nnlm. gov/hip/infoRx/interior.html. 43 U.S. Food and Drug Administration. “Janus Clinical Trials Repository (CTR) Project.” Retrieved July 7, 2014. http://www.fda.gov/ForIndustry/DataStandards/StudyDataStandards/ucm155327.htm. 44 The Centers for Disease Control and Prevention and The National Heart, Lung, and Blood Institute. “Prevention Impacts Stimulation Model (PRISM) for Chronic Disease Policymaking.” Retrieved July 7, 2014. http:// obssr.od.nih.gov/scientific_areas/translation/dissemination_and_implementation/DI2012/resources/PRISM_info_ and_bios_for_ISDC_2011_proceedings.pdf. 45 Ibid. 46 RTI International. Comparative Effectiveness Research.” Retrieved July 7, 2014. http://www.rti.org/page. cfm?objectid=4FA3323C-51CA-4DFE-A3E0246F90D5FA4B. 47 University of North Carolina Cecil G. Sheps Center for Health Services Research. “Incorporating Comparative Effectiveness Research Tools to Examine the Effect of a Reimbursement Policy Change on Local Public Health Service Outcomes.” Retrieved July 7, 2014. http://www.shepscenter.unc.edu/project/effect-of-a- reimbursement-policy-change-on-local-public-health-service-outcomes/. 48 University of Washington Department of Psychiatry and Behavioral Sciences. “Evidence Based Practice Institute.” Retrieved July 7, 2014. http://depts.washington.edu/pbhjp/evidence-based-practice-institute/about- institute. 49 Cathy Caldwell. Phone interview with authors. April 15, 2014. 50 Patient-Centered Outcomes Research Institute. “PCORI Funding Awards.” Retrieved July 7, 2014. http:// pfaawards.pcori.org/. 51 The Cochrane Collaboration. Retrieved July 7, 2014. http://www.cochrane.org/. 52 RTI International. Comparative Effectiveness Research.” Retrieved July 7, 2014. http://www.rti.org/page. cfm?objectid=4FA3323C-51CA-4DFE-A3E0246F90D5FA4B. 53 Colorado Foundation for Public Health and the Environment. Retrieved July 7, 2014. http://www.cfphe.org/. 54 Kansas Health Institute. Retrieved July 7, 2014. http://www.khi.org/. 55 The Missouri Institute for Community Health. Retrieved July 7, 2014. http://www.michweb.org/. 56 Georgia State University. “Georgia Health Policy Center.” Retrieved July 7, 2014. www.gsu.edu/ghpc. 57 Institute for Clinical and Economic Review. Retrieved July 7, 2014. http://www.icer-review.org/. 58 Institute for Clinical and Economic Review. “California Technology Assessment Forum (CTAF) Issues Final Report and Action Guides on New Treatments for Hepatitis C.” Retrieved July 7, 2014. http://www.icer-review. org/california-technology-assessment-forum-ctaf-issues-final-report-and-action-guides-on-new-treatments-for- hepatitis-c/. 59 Choosing Wisely. “Lists.” Retrieved July 7, 2014. http://www.choosingwisely.org/doctor-patient-lists/. 60 National Networks of Libraries of Medicine. “Information RX Tool Kit.” Retrieved July 7, 2014. http://nnlm. gov/hip/infoRx/interior.html. 61 American Academy of Family Physicians. “Clinical Recommendations A-Z.” Retrieved July 7, 2014. http:// www.aafp.org/patient-care/clinical-recommendations/a-z.html. 62 American College of Physicians. “ACP Clinical Practice Guidelines.” Retrieved July 7. 2014. http://www. acponline.org/clinical_information/guidelines/guidelines/. 63 Center for Science in the Public Interest. Retrieved July 7, 2014. https://www.cspinet.org/. 64 Hayes Inc. Retrieved July 7, 2014. http://www.hayesinc.com/hayes/.

65 Truven Health Analytics. “ComparativeNational AcademyEffectiveness for State Research.” Health Policy Retrieved July 7, 2014. http://truvenhealth.

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers 22

com/your-healthcare-focus/Life-Sciences/Comparative-Effectiveness-Research. 66 Cathy Caldwell. Phone interview with authors. April 15, 2014. 67 Other sources include The American College of Occupational and Environmental Medicine (ACOEM), The Workers Compensation Research Institute (WCRI), and The International Association of Industrial Accident Boards and Commissions (IAIABC). 68 Work Loss Data Institute. Retrieved July 7, 2014. http://www.worklossdata.com/about-wldi.html .

National Academy for State Health Policy

Programs Supporting The Use Of Comparative Effectiveness Research And Patient-Centered Outcomes Research By State Policymakers