Which Real-World Research Design Is Best?

Which Real-World Research Design Is Best? An Algorithmic Approach to Optimal Study Design for Outcomes Research By David Thompson, PhD © 2018 Syneos Health™. All rights reserved. WHICH RESEARCH DESIGN IS BEST? Introduction First Step: Engage Stakeholders Early and Often Manufacturers typically plan Phase II-III research with the utmost care, but all too often regard In clinical development, engagement with regulators is a routine component of study design, real-world evidence (RWE) generation as a “check-the-box” activity. This is a mistake. Real-world with consensus reached on study endpoints and analyses before proceeding to study conduct research design is more complicated than clinical trial design, its complexity due to a multitude and reporting of results. Historically, RWE generation has not followed this process but has been of factors including the differing evidentiary needs of diverse healthcare system stakeholders, more linear in nature, with manufacturers designing the research without stakeholder input and the differing outcome measures available to meet those needs, and the differing methodologic only engaging with stakeholders at the time of dissemination of study findings (Fig. 1). This approaches that can be used to collect clinical, economic and real-world data. approach runs the risk that the RWE generated will not meet stakeholder evidentiary needs, as their input was not solicited during research design. The most frequently utilized study designs for Phase IV research include: • Retrospective analyses of computerized health records (administrative claims and/or EHRs) • Manual chart review Figure 1. Historical approach to RWE generation fails in upfront engagement • Prospective observational studies and registries • Temporal flow is linear • Pragmatic clinical trials • Stakeholder engagement Real-World Real-World External • Randomized controlled trials efforts focused on back end Research Research Stakeholder • Economic modeling Design Execution Engagement • Key research design decisions disconnected from Selecting the most appropriate and cost-effective research design can be quite challenging. This stakeholder engagement is especially the case when stakeholders are not engaged at the beginning of the process. But even with robust stakeholder engagement from the start, identifying the most appropriate study design is still a challenge because of the multiple factors (data sources, methodologies, outcome measures, etc.) that must be considered. To facilitate this process, we have developed and tested an algorithm that has proven to be useful in structuring decision-making in the The optimal approach to RWE generation prioritizes stakeholders (Fig. 2). Similar to clinical design of real-world research. development, the process should begin with stakeholder engagement to ascertain evidentiary needs and agree upon the research design. This will help ensure that, upon study execution and dissemination of findings, health system stakeholders will find the evidence actionable to facilitate decision-making related to the product. Figure 2. Optimal Approach to RWE Generation Prioritizes Stakeholders External Stakeholder Stakeholder Temporal flow is engagement continuous Engagement occurs up front to inform design, on back for results Real-World Real-World Research Research Execution Design © 2018 Syneos Health™. All rights reserved. | 3 WHICH RESEARCH DESIGN IS BEST? Figure 4. Stakeholder interests vary for different real-world measures Treatment Costs, Cost- Comparative Patient-Reported Stakeholders Safety Patterns/ Effectiveness/ Effectiveness Outcomes Considering the (Many) Possibilities Adherence Budget Impact Regulatory ✔ ✔ ✔ ✔ ✔ ✔ Another important point of difference between clinical and RWE research is that, while clinical evidence generation Authorities is relatively straightforward, RWE is far from it. In clinical development, there is one major health system stakeholder to target (regulatory), two broad categories of study measures (safety and efficacy), and one type of HTA Agencies ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ study design (randomized controlled trial (RCT)). In contrast, RWE generation is complicated by the multiplicities of health system stakeholders to be targeted, real-world measures of interest, and methodologic approaches available (Fig. 3). The multidimensional nature of the real-world research design challenge is further complicated by the fact Payers ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ that stakeholders naturally vary in terms of their levels of interest in different real-world measures (Fig. 4) and different real-world study designs vary in terms of their suitability to the measures being assessed (Fig. 5). In some Patients and ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ sense, it’s as if the problem is exponentiating itself. Advocacy Groups Clinicians and Figure 3. RWE generation is anything but straightforward Professional ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Associations Institutions ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ (Hospitals, IDNs) Figure 5. Different real-world methods vary in suitability to measures being assessed Multiple health system Prospective stakeholders to target: Retrospective Medical Chart Pragmatic Clinical Economic Endpoint Types Observational/ Database Analysis Review Trials Modeling • Regulatory Registries • Payers Safety ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ • Clinicians • Patients Comparative ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ • Health technology Multiple real-world Effectiveness assessment agencies (HTAs) measures of interest: • Treatment patterns/ Patient-Reported ✔ ✔ ✔ ✔ ✔ ✔ compliance with therapy Outcomes • Comparative effectiveness Treatment Patterns/ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ • Long-term safety Multiple methodological Adherence • Cost-effectiveness and approaches: Costs, Cost- budget impact Effectiveness/ ✔ ✔ ✔ ✔ ✔ ✔ ✔ • Database analyses Budget Impact • Patient report outcomes (PROs) • Manual chart review • Prospective observational So how should we go about sorting through all of these for use in selecting the most appropriate modeling studies/registries complications? In other settings where decision-making approach. No such solution, however, exists to can take different twists and turns depending on support investigators in selecting the most • Pragmatic clinical trials multiple variables, algorithms have proven invaluable. appropriate real-world research design. We sought to • Economic modeling For example, algorithms are used in clinical guidelines bridge this gap by developing and testing an as a way to assist healthcare providers in making algorithm that provides structure for the study design decisions about optimal treatment approaches. selection process in real-world research. Likewise, health economists have developed algorithms © 2018 Syneos Health™. All rights reserved. | 5 WHICH RESEARCH DESIGN IS BEST? The Algorithm Figure 6. An algorithm to select optimal study design for real-world research Methodologic Approach Evidence Generation Algorithm Traditional RCT The algorithm, designed for ease of use and to strike a balance between being too simplistic or overly complex, consists of a series of structured yes/no questions (Fig. 6), as follows: Is the NO evidence need for an YES NO intervention? 1. Is the study focused on an intervention? 2. Is the intervention on the market? 3. Is the study intended to be comparative? Is the Is evidence YES product on the NO need for product 4. Is treatment assigned by study protocol? market? value? 5. Are data needed for the study available from existing sources? 6. Are those existing sources accessible in computerized form Is the (i.e., in administrative claims or electronic medical records)? NO study YES YES 7. Is the study setting real world? comparative? 8. Is the evidence need for product value? Is scientific Responding to each of these yes/no NO rigor of randomization questions within the structure of the needed? algorithm successfully guides the researcher to one of six different research designs: Are data (1) retrospective database analysis; available from YES (2) manual chart review; (3) prospective existing sources? non-interventional study / registry; (4) pragmatic clinical trial; (5) traditional randomized controlled trial; or All Some None (6) economic modeling. As research designs vary dramatically in terms of time and cost requirements, in those Possible instances in which multiple approaches Study Setting Hybrid are viable the algorithm steers the researcher to the most cost-effective option first Are data in claims (reading from left to right or EMRs? across the bottom row). YES NO Real World Experimental Database Analysis Chart Review Prospective Cohort/Registry Pragmatic Trial Phase IV Trial Economic Modeling © 2018 Syneos Health™. All rights reserved. | 7 WHICH RESEARCH DESIGN IS BEST? A Closer Look Discussion The first question asks whether or not the study is If none of the data are available from existing sources, If the study is comparative, the next question is whether An expanding array of healthcare decision- focused on a product. In nearly all instances, what we or if a hybrid approach is being used, then the study or not the scientific rigor of randomized treatment makers—including payers, providers, patient mean by “product” is a drug, a biologic or a medical would be classified as prospective observational or allocation is desired. If the answer is no, the algorithm advocacy groups, clinical guideline developers, device. However, in some instances, the focus might be disease registry. (In some regions, the term “registry” is takes us back over to the non-interventional study and regulators—now require, beyond

Load more