Welcome Letter from Paul and Sue.

Developing strong T3/T4 Pilot and Scholar Proposals– An Interactive, Pragmatic Workshop focusing on dissemination and implementation Science Research Day 1 – Monday, August 14

Day 1 Objectives:  Identify levels of translational research from T1 through T4 with a focus on how dissemination and implementation research reflect T3 and T4 levels.  Develop plans to initiate dissemination and implementation (D&I) research proposals while addressing the distinct areas where D&I proposals are similar to and different from other types of research proposals.  Identify key tips, strategies, and important steps in the development of innovative and high quality D&I grant proposals based on reviews of successful NIH applications.

Day 1 Agenda Location: College of Public Health Topic Learning Objective Time Speakers Registration To network with colleagues 8:00– 8:30 am Welcoming Distinguish between levels across the 8:30 – 9:00 Estabrooks Remarks translational research spectrum from T1-T4. Define the field of D&I Keynote 1 8:45 – 9:30 Glasgow Understand the progression of the field from initiation to present day Participant 9:30-10:00 Glasgow questions BREAK 10:00-10:10

Understand key features necessary for a Keynote 3 10:10-10:45 Almeida successful D&I research proposal

Participant 10:45-11:00 questions Clarifying key ingredients Almeida

Identify a strong D&I research question Applied Identify initial methods to address key D&I 11:00-12:00 Learning Harden proposal components.

BREAK Break time to pick up lunch before workshops 12-12:15 pm Topic Learning Objective Time Speakers Participants select 2, 30-minute workshops (Each workshop: minute overview; 10 minute case application teach back and Q&A INTERACTIVE To recognize how to identify evidence-based Facilitated Workshops: LUNCH 12:15 -1:15 pm interventions, D&I frameworks, and how social - Where to find evidence-based and pragmatic interventions WORKSHOPS networks have been used in D&I research and measures (Glasgow & Zimmerman) - D&I frameworks (Estabrooks) - Social networks and implementation science (Almeida) Understanding the concept of implementation science as it pertains to evidence-informed or Keynote 4 evidence-based principles rather than products. 1:15-2:00 Dzewaltowski Describing pitfalls and solutions when submitting such a proposal. BREAK 2:00-2:15

Participants work in small groups to identify Interaction what the ‘it’ that is being translated is and how 2:15-3:15 Moderator: Zimmerman Session best to define it. Q&A follow-up as large group Moderator: Zimmerman Panel: Glasgow, Dzewaltowski, Day 1 Closing To review common themes and ideas across 3:15 - 4:00 Harden, & Almeida Panel sessions End of Day 1 for public at 4:00 – internal Internal workshop team debrief with external 4:00-4:30 pm team debrief speakers to identify strengths of workshop follows Day 2 Objective: To compare consider practical methods to for pilot and scholar proposals that can set the stage for NIH funding. To generate ideas and methods to integrate costs and community-engaged approaches for T3/T4 research

Day 2 Agenda: Location: College of Public Health

Topic Learning Objective Time Speaker Activity Continental Breakfast or 7:30-8:00 Coffee/Tea ($$?) Describe methods, theories and research designs Title: “Being practical in the appropriate for D&I use of D&I outcomes, Keynote Address 2 research with a focus on GP 8:00 – 9:00 Estabrooks theories, and research designs” IDEA CTR scholar or pilot discussion applications.

Identify potential areas Presentation Title: for cost assessment and Developing cost assessment the potential to conduct 9:00 – 9:45 pm and analyses for D&I Interaction Session Michaud cost-effectiveness proposals. analysis 30 min talk 15 min discussion

BREAK 9:45-10:00 To understand community- engaged research and its application for D&I research King and Interaction Session 3 approaches to improve health 10:00-10:45 Su Title: To be determined. disparities 30 min talk 15 min discussion Describe benefits of, and ways to, engage with the GP Facilitated discussion 10:45-11:15 Palm IDeA CTR Community Advisory Board Humorous depiction of a D&I To understand common grant Glasgow 11:15-11:45 study section with focus on Mock Grant Reviews reviewer critiques of D&I Estabrooks underscoring key ingredients grants Harden for funding. To describe the GP IDeA CTR Presention and question and Resources and steps to Key Component areas, answers about how to submit scholar highlight the benefits of 11:45-12:30 Estabrooks engage appropriate KCAs to applications and pilots engaging KCAs in help with proposal development of proposal development

Box lunches to go 12:30 Dissemination and Implementation as defined by the National Institutes of Health… “Dissemination is the targeted distribution of information and intervention materials to a specific public health or clinical practice audience. The intent is to spread (“scale up”) and sustain knowledge and the associated evidence-based interventions.

Implementation is the use of strategies to adopt and integrate evidence-based health interventions and change practice patterns within specific settings....

Dissemination and implementation research intends to bridge the gap between public health, clinical research, and everyday practice by building a knowledge base about how health information, interventions, and new clinical practices and policies are transmitted and translated for public health and health care service use in specific settings.”

Source: Department of Health and Human Services. Part 1 Overview Information Dissemination and Implementation Research in Health (R01). http://grants.nih.gov/grants/guide/pa-files/PAR-13-055.html . Other Important concepts and definitions

Evidence-based intervention. The objects of D&I activities are interventions with proven efficacy and effectiveness (i.e., evidence-based). Interventions within D&I research should be defined broadly and may include programs, practices, processes, policies, and guidelines. More comprehensive definitions of evidence-based interventions are available elsewhere. In D&I research, we often encounter complex interventions (e.g., interventions using community- wide education) where the description of core intervention components and their relationships involve multiple settings, audiences, and approaches.

Adoption. Adoption is the decision of an organization or community to commit to and initiate an evidence-based intervention.

Sustainability. Sustainability describes the extent to which an evidence-based intervention can deliver its intended benefits over an extended period of time after external support from the donor agency is terminated. Most often sustainability is measured through the continued use of intervention components; however, Scheirer and Dearing suggest that measures for sustainability should also include considerations of maintained community or organizational-level partnerships, maintenance of organizational or community practices, procedures, and policies that were initiated during the implementation of the intervention, sustained organizational or community attention to the issue that the intervention is designed to address, and efforts for program diffusion and replication in other sites. Three operational indicators of sustainability are: (1) maintenance of a program's initial health benefits, (2) institutionalization of the program in a setting or community, and (3) capacity building in the recipient setting or community. Scale-up/Scalability. Efficacious interventions expanded under real world conditions to reach a greater proportion of the eligible population, while retaining effectiveness (Milat, King, Bauman, & Redman, 2011) or extending the reach of an intervention by institutionalizing the intervention within a given organization/region or by replicating it in other localities, cities, or states or both (Reis et al., 2016).

Reinvention/adaptation. For the success of D&I, interventions often need to be reinvented or adapted to fit the local context (i.e., needs and realities). Reinvention or adaptation is defined as the degree to which an evidence-based intervention is changed or modified by a user during adoption and implementation to suit the needs of the setting or to improve the fit to local conditions. The need for adaptation and understanding of context has been called Type 3 evidence (i.e., the information needed to adapt and implement an evidence- based intervention in a particular setting or population). Ideally, adaptation will lead to at least equal intervention effects as is shown in the original efficacy or effectiveness trial. To reconcile the tension between fidelity and adaptation, the core components (or essential features) of an intervention (i.e., those responsible for its efficacy/effectiveness) must be identified and preserved during the adaptation process.

Dissemination strategy. Dissemination strategies describe mechanisms and approaches that are used to communicate and spread information about interventions to targeted users. Dissemination strategies are concerned with the packaging of the information about the intervention and the communication channels that are used to reach potential adopters and target audiences. Passive dissemination strategies include mass mailings, publication of information including practice guidelines, and untargeted presentations to heterogeneous groups. Active dissemination strategies include hands on technical assistance, replication guides, point-of-decision prompts for use, and mass media campaigns. It is consistently stated in the literature that dissemination strategies are necessary but not sufficient to ensure wide-spread use of an intervention.

For additional terms see Rabin, B.A. and Brownson, R.C. (2012). Developing the terminology for dissemination and implementation research in health. In Brownson, R.C., Colditz, G.A., & Proctor, E.K. (Eds.), Dissemination and Implementation Research in Health: Translating Science to Practice. New York: Oxford University Press or visit http://makeresearchmatter.org/glossary.aspx Dr. Paul Estabrooks: Distinguish between levels across the translational research spectrum from T1-T4.

Questions to determine translational phase

1. Does the research involve use of animals? a. If yes, does it test a potential diagnostic, preventive, or therapeutic approach? a.i. If yes, your research is likely translational at the pre-clinical level. a.ii. If no, your research is likely not translational. 2. Does the research involve study of mechanisms, relationships, or modification of proteins, DNA, or cells? a. If yes, does it test a potential diagnostic, preventive, or therapeutic approach? a.i. If yes, your research is likely translational at the pre-clinical level. a.ii. If no, your research is likely not translational. 3. Is the research a Genome Wide Association Study, determining association of a SNP with a particular disease state? a. If yes, your research is likely not translational. 4. Does the study examine drug interactions with molecular receptor or enzyme; effects on cell biochemistry, or how to optimize interaction? a. If yes, your research is likely not translational. 5. Does the research test a new methodology with potential for use in diagnosis, treatment, or prevention, providing a basis for follow-up study but does not test an intervention. a. If yes, your research is likely not translational. 6. Is a new association between biological, social, and/or behavioral states determined, including association between presence or progression of a disease state and a biomarker, social, or behavioral state but does not test an intervention? a. If yes, your research is likely not translational. 7. Does the research explore a biological, social, or behavioral mechanism, including the mechanism underlying the presence or progression of a disease, but does not attempt to intervene on the presence or progression of a disease? a. If yes, your research is likely not translational. 8. Is the research a systematic review or meta-analysis of research that seeks to establish a correlation or elucidate a mechanism, or to establish need for further work at to establish a methodology? a. If yes, your research is likely not translational. 9. Is the research a Phase I clinical trial? a. If yes, your research is translational at the T1 level. 10. Does the study test the effect of a new intervention on healthy volunteers in a controlled clinical setting? a. If yes, your research is translational at the T1 level. 11. Does the research suggest a new method of diagnosis or new intervention, determine feasibility or safety, or test it in a small group? a. If yes, your research is translational at the T1 level. 12.Does research describe the use of a new device on a small population to determine potential usefulness and usability? a. If yes, your research is translational at the T1 level. 13.Does the research use a Phase II clinical trial methodology? a. If yes, your research is likely translational at the T1 level. 14.Does the research use a Phase III or Phase IV clinical trial methodology? a. If yes, your research is likely translational at the T2 level. 15.Does the study test efficacy and/or determine dosing levels of an intervention in a population with a given disease in a controlled clinical setting? a. If yes, your research is likely translational at the T2 level. 16.Does the study determine the optimal use of a new medical device? a. If yes, your research is likely translational at the T2 level. 17.Does the study determine the efficacy or optimal use of a new method of diagnosis or prevention, including through the use of existing datasets? a. If yes, your research is likely translational at the T2 level. 18.Does the research study effectiveness of an intervention or method of diagnosis in the clinic or community, either through the use of existing health datasets or new research? a. If yes, your research is likely translational at the T3 level. 19.Does the research study inconsistency or variation in the application of a diagnosis or intervention? a. If yes, your research is likely translational at the T3 level. 20.Does the research compare the effectiveness of existing health care interventions to determine which work best for which patients and which pose the greatest benefits and harms? a. If yes, your research is likely translational at the T3 level. 21.Does the research involve interventions in the community with input from community members? a. If yes, your research is likely translational at the T2, T3, or T4 level. 22.Does the research determine mechanisms underlying effective health care delivery in practice or community settings? a. If yes, your research is likely translational at the T3 level. 23.Does the research identify problems with effective health care delivery in practice or community settings? a. If yes, your research is likely translational at the T3 level. 24.Does the research test an intervention to improve healthcare delivery in practice or community settings? a. If yes, your research is likely translational at the T2, T3, or T4 level. 25.Does the research study real world factors affecting interventions (cost, convenience, accessibility, patient preferences)? a. If yes, your research is likely translational at the T2, T3, or T4 level. 26.Does the research determine reasons why gaps in care exist? a. If yes, your research is likely translational at the T3 or T4 level. 27.Does the research result in evidence-based guidelines/policies or best practices? a. If yes, your research is likely translational at the T3 level. 28.Does the research use large datasets to monitor morbidity, mortality, benefits, or risks of interventions in populations? a. If yes, your research is likely translational at the T4 level. 29.Does the research examine problems with, mechanisms underlying, potential interventions, impact, or real world outcomes of population level practices/interventions? a. If yes, your research is likely translational at the T4 level. 30.Does the research study the impact on health of a policy or environmental change that affects a population? a. If yes, your research is likely translational at the T4 level. 31.Does the research focus on the development or outcomes of population level behavioral/lifestyle interventions? a. If yes, your research is likely translational at the T4 level.

Questions adapted from Surkis et al., 2016 using definitions proposed by Khoury et al., 2007, 2010, 2011.

Dr. Glasgow Talk Dr. Almeida Talk

Key Ingredients to Writing Successful D&I Research Proposals

Getting Started: Is this a good D&I research question? o A significant and innovative research question that is appropriate for Dissemination and Implementation science (D&I) is the first and primary ingredient in a successful proposal. o Programs ready for D&I should show promise of reducing a gap in quality of care. D&I studies should be conducted in real world settings with representative patients and, in general, delivered by regular staff rather than researchers. See “ingredient 1” on the checklist below for more details. o D&I research in health focuses on a health condition or disease(s) in healthcare or community settings, and particular interventions or policies that are already evidence-based (e.g., in efficacy or effectiveness studies, systematic reviews, clinical care guidelines, etc.). See “ingredient 2” on the checklist below for more details.

Although we have framed this as a checklist, we recognize that it may be impossible to get all 10 ingredients in any given project. Click the box to check each item that you have addressed at least moderately well. □ 1. The Quality Gap or Care Gap:

● Provide clear evidence that a modifiable gap in care/quality exists at the population, organization or provider level. ● Include the following types of evidence of a Quality Gap: o Wide variation in care delivery, or poor/suboptimal care delivery – cite references in Significance o Potential to improve care through the proposed work – present Preliminary Data and/or Significance

□ 2. The Evidence-Based intervention or policy to be implemented or disseminated:

● Does the evidence demonstrate the efficacy/effectiveness of the program to be implemented? ● Can you support that you are using an evidence-based treatment? o Some experts suggest a minimum standard of demonstrated efficacy by at least 2 separate investigators using a manualized intervention. o Even stronger evidence would include the use of a program that has been promoted in an evidence-based guideline, systematic review, or promoted by 3rd parties as an effective program: Cochrane Review; CDC list of community health programs, especially related to tobacco and obesity; National Cancer Institute Research Tested Intervention Program related to cancer control/prevention; Canadian Best Practices Portal; Laura and John Arnold Foundation. o □ 3. Conceptual model/theoretical framework:

● Propose a clear conceptual model or theoretical framework that justifies the key relationships and variables that will be tested. o Even studies that address only a subset of variables within a conceptual model need to be framed conceptually, so that reviewers perceive the larger context (and body of literature) ● Present an overarching D&I model or D&I Theory to guide the investigation o Consider carefully which model or theory is most appropriate for your work—you may benefit from consultation with an expert D&I consultant o Examples of D&I models: Reach, Effectiveness, Adoption, Implementation, Maintenance (RE- AIM); Consolidated Framework for Advancing Implementation Science (CFIR) o Examples of D&I theories: Rogers’ Diffusion of Innovation theory ● Incorporate the D&I models/theories into the Approach, including measures and analyses – a mere mention of the model will not suffice

□ 4. Stakeholder priorities and engagement in change:

● Demonstrate that your program has a good fit with the preferences and priorities of those who shape, deliver, and participate in healthcare o Minimum requirement: include letters of support from stakeholders that highlight the shared priorities o If at all possible, include relevant preliminary qualitative or quantitative data demonstrating that stakeholders are engaged. Even better, publish such data with stakeholders as co-authors. ● Stakeholders may include all of the following, and will vary by project: health system administrators (or community organization leaders), health providers and staff (or community organization staff), patients and families (or community members), policymakers. □ 5. Setting’s readiness to adopt a new intervention: ● Beyond your specific setting - include citations on any innovations in the organizational and policy context that will influence setting’s readiness for change. ● Specific to your sites – include preliminary data regarding your settings’ appetite for change concerning the evidence-based practice that you propose. If no data are available, letters can also convey this type of information, albeit less strongly. ● Avoid cherry-picking sites for implementation that you know will adopt intervention well, as reviewers know that this will incompletely inform broader dissemination efforts. □ 6. Implementation strategy/process: ● At present, D&I research seeks to go beyond identifying barriers to implementation to “raise the bar” and generate knowledge about the D&I strategies that can overcome such barriers. ● To this end, it is extremely helpful to use existing implementation science strategy models to define proposed mechanisms by which you hypothesize that your implementation strategies will exert their effects, such as the WIDER and SQUIRE models. □ 7. Team experience with the setting, treatment, implementation process, and review environment.

● Include an experienced D&I researcher on the research team, as well as other investigators needed to address the methods required for the project (e.g., mixed methods, biostatistician, health economist). ● Convey the team’s expertise and the D&I infrastructure in your institution in several places, including the grant narrative, biosketches, budget justification, and the environment sections of the grant. □ 8. Feasibility of Proposed Research Design and Methods

● Preliminary data should include data on as many of the following categories as possible: o referral flow; o participant entry into the study; o participant retention; o the extent to which key measures are feasible to use, understood by participants, acceptable for use, are sensitive to change, and capture variability o demonstrate that the proposed methods are likely to work. □ 9. Measurement and Analysis ● Make sure you are measuring each key construct in the guiding theory (see ingredient #3 above- Conceptual Model/Theoretical Framework) ● Ensure measurements of fidelity to key components AND adaptation of the intervention or implementation strategies ● Consider the use of measures from the National Cancer Institute’s Grid-Enabled Measures (GEM), the NIH PROMIS measures set; or the Comprehensive Review of D&I Science Instruments supported by the Seattle Implementation Research Conference (SIRC) at the University of Washington □ 10. Policy environment will leverage, support, and sustain the change

● Address how the proposal aligns with current and anticipated policy changes, as this enhances the public health significance of the project and likelihood of adoption and sustainability. ● Dr. Harden Talk

Facilitated workgroup sessions (noon August 14th)

Dr. Dzewaltowski’s Talk

Dr. Estabrooks Talk II Dr. Michaud Talk Drs. Su and King Talk