Evaluation Guidelines for the Clinical and Translational Science Awards (Ctsas)

Special Reports Evaluation Guidelines for the Clinical and Translational Science Awards (CTSAs) William M. Trochim , Ph.D. 1 , Doris M. Rubio , Ph.D. 2 , Veronica G. Thomas , Ph.D. 3 , and the Evaluation Key Function Committee of the CTSA Consortium Abstract The National Center for Advancing Translational Sciences (NCATS), a part of the National Institutes of Health, currently funds the Clini- cal and Translational Science Awards (CTSAs), a national consortium of 61 medical research institutions in 30 states and the District of Columbia. The program seeks to transform the way biomedical research is conducted, speed the translation of laboratory discoveries into treatments for patients, engage communities in clinical research efforts, and train a new generation of clinical and translational researchers. An endeavor as ambitious and complex as the CTSA program requires high-quality evaluations in order to show that the program is well implemented, effi ciently managed, and demonstrably effective. In this paper, the Evaluation Key Function Committee of the CTSA Consortium presents an overall framework for evaluating the CTSA program and offers policies to guide the evaluation work. The guidelines set forth are designed to serve as a tool for education within the CTSA community by illuminating key issues and practices that should be considered during evaluation planning, implementation, and utilization. Additionally, these guidelines can provide a basis for ongoing discussions about how the principles articulated in this paper can most effectively be translated into operational reality. Clin Trans Sci 2013; Volume 6: 303–309 Keywords: evaluation , translational research , CTSA Introduction Translational research is a critically important endeavor in appropriate evaluative data and assessment systems, it would contemporary biomedical research and practice. Yet, there be diffi cult to guide development of policies for CTSAs, in have long been considerable stumbling blocks to successful general, and CTSA operations more specifi cally. Evaluation of translational research eff orts in science, medicine, and public the CTSA program can provide the prospective and retrospective health. Several years ago, a seminal paper published in Th e information necessary to direct its course and to assess the degree Journal of the American Medical Association stressed, “Without to which the program is accomplishing its goals. mechanisms and infrastructure to accomplish this translation Th e purposes of this paper are to present an overall framework in a systematic and coherent way, the sum of the data and for evaluating the CTSA program and to off er policies to guide the information produced by the basic science enterprise will not evaluation work. Th e term CTSA program refers here to the entire result in tangible public benefi t.” 6 To address this major concern, initiative that encompasses 61 sites across the United States and a through discussions with deans of academic health centers, national consortium consisting of representatives of the sites and of recommendations from the Institute of Medicine, and meetings the NIH. Th is document is not intended to be a series of prescriptive with the research community, the National Institutes of Health requirements. Rather, it is intended to provide general guidance on (NIH) recognized that a broad reengineering eff ort was needed evaluation in the CTSA context, to discuss the critically important to create greater opportunity to catalyze the development of a role of evaluation, and to present recommendations designed to new discipline of clinical and translational science. Th is resulted enhance the quality of current and future CTSA evaluation eff orts. in NIH launching the Clinical and Translational Science Awards While much of what is addressed in the document may be directly (CTSA) in October 2006. Th e program, supported initially by the generalizable to translational research eff orts outside of the CTSAs National Center for Research Resources (NCRR) and then by the or to other types of large multicenter grant initiatives, the focus of National Center for Advancing Translational Sciences (NCATS), the document is on the CTSA context. currently funds a national consortium of 61 medical research Th e guidelines provided in this paper are intended to off er institutions in 30 states and the District of Columbia that are recommendations for the myriad ways that CTSA evaluations can seeking to transform the way biomedical research is conducted, be accomplished given the range and complexity of individual and speed the translation of laboratory discoveries into treatments for collective CTSA evaluative eff orts. Th ese recommendations are patients, engage communities in clinical research eff orts, and train based upon the best practices from the discipline and profession a new generation of clinical and translational researchers. Th e of evaluation. Th e leading professional association in the fi eld, CTSA program, at a cost of approximately a half billion dollars the American Evaluation Association (AEA), has through its per year, is part of a larger 21st century movement to develop a Evaluation Policy Task Force (EPTF) produced the document discipline of clinical and translational science. “ An Evaluation Roadmap for a More Eff ective Federal Government ” An endeavor as ambitious and complex as the CTSA program (Th e “AEA Roadmap”) that we have learned from and modeled requires high-quality evaluation. Th e program needs to show that in developing CTSA evaluation guidelines. Th e authors, who are it is well implemented, effi ciently managed, and demonstrably members of the CTSA Evaluation Key Function Committee’s eff ective. Evaluation is key to achieving these goals. Without National Evaluation Liaison Workgroup, received consultation 1 Department of Policy Analysis and Management , Cornell University , Ithaca , New York , USA ; 2 Data Center , Center for Research on Health Care , Division of General Internal Medicine , Department of Medicine , School of Medicine , University of Pittsburgh , Pittsburgh , Pennsylvania , USA ; 3 Department of Human Development and Psychoeducational Studies , Howard University , Washington , DC , USA . Correspondence: William M. Trochim ([email protected]) D O I : 10 .1111/c t s .12036 WWW.CTSJOURNAL.COM VOLUME 6 • ISSUE 4 303 Trochim et al. ■ Evaluation Guidelines for CTSAs and feedback from the AEA’s EPTF in the formulation of the framed within selected topical areas. Th e recommendations guidelines provided in this paper. Additionally, considerable input are not prescriptive; they are meant to off er broad guidance was obtained from members of the Evaluation Key Function for CTSA evaluation. Th ey are based on the observations and Committee of the CTSA Consortium. experience of evaluators currently engaged in the CTSAs. Some of the recommendations may have direct implications for policy The CTSA Context or action. Others have the intention of clarifying a conceptual Th e CTSAs, awarded by the NIH, constitute one of the most concern or encouraging thinking about a complex evaluation ambitious and important endeavors in biomedical research and challenge. practice in the early part of the 21st century. Th e CTSA program was born out of the NIH Roadmap trans-NIH initiatives designed Scope of CTSA Evaluation to accelerate the pace of discovery and improve the translation Th e range and complexity of evaluation questions and issues that of research fi ndings to improve healthcare. Launched in 2006 by need to be addressed in CTSA contexts are both exciting and the NIH, the program supports a national consortium of medical daunting. Th e breadth of purpose and scope constitutes one of research institutions that seek to transform the way biomedical the major challenges for CTSA evaluation. research is conducted. Th e goals of the program are to accelerate the translation of laboratory discoveries into treatments for patients, to Evaluation should engage stakeholders in all phases of the engage communities in clinical research eff orts, and to train a new evaluation generation of clinical and translational researchers. At its core, the It is critical to identify and engage the stakeholders from the CTSA program is designed to enable innovative research teams beginning of the evaluation. Th is will ensure that the input, to speed discovery and advance science aimed at improving our guidance, and perspectives of stakeholders are incorporated nation’s health, tackling complex medical and research challenges, in all phases of the evaluation, and it will also ensure that the and turning discoveries into practical solutions for patients. stakeholders are kept informed and are able to utilize the fi ndings. Unlike most other large initiatives of the NIH, the CTSA Stakeholders may be engaged through an advisory panel that initiative included evaluation efforts at its outset. The NIH has input into identifying evaluation goals, developing an required each CTSA institution to develop an evaluation program evaluation strategy, interpreting fi ndings, and implementing and to undertake site-level evaluations. It also required a national recommendations. evaluation of the entire CTSA initiative to be conducted by an external evaluator. Evaluation should be an integral part of program planning and implementation The Importance of Evaluation A common misunderstanding is that evaluation is simply an For purposes of this document, we use the definition of “add-on” to program activities, rather than an integral

Load more