Assessment in Graduate Medical Education: a Primer for Pediatric Program Directors
Total Page:16
File Type:pdf, Size:1020Kb
Assessment in Graduate Medical Education: A Primer for Pediatric Program Directors The American Board of Pediatrics and The Association of Pediatric Program Directors Supported by the ABP Foundation Assessment in Graduate Medical Education: A Primer for Pediatric Program Directors Program Directors Committee of the American Board of Pediatrics Project Leaders: Carol L. Carraccio, MD, MA Patricia J. Hicks, MD Contributors: Ann E. Burke, MD M. Douglas Jones, Jr., MD Stephen Ludwig, MD Gail A. McGuinness, MD Julia A. McMillan, MD Richard P. Shugerman, MD Suzanne K. Woods, MD Alan Schwartz, PhD, Editor Copyright © 2011, American Board of Pediatrics, 111 Silver Cedar Court, Chapel Hill, NC, 27514. All rights re- served. This project was developed by the Program Directors Committee of American Board of Pediatrics (ABP) and the Association of Pediatric Program Directors (APPD) and sponsored by the ABP Foundation. The contributors grate- fully acknowledge the administrative support of Pam Moore at ABP. The document is available on the ABP Web site www.abp.org and the APPD Web site www.appd.org. Suggested citation: Schwartz A, editor. 2011. Assessment in Graduate Medical Education: A Primer for Pediatric Program Directors. Chapel Hill, NC: American Board of Pediatrics. Assessment in Graduate Medical Education: A Primer for Pediatric Program Directors Contents Part I: Introduction to Assessment Principles and Techniques ..................................................................... 5 1. Measurement Principles in Medical Education .................................................................................... 7 2. Assessment Methods .......................................................................................................................... 15 3. Faculty Development .......................................................................................................................... 21 4. Self-assessment ................................................................................................................................... 29 5. Portfolios in Medical Education ......................................................................................................... 37 6. Program Evaluation ............................................................................................................................ 47 Part II: Assessment of the ACGME Core Competencies ............................................................................ 55 7. Patient Care ......................................................................................................................................... 61 8. Medical Knowledge ............................................................................................................................ 71 9. Practice-based Learning and Improvement ........................................................................................ 79 10. Interpersonal and Communication Skills ............................................................................................ 87 11. Professionalism ................................................................................................................................... 97 12. Systems-based Practice ..................................................................................................................... 105 Resources for Further Learning ................................................................................................................ 111 Glossary .................................................................................................................................................... 113 About the Contributors ............................................................................................................................. 119 Part I: Introduction to Assessment Principles and Techniques Carol L. Carraccio, MD, MA, Patricia J. Hicks, MD, and Alan Schwartz, PhD At the cusp of the millennium, The Accreditation Council for Graduate Medical Education (ACGME) presented professionals in Graduate Medical Education (GME) with the challenge of implementing com- petency-based medical education with a focus on six broad and partially-overlapping domains. Although developing various instructional approaches to meaningfully incorporate these competencies into the GME curriculum has not been easy, assessment of the competencies has presented the most daunting task. To develop methods of assessment that provide a sound basis for making judgments about the progression of residents, program directors need to partner with medical educators with expertise in assessment meth- odology. This partnership is critically important because it pairs experts in the methodology of assessment with program directors who teach and assess learners in the context of delivering care. In other words, it grounds meaningful assessment in real world practice. The challenge of improving assessment in medical education brings many great opportunities for program directors. Most importantly, we have the opportunity to advance the field of assessment in medical educa- tion itself. Despite the difficulties and barriers this challenge presents, the rewards of our work as program directors rest on our contribution to the professional formation of physicians-in-training through informed assessment and feedback with the intent of helping them to continually improve. With meaningful indi- vidual performance assessment, program directors can also aggregate outcomes as metrics to guide pro- gram evaluation and improvement. The impact of assessment is powerful in driving change and pushing for transformation. If we want to have impact—to bring about transformation in medical education—we need to be learners as well as teachers. Medical school and residency did not provide us with the skills needed to be medical educators. We need to embrace our own continued learning so that we can live up to the work with which we have been entrusted. As academic faculty, program directors should seek evidence, through medical education research, that their work achieves the desired outcomes. Scholarship within medical education spans a wide array of domains, such as curriculum development, instructional methods, and constructive and effective feedback—all areas where assessment of outcomes is critical. Development of methods and tools to assess resident competence is a ripe area for such scholarship and requires us to rigorously study the evidence of the validity and utility of the results they produce. Medical education research provides opportunities for regional and national scholarly presentations as well as peer-reviewed publications. In short, improving the quality of assessments in our residency programs is not only beneficial to our resi- dents and their patients, but also provides opportunities for both personal satisfaction and advancing the field. The Primer is intended to be a stimulus for future workshops at our various program director, clerkship director, and other professional society meetings. But our responsibility does not end here. Once we our- selves have learned, we must transmit our knowledge through faculty development at our own institu- tions, without which we will never have the meaningful assessment of learners that is our vision and this Primer’s goal. We offer this Primer as a way to begin conversations around the complex problem of assessment in medi- cal education and training. In writing this Primer, program directors shared their many years of experi- ence with different levels of learners, and then invited an assessment expert, Dr. Alan Schwartz, to join 5 INTRODUCTION and guide the endeavor. Each chapter focuses on the principles of assessment within the context of famil- iar assessment challenges that program and clerkship directors and other faculty involved in medical edu- cation face on a routine basis. At the end of each chapter, an annotated bibliography of the most relevant articles is presented to assist you with further reading. We also include a more extensive bibliography for more seasoned educators who wish to go into greater depth. The authors purposefully tried to limit the use of educational jargon; however, familiarity with the terms that we include in italics within the text of the chapters as well as the glossary will enable you to approach the medical education literature with a greater understanding. We hope that the result is a user-friendly document that invites you to further explore in more depth the principles and practices of assessment where we have only exposed the surface. The primer is divided into two parts. Part I focuses on the foundation of assessment principles and meth- ods of assessment, faculty development and program evaluation. In Part II, we apply theory to practice by illustrating these principles and methods in the context of each of the ACGME domains of competence. We are grateful for the partnership role that the membership of the Association of Pediatric Program Di- rectors (APPD) has played in the development of a direction for this primer. Their needs established the content focus for the primer and their grounding in the trenches helped us to develop what we hope is a meaningful and practical guide to competency-based assessment of learners. Our vision for the future is that this primer will be step one in an ongoing process of program director development. The partnership with APPD provides a mechanism for feedback as to needed next steps