Training Evaluation Model: Evaluating and Improving Criminal Justice Training
Total Page:16
File Type:pdf, Size:1020Kb
The author(s) shown below used Federal funds provided by the U.S. Department of Justice and prepared the following final report: Document Title: Training Evaluation Model: Evaluating and Improving Criminal Justice Training Author(s): Kelly Bradley, Edward Connors Document No.: 244478 Date Received: December 2013 Award Number: 2003-DD-BX-K101 This report has not been published by the U.S. Department of Justice. To provide better customer service, NCJRS has made this Federally- funded grant report available electronically. Opinions or points of view expressed are those of the author(s) and do not necessarily reflect the official position or policies of the U.S. Department of Justice. Institute for Law and Justice Alexandria, Virginia Training Evaluation Model: Evaluating and Improving Criminal Justice Training Final Report September 2007 Submitted to National Institute of Justice Prepared by Kelly Bradley Edward Connors Institute for Law and Justice Acknowledgements The authors are deeply appreciative of the cooperation and support of all the individuals who made this project possible. We are indebted to Mark Gage, Marcia Williams, and Jim Foley of the National White Collar Crime Center; William Brunson of the National Judicial College and Nancy Yeend and John Paul Jones of the John Paul Jones Group; Liebe Geft, Sunny Lee, and Mark Katrikh of the Simon Wiesenthal Center Museum of Tolerance; and Steve Morrison, Michael Logsdon, Randall Milks, Regina Patrosi, and Michael Hummel of the National Corrections and Law Enforcement Training and Technology Center. These individuals showed the courage to open their programs to outside evaluators. We are also truly grateful for the cooperation of the training participants who agreed to be interviewed multiple times and complete surveys for this evaluation. We are also genuinely indebted to Detective Dave D’Amico of the Monmouth County, New Jersey Prosecutor’s Office, and Detective Alix Olson of the Madison, Wisconsin, Police Department, for allowing us to learn more about their efforts to combat crimes of hate and terror. A study of this depth and length does not thrive without strong support from the Office of Justice Programs' staff who provided guidance and oversight to the project, including Dr. Edwin Zedlewski, Dr. Katherine Browning and Maggie Heisler from NIJ, and Todd Brighton and Elaine Snyder from BJA. Additionally, we would like to thank Dr. James Wells, Dr. Kevin Minor, and Dr. Gary Cordner of Eastern Kentucky University for conducting and writing the evaluation of the NCLETTC training. They also contributed significantly to the development of the training evaluation model. Finally, we would like to thank the ILJ staff who worked so hard on this project, including Chera Baity, Tiffaney Hall, and Joan Peterschmidt for assistance in data collection; Deborah Spence for assistance with the literature review; and Dr. Thomas McEwen, Barbara Webster, and Laurie Samuel for their contributions in writing this report. This project was supported by grant number 2003-DD-BX-K101 awarded to the Institute for Law and Justice by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice. Points of view in this document are those of the authors and do not necessarily represent the official position or policies of the U.S. Department of Justice Table of Contents Chapter 1: Project Background and Report Overview Project Summary..............................................................................................................................1 Need for a Criminal Justice Training Evaluation Model.................................................................2 Overview of the Report....................................................................................................................3 Chapter 2: Planning for Evaluations Types of Evaluations........................................................................................................................6 Evaluation Planning Steps ...............................................................................................................9 Identify Program Goals, Objectives, and Evaluation Questions..............................................10 Develop Conceptual Framework and Logic Model.................................................................10 Design Evaluation Methodology .............................................................................................11 Conduct the Evaluation............................................................................................................15 Analyze and Communicate Evaluation Results.......................................................................18 Evaluating Criminal Justice Training Programs............................................................................20 Opportunities for Control and Comparison Groups.................................................................20 Challenges in Evaluating Criminal Justice Training ...............................................................20 Chapter 3: Factors That Contribute to Successful Practitioner Training Outcomes Training Objectives........................................................................................................................22 Adult Learning Concepts ...............................................................................................................23 Instructional Methods ....................................................................................................................25 Practical Training Matters..............................................................................................................28 Facilitation Skills ...........................................................................................................................28 Communication Skills..............................................................................................................29 Active Listening.......................................................................................................................31 Body Language ........................................................................................................................32 Sensitivity to Adult Students' Cultural Diversity.....................................................................32 Chapter 4: Criminal Justice Training Evaluation Model Kirkpatrick’s Training Evaluation Model......................................................................................34 Customizing and Expanding on Kirkpatrick’s Evaluation Model for Criminal Justice Training .37 Conduct Needs Assessment .....................................................................................................40 Design Training Plan ...............................................................................................................41 Develop and Test the Curriculum............................................................................................43 Deliver the Curriculum ............................................................................................................47 Evaluate the Training and Trainers and Revise .......................................................................48 This document is a research report submitted to the U.S. Department of Justice. This report has not been published by the Department. Opinions or points of view expressed are those of the author(s) and do not necessarily reflect the official position or policies of the U.S. Department of Justice. Chapter 5: Project Methodology Key Decision Processes for Site Selection ....................................................................................65 Overview of Methods ....................................................................................................................68 Chapter 6: Cross-site Comparisons and Findings Summary of the Training Evaluation Model's Applications .........................................................71 Needs Assessment....................................................................................................................71 Training Plan............................................................................................................................73 Develop and Test Curriculum..................................................................................................75 Pilot Test ..................................................................................................................................77 Trainer Selection......................................................................................................................78 Training Course Evaluation .....................................................................................................79 Conclusions....................................................................................................................................85 Recommendations and Lessons Learned: Tips for Evaluating and Improving Criminal Justice Training...............................................................................................................86 Costs of Training......................................................................................................................96 Chapter 7: National White Collar Crime Center's Foundations of Intelligence Analysis Training The National White Collar Crime Center ......................................................................................99