AN EVALUATION OF THE CORRECTIONAL TRAINING PROGRAM DELIVERED BY THE CORRECTIONAL SERVICE OF CANADA > by Shevaun Corey A thesis submitted to the Faculty of Graduate and Postdoctoral Affairs in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Psychology Carleton University Ottawa, Canada ©2012 Shevaun Corey Library and Archives Bibliotheque et Canada Archives Canada Published Heritage Direction du 1+1 Branch Patrimoine de I'edition 395 Wellington Street 395, rue Wellington Ottawa ON K1A0N4 Ottawa ON K1A 0N4 Canada Canada Your file Votre reference ISBN: 978-0-494-93659-7 Our file Notre reference ISBN: 978-0-494-93659-7 NOTICE: AVIS: The author has granted a non­ L'auteur a accorde une licence non exclusive exclusive license allowing Library and permettant a la Bibliotheque et Archives Archives Canada to reproduce, Canada de reproduire, publier, archiver, publish, archive, preserve, conserve, sauvegarder, conserver, transmettre au public communicate to the public by par telecommunication ou par I'lnternet, preter, telecommunication or on the Internet, distribuer et vendre des theses partout dans le loan, distrbute and sell theses monde, a des fins commerciales ou autres, sur worldwide, for commercial or non­ support microforme, papier, electronique et/ou commercial purposes, in microform, autres formats. paper, electronic and/or any other formats. The author retains copyright L'auteur conserve la propriete du droit d'auteur ownership and moral rights in this et des droits moraux qui protege cette these. Ni thesis. Neither the thesis nor la these ni des extraits substantiels de celle-ci substantial extracts from it may be ne doivent etre imprimes ou autrement printed or otherwise reproduced reproduits sans son autorisation. without the author's permission. In compliance with the Canadian Conformement a la loi canadienne sur la Privacy Act some supporting forms protection de la vie privee, quelques may have been removed from this formulaires secondaires ont ete enleves de thesis. cette these. While these forms may be included Bien que ces formulaires aient inclus dans in the document page count, their la pagination, il n'y aura aucun contenu removal does not represent any loss manquant. of content from the thesis. Canada AN EVALUATION OF CTP ii Abstract Despite the need for training evaluation, many organizations do not adequately evaluate their training programs. When organizations do evaluate their training, they often rely on Kirkpatrick’s (1959a) four-level evaluation framework, which focuses on trainee reactions, learning, behavioural outcomes, and results for the organization. This is one of the most widely used evaluation frameworks despite the fact that it has many limitations, one of the most serious being its narrow focus on just a few training outcomes at each level of the framework. An expanded version of this framework was used to evaluate the Correctional Training Program (CTP), which attempts to provide correctional officers who might be hired by the Correctional Service of Canada (CSC) with the knowledge, skills, and attitudes necessary to effectively manage offenders within institutions. Three specific issues were examined: (1) Does CTP produce positive changes in trainees (e.g., with respect to domain-relevant knowledge, skills and attitudes)?; (2) Do the training outcomes measured at each level of the expanded framework correlate with one another?; and (3) Do barriers to training effectiveness (e.g., lack of supervisor support) moderate the relationships between the training levels? By analyzing the data from 102 CTP trainees, valuable information was obtained concerning the effectiveness of CTP with respect to a wide variety of trainee reactions, learning gains, and the degree to which knowledge and skills transferred to the job. However, the results from this study do not suggest that the training outcomes measured at each level of the expanded framework correlate with one another, nor do they suggest that the training effectiveness factors, as measured in the current study, significantly moderate training outcomes, with the exception of content validity and self-efficacy. The theoretical and practical implications AN EVALUATION OF CTP iii of these results are discussed, along with a discussion of the suitability of an expanded version of Kirkpatrick’s four-level evaluation framework. Recommendations for CSC as well as directions for future research are also suggested. AN EVALUATION OF CTP iv Acknowledgements I once read that a mentor is someone who sees more talent and ability within you than you see in yourself, and that they help bring it out in you. Craig, you have certainly been a mentor to me over the past eight years and I would like to thank you for your unwavering support and confidence in my academic pursuits. I have learned many things from you that I will take forward into my own career, such as your objectivity, your dedication to your students, and your pragmatism. I feel so very fortunate to have had the opportunity to work with you and learn from you. My graduate career has been an amazing experience in large part because of you. I would like to thank all of my committee members for their time and thoughtful insights - Drs. Patti Phillips, Ralph Serin, Shelley Brown,, and George Pollard. I would like to express my sincerest gratitude to the staff of the Learning & Development Branch and the Regional Staff Colleges of the Correctional Service of Canada for their assistance, support and encouragement, as well as all of the correctional officer participants. A very special thank you to the members of my evaluation team - Steve Loeb, Jean-Luc Chamaillard, Carl Jalbert, Julia Crosier, and Jacques Brien - your expertise, time and support have been invaluable. Also, funding from the Social Science and Humanities Research Council has helped to make this research possible. For the countless hours spent listening, collaborating with and encouraging me, I must thank my colleagues past and present of the Police Research Lab, as well as Natalie Pressburger, Etelle Bourassa, April McGrath, and Clare Wilkinson. I am grateful to my friends Erin Crawley and John Edmunds who took care of my boys, Jax and Loki, during AN EVALUATION OF CTP v my many trips for data collection. For their endless love and support, I have to thank my parents, who have always believed in me. I share this success with you both. Lastly, but certainly not least, to Marc-Antoine, whose love has kept me going over the past few years. Thank you for your encouragement and support, your patience, and for putting my needs ahead of your own. The boys thank you too for playing with them when I was too busy... Here’s to the future and to putting us first! “Success is not final, failure is not fatal: it is the courage to continue that counts.” Winston Churchill AN EVALUATION OF CTP vi Table of Contents Abstract..................................................................................................................................ii Acknowledgements .............................................................................................................. iv Table of Contents .................................................................................................................. vi List of Tables ........................................................................................................................ xi List of Figures ....................................................................................................................... xv List of Appendices ................................................................................................................ xvi Chapter 1. Introduction .................................................................................................... 1 The Focus of the Research ................................................................................. :................. 1 A Multi-Level Framework for Training Evaluation ............................................................4 A Useful Starting Point: Kirkpatrick’s Four-Level Framework .............................. 5 Correlations Between the Levels of Kirkpatrick’s Framework ............................... 8 Potential Reasons for the Lack of Significant Correlations ......................................10 The Need for a Multi-Dimensional, Multi-Level Training Evaluation Framework ......... 10 The Uni-dimensionality of Kirkpatrick’s Levels .......................................................11 Expanding the Levels: Making Kirkpatrick’s Framework More Psychologically Sound .......................................................................................................................... 12 Level 1 ................................................................................................................. 12 Level 2 ................................................................................................................. 14 Level 3 ................................................................................................................. 16 The Need to Consider Training Effectiveness Factors ........................................................19 So Why Does Training Effectiveness Matter? ..........................................................22 Holton’s Learning Transfer System Inventory ..........................................................23 AN EVALUATION OF CTP vii The Current Study................................................................................................................