Evaluation of the US Army Asymmetric Warfare Adaptive Leader Program
Total Page:16
File Type:pdf, Size:1020Kb
Innovative Leader Development Evaluation of the U.S. Army Asymmetric Warfare Adaptive Leader Program Susan G. Straus, Michael G. Shanley, Carra S. Sims, Bryan W. Hallmark, Anna Rosefsky Saavedra, Stoney Trent, Sean Duggan C O R P O R A T I O N ELECTRONIC COPIES OF RAND RESEARCH ARE PROVIDED FOR PERSONAL USE; POSTING TO A NONRAND WEBSITE IS PROHIBITED. THIS PUBLICATION IS AVAILABLE FOR LINKING OR FREE DOWNLOAD AT www.rand.org ARROYO CENTER Innovative Leader Development Evaluation of the U.S. Army Asymmetric Warfare Adaptive Leader Program Susan G. Straus, Michael G. Shanley, Carra S. Sims, Bryan W. Hallmark, Anna Rosefsky Saavedra, Stoney Trent, Sean Duggan Prepared for the United States Army Approved for public release; distribution unlimited For more information on this publication, visit www.rand.org/t/rr504 Library of Congress Cataloging-in-Publication Data is available for this publication. ISBN 978-0-8330-8749-2 Published by the RAND Corporation, Santa Monica, Calif. © Copyright 2014 RAND Corporation R® is a registered trademark. Cover photo by Lt. Col. Sonise Lumbaca. Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions.html. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. Support RAND Make a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org Preface In this project, “Evaluation of U.S. Army Asymmetric Warfare Adap- tive Leader Program (AWALP),” the U.S. Army Training and Doctrine Command (TRADOC) asked RAND Arroyo Center to systemati- cally evaluate the effectiveness of AWALP and to design a set of instru- ments, tools, and protocols to foster ongoing assessment and improve- ment of AWALP and other courses or events that include adaptability training. This report describes the study design and results based on the collection of data from three AWALP classes. Using a pretest– posttest study design, the evaluation addresses multiple outcomes, including improvement in attitudes toward adaptability, cognitive learning, and adaptability behaviors; reactions to the course; and appli- cation of adaptability principles once graduates return to their units. The report will be of interest to those seeking to measure adaptability and other intangible skills in institutional and unit training environments. This research was sponsored by TRADOC and was conducted within the RAND Arroyo Center’s Manpower and Training Program. RAND Arroyo Center, part of the RAND Corporation, is a feder- ally funded research and development center sponsored by the United States Army. The Project Unique Identification Code (PUIC) for the project that produced this document is RAN126497. Correspondence regarding this report should be addressed to Susan Straus ([email protected]) or Michael Shanley (mikes@rand. org). For more information on the RAND Arroyo Center, contact the Director of Operations, Marcy Agmon (telephone 310-393-0411, iii iv Innovative Leader Development extension 6419; FAX 310-451-6952; e-mail Marcy_Agmon@rand. org); or visit Arroyo’s website at http://www.rand.org/ard/. Contents Preface ............................................................................. iii Figures ............................................................................. ix Tables .............................................................................. xi Summary .........................................................................xiii Acknowledgments ............................................................xxvii Abbreviations ...................................................................xxix CHaptER ONE Introduction ....................................................................... 1 Background ......................................................................... 1 The Need for Adaptability ...................................................... 1 Definition and Dimensions of Adaptability .................................. 2 AWALP Structure and Content ................................................ 4 Objectives ........................................................................... 8 Organization of This Report ...................................................... 9 CHaptER TWO General Approach and Study Design ........................................11 General Approach .................................................................11 Underlying Model of Training Evaluation and Effectiveness..............11 Research Literature Relevant to Measuring of AWALP Effectiveness ....12 Other Methodological Considerations .........................................18 Challenges to Measuring Adaptability .......................................18 Addressing Challenges to Measuring Adaptability ........................ 20 Study Design ......................................................................21 Participants ......................................................................21 v vi Innovative Leader Development Methods, Measures, and Procedures ........................................ 24 CHaptER THREE Reactions Toward AWALP and Attitudes Toward Adaptability ........25 Reactions to AWALP .............................................................25 Method ...........................................................................25 Results ............................................................................25 Summary of Reactions to Training ...........................................33 Attitudes Toward Adaptive Performance .......................................33 Method ...........................................................................33 Results ............................................................................35 Conclusions ....................................................................... 46 CHaptER FOUR Results: Learning ................................................................49 Knowledge of Course Concepts .................................................49 Method ...........................................................................49 Results ........................................................................... 50 Peer Evaluations ...................................................................51 Method ...........................................................................51 Results ............................................................................52 Student and Guide Ratings of Teams ......................................... 54 Method ...........................................................................55 Results for Requirements ......................................................57 Results for Performance ....................................................... 60 Conclusions ....................................................................... 64 CHaptER FIVE Results: Application of AWALP Principles on the Job and Longer-Term Attitudes Toward AWALP ...............................67 Purpose of Interviews .............................................................67 Application of AWALP Principles and Attitudes Three Months Post- AWALP ....................................................................... 68 Method and Response Rates ................................................. 68 Results: Report of Changes in Work Activities .............................69 Contents vii Results: Use of Outcomes-Based Strategies for Training, Delegating, and Mentoring ...............................................74 Results: Who Is Most Affected by Graduates’ Changes ....................75 Results: Changes in Attitudes .................................................76 Application of AWALP Principles and Attitudes Six Months Post- AWALP ........................................................................78 Method ...........................................................................78 Results ............................................................................78 Interviews of Supervisors .........................................................79 Method ...........................................................................79 Results ........................................................................... 80 Conclusions ....................................................................... 80 CHaptER SIX Summary, Conclusions, and Recommendations ...........................83 Summary of Key Findings and Conclusions from AWALP Evaluation ....83 Reactions .........................................................................83 Changes in Learners ............................................................85 Application of AWALP Principles on the Job .............................. 86 Improving AWALP .............................................................