A Strategic Governance Review for Multi-Organizational Systems of Education, Training, and Professional Development
Total Page:16
File Type:pdf, Size:1020Kb
National Defense Research Institute A Strategic Governance Review for Multi-organizational Systems of Education, Training, and Professional Development Glenn A. Daley Dina G. Levy Tessa Kaganoff Catherine A. Augustine Roger Benjamin Tora K. Bikson Susan M. Gates Joy S. Moini R Prepared for the Office of the Secretary of Defense Approved for public release; distribution unlimited The research described in this report was sponsored by the Office of the Secretary of Defense (OSD). The research was conducted in RAND’s National Defense Research Institute, a federally funded research and development center supported by the OSD, the Joint Staff, the unified commands, and the defense agencies under Contract DASW01-01-C-0004. Library of Congress Cataloging-in-Publication Data A strategic governance review for multi-organizational systems of education, training, and professional development / Glenn A. Daley ... [et al.]. p. cm. “MR-1560.” Includes bibliographical references. ISBN 0-8330-3267-4 1. United States. Dept. of Defense—Reorganization. 2. United States. Dept. of Defense—Employees—Training of. 3. United States. Dept. of Defense— Employees—Education (Higher) 4. United States. Dept. of Defense—Employees— Continuing education. I. Daley, Glenn A., 1953– UB193 .S75 2003 355.6'1—dc21 2002031848 RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND® is a registered trademark. RAND’s publications do not necessarily reflect the opinions or policies of its research sponsors. © Copyright 2003 RAND All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from RAND. Published 2003 by RAND 1700 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138 1200 South Hayes Street, Arlington, VA 22202-5050 201 North Craig Street, Suite 202, Pittsburgh, PA 15213-1516 RAND URL: http://www.rand.org/ To order RAND documents or to obtain additional information, contact Distribution Services: Telephone: (310) 451-7002; Fax: (310) 451-6915; Email: [email protected] PREFACE As one of the largest employers in the country, the Department of Defense (DoD) seeks to maintain a high-quality civilian workforce in the face of shifting organizational priorities as well as technological and social changes. Like most employers, DoD offers its employees opportunities for education, training, and professional development (ET&D) through both DoD and non-DoD providers. This network of ET&D providers and users constitutes a multi-organizational, multi- stakeholder system that is at present loosely coordinated and in- completely understood. Because of the division of financial respon- sibility, command authority, and workforce planning within DoD, the effectiveness of the system depends on its capacity for gover- nance of the organizations that provide the ET&D needed. At the request of the DoD Office of the Chancellor for Education and Professional Development, RAND undertook a study to develop tools for describing and evaluating external governance in such a multi- organizational system of ET&D. This report describes an approach for accomplishing those aims. The approach described here consists of two major tasks: a structural analysis that maps governance ar- rangements and an implementation audit that assesses effectiveness based on operational evidence. The framework and taxonomy pre- sented here were developed based on reviews of relevant literature and limited mapping exercises, but not through field audits or as- sessment exercises with organizations. We recommend pilot testing of the approach on a limited scale before systemwide implementa- tion. iii iv A Strategic Governance Review This report is one of a series of reports on ET&D prepared for the Chancellor’s office by RAND. A companion document (Augustine et al., 2002) develops an approach for assessing interorganizational influence in complex, nonhierarchical systems. Earlier reports ad- dress strategic planning approaches for organizations such as the Chancellor’s office (Levy et al., 2001a) and approaches for ensuring quality and productivity in systems of education (Gates et al., 2001). This report focuses on the DoD challenge but also attempts to ad- dress issues in common with other large systems of ET&D. As a re- sult, it may be of interest to higher education administrators and managers of corporate ET&D systems. This research was conducted for the U.S. Department of Defense Of- fice of the Chancellor for Education and Professional Development within the Forces and Resources Policy Center of RAND’s National Defense Research Institute, a federally funded research and devel- opment center sponsored by the Office of the Secretary of Defense, the Joint Staff, the unified commands, and the defense agencies. CONTENTS Preface ......................................... iii Figures ......................................... ix Summary ....................................... xi Acknowledgments................................. xix Acronyms ....................................... xxi Chapter One INTRODUCTION .............................. 1 Objective .................................... 1 Background .................................. 2 The Value of Reviewing External Governance .......... 3 Approach .................................... 6 Components of the Strategic Governance Review ..... 6 Development of the Review ..................... 8 Organization of the Report ....................... 10 Chapter Two THE DOD CHALLENGE ......................... 11 DoD Civilian Education, Training, and Development .... 12 Relevant Characteristics of the System ............... 14 Heterogeneity ............................... 14 Independent Evolution ........................ 15 Hybrid Form ................................ 15 The Chancellor’s Office .......................... 16 v vi A Strategic Governance Review Chapter Three THE STRUCTURAL ANALYSIS OF EXTERNAL GOVERNANCE ................................ 19 A Taxonomy .................................. 19 External Governance .......................... 20 Providers ................................... 21 Stakeholders ................................ 22 Systems .................................... 22 Special Stakeholder Types ...................... 23 Stakeholder Influence ......................... 28 Governance Functions ......................... 29 Domains ................................... 31 Mapping Governance Arrangements ................ 32 Sources of Information ........................ 33 Cataloging Arrangements ....................... 34 Drawing a Governance Map ..................... 35 Mapping Indirect Governance Relationships ........ 36 Additional Lessons from the Mapping Exercise ....... 40 Aligning Governance Arrangements ................. 41 Chapter Four THE IMPLEMENTATION AUDIT OF EXTERNAL GOVERNANCE ................................ 49 Overview..................................... 49 Defining the Assessment of Governance Effectiveness .. 51 Planning the Audit ............................. 53 The Purpose of the Audit ....................... 54 The Scope of the Audit ......................... 55 Selecting Criteria ............................. 56 Determining Who Will Conduct the Audit ........... 58 Gathering Operational Information ............... 59 Assessing the Effectiveness of External Governance ..... 60 Criterion 1. There Is Evidence that System Outcomes Are Being Defined, Measured, and Evaluated Relative to Goals .................................. 61 Criterion 2. There Is Evidence that Providers, Intermediaries, and Key Stakeholders Are Communicating Frequently and Transparently ..... 62 Criterion 3. There Is Evidence that Providers and Intermediaries Are Responsive to Key Stakeholder Guidance................................. 63 Contents vii Criterion 4. There Is Evidence that Key Stakeholders Are Being Satisfied .......................... 65 Reporting and Follow-Up ........................ 66 Responding to Preliminary Findings ............... 67 Generating Recommendations and Conclusions ...... 67 Implementation and Evaluation .................. 67 Reviewing the Audit ........................... 68 Chapter Five RECOMMENDATIONS AND CONCLUSIONS .......... 69 Bibliography ..................................... 73 FIGURES 1.1. A Strategic Review of External Governance ......... 6 3.1. A Multi-Organizational System of ET&D .......... 23 3.2. Examples of Stakeholders, Governance Functions, and Domains of ET&D ....................... 32 3.3. External Governance Map for a Single Provider— Direct Links ............................... 37 3.4. External Governance Map for a Single Provider —with Indirect Links and System Included .............. 38 ix SUMMARY INTRODUCTION AND BACKGROUND The Department of Defense (DoD) provides its civilian employees with numerous opportunities for education, training, and profes- sional development (ET&D). For this purpose, it operates, sponsors, or contracts with various providers of ET&D ranging from line agen- cies that offer short courses to major institutions of higher education. Thus, the provision of ET&D forms a loose and complex multi-orga- nizational network of providers, employers, sponsors, and other stakeholders, including licensing boards, boards of visitors and advi- sors, and accrediting bodies. We call the influencing functions as- sumed by those stakeholders external governance. Systems can be governed through