Global Perspectives on Measuring Quality Proceedings of the 2010 Strategic Leaders Global Summit on Graduate Education September 13-15, 2010 Brisbane, Australia CounCil of GrAduAte SChoolS Global Perspectives on Measuring Quality Proceedings of the 2010 Strategic leaders Global Summit on Graduate education Managing Editor: Julia D. Kent This project was supported in part with funding from ProQuest and by the Australian government through the Department of Education, Employment and Workplace Relations (DEEWR). The views expressed here are those of the authors and do not necessarily represent the views of the funders. Copyright © 2011 Council of Graduate Schools, Washington, D.C. ALL RIGHTS RESERVED. No part of this work covered by the copyright herein may be reproduced or used in any form by any means—graphic, electronic, or mechanical, including photocopying, recording, taping, Web distribution, or information storage and retrieval systems—without the written permission of the Council of Graduate Schools, One Dupont Circle, NW, Suite 230, Washington, D.C. 20036-1173. ISBN-13: 978-1-933042-29-9 ISBN-10: 1-933042-29-X 10 9 8 7 6 5 4 3 2 1 10 09 08 TABLE OF CONTENTS foreword ................................................................................................... vii Acknowledgments ................................................................................... viii i. introduction: Measuring Quality in Graduate education and research training ............................................................................... 1 Debra W. Stewart, Council of Graduate Schools ii. national Contexts for Assessment: What are we measuring and why? .................................................................................. 5 Helene Marsh, James Cook University (Australia) Jeffery Gibeling, University of California, Davis (U.S.) Douglas Peers, York University (Canada) He Kebin, Tsinghua University (China) Narayana Jayaram, Tata Institute of Social Sciences (India) Kyung-Chan Min, Yonsei University (South Korea) iii. laying A Strong foundation for institutional Assessment ................................................................................................ 41 Benefi ts and Challenges of Quality Assessment ..............................................46 Kim D. Nguyen, HoChiMinh City University of Education (Vietnam) Surasak Watanesk, Chiang Mai University (Thailand) Communicating the Value of Assessment .........................................................53 Patrick Osmer, Ohio State University (U.S.) Administrative Challenges & Opportunities ...................................................56 Ursula Lehmkuhl, Freie Universität Berlin (Germany) Alan Dench, University of Western Australia, and Maxwell King, Monash University (Australia) iV. Supporting the development of research training environments .................................................................... 66 iii CONTENTS External Assessments of the Research Environment ........................................71 Akihiko Kawaguchi, National Institute of Academic Degrees and University Evaluation (Japan) Paul K.H. Tam, University of Hong Kong (Hong Kong) The Role of Global Rankings ...........................................................................82 Chao Hui Du, Shanghai Jiao Tong University (China) Jan Botha, Stellenbosch University (South Africa) University Assessments of the Research Environment .....................................92 Sang-Gi Paik, Chungnam National University (South Korea) Marie Carroll, University of Sydney, and Richard Russell, University of Adelaide (Australia) Admission, Retention and Completion in Doctoral Degree Programs ..........104 Charles Tustin, University of Otago (New Zealand) V. using Quality Measures to Support Program Content and design ..........................................................................................110 Outcomes of Student Learning and Research Training .................................116 Zhen Liang, Harbin Institute of Technology (China) Julia Kent, Council of Graduate Schools (U.S.) Mentoring and Supervision ............................................................................125 Barbara Evans, University of British Columbia (Canada) Gregor Coster, University of Auckland (New Zealand) Interdisciplinary Programs ............................................................................139 Zlatko Skrbis, University of Queensland, and Mandy Thomas, Australian National University (Australia) Karen DePauw, Virginia Polytechnic Institute and State University (U.S.) Vi. Skills, Competencies, and the Workforce ..................................... 150 Defining and Measuring Professional Skills .................................................156 Austin McLean, ProQuest (U.S.) Illah Sailah, Ministry of National Education (Indonesia) Linking Professional Training Programs to Workforce Needs ......................162 Laura Poole-Warren, University of New South Wales, and Richard Strugnell, University of Melbourne (Australia) iv CONTENTS Rose Alinda Alias, Malaysia Dean of Graduate Studies Council, Universiti Teknologi Malaysia (Malaysia) Career Pathways ............................................................................................172 Jianhua Yan, Zhejiang University (China) Allison Sekuler, McMaster University (Canada) Vii. Measurements without Borders? ................................................. 183 Tools and Methods for Assessing Quality Internationally .............................188 Jean Chambaz, University Pierre and Marie Curie (France) Maxwell King, Monash University (Australia) Promising Practices for Administering Quality Assessments ........................198 Tan Thiam Soon, National University of Singapore (Singapore) Assessing Quality in (Post)-Graduate International Collaborations ............202 Le Thi Kim Anh, Ministry of Education (Vietnam) Andrew Comrie, University of Arizona (U.S.) Viii. Conclusion: Guiding Principles for Measuring Quality .................................................................................................... 216 Debra W. Stewart, Council of Graduate Schools Appendices .............................................................................................. 218 A. Principles and Practices for Assessing the Quality of (Post)-Graduate Education and Research Training ........................................218 B. Supplementary Country Papers ..............................................................222 Illah Sailah (Indonesia) Akihiko Kawaguchi (Japan) Gregor Coster and Charles Tustin (New Zealand) Tan Thiam Soon (Singapore) Jan Botha (South Africa) Kim Anh le Thi and Kim D. Nguyen (Vietnam) C. Participant Biographies ..........................................................................247 Note: The names of co-authors who did not attend the summit appear at the top of the essay to which they contributed. v FOREWORD he 2010 Global Summit in Brisbane, Australia, “Measuring Quality in Graduate Education,” addressed a topic of growing Timportance in graduate education systems worldwide. Whereas in undergraduate education, the assessment of quality for the purposes of accountability and improvement has become a well-established practice, quality assessment at the graduate level, particularly in doctoral education, is a relatively new phenomenon. Increasingly, governments and other external stakeholders are expanding efforts to measure the quality of their graduate education systems along with the capacity of universities to serve national needs. Meanwhile, institutions are also developing strategies to enhance the quality of their programs and the preparation of graduate students to solve pressing global problems. The 2010 Global Summit provided an opportunity for a diverse group of graduate leaders to better understand the differences among their national quality assessment frameworks, share assessment methods and practices, and identify emerging best practices in this area. Global Perspectives on Measuring Quality highlights both the differences and areas of common ground that emerged at the 2010 Global Summit and aims to spark future thinking and discussion of this important topic. Debra W. Stewart President Council of Graduate Schools vii ACKNOWLEDGMENTS his volume refl ects the contributions of many collaborators, both individuals and groups, from around the world. For ensuring that the Tsummit refl ected a diverse range of national perspectives, I fi rst thank the international Steering Committee that oversaw the development of the summit agenda, and all of the 2010 summit delegates, who gave the agenda life through thoughtful papers and contributions to the discussion sessions. On behalf of the Council of Graduate Schools, I express the warmest thanks to our summit co-hosts in Australia, the Group of Eight (Go8) and the Deans and Directors of Graduate Studies (DDoGS), partners at every stage of the process. Michael Gallagher, Executive Director at Go8, committed vision and resources, along with the time of an exceptional staff: Susie Barrett, Jasonne Grabher, Meagan James, Jane Liang, Les Rymer, and Kerrie Thornton. As an ongoing contributor to the Summit, Maxwell King of DDoGS ensured that the Brisbane summit built on accomplishments of past years while also paving the way for future best practice exchange among international leaders in graduate education. Finally, I thank The University of Queensland and its Vice-Chancellor, Paul Greenfi eld, for providing the historic Brisbane Customs