MEASURING WHAT MATTERS Using Assessment and Accountability to Improve Student Learning A STATEMENT BY THE RESEARCH AND POLICY COMMITTEE OF THE COMMITTEE FOR ECONOMIC DEVELOPMENT MEASURING WHAT MATTERS Using Assessment and Accountability to Improve Student Learning A STATEMENT BY THE RESEARCH AND POLICY COMMITTEE OF THE COMMITTEE FOR ECONOMIC DEVELOPMENT Library of Congress Cataloging-in-Publication Data Committee for Economic Development. Research and Policy Committee. Measuring what matters : using assessment and accountability to improve student learning / a statement by the Research and Policy Committee of the Committee for Economic Development p. cm. “December 15, 2000.” Includes bibliographical references. ISBN 0-87186-139-9 1. Educational tests and measurements — United States. 2. Educational accountability — United States. 3. School improvement programs — United States. I. Title. LB3051 .C632 2001 371.26'0973 — dc21 2001017186 First printing in bound-book form: 2001 Paperback: $15.00 Printed in the United States of America Design: Rowe Design Group COMMITTEE FOR ECONOMIC DEVELOPMENT 477 Madison Avenue, New York, N.Y. 10022 (212) 688-2063 2000 L Street, N.W., Suite 700, Washington, D.C. 20036 (202) 296-5860 www.ced.org CONTENTS RESPONSIBILITY FOR CED STATEMENTS ON NATIONAL POLICY iv PURPOSE OF THIS STATEMENT viii CHAPTER 1: INTRODUCTION AND SUMMARY 1 CHAPTER 2: MEASURING STUDENT ACHIEVEMENT 5 Issues in designing direct measures of student learning 5 Setting standards for learning 5 Designing assessments 7 Reporting scores 9 Challenges to measurement practices 10 Appropriate test use 10 Testing all students 12 Cost implications 13 Limitations of tests as measures of student learning 14 CHAPTER 3: USING ACHIEVEMENT MEASURES TO IMPROVE TEACHING AND LEARNING 15 Measurement that enhances instruction 15 Teacher capacity 17 CHAPTER 4: USING ACHIEVEMENT MEASURES TO HOLD STUDENTS AND EDUCATORS ACCOUNTABLE 19 Using performance measures to reward and sanction students 21 Using multiple measures 21 Availability of appropriate instruction and assistance 23 Motivating students to do their best 24 Using performance measures to reward and sanction educators 25 Accountability for individual teachers 25 Accountability for schools 28 Guidelines for good practice 30 CHAPTER 5: MONITORING EDUCATIONAL PROGRESS AND THE MEASUREMENT SYSTEM ITSELF 32 National and international assessments 32 Reporting to the public 33 Monitoring the measurement system 35 ENDNOTES 37 REFERENCES 39 MEMORANDA OF COMMENT, RESERVATION, OR DISSENT 42 OBJECTIVES OF THE COMMITTEE FOR ECONOMIC DEVELOPMENT 45 iii RESPONSIBILITY FOR CED STATEMENTS ON NATIONAL POLICY The Committee for Economic Develop- ing specific legislative proposals; its purpose is ment is an independent research and policy to urge careful consideration of the objectives organization of some 250 business leaders set forth in this statement and of the best means and educators. CED is nonprofit, nonparti- of accomplishing those objectives. san, and nonpolitical. Its purpose is to pro- Each statement is preceded by extensive pose policies that bring about steady eco- discussions, meetings, and exchange of memo- nomic growth at high employment and randa. The research is undertaken by a sub- reasonably stable prices, increased productiv- committee, assisted by advisors chosen for their ity and living standards, greater and more competence in the field under study. equal opportunity for every citizen, and an The full Research and Policy Committee improved quality of life for all. participates in the drafting of recommenda- All CED policy recommendations must tions. Likewise, the trustees on the drafting have the approval of trustees on the Research subcommittee vote to approve or disapprove a and Policy Committee. This committee is di- policy statement, and they share with the rected under the bylaws, which emphasize Research and Policy Committee the privilege that “all research is to be thoroughly objec- of submitting individual comments for publi- tive in character, and the approach in each cation. instance is to be from the standpoint of the general welfare and not from that of any The recommendations presented herein are special political or economic group.” The those of the trustee members of the Research and committee is aided by a Research Advisory Policy Committee and the responsible subcom- Board of leading social scientists and by a mittee. They are not necessarily endorsed by other small permanent professional staff. trustees or by non-trustee subcommittee members, The Research and Policy Committee does advisors, contributors, staff members, or others not attempt to pass judgment on any pend- associated with CED. iv RESEARCH AND POLICY COMMITTEE Co-Chairmen PATRICK W. GROSS, Founder and RONALD R. DAVENPORT, Chairman of ALONZO L. MCDONALD, Chairman Chairman, Executive Committee the Board and Chief Executive Officer American Management Systems, Inc. Sheridan Broadcasting Corporation Avenir Group, Inc. BRUCE K. MACLAURY, President JOHN DIEBOLD, Chairman NICHOLAS G. MOORE, Chairman Emeritus John Diebold Incorporated PricewaterhouseCoopers The Brookings Institution FRANK P. DOYLE, Retired Executive JOHN D. ONG, Chairman Emeritus Vice President The BFGoodrich Company Vice Chairmen GE STEFFEN E. PALKO, Vice Chairman IAN ARNOF, Retired Chairman T.J. DERMOT DUNPHY, Retired Bank One, Louisiana, N.A. and President Chairman Cross Timbers Oil Company ROY J. BOSTOCK, Chairman Sealed Air Corporation B/com3 Group, Inc. * CAROL J. PARRY, Retired Executive CHRISTOPHER D. EARL, Managing Vice President CLIFTON R. WHARTON, JR., Former Director The Chase Manhattan Corporation Chairman and Chief Executive Officer Perseus Capital LLC VICTOR A. PELSON, Senior Advisor TIAA-CREF W. D. EBERLE, Chairman Warburg Dillon Read LLC Manchester Associates, Ltd. PETER G. PETERSON, Chairman EDMUND B. FITZGERALD, Managing The Blackstone Group Director Woodmont Associates S. LAWRENCE PRENDERGAST, REX D. ADAMS, Dean Executive Vice President of Finance The Fuqua School of Business HARRY L. FREEMAN, Chair LaBranche & Co. The Mark Twain Institute Duke University NED REGAN, President ALAN BELZER, Retired President and RAYMOND V. GILMARTIN, Chairman, Baruch College Chief Operating Officer President and Chief Executive Officer Merck & Co., Inc. JAMES Q. RIORDAN AlliedSignal Inc. Quentin Partners Inc. PETER A. BENOLIEL, Chairman, BARBARA B. GROGAN, President Western Industrial Contractors MICHAEL I. ROTH, Chairman and Executive Committee Chief Executive Officer Quaker Chemical Corporation RICHARD W. HANSELMAN, Chairman The MONY Group Inc. Health Net Inc. FLETCHER L. BYROM, President and LANDON H. ROWLAND, Chairman, Chief Executive Officer RODERICK M. HILLS, President President and Chief Executive Officer MICASU Corporation Hills Enterprises, Ltd. Stillwell Financial Inc. DONALD R. CALDWELL, Chairman and MATINA S. HORNER, Executive GEORGE RUPP, President Chief Executive Officer Vice President Columbia University Cross Atlantic Capital Partners TIAA-CREF HENRY B. SCHACHT, Director and ROBERT B. CATELL, Chairman and Chief H.V. JONES, Office Managing Director Senior Advisor Executive Officer Korn/Ferry International, Inc. E.M. Warburg, Pincus & Co., Inc. LLC KeySpan Corporation EDWARD A. KANGAS, Chairman, ROCCO C. SICILIANO JOHN B. CAVE, Principal Retired Beverly Hills, California Avenir Group, Inc. Deloitte Touche Tohmatsu MATTHEW J. STOVER, President CAROLYN CHIN, Chief Executive Officer JOSEPH E. KASPUTYS, Chairman, edu.com Cebiz President and Chief Executive Officer Thomson Financial Primark ALAIR A. TOWNSEND, Publisher A. W. CLAUSEN, Retired Chairman and Crain's New York Business Chief Executive Officer CHARLES E.M. KOLB, President BankAmerica Corporation Committee for Economic Development ARNOLD R. WEBER, President Emeritus Northwestern University JOHN L. CLENDENIN, Retired Chairman ALLEN J. KROWE, Retired BellSouth Corporation Vice Chairman JOSH S. WESTON, Honorary Chairman Automatic Data Processing, Inc. GEORGE H. CONRADES, Chairman and Texaco Inc. Chief Executive Officer CHARLES R. LEE, Chairman and DOLORES D. WHARTON, Chairman and Akamai Technologies, Inc. Co-Chief Executive Officer Chief Executive Officer The Fund for Corporate Initiatives, Inc. KATHLEEN B. COOPER, Chief Economist Verizon Communications and Manager, Economics & Energy MARTIN B. ZIMMERMAN, Vice Division President, Governmental Affairs Exxon Mobil Corporation Ford Motor Company *Voted to approve the policy statement but submitted memoranda of comment, reservation, or dissent. See page 42. v SUBCOMMITTEE ON EDUCATION POLICY Co-Chairmen ROY J. BOSTOCK JEROME H. GROSSMAN HOWARD M. ROSENKRANTZ Chairman Chairman and Chief Executive Chief Executive Officer B/com3 Group, Inc. Officer Grey Flannel Auctions EDWARD B. RUST, JR. Lion Gate Management Corporation MICHAEL I. ROTH Chairman and Chief Executive Officer JUDITH H. HAMILTON Chairman and Chief Executive Officer State Farm Insurance Companies President and Chief Executive Officer The MONY Group Inc. Classroom Connect NEIL L. RUDENSTINE WILLIAM A. HASELTINE President Trustees Chairman and Chief Executive Officer Harvard University HENRY P. BECTON Human Genome Sciences, Inc. WILLIAM RUDER President and General Manager HEATHER R. HIGGINS President WGBH Educational Foundation President William Ruder Incorporated PETER A. BENOLIEL Randolph Foundation ALAN G. SPOON Chairman, Executive Committee DAVID KEARNS Managing General Partner Quaker Chemical Corporation Chairman Emeritus Polaris Venture Partners JON A. BOSCIA New American