Analytic Issues in the Assessment of Student Achievement the Charles Sumner School
Total Page:16
File Type:pdf, Size:1020Kb
Analytic Issues in the Assessment of Analytic Issues in the Assessment of Student Achievement in the Assessment Issues Analytic Student Achievement Proceedings from a Research Seminar Jointly Sponsored by National Center for Education Statistics National Institute on Student Achievement, Curriculum and Assessment and RAND U.S. Department of Education NCES 2000-050 The Charles Sumner School More than any other school founded after the Civil War, the Charles Sumner School served as the cornerstone for the development of educa- tional opportunities for black citizens in the District of Columbia. The sig- nificance assigned to its design and construction was indicated by the selec- tion of Adolph Cluss as architect for the new building. In 1869, Cluss had completed the Benjamin Franklin School; in 1872, he completed Sumner School; and in 1873, he won a medal for “Progress in Education and School Architecture” for the City of Washington at the International Exposition in Vienna, Austria. Dedicated on September 2, 1872, the new school was named in honor of United States Senator Charles Sumner of Massachusetts, who ranked alongside Abraham Lincoln and Thaddeus Stevens in leading the struggle for abolition, integration, and nondiscrimination. Upon opening, the Sumner building housed eight primary and grammar schools, as well as the executive offices of the Superintendent and Board of Trustees of the Colored Schools of Washington and Georgetown. The building also housed a secondary school, with the first high school graduation for black students held in 1877. The school also offered health clinics and adult education night classes. A recipient of major national and local awards for excellence in restora- tion, Sumner School currently houses a museum, an archival library, and other cultural programs that focus on the history of public education in the District of Columbia. Note: Cover photographs of the Charles Sumner School by David W. Grissmer Analytic Issues in the Assessment of Student Achievement David W. Grissmer and J. Michael Ross Editors Proceedings from a research seminar jointly sponsored by National Center for Education Statistics National Institute on Student Achievement, Curriculum, and Assessment RAND U.S. Department of Education Richard W. Riley Secretary Office of Educational Research and Improvement C. Kent McGuire Assistant Secretary National Center for Education Statistics Gary W. Phillips Acting Commissioner The National Center for Education Statistics (NCES) is the primary federal entity for collecting, analyzing, and reporting data related to education in the United States and other nations. It fulfills a congressional mandate to collect, collate, analyze, and report full and complete statistics on the condition of education in the United States; conduct and publish reports and specialized analyses of the meaning and significance of such statistics; assist state and local education agencies in improving their statistical systems; and review and report on education activities in foreign countries. NCES activities are designed to address high priority education data needs; provide consistent, reliable, complete, and accurate indicators of education status and trends; and report timely, useful, and high quality data to the U.S. Department of Education, the Congress, the states, other education policymakers, practitioners, data users, and the general public. We strive to make our products available in a variety of formats and in language that is appropriate to a variety of audiences. You, as our customer, are the best judge of our success in communicating information effectively. If you have any comments or sugges- tions about this or any other NCES product or report, we would like to hear from you. Please direct your comments to: National Center for Education Statistics Office of Educational Research and Improvement U.S. Department of Education 1990 K Street, NW Washington, DC 20006-5574 June 2000 The NCES World Wide Web Home Page is http://nces.ed.gov. Suggested Citation: U.S. Department of Education. National Center for Education Statis- tics. Analytic Issues in the Assessment of Student Achievement, NCES 2000–050 by David W. Grissmer and J. Michael Ross. Washingon, DC: 2000. For ordering information on this report, write: Contact: U.S. Department of Education David Grissmer ED Pubs (703) 413–1100 ext. 5310 P.O. Box 1398 J. Michael Ross Jessup, MD 20794-1398 (202) 502–7443 Or call toll free 1–877–4ED–Pubs. iii Dedication Dedicated to David W. Stevenson (1951–1998) Senior Advisor to the Acting Deputy Secretary of Education, 1993–98 This book is dedicated to the memory of David W. Stevenson. His under- standing of the interplay between basic research and education policy facilitated the development of this research seminar. From his early days in the sociology program at Yale, David began to develop a discipline-specific understanding of the structural factors mediating social change. As he became more involved in controversial policy issues, he saw the necessity for more definitive empiri- cal evidence in their resolution. In the continual efforts of the research and policy communities, David’s perspective will continue to enrich conversations about the direction of and appropriate methodologies for education reform. We acknowledge, with this dedication, his memorable accomplishments and our appreciation for his influence on this research seminar. v Table of Contents Foreword .............................................................................. ix Peggy G. Carr Acknowledgments ............................................................... xi David W. Grissmer and J. Michael Ross, Editors Introduction: Toward Heuristic Models of Student Outcomes and More Effective Policy Interventions............ xv C. Kent McGuire SECTION I. USING EXPERIMENTS AND STATE-LEVEL DATA TO ASSESS STUDENT ACHIEVEMENT Synthesizing Results from the NAEP Trial State Assessment ................................................................... 3 Stephen W. Raudenbush Moving Educational Research Toward Scientific Consensus ........................................................... 43 David W. Grissmer and Ann Flanagan Response: Guidance for Future Directions in Improving the Use of NAEP Data ...................................... 91 Sylvia T. Johnson vi Table of Contents SECTION II. USING LONGITUDINAL DATA TO ASSESS STUDENT ACHIEVEMENT Understanding Ethnic Differences in Academic Achievement: Empirical Lessons from National Data ...... 103 Meredith Phillips Certification Test Scores, Teacher Quality, and Student Achievement ........................................................ 133 Ronald F. Ferguson with Jordana Brown Response: Two Studies of Academic Achievement .......... 157 Robert M. Hauser SECTION III. RELATING FAMILY AND SCHOOLING CHARACTERISTICS TO ACADEMIC ACHIEVEMENT Improving Longitudinal Data on Student Achievement: Some Lessons from Recent Research Using NELS:88..... 169 Dominic J. Brewer and Dan D. Goldhaber School-level Correlates of Reading and Mathematics Achievement in Public Schools ........................................ 189 Donald McLaughlin and Gili Drori Response: Opportunities for Design Changes .................. 237 Valerie E. Lee Table of Contents vii SECTION IV. POLICY PERSPECTIVES AND CONCLUDING COMMENTARY Assessment Trends in a Contemporary Policy Context................................................................... 249 Marshall S. Smith Educational Research and Educational Policy: An Historical Perspective.................................................. 279 Christopher Jencks Improving Research and Data Collection on Student Achievement ........................................................ 299 Brenda J. Turnbull, David W. Grissmer, and J. Michael Ross SECTION V. APPENDIX Seminar Attendees ............................................................ 319 ix Foreword Peggy G. Carr Associate Commissioner Assessment Division National Center for Education Statistics In November 1998, a group of outstanding researchers and scholars gath- ered at the Charles Sumner School in Washington, DC to explore methodological issues related to the measurement of student achievement. Within this broad topic, the research seminar also focused more specifically on the sharing of perspectives related to the black-white test score gap. This sharing enabled the participants to compare their analyses and findings and to recommend improvements in data collection and analysis to the National Cen- ter for Education Statistics (NCES). Thus, eventually this collegial exchange promises to improve the utility of NCES data sets for policymakers in their efforts to ensure both excellence and equity in American education. Seeking deeper explanations of the test score gap is a critical first step in the process of assessing student achievement more accurately. Toward that end, the seminar demonstrated the need for NCES to pursue more aggressively the development of concepts and methodologies that allow independent analysts to unravel the causes of such gaps. Such an “unraveling” requires closer examina- tion of the complex interrelationships among resource factors, home and schooling influences, family configurations, and achievement outcomes. Further, NCES needs to place both cross-sectional and longitudinal data in a broader framework and to explicate our findings within diverse social contexts in richer detail. The work of the Assessment Division in NCES, in particular, will benefit from the development of more explicit