Alternative Assessment Programs: What Are the True Costs?
Total Page:16
File Type:pdf, Size:1020Kb
Alternative Assessment Programs: What Are the True Costs? An Analysis of the Total Costs of Assessment in Kentucky and Vermont CSE Technical Report 4411 Lawrence O. Picus and Alisha Tralli CRESST/University of Southern California February 1998 National Center for Research on Evaluation, Standards, and Student Testing (CRESST) Center for the Study of Evaluation (CSE) Graduate School of Education & Information Studies University of California, Los Angeles Los Angeles, CA 90024-6511 (310) 206-1532 1 Because of a revision to this report, please use the February 1998 and not the April 1997 publication date for citation purposes. Copyright © 1997 The Regents of the University of California The work reported herein was supported under the Educational Research and Development Center Program cooperative agreement R117G10027 and CFDA Catalog number 84.117G as administered by the Office of Educational Research and Improvement, U.S. Department of Education. The findings and opinions expressed in this report do not reflect the policies of the Office of Educational Research and Improvement or the U.S. Department of Education. ALTERNATIVE ASSESSMENT PROGRAMS: WHAT ARE THE TRUE COSTS? AN ANALYSIS OF THE TOTAL COSTS OF ASSESSMENT IN KENTUCKY AND VERMONT Lawrence O. Picus and Alisha Tralli CRESST/University of Southern California INTRODUCTION At the March 1996 education summit in New York, President Clinton, the nationÕs governors, and a select group of chief executive officers from large companies agreed that a critical component of education reform was the need to hold schools accountable for student learning. Although the exact methods for doing this were not specified, it seems certain that a major component of this effort will include some form of assessment. While it is doubtful that we will see a national standardized test, the importance of assessment programs will continue to grow in the foreseeable future. Although generally not asked, an important question is how much will this effort cost? On the surface, the costs of assessment appear relatively low. Picus, Tralli, and Tacheny (1995) point out that a simple analysis of state level expenditures for assessment in Kentucky and North Carolina shows assessment represents less than 1% of total K-12 expenditures in each state. Yet, as Picus (1994) suggests, this analysis may mask the true cost of assessment programs. Two factors are at the root of this suggestion. First, new assessment techniques including performance assessments and the development of more short answer and essay type questions make scoring of these tests more complex and labor intensive than traditional multiple choice tests. Second, even when the full costs of newer assessment instruments are calculated, they often ignore the ÒeconomicÓ costs of that assessment to the school systems across a state. For example, states do not reimburse schools for the time it takes to administer these tests, nor is the time local school officials spend preparing for the tests, administering the tests, and using the results included in the cost estimates of assessment 1 programs. Yet as Haney, Madaus, and Lyons (1993) point out, these costs are not only substantial, they probably represent the bulk of the costs of any testing system. How important are issues of cost? It is difficult to say, but one of the many, criticisms levied against CaliforniaÕs CLAS (California Learning Assessment System) tests before they were discontinued after only two years, was concern over the high cost. Yet the state appropriation of approximately $32 million represented less than two-tenths of 1% of the $26 billion in total outlays for K-12 education during the last year of the CLAS assessment. The purpose of this report is to provide a first detailed analysis of the ÒeconomicÓ or opportunity costs of the testing systems in two states, Kentucky and Vermont. Using a framework developed by Picus (1994), this study looked closely at the amount of time local school officials spent supporting the assessment programs in their respective states in 1995-96, and estimates the value of that effort. As the results show, when the full ÒeconomicÓ costs of an assessment system are estimated, the costs of assessment programs are considerably higher than they appear when only state level appropriations are considered. The next section of this paper provides additional background on testing in general and on cost issues specifically. This is followed by a description of our methodology. Next, individual sections on Kentucky and Vermont provide estimates of the costs of those statesÕ assessment programs. For Vermont, both district and state data are provided. For Kentucky, only district level estimates are provided as the state level assessment costs are detailed in our earlier work (Picus, Tralli, & Tacheny, 1995). In those sections, detailed analyses of the actual costs and the ÒeconomicÓ costs of assessment are provided. Finally, the last section of this document provides some conclusions and policy recommendations based on our findings. BACKGROUND Tests are being widely touted as instruments of educational reform and renewal. State minimum competency testing has increased dramatically over the last several decades from one program in 1972 to 34 programs in 1985 (Haney, Madaus, & Lyons, 1993). Many states have been shifting from traditional, 2 multiple-choice testing formats to alternative forms of student assessment. These new assessment methods include performance assessment, authentic testing, and portfolio review. Critics of traditional multiple-choice assessments see this testing method as part of the cause of low educational standards because multiple-choice testing reinforces rote learning rather than complex thinking and active learning (Nickerson, 1989; Wiggins, 1991). Wiggins (1992) argues that performance assessment is a better method of determining a studentÕs higher order thinking skills than the use of traditional multiple-choice tests. Higher order thinking is often nonalgorithmic, complex, involves the application of multiple criteria and uncertainty, and potentially has multiple solutions. Some research documents the negative effects of standardized testing on both teaching and learning (Shepard, 1991; Smith, 1991). In order to assess more sophisticated cognitive skills, many researchers promote the use of alternative assessment methods as a means of educational reform (Barone, 1991; Krechevsky, 1991; Wiggins, 1989). Surveys show that teachers spend considerable time teaching to test objectives, test-taking skills, and even specific test items (Haney, Madaus, & Lyons, 1993). For this reason, it is suggested that the tests become Òworth teaching toÓ (Wiggins, 1992). Tests may be used as instruments to improve teaching and learning through focusing instruction on areas deemed important, such has higher-order thinking skills. National and state projects have been implemented towards these goals. The New Standards Project, a joint effort of the National Center on Education and Economy and the University of PittsburghÕs Learning Research and Development Center, is a nationwide assessment project which includes 18 states and six school districts, and was begun in 1991. This program has received a lot of attention with its focus on assessment through performance tasks, projects, and portfolios of student work. The aim of the program is to make thinking, problem solving, and communication skills ÒcountÓ by creating and fostering assessments designed to elicit them (Viadero, 1994a). The National Council on Education Standards and Testing (NCEST), a commission convened at the national level, endorses an assessment system that consists of multiple methods of measuring student progress (Koretz, Madaus, Haertel, & Beaton, 1992). One objective of Goals 2000, an act legislated by Congress in 1994, is for students to Òdemonstrate the ability to reason, solve problems, apply knowledge, and write and communicate effectivelyÓ (National Education Goals Panel, 1994, 3 p.Ê9). These aims are similar to those of many states that have implemented new assessment methods. Over the past several years, numerous state level programs have been implemented that reflect more ÒauthenticÓ testing practices. KentuckyÕs assessment system was developed to Òelicit authentic performances through which students can accurately demonstrate their ability to develop, understand, and use knowledgeÓ (Foster, 1991, p. 35). The California Learning Assessment System (CLAS) included authentic, performance-based assessment methods in order to provide a more Òin-depth pictureÓ of student abilities (California Department of Education, 1993). The North Carolina Standard Course of Study (SCS), the state-adopted curriculum, includes portfolio assessment in the first and second grades, as well as open-ended testing in the later grades. Vermont has also implemented a portfolio program through voluntary participation. State education agencies are increasingly seeking alternative assessment strategies. Despite the importance placed on assessment systems, it is surprising how small a portion of state education funding is devoted to the development, implementation, and evaluation of student assessments. As Picus (1994) suggests, simple analysis of state level expenditures for assessment programs may mask the true cost of these assessment programs, both the newer forms of assessment and more traditional multiple-choice methods. Cost information is vital to solid policy making. Administrators and policy makers