Kappan Classic KAPPAN digital edition exclusive Inside the Black Box: Raising Standards Through Classroom Assessment Formative assessment is an essential component of classroom work and can raise student achievement. By Paul Black and Dylan Wiliam This article was originally published as “Inside the Black Raising the standards of learning that are achieved through school­ Box: Raising ing is an important national priority. In recent years, governments Standards throughout the world have been more and more vigorous in making Through Classroom changes in pursuit of this aim. National, state, and district standards; Assessment,” by target setting; enhanced programs for the external testing of students’ Paul Black and performance; surveys such as NAEP (National Assessment of Edu­ Dylan Wiliam. Phi cational Progress) and TIMSS (Third International Mathematics and Delta Kappan 80, Science Study); initiatives to improve school planning and manage­ no. 2 (October 1998): 139-144, ment; and more frequent and thorough inspection are all means to­ 146-148. ward the same end. But the sum of all these reforms has not added up to an effective policy because something is missing. Learning is driven by what teachers and pupils do in classrooms. Teachers have to manage complicated and demanding situations, channeling the personal, emotional, and social pressures of a group of 30 or more youngsters in order to help them learn immediately and become better learners in the future. Standards can be raised only if teachers can tackle this task more effectively. What is missing from the efforts alluded to above is any direct help with this task. This fact Deepen your was recognized in the TIMSS video study: “A focus on standards and understanding of accountability that ignores the processes of teaching and learning in this article with classrooms will not provide the direction that teachers need in their questions and quest to improve” (Stigler and Hiebert 1997). activities from this In terms of systems engineering, present policies in the U.S. and in month’s Kappan many other countries seem to treat the classroom as a black box. Certain inputs from the outside —pupils, Professional teachers, other resources, management rules and requirements, parental anxieties, standards, tests with high Development Discussion Guide stakes, and so on — are fed into the box. Some outputs are supposed to follow: pupils who are more knowl­ by Lois Brown edgeable and competent, better test results, teachers who are reasonably satisfied, and so on. But what is hap­ Easton, free to pening inside the box? How can anyone be sure that a particular set of new inputs will produce better outputs members in the if we don’t at least study what happens inside? And why is it that most of the reform initiatives mentioned in digital edition at the first paragraph are not aimed at giving direct help and support to the work of teachers in classrooms? kappanmagazine .org. PAUL BLACK is emeritus professor of science education at King’s College London, United Kingdom. DYLAN WILIAM is deputy director of the Institute of Education, University of London, United Kingdom. kappanmagazine.org V92 N1 Kappan 81 The answer usually given is that it is up to teach­ 3. Is there evidence about how to improve ers: They have to make the inside work better. This formative assessment? answer is not good enough, for two reasons. First, it is at least possible that some changes in the inputs In setting out to answer these questions, we have may be counterproductive and make it harder for conducted an extensive survey of the research liter­ teachers to raise standards. Second, it seems strange, ature. We have checked through many books and even unfair, to leave the most difficult piece of the through the past nine years’ worth of issues of more standards-raising puzzle entirely to teachers. If there than 160 journals, and we have studied earlier re­ are ways in which policy makers and others can give views of research. This process yielded about 580 ar­ ticles or chapters to study. We prepared a lengthy re­ view, using material from 250 of these sources, that There is a wealth of research evidence that the has been published in a special issue of the journal everyday practice of assessment in classrooms is beset Assessment in Education, together with comments on with problems and shortcomings. our work by leading educational experts from Aus­ tralia, Switzerland, Hong Kong, Lesotho, and the direct help and support to the everyday classroom U.S. (Black and Wiliam 1998). task of achieving better learning, then surely these The conclusion we have reached from our re­ ways ought to be pursued vigorously. search review is that the answer to each of the three This article is about the inside of the black box. questions above is clearly yes. In the three main sec­ We focus on one aspect of teaching: formative as­ tions below, we outline the nature and force of the sessment. But we will show that this feature is at the evidence that justifies this conclusion. However, be­ heart of effective teaching. cause we are presenting a summary here, our text will appear strong on assertions and weak on the details THE ARGUMENT of their justification. We maintain that these asser­ We start from the self-evident proposition that tions are backed by evidence and that this backing is teaching and learning must be interactive. Teachers set out in full detail in the lengthy review on which need to know about their pupils’ progress and diffi­ this article is founded. culties with learning so that they can adapt their own We believe that the three sections below establish work to meet pupils’ needs — needs that are often a strong case that governments, their agencies, unpredictable and that vary from one pupil to an­ school authorities, and the teaching profession other. Teachers can find out what they need to know should study very carefully whether they are seri­ in a variety of ways, including observation and dis­ ously interested in raising standards in education. cussion in the classroom and the reading of pupils’ However, we also acknowledge widespread evidence written work. that fundamental change in education can be We use the general term assessment to refer to all achieved only slowly — through programs of pro­ those activities undertaken by teachers — and by fessional development that build on existing good their students in assessing themselves — that pro­ practice. Thus we do not conclude that formative as­ vide information to be used as feedback to modify sessment is yet another “magic bullet” for education. teaching and learning activities. Such assessment be­ The issues involved are too complex and too closely comes formative assessment when the evidence is ac­ linked to both the difficulties of classroom practice tually used to adapt the teaching to meet student and the beliefs that drive public policy. In a final sec­ needs. (There is no internationally agreed-upon tion, we confront this complexity and try to sketch term here. “Classroom evaluation,” “classroom as­ out a strategy for acting on our evidence. sessment,” “internal assessment,” “instructional as­ sessment,” and “student assessment” have been used DOES IMPROVING FORMATIVE ASSESSMENT by different authors, and some of these terms have RAISE STANDARDS? different meanings in different texts.) A research review published in 1986, concentrat­ There is nothing new about any of this. All teach­ ing primarily on classroom assessment work for chil­ ers make assessments in every class they teach. But dren with mild handicaps, surveyed a large number there are three important questions about this of innovations, from which 23 were selected (Fuchs process that we seek to answer: and Fuchs 1986). Those chosen satisfied the condi­ tion that quantitative evidence of learning gains was 1. Is there evidence that improving formative obtained, both for those involved in the innovation assessment raises standards? and for a similar group not so involved. Since then, 2. Is there evidence that there is room for many more papers have been published describing improvement? similarly careful quantitative experiments. Our own 82 Kappan September 2010 kappanmagazine.org review has selected at least 20 more studies. (The back between those taught and the teacher, ways that number depends on how rigorous a set of selection will require significant changes in classroom prac­ criteria are applied.) All these studies show that in­ tice. novations that include strengthening the practice of Underlying the various approaches are assump­ formative assessment produce significant and often tions about what makes for effective learning — in substantial learning gains. These studies range over particular the assumption that students have to be age groups from 5-year-olds to university under­ actively involved. graduates, across several school subjects, and over several countries. For research purposes, learning gains of this type What is needed is a culture of success, backed by a are measured by comparing the average improve­ belief that all pupils can achieve. ments in the test scores of pupils involved in an in­ novation with the range of scores that are found for For assessment to function formatively, the re­ typical groups of pupils on these same tests. The ra­ sults have to be used to adjust teaching and learning; tio of the former divided by the latter is known as thus a significant aspect of any program will be the the effect size. Typical effect sizes of the formative ways in which teachers make these adjustments. assessment experiments were between 0.4 and 0.7. The ways in which assessment can affect the mo­ These effect sizes are larger than most of those found tivation and self-esteem of pupils and the benefits of for educational interventions.