CHAPTER 1 Classroom Decision Making and Using

Home , Educational assessment

A01_BROO6495_07_SE_FM.inddA01_BROO6495_07_SE_FM.indd PagePage xiixii 11/21/1311/21/13 7:207:20 PMPM f451f451 ~~/Desktop/21:11:13/BROOKHART_PH01411_203/Desktop/21:11:13/BROOKHART_PH01411_203

Preface

As for the previous editions, the goal of Educa- the book is a comprehensive treatment of tradi- tional Assessment of Students, Seventh Edition, is to tional and alternative assessments, we give exam- help teachers and those in training to teach to ples, discuss the pros and cons, and give guidance improve their skills through better assessment of for crafting every assessment technique that we students. It focuses directly on the professional introduce. Research is cited that supports or practices of elementary and secondary school- refutes assessment and teaching practices. teachers. This edition features: The text prepares teachers and those in training to teach as professionals. We recognize that ■ A continued strong emphasis on classroom teachers’ experiences and judgments are neces- assessment, both formative and summative. sary for proper and valid use of educational ■ Complete coverage of the basics as well as assessment. We do not hesitate to point out teach- advanced topics and topics of contemporary ers’ and school administrators’ erroneous judg- interest. ments and assessment abuses, however, where ■ Practical advice and examples of how good good lessons can be learned from them. and poor classroom assessments affect students’ learning. NEW AND REVISED CONTENT Educational Assessment of Students is a core text In preparing this edition, we made a special effort written for a first course in educational testing and to make it easy for the reader to apply the mate- constructing classroom assessments, and it serves rial to classroom practice through improved equally as the textbook for an undergraduate course explanations, improved practical examples and or a first graduate course in educational assess- illustrations, checklists, and step-by-step, how-to ment. No formal coursework in statistics or college instructions. As with previous editions, we have mathematics is necessary to understand the text. written the text from the viewpoint that assess- The book provides complete coverage of edu- ment is part of good teaching practice that helps cational assessment, including developing plans the teacher improve students’ learning. Material that integrate teaching and assessment; using for- new to the seventh edition includes: mative assessment strategies and providing effective feedback to students; crafting objective, 1. A new chapter ( Chapter 8 ) on Providing For- performance, and portfolio assessments; evaluat- mative Feedback and expanded coverage of ing students and discussing evaluations with par- Formative Assessment ( Chapter 7 ), including ents; and interpreting state-mandated tests and a section on Feedback for English Language standardized achievement tests. Learners. It is important in a first course that students 2. Expanded coverage of the role and use of receive a balanced treatment of the topics. Because technology in assessment, throughout the

xii A01_BROO6495_07_SE_FM.inddA01_BROO6495_07_SE_FM.indd PagePage xiiixiii 11/21/1311/21/13 7:207:20 PMPM f451f451 ~~/Desktop/21:11:13/BROOKHART_PH01411_203/Desktop/21:11:13/BROOKHART_PH01411_203

PREFACE xiii

chapters but especially in Chapters 6 , 11 , 13 , calculating reliability coefficients for instruc- and 16 . tors and students interested in a more quanti- 3. Updated information on the Common Core tative approach than the text provides. State Standards (CCSS), throughout the text and especially in Chapter 2 (Describing the ACKNOWLEDGMENTS Goals of Instruction) and Chapter 16 , which contains a summary of current plans of the A project of this magnitude requires the help of two state consortia working on the assess- many persons. We are very much indebted to the ment of the CCSS and the two consortia reviewers whose critical reading contributed working on alternative assessments of the greatly to the technical accuracy, readability, and CCSS. pedagogy of the seventh edition: Kathryn Anderson Alvestad, University of Maryland, College Park; 4. Expanded treatment of Learning Progressions Heidi Legg Burross, University of Arizona; Marianne in Chapter 7 . Robin Russo, Florida Atlantic University; Charles 5. Updated and expanded treatment of Standards- L. Thomas, George Mason University; and Tary Based Report Cards in Chapter 1 5 . L. Wallace, University of South Florida, Sarasota- 6. A new section on Student-Led Parent Confer- Manatee. Special thanks go to Janet Carlson and ences in Chapter 15 . Jessica Jonson, Buros Center for Testing; to Leslie 7. Up-to-date discussion of published achieve- Lukin, Lincoln Public Schools; to Gail McGregor, ment tests (including a new section on bench- University of Montana; and to Michael J. Young, mark and interim assessments in Chapter 16 ), Pearson Assessment. scholastic ability tests, and vocational interest We would also like to thank the reviewers for inventories. the second, third, fourth, fifth, and sixth editions: 8. Update of websites related to assessment in Peter W. Airasian, Boston College; Lawrence most chapters. M. Aleamoni, University of Arizona; Carol E. Baker, University of Pittsburgh; W. L. Bashaw, University of Georgia; Gary Bingham, Georgia SPECIAL FEATURES State University; Pamela Broadston, University of The following special features highlight the prac- Arkansas at Little Rock; Deborah Brown, West ticality of this text: Chester University; Marcia Burell, SUNY Oswego; Alice Corkill, University of Nevada at 1. Examples of how to craft classroom assess- Las Vegas; Lee Doebler, University of Monte- ments and what they typically look like. vallo; Leonard S. Feldt, University of Iowa; Terry 2. Checklists with succinct tips for evaluating Fogg, Minnesota State University; Betty E. Gridley, the quality of each type of assessment taught Ball State University; Gretchen Guiton, Univer- in the book. sity of Southern California; Anthony E. Kelly, 3. Strategies for assessing higher-order thinking George Mason University; Jin-Ah Kim, Illinois State that serve as models and descriptions for University; Thomas M. Haladyna, Arizona developing problem-solving and critical- State University; Charles Hughes, Pennsylvania thinking assessments. State University; Louise F. Jernigan, Eastern 4. Key concepts that serve to introduce each Michigan University; Suzanne Lane, University chapter. of Pittsburgh; Robert Lange, University of Central Florida; Robert W. Lissitz, University of 5. Important terms and concepts listed at the Maryland; Nancy Martin, University of Texas– beginning of the chapter and defined in both San Antonio; Craig Mertler, Bowling Green State the chapter’s text and in a glossary. University; William P. Moore, University of 6. End-of-chapter exercises that let students Kansas; Pamela A. Moss, University of Michigan; apply their learning to practical situations Robert Paugh, University of Central Florida; and an appendix with answers to even- Susan E. Phillips, Michigan State University; numbered exercises. Bruce Rogers, University of Northern Iowa; John 7. Appendixes of statistical concepts with Shimkanin, California University of Pennsylvania; spreadsheet applications and tutorials for William M. Stallings, Georgia State University; A01_BROO6495_07_SE_FM.inddA01_BROO6495_07_SE_FM.indd PagePage xivxiv 11/21/1311/21/13 7:207:20 PMPM f451f451 ~~/Desktop/21:11:13/BROOKHART_PH01411_203/Desktop/21:11:13/BROOKHART_PH01411_203

xiv PREFACE

Hoi K. Suen, Pennsylvania State University; the Instructor’s Manual for the fourth edition. To James S. Terwilliger, University of Minnesota; all of these persons, and others we have failed to Michael S. Trevisan, Washington State Univer- mention, we offer our most sincere thanks and sity; Anthony Truog, University of Wisconsin– appreciation. Whitewater; Kinnard White, University of North We are grateful for permission to use checklists Carolina; Richard Wolf, Teachers College, Colum- and examples that Anthony Nitko originally pub- bia University; and David R. Young, State Uni- lished with colleagues Harry Hsu and Maury versity of New York–Cortland. Lindvall. Specifically, the checklists for evaluat- We thank our students at the School of Educa- ing the quality of a test blueprint (Chapter 6 ), tion, University of Pittsburgh; the School of Edu- multiple choice items (Chapter 10 ), matching cation, Duquesne University; the College of exercises ( Chapter 10 ), and essay items ( Chapter Education, University of Arizona; the Curricu- 11 ) and the example in Figure 6.4 originally lum Development and Evaluation Centre, appeared in A. J. Nitko and T-C. Hsu, Teacher’s Botswana Ministry of Education; teachers work- Guide to Better Classroom Testing: A Judgmental ing with the Jamaica Ministry of Education; teach- Approach, 1987, Pittsburgh, PA: Institute for Prac- ers and assessors at the Examination Development tice and Research in Education, School of Educa- Center, Indonesia Ministry of Education and Cul- tion, University of Pittsburgh. The examples in ture; and trainers with the Integrated Language Figures 13.4, 17.4, and 17.12 originally appeared Project in Egypt, who used the second, third, and in C. M. Lindvall and A. J. Nitko, Measuring Stu- fourth editions. They provided insightful feed- dent Achievement and Aptitude ( Second Edition), back and corrections of errors that have greatly 1975, New York: Harcourt Brace Jovanovich. improved the usefulness of the text. Francis Special thanks to Veronica Nitko and Frank Amedahe helped classify chapter learning targets Brookhart, whose support and encouragement and write test items for the third edition. Sarah were invaluable throughout the work on this text Bonner contributed test items, practical examples and its previous editions. for classroom activities, and many elements of M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PagePage 1 11/7/1311/7/13 4:404:40 PMPM user-s138user-s138 ~/Desktop/07:11:13/BROOKHART_PH01411_203~/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER 1

Classroom Decision Making and Using Assessment

KEY CONCEPTS IMPORTANT TERMS 1. Assessment provides teachers with information accountability testing to make decisions about teaching and provides assessment students with information to make decisions classification decisions about learning. content standards 2. Different kinds of educational decisions require different types of assessment information. credentialing 3 . Assessment, test, measurement, and evaluation diagnostic assessments are different but related terms. disaggregation of test results 4. Classroom formative and summative assess- evaluation ments provide teachers and students with the formative evaluation of schools, programs, or information they need to improve learning. materials 5. High-stakes assessments provide those in formative assessment of students’ achievement authority with the information they use to high-stakes assessments (tests) classify and sanction. measurement 6. Professional guidelines for assessment competencies and assessment literacy are available. performance standards placement decisions selection decisions sizing up summative evaluation of schools, programs, or materials summative assessment of students’ achievement test

1 M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 2 111/7/131/7/13 44:40:40 PPMM user-s138user-s138 ~/Desktop/07:11:13/BROOKHART_PH01411_203~/Desktop/07:11:13/BROOKHART_PH01411_203

2 PART ONE ■ The Bases for Assessment

ASSESSMENT AND CLASSROOM have happened had she gone straight to first DECISIONS grade, because she didn’t. Assessment is everywhere in schooling, some- Decisions involve using different kinds of times hidden in plain sight. Consider this example: information. Sometimes test scores play a major role; at other times, less formal assessments play a more dominant role. In Meghan’s case, both Example informal (teachers’ observations, interviews) and formal (readiness test, intelligence test, projective Meghan’s educational assessment began in kinder- test) assessments were administered. garten with an interview and an observation. On reg- Making good classroom decisions requires istration day, Meghan and her mother came to school more than good intentions or previous experi- and were interviewed briefly. A teacher rated ence. Good decisions, such as what to teach, how Meghan’s cognitive and social-emotional skills. Her development was judged normal, and she attended to teach it, and how to evaluate students’ achieve- kindergarten. ment, are based on high-quality information. Suc- During the year, she had difficulty paying attention cessful teachers obtain information about their to the teacher and participating in group activities, students from high-quality assessments. although she was neither aggressive nor hostile. She Similarly, assessment involves more than was given a “readiness test” at the end of kindergar- testing and grading students. Assessment ten and performed as an average child. Her teacher involves gathering and using information to recommended that she continue on to first grade, but improve your teaching and your students’ learn- her parents balked: They didn’t think she was ready. ing. Whether you use teacher-made assessment They took her to a child guidance clinic and procedures, assessments from your district’s cur- requested further psychological assessment. The riculum materials, or state and standardized clinical psychologist administered an individual intel- assessments, you need to be able to explain the ligence test and a “projective test” in which Meghan was asked to tell a story about what was happening results correctly to students, parents, other teach- in each of a set of pictures. The psychologist inter- ers, and school administrators. Further, as you viewed her, her parents, and her teacher. The psy- develop professionally, you may have the oppor- chologist described her as normal, both in cognitive tunity to participate in local and state committees ability and in social-emotional development. concerned with assessment issues. The media Her parents withdrew her from the school she emphasize assessment as a major concern and was attending and placed her in another school to consider it a newsworthy issue. It is likely to repeat kindergarten. Later, they reported that whereas remain so for much of your professional career. her first experience was difficult for her, her second This book discusses a variety of educational deci- kindergarten year was a great success. In their view, sions that depend on assessments, especially in a teacher who was particularly sensitive to Meghan’s the classroom. needs helped accelerate her cognitive development. By the end of the year, she had also become more confident and regularly participated in group activities. ASSESSMENT AND EDUCATIONAL DECISIONS ABOUT STUDENTS This brief anecdote shows assessments being Assessment provides information for decisions used early in life. Most of us recall more easily the about students; schools, curricula, and programs; assessments applied to us later in our lives, as and educational policy. This section discusses older children and as adults. You may not even several types of educational decisions made associate the term assessment with Meghan’s inter- about students. It puts assessment into a broader views. Yet, as we explain later, interviews are context to give you a better idea of the purposes included in the broad definition of assessments. for which assessments are used (see Figure 1.1 ). Meghan’s situation also illustrates that assess- Understanding the features of different types ment results can contribute to a decision, but of decisions will help you evaluate various assess- everyone concerned may not interpret the results ment techniques that you may be considering. in the same way. Although Meghan’s parents There is no simple answer to the question, “Is this may have been right to have her repeat kinder- a good assessment procedure?” An assessment garten, there is no way of knowing what would procedure may serve some types of decisions M01_BROO6495_07_SE_C01.indd Page311/7/134:40PMuser-s138 M 0 1 _ B R O O 6 4 9 5 _ 0 7 _ S E _

FIGURE 1.1 Examples of types of educational decisions for which assessments may be used. C 0 1 . i n d d

Educational assessments P a g e

are used for informing 1 1 / 7 / 1 3

decisions about policy decisions about decisions about students 4 : curricula, programs, and schools 4 0

P M

u s

that include that include that include e r - s 1 3 8 district state national formative summative managing placing classifying counseling selecting credentialing, policy policy policy evaluations evaluations instruction students students and guiding students certifying into students students programs

that includes

formative assessments summative assessments

for purposes of for purposes of

planning placing students monitoring diagnosing feedback to reporting assigning instructional into learning students’ learning students on grades to activities sequences progress difficulties how to students ~/Desktop/07:11:13/BROOKHART_PH01411_203 ~ /

improve D e s k t o p /

to students to teacher 0 7 :

and parents about 1 1 :

about achievement effectiveness 1 3 / B R O O K H 3 A R T _ P H 0 1 4 1

1 _ 2

3 M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 4 111/7/131/7/13 44:40:40 PPMM user-s138user-s138 ~/Desktop/07:11:13/BROOKHART_PH01411_203~/Desktop/07:11:13/BROOKHART_PH01411_203

4 PART ONE ■ The Bases for Assessment

very well, others not so well. Understanding the records to learn previous teachers’ evaluations different types of decisions discussed in this sec- and the students’ standardized achievement test tion will also help you explain to parents why results. you used various assessments with their children. 4 . With what learning activities will my students Finally, although you may not be required to and I need to be engaged as I teach the lesson (unit, make all of these types of student decisions your- course)? Possible assessment methods: Review self, by the time your students have completed the types of activities used previously; knowl- their education they will have experienced virtu- edge of typical student learning progressions in ally all of them. this area; analysis of the sequence of the learning activities students will follow; review of students’ Instructional Decisions interest and achievement when those activities Teachers make decisions about students at the were used previously. rate of one every 2 to 3 minutes (Shavelson & 5. What learning objectives do I want my stu- Stern, 1981 ). That’s about 20 decisions every class dents to achieve as a result of my teaching? Possible period! Sound teaching decisions require sound assessment methods: Review of statements of information. Sound assessment procedures goals and learning objectives; review of test ques- gather sound information. Researchers estimate tions students should be able to answer; review of that teachers may spend from one third to one the things students should be able to do and of half of their time in assessment-related activities the thinking skills students should be able to ( Stiggins, Conklin, & Associates, 1992 ). demonstrate after learning. To help you think about the many decisions a teacher must make, we have organized a set of 6. How should I organize and arrange the students questions teachers must answer before, during, in the class for the upcoming lessons and activities? and after teaching. Examples of assessment meth- Possible assessment methods: Informal observa- ods that may give you useful information for mak- tion of students with special learning and social ing the decisions are listed after each question. needs; recollection of students’ behavior during previous learning activities; information about Decisions Before Beginning Teaching what classroom arrangements worked best in the past when students were pursuing similar learn- 1. What content do I need to cover during the ing goals. next day, week, month, marking period, and so on? Possible assessment methods: Review state standards, the curriculum, the syllabus, and the text- Decisions During Teaching book; examine copies of the standardized tests 1 . Is my lesson going well? Are students catching my students will need to pass. on (i.e., learning)? Possible assessment methods: Observations of students during learning activi- 2. What student abilities (cultural background ties; student responses to questions; observations factors, interests, skills, etc.) do I need to take into of students’ interactions. account as I plan my teaching activities? Possible assessment methods: Informal observation of stu- 2 . What should I do to make this lesson (activity) dents during class discussions; conversations work better? Possible assessment methods: Diag- with students and students’ previous teachers; nosis of the types of errors students made or mis- studying students’ permanent records to see their conceptions students have; identifying alternative scholastic aptitude test results, past grades, and ways to teach the material; identifying which stu- standardized test results; knowledge of the students are not participating or are acting inappro- dent’s personal family circumstances. priately. 3. What materials are appropriate for me to use 3 . What feedback should I give each student about with this group of students? Possible assessment how well he or she is learning? Possible assess- methods: Class discussions in which students’ ment methods: Informal observation and experi- motivations, interests, beliefs, and experiences ence on the amount and type of feedback with learning topics can be observed; results from information different students require; informa- short pretests; study of the students’ permanent tion about how close each student has come to M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 5 111/7/131/7/13 44:40:40 PPMM user-s138user-s138 ~/Desktop/07:11:13/BROOKHART_PH01411_203~/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER ONE ■ Classroom Decision Making and Using Assessment 5

achieving the learning objective; students’ home- teaching decisions require you to use many dif- work and quiz results; interviews of students. ferent types of information. Further, they illustrate that the exact type of information you need 4. Are my students ready to move to the next varies greatly from one situation to the next. The activity in the learning sequence? Possible assess- following sections describe different kinds of ment methods: Informal observation and check- instructional decisions. ing of students’ completed work and questioning students about their understanding; analysis of Instructional Diagnosis and Remediation students’ homework, quizzes, and test results; Sometimes the instruction an individual student results of student self-assessment. receives is not effective: The student may need special remedial help or special instruction that Decisions After a Teaching Segment relies on alternative methods or materials. Assess- ments that provide some of the information 1 . How well are my students achieving the short- needed to make this type of decision are called and long-term instructional objectives? Possible diagnostic assessments. Diagnostic decisions assessment methods: Classroom tests, projects, center on the question, “What learning activities observations, interviews with students; analysis should I use to best adapt to this student’s indi- of standardized test results. vidual requirements and thereby maximize the 2 . What strengths and weaknesses will I report to student’s opportunities to attain the chosen learn- each student and to his or her guardian or parent? ing objective?” Diagnosis implies identifying Possible assessment methods: Observations of both the appropriate content and the types of each student’s classroom participation; review of learning activities that will help a student attain each student’s homework results; review of each the learning objective ( Nitko, 1989). student’s standardized achievement and scholastic aptitude test results when they become avail- Feedback to Students Assessments can pro- able; review of information about a student’s vide feedback to students about their learning. personal family circumstances. Feedback, however, is likely to improve learning only under certain conditions. Simply assessing 3. What grade should I give each student for the students and reporting the results to them is not lesson or unit, marking period, or course? Possible likely to affect their performance. Learners must assessment methods: Combining results from review both correct and incorrect performance classroom assessments, quizzes, tests, class proj- and, in addition, be able to correct their incorrect ects, papers, labs, etc.; observation about how performance. Feedback must give specific guid- well the student has attained intended learning ance to students about what to do to improve. goals. Therefore, teachers who give students only their 4. How effectively did I teach this material to the grade on a paper or test are not providing enough students? Possible assessment methods: Review of feedback to help students improve. summaries of the class’s performance on the impor- Assessments can be used to provide feedback tant instructional objectives and on selected ques- that helps learning, provided you integrate them tions on standardized tests, and of how well the into your instructional process. Feedback from students liked the activities and lesson materials. classroom assessment procedures will not help your students learn if the students lack a com- 5 . How effective are the curriculum and materials mand of the prerequisite learning and/or have I used? Possible assessment methods: Review of comprehended little of the lesson prior to the summaries of informal observations of students’ assessment. It is especially important that stu- interests and reactions to the learning activities dents correct their errors before you go on to new and materials, of the class’s achievement on class- instruction. Additional discussion of feedback room tests that match the curriculum, and of sev- appears in Chapter 8 . eral past classes’ performance on selected areas of standardized tests. Feedback to the Teacher Assessments provide These lists of questions and assessments are not feedback to the teacher about how well students exhaustive. These examples illustrate that your have learned and how well the teacher has taught. M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 6 111/7/131/7/13 44:40:40 PPMM user-s138user-s138 ~/Desktop/07:11:13/BROOKHART_PH01411_203~/Desktop/07:11:13/BROOKHART_PH01411_203

6 PART ONE ■ The Bases for Assessment

Of course, if students have failed to grasp impor- evaluations. Do not use test scores alone to justify tant points, the teacher should reteach the mate- your grades. Some teachers do this because rial before proceeding to new material. assigning grades involves evaluative decisions, and judgments are often difficult to justify and Modeling Learning Expectations Assessments explain. Tests, especially those of the objective serve as examples for students by showing them variety, seem to reduce judgment and subjectiv- what you want them to learn. Assessments, as ity, even though this is not necessarily true. A well as other assignments, should therefore more complete discussion of grading, including embody the students’ learning target (Shepard, suggestions for assigning grades, appears in 2006 ) so that students get an accurate and clear Chapter 15 . idea of what they are to learn. Students can compare their current performance with desired performance. You may teach students to identify the Selection Decisions way(s) in which their current performance Most people are familiar with selection deci- matches the expected performance and how to sions : An institution or organization decides that remedy any deficiencies. In this way, good assess- some persons are acceptable, whereas others are ment is good instruction. Also, as students evalu- not; those who are unacceptable are rejected and ate their own performance, you may teach them are no longer the concern of the institution or the appropriate criteria for judging how well they organization. This feature—rejection and the are learning as well as what is important to learn. elimination of those rejected from immediate institutional concern—is central to a selection Motivating Students Assessments may also decision. For example, college admissions are often motivate students to study. Unfortunately, some selection decisions: Some candidates are admitted teachers use this form of accountability as a weapon and others are not; those who are rejected are no rather than as a constructive force. Teachers may longer the college’s concern. (Some critics may hope that using an assessment as a possible threat argue that those rejected should still be of con- will encourage their students to take studying seri- cern to society generally.) ously. Sometimes teachers use the “surprise quiz” When an institution uses an assessment pro- or “pop quiz” in this manner to encourage more cedure for selection, it is important to show that frequent studying and less cramming. candidates’ results on the assessments bear a sig- Studies have not justified use of assessments nificant relationship to success in the program or this way. Rather, assessments ought to be viewed job for which the institution is selecting persons. in a more positive light: as tools for instruction If data do not show that these assessment results and feedback to students. Positive motivation can distinguish effectively between those candi- comes as students understand what they need to dates likely to succeed and those unlikely to suc- do to improve and realize they are capable of tak- ceed, then these assessment procedures should be ing those steps. Also, teachers or parents who improved or eliminated. In fact, it may be illegal stress test performance as the sole or major crite- to continue to use assessment results that bear no rion for school success may create undue test anx- relationship to success on the job ( Equal Employ- iety for students. As a result, students may ment Opportunity Commission, Civil Service perform less well in the long run. Commission, Department of Justice, Department of Labor, & Department of the Treasury, 1979; Assigning Grades to Students One of the most United States Supreme Court, 1971 ). obvious reasons for giving classroom assessments Selection decisions need not be perfect to be is to help you assign grades to students. Periodi- useful, however. Assessment results cannot be cally, teachers must officially record their evalua- expected to have perfect validity for selection, or tions of students’ progress. The grades or symbols any other, decisions (see Chapter 3 ). Figure 1.2 (A, B, C, etc.) that you report represent your sum- illustrates the use of imperfect assessments in mative evaluations or judgments about how well selection. Some applicants would have been suc- your students have achieved important learning cessful had they been selected instead of rejected goals. Use a mixture of assessment formats to (false negative decisions); and some, even though provide the information you need to make these they were accepted, turned out to be unsuccessful M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 7 111/7/131/7/13 44:40:40 PPMM user-s138user-s138 ~/Desktop/07:11:13/BROOKHART_PH01411_203~/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER ONE ■ Classroom Decision Making and Using Assessment 7

FIGURE 1.2 A simplified illustration of how a selection situation Decisions: Consequences: uses assessments and the consequences of those Successful persons decisions. The assessments (correct decisions) and the decision rules are Accept some evaluated in terms of persons Persons unsuccessful their consequences. on the job or in the program (erroneous decisions; persons should have been Decision rules combining rejected) Persons take assessment results assessments and other information are applied Persons unsuccessful on the job or in the program (correct decisions) Reject other persons Persons successful on the job or in the program (erroneous decisions; persons should have been accepted)

(false positive decisions). Assessments can be placement decisions because their ultimate pur- evaluated, then, in terms of the consequences of pose is to place all students in appropriate educa- the decisions made when using them. tional programs. The schools are not free to teach some students and to reject the rest. If one instruc- Placement Decisions tional method is inappropriate for a particular student, then an appropriate alternative method In placement decisions, persons are assigned to needs to be found. In the end, all students are different levels of the same general type of served. instruction, education, or work; no one is rejected, but all remain within the institution to be assigned to some level. Students not enrolled in honors Classification Decisions sections, for example, must be placed at other Sometimes a decision results in a person being educational levels. Or, first-grade students with assigned to one of several different but unordered low scores on a reading readiness test cannot be categories, jobs, or programs. These types of deci- sent home. They must be placed in appropriate sions are called classification decisions . For exam- educational settings and taught to read. You may ple, educational legislation concerning persons recognize a decision as a placement decision by with disabilities has given a legal status to many noting whether the institution must account for labels for classifying children with disabilities and all candidates instead of sending some away, as strongly encourages classifying them into one (or in selection decisions. more) of a few designated categories. These cate- Most decisions in schools are placement deci- gories are unordered (blindness is not higher or sions. Educators who use the language of selec- lower than deafness), so these are classification tion are often using the language incorrectly. On decisions rather than placement decisions. closer examination, they are speaking about You may consider classification as a more gen- placement decisions. For example, when an edu- eral term that subsumes selection and placement as cator speaks of “screening” students for a gifted special cases. Classification refers to cases in which and talented program, the decisions are actually the categories are essentially unordered, placement M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 8 111/7/131/7/13 44:40:40 PPMM user-s138user-s138 ~/Desktop/07:11:13/BROOKHART_PH01411_203~/Desktop/07:11:13/BROOKHART_PH01411_203

8 PART ONE ■ The Bases for Assessment

refers to cases in which the categories represent lacked the necessary resources or a particular ordered levels of education without rejection, and teacher failed to deliver appropriate instruction, selection refers to cases in which students are the assessment-based certification process seems accepted or rejected. This book considers the three inherently unjust. types of decisions separately. Data Driven Decision Making Counseling and Guidance Decisions The phrase “data driven decision making” means Assessment results frequently assist students in analyzing, interpreting, and using data in a man- exploring, choosing, and preparing for careers. A ner that contributes to improved instruction and single assessment result is not used for making increased student learning (Boudett, City, & guidance and counseling decisions. Rather, a Murnane, 2005). The phrase is recently popular, series of assessments is administered, including but the concept behind it, using assessment an interview, an interest inventory, various apti- information—of all kinds, not just test scores—to tude tests, a personality questionnaire, and an inform educational decisions, is not. This section achievement battery. Information from these has described types of decisions educators make assessments, along with additional background and the assessment information on which they information, is discussed with the student during should be based, and these concepts have been a series of counseling sessions. This process facili- around for decades. The contents of this book are tates a student’s decision making and provides a intended to help you develop the skills of under- beginning for exploring different careers. Explor- standing, interpreting, and using assessment ing career options is likely to involve an ongoing information—the “data” in “data driven decision and changing series of decisions that occur making”—in your work. In a sense, then, this throughout a person’s life. whole book is about data driven decision making. The discussion of pretesting and differentiating Credentialing and Certification Decisions instruction in Chapter 6 is especially relevant for Credentialing and certification decisions reflect classroom instructional decision making. The dis- whether a student has attained certain standards cussion of appropriate uses of standardized tests of learning. Certification decisions may focus on in Chapter 16 is especially relevant for school- and whether a student has attained minimum compe- program-level decision making. However, most tence or obtained a high standard, depending on educational decisions benefit from considering the legal mandate. Certification and credentialing multiple assessments at multiple levels, and we may be mandated by a state’s legislation or may encourage you to use all the assessment concepts be voluntary. If a state law requires students to in this book as you make educational decisions. achieve certain standards of performance, most often students are administered an assessment DISTINCTIONS AMONG ASSESSMENTS, procedure created at the state level. Those who TESTS, MEASUREMENTS, AND meet the standards are awarded a credential EVALUATIONS (such as a high school diploma). The general public often uses the terms assess- Assessment procedures for certification pres- ment, test, measurement, and evaluation inter- ent special problems for validation. Individual changeably, but it is important for you to students cannot reasonably be held accountable distinguish among them. This section explains for instruction that the teacher failed to deliver or the relationship among these terms (shown in which was delivered poorly, even though, on the Figure 1.3 ) and the way assessments inform edu- average, teaching was adequate. A critical point, cational decisions ( Figure 1.1 ). therefore, is whether the quality of instruction corresponds to what the assessment procedure covers. The closer the correspondence, the fairer Assessment the certification is to the student. If students did Assessment is a broad term defined as a process not have the opportunity to learn how to perform for obtaining information for making decisions the tasks that appear on the certification assess- about students; curricula, programs, and schools; ment procedure, either because a specific school and educational policy. When we say we are M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 9 111/7/131/7/13 44:40:40 PPMM user-s138user-s138 ~/Desktop/07:11:13/BROOKHART_PH01411_203~/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER ONE ■ Classroom Decision Making and Using Assessment 9

FIGURE 1.3 Relationship among the terms assessments, tests, Assessments measurement, and evaluation . are used to gather information about students and include

Tests Nontests

are systematic procedures for describing certain characteristics of students using either

Classification schemes Numerical scales

use psychological use a process called theories

Measurement

to assign qualitative to assign scores labels to students to students

One or more of these may be combined with a teacher's experience to judge the worth of a student's achievement using a process called

Evaluation

“assessing a student’s competence,” for example, become competent in selecting and using assess- we mean we are collecting information to help us ments. These five guiding principles will help decide the degree to which the student has achieved you select and use educational assessments intended learning outcomes. A large number of meaningfully. assessment techniques may be used to collect this 1 . Be clear about the learning objectives you want information: formal and informal observations of a to assess. Before you can assess a student, you student; paper-and-pencil tests; a student’s perfor- must know the kind(s) of student knowledge, mance on homework, lab work, research papers, skill(s), and performance(s) about which you projects, and during oral questioning; and analyses need information. The knowledge, skills, and of a student’s records. This book will help you performances you want students to learn are decide which of these techniques are best for your sometimes called learning goals or standards. particular teaching situations. The more clearly you are able to specify these learning goals, the better you will be able to select Guidelines for Selecting and Using the appropriate assessment techniques. Classroom Assessments 2. Be sure that the assessment techniques you In order to focus your assessment activities on the select match the learning goal. For example, if the information you need to make particular educa- goal specifies that students will be able to write tional decisions in the classroom, you need to poetry, solve a mathematical problem, or design M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 1100 111/7/131/7/13 44:40:40 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

10 PART ONE ■ The Bases for Assessment

a scientific experiment, the assessments should of a learning goal. Because of this, information require students to do these things. The assess- from assessment contains sampling error. Also, ment techniques selected should be as practical factors such as a student’s physical and emotional and efficient to use as possible, but practicality conditions further limit the extent to which we and efficiency should not be the overriding can obtain truly accurate information. Teachers, considerations. students, and others must make decisions nevertheless. Those decisions must keep an assess- 3. Be sure that the selected assessment techniques ment’s limitations in mind. serve the needs of the learners. Proper assessment tools are concrete examples for students of what they are expected to do with their learning. Tests Assessment techniques should provide learners A test is defined as an instrument or systematic with opportunities for determining specifically procedure for observing and describing one or what they have achieved and specifically what more characteristics of a student using either a they must do to improve their performance. numerical scale or a classification scheme. Test is Therefore, you should select assessment methods a concept narrower than assessment . In schools, that allow you to provide meaningful feedback to we usually think of a test as a paper-and-pencil the learners. You should be able to tell students instrument with a series of questions that stu- how closely they have approximated the learning dents must answer. Teachers usually score these goals. Good assessment is good instruction. tests by adding together the “points” a student 4. Whenever possible, be sure to use multiple earned on each question. By using tests this way, indicators of performance for each learning objective. teachers describe the student using a numerical One format of assessment (such as short-answer scale. Similarly, a preschool child’s cognitive questions or matching exercises) provides an development could be observed by using the incomplete picture of what a student has learned. Wechsler Preschool and Primary Scale of Intelli- Because one assessment format tends to empha- gence (see Chapter 19 ) and described as having a size only one aspect of a complex learning goal, it percentile rank of 50 (see Chapter 17 ). Not all typically underrepresents that goal. Getting tests use numerical scales. Others use systematic information about a student’s achievement from observation procedures to place students into several assessment modalities usually enhances categories. the validity of your assessments. Matching exer- Although it is natural to assume that tests are cises, for example, emphasize recall and recogni- designed to provide information about an indi- tion of factual information; essay questions vidual, this is not always true. States have testing emphasize organizing ideas and demonstrating programs designed to determine whether their writing skill under the pressure of time limits; schools have attained certain goals or standards. and a monthlong project emphasizes freely using Although these tests are administered to individ- resources and research to more thoroughly ana- ual students, a state uses the results to measure lyze the topic. All three of these assessment tech- the effectiveness of a school. In such cases, indi- niques may be needed to ascertain the extent to vidual names are not associated with scores when which a student has achieved a given learning reporting to the government. The “score” for the standard. school system (or for a specific school at a specific grade level) is usually the percentage of the 5. Be sure that when you interpret—or help stu- school’s students who meet or exceed that state’s dents interpret—the results of assessments, you take standards. the limitations of such results into account. Another example of an assessment program Although Guiding Principle 2 calls for increasing designed to survey the educational system rather the authenticity or meaningfulness of the assess- than individual students is the National Assess- ment techniques, assessments in schools cannot ment of Educational Progress (NAEP) ( nces. completely reproduce those things we want stu- ed.gov ). The NAEP assesses the impact of the dents to learn in “real life.” The information you nation’s educational efforts by describing what obtain, even with several different types of assess- students are able to do. Assessment tasks are ments, is only a sample of a student’s attainment assigned to students on a random sampling basis M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 1111 111/7/131/7/13 44:40:40 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER ONE ■ Classroom Decision Making and Using Assessment 11

so that not every student has the same or even Evaluation comparable tasks. Thus, it is not meaningful to Evaluation is defined as the process of making a use the scores with individual students. The value judgment about the worth of a student’s assessment is intended to pool the results from all product or performance. For example, you may students in the sample to show the progress of judge a student’s writing as exceptionally good education in the entire country. for his grade placement. This evaluation may lead The NAEP surveys are efficient ways to you to encourage the student to enter a national gather information about the average perfor- essay competition. To make this evaluation, you mance of a group of students because they assess would first have to assess his writing ability. You each student using very few tasks, but pool the may gather information by reviewing the stu- results to estimate the average. However, this dent’s journal, comparing his writing to that of gain in efficiency of assessing the group comes at other students and to known quality standards of the expense of not being able to describe validly writing, and so on. Such assessments provide the achievement of individual students. information you may use to judge the quality or worth of the student’s writing. Your judgment Measurement that the student’s writing is of high quality would lead you to decide to encourage him to enter the Measurement is defined as a procedure for competition. Evaluations are the bases for deci- assigning numbers (usually called scores) to a sions about what course of action to follow. specified attribute or characteristic of a person in Evaluation may or may not be based on mea- such a way that the numbers describe the degree surements or test results. Among others, evalua- to which the person possesses the attribute. An tions may be based on counting things, using important feature of the number-assigning proce- checklists, or using rating scales. Clearly, evalua- dure in measurement is that the resulting scores tion does occur in the absence of tests, measure- maintain the order that exists in the real world ments, and other objective information. You among the people being measured. At the mini- can—and probably often do—evaluate students mum, this principle would mean, for example, on the basis of assessments such as systematic that if you are a better speller than we are, a test observation and qualitative description, without that measures our spelling abilities should result measuring them. Even if objective information is in your score (your measurement) being higher available and used, evaluators must integrate it than ours. into their own experiences to come to decisions. For many of the characteristics measured in So degrees of subjectivity, inconsistency, and bias education and psychology, the number-assigning influence all evaluations. Testing and measure- procedure is to count the correct answers or to ment, because they are more formal, standard- sum points earned on a test. Alternatively, we ized, and objective than other assessment may use a scale to rate the quality of a student’s techniques, reduce some of the inconsistency and product (for example, an essay or a response to subjectivity that influence evaluation. The gen- an open-ended mathematics task) or performance eral public, however, sometimes thinks that (how well the student carries out chemistry lab because numbers look objective they remove the procedures). (See Chapter 13 for examples.) Most element of judgment from evaluation; this is measurement specialists would probably agree called the illusion of “mechanical objectivity” that although a counting or rating procedure is ( Porter, 1995 , p. 4 ). crude, as a practical matter scores from assessments are useful when they are validated by Evaluation of Schools, Programs, or Materials using data from research ( Kane, 2006 ). Not all evaluations are of individual students. Thus an assessment may or may not provide You also can evaluate a textbook, a set of instruc- measurements. If a procedure describes a student tional materials, an instructional procedure, a by qualitative labels or categories but not by curriculum, an educational program, or a school. numbers, the student is assessed, but not mea- Each of these things may be evaluated during sured in the sense used here. Assessment is a development as well as after they are completely broader term than test or measurement because not developed. The terms formative and summative all types of assessments yield measurements. evaluation are also used to distinguish the roles M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 1122 111/7/131/7/13 44:40:40 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

12 PART ONE ■ The Bases for Assessment

of evaluation during these two periods ( Cronbach, process of learning. We make formative assess- 1963 ; Scriven, 1967 ). Historically, these terms ments of students to guide their next learning steps. arose first in the context of evaluation of schools When you ask questions in class to see whether or programs and were then applied to students. students understand the lesson, for example, you The convention has become that “formative and are obtaining information to formatively evaluate summative evaluation” refers to schools, pro- their learning. You can then adjust your lesson if grams, or materials, and “formative and summa- students do not understand. Students participate in tive assessment” refers to students. We will formative assessment as well, interpreting informa- follow that convention. tion about their own performances to adjust their Formative evaluation of schools, programs, learning strategies (Moss & Brookhart, 2009). High- or materials is judgment about quality or worth quality formative assessment and feedback to made during the design or development of students increase student learning (Hattie & instructional materials, instructional procedures, Timperley, 2007 ). In general, formative assessments curricula, or educational programs. The evalua- are less formal than summative assessments. We tor uses these judgments to modify, form, or oth- recommend that you record the results of these erwise improve the school, program, or assessments to help your memory; however, you educational material. A teacher also engages in do not use them to report official letter grades or formative evaluation when revising lessons or achievement progress. learning materials based on information obtained Typically, you use the most informal assess- from their previous use. ments for sizing-up purposes. Sizing up means Summative evaluation of schools, programs, to form a general impression of a student’s or materials is judgment about the quality or strengths, weaknesses, learning characteristics, worth of schools, already-completed instructional and personality at the beginning of a course or at materials, instructional procedures, curricula, or the start of the year. The following example illus- educational programs. Such evaluations tend to trates how a teacher pulled together various summarize strengths and weaknesses; they informally obtained pieces of information to size describe the extent to which a properly imple- up Saleene, a fifth-grade student: mented program or procedure has attained its Saleene (a fifth grader) walks into class each stated goals and objectives. Summative evalua- day with a worried and tired look on her face. tions appraise the effectiveness of a particular Praising her work, or even the smallest posi- educational product as well as under what condi- tive action, will crack a smile on her cheeks, tions it is effective. Summative evaluations usu- though the impact is brief. She is inattentive, ally are directed less toward providing suggestions even during the exercises we do step by step. for improvement than are formative evaluations. Saleene has a hearing disability that makes it hard for her to follow directions and class- Evaluation of Students You may evaluate stu- room discussions. She is shy, but sometimes dents for formative or summative purposes, as will ask for help. But before she gives herself a well. Classroom formative and summative assess- chance, she will put her head down on her ments both should be based on the same intended desk and close her eyes. Her self-esteem is low. I am concerned that she will be this way learning outcomes. Figure 1.4 shows common uses all year. ( Airasian, 2001, p. 38 ) for classroom assessment results. The uses are organized into two groups: formative and summa- You can see that this teacher used information tive. One use of assessment, controlling students’ about Saleene’s cognitive, affective, and psycho- behavior, is not listed in Figure 1.4 because it is a motor traits to help form a general strategy for poor, and sometimes unethical, practice. Control- how to teach her. ling students through assessments turns a process Other formative decisions also require quality of information gathering into a process of threat- information. These include diagnosing individual ening and punishing with negative consequences students’ learning needs, communicating achieve- for learning and self-efficacy. ment expectations, using assessment in instruction, Formative assessment of students’ achieve- diagnosing the group’s learning needs, providing ment means judging the quality of a student’s feedback, promoting student self-assessment, and achievement while the student is still in the planning instruction (see Chapters 7 and 8 ). These M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 1133 111/7/131/7/13 44:40:40 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER ONE ■ Classroom Decision Making and Using Assessment 13

FIGURE 1.4 Examples of basic purposes for which classroom assessment results are used.

I. Formative uses help teachers monitor or guide student learning F. Providing specific feedback gives students information about while it is still in progress. how to improve. A. Sizing-up uses help a teacher form initial impressions of G. Promoting students’ self-assessment helps students students’ strengths, weaknesses, learning characteristics, monitor their own learning, set goals, and take action to and personalities at the beginning of the year or course. meet them. B. Diagnosing individual students’ learning needs helps a H. Planning instructional uses helps a teacher design and teacher and the student identify what the student has learned implement appropriate learning and instruction activities, and what still needs to be learned, decide how instruction decide what content to include or emphasize, and organize needs to be differentiated, and decide what feedback each and manage the classroom as a learning environment. student needs about how to improve. II. Summative uses help a teacher evaluate student learning after C. Diagnosing the group’s learning needs helps a teacher iden- teaching one or more units of a course of study. tify how the class as a whole has progressed in its learning, what might need to be reinforced or retaught, and when the A. Assigning grades for report cards is a way in which a teacher group is ready to move on to new learning. records evaluations of each student’s learning progress to communicate evaluations to students, their parents, and D. Using assessment procedures as teaching tools is a way in responsible educational authorities. which a teacher uses the assessment process as a teaching strategy. For example, a teacher may give practice tests B. Placing students into remedial and advanced courses is or “mock exams” to help students understand the types a way in which a teacher attempts to adapt instruction to of tasks used on the assessment, practice answering and individuals’ needs when teaching is group based. Students recording answers in the desired way, or improve the speed who do poorly in the teacher’s class may be placed into at which they respond. In some cases, the performance remedial classes that provide either alternate or supplemen- assessed is identical or nearly identical to the desired learn- tal instruction that is more suitable for the students’ current ing objective so that “practicing the assessment” is akin to level of educational development. Similarly, students whose teaching the intended knowledge or skill. educational development in the subject is above that of the rest of the class may be placed into a higher-level or more E. Communicating achievement expectations to students helps enriched class. teachers clarify for students exactly what they are expected to be able to perform when their learning is complete. This C. Evaluating one’s own teaching requires a teacher to review communication may be done by showing the actual assess- the learning that students have been able to demonstrate ment tasks or by reviewing the various levels or degrees of after the lessons are complete, identify which lessons were performance of previous students on specific assessment successful with which students, and formulate modifica- tasks so that current students may be clear about the level of tions in teaching strategies that will lead to improved student learning expected of them. performance the next time the lessons are taught.

decisions require valid information from carefully assessments and ensure the validity of each result planned assessment. for supporting the decisions based on them. Summative assessment of students’ achievement means judging the quality or worth of a HIGH-STAKES ASSESSMENT student’s achievement after the instructional AND ACCOUNTABILITY process is completed. Giving letter grades on report cards is one example of reporting your It may not come as a surprise to you that what summative evaluation of a student’s achievement you teach and how you teach it are not entirely (see Chapter 15 ). Parents and school authorities under your control. Legally mandated external interpret those grades as the progress students assessment programs place constraints on your have made toward achieving the curriculum’s teaching. You need to be aware of these as you learning goals. Summative uses of assessment plan your classroom instruction. also help you evaluate your own teaching after you finish teaching one or more units. Placement High-Stakes Testing and evaluation decisions are also summative uses High-stakes assessments (tests) are used for for assessment results. Because of the finality of decisions that result in serious consequences for summative assessment, you should prepare to school administrators, teachers, or students. Here keep records of students’ results on summative are some examples: M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 1144 111/7/131/7/13 44:40:40 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

14 PART ONE ■ The Bases for Assessment

Examples fact, the tests may be low stakes for the students because there appear to be no consequences for High-Stakes Testing their doing poorly on the tests. Example 1. In a certain country, at the end of their secondary schooling, students must pass an exami- Accountability Testing nation for each subject they studied. The examina- tions cover the concepts and skills that are in the Although the use of high-stakes testing in the curriculum. Students are marked as A, B, C, D, and F United States can be traced back to Horace Mann for each examination. Students must get no Fs in in the 1850s, modern high-stakes testing in the order to be awarded the secondary school certificate. United States grew out of school reform move- Persons without a secondary school certificate find it ments that developed during the 1980s. Educa- difficult to get a job in the country because employers tional reformers and state legislators wanted to see the certificate as indicating that candidates for a ensure that virtually all students could meet edu- job have necessary minimum competencies. Stu- cational standards set by the state and demanded dents who fail may study on their own time and take by employers. Employers needed to increase pro- the examination again, but they cannot repeat the ductivity and to be competitive in world markets. schooling because there are only a limited number of They needed a better-educated workforce to han- places in secondary schools. Students must have As and Bs but no Ds or Fs to be considered for a place dle the demands of rapidly increasing technology in one of the few universities. and greater intellectual skills needed in the work- place. State legislators considered testing to be Example 2. In a certain state, students must pass one way of holding schools accountable for stu- tests in English, writing, and mathematics before dents learning the educational standards set by a Grade 12; otherwise they cannot receive a high state. school diploma. They begin taking the test in Grade Assessment that is used to hold individual 10, and they may repeat the tests they failed once students or school officials responsible for ensur- each year up to Grade 12. Students who do not pass ing that students meet state standards is called all of the tests by the end of Grade 12 receive only an accountability testing. Usually accountability attendance certificate. testing is accompanied by high-stakes consequences. A state’s accountability testing may Example 3. In another state, students take annual state-mandated tests in reading and mathematics take several forms, as is shown by the examples from Grades 3 through 11. Students do not have to above. A state may require both individual and pass the tests, but each school is evaluated by how school accountability, too. Check your state’s well its students do. If a school’s students do not education department website for its current reg- show a pattern of continued improvement on the ulations regarding individual and school tests, the state sanctions the school by dismissing accountability. the administrative staff and perhaps some of the teachers. It turns over the running of the school to a No Child Left Behind Act state-appointed team until the test scores show regular improvement. The No Child Left Behind Act (NCLB) is important to our discussion of high-stakes assessment because it requires states to establish challenging In Example 1, the consequences of assessment content standards and performance standards are quite serious for individual students: If they (referred to as achievement standards in the NCLB fail to pass all subjects, they may not get a job literature) and to demonstrate by way of tests and because employers require a secondary school other assessments how well students have certificate; if they fail to do well on the examina- attained high levels of achievement on these stan- tions, they have no opportunity for attending a dards. A state’s failure to provide this demonstra- university. The stakes are high in Example 2, but tion results in loss of federal education funds that not quite as high as in Example 1. Students can are authorized under NCLB. Assessment under stay in school for several years, prepare for the NCLB is a school-level accountability tool. tests, and retake the tests each year. In Example 3, there are high stakes for school administrators Standards-Based Proficiency Requirements and teachers, but not for individual students. In Content standards describe the subject-matter M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 1155 111/7/131/7/13 44:40:40 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER ONE ■ Classroom Decision Making and Using Assessment 15

facts, concepts, principles, and so on that students them. The Common Core State Standards Ini- are expected to learn. Performance standards tiative, coordinated by the National Governors describe the things students can perform or do Association Center for Best Practices (NGA once the content standards are learned. (We dis- Center) and the Council of Chief State School cuss state standards and how to align your learn- Officers (CCSSO), released standards in Eng- ing objectives to them in Chapter 2 .) When lish/language arts and mathematics in June students are assessed on a state’s standards, they 2010. The two federally funded assessment are classified into one of three categories for pur- consortia are the Partnership for Assessment poses of reporting to the federal government: of Readiness for College and Careers (PARCC, basic, proficient, and advanced. A state may have parcconline.org ) and the Smarter Balanced more than three categories, but all must be Assessment Consortium (smarterbalanced. aligned to these three. Under NCLB, the goal org). Both consortia aim to have assessments originally was for 100% of the students in each ready for states’ implementation by the school to reach the proficient level or higher on 2014–2015 school year. the state’s content and performance standards by 2. State-developed differentiated recognition, 2014. In addition, schools were to show adequate accountability, and support. To qualify for a yearly progress (AYP) toward this goal or have waiver of the original NCLB sanctions, which sanctions imposed. were couched in punitive language (e.g., “failing schools”) and could include staff High-Stakes Sanctions NCLB sanctions and replacement or a state takeover of the school, corrective actions for schools that failed to make states must develop and implement a system adequate yearly progress after 2 years were to identify schools into different categories for quite restrictive and entirely punitive; there support. The state must set new, achievable, were no official rewards for schools that did annual measurable objectives for all local edu- well. In addition, reaching 100% proficiency was cation agencies (districts), schools, and sub- a statistically unattainable goal, and, as 2014 groups. Using these, states will publicly approached, more and more schools failed to recognize Title I schools making the most meet AYP goals. Beginning in 2011, the federal progress or having the highest performance as government allowed states to apply for waivers “reward schools”; publicly identify the from some of the NCLB requirements—such as lowest-performing schools as “priority meeting original AYP goals—and sanctions. In schools,” implementing and evaluating inter- order to qualify for a waiver, states had to vention plans for those schools; and publicly address four principles (U.S. Department of identify Title I schools with the greatest Education, 2012). achievement gaps, or in which subgroups are furthest behind, as “focus schools,” also imple- 1. College- and career-ready expectations for all stu- menting and evaluating intervention plans for dents. To qualify for a waiver of NCLB restric- those schools. tions, states must demonstrate that they have college- and career-ready expectations for stu- 3 . Supporting effective instruction and leadership. To dents, in at least reading/language arts and qualify for a waiver of NCLB restrictions, states mathematics, by adopting standards (a) that must develop evaluation systems for teachers are common to a significant number of states and principals that use multiple sources of or (b) that are approved by a state network of data, including measures of student growth institutions of higher education. They also and measures of professional practice. The must develop and administer annual, state- evaluation systems should be used for contin- wide, aligned high-quality assessments and uous improvement and provide clear and report student growth on these standards in at timely feedback to the teachers and principals. least Grades 3 through 8 and once in high school. 4 . Reducing duplication and unnecessary burden. To Most states will do this by signing on to the qualify for a waiver of NCLB restrictions, states Common Core State Standards ( corestandards. must remove school administrative reporting org ) and participating in one of two consortia requirements that have little or no impact on of states developing assessments to match student outcomes. M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 1166 111/7/131/7/13 44:40:40 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

16 PART ONE ■ The Bases for Assessment

Disaggregation An important provision of the ensure that all students demonstrate that learn- NCLB Act is that a state must report test summa- ing has occurred. There is some evidence that ries at the school level and must disaggregate the Black-White and Hispanic-White achievement data. Disaggregation of test results means that gaps have decreased since NCLB, although the test results for the total population of students male-female gaps remain unchanged ( Reardon, are separated in order to report on individual Greenberg, Kalogrides, Shores, & Valentino, subgroups of students—such as students who are 2012 ). poor, who are members of minority groups, who Critics point to the inevitable corruption of have limited English proficiency, and who have test scores when stakes are high, including the disabilities—in addition to reporting on the total narrowing of the curriculum to easily tested student population. The reason for this require- objectives whenever the focus of the school is ment is that the federal government wants to on improving scores on tests (e.g., Nichols & ensure that states are accountable for all students Berliner, 2008). Some have argued that the learning the challenging state standards, includ- large-scale accountability tests are not instruc- ing those in these subgroups. In some instances in tionally sensitive, that is, they do not reflect the past, states reported only on the whole popu- students’ actual classroom learning (Popham, lation of their students, thus masking the fact that 2005). Others point out that “for special educa- some subgroups of students were not receiving tion students and the schools that serve them, quality education and were failing to meet the the requirements of two federal education laws standards. and their implementing regulations, the Indi- viduals with Disabilities Education Act (IDEA) Assessment of Students With Disabilities and the No Child Left Behind Act (NCLB), are Under NCLB, all students must be assessed, in conflict” (Phillips, 2005). Finally, as tests including students with disabilities and students become more rigorous, pass rates will decline, with limited English proficiency. Ninety-five per- at least initially. Some states have already cent of students with disabilities must participate raised passing scores on their state accountabil- in the assessment. Students’ disabilities may be ity tests to get ready for the higher expectations used as a basis for accommodations to the assess- for student achievement in the new Common ment process when they are unable to participate Core assessments, and have seen such results under the standardized conditions set for the (Ujifusa, 2012). They are working to inform the general student population. Further, alternative public about changes in their accountability assessment methods must be found to assess systems. those students who cannot participate even with accommodations. States are now granted some limited flexibility in adjusting content and perfor- ACQUIRING THE KNOWLEDGE mance standards for students with severe cogni- AND SKILLS TO ASSESS STUDENTS tive impairments (U.S. Department of Education, The American Federation of Teachers, the 2005 ). National Council on Measurement in Education, and the National Education Association pub- Issues in the Current Accountability Climate lished Standards for Teacher Competence in Educa- Whether NCLB assessment and accountability tional Assessment of Students in 1990. These requirements have improved or hindered edu- standards are somewhat dated now. Most impor- cation is controversial. Advocates of strong tantly, they do not address formative assessment accountability testing support the federal gov- skills like being able to help students generate ernment’s position that “No Child Left Behind is and use assessment information for their own designed to change the culture of America’s learning. Appendix A presents a more recent syn- schools by closing the achievement gap, offering thesis of the various knowledge and skills that, more flexibility, giving parents more options, taken together, comprise what today would be and teaching students based on what works” called “assessment literacy” for teachers. The aim (U.S. Department of Education, n.d.). Propo- of this book is to develop these understandings nents view assessment as an objective way to and skills in its readers. M01_BROO6495_07_SE_C01.inddM01_BROO6495_07_SE_C01.indd PPageage 1177 111/7/131/7/13 44:40:40 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER ONE ■ Classroom Decision Making and Using Assessment 17

CONCLUSION on your state’s education department website. How does your state currently meet NCLB This chapter introduced you to basic assessment terms reporting requirements? and concepts as well as basic types and purposes of b. Has your state recently received a waiver of the educational decisions. It would not be exaggerating to original NCLB requirements, and if so, what say that appropriate assessment information should other requirements has it put in place? support everything teachers and administrators do in c. Has your state endorsed the Common Core schools. The remainder of this book is devoted to State Standards Initiative? Is your state a partici- developing the knowledge and skills you will need to pant in the PARCC or Smarter Balanced assess- accomplish that assessment well. In Chapter 2 , we turn ment consortia? What is its role in the consortium to defining instructional goals, which are the founda- (is it a governing or participating state)? tion on which formative and summative assessment, 4. Classify each of these statements as reflecting a as well as instruction, must be based. selection, classification, placement, career guid- EXERCISES ance, diagnostic/remediation, or certification decision. Defend your answers. 1. Self-reflect on a specific lesson you have taught or a. After students begin kindergarten, they are would like to teach. Make a list of the decisions given a battery of perceptual skills tests to you made (or need to make) before, during, and decide which children should receive special after this lesson. Next to each decision, identify perceptual skills training and which should how you will obtain the information needed to remain in the “regular” program. make the decision. What criteria might you use to b. A child study team decides whether each child judge the quality of each piece of information? who has been administered a series of screen- 2. Decide whether each of the following statements ing tests should be included in a particular cat- is true or false. Defend your answers. egory of disability (students with hearing a. To make evaluations, one must use measure- impairments, learning disabilities, etc.). ments. c. After a school psychologist assesses a student, b. To measure an important educational attribute local education authorities assign the student to of a student, one must use a test. the resource room, where the teacher for stu- c. To evaluate a student, one must measure that dents with learning disabilities gives the stu- student. dent special instruction each day. d. To test a student, one must measure that student. d. Each graduate of this department of education e. Any piece of information a teacher obtains is required to take and pass the state’s test about a student is an assessment. before being allowed to teach in the schools. f. To evaluate a student, one must assess that 5. Self-reflect on each of the teacher assessment student. knowledge and skills found in Appendix A. Under 3. Describe the accountability context in your state. each standard, describe the kinds of competence a. Obtain a copy of your state’s most recent you now have and those that you hope to have at accountability report, which should be available the end of this course. M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PagePage 1818 11/7/1311/7/13 4:424:42 PMPM user-s138user-s138 ~/Desktop/07:11:13/BROOKHART_PH01411_203 ~/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER 2

Describing the Goals of Instruction

KEY CONCEPTS IMPORTANT TERMS 1. Learning objectives focus instruction and affective domain assessment, and they also focus students and alignment study teachers on the knowledge and skills intended analysis, application, comprehension, evaluation, for learning. knowledge, synthesis 2. Different levels of specificity are used for cognitive domain statements of learning goals, state standards, content and performance standards, general conceptual knowledge, factual knowledge, and specific learning goals, and developmen- procedural knowledge, and metacognitive tal and mastery learning goals. knowledge 3. Sources for locating learning goals include content centered state standards (including Common Core developmental learning objective State Standards), instructional materials, and educational goals professional associations. general learning goal 4. Taxonomies of thinking skills help you get the learning objective most out of your learning goals and assessment tasks. mastery learning objectives 5. Before teaching, list and evaluate all your performance centered learning goals for a unit or course. psychomotor domain 6. Specific learning objectives should be student specific learning objectives centered, performance centered, and content standards centered. student centered 7. Align both instruction and assessment to your taxonomies of instructional objectives learning goals.

18 M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 1199 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER TWO ■ Describing the Goals of Instruction 19

IMPORTANCE OF SPECIFYING Activity 2 is the heart of the teaching process LEARNING OUTCOMES itself. Here you provide the conditions and activi- A learning objective specifies what you would ties for students to learn. These include formative like students to achieve when they have com- assessment procedures like monitoring students’ pleted an instructional segment. The goal of progress and giving them feedback on what they teaching should involve more than “covering the need to improve their achievement of their learn- material” and “keeping students actively ing targets. engaged.” The focus of your teaching should be Activity 3, evaluating whether learning has on student achievement as well as on the learning occurred, is summative assessment. Through it process. So, your learning objectives should state you and your students come to know how well what students ought to be able to do or value the learning objectives have been reached. The after you have taught them. more clearly you specify the learning targets, the Some learning objectives are cognitive, mean- more directed your teaching efforts and your stu- ing that they deal primarily with intellectual dents’ learning efforts will be. knowledge and thinking skills. For example, you These three fundamental activities are inter- may want students to read a claim made by a active rather than a straight one–two–three pro- political figure and determine whether there is cess. Setting clear learning objectives helps you evidence available to support that claim. Other plan your teaching efficiently, conduct your learning outcomes are affective, meaning that they instruction—whether whole-class, differentiated deal with how students should feel or what they by groups, or individualized—effectively, and should value. For example, you may want stu- assess student outcomes validly. Assessing and dents to value the right to vote in elections over evaluating students using clearly specified learn- other activities competing for their time. Still ing objectives provides you with information other learning objectives are psychomotor, mean- about how to guide students’ learning and how ing that they deal primarily with motor skills and effective your instruction has been. This informa- physical perceptions. For example, you may want tion, in turn, may be used to adjust your teaching, students to set up, focus, and use a microscope to plan the next instructional activities, or to bet- properly during a science investigation of pond ter specify the instructional objectives. Setting water. clear learning objectives also helps you communi- Deciding the specific objectives you expect cate them to others. students to achieve in their learning is one impor- Before you can design procedures to evaluate tant step in the teaching process. Instruction may students’ learning, you should have clearly in be thought of as involving three fundamental but mind the students’ performances you want to interrelated activities: evaluate. If you are not clear on which important learning outcomes you want to evaluate, it is 1. Deciding what students are to learn. hardly possible to make a valid assessment of 2. Carrying out the actual instruction. those outcomes. Statements of specific learning 3. Evaluating the learning. objectives are important for the following four aspects of classroom assessment: Activity 1 requires you to articulate in some 1 . The general planning for an assessment procedure way what you expect students to be able to do is made easier by knowing the specific out- after you have taught them. Usually, you do this comes you wish students to achieve. by specifying student-friendly learning targets or Selecting and creating assessment procedures by providing several concrete examples of the 2. tasks students should be able to do to demon- depend on your knowing which specific strate that the learning targets have been reached. achievements you should assess. Activity 1 informs you and the students about 3. Evaluating an existing assessment procedure is what is expected as a result of teaching and study- easier when you know the specific learning ing. Your understanding of the learning objec- objectives. tives guides your teaching and provides a 4. Properly judging the content relevance of an assess- criterion for deciding whether students have ment procedure requires you to know the specific attained the desired change. achievements you should assess (see Chapter 3 ). M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 2200 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

20 PART ONE ■ The Bases for Assessment

EDUCATIONAL GOALS, STATE Educational goals are stated in broad terms. STANDARDS, AND LEARNING They give direction and purpose to planning OBJECTIVES overall educational activities. Examples of statements of broad educational goals appear in This section discusses several closely related con- reports prepared by state departments of educa- cepts. You might find it helpful to refer to Figure 2.1 tion, local school systems, and associations such when studying them. as the National Council of Teachers of Mathemat- Educational Goals Versus Specific ics, the American Association for the Advance- Learning Objectives ment of Science, and the Association of American Schooling and other organized instruction help stu- Geographers. Here is one example of an educa- dents attain educational goals. One of the many ways tional goal: to define educational goals is that they “are those Example human activities which contribute to the functioning of a society (including the functioning of an individ- Every student should acquire skills in using scientific ual in society), and which can be acquired through measurement. learning” (Gagné, Briggs, & Wager, 1988, p. 39 ).

FIGURE 2.1 Relationships among the concepts of standards, goals, and learning objectives.

Educational goals

can be the basis for several

State standards

may be written as

Content Performance Often the same General learning standards standards as objectives

describe describe

only the subject-matter what students should do with content students the content they learned should learn

Useful for planning a course or a curriculum

Must be made into several

Specific learning objectives

that can be used for

Lesson planning Assessment planning

These specific learning targets may be either

Mastery learning Developmental objectives learning objectives

That usually can be assessed validly using a That usually require using multiple types single type (mode) of assessment (modes) of assessment for valid assessment M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 2211 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER TWO ■ Describing the Goals of Instruction 21

These types of broad goals are organized into Released in June 2010, these standards have been subject-matter areas such as mathematics and his- adopted by 45 states. At the present time, two tory. The broad goals, as well as statements of consortia of states are working on common state subject-matter area and content-specific thinking assessments and assessment systems that could processes, serve as a curriculum framework take the place of individual state assessments in within which you and other educators can define these content areas. specific learning objectives. State Standards Common Core State Standards State education agencies take the process further Most states are beginning to use the Common by publishing expected learning outcomes or Core State Standards (corestandards.org ) in standards, and your school is held accountable for English/Language Arts and Mathematics as 85% students’ achieving these particular standards. or more of their state standards in these two areas. You can obtain a copy of your state’s standards There are two main standards documents, which from your school principal, central administra- both have appendices, and two supporting docu- tion office, or your state’s education department ments: “Application of the Standards for English website. Standards are statements about what Language Learners” and “Application to Stu- students are expected to learn. Some states call dents with Disabilities.” Common Core State Standards for English these statements essential skills, learning expecta- The Language Arts & Literacy in History/Social Studies, tions, learning outcomes, achievement expectations, or Science, and Technical Subjects other names. are organized by Often there are two sets of achievement stan- grade level under the categories of reading, writ- dards. Content standards are statements about the ing, speaking, listening, and language (grammar subject-matter facts, concepts, principles, and so and usage), in two levels (K through 5 and 6 on that students are expected to learn. For exam- through 12). In K through 5, standards for ple, a standard for life science might be, “Students English Language Arts and for literacy in History/ should know that the cell nucleus is where genetic Social Studies, Science, and Technical Subjects are information is located in plants and animals.” grouped together. In 6 through 12, the English Performance standards are statements about the Language Arts standards are described separately things students can perform or do once the con- from the standards for literacy in History/Social tent standards are learned. For example, “Stu- Studies, Science, and Technical Subjects. dents can identify the cell nucleus in microscopic Example slides of various plant and animal cells.” State education departments prepare stan- The Common Core “Reading Standards for Literature dards used in schools. Local school districts are K5” include five areas, one of which is “Key Ideas required to teach students to achieve these stan- and Details.” Anchor Standard 1 in this area states: dards and are held accountable for students “Read closely to determine what the text says explic- achieving them through the state’s assessment itly and to make logical inferences from it; cite spe- system. Professional organizations can prepare cific textual evidence when writing or speaking to standards, too. These organizations try to influ- support conclusions drawn from the text.” To support ence what is taught by publicly promoting their this anchor standard, a progression of learning is expected through the grade levels. own standards. Examples of professional organizations with published standards are the National Kindergarten: With prompting and support, ask and Academy of Sciences, National Council of Teach- answer questions about key details in a text. ers of English, and National Council of Teachers Grade 1: Ask and answer questions about key details of Mathematics. Most standards from profes- in a text. sional organizations can be found on the organi- Grade 2: Ask and answer such questions as who, zations’ websites. what, where, when, why, and how to demonstrate The Common Core State Standards Initiative understanding of key details in a text. has prepared standards in English Language Arts Grade 3: Ask and answer questions to demonstrate and Literacy in History/Social Studies, Science, understanding of a text, referring explicitly to the text and Technical Subjects and in Mathematics. as the basis for the answers. M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 2222 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

22 PART ONE ■ The Bases for Assessment

Grade 4: Refer to details and examples in a text when Recognize that in a multi-digit whole number, a explaining what the text says explicitly and when digit in one place represents ten times what it drawing inferences from the text. represents in the place to its right. For example, Grade 5: Quote accurately from a text when explain- recognize that 700 4 70 5 10 by applying con- ing what the text says explicitly and when drawing cepts of place value and division. inferences from the text.

The Common Core State Standards in Mathe- These standards are a bit different from most matics also have standards for mathematical states’ English Language Arts (ELA) standards. practices. These standards apply to both elemen- They emphasize reading and comprehending tary and high school content standards. The informational texts more than most states’ prior mathematical practice standards are as follows ELA standards. The level of cognitive demand is ( CCSS, 2010 ). Students: also greater than most states’ prior standards with much more emphasis on analysis ( Porter, 1. Make sense of problems and persevere in solv- McMaken, Hwang, & Yang, 2011). The Common ing them. Core ELA Standards have three appendices. Appendix A presents research supporting key 2. Reason abstractly and quantitatively. elements of the standards and a glossary of key 3. Construct viable arguments and critique the terms. Appendix B presents sample reading reasoning of others. texts and sample performance tasks and is very 4. Model with mathematics. helpful for illustrating the standards for ELA 5. Use appropriate tools strategically. teachers. Appendix C presents samples of stu- 6. Attend to precision. dent writing with annotations, again very helpful for illustrating the standards for ELA 7. Look for and make use of structure. teachers. 8. Look for and express regularity in repeated The Common Core State Standards for Mathe- reasoning. matics are organized by grade level from K through 8 and by topic for high school. K through The Common Core mathematics standards have 8 standards are in clusters. In Grades K through one appendix, Designing High School Mathemat- 5, clusters include counting and cardinality (K ics Courses Based on the Common Core State only), operations and algebraic thinking, number Standards. and operations in base 10, measurement and Currently, states use their own standards for data, and geometry. In Grades 6 through 8, clus- subjects not covered by the Common Core State ters include ratios and proportional relation- Standards. The Next Generation Science Stan- ships, the number system, expressions and dards for Today’s Students and Tomorrow’s equations, geometry, and statistics and probabil- Workforce have just been prepared ( nextgen- ity. High school topics include number and science.org/next-generation-science-standards ). quantity, algebra, functions, modeling, geometry, and statistics and probability. As for ELA, the Common Core mathematics standards are a General Learning Goals Versus Specific bit different from most states’ previous mathe- Learning Objectives matics standards. They emphasize number sense, A general learning goal is a statement of an operations, measurement, and basic algebra expected learning outcome that is derived from much more, and consumer applications, an educational goal or state standard. General advanced algebra, and instructional technology learning goals are more specific than educational much less ( Porter et al., 2011 ). goals or standards and usually clear enough for Example general planning of a course. However, they need to be made more specific before they can Common Core State Standard in Mathematics 4. become learning objectives that you can use NBT.1 (Grade 4, cluster Numbers and Operations in when planning lessons. The following example Base 10, Standard 1): of a general learning goal might be stated for a M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 2233 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER TWO ■ Describing the Goals of Instruction 23

primary school science unit on measurement in Example the metric system: The student is able to: Example Too specific: Explain the meaning of the term cold front. Acquire the skills needed to use common instruments Better: Explain the meaning of key weather to measure length, volume, and mass in metric units. terms. Rationale: “Cold front” is only one of To teach and assess this general learning goal, several key weather terms that you may need to break it down into two or more are included in a unit. Listing a specific learning objectives. A specific learning separate learning objective for objective is a clear statement about what students each term taught in the unit are to achieve at the end of a unit of instruction. fragments the unit’s focus on general weather terminology. Here are three examples of specific learning objectives that are derived from the preceding general learning goal: A second danger is that lists of specific objec- Example tives may become too long and be ignored. Iden- tify a few of the most important learning 1. Measure the length of objects to the nearest tenth of a meter using a meter stick. objectives for each instructional unit and focus on these. 2. Measure the mass of objects to the nearest tenth of a kilogram using a simple beam balance and one set of weights. Deriving Learning Goals and Instructional 3. Measure the volume of liquids to the nearest tenth Objectives from State Standards of a liter using a graduated cylinder. After officially adopting a state’s standards, a school must make sure all students are taught When learning objectives are made more spe- and achieve those standards. Many districts use cific, the achievement you are to teach and to curriculum maps to specify how each of the learn- assess becomes clear. But beware of overspecific- ing goals in their curriculum are derived from ity. Long lists of very narrow “bits” of behavior standards. Different terms are sometimes used: can fragment the subject to be taught. The follow- “objectives,” “standards,” or even “learning tar- ing examples show learning objectives that are gets.” We reserve the term “learning targets” for too specific, along with a suggested revision: a particular student-centered function in formative assessment (see Chapter 7 ), and therefore we Example do not use the term for statements that are written for teachers’ planning purposes. The student is able to: The example below shows how specific learn- Too specific: Estimate the number of beans in ing objectives are developed from a state stan- a jar. dard and compares statements of standards, Better: Solve practical problems using general learning goals, and specific learning calculations and estimation. objectives for third-grade reading in one school Rationale: “Beans in a jar” is not the real district: objective of learning. Rather, it is but one of the many possible Example tasks that a student should complete to demonstrate State standard achievement of estimation and • Communicate well in writing for a variety of pur- calculation. The learning objective poses. should describe this less specific General learning goal achievement. • Write for narrative, persuasive, imaginative, and expository purposes. M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 2244 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

24 PART ONE ■ The Bases for Assessment

Specific learning objectives functions of cells.” But what can the student do to • Explains the difference between narrative, persua- demonstrate learning of this general goal? There sive, imaginative, and expository writing purposes. may be several answers to this question, each • Applies prewriting skills and strategies to generate phrased as a specific instructional objective and ideas, clarify purpose, and define audience before each describing what a student can do, as shown beginning to write. in the following example: • After receiving feedback on the first draft in the areas of ideas, organization, voice, word choice, Example and sentence fluency, uses the feedback to revise the draft. 1. The student can draw models of various types of • Reviews and revises the second draft for gram- cells and label their parts. matical correctness and proper use of standard 2. The student can list the parts of a cell and writing conventions. describe the structures included in each. 3. The student can explain the functions that different cells perform and how these functions are related Creating Assessments That Require to each other. Students to Use Combinations of Learning Objectives Statements of what students can do at the end It is important to create learning and assessment of instruction may be called mastery learning situations that require students to use combina- objectives. They have also been called specific tions of specific skills and knowledge to perform learning outcomes and behavioral objectives . complex tasks and solve real-life problems. Fig- ure 2.2 shows a beans-in-a-jar problem. In solving this problem, students are expected to use Mastery Learning Objectives Versus several specific skills and knowledge (listed at Developmental Learning Objectives the upper right of the figure) to accurately esti- Some skills and abilities are more aptly stated at a mate the number of beans in the jar. “Beans in a somewhat higher level of abstraction than mas- jar” is not the learning objective itself, of course. tery learning objectives to communicate that they Rather, it is only one example of many possible are continuously developed throughout life. Con- tasks in which the learning objective is to apply a sider the following examples: combination of proportional reasoning, estimation, measurement, and other skills to solve com- Examples plex problems. Notice that in this example, the most impor- 1. Combine information and ideas from several tant outcomes teachers should assess are the pro- sources to reach conclusions and solve problems. cesses and strategies students use to solve these 2. Analyze and make critical judgments about the problems. The criteria for these are listed under viewpoints expressed in passages. “criterion-referenced interpretations” in Figure 3. Use numerical concepts and measurements to 2.2 . An assessment procedure that focuses exclu- describe real-world objects. sively on the degree of correctness of students’ 4. Interpret statistical data found in material from a answers to tasks like this would be invalid because variety of disciplines. it misses assessing the processes that students use. 5. Write imaginative and creative stories. 6. Use examples from materials read to support your Specific Learning Objectives as Mastery point of view. Statements 7. Communicate your ideas using visual media such as drawings and figures. Assessment focuses on what you can see students doing. From this observation, you will infer whether they have attained the learning objec- Because of the lifelong nature of these objectives. For example, a high school biology unit on tives, they may be called developmental objectives living cells may have as a general learning goal (Gronlund & Brookhart, 2009) or developmental that students should “learn the organizations and learning objectives . M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 2255 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER TWO ■ Describing the Goals of Instruction 25

FIGURE 2.2 An example of a complex assessment task.

BEANS IN A JAR Key objectives from the Ontario Ministry of Applying rate and ratio Education and Toronto Board guidelines In the task for this benchmark, students were first • Apply ratio and rate in problem solving shown a jar filled with beans and asked to estimate • Consolidate conversions among commonly used the number of beans. They were then asked to work metric units out the number of beans more accurately using any • Collect and organize data of the following materials: a calculator, a balance • Consolidate and apply operations with whole scale and masses, a ruler, a graduated cylinder, and numbers and decimals with and without a calculator a transparent centimeter-squared grid. They were • Apply estimation, rounding and reasonableness told they could count some but not all of the beans. of results in calculations, in problem solving and in If the students did not know how to proceed, the applications evaluators suggested they weigh a small handful of • Develop facility in communication skills involving beans. The students were asked to keep an ongoing the use of the language and notation of mathematics record of their solutions. After they had solved the • Develop problem-solving abilities problem, they were asked to describe the problem and their solutions. Norm-referenced interpretations Task score Criterion-referenced interpretations 20% of the students scored 5 5 The student understands the problem and immediately begins (80% scored lower than 5) to search for a strategy, perhaps experimenting with different methods and materials before proceeding. The student monitors the solution as it develops and may check and remeasure. The student uses the materials efficiently and accurately and keeps a good record of the data. All the calculations are performed accurately and a reasonable answer is produced. The student gives a clear explanation of the solution demonstrating sound reasoning with proportions. The student takes ownership of the task and enjoys its challenge. 19% of the students scored 4 4 The student may make some false starts and may be helped (61% scored lower than 4) by the evaluator to get focused. The student may use some materials to no purpose or inaccurately, perhaps confusing volume and mass.The student reasons with proportions correctly. Although stuck at various points in the solution, the student perseveres and usually produces a reasonable answer. The student usually gives a clear explanation and enjoys the activity.

20% of the students attained 3 3 There is some confusion in one or more aspects of the solution (41% scored lower than 3; to the problem. The student may confuse units, make arithmetic the average score is 3.0) errors, or perform incorrect operations. The student may have some idea of proportionality but is unable to use it correctly. The student does not use the materials to the best advantage. The student seeks assistance from the evaluator. Although not totally confident, the student may persevere in an attempt to arrive at an answer to the problem. 24% of the students attained 2 2 The student may make a start at solving the problem but is (17% scored lower than 2) unable to complete a solution. The student may repeatedly switch methods and materials and be unable to find an effective strategy. There is considerable confusion with units and the interpretation of various measurements. The student usually guesses at the operations that should be performed with the data. The student lacks confidence and seeks a great deal of assistance from the evaluator. 17% of the students attained 1 1 The student may estimate the number of beans but gives no response or very limited response to working out the number more accurately.

Source: From John L. Clark (1992). The Toronto Board of Education’s Benchmarks in Mathematics. The Arithmetic Teacher: Mathematics Through the Middle Grades, 39 (6), pp. 51 – 55 . Reprinted with permission from The Arithmetic Teacher , copyright 1992 by the National Council of Teachers of Mathematics. All rights reserved. M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 2266 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

26 PART ONE ■ The Bases for Assessment

At first glance, it might seem that all one The Issue of Continuous Development of needs to do is to insert a “can do” phrase in front Skill The second concern, the continuous or of each of the preceding statements to transform developmental nature of these learning objec- them to mastery learning objectives. However, it tives, stems from the fact that even the simplest is not that simple. First, each statement repre- developmental objective is a matter of degree. sents a broad domain of loosely related perfor- Continuous development is possible throughout mances. Second, each statement represents skills life. All we can reasonably expect to do for a par- or abilities typically thought of as developing ticular course or unit of instruction is to identify a continuously to higher levels rather than the all- sample of specific learning outcomes that repre- or-none dichotomy implied by the mastery learn- sent degrees of progress toward the objectives. ing objectives. The essential concern here is that the skills represented by these learning objectives are complex, The Problem of a Broad, Heterogeneous the number of tasks that can be used to demon- Domain Consider Developmental Learning strate learning is vast, and each represents goals Objective 2 in the previous list. Now, think about to work toward continuously rather than to mas- questions you could ask students to assess how ter completely ( Gronlund & Brookhart, 2009). well they have achieved this learning objective. Your questions need to require students to analyze Teaching and Assessing Developmental Learning a reading passage and make inferences based on Objectives One way to begin designing instruc- information in it. The example below shows three tion and assessing progress toward developmental possible questions. These questions are passage- objectives is to list several specific learning out- based items from the National Assessment of Educa- comes for each objective. The outcomes should rep- tional Progress civics test. The numbers in the resent the key performances expected of a student at brackets are the percentage of twelfth-grade stu- a particular grade level. This principle is illustrated dents who answered each question correctly. in the following example, which clarifies a broad instructional objective in science by listing several specific learning outcomes that support it: Example

1. In what way does the article show one of the Example strengths of federalism? [32%] (2006-12C7, Developmental learning Interprets and uses Boyle’s question 9) objective: law to explain phenomena 2. In what fundamental way do the two quotes above and solve problems. show different understandings of the rights of citizens? [51%] (2006-12C7, question 3) 3. The events at Central High School in Little Rock Specific learning outcomes clarifying this showed that . . . [60%] (2006-12C5, question 17) developmental objective: 1. States a definition of Boyle’s law. 2. States the domain to which Boyle’s law applies. You can see that each question refers to a dif- 3. Describes the relationship between Boyle’s law ferent passage with different viewpoints and Charles’s law. expressed. Further, the percentage of students 4. Uses Boyle’s law to explain an observation in a lab answering one question is quite different from experiment. the percentage answering another. Studies of 5. Appropriately analyzes a new (to the student) situ- these types of questions show that those who ation in terms of Boyle’s law. answer one question right are not necessarily the 6. Solves a new problem or makes an appropriate same students who get another question right. choice for a course of action, taking into account We can conclude from this that Developmental the implications of Boyle’s law. Learning Objective 2 represents a broad domain of reading passages and that mastering one part of the domain does not mean mastering another. Although this list of six specific outcomes This is the case with developmental learning might be made longer, the six would likely be objectives like those listed previously. considered adequate for describing what is meant M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 2277 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER TWO ■ Describing the Goals of Instruction 27

by “interpreting and using Boyle’s law” at the Taxonomies of instructional learning objec- end of an introductory course in high school tives are highly organized schemes for classifying physics. Specific tasks could then be prepared for learning objectives into various levels of com- assessing achievement of the six specific out- plexity. Generally, educational learning objec- comes. Some tasks might assess only one of these tives fall into one of three domains, although a learning outcomes; others could require a student complex performance may involve more than to use several of these outcomes in combination. one of them. A student’s overall score could be interpreted as 1 . Cognitive domain: Objectives focus on knowl- indicating the degree to which a student has edge and abilities requiring memory, thinking, acquired the ability to interpret and use Boyle’s and reasoning processes. law, rather than as a “mastery/nonmastery” description. 2. Affective domain : Objectives focus on feelings, interests, attitudes, dispositions, and emotional states. SOURCES FOR LOCATING LEARNING 3. Psychomotor domain: Objectives focus on OBJECTIVES motor skills and perceptual processes. You may find lists of learning objectives in Learning objectives within each domain may instructional materials and teachers’ manuals, be classified by using a taxonomy for that domain. local and state curriculum frameworks, the Com- Because there is more than one way to define a mon Core State Standards, state websites contain- classification scheme, several different taxono- ing performance standards, reports of the mies have been developed for sorting learning National Assessment of Educational Progress, objectives in a given domain. Only two of these books on teaching methods, manuals accompa- taxonomies for the cognitive domain are nying tests (especially criterion-referenced tests), described here. Other cognitive domain taxono- and reports from educational associations. More mies are summarized in Appendix D . Chapter 6 than likely you will have to adapt the learning will discuss using taxonomies to develop an objectives you find in these sources to your own assessment plan. The other chapters in Part II dis- situation. Nevertheless, these sources do provide cuss creating tasks to assess learning objectives at a starting place: It is much easier to adapt and different taxonomy levels. revise learning objectives statements than to write them without any assistance. Also, a learning objective often will cut across COGNITIVE DOMAIN TAXONOMIES several lessons or subject areas. The objective of Taxonomies are not teaching hierarchies. Their being able to use library and print resources to only purpose is to classify various learning objec- obtain information for a report, for example, is tives and assessment tasks. For example, you likely to be common to social studies, mathemat- should not teach “recall” first and “higher order ics, and language arts curricula. The taxonomies thinking” second. If you did that, younger and in the next section were created so that each cate- lower-achieving students would be doomed to gory would apply across several curricular areas. spend all their time on drill. Use the taxonomy to help you explore each learning objective at several levels. A taxonomy gives you a tool to judge TAXONOMIES OF LEARNING whether you have taught and assessed a wide OBJECTIVES enough range of thinking skills. Including a wide Simply writing learning objectives “off the top of range of thinking skills in an assessment usually your head” can be frustrating because a seem- improves its validity. ingly endless number of possible objectives exist. It is also important to recognize that student Further, you are likely to write first those objec- performance on complex tasks involves using sev- tives that have a very narrow focus, specify con- eral thinking skills at the same time. It is possible, tent topics, and represent lower level cognitive therefore, to classify a given learning objective or skills. A taxonomy can help you bring to mind assessment task into more than one taxonomy cat- the wide range of important learning objectives egory ( Krathwohl, 2002 ). For most classroom pur- and thinking skills. poses, classify each learning objective into the M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 2288 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

28 PART ONE ■ The Bases for Assessment

category that represents the thinking skill that is 5 . Synthesis involves the putting together of ele- (a) most prominently used or (b) the main intent ments and parts so as to form a whole. This of the learning objective or assessment task. Then process involves working with pieces, parts, use the classification to decide if some important elements, and so on, arranging and combining skills have received too little or too much atten- them in such a way as to constitute a pattern or tion in your teaching and assessment. structure not clearly there before (p. 206 ). 6 . Evaluation requires judgments about the value Bloom’s Taxonomy of material and methods for given purposes, quantitative and qualitative judgments about The Taxonomy of Educational Objectives: The Classi- the extent to which materials and methods sat- fication of Educational Goals, Handbook I: Cognitive isfy criteria, and the use of a standard of Domain (Bloom, Englehart, Furst, Hill, & appraisal. The criteria may be determined by Krathwohl, 1956 ) had an enormous influence on the student or supplied to him (p. 207 ). how we think of educational goals and teaching practice. This taxonomy is a comprehensive outline of a range of cognitive abilities that you might Revised Bloom’s Taxonomy teach, classified into six major headings arranged Relationship of the Revision to the Original The from simple to complex. original Taxonomy of Educational Objectives has The six main headings of the original Bloom’s been revised as A Taxonomy for Learning, Teaching, taxonomy are Knowledge, Comprehension, and Assessing: A Revision of Bloom’s Taxonomy of Application, Analysis, Synthesis, and Evaluation. Educational Objectives ( Anderson et al., 2001 ). The 1 . Knowledge involves the recall of specifics and revised taxonomy improves on the original by universals, the recall of methods and processes, adding a two-dimensional framework into which or the recall of a pattern, structure, or setting. you may classify learning objectives and assess- For measurement purposes, the recall situation ment items. The two dimensions are the Knowl- involves little more than bringing to mind the edge Dimension and Cognitive Process Dimension. appropriate material (p. 201 ). The Cognitive Process Dimension is very much like the original Bloom’s taxonomy . Its cate- 2 . Comprehension represents the lowest level of gories are Remember, Understand, Apply, Ana- understanding. It refers to a type of under- lyze, Evaluate, and Create. The cognitive standing or apprehension such that the indi- processes of Synthesis and Evaluation from the vidual knows what is being communicated old taxonomy have switched their order and and can make use of the material or idea being become Evaluate and Create in the new taxon- communicated without necessarily relating it omy . This change makes sense, in that evaluation to other material or seeing its fullest implica- requires making a judgment after analyzing tions (p. 204 ). something against criteria while creating requires 3. Application involves the use of abstractions in putting together something new. The definitions particular and concrete situations to solve new or of the Cognitive Process Dimension categories novel problems. The abstractions may be in the remain like the original Bloom’s taxonomy defini- form of general ideas, rules of procedure, or gen- tions presented above, with one exception: eralized methods. The abstractions may also be Knowledge. technical principles, ideas, and theories, which Bloom’s original Knowledge category has must be remembered and applied (p. 205 ). been divided into two parts: the Knowledge 4 . Analysis involves the breakdown of a commu- Dimension and the Cognitive Process category nication into its constituent elements or parts Remember. The Knowledge Dimension has four such that the relative hierarchy of ideas is subcategories: Factual Knowledge , Conceptual made clear and/or the relations between the Knowledge, Procedural Knowledge, and Meta- ideas expressed are made explicit. Such analy- cognitive Knowledge . The Knowledge Dimen- ses are intended to clarify the communication, sion contains the type of content a learning to indicate how the communication is orga- objective refers to: factual knowledge, conceptual nized and conveys its effects, and to identify its knowledge, procedural knowledge, or metacog- basis and arrangements (p. 205 ). nitive knowledge. M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 2299 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER TWO ■ Describing the Goals of Instruction 29

FIGURE 2.3 Taxonomy Table from the revised taxonomy. Cognitive Process Dimension

Knowledge Dimension 1. Remember 2. Understand 3. Apply 4. Analyze 5. Evaluate 6. Create

A. Factual Knowledge

B. Conceptual Knowledge

C. Procedural Knowledge

D. Metacognitive Knowledge

Source: Anderson, Lorin W.; Krathwohl, David R.; Airasian, Peter W.; Cruikshank, Kathleen A.; Mayer, Richard E.; Pintrich Paul R.; Raths, James; Wittrock, Merlin C., A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives, Complete Edition, 1st ed., © 2001, pp. 28 , 46 , 67 – 68 . Reprinted and electronically reproduced by permission of Pearson Education, Inc., Upper Saddle River, New Jersey.

The Taxonomy Table A two-dimensional table the taxonomy and that they may also overlap into is constructed to describe the location of a learn- some of the other cells. ing objective and its corresponding assessment on both dimensions simultaneously (see Figure 2.3 ). Example The figure shows 24 cells, each defined by one Knowledge and one Cognitive Process subcate- Remember Factual Knowledge [1A] gory. Note that the subcategories of the Knowl- Sample learning Recall the main characters edge Dimension are lettered, whereas the objective: in each of the short stories subcategories of the Cognitive Process Dimen- read and what they did. sion are numbered. As a shortcut, we can refer to Sample assessment (1) List the names of all of a particular cell by its letter and number. Thus, a items: the characters in “Witch’s learning objective that requires students to Forest.” remember some factual knowledge is placed in (2) In “Witch’s Forest,” what cell 1A. did Sally do when her mother refused to let her go Classifying Learning Objectives and Assess- into the forest? ment Items Suppose you are teaching students to understand the elements that authors use when writing short stories. Suppose the short stories you select all concern people’s personal problems Example and that the characters in these stories handle their personal problems inappropriately. The Understand Conceptual Knowledge [2B] sample learning objectives and questions that fol- Sample learning Explain the main ideas and low may help you direct your assessment plans. objective: themes of the short stories Later chapters will detail how to design assess- that we read. ment tasks. At this point, we are studying only Sample assessment Write using your own words the range of thinking skills that should be taught item: what “Witch’s Forest” was and assessed. Also, remember that the examples all about. are classified into the most appropriate cell(s) in M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 3300 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

30 PART ONE ■ The Bases for Assessment

Example Example Apply Conceptual Knowledge [3B] Create Using Conceptual and Procedural Knowledge [6B,C] Sample Relate the personal learning problems of the characters in the Sample Describe, across all of the stories objective: short stories that we read to prob- learning read, the general approach that the lems that real people face. objective: characters used to resolve their Sample Are the problems Sally had with her problems unsuccessfully. assessment mother in the story similar to the Sample So far we have read “Witch’s For- item: problems you or someone you assessment est,” “Dog Long Gone,” “Simon’s know has with his or her mother? item: Top,” and “Woman With No Man- Explain why or why not. ners.” In every story, one character was not able to solve the personal problem he or she faced. What Example were the ways these characters tried to solve their problems? What Analyze Procedural Knowledge [4C] do these unsuccessful ways to solve problems have in common? Sample Identify the literary devices that learning authors use to convey their objective: characters’ feelings to the reader. Figure 2.4 shows how learning objectives in Sample In “Witch’s Forest,” Sally was upset science and social studies may be classified in the assessment with her mother. In “Dog Long revised taxonomy. The value of such a taxonomy item: Gone,” Billy was upset with his brother. What words and phrases is that it calls your attention to the variety of abili- did the authors of these two stories ties and skills toward which you can direct use to show how upset these instruction and assessment. characters were? Explain and give examples. Advantage of the Revised Taxonomy The advantage of the revised taxonomy is that it allows you to consider a broader range of learn- Example ing objectives than the original one-dimensional taxonomy. If you classify your learning objec- Evaluate and Create Using Conceptual and Procedural tives, your assessment items, and your teaching Knowledge [5B,C;6B,C] activities into the Taxonomy Table shown in Sample (1) Develop one’s own set of three Figure 2.3 , you can immediately see the types of learning or four criteria for judging the qual- knowledge and thinking on which your instruc- objectives: ity of a short story. [6B,C] tional unit focuses. Not every unit should have (2) Use the three or four criteria to learning objectives and assessments in every one evaluate several new stories that of the 24 cells, of course. But over the semester, were not read in class. [5B,C] your teaching should address and evaluate stu- Sample (1) So far we have read four short dents’ learning in all (or nearly all) of them. assessment stories. What are three or four dif- items: ferent traits that make a story high Different Modes of Assessments quality? Use these traits to develop for Different Taxonomy Levels three or four criteria that you could use to evaluate the quality of any Learning objectives classified in the first three cog- short story. nitive categories are more easily assessed with (2) Read the two new short stories short-answer, true-false, multiple-choice, or assigned to you. Use the criteria you matching test items. Learning objectives classified developed to evaluate these two in the last three cognitive categories might be par- stories. Evaluate each story on every tially tested by such item formats, but their assess- criterion. Summarize your findings. ment usually requires a variety of other procedures such as essay questions, class projects, observing M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 3311 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER TWO ■ Describing the Goals of Instruction 31

FIGURE 2.4 How different outcomes for science and social studies may be classified using the Anderson et al. revised taxonomy.

Anderson et al. Category Science Social Studies

Remember • Recall the names of parts of a flower 1A • List known causes of the Civil War 1B • Identify and label the parts of insects 1A • Recall general principles of migration of peoples of • List the steps in a process 1C Africa 1B Understand ••Explain the digestive processes in one’s own 2B Explain the meaning of technical concepts in 2B words one’s own words • Give examples of propaganda usage from current events 2B

Apply • Use scientific principles to make a simple 3B,C • Use specified principles to explain 3B,C machine current events ••Find real examples of igneous rock and mineral Carry out a survey and collect data from the field 3C formations 3B Analyze • Show how scientific principles or • Identify the credible and noncredible claims of an 4B,C concepts are applied in the design of advertisement for clothing 4B,C a refrigerator • Show the different component parts of a political 4B speech

Evaluate • Use criteria or standards to evaluate the 5B,C • Use a specific set of criteria to evaluate several 5B conclusions drawn from the research findings political speeches

Create • Determine what the rule is that underlies • Show the similarities among several schools of 6B the results obtained from several experiments or social thought investigations 6B • Develop plans for peace between two countries 6B,C

performance in labs, and portfolios. Learning taxonomy. A disadvantage of using a condensed objectives at more complex thinking levels require version of the taxonomy is that teachers may stop students to actually produce or create something, trying to teach learning objectives in the Evaluate rather than simply to answer questions. Carefully and Create categories because, after condensing, reading the various subcategories of the taxonomy Apply and Analyze—generally easier skills to in Appendix D should make this more apparent. assess—will be in the same category as Evaluate and Create. Condensing the Taxonomy Because it is sometimes difficult for teachers to classify learning objectives into all six cognitive categories, some Webb’s Depth of Knowledge Levels schools have opted to use a shorter version of the As part of the No Child Left Behind legislation, Bloom taxonomy. For example, some have states have been required to show evidence that reduced it to three categories: Remember (or the state accountability tests are well matched Knowledge), Understand (or Comprehension), with the state standards they are intended to and Higher-Order Thinking. The “Higher-Order measure. Alignment studies are conducted to Thinking” category collapses Apply, Analyze, document this match. Most alignment studies are Evaluate, and Create learning objectives into one concerned with both the content and the level of group. Other schools formed three categories cognition represented in the test. One widely somewhat differently: Remember and Under- used method for conducting alignment studies stand (Knowledge and Comprehension), Apply, was developed by Webb (1997). Expert review and Higher-Order Thinking (including Analyze, data are often sought to document whether the Evaluate, and Create). The advantage of these test items and standards match in content, condensations is that they eliminate the need for whether they match in level of cognitive chal- struggling with how to classify learning objec- lenge, and whether they match in relative empha- tives into one of the top three categories of the sis or weight given to each standard. As part of M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 3322 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

32 PART ONE ■ The Bases for Assessment

this methodology, Webb developed a taxonomy Appendix D of this book. Which one should you called Depth of Knowledge ( Webb, 1997 ; Webb, use? That depends on whether this is a personal Herman, & Webb, 2007). decision for use in your classroom only or a more The Depth of Knowledge levels are four broad general decision about a taxonomy that will be categories that can apply across different content used throughout your school system. areas. Their general descriptions are as follows ( Webb, Herman, & Webb, 2007 ). ✔ Checklist Level 1—Recall Criteria for Selecting a Taxonomy of Cognitive Learning Objectives ■ Recall a fact, definition, or simple procedure ■ Apply a simple algorithm or formula (solve 1. Completeness: To what degree can your major learn- a one-step problem) ing objectives be classified within this taxonomy? Not at all Somewhat To a great extent ■ Example verbs: identify, recall, recognize, use, and measure 2. Point of view: To what extent can this taxonomy be used as a platform for explaining your teach- Level 2—Skill/Concept ing methods or your curriculum characteristics ■ Make a mental decision requiring more than to others? rote action or recall Not at all Somewhat To a great extent ■ Interpret information, read a graph, notice 3. Reform: To what extent can this taxonomy help patterns you evaluate your curriculum and learning objec- ■ Solve a two-step problem tives and revise as needed? ■ Example verbs: classify, organize, estimate, Not at all Somewhat To a great extent make observations, collect and display data, 4. Simplicity: How easy is it for parents, teachers, and compare data education officials to understand this taxonomy? Level 3—Strategic Thinking Not at all Somewhat To a great extent 5. Reporting: How useful is this taxonomy in organiz- ■ Reason, plan, use evidence, make conjec- ing reports on assessment results for individual tures students, educational officials, government offi- ■ Explain one’s thinking cials, or the public? ■ Perform tasks that have multiple ways to Not at all Somewhat To a great extent give good responses ■ Example verbs: explain, plan, reason To choose among the various taxonomies, Level 4—Extended Thinking apply the practical criteria in the checklist to ■ Use complex reasoning, perform extended judge each taxonomy or classification scheme tasks over time you are considering. If the decision is a personal ■ Design and conduct experiments one for a single classroom, then not all criteria ■ Draw conclusions or synthesize ideas across may apply. multiple texts ■ Example verbs: design, make connections, EVALUATING THE LEARNING synthesize OBJECTIVES OF A COURSE OR UNIT Appendix D presents more specific examples of Prepare a complete list of the learning objectives work at each of the four levels in specific subject for a unit before you teach it. A complete list is areas: English language arts, mathematics, sci- not necessarily long. You can use this checklist to ence, and social studies. evaluate your list of learning objectives: ✔ Checklist Choosing a Taxonomy We have discussed two different schemes for Checklist for Evaluating a List of Learning Objectives classifying cognitive learning objectives, Bloom’s for a Course or Unit and Webb’s. There are many more taxonomies 1. Are all the learning objectives appropriate for stu- that we have not discussed, some of which are in dents’ educational level? M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 3333 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER TWO ■ Describing the Goals of Instruction 33

2. Is the list of learning objectives limited to only the Example important outcomes for the course or unit? 3. Are all the learning objectives consistent with your Poor: Provide the opportunity for students to express state’s published learning standards? their opinions in classroom discussions about why peace is so difficult to attain. 4. Are all the learning objectives consistent with your local school’s philosophy and general goals? 5. Can all the learning objectives be defended by currently accepted learning principles? The problem with the preceding statement is that it is an activity statement for teachers rather 6. Can all the learning objectives be taught within the than a learning objective for students. You may time limits of the course or unit? “provide the opportunity for students to express 7. Can all the learning objectives be taught with avail- their opinions,” yet each student may not express able teaching resources? his or her opinion. Learning objectives need to be student centered if they are to be the basis for crafting assessment procedures. Thus, you should say: HOW TO WRITE SPECIFIC LEARNING Example OBJECTIVES To be useful for classroom instruction and assess- Better: A student will express his or her opinion in ment (Gronlund & Brookhart, 2009) learning classroom discussions about why peace is so diffi- objectives must be: cult to attain. 1. Student centered: Objectives focus on the student. Student-centered learning objectives allow 2 . Performance centered : Objectives are worded you to decide whether the students actually have in terms of what a student can perform after achieved what you intended from the lesson. the required learning experiences. 3 . Content centered : Objectives state the specific Performance Centered content to which the student should apply the A learning objective should state a performance— performance. that is, an observable activity. You can accomplish this kind of statement by including an action Student Centered verb that specifies a student performance. Because instruction focuses on changes in stu- To help beginners write learning objectives dent performance, learning objectives should that describe students’ performances, Figure 2.5 describe student performances. It is not unusual, lists further examples of various action verbs. however, for some teacher guides, curriculum These verbs are organized according to the cogni- frameworks, and other materials to contain tive dimension of the Anderson et al. revised tax- statements that do not focus on the student. onomy. When verbs such as these are used in Consider this statement: statements of learning objectives, the objectives

FIGURE 2.5 Action verbs to use when writing learning objectives. Remember Define, describe, explain, identify, label, list, match, name, outline, reproduce, select, state Understand Convert, describe, distinguish, estimate, extend, generalize, give examples, paraphrase, rewrite, summarize Apply Apply, change, classify (examples of a concept), compute, demonstrate, discover, modify, operate, predict, prepare, relate, show, solve, use Analyze Analyze, arrange, associate, compare, contrast, infer, organize, solve, support (a thesis) Evaluate Appraise, compare, conclude, contrast, criticize, evaluate, judge, justify, support (a judgment), verify Create Classify (infer the classification system), construct, create, extend, formulate, generate, synthesize M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 3344 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

34 PART ONE ■ The Bases for Assessment

will usually satisfy the second criterion of express- Figure 2.6 suggests some verbs that maintain ing observable student performance. this balance and illustrates other verbs that are A balance is necessary between verbs that are too specific or too broad to make useful state- too broad (and thus imply too many nonequiva- ments of learning objectives. lent performances) and those that are too specific (and which are often just ways of marking Content Oriented answers). Consider this learning objective, which A learning objective should indicate the content is stated too specifically: to which a student’s performance is to apply. The Example following learning objective is poor because it lacks a reference to content: Poor: The student is able to put an X on the picture of the correct geometric shape (circle, triangle, rectangle, Example square, or ellipse) when the name of the shape is given. Poor: The student is able to write definitions of the important terms used in the text. The main intent of such an objective is to select or identify the correct shape, not just to make Xs. Any response that indicates the student has cor- To modify this learning objective you need to rectly identified the required shape is acceptable. include a reference to a specific list of “important Thus, the learning objective should be written as: words” or in some other way describe them: Example Example

Better: The student is able to identify a picture of a Better: The student is able to write definitions of the geometric shape (circle, triangle, rectangle, square, terms listed in the “Important Terms” sections of or ellipse) when the name of the shape is given. Chapters 1 through 5 of the textbook.

FIGURE 2.6 Action verbs sometimes used in learning objectives.

Specific but acceptable verbs add, total describe match rename alphabetize divide measure rephrase choose draw multiply select complete, supply explain name sort, classify construct, make identify order, arrange state convert label pick out subtract, take away count list regroup weigh delete

Too broad, unacceptable verbs apply examine interpret respond deduce generate observe test do infer perform use

Too specific, essentially indicator verbs check draw a line between put a mark on underline circle draw a ring around put an X on write the letter of color the same as put a box around shade write the number of

Toss-up verbs, requiring further clarification answer contrast differentiate give collect, synthesize demonstrate discriminate locate compare determine distinguish predict M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 3355 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

CHAPTER TWO ■ Describing the Goals of Instruction 35

If you do not refer to content in your learning Aligning Assessments to Developmental objective, you will be uncertain whether an assess- Learning Objectives ment task is valid for evaluating the student. For As is often the case, developmental learning example, the assessment may require students to objectives define a broad domain of performance define words that, although in the text, may be application. To ensure the validity of your class- unimportant. Without knowing the content, it is room assessment, you may need to assess the difficult for anyone to determine what, if any- same learning objective in several different thing, was learned. ways. For example, you might assess writing achievement both by scoring several samples of students’ written assignments and by using a ALIGNING ASSESSMENT TASKS grammar and usage test. The test provides the opportunity to assess grammar and usage that WITH LEARNING OBJECTIVES might not appear in the natural course of the Chapters 6 through 14 discuss the details of cre- student’s writing, but that may well be part of ating high-quality assessments. Here we wish the learning objective. Observing a student’s simply to point out that the basic purpose of natural writing habits permits you to infer how any assessment is to determine the extent to well the student is likely to use language in typi- which each student has achieved the stated cal writing situations. Using both procedures learning objectives. Although this purpose increases how comprehensively you assess the sounds straightforward, it is not always an easy student’s writing ability and the validity of your criterion to meet. The validity of your assess- evaluation. ment results determines the quality of your Another reason for using more than one evaluation. Validity has many aspects (see assessment procedure is to obtain more reli- Chapter 3 ); here we discuss validity only in rela- able results. Your subjective evaluation of a tion to matching assessment tasks to learning student’s written essay on a topic might be objectives. supplemented by a test made up of more objectively scored items. Combining the less reliable information about the student’s written work Aligning Assessments to Mastery Learning (that is, your subjective evaluation) with the Objectives more reliable information (the objectively scored test) yields a more reliable overall eval- The specific tasks or procedures you use in an uation result. Reliability is discussed in more assessment should require the student to display detail in Chapter 4 . the skill or knowledge stated in the learning objective. For instance, if the main intent of your learning objective is for a student to build an apparatus, write a poem, or perform a physical Aligning Assessments to State skill, your assessment procedure must give the Standards student the opportunity to perform . Assessment Earlier in this chapter, we showed how you can procedures that require a student only to name derive your learning objectives from your state’s the parts of an apparatus, to analyze an existing standards. It is important that you maintain con- poem, or to describe the sequence of steps sistency by aligning your classroom assessments needed for performing a physical skill do not as well as the learning objectives with the state’s require the performance stated in these learning standards. First, align your assessments with the objectives. Therefore, they would be invalid for learning objectives that you derive from the assessing them: They are not aligned to the state’s standards (in the manner we showed ear- learning objectives’ main intents. A very basic lier). Second, ensure that your assessments requirement for the validity of classroom assess- match the span of content covered by the stan- ment procedures is that the assessment proce- dards, the depth of thinking implied by the stan- dures should be aligned with the intentions of dards, the topical emphasis in the standards, the specific learning objectives that you include and the same types of performances as are speci- in your assessment plan. fied in the standards. M02_BROO6495_07_SE_C02.inddM02_BROO6495_07_SE_C02.indd PPageage 3366 111/7/131/7/13 44:42:42 PPMM uuser-s138ser-s138 ~ ~/Desktop/07:11:13/BROOKHART_PH01411_203/Desktop/07:11:13/BROOKHART_PH01411_203

36 PART ONE ■ The Bases for Assessment

CONCLUSION Select one unit you are teaching or plan to teach in the future that is based on one or more of these Well-conceived learning objectives are the foundation standards. Explain what you would need to do to for both instruction and assessment. They are also the align your classroom learning objectives and stu- means by which instruction and assessment are coor- dent assessments with the standard(s) you dinated. Such coordination or alignment is the basis selected. Summarize the results and report them for valid classroom assessment. In Chapter 3 , we con- to your class. sider the broader concept of validity for both class- 4. Decide whether each learning objective listed here room and large-scale assessment. belongs to the cognitive, affective, or psychomotor domain. Does the performance of each learn- EXERCISES ing objective require some use of elements from 1. Write three specific learning objectives for a lesson domains other than the one into which you classi- you plan to teach. Explain how each objective fied it? Which one(s)? Explain why. Does this meets the three criteria: student centered, perfor- mean you should reclassify that objective? mance centered, and content centered. Explain. 2. Following are three learning objectives. Decide a. The student is able to adjust a television to get whether each is a mastery learning or a develop- the best color resolution. mental learning objective. Explain your choices. b. The student demonstrates knowledge of parlia- a. The student is able to take the square root of mentary law by conducting a meeting without any number using a handheld calculator. violating parliamentary procedures. b. The student is able to determine whether the c. The student contributes to group maintenance thesis of the argument is supported adequately. when working with classmates on a science c. When given data, the student is able to con- project. struct a graph to describe the trend in the data. d. The student makes five baskets in 10 attempts 3. Obtain a copy of the Common Core State Standards on the basketball court while standing at the in either English/language arts or mathematics. foul line.