AEA 267 Assessment for Learning: Training/Support Team

AEA 267 Assessment for Learning: Training/Support Team Joe Kremer Jon McKenzie

Assessment for Learning Archdiocese School Leadership Professional Learning

October 1, 2015

Big Idea 1

Implementation of the Assessment for Learning process can positively impact teaching and learning.

Learning Goals

Gain a basic understanding of: • The research-base regarding the Assessment for Learning/Formative Assessment process. • The critical attributes of the Assessment for Learning/Formative Assessment process and what it might look like as they implemented in a classroom.

Success Criteria

• I can summarize the research-base surrounding Assessment for Learning/Formative Assessment practices • I can briefly describe how each of the attributes support teaching and learning. • I can begin to apply this information in my day to day teaching/coaching responsibilities.

Assessment for Learning Attributes Assessment for Learning Attributes –

–“What I Know” (before) “What I Now Know” (after) 1. Learning Progressions

2. Learning Goals and Success Criteria

3. Eliciting Evidence of Learning

4. Self and Peer Assessment 2

5. Descriptive Feedback

6. Collaboration

Formative Assessment (Iowa’s Definition)

Formative Assessment is a process used by teachers and students as part of instruction that provides feedback to adjust ongoing teaching and learning to improve students’ achievement of core content. As assessment for learning, formative assessment practices provide students with clear learning targets, examples and models of strong and weak work, regular descriptive feedback, and the ability to self-assess, track learning, and set goals.

Effect Size- is a way of quantifying the size of the difference between two groups. It is particularly valuable for quantifying the effectiveness of a particular intervention, relative to some comparison. An effect size of 0.2 to 0.3 might be a "small" effect, around 0.5 a "medium" effect and 0.8 to infinity, a "large" effect.

For example, an effect size of .4 represents change from the 16th percentile to the 32nd percentile. An effect size of .7 would result in change from the 35th percentile to the 61st percentile..  Stats 101 - Percentage Interpretation of Effect Sizes • ES = 0.00 means that there was no difference between the treatment group and the control group.  • ES = 0.40 means that the average treatment participant outperformed 65% of the control participants • ES = 0.85 means that the average treatment participant outperformed 80% of the control participants 



Meta-Analysis results for selected Teacher/School Instructional Influences (Hattie, 2009, 2012 and 2013)

Response to Intervention 1.07 The effect size of 0.40 sets a level where the Teachers Working Together, as Evaluators of their impact .93* effects of innovation enhance achievement in Providing formative evaluation (program evaluation) .90 such a way that we can notice real-world Micro teaching .88 differences, and this should be a benchmark of Moving from what students know to explicit success criteria .77* such real-world change. (Hattie, 2009) 3 Comprehensive interventions for learning disabled students .77 Teacher clarity (clear learning goals communicated) .75 Reciprocal teaching .74 Feedback .73 Hattie, J. (2009). Visible learning: a synthesis of over 800 Maximize feedback to teachers about their impact .72* meta-analysis relating to achievement. New York: Teacher-student relationships .72 Routledge. Errors and trust are welcomed as opportunities to learn .72* Hattie, J. (2012). Visible learning for teachers: maximizing Getting the proportions of surface to deep correct .71* impact on learning. New York: Routledge. Spaced vs. mass practice .71 Hattie, J. (2013). Ted Talk. Sweden * Meta-cognitive strategies .69 The Goldilocks principles of challenge, and deliberate practice .60* to attain these challenges

What does the research say about the effectiveness of Assessment for Learning / Formative Assessment as an educational practice? (Summary of Selected Research Surrounding Formative Assessment)

“The research reported here shows conclusively that formative assessment does improve learning.” (Assessment in Education, 1998)

When teachers effectively used data and information from Formative Assessment activities, effect size gains ranged from .4 to . 7 on large-scale assessments; The student gains in learning triggered by formative assessment were “amongst the largest ever reported for educational interventions.” (Black and Wiliam,1998).

Formative Assessment is particularly effective in narrowing the achievement gap while raising overall achievement, especially with students who previously have not done so well in school (Black and Wiliam,1998).

Students who know the learning objectives, assessment criteria and have time to reflect on their work, had greater gains than students that did not; these gains were found to be even more impressive for students considered to be at-risk (Fontana &

Fernandes, 1994; Frederickson & White, 1997).

Black and Wiliam (1998a) found that ―…irrespectively of the particular approach adopted, we have not come across any reports of negative effects following an enhancement of formative assessment practices. Usually new initiatives/strategies will result in a dip in student learning as the teacher learns to hone the strategy or instructional practice and as the students become accustomed to it. Then after a period of time the increases in student learning occur. What Black and Wiliam found was that no dip in student learning occurs as teachers begin to implement formative assessment practices.

Another strong finding of the research being conducted on formative assessment practices is related to the frequency of its use by classroom teachers. In 1991, Robert Bangert-Drowns, James Kulik, and Chen-Lin Kulik analyzed findings from 29 studies on the frequency of assessments. They found that even a single formative assessment practice used in a 15-week unit of study resulted in an effect size gain of 0.34 and the more frequently the practices are used the greater the effect size.

Fuchs and Fuchs (1986) discovered a similar gain in effect size with multiple uses of formative assessment practices. They found that if a teacher provided two formative assessment practices per week in the form of curriculum-based data collection, they resulted in an effect size of 0.85, or a percentile gain of 30 points on a standardized test. Additionally, if data were visually depicted, the chances were greater that corrective action would be prompted and any resulting gains were greater.

John A. Ross, Gary Hogaboam, and C. Rolheiser (2002) examined the effects of training 5th and 6th grade students to self assess and compared it to student achievement in mathematics problem solving. They found—in a study of over 500 students—that those students who practiced self assessment strategies as a part of formative assessment practices in the classroom outperformed students who did not use self-assessment strategies (effect size of 0.40).

In a study comparing achievement gains in classrooms using four different types of teacher feedback, the greatest pre-and post- test achievement gains were obtained by students who received comments from teachers compared to those who received numerical grades, praise, or no feedback (Butler, 1988).

Formative Assessment can improve learning when it provides students with feedback about specific qualities of their work, and about how to improve (Black and William, 1998; Crooks, 1998; Kluger and DeNisi, 1996; Hattie and Timperly, 2007; Natriello, 4 1987; Rea-Dickens, 2001; Turnstall and Gipps, 1996).

When regular testing (assessment) and feedback are coupled with a practice of communicating high expectations to students (age 8-14), achievement was often found to be higher than for students for which these elements were not present (Whiting, etal, 1995).

Bergen and colleagues (1991), investigated the early acquisition of skills for kindergarten students from disadvantaged home backgrounds. Those students who had a teacher who frequently collected data to ascertain the student’s status on a set of learning goals/standards and actively used these data to adjust or differentiate their learning experiences were less likely to be identified with a particular learning need or placed in special education.

Students who are provided with the opportunity to self-assess/reflect on their progress demonstrated higher scores on outcome measures of skill, motivation and self-efficacy than students who did not have opportunity to self-assess/reflect upon their work regardless if they were provided with learning goals which stressed the quality of their work or performance goals which stressed the quantity of their work. When students are provided with specific learning goals which stressed work quality, these outcome measures were even higher (Schunk, 1996).

Because compelling evidence now exists that formative assessment, when appropriately implemented, has a profound positive impact on students’ learning, to delay in expanding the use of formative assessment short-changes our students educationally and, thereby, demeans our profession. (Popham, 2009)

Very Important!!! These rather significant changes in learning were due to students changing how they interact with the learning process; not just teachers doing something different! Although teachers are the choreographers of this performance, the students must actually be the ones performing!

Processing Activity: Considering the research outlined above, if you were trying to convince another educator that the formative assessment process was in fact something they should spend time learning about and practicing, what research would you highlight for them? The Formative Assessment process will assist in answering the following questions:

1. Where am I going?/Where is the learner going (what are the goals of learning)? 2. Where am I now?/Where is the learner right now (what progress is being made toward the goal)? 3. How can I close the gap?/How will the learner get there (what activities need to be undertaken to make better progress)?

Key Strategies of Formative Assessment Where is the learner going Where the learner is right now How to get there Teacher Clarifying and sharing learning intentions Engineering effective classroom discussions, activities, Providing feedback that moves and task that elicit evidence of learning learning forward Peer Understanding and sharing learning intentions and criteria for success Activating learners as instructional resources for one another

Learner Understanding learning intentions and criteria for success Activating learners as the owners of their own learning The five key strategies of formative assessment. Adapted from Leahy, Lyon, Thompson and Wiliam, 2005 All too often, the term “formative assessment” conjures images of Formative Assessment isn’t something you buy quizzes and tests, while in reality, formative assessment is a process – it is something you practice. used by teachers and students during instruction that provides Richard Stiggins feedback to adjust ongoing teaching and learning. Margaret Heritage, 2010

The Critical Attributes of the Formative Assessment Process: 1. Learning Progressions 2. Learning Goals and Success Criteria 3. Eliciting Evidence 4. Descriptive Feedback 5. Self and Peer Assessment 6. Collaboration

Start – Iowa Core/Early Learning Standards, Content Area Standards - Iowa Core Shifts

Promotes student learning through the development of conceptual understanding and application of knowledge. Examples: Math: Shifts from memorizing and practicing facts and procedures to understanding and applying concepts, practices, and facts.

Literacy: Shifts from a primary focus on reading and writing in English/Language Arts classes to the integration and practical application of all five literacy skills – reading, writing, speaking, viewing, and listening – across all content domains.

Science: Shifts from lecture, an over-emphasis on textbook readings and “cookbook” labs to learning through actively investigating, designing experiments, questioning, exploring, and defending conclusions.

Social Studies: Shifts from lecture and recitation of discrete pieces of information to the acquisition of knowledge and skills associated with the economic, political, civic, and social forces that are relevant to the world in which students live.

21 st Century Skills: Shifts from a traditional system that may not include the incorporation of these skills into curriculum for all, to one that requires that all students leave school prepared to succeed in the complex new, 21st century. Essential concepts and skill sets include employability skills, financial literacy, health literacy, and technology literacy.

The Iowa Core and Depth of Knowledge (DOK) Measures the degree to which the knowledge elicited from students on assessments is as complex as what students are expected to know and do as listed in the state standards. 6 Level 1 – Recall of Fact, information or procedure Level 2 – Use information, conceptual knowledge, procedures, two or more steps Level 3 – Requires reasoning, developing a plan or sequence of steps; has some complexity; more than one possible answer Level 4 – Requires an investigation; time to think and process multiple conditions of the problem or task

1. Learning Progressions

• Learning progressions are a "carefully sequenced set of building ______that students must master en route to a more distant curricular aim. The building blocks consist of sub skills and bodies of enabling knowledge." (Popham, 2007)

• The learning progression describes a sequence along which students can move incrementally from ______to more ______performance. Implicit in progression is the notion of continuity and coherence. Learning is not viewed as a series of discrete events, but rather as a trajectory of development that connects knowledge, concepts and skills within a domain. (Heritage, 2010)

How do individual teachers use learning progressions?

• They identify a target component of the learning progression based on what students already know and what learning is ahead.

• They identify missing building blocks of understanding for struggling students and next steps for students with grade level mastery.

• They organize the unit by establishing clear learning goals, success criteria and examples of strong and weak work samples. Help plan evidence gathering activities Time to Think! With your learning partners, discuss the following questions:

How do Learning Progressions support instruction and learning (i.e., assist in answering the 3 pivotal questions)?

What are some specific examples of things I currently do or can do in the future to apply this attribute into my instructional planning and/or classroom instructional activities?

______2. Learning Goals and Success Criteria

Learning Goals- general statements of intended student attainment of essential concepts and skills. Identifies what the student will learn during the course of the lesson or lessons. Generally written as “I understand” statements. Students need to learn what they need to learn to be able Qualities of a Learning Goal: to know if they learned it – M. Heritage  are brief • are written in student-friendly terms; • are based on learning progressions and future learning; • allow students to make connections to prior learning; 7 • guide the development of success criteria; • guide the development of embedded formative and summative assessments; and • guide teacher’s actions.

Success Criteria- Success criteria are the indicators that teachers and students use to understand that learning is progressing. They describe, from a student’s perspective, what successful attainment of a learning goal looks like and are often written as "I can.." statements. They provide students with the tools to monitor their learning and to determine their level of proficiency with respect to a learning goal. Criteria can be made explicit through exemplars.

Success criteria should: • Describe what success looks like; Be measureable • Be communicated to students and be described in student-friendly language • Be used by students during the learning process (self and peer assessment)

“The single most important method for routinely sharing “A learning goal is only as good as the learning targets is using assignments that match – really instructor’s ability to imagine what it would look match- the learning goal. It is in the assignment that the like when it is being met” Sara Fine 2010 teacher translates the learning goal into action for the student.” C.M. Moss and S.M. Brookhard

Evaluating Quality: Do the Learning Goals and Success Criteria:

1. Do they align with the Iowa Core/EL standards, other district content area standards and available learning progressions? 2. Do they demonstrate appropriate Rigor and Relevance? – Iowa Core – DOK, Rigor/Relevancy Quadrants and Blooms Taxonomy Examples of communication strategies  display in classroom previous to start of lesson  after an inquiry activity (so the journey is not spoiled) “ Students can hit any target  rubric they can see that holds still for them”  co-creation (R. Stiggins, 2006)  sharing examples of high and low quality work

Time to Think! With your learning partners, discuss the following questions:

How do Learning Goals and Success Criteria support instruction and learning (i.e., assist in answering the 3 pivotal questions)?

What are some specific examples of things I currently do or can do in the future to apply this attribute into my instructional planning and/or classroom instructional activities?

______3. Eliciting Evidence to Close the Gap 8 Planned activities used for the purpose of gathering student learning information so current performance can be compared to the success criteria for a given learning goal. This information will assist the teacher in providing feedback to the student and choosing the instructional modifications that are necessary to close the learning gap.

Strategies must be:  Appropriate to ______ ______with learning goals  Able to provide ______and ______with sufficient information

Intended Purpose These eliciting evidence activities should To increase students' learning generally be Ungraded and help both you and To diagnose student needs the student answer the 3 critical questions….. To adjust instruction To improve the instructional program “A good assessment activity makes a good teaching activity, and a good teaching activity makes a good assessment activity”  Questioning  Discussion (student/student - teacher/student) Shavelson, Baxter, and Pine, 1992  Observation (watching and listening)  Tasks (representations, explanations, problem solving strategies)  All Response Systems  Exit cards  Notes to teacher

 Curriculum-embedded assessments (e.g. from textbook or guides)  Work samples  Quizzes When possible, use response systems that engage all students

Time to Think! With your learning partners, discuss the following questions:

How do Elicit Evidence of Learning activities support instruction and learning (i.e., assist in answering the 3 pivotal questions)?

What are some specific examples of things I currently do or can do in the future to apply this attribute into my instructional planning and/or classroom instructional activities?

______4. Self and Peer Assessment

• Self Assessment - a process in which a student monitors their own learning in relation to a specific goal, and then makes necessary changes, using appropriate strategies, to meet the goal. Self-assessment involves the processes of metacognition and self-regulation.

• Peer Assessment - a learning activity in which students provide feedback relative to the learning goal. Feedback should be specific to the process or task, not the student 9

Why self and peer assessment is important: Teachers do not create learning; only • Increases student: learners create learning. – Involvement – Responsibility Activating students as owners of their own – Motivation learning can produce extraordinary • Builds a “community” of learners improvements in their learning. Wiliam, 2011 • Teacher and student roles change – Teacher’s role is not to judge, but to assist the student in achieving their goals. – Student’s role is to identify where they are and how to meet their learning goal and to assist others in doing the same.

Examples of self and peer assessment strategies

Learning Diary/logs - • When providing written feedback to students ask questions such as: • “Do you think that your response showed your understanding of ….? • If yes, why do you think this? If not, why do you think you did not show you understood X?” • “ What strategy could you use to help you understand? “ Learning Pairs - Ask pupils to use the expected learning goal to comment on strengths of each other’s work and to identify areas for improvement. Pre-Flight Checklist - Pair students up. Provide them with a checklist. Students trade their work. They check and give

feedback to one another. Papers are returned to the owner. Mistakes are corrected. Work goes back to partner for a “pre- flight check” before it is submitted. Blogs - Allow students to post samples of work for peers to access and provide descriptive feedback. Two Stars & a Wish - Student pairs trade their work. Each reviews the other’s work. They provide feedback to their partner by giving them positive comments (two stars) and making one suggestion for improvement (a wish). Thumbs Up Thumbs Down - Have pupils show thumbs up, sideways or down to indicate their perception of their achievement. • Thumbs up – confident they have achieved what was expected, Thumbs sideways – some way there, but could achieve more, Thumbs down – little progress towards achieving Find it and Fix it - Check student work quickly but do not mark the work. Identify the number of mistakes. Return the work to the student to find and fix the mistakes Surveying students - Ask pupils for their confidence level with a particular piece of work. This can be in the form of a student survey, a written statement, shared orally, or self-scored rubric. What-do you-think? - Ask pupils to decide whether they think an answer is reasonable, whether they can add to the answer, or whether they would have given another answer. Begin with the end in mind - Encourage children to develop learning outcomes for themselves for units of work. Self-Assessment Protocol - When reading aloud, stop from time to time and orally complete sentences like these: So far, I've learned... This made me think of...That didn't make sense. Scoring Guides/Rubrics - Build the scoring guides from the clear learning goals and success criteria. Provide students with examples of high quality work, Provide the scoring guides to students when the assignment is issued. The Muddiest Point – This is reverse thinking for many students. Here they are asked to think about what they don’t understand rather than what they do understand, but it helps them also clarify what they do know. Students are asked to identify their muddiest point and submit it anonymously in writing. The teacher collects these and addresses each muddiest point or invites students to address it. Turn to your Neighbor - Students debrief each other in pairs immediately following an activity or lesson. Students formulate an individual response, and then turn to a partner to share their answers. Teacher calls on several random pairs to share their answers with the class.

Time to Think! With your learning partners, discuss the following questions: 10

How do Self and Peer Assessment Activities support instruction and learning (i.e., assist in answering the 3 pivotal questions)?

What are some specific examples of things I currently do or can do in the future to apply this attribute into my instructional planning and/or classroom instructional activities? ______5. Descriptive Feedback

Information (based upon the analysis of data collected) provided to the student which serves to describe their current status as compared to the learning goal (i.e. success criteria)

 Feedback should be directly connected with the success criteria and the learning goal and the 3 pivotal questions  Feedback is only formative if it is USED by students; should create more work on the part of the recipient than the donor  Teachers must allow Time for students to use feedback  Using feedback helps students develop LEARNING STRATEGIES and become intrinsically motivated learners  Feedback assists the teacher in becoming evaluators of their IMPACT

Additional Key Guidelines about Feedback

 Feedback should provide information to the student relating to the task or process of learning that fills a gap between what is understood and what is aimed to be understood  Feedback should provide the learner with suggestions, hints or cues for how to improve rather than correct answers; Feedback should avoid comparisons with other students  Feedback should match the student’s cognitive needs – not too complex and not too vague  Feedback should be given after a student has responded to initial instruction. In the case when no learning has occurred it is better to continue with instruction rather than provide feedback

What are some non-examples of productive Descriptive Feedback? Time to Think! With your learning partners, discuss the following questions:

How does Descriptive Feedback support instruction and learning (i.e., assist in answering the 3 pivotal questions)?

What are some specific examples of things I currently do or can do in the future to apply this attribute into my instructional planning and/or classroom instructional activities?

______6. Collaboration: Classroom/Building Culture

Building a classroom/building culture and learning environment that is conducive to the implementation of the attributes of the assessment for learning process and is based upon: • Trust With this type of classroom culture, we • Norms of respect will increase the odds that students • Transparency will: 11 • Appreciation of differences • Use and value feedback • A non-threatening environment • Engage in Peer and self assessment • Clear learning goals • Become self regulated and self- • Partnership in teaching, learning and in making some decisions motivated learners • Become partners in learning

Examples of collaborative classroom / climate strategies:

• Establishing high standards and clear expectations • Providing on-the-spot coaching • Identifying and rewarding desired behavior • Exhibiting student-produced work • Including students in decision-making • Arranging space for flexible use • Interacting in a respectful and encouraging manner • Conveying confidence in student abilities • Providing the learning targets before the instructional/learning phase. • Providing multiple opportunities for applying new understandings, analyzing and assessing the results, and revising the product and process (the learning loop). • Providing descriptive feedback so students know they are “on track” to the learning target and, upon completion, if they achieved it – and why.

Additional thoughts around developing a conducive classroom culture to develop “self-regulated” learners 1. Share learning goals with students so that they are able to monitor their own progress.

2. Promote the belief that ability is incremental rather than fixed; when student think they can’t get smarter, they are likely to devote their energy to avoiding failure.

3. Make it difficult for student to compare themselves with others in terms of achievement.

4. Provide feedback that contains a recipe for future action rather than a review of past failures (a physical rather than an autopsy).

5. Use every opportunity to transfer executive control of the learning from the teacher to the students to support ther development as autonomous learners.

From: Dylan Wiliam, Embedded Formative Assessment, 2011, page 152

What do you think!

Effective use “Gone wrong”

Time to Think! With your learning partners, discuss the following questions:

How does a positive Classroom Culture support instruction and learning (i.e., assist in answering the 3 pivotal questions)?

What are some specific examples of things I currently do or can do in the future to apply this attribute into my instructional planning and/or classroom instructional activities?

The quality of teachers is the single most important factor in the The quality of an education system cannot exceed educational system. the quality of it teachers.

William, 2012 Barber and Mourshead, 2007

“How well we teach = How well they learn” Archer, 2015

AEA PD Online Assessment for Learning Modules http://moodlesw.aeapdonline.org/login/index.php