Assessment principles and practices—Quality assessments in a digital age Table of contents Overview ................................................................................................................................................................................................................. 5 Table of contents 5 Foreword 5 Why read this document? 8 Using this resource—different routes through the document 9 Introduction and overview of this resource ................................................................................................. 21 Introduction 21 What are assessment principles and practices? 21 Which IB programmes does it cover? 22 Who is this resource for? 23 What does the tag line “quality assessment in a digital age” mean? 23 How does this resource relate to other IB resources and guides? 24 Assessment using technology 28 Language of assessment 33 Emerging terms for eAssessment 35 Section A—Principles of Assessment ..................................................................................................................... 39 What is assessment about? 39 Fit for purpose? Validity 46 Elements of the validity chain 55 Defining standards 73 Describing success— candidate achievement for summative assessment 79 Marking assessments 82 What is a good assessment? 90 IB’s principles of assessment 106 Section B—IB assessment practices ..................................................................................................................... 108 What do we mean by a practice? 108 Reporting candidate achievement 108 Roles and responsibilities 114 Integrity of the assessment 120 Fairness for all—meeting candidates’ needs 125 The assessment cycle 131 Examination paper preparation—development and quality 135 Examinations 142 Standard setting—Preparing examiners for marking 147 Marking 153 Moderation 164 Grade awarding (and aggregation) 174 “At risk” based quality checks 191 The final award committee 193 Preparation for release of results 194 Enquiries upon results (EUR), appeals and general feedback 197 Setting next year’s assessments 203 Feedback to schools 207 Section C—IB programme-specific processes ....................................................................................... 211 What are programme-specific processes? 211 Elements common to all programmes 213 IB Diploma Programme 216 IB Career-related Programme 225 IB Middle Years Programme 229 IB Primary Years Programme 235 Annexes ............................................................................................................................................................................................................ 237 Moderation of internal assessment 237 Roadmap for creating a validity argument 243 Bibliography 247 Glossary 252 Assessment principles and practices—Quality assessments in a digital age Overview Table of contents This resource has been designed to have many routes through, depending on the interests and needs of the reader. The following table of contents is intended only as a reference list of all the chapters rather than a suggested order in which they should be read. We would encourage you to start with the chapter on Using this resource, to understand how to navigate to the sections which are most important to you. Foreword "Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted. — (Albert Einstein/William Cameron 1963) "Clearly, if the other criteria are less reliable than the examinations, greater reliance on them will lead to less reliable selection decisions. — (Mike Cresswell 1986) These two quotes indicate the scope of the challenge that we face with assessment. Many of the objectives for an International Baccalaureate (IB) education are not easy to assess, but without detailed assessments of our learners, important decisions that will affect their lives will be made on less fair and reliable grounds. IB programmes are taught in over 140 countries by schools representing a wide variety of educational contexts and traditions. In some of these contexts, the philosophy and approaches adopted by the IB in assessing their students will seem familiar, while to others, the system might seem mysterious and obscure. In this document, we are seeking to explain the principles the IB has adopted to make sure that the assessment we undertake is meaningful, fair and in the best interest of the students involved. 5 Assessment principles and practices—Quality assessments in a digital age Overview Such clarity is even more important during a period of change, and the impact of technology on education, including assessment, will continue to be felt over the next decade. We strongly believe that technology should support assessment and the move towards computerized on-screen examinations will not change our principles; but it may open up new possibilities in turning these principles into practices. More details on this can be found in the chapter on Assessment using technology. We believe that it is important that everyone in the IB community understands how our external assessment process works, what its strengths and limitations are, and the reasons why decisions are taken. Increased transparency can only lead to better understanding and ultimately a better education for our students. By using the opportunities offered by on-screen resources, we hope to provide teachers with a clear guide that is accessible but also contains the depth of information they need to understand IB assessment. Figure 1: All assessments are a balance 6 Assessment principles and practices—Quality assessments in a digital age Overview All assessments are a balance between conflicting demands and many concerns about testing processes fail to take this into account. An example might be the tension between reducing the assessments burden and the risk of candidates only having one opportunity to show what they can do. For more information on this see the section on validity. Need to balance between conflicting priorities Another aspect of balance is the fact that the focus of the IB is to develop students through a holistic programme of study, and we must reflect this in our approach to assessment. This means we should make decisions about the impact on the overall programme, not a narrow focus on one subject, discipline or assessment. In order to understand the nature of the IB assessment philosophy and operation, it is necessary to provide some background on the historical and theoretical development of assessment practice. Many significant issues are only briefly touched upon, but it is important to highlight them as they have a significant impact on current practice. For readers who wish to find out more, the academic papers quoted in the text will make a suitable starting point for further investigation. We started by recognizing the difficult task the IB sets itself; to focus on what is important to assess and not what is easy. This is perhaps most eloquently expressed by Alec Peterson, the first Director General of the IB: "What is needed is a process of assessment which is as valid as possible, in the sense that it really assesses the whole endowment and personality of the pupil in relation to the next stage of his life, but at the same time sufficiently reliable to assure pupils, parents, teachers, and receiving institutions that justice is being done. Yet such a process must not, by its backwash effect, distort good teaching, nor be too slow, nor absorb too much of our scarce educational resources. — (Peterson 1971) We hope the rest of this resource explains how we believe we can deliver on this challenging objective and support the wider educational intentions of the International Baccalaureate in providing 7 Assessment principles and practices—Quality assessments in a digital age Overview a world class experience for its students. If you have further queries which are beyond the scope of this resource please contact IB assessment staff, by emailing [email protected]. Why read this document? Because assessment results have an impact on students’ lives The majority of the content of this document refers to the way in which the IB assesses candidates to award Diploma Programme (DP), Career-related Programme (CP) and the optional Middle Years Programme (MYP) outcomes, which are then used by students to progress into further education or work. Like everything else, assessment is only a tool which can lead to positive or negative outcomes for those being evaluated with them. As a teacher, parent or student, you are involved in assessments and should understand the strengths, weaknesses and decisions that those offering and using assessment need to make. Below are some of the common questions and comments we receive from teachers and how this document will help answer them. “Why would the IB do that?” This is the heartfelt question that we often hear from teachers and students when faced with external assessment. By reading this document you should understand the principles that drive our assessment practices. “It’s not fair sir, I needed a higher grade!” For students, a great deal will depend on their examination results, such as university entrance and future career; and they need to understand why decisions are made. Assessment is all about balancing conflicting and competing demands and this document will help you to explain to students the wider implications of bending the rules in their case, and maybe even convince them that their grade was “fair”.