Proceedings of the Symposium on Assessment and Learner Outcomes Rutherford House, Victoria University, Wellington, New Zealand 1-3 September 2011 Edited by Mimi Hodis and Susan Kaiser Jessie Hetherington Centre for Educational Research Victoria University Wellington, New Zealand Publication date: October 2012 ISBN: 978-0-475-12398-5 Table of Contents INTRODUCTION Editorial Board ................................................................................................................... ii Editorial Reviewers ............................................................................................................ ii SECTION 1: CONFERENCE PAPERS External and Internal Moderation: The Other Side of the Story .......................................... 2 Tahera Afrin The Assessment of Literacy Capability in Adults: Tensions and Complexities .................. 15 Helen Anderson and Lisa Maurice-Takerei Formative Learning: A Description of an Assessment Framework in Initial Teacher Education ......................................................................................................................... 25 Ann Balcombe, Liz Everiss, and Margaret Brennan The Future of Secondary School Examinations: The Use of Technology in the New Zealand Context .............................................................................................................. 41 Stephen Bargh Exploring Four Self-Assessment Strategies to Sustain Learning in Tertiary Education .... 57 Roseanna Bourke How do Students use Self-assessment to Learn in a Tertiary Institution? ........................ 70 Roseanna Bourke and Carolyn Tait Two Uses of Value-Added Modelling in New Zealand Secondary Schools ...................... 81 Therese M. Boustead Assessing Group Work in Student Industry Projects: Is Fairness Achievable? ................ 92 Jill Clark and Trish Baker Popular Culture, Overall Teacher Judgments and National Standards ........................... 107 John Dickie and Mary Jane Shuker Promoting Information Literacy Development by Assessing the Research Process in First-year Tertiary Assessments..................................................................................... 121 Angela Feekery & Lisa Emerson Reducing exclusions and improving attainment – how can that be? Evidence from a United Kingdom secondary school ................................................................................. 137 Gwen Gilmore Enhancing Learner Outcomes Through the Use of Rubrics in the Summative Evaluation Process ........................................................................................................ 152 Trudy Harris, Dorothy Spiller, Michele Schoenberger-Orgad and Cheryl Cockburn-Wootten Design Experiments for Assessment of ―21st Century‖ Learning Outcomes .................. 164 Rosemary Hipkins An Alignment of the Trends in International Mathematics and Science Survey Mathematics Scale with the ‗By the End of Year 5‘ National Standard in Mathematics .. 174 Elliot Lawes Beyond Assessment: Assuring Student Learning in Higher Education ........................... 187 Victoria J. Mabin and Stephen J. Marshall Schools Out – Gone to the Beach: How Some Year 11 New Zealand High School Students Perceive Study Leave ..................................................................................... 204 Peter Sanders The ORBIT Online Analysis, Profiling and Reporting Tool............................................. 221 Peter Tait, Michael Davison, John Hattie, Wendy Kofoed The Benefits of Effective Behaviour Screening for Children in Schools and Kindergartens ................................................................................................................ 231 Gaye Tyler-Merrick and John Church Reading Achievement over the Transition to Secondary School: Profiling a Problem .... 242 Aaron Wilson, Stuart McNaughton, and Mei Kuin Lai New Zealand‘s First Tertiary High School: Students‘ Perspectives of Various Aspects of Their Education and Experiences in School ............................................................... 258 Colleen Young SECTION 2: KEYNOTE SPEAKERS Assessment in a Changing World .................................................................................. 275 Eva Baker National Education Standards for New Zealand: A Research Agenda ........................... 285 John Hattie Researching Quality and Equity in Assessment for Improved Learner Outcomes .......... 296 Val Klenowski The NCEA as an Exemplar of Research into Practice: Motivation, Achievement and Assessment Design ....................................................................................................... 307 Luanna H. Meyer Full List of Symposium Contributors ............................................................................... 313 Introduction The Symposium that resulted in these published papers was co-sponsored by Victoria University, the University of Auckland, the Ministry of Education, and the New Zealand Qualifications Authority. Special appreciation is due to the conference organising committee comprising John Hattie, Rawiri Hindle, Mimi Hodis, Jenny Horsley, Michael Johnston, Kiritina Johnstone, Mary Klaver, Mei Kuin Lai, Luanna H. Meyer, Boaz Shulruf, and Louise Starkey. Thank you especially to Graeme Aitken, Bali Haque, Lisa Rodgers, and Dugald Scott, for their facilitation of support from their respective institutions and agencies for the Symposium itself. Special thanks are also due to Pam Ritchie who was the conference administrator and provided key support to the organisers and participants. The individual papers published in these online Proceedings from the Symposium represent those submitted by their authors following the symposium. Each of these published papers was quality assured through an independent, anonymous peer review process and editorial review by both New Zealand and international experts in the relevant specialised areas of assessment. Not all papers presented at the Symposium are published in these proceedings, and interested persons may consult the full list of papers at the end of the Proceedings and contact authors directly for materials of interest. We owe a debt of gratitude to many who contributed their expertise to the Symposium through presentations and/or reviewing papers as part of our quality assurance process. Thank you. Conference Organising Committee October 2012 i Editorial Board Rawiri Hindle Luanna H. Meyer Victoria University of Wellington Victoria University of Wellington New Zealand New Zealand Mimi Hodis Boaz Shulruf Victoria University of Wellington Auckland University New Zealand New Zealand Jenny Horsley Louise Starkey Victoria University of Wellington Victoria University of Wellington New Zealand New Zealand Editorial Reviewers Tahera Afrin Michael Drake ICL Business School Victoria University of Wellington Auckland New Zealand New Zealand Ian M. Evans Dayle Anderson Massey University Victoria University of Wellington Wellington New Zealand New Zealand Helen Anderson Lester Flockton Manakau Institute of Technology University of Otago Auckland Dunedin New Zealand New Zealand Roseanna Bourke Cedric Hall Victoria University of Wellington Victoria University of Wellington New Zealand New Zealand Gavin Brown Roger Harvey Auckland University Victoria University of Wellington New Zealand New Zealand Sue Cherrington Trudy Harris Victoria University of Wellington Waikato University New Zealand New Zealand Vijaya Dharan Eva Heinrich Victoria University of Wellington Massey University New Zealand New Zealand Stephanie Doyle Rose Hipkins Victoria University of Wellington New Zealand Council for Educational New Zealand Research Wellington New Zealand ii Flaviu Hodis Stephen Marshall Victoria University of Wellington Victoria University of Wellington New Zealand New Zealand Mimi Hodis Azra Moeed Victoria University of Wellington Victoria University of Wellington New Zealand New Zealand Anne Hynds Eleanor Rowe Victoria University of Wellington Auckland University New Zealand New Zealand Michael Johnston Catherine Savage Victoria University of Wellington Te Tapuae o Rehua New Zealand Christchurch New Zealand Liz Jones Victoria University of Wellington Boaz Shulruf New Zealand Auckland University New Zealand Ella Kahu Massey University Louise Starkey Palmerston North Victoria University of Wellington New Zealand New Zealand Mei Kuin Lai Alison Stephenson Auckland University Victoria University of Wellington New Zealand New Zealand Kathleen Liberty Rob Strathdee Canterbury University Victoria University of Wellington New Zealand New Zealand David Lillis Carolyn Tait New Zealand Qualifications Authority Victoria University of Wellington Wellington New Zealand New Zealand iii SECTION 1: Conference Papers 1 External and Internal Moderation: The Other Side of the Story Tahera Afrin Early Childhood Education (ECE), ICL Business School Email: [email protected] Abstract Tertiary Education Organisations (TEOs) that offer programmes including unit standards set by the New Zealand Qualifications Authority (NZQA) are required to participate in the national external moderation system. In spite of conducting regular internal moderation, TEOs often fail when internally moderated unit standard materials are sent for external moderation. This paper shares the experience of a non-university training provider and analyses it further to identify factors that lead to ineffective moderation. The initial discussion concerns the two types of moderation that TEOs are involved in: internal and