Literacy, Numeracy and Problem Solving in Technology-Rich Environments FRAMEWORK FOR THE OECD SURVEY OF ADULT SKILLS skills.oecd: translating better skills into better outcomes Literacy, Numeracy and Problem Solving in Technology-Rich Environments FRAMEWORK FOR THE OECD SURVEY OF ADULT SKILLS This work is published on the responsibility of the Secretary-General of the OECD. The opinions expressed and arguments employed herein do not necessarily reflect the official views of the Organisation or of the governments of its member countries. This document and any map included herein are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Please cite this publication as: OECD (2012), Literacy, Numeracy and Problem Solving in Technology-Rich Environments: Framework for the OECD Survey of Adult Skills, OECD Publishing. http://dx.doi.org/10.1787/9789264128859-en ISBN 978-92-64-12880-4 (print) ISBN 978-92-64-12885-9 (PDF) Revised version, October 2013 Details of revisions available at: http://www.oecd.org/about/publishing/Corrigendum_literacy-numeracy-and-problem-solving-in-technology-rich-environments.pdf Photo credits: Getty Images © Tetra Images Fotolia © ag visuell iStockphoto © Joshua Hodge Photography Stocklib © Kheng Ho Toh Getty Images © Monty Rakusen Stocklib © Fernando Blanco Calzada Corrigenda to OECD publications may be found on line at: www.oecd.org/publishing/corrigenda. © OECD 2012 You can copy, download or print OECD content for your own use, and you can include excerpts from OECD publications, databases and multimedia products in your own documents, presentations, blogs, websites and teaching materials, provided that suitable acknowledgement of OECD as source and copyright owner is given. All requests for public or commercial use and translation rights should be submitted to [email protected]. Requests for permission to photocopy portions of this material for public or commercial use shall be addressed directly to the Copyright Clearance Center (CCC) at [email protected] or the Centre français d’exploitation du droit de copie (CFC) at [email protected]. Foreword T he Programme for the International Assessment of Adult Competencies (PIAAC) is an international assessment of adult skills managed by the OECD which is currently being implemented by 25 countries in Europe, the Americas and Asia. PIAAC will provide one of the richest sources of data regarding the foundation skills of adults, the skills that they use in work and other contexts and correlates of skills such as demographic characteristic, family background, education, employment, income and outcomes such as civic participation and health. Development of assessment frameworks is a central component of the approach to assessment underlying PIAAC. These frameworks provide an agreed definition of what should be measured and the identification of characteristics which can be used in the construction and interpretation of tasks. In other words they define what is meant by “literacy”, “numeracy” and “problem solving in technology-rich environments” in PIAAC. Draft frameworks for each of the assessment domains were developed by dedicated expert groups under the leadership of Stan Jones (literacy), Iddo Gal (numeracy), Jean-François Rouet (problem solving in technology-rich environments). The reading components framework was prepared by John Sabatini and Kelly Bruce. This document which summarises the work of the expert groups was prepared by William Thorn. Andreas Schleicher, Irwin Kirsch and Claudia Tamassia offered valuable comments during the drafting. Marilyn Achiron, Fionnuala Canning and Elizabeth Del Bourgo provided editorial assistance. Niccolina Clements, Sabrina Leonarduzzi and Elisabeth Villoutreix co-ordinated the production process. Angel Gurría OECD Secretary-General LITERACY, NUMERACY AND PROBLEM SOLVING IN TECHNOLOGY-RICH ENVIRONMENTS – © OECD 2012 3 Table of Contents INTRODUCTION .....................................................................................................................................................................................................................................7 CHAPTER 1 WHY ASSESS THE SKILLS OF ADULTS? ......................................................................................................................................................9 Changing demand for skills .............................................................................................................................................................................................................10 Low proficiency – its scale and impact ...................................................................................................................................................................................10 Skills for the information age .......................................................................................................................................................................................................11 The output and performance of education systems.......................................................................................................................................................11 CHAPTER 2 SOME DESIGN FEATURES OF PIAAC ........................................................................................................................................................15 Links to previous surveys ..................................................................................................................................................................................................................16 Computer-based assessment ...........................................................................................................................................................................................................16 CHAPTER 3 LITERACY AND READING COMPONENTS .........................................................................................................................................19 Definition of the domain ..................................................................................................................................................................................................................20 Categorising texts (task characteristics) .................................................................................................................................................................................20 Aspects of tasks .......................................................................................................................................................................................................................................23 Factors that affect task difficulty .................................................................................................................................................................................................24 Distribution of test items by task characteristics .............................................................................................................................................................24 Literacy – Examples of items ..........................................................................................................................................................................................................25 Reading components ...........................................................................................................................................................................................................................27 Reading components – Example of items ..............................................................................................................................................................................28 Sentence processing .............................................................................................................................................................................................................................30 Passage comprehension .....................................................................................................................................................................................................................30 CHAPTER 4 NUMERACY ................................................................................................................................................................................................................33 Definition of the domain ..................................................................................................................................................................................................................34 Facets of numerate behaviour .......................................................................................................................................................................................................35 Principles for assessing numeracy in PIAAC .......................................................................................................................................................................39 Numeracy – Examples of Items ....................................................................................................................................................................................................40