How Does PISA for Development Measure Reading Literacy?

PISA for Development Brief 8

How does PISA for Development measure reading literacy?

 The term “reading literacy” as used in PISA includes a wide range of cognitive competencies, from basic decoding, to knowledge of words, grammar and larger linguistic and textual structures and features, to knowledge about the world.  Building on the OECD’s experience with measuring reading literacy in middle-income countries, PISA for Development (PISA-D) extends PISA’s reading literacy framework to better measure basic processes, such as comprehending sentences and longer texts.  In particular, the PISA-D reading test measures skills, such as understanding the meaning of individual written words and short phrases, thus allowing countries to know more about the kinds of tasks 15-year-olds with poor reading proficiency can perform. ______

PISA defines reading literacy as: understanding, using, reflecting on and engaging with written texts, in order to achieve one’s goals, develop one’s knowledge and potential, and participate in society. This definition acknowledges the diversity and complexity of the processes involved in daily reading activities. PISA also establishes a baseline level – proficiency Level 2, on a scale with 6 as the highest level and 1b the lowest – at which readers begin to demonstrate the competencies that will enable them to participate effectively and productively in life as continuing students, workers and citizens.

The tasks that make up the PISA reading assessment often require students to answer questions based on their comprehension of extended passages of text. This approach enables PISA to identify and describe a wide range of proficiency levels. But proficiency below the baseline Level 2 is less clearly defined.

The OECD analyses reported in PISA 2012 show that for many of the participating middle- and low-income countries, reading scores are concentrated below proficiency Level 2 (see figure).

Percentage of students scoring at Levels 1a and 1b and below in reading in 18 low- and middle-income countries, PISA 2012

60 Level 1a Level 1b Below level 1b 50 40 30 20 10 0

Source: OECD (2014), PISA 2012 Results: What Students Know and Can Do (Volume I, Revised edition, February 2014): Student Performance in Mathematics, Reading and Science, PISA, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264208780-en.

PISA for Development Brief – 2016/12 (December) © OECD 2016

PISA-D is extending PISA’s assessment of basic reading literacy processes to better capture the abilities of low performers.

In light of the OECD’s analyses of reading scores in middle- and low- income countries, the conceptual framework for the reading assessment in PISA-D was designed to extend the PISA framework to lower levels of proficiency. PISA-D includes more items at Level 2 and below than the main PISA test, providing a wider and more fine-grained picture of what 15-year-olds know and can do at these lower levels of performance. While closely aligned with PISA, PISA-D provides better coverage of basic processes, such as sentence and passage comprehension. Maintaining the same dimensions and descriptors as the main PISA test, PISA-D outcomes will align with the PISA scale.

The PISA reading assessment is built on three main task characteristics:  Processes (aspects) – the cognitive strategies, approaches or purposes that readers use to negotiate their way into, around and among texts  Text – the range of material that is read  Situation – the range of broad contexts or purposes in which reading takes place.

The text and situation variables for PISA-D are the same as those used in the main PISA test, but PISA-D has enhanced the processes targeted in the reading test. Five processes were defined: retrieving information; forming a broad understanding; developing an interpretation; reflecting on and evaluating the content of a text; and reflecting on and evaluating the form of a text. PISA-D adds a process called “literal comprehension” that requires students to comprehend explicitly stated information that may be found in individual words, sentences or passages.

In addition, the concept of “retrieving information” is broadened in PISA-D to range from locating explicitly stated individual pieces of information, such as individual words or phrases, to finding information in long passages. Hence, PISA-D has created Level 1c as the new lowest level on the reading proficiency scale, and includes new test items to measure performance on tasks at this level.

With a clearer understanding of the reading skills of low-performing 15-year-olds, policy makers in middle- and low-income countries will be able to design more effective and targeted policies to help students learn to read better, help teachers teach reading better, and help school systems promote reading as a fundamental skill.

References OECD (forthcoming), PISA for Development Assessment and Analytical Framework, PISA, OECD Publishing, Paris. OECD (2016), PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic and Financial Literacy, PISA, OECD Publishing, Paris, DOI: http://dx.doi.org/10.1787/9789264255425-en

For more information Contact Michael.Ward@oecd.org, [email protected] or [email protected]

Visit Coming next month www.oecd.org/pisa/aboutpisa/pisafordevelopment.htm How does PISA-D measure mathematics literacy?

Photo credits: © Curt Carnemark/World Bank © epicurean / iStockphoto This paper is published under the responsibility of the Secretary-General of the OECD. The opinions expressed and the arguments employed herein do not necessarily reflect the official views of OECD member countries. This document and any map included herein are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area.