ASSESSMENT GEMS SERIES No. 5

The ‘’ idea March 2014 Ross Turner

The ACER Centre for Global Literacy in learning domains Education Monitoring The phrase has been used since Much of ACER’s assessment work aims to the early 1950s, and from the early 1960s it began measure levels of student literacy in the various to appear frequently in papers and discussions knowledge domains of interest, rather than about science education. The phrase mathematical narrowly defined curriculum-based achievement. literacy dates from even earlier. These days we This document explains ACER’s approach. often see references in academic literature and the general media to a wide range of , Knowledge, education and literacy including , environmental literacy, digital literacy, economic literacy, statistical literacy Good practice in teaching and learning has long and . been concerned with the application of acquired knowledge. In an influential early essay about One reason that education researchers and this issue, Alfred North Whitehead describes the practitioners describe a learning domain in terms central problem of education as ‘the problem of of literacy is to emphasise the fact that the domain keeping knowledge alive, of preventing it from has dimensions that extend beyond any traditional, becoming inert’ (Whitehead, 1929, p.4). According narrow definition. In this way they are aiming to to Whitehead, elements of knowledge risk promote appreciation of the domain as a significant becoming inert if they are ‘merely received into the focus of human endeavour. mind without being utilised, or tested, or thrown Another, more central reason researchers and into fresh combinations’. practitioners refer to domain literacy is to draw Later, cognitive scientists such as Herbert Simon attention to the kinds of things students learn contrasted the notion of inert knowledge with in the domain. In a traditional learning domain that of conditionalised knowledge (Simon, 1980). the focus might be on the acquisition of discrete Conditionalised knowledge is transferable – facts, skills and procedures that have little obvious items of conditionalised knowledge are readily connection or utility. In a learning domain with a thrown into the ‘fresh combinations’ that different literacy orientation, the focus is on applying the contexts demand. domain’s facts, skills and procedures to support creativity and inventiveness, to solve novel Concepts and skills that make up conditionalised problems and to deal with the kinds of challenges knowledge are transferable because they have that life presents outside the classroom. been consolidated and generalised. An education In the case of mathematics, for example, a that enables this consolidation and generalisation literacy orientation enables students to forge the is an education that empowers students to take connections between the facts and procedures their abilities beyond the classroom to other that constitute the basis of mathematical theatres of daily activity, and beyond schooling to knowledge and the real-life situations in which work and other areas of adult life. mathematical knowledge can be used. Specifically, Such an education is said to have a through studying mathematics with a literacy literacy orientation. orientation students learn to:

Australian Council for Educational Research

ISSN 2203-9406 (Online) • express mathematical ideas and mount In the case of and , the literacy mathematical arguments orientation is perhaps more immediately accepted • understand the mathematical ideas and than it is in domains that are traditionally regarded arguments of other people as highly content driven, such as science and mathematics. Of course, this is partly to do • reason mathematically with the fact that the original meaning of the • use a variety of different kinds of word ‘literacy’ is the ability to read and write. representations of mathematical phenomena, Nevertheless, our use of the terms ‘reading including graphs, tables, charts, equations literacy’ and ‘writing literacy’ deserves some • think strategically, and plan and implement a elaboration. Like mathematical literacy, reading sequence of mathematical processing steps and writing literacy emphasise the application of • recognise and use, or devise and manipulate sets of skills, knowledge and understanding – mathematical models of real-world phenomena applications across many contexts both inside and • reflect on which elements of their mathematical outside school. skills and technical knowledge might be relevant While reading literacy necessarily depends on to a particular challenge knowledge of the basic components of decoding • identify when particular mathematical tools skill – phonemic awareness and (for (such as computer-based tools, or measuring alphabetic languages), knowledge of symbols (for instruments, or calculating devices) might be non-alphabetic languages), fluency and vocabulary useful, and make effective use of those tools. knowledge – these components are not sufficient indicators of reading literacy, which requires also Box 1: Real-world mathematical challenges the ability to understand, reflect upon, evaluate Two examples of real-life situations in and apply what is encountered in written texts. which mathematical knowledge is used are This complex and interwoven set of proficiencies formulating a personal budget and planning a gives access to a range of human knowledge trip on public transport. far beyond what can be experienced directly; it allows readers to learn from what they encounter To formulate a personal budget, an individual must model projected income and expenditure. in texts, to combine what they have learnt with This involves making assumptions about what prior knowledge, and thus to create new meanings might transpire over a specified period of time, and solve problems. Making meaning from transforming those assumptions into numerical written texts by activating knowledge of language form, selecting an appropriate tool for recording structures and features, and by combining what the information and using that tool effectively. is already understood with new information in To plan a trip by public transport, an the text – these capabilities are at the heart of individual must exercise spatial awareness in reading literacy. determining directions and possible routes Similarly, the literacy orientation conceives of for the proposed journey, analyse timetable writing as making meaning through expression information (perhaps even linking timetables and communication in language, rather than as the of different transport modes) with reference to exercise of a mechanistic set of language rules or assumptions about desired departure, duration and arrival times, and put all findings together the display of atomised pieces of knowledge about in the form of an action plan. grammar, vocabulary and other linguistic features. Writing literacy is the use of language to express In both examples, successful negotiation of thoughts, feelings and ideas. Increasing proficiency the situation depends on the individual’s ability in writing literacy is indicated by increasing to connect and operationalise mathematical facts, skills and processes. This connecting range and depth of expression, and flexibility in and operationalising is mathematical literacy. communicating with a variety of audiences and for multiple purposes.

The ‘literacy’ idea 2 Assessing literacy Literacy in ACER’s Monitoring Trends in Educational Growth partnership program Assessments of literacy and ACER’s Monitoring Trends in Educational Growth curriculum-based assessments (MTEG) partnership program has a flexible, Assessments with a literacy orientation are often collaborative approach to assessment. ACER contrasted with curriculum-based assessments. staff members work with individual participating Yet this contrast sets up an unhelpful, outdated countries to set the policy direction for the dichotomy. Since it is now widely recognised assessment, to determine the target populations that curriculum goals should be informed by and to formulate the test design. literacy goals, it follows that curriculum-based MTEG’s three core assessment domains of assessments should in some ways have a literacy mathematics, reading and writing all have a orientation themselves. literacy orientation. ACER believes that a literacy Indeed, modern conceptions of effective orientation to assessment is important for all assessment increasingly focus on a complex countries, but particularly important for countries set of benefits and outcomes that go far beyond with rapidly developing education systems. summarising individuals’ levels of mastery of In a rapidly developing education system, knowledge taught in a particular school year. advances in teaching and learning practices Assessment is most useful when it generates can lead to significant improvements in student information that describes progress; when it performance in a relatively short period of informs future action; when it takes account time. Assessments should yield data that give of current research on learning in the relevant a full picture of the extent and nature of this domain; when it accommodates the need improvement. An assessment with a literacy for information about broader life skills and orientation is better able to do this than an attributes, especially those relevant to 21st- assessment that merely measures the mastery of century workplaces and life circumstances; facts and processes. and when it takes advantage of advances in technology and engages the kinds of technology The seminal report of the Committee on the in common use among citizens of the 21st century Prevention of Reading Difficulties in Young (Masters, 2013). Children (Snow et al, 1998) points to the complexity that lies behind a well-developed Literacy in ACER’s assessment activities ability to find meaning in text. It presents a clear When developing the Programme for International case for the need to base reading and writing Assessment (PISA) for the OECD in the late instruction on a broad and inclusive literacy notion 1990s, ACER introduced and then advocated a that includes specific skills and techniques as well literacy orientation for the assessment domains. as a range of processes essential to developing a full understanding of text written for a variety of This action was based on the firm belief that purposes, and forging the required links to relevant what an assessment of learning outcomes should contextual elements. This complexity is missed in measure is the extent to which students can assessments limited to basic skills of reading and demonstrate the capacity to use their knowledge writing such as phonological knowledge, alphabet and skills effectively in a range of contexts. knowledge, grammar, punctuation and the location The literacy orientation has subsequently been of directly stated information in short, simple used successfully in many other ACER projects. texts. While appearing attractive because they One example is the International Schools’ can be relatively cheap and easy to administer, Assessment (ISA) project, which assesses such basic assessments can create a cycle of students from grades 3 to 10 in international misunderstanding about what being a literate schools worldwide. Developers of the ISA see reader and writer means that reinforces inadequate both traditional domain content knowledge and teaching methodologies and poor student critical mathematical, writing and reading process outcomes. Assessments with such a limited skills as central goals of instruction and key focus can give inflated measures of students’ targets of assessment. ‘reading’ ability, because they do not address the

The ‘literacy’ idea 3 complex skills that underpin finding substantive – different students, different classes, different meaning in texts. This can lead to a consequent schools, different grades, even different countries. lack of intervention at the very point at which it is most critical. Misjudging students’ reading skills ACER’s approach also leads to erroneous assumptions by teachers ACER draws upon and constantly updates its in later years about why some students fail to research on what is known about teaching and progress. Teachers and systems assume that the learning in the various knowledge domains assessments used in the early years have correctly in which its projects are located. It takes into identified that most students can read, and the account current contextual information such as issues in later years concern students’ ability to the curriculum and assessment arrangements understand the subjects they have been taught. that apply in a particular jurisdiction or learning Teachers may seriously overestimate students’ environment, and while understanding the ability to read and understand the prescribed need to work within existing contexts, seeks workbooks and other texts. Students’ ability to means of advancing practice to take account of progress is seriously hampered if a limited ability to relevant research. read remains undiagnosed. Rigorous assessments of reading in the early years of school model ACER bases its work wherever possible on a the complexity of the skills required and identify developmental approach, which seeks to generate what students need to learn so that they have and promulgate information that can be used by the requisite skills to progress in the later years systems, schools or individuals to make and map of schooling. progress in relation to relevant learning objectives. ACER seeks to take into account broad learning Assessments of literacy and objectives in teaching and learning, and in developmental scales assessment processes. It conceives of learning In an assessment that measures the mastery domains as comprising an intricate web of of facts and processes, if students manage knowledge, skills and understanding that will to complete a discrete task successfully, the be relevant to and can be applied in the work results do little more than reveal their successful and life of individuals in the 21st century. These completion, and perhaps highlight which other approaches taken by ACER can be summarised as discrete tasks they should be able to complete the literacy orientation. successfully. If students do not complete a task successfully, the results merely reveal that their attempts were unsuccessful, and perhaps highlight which other discrete tasks they may not be able to complete successfully. The results are atomised and lacking in nuance, and they do not tell us how well the education system is preparing students for the demands of life. In an ACER assessment with a literacy orientation, the results locate students on empirically derived developmental scales. The scales are continuous and cover a range of proficiency levels. Each proficiency level is described in real-life terms. Given their continuous and wide-ranging nature, and their literacy – or real-life – orientation, these scales can be powerful tools for tracking student progress towards the attainment of a set of skills that enable them to participate fully in life beyond the classroom. In addition, since the proficiency levels in the scales are not tied to curriculum goals, they can be applied across different settings

The ‘literacy’ idea 4 References Masters, G.N. (2013). Reforming Educational Assessment: Imperatives, principles and challenges. Australian Education Review 57. Melbourne: ACER. Available at http://research.acer.edu.au/aer/12

Simon, H. A. (1980). Problem Solving and education. In Tuma, D. T. and Reif, R. (eds.) Problem Solving and Education: Issues in Teaching and Research, Hillsdale, HJ: Erlbaum.

Snow, C.E., Burns, M.S. & Griffin, P. (Eds.) (1998). Preventing Reading Difficulties in Young Children: Report of the Committee on the Prevention of Reading Difficulties in Young Children Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

Whitehead, A. N. (1929). The Aims of Education and Other Essays. New York: The Free Press.

The ACER Centre for Global Education Monitoring supports the monitoring of educational outcomes worldwide, holding the view that the systematic and strategic collection of data on educational outcomes, and factors related to those outcomes, can inform policy aimed at improving educational progress for all learners.

The ‘literacy’ idea 5