Literacy Assessment New Zealand Style

University of Kentucky UKnowledge

Educational, School, and Counseling Educational, School, and Counseling Psychology Faculty Publications Psychology

10-2006

Literacy Assessment New Zealand Style

Thomas R. Guskey University of Kentucky, [email protected]

Jeffrey K. Smith University of Otago, New Zealand

Lisa F. Smith Educational Assessment Research Unit, New Zealand

Terry Crooks University of Otago, New Zealand

Lester Flockton Educational Assessment Research Unit, New Zealand

Follow this and additional works at: https://uknowledge.uky.edu/edp_facpub

Part of the Educational Assessment, Evaluation, and Research Commons Right click to open a feedback form in a new tab to let us know how this document benefits ou.y

Repository Citation Guskey, Thomas R.; Smith, Jeffrey K.; Smith, Lisa F.; Crooks, Terry; and Flockton, Lester, "Literacy Assessment New Zealand Style" (2006). Educational, School, and Counseling Psychology Faculty Publications. 12. https://uknowledge.uky.edu/edp_facpub/12

This Article is brought to you for free and open access by the Educational, School, and Counseling Psychology at UKnowledge. It has been accepted for inclusion in Educational, School, and Counseling Psychology Faculty Publications by an authorized administrator of UKnowledge. For more information, please contact [email protected]. Literacy Assessment New Zealand Style Notes/Citation Information Published in Educational Leadership, v. 64, issue 2, p. 74-79.

Republished for research purposes only. No further duplication or transmission of this article is allowed without the written permission of ASCD. Please contact the Permissions Unit at [email protected].

This article is available at UKnowledge: https://uknowledge.uky.edu/edp_facpub/12

Published in Educational Leadership, v. 64, issue 2, p. 74-79.

Literacy Assessment

Ws mostly performance based. It assesses students in teams as well as individually. Whats more, students like it.

Thomas R. Guskey, Jeffrey K. Smith, Lisa F. Smith, Terry Crooks, and Lester Flockton

magine students or Year 13, depending on striving to do well on the students postsecondary literacy assessments aspirations. Students because the tasks are leaving after Year 12 usually interesting and enter the workforce, engagingI , not because of the whereas those leaving after consequences that might Year 13 typically go on to follow poor performance. higher education. New Consider the advantages of Zealand schools have no having data that issue from equivalent for a U.S. High authentic literacy tasks, c school diploma or gradua- rather than from items that l tion certificate. High school require students to simply I students earn "qualifica- fill tn the hianks or read a t tions" and even scholarships passage and answer a few lor university study through related questions. Picture a I a system of assessments literacy assessment program ^ controlled hy the New driven by teachers' need for Zealand Qualifications formative information about student standing student achievement in Authority All New Zealand universities learning instead of by high-stakes language arts. have open admission policies and accountability demands. admit any student who has earned the U.S. educators, struggling under the Education in New Zealand: necessary qualifications. The country accountability requirements of No A Quick Overview has no private universities. Child Left Behind, might consider such New Zealand has roughly 4 million an assessment program the stuff of inhabitants, with a student population The National Educational education fantasies. But for New of approximately three quarters of a Monitoring Project Zealand educators, it is an integral part million- For most children, school Teachers in New Zealand schools have of a comprehensive assessment begins on the day that they turn 5, a variety of assessment resources avail- program designed to help teachers regardless of whether that takes place able to them, ranging from standard- enhance students' literacy skills. New in March or November. On that day, ized tests and national exemplars of Zealand assesses the learning progress the child joins an existing Year 1 class student work to the reports and of elementary and middle-level in the school of the parents' choice. The samples of assessment tasks offered by students through the National Educa- vasl majority of New Zealand children the National Educational Monitoring tional Monitoring Project, which offers attend the school nearest their home. Project (NEMP). NEMP is a national an innovative approach to under- Schooling t)-picaliy ends after Year 12 formative evaluation effort that

74 EDUCATIONAL LEADERSHIP/OCTOBER 2006 New Zealand Style

provides New Zealand educators with randomly selected from each school number of tasks that the assessment an accurate picture of students' educa- (12 from Year 4 and 12 from Year 8), can include. The 2004 Reading and tion progress at Year 4 and Year 8, unless fewer tban 12 students are Speaking assessment included 52 tasks, which are roughly comparable to enrolled in either oi the two grades. In the 2002 Listening and Viewing assess- grades 3 and 7 in the United Slates. tbat event, the school is paired with a ment included 37 tasks, and tbe 2002 Each year, the project tests a random neighboring small school to get a group Wnting assessment included 36 tasks. sample of 1,440 Year 4 students and of 12. Although school and student Because students are di\'ided mto three 1,440 Year 8 students (Crooks &r participation in NEMP is voluntary, groups, each student is involved in Flockton, 2003; Elockton & Crooks, participation rates consistently exceed approximately one-third of the tasks in 2003a. 2003b). The sampling proceeds 95 percent. eacb topic area. in two stages. First, participating The selected 1,440 students at eacb Unlike the National Assessment of schools are randomly selected on a of the two grade levels are randomly Educational Progress (NAEP) or various stratified basis, witb the region of tbe assigned to one of three groups. Each statewide assessment programs in the country, the district withm the region, group of 480 students receives a sepa- United States, the NEMP assesses and the school size as the basis for rate set of tasks for the subject area nearly all areas of the New Zealand stratification. Second, 12 students are tested, thereby increasing the range and curriculum on a four-year, rotating

ASSOCIATION FOR SUPERVISION AND CURRICULUM DEVELOPMENT 75 cycle. Since its inception in 1995, NEMP has assessed the following: • In 2003. 1999, and 1995: science, visual arts, and information skills (working with graphs, tables, maps, charts, and diagrams). • In 2004, 2000, and 1996: language (reading and speaking), aspects of tech- nology, and music- • In 2005, 2001. and 1997: mathe- matics, social studies, and information skills (library research). • In 2006, 2002. and 1998: language (writing, listening, viewing) and health/physical education. The NEMP approach begins vAih a one-page assessment framework for each assessed area that describes the specific skills, knowledge, understand- ings, and attitudes of enduring impor- tance that the project will assess. For example, the central organizing theme New Zealand educators see literacy as of the 2004 Reading Framework was constructing meaning from a range of comprising skills in reading, speaking, texts for a variety of purposes: reading for enjoyment, reading to fallow instruc- writing, listening, viewing, and presenting. tions, reading to search for information, reading to assimilate knowledge, and tasks, which involve four different reading to analyze critically. aspect of the assessment framework; (2) presentation approaches: they are relevant to the student's world; The framework lists 15 characteris- • One-to-one interviews, in which (3) they are engaging to students; and tics of reading—such as "reading is students work individually with the (4) they generate enough useful infor- both a social and a personal activity" teacher and student responses are mation to merit the time spent on the and "reading in one language can recorded on videotape. task. enrich and support reading in another • Stations, where four students work New Zealand educators receive the language"—as well as 16 skills and independently rotating among a set of results for each task scored separately, processes that the assessment tasks, some of which are computer with a complete description of the task involves—such as "making use of based. provided. Reports include the total semantic, syntactic, and visual cues in • Teams, in which groups of four score for each task but do not provide text" and "making self-corrections." students work collaboratively on tasks an overall score for the assessed area, Three motivation items are also listed supervised by the teacher, with their which would encourage a simplistic that focus on enthusiasm, voluntary interactions videotaped. reading of the results rather than the engagement, and commitment. Each • Independent, in which students more in-depth reading required to go year, national panels of content area work individually, completing paper- through task results individually. experts, including practicing teachers and-pencil tasks, creating works of art, A novel aspect of the NEMP and curriculum specialists, review and or performing physical tasks, with their approach is that Year 4 and Year 8 revise that year's frameworks. performance videotaped. students complete the same tasks. This The assessment framework provides Tasks are chosen on the basis of four enables teachers to examine the nature the basis for developing the assessment criteria: (1) they measure an important and extent of the growth that occurs

76 EDUCATIONAL LEADERSHiP/OcroBER 2006 during this time period. Also, roughly 55 percent of the tasks are released The Zippo Task from tlie 2002 Nationai Educationai Monitoring Project with the results, but about 45 percent are "held back," meaning that full Trend Task: 2\PP0S details are not presented on these tasks Approach: Independent Year: 4 & 8 so that they can be administered four Focus: Recalling infornnation, drawing appropriate conclusions, representing years later, on the next cycle of assess- information visually ment in that content area. Once these "held-back" tasks have been adminis- Resources: Audio recording on Year 4 Exemplars—High tered a second time, both sets of results laptop computer are released to provide a look at change Audio script: In this activity, you are over time. Results are reported only at going to drav*/ a Zippo. Zippos live in the national level and are disaggregated caves and never come Into the light. by gender, ethnicity, and socioeconomic Nobody has ever seen a Zippo so we status. No individual student or school don't know exactly what they look like. scores are provided. You are going to hear some information about Zippos. Use this information to Assessing Students in Literacy draw what you think a Zippo would look New Zealand educators see literacy as like. You can start drawing as the infor- comprising skills in reading, speaking, mation is given to you. The information will be given twice. You can check your writing, listening, viewing, and drawing when the information is given presenting, Reading and speaking are the second time. assessed in one year of the four-year cycle, whereas listening, viewing, and Facts about Zippos; writing are assessed in a separate year. • Zippos are round in shape. This Presenting is currently not assessed due helps them to roll from place to place. 10 time constraints. • They have 2 large round eyes that help them to see in the dark caves. Reading and Speaking Each eye is made up of 4 circles. New Zealand's Maori heritage and the • Zippos use their large wings to fly country's concern for the environment about in the cave, where they feed ligure prominently in NEMP tasks. One on spiders and insects. of the independent tasks from the 2004 • Zippos are friendiy. They have big, Reading and Speaking assessment happy smiles. measures students' ability to properly • Zippos have 3 short hairy legs and pronounce a number of Maori words. claws on their feet. Many of these words appear in everyday Now listen to the information again. speech and in place-names throughout Source.-From the WEMP2002 m-ir/ngfteport, byT. Crooks and L. Flockton, 2002, Educational New Zealand, such as the word Assessment Research Unit, New Zealand. Adapted with permission. mapehu, which means volcano and is also the name of one of the largest active volcanoes in New Zealand. done four years earlier. This informa- periortn in a group. Teams of four Results from the 2004 assessment tion shows teachers that although students are given the poem "Night showed that Year 8 students did 25 substantial work remains, older Noises," which describes some of the percent better on the pronunciation students are making clear progress on noises that animals make at night. msk than Year 4 students did. Further- Maori pronunciation. Students develop a lively oral presenta- more, Year 8 students performed better A speaking task used in the 2000 tion of the poem. The test adminis- on this task than Year 8 students had assessment measures how well students trator observes and scores the students'

ASSOCIATION FOR SUPERVISION AND CURRICULUM DEVELOPMENT 77 work during the 10 minutes of One task m the 2002 Writing preparation time as well as assessment calls for students to during the performance; the choose one picture from seven rubric includes both perfor- shown to them and write a story mance and collaboration about it within 15 minutes. measures. Students are told not to worr)' The ensuing repon on this about spelling because the focus task evaluates students on a is on writing and using their number of issues: Did all imaginations. Results do include students participate? How scores for spelling performance, expressively did students present however—not for individual the poem? How fluently? Did -: students, but tor the group. They different students present 3 reflect the growth from Year 4 to different parts of the perfor- ^ Year 8 and show teachers the mance? Did they imitate the degree to which students are animal sounds in the back- X, willing to use less familiar words ground? The report shows that even though they may be unsure Year 4 students outperformed I of the correct spellings. This task their Year 8 counterparts in Students work with manipulatives in numerous NEMP gives teachers clear ideas on how expression, whereas Year 8 assessment tasks, and tasks are often videotaped. to structure writing assignments, students were more fluent in offers exemplars to guide scoring their presentation and more likely to made up of four circles. Both Year 4 and structure feedback to students, have different students present different and Year 8 students managed well on and provides direct evidence of parts of the poem. Year 4 students this task, but Year 8 students did improvements in language use and improved somewhat in this task notably better on the finer details. vmting skills. compared with four years earlier. The Zippo task holds great potential Teachers can see from these results how for formative assessment. First, teachers Subject Area Surveys well students do in general working can try this in their o\\n classrooms The NEMF assesstnents include tv^'o together on a task and how their and compare their results with national additional components. First, students performance compares with national norms. Second, this task provides a are asked whether they enjoyed doing standards. great example of how to assess—and the assessment tasks. Assessment devel- have students self-assess—their ability opers use this information both in Listening and Viewing to follow directions in an enjoyable and considering the validity of a task and in One independent task used in the 2002 nonthreatening fashion. developing future tasks. Results to date Listening and Viewing assessment asks show that a large majority of students students to listen and respond to mate- Wtiling have given the tasks positive reviews. In rial read aloud, which includes literary Writing is a particularly difficult area to addition, students complete a brief texts as well as practical messages, like assess within the time limits of conven- survey that asks about their in-school weather reports, phone messages, and tional assessment. The NEMP approach and out-of-school activities, how much instructions. The computer-based presents students with a variety of they like a given subject area, and how Zippo task (see p. 77) requires students writing prompts and tasks, including they, their teachers, and their parents to listen to a soundtrack that pro\ides one multiple-day task that takes perceive their performance in that information that will help them draw a students three separate writing sessions subject area. "Zippo." The resulting drawings, to complete. In addition to both With more than 10 years of data on regardless of level of artistry, reflect expressive and functional writing, these affective dimensions, the NEMP students' abilities to listen to and follow students are assessed on such conven- team has developed a strong knowledge instructions, which call for the Zippo to tions as spelling and grammar and on base about the sorts of tasks that both have three legs as well as two eyes their editing ability. appeal to students and provide teachers

78 EDUCATIONAL LEADERSHIP/OCTOBER 2006 YouVe changed lives. Now change the system. A novel aspect of the NEMP approach is that Year 4 and Year 8 students complete the same tasks.

with good fonnative information. The Zealand educators realize the value of tasks in NEMP also serve as exemplars high-quality formative assessment data to teachers of how they can effectively drawn from authentic literacy tasks. assess students in more engaging ways. For additional information on New Assessments for Learning Zealand's National Educational Monitoring Project, visit http://nemp.otago.ac.nz/. In Maori, the native language of New Zealand, the root word ako connotes References both teaching and learning. To the Crooks, T., & Flockton, L. (2003). Reading EdD —Educational native Maori people, teaching and and speaking assessmenl results 2002: Leadership & Change learning are inseparable. Even indepen- National Educational Monitoiing Report 27. dent learning implies some form of self- Dunedin, New Zealand: Educational The EdD program's dynamic, multicul- Assessment Research Unit. teaching. By focusing primarily on tural learning community promotes Flockton, L.,

ASSOCIATION FOR SUPERVISION AND CURRICULUM DEVELOPMENT 79