DOCUMENT RESUME ED 365 144 FL 021 757 AUTHOR Anivan, Sarinee, Ed. TITLE Current Developments in Language Testing. Anthology Series 25. INSTITUTION Southeast Asian Ministers of Education Organization (Singapore). Regional Language Centre. REPORT NO ISBN-9971-74-037-0; ISSN-0129-8895; RELCP383-91 PUB DATE 91 NOTE 271p.; Selected papers presented at the Regional Language Centre Seminar on Language Testing and Language Programme Evaluation (April 9-12, 1990). For selected individual papers, see also ED 328 072; ED 350 819, ED 322 725, ED 342 211, ED 317 073, ED 322 729. PUB TYPE Collected Works General (020) Collected Works Serials (022) EDRS PRICE MF01/PC11 Plus Postage. DESCRIPTORS Communicative Competence (Languages); Comparative Analysis; Computer Assisted Testing; Cues; English (Second Language); Evaluation CriterCa; Instructional Materials; International Programs; Interviews; Item Response Theory; Language Aptitude; *Language Proficiency; Language Research; *Language Tests; *Second Languages; Simulation; Sociocultural Patterns; Speech Skills; Teacher Developed Materials; *Test Format; *Testing; Testing Programs; Test Validity; Trend Analysis; Writing Evaluation; Writing Exercise IDENTIFIERS *Oral Proficiency Testing ABSTRACT The selection of papers on language testing includes: "Language Testing in the 1990s: How Far Have We Come? How Much Further Have We To Go?" (J. Charles Alderson); "Current Research/Development in Language Testing" (John W. 011er, Jr.); "The Difficulties of Difficulty: Prompts in Writing Assessment" (Liz Hamp-Lyons, Sheila Prochnow); "The Validity of Writing Test Tasks" (John Read); "Affective Factors in the Assessment of Oral Interaction: Gender and Status" (Don Porter); "Authenticity in Foreign Language Testing" (Peter Doye); "Evaluating Communicative Tests" (Keith Morrow); "Materials-Based Tests: How Well Do They Work?" (Michael Milanovic); "Defining Language Ability: The Criteria for Criteria" (Geoff Brindley); "The Role of Item Response Theory in Language Test Validation" (T. F. McNamara); "The International English Language Testing System IELTS: Its Nature and Development" (D. E. Ingram); "A Comparative Analysis of Simulated and Direct Oral Proficiency Interviews" (Charles W. Stansfield); "Southeast Asian Languages Proficiency Examinations" (James Dean Brown, H. Gary Cook, Charles Lockhart, Teresita Ramos); "Continuous Assessment in the Oral Communication Class: Teacher Constructed Test" (Shanta Nair-Venugopal); and "What We Can Do with Computerized Adaptive Testing...And What We Cannot Do!" (Michel Laurier). (MSE) CURRENT DEVELOPMENTS INLANGUAGE TESTING REGIONAL LANGUAGE CENTRE Edited by SARINEE ANIVAN SEAMEO ANTHOLOGY SERIES25 fill 11 1111111 III111 till PERMISSION 1 0 REPRODUCE. T HIS U S. DEPAWTMENT OF EDUCATION Othce or EctucahonaIResevch and Improvement MATERIAL HAS BEEN GRANTED BY EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) 9,17.../us document has been reproduced as rece.ne4 from the person or organization originating 0 Mmor changes have been made to improve reproduction duality Pthntsol ,/vr Or opinions stated in this docu- TO THE EDUCATIONAL RESOURCES ment do not neCessanly rnpresentotfrthal INFORMATION CENTER IERICI- OERI ouseton cr ooliCy III 40 .., CURRENT DEVELOPMENTS IN LANGUAGE TESTING CURRENT DEVELOPMENTS IN LANGUAGE TESTING Edited by Sarin= Anivan Anthology Series 25 Published by SEAMEO Regional Language Centre c.4. RELCP383-91 se. Copyright 1991 by SEAMEO Regional Language Centre 30 Orange Grove Road Singapore 1025 ISBN 9971-74-037-0 ISSN 0129-8895 Printed by Fong & Sow, Printets Pte. 1.td. () CONTENTS Page Foreword iv Introduction Language Testing in the 1990s: How Far Have 1 We Come? How Much Further Have We To Go? I Charles Alderson Current Research/Development in Language 27 Testing John W 01 ler Jr The Difficulties of Difficulty: Prompts in 58 Writing Assessment Liz Hamp-Lyons and Sheila Prochnow The Validity of Writing Test Tasks 77 John Read Affective Factors in the Assessment of Oral 92 Interaction: Gender and Status Don Porter Authenticity in Foreign Language Testing 103 Peter Dove Evaluating Communicative Tests 111 Keith Morrow Materials-13a=! Tests: How Well Do They 119 Work? Michael Milanovic Defining Language Ability: T ne Criteria 139 For Criteria Geoff Brindley The Role of Item Resporse Theory in 165 Language Test Validation T F McNamara The International English Language Testing 185 System 1ELTS: Its Nature and Development D E Ingrain A Comparative Analysis of Simulated and 199 Direct Oral Proficiency Interviews Charles W Stansfield 7 Southeast Asian Languages Proficiency 210 Examinations James Dean Brown H Gary Cook Charles Lockhart Teresita Ramos Continuous Assessment in the Oral 230 Communication Class: Teacher Constructed Test Shanta Nair-Venugopal What We Can Do with Computerized Adaptive 244 Testing... And What We Cannot Do! Michel Laurier List of Contributors 256 iii FOREWORD Considerable interest in current language testing and the evaluation of language teaching programmes amongst both researchers and language educators was evidenced by the excellent and wide-ranging attendance at RELC's Regional Seminar on Language Testing and Language Programme Evaluation held from 9-12 April 1990. Many well-known personalities in the field, from within and outside the SEAMEO member colntries, attended and gave papers or workshops.I am happy that RELC is able to publish selected papers that appear in this volume. The importance of language testing is recognized by virtualiy all professionals in the field of language education. It is of special importance in education systems that are highly competitive, as education systems in Southeast Asia tend to be, as testing is not only an indirect stimulus to learning, but plays a crucial role in determining the success or failure of an individual's career, with direct implications for his future earning power. Moreover, in countries where education is seen as central tosocio- economic development, it is important that tests be valid and reliable.It is our belief that where validity and reliability arc lacking, individuals as well as educational programmes suffer, and society at large is the loser. Thus, testing is an important tool in educational research and for programmeevaluation, and may even throw light on both thc nature of language proficiencyand language learning. Although the theme of the 1990 Seminar encompassed both testing and programme evaluation, it has not been possible to coverboth areas in one anthology, and the present volume deals with papers on testing. Thosethat deal with programme evaluation will be published in a separatevolume later. Thc papers presented hcrc are inevitably a selection, limitations of space having forced us to omit a number of papers.However, I am confident that the papers offered all make worthwhile contributions to the importantfield of language testing, and that thcy will be of interest to languageeducators both within and beyond Southeast Asia. Eatnest Lau Director, RELC 9 iv INTRODUCTION The hoped-for developments in language testing did not materialiseduring the 1980s.In spite of rapid growth in the field, many of the problemsthat existed a decade ago arc still with us. Alderson in his paper inthis volume sums up the past decade as one in which,"there has been much movement, a lot of it sideways and backwards and not much of it ... forwards% Hedescribes areas where language testing has made progress, but also lists problemswhich are as yet intractable, and includes recommendationsfor a wide range of topics on which research can bc target*.ed. Naturally, as in all research, thc approach taken and thebasis of the research is crucial to the findings. Oiler points out some ofthe fundamental problems of research in language testing. He argues for a greaterunderstanding of language proficiency from the point of view of semioticabilities and processes, as well as thc different perspectivesof the various people that are involved in the test making and test taking processes. Thevariances that show up in language proficiency tests could thus be correctly identified and their sourcesproperly attributed and controlled. The validity of the tests wouldbe more secure and their significance unambiguous. Language testing couldthen be a "proving ground" for language theories, and hence hdp to define the natureof language proficiency. A number of writers in this collection discuss sourcesof variance in tcst results-prompts arc discussed by Prochnow andHamp-Lyons, experience by Read, affective factors by Porter. Prochnow andHamp-Lyons report that prompts such as topic assignment andselection make a difference in grades. Read similarly shows that tcst type makes adifference in getting different performance from test takers. Porter argues that the genderof the interviewer is anothcr source of variance in tcst results. v According to Oiler, language learning will take place when learners can connect their text (discourse) with their own experience, and it is the congruence between tests and experience that is crucial in determining the validity of tests. So long a ; this congruence or connection exists, there is less need for real world experience or authenticity to be inserted into the tests. Boyé similarly offers the view that complete authenticity is neither possible nor desirable. He calls for some balance between authenticity and abstraction, since hc believes that what we gain in validity through authenticity may result in a loss in tcrms of generalisability. Broader educational traditions