Saif Shahrzad Phd 1999.Pdf
Total Page:16
File Type:pdf, Size:1020Kb
INFORMATION TO USERS This manuscript has been reproduced from the microfilm master. UMI films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may t>e from any type of computer printer. The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to t>e removed, a note will indicate the deletion. Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing from left to right in equal sections with small overlaps. Photographs included in the original manuscript have been reproduced xerographically in this copy. Higher quality 6” x 9” black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UMI directly to order. Bell & Howell Information and Learning 300 North Zeeb Road. Ann Arbor, Ml 48106-1346 USA UIVQ 800-521-0600 Theoretical and Empirical Considerations in Investigating Washback: A Study of ESL/EFL Learners by Shahrzad Saif B.A., Allameh Tabatabai University, 1984 M.A., Shiraz University, 1987 A Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of DOCTOR OF PHILOSOPHY in the Department of Linguistics We accept this dissertation as conforming to the required standard Dr. Jo Supervisor (Departmnt of Linguistics) Dr. Joseph F. KessTDepartmental Member (Department o f Linguistics) Dr. -----------Barbara (Department of Linguistics) Dr. Jonn o!^ And^on, Outside Member (Department of Educational Psychology and Leadership Studies) _______________________________ Dr. Jared Bernstein, External Examiner (Department of Linguistics, Stanford University) © Shahrzad Saif, 1999 University of Victoria All rights reserved. This dissertation may not be reproduced in whole or in part, by photocopy or other means, without the permission of the author. u Supervisor: Dr. John H. Esling ABSTRACT Researchers’ and educators’ recognition of the positive/negative effects of tests on teaching and learning activities goes back at least four decades. However, this phenomenon, referred to as “washback” in the applied linguistic literature, has been examined empirically by only a few studies in the field o f language testing. Even fewer have based their investigation into washback on an a priori theory outlining the scope and design of the study. This study examines washback as a phenomenon relating those factors that directly affect the test to those areas most likely to be affected by the test. The goals of the study are: (i) to investigate the existence and nature o f the washback phenomenon, (ii) to identify the areas directly/indirectly affected by washback, and (iii) to examine the role of test context, construct, task, and status in promoting beneficial washback. Theoretically, this study conceptualizes washback based on the current theory of validity proposed by Messick (1989, 1996). It is defined as a phenomenon related to the consequential aspect of the test’s construct validity and thus achievable, to a large extent, through the test’s design and administration. Given this assumption, a conceptual and methodological fiomewoik is proposed that identifies “needs”, “means”, and “consequences” as the major focus areas in the study of washback. While the model recognizes tests of language abilities as instrumental in bringing about washback effects, it highlights an analysis of the needs and objectives of the learners (and of the educational system) and their relationship with the areas influenced by washback as the starting point U1 for any study of washback. Areas most likely to be affected by the test, as well as major variables that can potentially promote or hinder the occurrence of washback, are also delineated by the model. This theoretical fiamework is examined empirically in this study through a long term multi-phase investigation conducted in different educational contexts (EFL/ESL), at different levels of proficiency (advanced/intermediate), with different tasks (oral/written) and different groups o f subjects. The stages in the experimental part o f the study correspond to the different phases of the theoretical fiamework underlying the investigation. The approach to data collection is both quantitative and qualitative. The results o f the study indicate that positive washback can in fact occur if test constructs and tasks are informed by the needs of both the learners and the educational context for which they are intended. The extent, directness, and depth of washback, however, are found to vary in different areas likely to be influenced by washback. The areas most influenced by washback are found to be those related to immediate classroom contexts: (i) teachers’ choice o f materials, (ii) teaching activities, (iii) learners’ strategies, and (iv) learning outcomes. The study also reveals that non-test-related forces and factors operative in a given educational system might prevent or delay beneficial washback fixim happening. Based on the theoretical assumption underlying the definition of washback adopted in this study, such consequences which cannot be traced back to the construct of the test are outside the limits o f a washback study. IV Examiners: Dr. Esfing, Supervisor (Depafment of Linguistics) Dr. Joseph F. Kess“, uepartmental Member (Department of Linguistics) DrBarbâr Member (Department of Linguistics) Dr. John O. Anderson, Oytside Member (Department o f Educational Psychology and adership Studies) Dr. Jared Bernstein, External Examiner (Department o f Linguistics, Stanford University) TABLE OF CONTENTS ABSTRACT............................................................................................................................ U TABLE OF CONTENTS...................................................................................................... v TABLES ..................................................................................................................................ix FIGURES................................................................................................................................ I ACKNOWLEDGEMENTS ................................................................................................xi DEDICATION .................................................................................................................... xüi PART ONE: PRELIMINARY REMARKS.......................................................................1 CHAPTER ONE: INTRODUCTION...............................................................................2 1.1 Teaching and testing.............................................................................................. 2 1.2 The study..................................................................................................................4 1.3 The organization o f the text ...................................................................................6 PART TWO: THEORETICAL CONSIDERATIONS...................................................12 CHAPTER TWO: VALIDITY.........................................................................................13 2.1 Introduction ......................................................................................................... 13 2.2 Traditional approaches to validity .................................................................... 14 2.2.1 Types o f validity ................................................................................... 15 2.2.1.1 Content validity ................................................................15 2.2.1.2 Criterion validity ............................................................ 17 2.2.1.3 Construct validity ........................................................... 20 2.2.2 The evolution o f the validity concept: A historical approach 21 2.3 Current issues in validity.....................................................................................25 2.4 General view o f validity adopted in this study ...................................................33 CHAPTER THREE: WASHBACK................................................................................39 3.1 Introduction ...........................................................................................................39 3.2 Defining washback ................................................................................................40 3.3 The Washback mechanism ...................................................................................44 3.4 Towards positive washback ..................................................................................48 3.5 Measuring washback........................................................................................... 54 3.6 Previous studies in washback .............................................................................55 VI 3.7 The washback hypothesis proposed in this study ............................................. 66 CHAPTER FOUR: PERFORMANCE TESTING.........................................................72 4.1 Introduction .........................................................................................................