Characteristics of Group Oral Interactions Performed by Japanese Learners of English
Total Page:16
File Type:pdf, Size:1020Kb
Characteristics of Group Oral Interactions Performed by Japanese Learners of English A doctoral dissertation submitted for the degree of Doctor of Education Graduate School of Education Waseda University Junko Negishi 2011 Abstract Group oral test formats have recently been added to the types of oral performance tests. This is because the assessment of second language (L2) learners’ authentic conversational competence is considered important in the current era of globalization. However, only a few studies dealing with group oral interaction have been carried out to date. This study aimed to identify some characteristics and developmental phenomena of Japanese learners of English by investigating what happened when they underwent group oral interactions in groups of three. Participants comprised of 145 students from junior high schools, senior high schools, and universities. Ten Japanese raters assessed the participants utilizing criteria of the Common European Framework of Reference (CEFR; Council of Europe, 2001) which is currently being used throughout the world. In order to explore the feasibility of employing Japanese raters and adopting the CEFR criteria, first, three facets—the severity of raters, the difficulty of rating categories, and participants’ speaking ability—were examined by means of multi-faceted Rasch measurement, which analyzed sources of variation and estimated “measures” from the raw scores given by raters. Secondly, the participants’ discourse was investigated from various points of view based on five subcategories in the CEFR criteria: Range, Accuracy, Fluency, Interaction, and Coherence. The analysis was carried out ii on 82 discrete items that might have the potential to explain speakers’ development. Next, participants’ characteristics which exhibited developmental features and had relationships to the CEFR criteria were identified and further explored. The last method taken was to find an overall picture of the study, utilizing neural network analysis. The major findings were as follows: the raters were considered to be self-consistent, but showed significant differences among themselves as well as with other researches. The CEFR rating criteria were evidenced to be valid; however, more detailed rating scales should be created for the situation in which all participants are Japanese, as the speaking ability of most participants was judged as “low”. As for the relationship between the various range-related indices and the CEFR measures, lexical diversity could indicate wider range of talk. According to the results of error analysis, accuracy did not significantly demonstrate the participants’ developmental characteristics or relationships with the CEFR measures. Although fluency is said to be a major factor in judging L2 speakers’ proficiency, this study did not clearly support the accepted notions except for temporal variables. Qualitative analysis concerning Interaction discovered a new pattern termed “under-developed” in addition to the three basic patterns of interaction that Galaczi (2004, 2008) advocated. As for the relationship between the interactional patterns and the CEFR measures, the observed high correlation was largely explained by group rather than individual iii characteristics, which was also demonstrated in the analysis of Coherence. The result suggested the possibility of assigning joint scores to a group and the necessity of further research. Neural network analysis helped estimate the participants’ global oral interactional ability from a few number of items, which might be more practical than to analyze the speakers’ discourse on a number of items. The eight items used to estimate were: the number of types, the total number of formulaic sequences (Range), the ratio of self-corrections per error (Accuracy), the total speaking time including pause time, the number of syllables including dysfluency, the total number of words (Fluency), the total scores of Global Interactional Patterns (group + individual; Interaction), and the number of words used as a group (Coherence). The last two items from Interaction and Coherence were group-related data, and they might have contributed, in a certain sense, to the group oral interaction. The correlation coefficient between the estimated values and the CEFR measures (observed values) calculated by the help of neural network analysis was 0.840. The group oral is an effective format for L2 learners, providing more opportunities to interact with peers, irrespective of its threat to construct validity. What teachers need to do in a classroom is to provide students with opportunities to express themselves interactively using English. The group oral interaction activity will bring about good relations between classroom activity and the group oral test format. iv Acknowledgements I wish to acknowledge my deep appreciation to the following people for their ongoing encouragement and advice: this doctoral thesis could not have been completed without their support. I would like to express my utmost appreciation to my supervisor, Professor Michiko Nakano, for her guidance throughout my work on this study. Her insightful comments and advice have motivated me to strive for the best. My special thanks and gratitude to Mr. Hajime Tsubaki and Dr. Yoshihito Sugita for their professional advice concerning statistical analysis. I am deeply grateful to my colleagues and to all the students who participated in the study. I appreciate their willingness to spend time rating the participants and talking in groups of three. This study would not have been possible without their support and participation. I am grateful to Mr. Chris Kennard and Ms. Lia Hasegawa for helping to edit the final drafts of this doctoral dissertation. Finally, I owe my deepest gratitude to my husband, Masami, for his constant encouragement and support, and for giving me an opportunity to study. I also am personally indebted to my family for their understanding and warm-hearted affection. v Table of Contents Abstract ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ ii Acknowledgement ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ v Table of Contents ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ vi List of Tables ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ xii List of Figures ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ xv Transcript Conventions ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ xvii Chapter 1 INTRODUCTION 1 1.1 Introduction ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 1 1.2 RationalefortheStudy ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 4 1.2.1 Course of Study and Communication Skills ・・・・・・・・・・・・・・・・ 4 1.2.2 Performance Test Format ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 6 1.2.3 Performance Assessment ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 8 1.3 Purpose of the Study ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 10 1.4 Research Questions ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 11 1.5 Significance of the Study ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 12 1.6 Definition of the Terms ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 14 1.6.1 Communicative Competence and Language Ability ・・・・・・・・・ 14 1.6.2 Interactional Competence ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 16 1.6.3 Co-constructed Interaction ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 18 1.7 Organization of the Dissertation ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 19 Chapter2 BACKGROUNDOFTHESTUDY 22 2.1 Introduction ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 22 2.2 Previous Studies on Paired Orals ・・・・・・・・・・・・・・・・・・・・・・・・・・・ 23 2.3 Previous Studies on Group Orals ・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 25 2.4 Disadvantages of Group Orals ・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 27 2.5 Advantages of Group Orals ・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 30 2.6 Summary ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 34 Chapter3 PROCEDURE 35 3.1 Introduction ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 35 3.2 Participants ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 35 3.3 Data Collection ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 37 3.4 Transcription Procedure ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 39 3.5 Rating Procedure ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 41 3.6 Analysis Procedure ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 43 vi 3.7 Summary ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 44 Chapter4 RATINGS 46 4.1 Introduction ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 46 4.2 Common European Framework of Reference: CEFR・・・・・・・・・・・ 46 4.3 Performance Assessment・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 54 4.4 Multi-Faceted Rasch Measurement・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 57 4.5 Unidimensionality of the Rasch Model・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 60 4.5.1 Objections to the Assumption of Unidimensionality ・・・・・・・・・ 60 4.5.2 Approval of the Assumption of Unidimensionality ・・・・・・・・・ 61 4.6 FACETS・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 63 4.7 Raw Scores ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 64 4.8 DataAnalysis by Multi-Faceted Rasch Measurement (FACETS)・・ 67 4.8.1 Rating Categories ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 68 4.8.2 Rater Measurement Report ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 74 4.8.3 All FACETS Summary ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 77 4.8.4 Person Fit ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 80 4.8.5 Bias Analysis ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 83 4.9 Discussion ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 92 4.10 Summary ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 95 Chapter5 CEFR(I)RANGE ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・ 97 5.1 Introduction ・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・・