Communicative Competence and Writing( Dissertation 全文 )

Game-Based Language Learning in ESL Classrooms: Effective Interventions and Influences on Students’ Vocabulary Title Acquisition, Communicative Competence and Writing( Dissertation_全文 )

Author(s) Wang, Qiao

Citation 京都大学

Issue Date 2020-03-23

URL https://doi.org/10.14989/doctor.k22538

Right 許諾条件により本文は2020-12-31に公開

Type Thesis or Dissertation

Textversion ETD

Kyoto University

Game-Based Language Learning in ESL Classrooms: Effective Interventions and Influences on Students’ Vocabulary Acquisition, Communicative Competence and Writing

Wang Qiao

Abstract

The role of games is widely investigated in language learning and an increasing number of practitioner-researchers are exploring ways to incorporate games into language classrooms to secure both higher student motivation and satisfactory learning outcomes. In this study, the researcher conducted three mixed-method studies with a stand-alone digital simulation game, The Sims 4, and focused on different aspects of language skills with iterated classroom interventions involving university students in Japan and China.

In the first study, the researcher started with a controlled experiment on classroom intervention that could help integrate the game into an ESL class focused on daily vocabulary for communication. The study targeted receptive knowledge of vocabulary, evaluating how certain classroom interventions could influence receptive knowledge of game vocabulary. Results showed that teacher instruction, appropriate gameplay tasks and presentational activities facilitated the acquisition of lexical items in the game, while peer interaction made little contribution as students barely communicated in English and topics for communication were overwhelmingly focused on gameplay techniques, rather than on game vocabulary. Based on results from the first study, in the second study the researcher changed peer interaction from mandatory to voluntary, allowing more time for other forms of classroom intervention. The second study continued with receptive vocabulary knowledge and went further to explore how the classroom-situated GBLL would influence students’ communicative competence. The students presented their gameplay episodes in class and answered questions from the audience. Their communicative competence was evaluated in accordance with a rubric specifically designed for the class, including three major metrics of fluency, interaction and content. Results reaffirmed the positive influence of GBLL on vocabulary acquisition and suggested that the GBLL class contributed most to content in presentational activities while interaction was limited to teacher-student communication. In addition, fluency showed little improvement, which the students attributed to the lack of preparation and explicit instruction on how to speak fluently. The third study is a single-participant experiment, where the researcher added writing tasks into the classroom and ensured ample interaction for the subject. Through this experiment, the researcher explored how GBLL would influence student writing and how writing as a scaffold would influence communicative competence. The study concluded that the student

was able to use game vocabulary in writing and retainment rate of such vocabulary was satisfactory. In addition, with better preparation and more interaction opportunities, the student developed enormously in interaction and in content during communicative activities, albeit showing less progress in fluency.

Apart from receptive and productive vocabulary knowledge and writing and speaking skills investigated in this research, the findings of the studies suggest that games in classrooms can benefit other language skills as well, such as reading and listening. Moreover, this study highlights the importance of the classroom teacher who plays a pivotal role in selecting appropriate games and designing targeted intervention, activities and evaluation methods. In future studies, the researcher proposes to extend the scope of this research to include the meta- game environment, including gaming communities such as online discussion fora and social network sites, as well as game streaming platforms as arenas for out-of-class activities that complement classroom-incorporated GBLL.

要約

言語学習におけるゲームの役割は広く研究されており、学生のモチベーションと学習成果の両方を確保するために、語学教育にゲームを取り入れる方法を模索し

ている実践研究者（Practitioner-researchers）が増えている。この文章では、研究者がスタンドアロン型シミュレーションゲーム「The Sims 4」を用いて混合研究法による 3 つの研究を実施し、言語スキルのいくつかの側面に焦点を合わせて日本と中国の大学生を対象とした教室において教育介入方法の反復改善を行った。最初の研究は、日常会話の語彙の学習に特化した ESL 教室（第二言語としての英語教室）にゲームを取り入れるための教育介入方法の対照実験から始まった。この研究は語彙の受容的知識を対象として、特定の教育介入方法によって学生のゲームの語彙に関する受容的知識がどう影響されるかを評価した。研究の結果は、教師の指導、ゲームプレイの適切なタスクおよび発表活動はゲーム内語彙の習得を促進する効果があったが、ピア・インテラクション（学習者同士のやりとり）の効果は薄かったことを示した。そのような結果に至ったのは、学生らがほとんど英語で交流せず、交流があってもゲーム内語彙より圧倒的にゲームプレイのテクニックのほうを中心としていたからである。最初の研究の結果に基づいて研究者は第二の研究を行った。この研究では、ピア・インテラクションを強制せずに、他の介入方法の実施により多くの時間を割り当てていた。第二の研究は継続して語彙の受容的知識を対象として、教室内の GBLL(ゲームベースの言語学習)によって学生のコミュニケーション能力がどう影響されるかについて更に深い探究をした。学生らはクラスでゲームプレイのエピソードについて発表し、質疑応答もしていた。そして流暢さ、やりとりのスキルおよび発表内容など 3 つの指標を含めてクラスごとに設けられたルーブリックによって学生らのコミュニケーション能力が評価された。その結果、 GBLL による語彙習得のプラス効果が再確認された。具体的な結論として、GBLL の教室は発表の内容に最も貢献があるが、教室でのやりとりは教師と学生の間に限定されていた。また、流暢さに英語で話せるための準備と明確的な指導が足りないため、流暢さの改善はほとんど見えていなかった。第三の研究は単一被験者法を用いた。被験者の十分なやりとりを確保するために、教室にライティングタスクを加えた。

iii

この実験では、GBLL によって学生のライティングスキルがどう影響されるか、足がかりとしてのライティングスキルによって学生のコミュニケーション能力がどう変わるかを研究した。研究の結果によって、学生らがゲーム内語彙をライティングに生かすことができることと習得した語彙の定着率が良好であることがわかった。さらに、より良く準備させてやりとりのチャンスをより多く与えたため、コミュニケーション活動では、学生らのやりとりと内容面におけるパフォーマンスは大幅な進

歩を見せたが、流暢さの進歩は少なかった。語彙の受容的知識や発表的知識、ライティングスキルおよび会話スキル以外に、前述の研究によって教室にゲームを取り入れることで読解や聴解など他の語学スキルの習得にも役立つことがわかった。さらに、前述の研究結果によって、適切なゲームを選別した上、介入方法や教室活動、評価手法の設計も行って中枢のような役割を果たす教師の重要性が強調されている。将来の研究について、研究者は今までの研究の範囲を拡大してメタゲーム環境を整備しようとしている。例えば、オンラインディスカッションや SNS などゲームプレイコミュニティを取り入れ、教室内の GBLL を補完するための教室外活動の場としてゲームストリーミングプラットフォー

ムも加えるなどが考えられる。

Acknowledgement

I would like to first thank every student who participated in and contributed to my research and subsequently to this thesis. My thanks also go to the teachers who acted as evaluators in assessing the students’ communicative competence and the China-based language institution Elixir English for providing facilities in the third study of this thesis. Without them, I would not have completed this coherent series of studies on game-based language learning.

I owe my greatest gratitude to my supervisor, Professor Mark Peterson, for his enormous support and kind help throughout my Ph.D. research. Professor Peterson helped recruit participants for my studies and offered very constructive feedback on my research design. Under his guidance, I was able to publish my studies in relevant journals and present my research to a larger audience. In particular, I really appreciate the light-hearted conversations with Prof. Peterson, as they gave me motivation and the courage to push forward with my research and Ph.D. study. He is both a mentor and a friend to me, the most one can expect from a supervisor.

My gratitude also extends to the Nishimura Financial Foundation, for providing me with its scholarship for overseas students throughout my Ph.D. years. I sincerely hope that my research would contribute to the cultural exchange between Japan and China, as is the core spirit of the Foundation.

A special thanks to Wang Haitao, my friend in graduate school who has helped me in submitting various administrative documents while I was away. Last but not the least, I would like to thank my husband Frank for his technical support these past years. He helped format my manuscripts to meet requirements from different journals. I did not expect such support and encouragement from all of these wonderful individuals.

Table of Contents

Abstract...... i

Acknowledgement ...... v

Table of Contents ...... vi

List of Tables ...... xii

List of Figures ...... xiv

List of Abbreviations ...... xvi

General Introduction ...... 1

1.1 Computer games in education ...... 1

1.2 Computer games in language learning ...... 2

1.3 Computer games in classrooms ...... 3

1.4 Computer games in language classrooms ...... 4

Game Selected for this Research ...... 6

2.1 Simulation games ...... 6

2.2 Introduction to The Sims 4 ...... 8

2.3 The Series of Studies in this Research and Their Relationships ...... 11

The First Study: Effective Intervention for Vocabulary Acquisition in a Game-Based Language Learning Classroom ...... 13

3.1 Introduction ...... 13

3.2 Background ...... 13

3.2.1 Classroom intervention ...... 13

3.2.2 Research Questions ...... 14

3.3 Methodology ...... 14

3.3.1 Participants ...... 15

3.3.2 Research design ...... 16

3.3.3 Data collection ...... 17 vi

3.3.3.1 Vocabulary quizzes ...... 17

3.3.3.2 Recordings ...... 17

3.3.3.3 Surveys ...... 18

3.3.4 Data Analysis ...... 18

3.3.4.1 Vocabulary quizzes...... 18

3.3.4.2 Recordings...... 19

3.3.4.3 Surveys ...... 20

3.4 Results ...... 20

3.4.1 Quantitative results ...... 20

3.4.2 Qualitative results ...... 22

3.4.2.1 Recordings ...... 23

3.4.2.2 Survey results...... 26

3.5 Discussion ...... 28

3.5.1 Quantitative data ...... 28

3.5.1.1 Qualitative data ...... 30

3.5.1.2 In-class activities ...... 31

3.6 Conclusions and limitations ...... 32

The Second Study: Communicative Competence in a GBLL Classroom ...... 34

4.1 Introduction ...... 34

4.2 Background ...... 34

4.2.1 Communicative Competence in Language Classrooms ...... 35

4.2.2 Communicative Competence in GBLL ...... 35

4.3 Methodology ...... 36

4.3.1 Research Question ...... 36

4.3.2 Participants ...... 38

4.3.3 Study Configurations ...... 41 vii

4.4 Data Collection and Analysis ...... 42

4.4.1 Data collection instruments ...... 42

4.4.1.1 Recordings ...... 42

4.4.1.2 Surveys ...... 42

4.4.1.3 Vocabulary tests ...... 42

4.4.2 Data analysis...... 43

4.4.2.1 Rubric for Evaluating Participants’ Communicative Competence ...... 43

4.4.2.2 Surveys analysis ...... 43

4.4.2.3 Vocabulary tests ...... 43

4.5 Results ...... 43

4.5.1 Evaluation of Communicative Competence ...... 43

4.5.1.1 Quantitative Results ...... 44

4.5.1.2 Qualitative Results ...... 45

4.5.2 Receptive Vocabulary Knowledge ...... 48

4.5.3 Post-questionnaire...... 50

4.5.4 Interview ...... 53

4.6 Discussion ...... 54

4.7 Conclusions ...... 57

The Third Study Part I: Narrative Writing in a GBLL Classroom ...... 59

5.1 Introduction ...... 59

5.2 Background ...... 59

5.2.1 Computer games and writing in general ...... 61

5.2.2 Computer games, topic familiarity and genre-based writing ...... 62

5.2.3 Current issues with evaluation of second language writing ...... 64

5.2.4 The present study ...... 65

5.3 Methodology ...... 66 viii

5.3.1 Pre-study survey ...... 66

5.3.1.1 Students interests ...... 66

5.3.1.2 Genre of writing ...... 67

5.3.1.3 Teacher feedback ...... 69

5.3.2 Indicators of writing quality ...... 70

5.3.3 Research questions...... 70

5.3.4 Study design ...... 71

5.3.4.1 Participant ...... 71

5.3.4.2 Study design ...... 72

5.3.4.3 Teacher instruction ...... 72

5.3.4.4 Feedback on writing ...... 73

5.3.4.5 Student gameplay and writing ...... 73

5.3.5 Data collection and analysis tools ...... 73

5.3.5.1 Data sources ...... 73

5.3.5.2 Syntactic Complexity Measures (SCMs) for generating syntactic similarity ...... 74

5.3.5.3 Jaccard similarity coefficient ...... 74

5.3.5.4 Analysis tools...... 75

5.4 Data analysis and results ...... 76

5.4.1 Vocabulary ...... 76

5.4.1.1 Productive use of game vocabulary ...... 77

5.4.1.2 Syntax ...... 79

5.4.1.3 Semantics ...... 82

5.4.2 Survey results ...... 85

5.5 Discussion ...... 86

5.5.1 Lexicon...... 86 ix

5.5.1.1 Saturation point for productive vocabulary in writing ...... 87

5.5.2 Syntax ...... 90

5.5.3 Semantics ...... 92

5.6 Limitations of the study ...... 92

5.7 Conclusion and Future Directions ...... 93

The Third Study Part II: Back to Communicative Competence ...... 95

6.1 Introduction ...... 95

6.2 Background ...... 95

6.2.1 Relationship between writing and speaking in second language classrooms ..... 95

6.2.2 Interaction in second language classrooms ...... 97

6.2.3 The present study ...... 99

6.3 Methodology ...... 99

6.3.1 Study design ...... 99

6.3.2 Data collection and analysis ...... 100

6.4 Result ...... 101

6.4.1 Rubric-based teacher evaluation ...... 101

6.4.1.1 Pre-test ...... 101

6.4.1.2 Writing-scaffolded interaction-rich speaking activity ...... 102

6.4.2 Speech rate ...... 105

6.4.2.1 Post-test ...... 107

6.4.2.2 Interview ...... 107

6.5 Discussion ...... 109

6.5.1 Research question ...... 109

6.5.2 Speech rate ...... 109

6.6 Conclusions ...... 110

Conclusions, Limitations and Future Research ...... 111 x

7.1 Conclusions ...... 111

7.2 Limitations ...... 112

7.3 Future Research ...... 113

References ...... 115

Sessional themes and activities for both groups ...... 126

The pre-questionnaire ...... 128

The post-questionnaire ...... 130

List of words in the vocabulary pre-test ...... 131

Requirements from the English entrance exam ...... 132

Sessional activities ...... 133

Evaluation rubric for presentation recordings ...... 135

The pre-questionnaire ...... 137

The post-questionnaire ...... 139

The pre-study survey ...... 142

List of student writing samples ...... 144

The vocabulary post-test ...... 146

Writing samples with revision: #5 and #14 ...... 147

The post-interview ...... 149

The 10th samples (original and revised) ...... 150

Pictures used in the pre-test and post-test ...... 152

Evaluation rubric for presentations ...... 153

List of Tables

Table 1. Evaluation of SIMs 4 against Chapelle’s criteria ...... 11

Table 2. Coding schemes of recordings ...... 19

Table 3. P-values of the control and experimental groups ...... 22

Table 4. The extent to which students were happy with the class ...... 27

Table 5. Mean scores, standard deviations and median scores of class performance ...... 44

Table 6. Individual scores on six vocabulary tests ...... 48

Table 7. Participants’ ratings of the game ...... 51

Table 8. Participants’ ratings of GBLL activities in improving communicative competence 52

Table 9. Participants opinions towards GBLL as compared to traditional classes ...... 52

Table 10. Overall rating of the course ...... 53

Table 11. Students’ interest towards GBW ...... 66

Table 12. Most common writing genres for the students ...... 67

Table 13. Use of background information in writing...... 68

Table 14. Most practical areas of writing where students wanted to improve their skills ...... 68

Table 15. Types of feedback the students received ...... 69

Table 16. Number of game words in two sets ...... 77

Table 17. Forgotten words in the delayed post-test ...... 78

Table 18. MLS, C/T and T/S in the original and revised writing samples...... 79

Table 19. Difference between the two sample sets on the three SCMs ...... 81

xii

Table 20. Jaccard similarity coefficient and semantic similarity between the original and revised writing samples ...... 83

Table 21. Rate of distinct words in each original sample ...... 88

Table 22. Teacher evaluation on the student’s communicative competence during the experiment ...... 102

Table 23. Speech rates of the students in the 13 sessions ...... 105

Table 24. Sessional activities...... 133

xiii

List of Figures

Figure 1. Typology of simulation games (Lean et al., 2008) ...... 7

Figure 2. Creating a Sim ...... 9

Figure 3. Sims’ interaction with the environment ...... 10

Figure 4. Input from explanatory/notification texts ...... 10

Figure 5. Overview of the three studies and their respective focuses in this research ...... 12

Figure 6. Three scenarios of students’ responses ...... 19

Figure 7. Quiz results of the experimental, control and baseline groups (full score: 100 points)

...... 21

Figure 8. Participants’ self-reporting on communicative competence ...... 39

Figure 9. Participants’ gaming experience ...... 39

Figure 10. Participants’ computer use ...... 40

Figure 11. Preference for gaming platforms (multiple choices) ...... 40

Figure 12. Perceptions towards the usefulness of GBLL ...... 41

Figure 13. Boxplot results on interaction, fluency and content ...... 44

Figure 14. Picture in excerpt 1 ...... 46

Figure 15. Boxplots of the vocabulary scores ...... 49

Figure 16. Comparison between the experimental group in the first study and students in the second study ...... 50

Figure 17. Principle of Jaccard similarity ...... 75

Figure 18. Distinct words in two sets ...... 76

xiv

Figure 19. Comparison on MLS between the two sets of samples ...... 80

Figure 20. Comparison on C/T between the two sets of samples ...... 80

Figure 21. Comparison on T/S between the two sets of samples ...... 81

Figure 22. Trends on the three differences ...... 82

Figure 23. Jaccard similarity coefficient and semantic similarity between the original and revised writing samples...... 83

Figure 24. Rate of new distinct words in a writing sample (Ri) ...... 89

Figure 25. Trendline for rates of new words in writing samples ...... 90

Figure 26. Teacher evaluation on the student’s communicative competence during the experiment ...... 102

Figure 27. Teacher evaluation of the student’s communicative competence overall ...... 103

Figure 28. Linear regression on speech rates ...... 106

Figure 29. Trendline of the student’s speech rates ...... 107

Figure 30. Entrance exam guidelines for English ...... 132

List of Abbreviations

ACTFL OPI American Council on the Teaching of Foreign Languages Oral Proficiency Interview

C/S Clause per sentence

C/T Clause per T-unit

CALL Computer-assisted language learning

CEFR Common European Framework of Reference for Languages

CET College English test

CLT Communicative language teaching

CMC Computer-mediated communication

COTS Commercial off-the-shelf

EFL English as a foreign language

ESL English as a second language

FLOSEM Foreign Language Oral Skill Evaluation Matrix

GBLL Game-based language learning

GBW Game-based writing

L1 First language

L2 Second language

L2SCA L2 Syntactic Complexity Analyzer

MLC Mean length per clause

xvi

MLS Mean length per sentence

MLT Mean length per T-unit

MMORPG Massive multiplayer online role-playing game

NLP Natural language processing

PUBG PlayerUnknown's Batterground

SCM Syntactic complexity measure

T/S T-unit per sentence

TBLT Task-based language teaching

TL Target language

VR Virtual reality

WTC Willingness to communicate

xvii

General Introduction

1.1 Computer games in education

According to a 2019 study (“Number of Gamers Worldwide 2021”, 2019) on the number of video gamers worldwide provided by Statista (“Statista”, n.d.), a global business data platform, the number of gamers worldwide registered 1.8 billion in 2014 and is expected reach 2.7 billion in 2021. The Asia Pacific region is at the heart of the global video gaming industry, with over 1.2 billion video gamers in the region in 2018 according to estimates. The data implies that many students in the classroom today are experienced gamers and in fact many researchers are investigating the role of games in promoting learning. In the field of education, a study designed to examine how games engage players and how learning occurs through gameplay reported by Squire and Barab (2004) brought the game Civilization III into an urban school for underprivileged students, who gradually developed familiarity with game concepts and deeper understanding of content in their history courses. With the same game, Charsky & Ressler (2011) conducted a study to examine ninth graders’ motivation to learn history concepts. By dividing students into three groups with different treatments, they discovered that students treated with pure gameplay showed higher motivation in learning while students treated with more classroom materials were less motivated as such materials made gameplay less autonomous, less creative and less active. They further suggested that the principal potential pedagogical power of games is: fun. To determine the effect of contextualized advice and competition on transfer of mathematics skills, Van Eck and Dempsey (2002) utilized a computer-based simulation game in which participants played a peer-aged character working for their “aunt and uncle’s” home remodeling business. Results indicated an interaction between competition and contextualized advice. In follow-up research, Van Eck (2006) focused on student’s attitude towards mathematics and its instruction. He concluded that contextual pedagogical advice can result in lower anxiety toward mathematics scores, especially under competitive conditions. Tüzün, Yilmaz-Soylu, Karakuş, Inal, & Kizilkaya (2009) implemented a computer game Global Village in a geography class at a primary school. An analysis of pre- and post- achievement tests showed that students made significant learning

gains by participating in the game-based learning environment. Bachen, Hernández-Ramos and Raphael (2012) examined the role of the simulation game REAL LIVES in fostering students’ global empathy and greater interest in learning about other countries. They concluded that computerized simulations could cultivate important qualities for global learning and citizenship.

1.2 Computer games in language learning

In research related to computer games and education, an increasing number of studies are exploring how computer games can promote language learning. Current studies on computer game-based language learning (GBLL), an increasingly influential area of computer-assisted language learning (Peterson, 2016), base themselves on learning theories and draw on psycholinguistic, cognitive, and sociocultural rationales. In his book What Video Games Have to Teach Us about Learning and Literacy, Gee (2007), one of the best-known video games researchers, presented 36 principles of learning applied in video games and discussed how cognitive science-supported games enhanced learning. Researchers have also concluded that GBLL increases learner motivation (Dickey, 2007; Papastergiou, 2009), reduces learning anxiety (Kiili, 2005; Hwang, Hsu, Lai, & Hsueh, 2017), exposes learners to rich sources of target language input (Berns, Gonzalez-Pardo, & Camacho, 2013), and provides access to conditions conducive to peer learning (Sylvén & Sundqvist, 2012). There are also studies that target specific aspects of language learning with the application of computer games (see for example Alyaz, Spaniel-Weise, & Gursoy, 2017; Reinders & Wattana, 2015). In a study on the effect of digital games on Iranian children’s English vocabulary learning, Aghlara and Tamjid (2011) concluded that the mean score of children who learned vocabulary through a digital game was significantly higher than those who learnt without the game. DeHaan (2005) conducted a one-month study to investigate how one intermediate Japanese-as-a-foreign- language student improved his listening and vocabulary in terms of kanji character recognition by playing a baseball video game. Results suggested that language acquisition was facilitated by the subject's ability to control the video game's repetitive, highly contextualized, and simultaneously presented aural and textual language.

1.3 Computer games in classrooms

Although research on the use of games has produced broadly positive results, researchers caution that the pedagogical benefits are not guaranteed (Anderson, Reynolds, Yeh, & Huang, 2008). This is mainly because commercial off-the-shelf (COTS) games are not designed specifically for educational purposes and that their use is mostly limited to incidental learning. Moreover, most studies on games situate themselves in informal learning contexts (Yudintseva, 2015). Therefore, there exists one major issue in GBLL research: pedagogical relevance. In a noteworthy dissertation, Egenfeldt-Nielsen (2005) concluded that more classroom-based research is necessary in order to establish the most effective application of computer games in educational settings. However, the integration of computer games into the classroom is frequently accompanied by many difficulties. In one comprehensive study focusing on factors inhibiting teachers' use of computer games in the classroom setting Baek (2008), identified six factors: inflexibility of curriculum, negative effects of gaming, students' lack of readiness, lack of supporting materials, fixed class schedules, and limited budgets. Further, he examined the degree to which teaching experience and gender affect attitudes toward using games. Lack of supporting material, fixed class schedules, and limited budgets were factors that female teachers believed to be serious obstacles to game use in the classroom than male teachers did. Experienced teachers, more so than inexperienced teachers, believed that adopting games in teaching was hindered by inflexibility of curriculums and negative effects of gaming. Mayer (2016) echoed Baek’s finding on the inflexibility of curriculums and argued that targeted games that fit within the existing educational program should be adopted. The targeted games should not be stand-alone activities that are disconnected from the overall program of instruction, but rather should be used as part of a package of instructional activities to achieve specific instructional goals.

To effectively integrate games into formal educational contexts, teachers have a key role to play in securing beneficial outcomes (Peterson, 2016). A study by Wouters & Van Oostendorp (2013) used meta-analytical techniques to test if instructional support enhances learning in game-based learning, and the result was positive. The teacher plays an important role in the integration of games into classrooms through designing pedagogical activities and instruction, which require knowledge and skills that differ from teaching without games (McNeil, 2018). Teachers must be able to, for example, decide “what kind of games to

choose …; how to find the opportunities for language learning within gameplay; and how to integrate gameplay and its associated activities into the curriculum” (Godwin-Jones, 2014, p.9). However, the literature on preparing L2 teachers to do this is still in its infancy. To fully consider and use games in instruction, teachers may need extended periods of gaming experience, and more. In this context, Squire (2005) described several sessions using simulation games which highlighted the considerable effort required on the part of the teaching staff to use the game effectively in line with the curriculum. Molin ( 2017) contended that instead of seeing the role of game-based learning to motivate and engage students, games should be viewed as an opportunity for teacher learning and empowerment, giving teachers a sense of ownership of game-based teaching and learning. Chik (2011) reported that the teachers in her study who were inexperienced gamers perceived digital games negatively and that a 7- day period of gameplay was not enough for them to learn how to play games, understand gaming, or change their negative perspectives about them. Therefore, more research focusing on teacher’s roles, or effective classroom intervention by the teacher, is required as such an effort generates evidence that can persuade teachers to implement games in their classrooms.

1.4 Computer games in language classrooms

In the relatively limited number of studies that have integrated games into language classrooms, research focuses/results tend to concentrate on two aspects. The first and probably the most prominent is the ubiquitous positive influence of games on students’ learning motivation. In a study conducted in Taiwan, Wu, Chen, and Huang (2014) utilized a digital board game in a language classroom and discovered that the game enhanced performance and contributed to an immersive environment that fostered higher willingness to communicate (WTC). Similarly, Ebrahimzadeh and Sepideh (2017) found that video games significantly enhanced the language learning motivation of high school students in Iran. In a controlled experiment by Liu & Chu (2010) that incorporated the game Handheld English Language Learning Organization into English classrooms in a high school, the researchers discovered that the game helped students to engage in learning activities based on the ARCS motivation theory (attention, relevance, confidence and satisfaction) and produced better learning outcomes and motivation than the non-gaming method. The study also further revealed a positive relationship between learning outcomes and motivation. James & Mayer (2019) reaffirmed this value of gamification with a game-based learning study involving 64 college students learning Italian. They pointed out that 4

the motivational and affective benefits of Duolingual, a website-based vocabulary game used in the study, served as a promising alternative to more traditional methods for students who may require additional motivation not to disengage from the material. The second major focus in literature is on such receptive linguistic knowledge as vocabulary (see for example Ashraf, Motlagh, & Salami, 2014; Ghanbaran & Ketabi, 2014). Abrams & Walsh (2014) explored the ways “gamification” can play a role in adolescents’ development of vocabulary. The two authors drew upon data from in-school and after-school settings to examine how eleventh grade students’ use of adaptive online resources helped to support differentiated learning and improved understanding of vocabulary in context. The authors discovered that not only did students enjoy the adaptive and independent practices associated with a gamified approach to learning vocabulary, they also became more aware of their own word knowledge and developed a nuanced understanding of language.

An examination of literature shows that there is an increasing demand for studies that justify the rationales for incorporating games into formal instructional settings. While doing so, researchers are called upon to investigate more in-depth aspects of the language classrooms, rather than staying at the motivation level. Various language skills or competences should be targeted to better serve curriculum goals, for example, promoting students’ listening, reading, and productive language use such as writing or speaking. In addition, more quantitate data are needed to complement such forms of qualitative data as classroom observation and self- reporting. In this context, this thesis consists of a series of mixed-method studies that explore effective ways to integrate computer games into language classroom with clear curriculum goals. In the following chapters, the researcher first introduces the game used in this research and then goes on to the three studies of different focuses. Finally, the researcher presents findings from this research and propose suggestions for future research.

Game Selected for this Research

2.1 Simulation games

In an expanding body of GBLL research, computer-based simulation games (henceforth simulation games) have become a major subject of investigation (Peterson, 2010). These environments offer highly engaging virtual reality-based simulations where players are presented with language learning opportunities through their exposure to game content and, if the game is network-based, through online socialization (Peterson, 2012). Simulation games are a subcategory of games that have attracted the attention of language classroom researchers. Fletcher, Tobias and Wisher (2007) defined a computer-based simulation game as a piece of entertainment computer software that is reality-based, goal-focused and interactive. They further claim that in this type of game decisions must be made and reactions to decisions are discernable. Such games are believed to provide content for language learning that is “naturally rich in associations” through exposure to meaningful contexts (Purushotma, 2005, p.84).

Many simulation games are widely held to be intrinsically motivating (Malone, 1981). In an attempt to evaluate the educational effectiveness of simulation games, Bredemeier and Greenblat (1981) identified motivation as one of the three major variables affected by simulations games, with the other two being substantive learning, which may be either cognitive or affective learning, and learning atmosphere. Despite consensus on the positive effect of simulation games on learning motivation, there appears to be no generally accepted definition of what constitutes a simulation game. In exploring the use of simulations and games in tertiary education, Lean et al. (2006) presented a typology of simulation types derived from the findings of previous studies (see Figure 1).

Figure 1. Typology of simulation games (Lean et al., 2008)

In an article reviewing CALL software, Healey (1999) proposed the use of the simulation game SimCity to teach students reading in an English classroom. In support of Healey’s work, Schwienhorst (2002) discussed the autonomous and interactive learning opportunities offered by simulation games. Furthermore, it was suggested that vocabulary acquisition was facilitated by the game’s repetitive, highly contextualized, and simultaneously presented textual language (Calvo-Ferrer, 2017). In particular, these input-rich environments allow learners to guess the meaning of a word through contextual clues, which promotes vocabulary learning in an incidental and rather gradual manner (Coady, 1993). Miller and Hegelheimer (2006) incorporated an authentic simulation game, The Sims, into their language classroom and found that the use of supplementary materials enhanced vocabulary learning. Ranalli (2008) used the same game in his classroom and utilized Miller and Hegelheimer’s work by focusing on task appropriateness in order to enhance the pedagogical benefits of supplementary materials.

In light of the advantages simulation games offer, the researcher decided to choose an engaging and easy-to-play simulation game in a series of GBLL studies involving university students.

When selecting the simulation game for this study, the researcher first referred to the six criteria for CALL software selection proposed by Chapelle (2001), namely language learning potential, learner fit, meaning focus, authenticity, positive impact, and practicality. Naturally, simulation games provide authentic learning environments and have been proven to have a positive impact on students’ learning motivation (see for example Garris, Ahlers, & Driskell, 2002; Sitzmann, 2011).

The first study in this research was situated in a course designed to improve students’ daily communication from the aspect of vocabulary, so an ideal game would be one that has vocabulary acquisition potential, fits the needs of the learners in this study, focuses on meaning instead of forms and runs on the laptops in the research lab the study would take place. Although research on network-based simulation games is becoming increasingly popular (Peterson, 2017; Sadler, 2012), the unpredictability of second language (L2) exposure from socialization in such games also renders peer interaction uncontrollable and thus unsuitable for this vocabulary-focused research. Thus, games with more predictable and controllable contents that nevertheless do not compromise literature-supported advantages of higher motivation should be explored. A stand-alone simulation, for instance, is a good choice given its engaging experience and definitive L2 input from the game itself. In this context, the researcher searched online a list of most rated standalone simulation games (“The Best Life Simulation Games of All Time”, n.d.), and the game The Sims 4 (n.d.) by Electronic Arts emerged as the most suitable for the purposes of this research.

2.2 Introduction to The Sims 4

The Sims 4 is a life simulation game, the fourth major title in the series The Sims developed by Maxis, one of the studios under The Sims 4’s publisher Electronic Arts (Official EA site, n.d.). In the game, players create characters called “Sims” by designing their appearance and assigning them aspirations and personality traits (see Figure 2) that influence their daily “whims”, wishes or goals the characters attempt to achieve. Sims live in a simulated world with players taking care of their basic needs, such as housing, food, hygiene and sleep. The Sims in the game go to school/work, grow old, give birth to babies and pass away as in the real

world. One second in real life equals one minute in the game in normal mode, and players can switch to faster modes as needed. With various game expansion packs, features such as seasonal activities and celebrations, tourist resorts, club gatherings, dining at restaurants and opening up businesses are also available. There is no ultimate goal set by the game itself and players can try out numerous possibilities in life.

Figure 2. Creating a Simi

The Sims 4 is available in many languages including Chinese, English, Japanese and Spanish. The game provides language learners with rich exposure to language use in daily life. The game further offers visual/written linguistic input, and Sims’ utterances in the audio form are reproduced in a non-natural language created by developers. Visual input is triggered in two ways: Sims’ interaction with other Sims or the environment, and pop-up explanatory/notification texts. Input from the former is presented in single words and short phrases while the latter in complete sentences or paragraphs that often appear when the mouse hovers over certain items or when notifications pop up. Figure 3 and Figure 4 show the two modes of input. The reason why the researcher chose this game to conduct a series of GBLL studies in ESL classrooms is that simulation games give players immediate multimodal feedback on their decision-making in the game, which enables them to acquire vocabulary through trying out options marked by unknown words, phrases or short sentences. In addition, 9

the game provides interesting events that can be turned into narrative writing or storytelling, affording opportunities for productive use of linguistic input from the game. Moreover, as a stand-alone game, it offers an enclosed environment with an extractable language package, a factor that makes research design and data analysis more controllable.

Figure 3. Sims’ interaction with the environment

Figure 4. Input from explanatory/notification texts 10

There are two studies on the previous series of The Sims (see Miller & Hegelheimer, 2006; Ranalli, 2008), both of which, however, were conducted more than 10 years ago. With the development of games and language, it is worth reinvestigating the potential of this game in language classrooms.

Table 1 shows the evaluation of The Sims 4 against the six criteria proposed by Chapelle.

Table 1. Evaluation of The Sims 4 against Chapelle’s criteria

Criteria Classroom features Sims 4 features Language learning Vocabulary acquisition as the Rich daily vocabulary potential curriculum goal Suitable for both genders; Twelve female and male students Learner fit intuitive user interface; non- of varying levels of English academic; inter-mediate Meaning focus Focus on meaning Focus on meaning Authenticity Daily communication Daily life scenarios Vocabulary acquisition; positive Game-based vocabulary Positive impact student attitudes acquisition; higher motivation Laptops of medium-level Practicality Compatible with average PCs hardware

2.3 The Series of Studies in this Research and Their Relationships

With The Sims 4, the researcher conducted three studies focusing on different aspects of language skills with iterated classroom interventions involving university students in Japan and China. In the first study, the researcher started with a controlled experiment on classroom intervention (classroom intervention 1.0) that could help integrate the game into an ESL class focused on daily vocabulary for communication. The study targeted receptive knowledge of vocabulary, evaluating how certain classroom interventions could influence receptive knowledge of game vocabulary. Based on results from the first study, the researcher made changes to classroom intervention 1.0 and applied it into the second study (classroom intervention 2.0) to explore how the classroom-situated GBLL would influence students’

communicative competence, as well as reaffirming the positive result on vocabulary acquisition with a post-test which was absent in the first study. Again, based on students’ feedback on classroom intervention 2.0, the researcher made adjustments to previous classroom interventions and added out-of-class intervention to complement classroom learning in the third study. The third study was a single-participant experiment and focused on both writing and communicative competence. For all studies, the researcher has obtained either written or oral consent from participants for publishing data using pseudonyms. Figure 5 below gives an overview of the series of studies in this thesis.

Receptive Productive

Communicative Vocabulary Writing Competence

Study 3 Study 1 Study 2 Single-Participant Controlled Experiment Exploratory Study Experiment Classroom intervention Classroom intervention Classroom intervention 3.0 1.0 2.0

+ Figure 5. Overview of the three studies and their respective focuses in this research

The First Study: Effective Intervention for Vocabulary Acquisition in a Game-Based Language Learning Classroom

3.1 Introduction

This chapter reports on a study that explored three forms of classroom intervention: teacher instruction, peer interaction and in-class activities, for the purpose of integrating computer games into a vocabulary-focused English classroom in a university in Japan. The aim was to establish which intervention is most effective, as well as what improvements should be made for future application. The study took the form of a controlled experiment and evaluation of the forms of intervention was based on concurrently collected quantitative and qualitative data. Receptive vocabulary knowledge was chosen as the focus in the first study as words and phrases are the most direct linguistic input from the game and can therefore lay a further foundation for later studies focusing on more complex language skills such as speaking and writing.

3.2 Background

3.2.1 Classroom intervention

Interventions are change strategies purposively implemented at the individual, family, group, organizational, community and societal levels (Fraser & Galinsky, 2010). In educational settings, classroom intervention is comprised of interventions with class participants as intervention agents and therefore should be subject to unique adaptations by individual teachers and students according to the exigencies of their own curricula, values, and beliefs (Randi & Corno, 1997). Such interventions in language classrooms can be theory-based or practice-based, ranging from teacher’s corrective feedback to student discourse, from the arrangement of classroom activities to classroom management and disciplines. Nagahashi (2007) conducted an intervention study among freshman students in an English for academic purposes course in Japan. He found that classroom intervention by means of the teacher arranging cooperative learning activities was effective in reducing language anxiety by providing a non-threatening, 13

supportive environment that helped develop language skills. Jean and Simard (2011) conducted a descriptive observational study aimed at exploring the form-focused instruction interventions used by four French and four English high school teachers in classrooms. They discovered that grammar-oriented interventions were rather frequent in both contexts while some differences in preferences existed. Under a circumstance where pronunciation teaching in Japan depends heavily on decontextualized practice such as mechanical drills and repetition, Saito (2012) investigated the intervention of teacher instruction in students’ pronunciation improvement and found significant improvement resulting from instruction. In intervention research conducted in an EFL classroom in Turkey, Bush (2015) concluded that games were an effective intervention in learner motivation although statistical results showed no significant difference between the experiment group and the control group.

3.2.2 Research Questions

There are also several studies in GBLL that highlight measures resembling classroom intervention, such as teacher-produced scaffolds in the form of supplementary materials based on game content (Coleman, 2002; Ranalli, 2008). However, there is still ample room to systematically investigate classroom intervention that makes simulation games suitable for formal classroom instruction. Therefore, investigation of different classroom interventions in GBLL represents an area of major interest. In light of this, this study aims to explore interventions that have the potential to increase the pedagogical relevance of simulation games. This study is exploratory in nature and serves as part of a larger project to identify effective and generalizable classroom interventions which teachers can apply to their classrooms with varying local needs.

Accordingly, the researcher proposes the following two research questions:

1. Do the interventions selected for this study promote the integration of a simulation game into the language classroom? 2. What improvements can be made to these classroom interventions?

3.3 Methodology

The study took place in an English communication class at freshman level in a Japanese university and the curriculum goal was the acquisition of daily vocabulary as part of cognitive 14

competence in English communication. The researcher drew on and evaluated existing GBL studies that are related to, but not necessarily focused on, integration of games into language curriculum (see for example Miller and Hegelheimer, 2006; Ranalli, 2008), summarized the measures, and profiled these measures against the configuration of this study. The result was three forms of classroom intervention: teacher instruction, peer interaction and in-class activities. Teacher instruction was in English and consisted of two components: group-oriented instruction before gameplay where the teacher instructed the whole class in game vocabulary and gaming skills, and teacher-student interaction during gameplay where the teacher walked around and resolved specific questions from each pair. Peer interaction included communication in both first and second languages, to ensure that the students would feel comfortable in interaction, but of course, the latter was strongly encouraged. Regarding in- class activities, participants were given game quests or tasks they were required to complete in the game. They were also required to record interesting gameplay episodes for later presentations. The presentations were expected to solicit the use of game vocabulary in a meaningful context and encourage participants to interact with the audience in a Q&A session after each presentation. To evaluate these interventions both qualitatively and quantitatively, the researcher decided to conduct a controlled experiment where another group of students would play the game outside the classroom without the interventions experienced by the control group. In the experimental group which was situated in the class, the researcher would also assume the role of the teacher.

3.3.1 Participants

This study consisted of three groups of participants: a baseline group, an experimental group, and a control group. Participants in the baseline group were all native Chinese first-year students randomly selected from a tier 2 university (ranking 100-200 nationwide) in China. They regularly took compulsory intensive and extensive English reading classes twice a week. Before the controlled experiment started, the researcher asked the baseline group, consisting of 10 participants, to take vocabulary quizzes designed for this study online without playing the game. The purpose of the baseline group was to give a general idea of how random university students would perform on the quizzes. Apart from that, the baseline group was not involved in any part of the controlled experiment. The experimental group included 12 first- year education majors who were taking the English communication course once a week in a 15

Japanese university. The students in the experimental group were of different English learning backgrounds and varying levels of English proficiency: one student came from Norway speaking fluent English and three Japanese students had overseas experience in English speaking countries. The Japanese natives were in the B1 to B2 (intermediate) competence range under CEFR (Common European Framework of Reference for Languages: Learning, Teaching, Assessment, n.d.) according to English test scores they reported in the pre-questionnaire (TOEFL, IELTS, TOEIC or EIKEN, a Japanese national English test). Lastly, the control group was situated in an out-of-class environment in China and included 12 Chinese first-year students in the same university as the baseline group. They were volunteer students who reported intermediate competence range under CEFR according to scores from their national college entrance exam. None of them had overseas experience. For both the experimental and control groups, English was a compulsory course for general education and none of the participants were majoring in English for academic purposes.

In terms of gaming experience, the students in the control group were more familiar with gaming environments, with all reporting having played games on PCs and mobile phones. Five of them were frequent gamers but none of them played the game used in this study before. In the experimental group, only one student said he frequently played games in his spare time. However, two students said they had previous experience playing earlier versions of the game used in the study. The most preferred gaming environment of the experimental group was the mobile phone, while the PC the most used platform in the control group.

3.3.2 Research design

Before the experiment started, the researcher invited the 10 university students from China, the baseline group, to complete all the vocabulary quizzes. Their results would provide a reference for the consistency among all quizzes so that the researcher would know whether in-group fluctuations of scores should be attributed to varied levels of test difficulty. For evaluating the interventions as a treatment, the experimental group were divided into pairs for enhanced peer interaction, received instruction from the teacher before and during gameplay, and completed structured in-class activities; whereas the control group played the game alone outside of a classroom environment. Both the experimental and control groups were introduced to the basics of gameplay at the beginning of the experiment and took regular vocabulary tests as an assessment of their learning results. In the experimental group, there were 11 weekly sessions 16

(each session lasted approximately 1.5 hours) that entailed 4 themed units of gameplay. These sessions consisted of teacher instruction (around 100 minutes), participants’ autonomous gameplay (around 10 hours for each participant), presentations (around 120 minutes for the whole class, with individual presentations lasting 1 to 5 minutes) and vocabulary tests and surveys (around 40 minutes for each participant). Each week participants in the control group were asked to submit automatically generated “saved data” and to indicate from and to what time they were playing the game in the file name. Each of them submitted 11 files that added up to 16.5 hours, exactly the total duration of the 11 sessions in the experimental group. Refer to Appendix A for a general view of sessional themes and activities in both groups.

3.3.3 Data collection

In this research, data from the experimental group came from three sources: surveys, recorded gameplay sessions and vocabulary quizzes. Data from the control group consisted solely of vocabulary quizzes.

3.3.3.1 Vocabulary quizzes

The researcher administered 5 quizzes to evaluate students’ vocabulary acquisition: a pre-test and four unit tests, each with 50 words. The vocabulary base for the tests was extracted from the game language pack and profiled against the most frequent 1000 word families, the second 1000 and the Academic Word List (Vocabularyprofile English, n.d.). Then the researcher reviewed the words one by one to finalize a pool that contained words mostly likely to be encountered in gameplay. The pre-test was conducted in the introductory session before students began to play the game. The test consisted of randomly selected words from the pool to provide a general idea of students’ vocabulary level against the game environment. The four unit tests included theme-specific words the researcher selected from the pool. They were conducted one week later, instead of immediately after the completion of a certain theme. By doing so, delayed investigation of students’ retention of words was allowed and thus there was no post-test.

3.3.3.2 Recordings

Recordings in the experimental group were screen recordings of the game that also captured environment sounds so that interaction during gameplay and in-class activities were also included in audio format (the students were reluctant to show their faces, so no videos of 17

activities were recorded). The researcher used the screen recording software, Bandicam (n.d.), to record full-time sessions of gameplay by the students. This provided valuable qualitative data regarding how students acquired and used vocabulary through gameplay and how effective the three interventions were respectively. A total of 72 hours of recordings from 6 pairs during a span of 10 sessions were examined in this study.

3.3.3.3 Surveys

Surveys included a pre-questionnaire, a post-questionnaire, and interviews for the experimental group, and a pre-questionnaire for the control group. The pre-questionnaire focused on students’ English background and their attitudes towards GBL and was also the base for selecting similar-level students in the control group. The post-questionnaire focused on the perceived learning experience and the influence of GBL on learning outcomes of the experimental group. Follow-up interviews were administered based on the post-questionnaire to acquire more in- depth qualitative information. Refer to Appendix B and Appendix C for the two questionnaires.

3.3.4 Data Analysis

3.3.4.1 Vocabulary quizzes.

In these quizzes, participants from the three groups reported whether they knew and how much they knew a certain word. Two points were credited for clearly knowing a word, including its meaning and usage; 1 for a vague impression of a word; and 0 for not knowing a word at all. In the first two scenarios, participants were also required to provide the meaning of the word with which the researcher double-checked their final scores to ensure the accuracy of self- reporting. To ensure integrity, all participants were told that the test results would only be used for research and would have no bearings on their credit or final grading. The following are examples of the three scenarios:

Figure 6. Three scenarios of students’ responses

3.3.4.2 Recordings.

The researcher used a closed coding scheme, or what is more popularly known as concept- driven analysis, to analyze the recordings. Working in a concept-driven way means basing the categories, or codes, on previous knowledge: a theory, prior research, everyday knowledge, logic, or an interview guide (Schreier, 2012). Also, according to Crabtree and Miller (1999), there are several approaches to coding. On one hand, researchers can rely on predefined or priori codes, generally based on understandings from prior research or theoretical considerations. On the other hand, the researcher can develop codes only after some initial exploration of the data has taken place. The latter is also referred to as the open coding scheme. The reason why the researcher chose a closed coding scheme is that she is a teacher-researcher who had observed the classroom before analyzing the recordings. Therefore, rather than exploring the recordings with the open coding scheme, she predetermined a set of codes based on previous knowledge: classroom observation and the research focus (see Table 2). Transcripts typical of certain audible codes were yielded as examples.

Table 2. Coding schemes of recordings

Code Description PIJ Peer interaction in Japanese PIE Peer interaction in English

TSIE Teacher-student interaction in English GD Gameplay development ICA In-class activities

3.3.4.3 Surveys

The pre-questionnaire focused on the learning background of students and was already analyzed and presented in Participants. The post-questionnaire was dominated by quantitative data and focused on students’ perceptions of the class. From the questionnaire, the researcher identified salient patterns and peculiarities in students’ responses, both of which formed the basis for the follow-up interview. The researcher then took field notes during the interview and categorized individual data under the corresponding salient patterns and peculiarities according to students’ post-questionnaire responses.

3.4 Results

3.4.1 Quantitative results

The quantitative results of this research came from vocabulary quizzes. Figure 7 represents a box plot showing the quiz results of three groups: the baseline group, the experimental group, and the control group.

Figure 7. Quiz results of the experimental, control and baseline groups (full score: 100 points) In the box plot, the green triangular markers represent mean scores; and horizontal bars in the middle of the boxes, median scores. The boxes along with their whiskers at both ends show the distribution of data in three parts. The bottom edge of the box represents the first quartile (Q1), which is the median score of the lower half of the scores; and the top edge of the box represents the third quartile (Q3), which is the median score of the upper half of the scores. Gray diamond markers outside the boxes represent outliers, or extreme values, that lie more than one and a half times the length of a box from either end of the box.

It can be seen from the boxes of the baseline group that the quizzes were of varying difficulty levels, with the pre-test being obviously easier than other unit tests (see Appendix D for the pre-test). In addition, the dynamics of the boxes of the baseline group roughly matched those of the other two groups. Therefore, in-group fluctuation of scores should be attributed to varying levels of test difficulty, rather than to experiment treatment. In terms of outliers, the experimental group produced more outliers than the control group, especially those in the lower part of the box.

At a glance, it is apparent that the control group did not perform so well as the experimental group at the beginning, but soon caught up with and even exceeded the experimental group. This finding did not conform to the researcher’s expectation that the experimental group would perform better all the way through the treatment. To better 21

understand this phenomenon, the researcher also conducted a 2-tailed t-test to determine whether such differences were statistically significant. The sample sizes of the control group (N1) and experimental group (N2) are both 12, and the total sample size (N) is 24. As this sample size is less than 30, it means that the distribution of these samples is not normal, and therefore it is more appropriate to use the t-test instead of the z-test. In the t-test, there are two kinds of hypotheses, the null hypothesis (H0) and the alternative hypothesis (H1). The alternative hypothesis assumes that some differences exist between two groups in comparison, whereas the null hypothesis assumes that no difference exists. In this study the null hypothesis (H0) is set to be that the experimental group and the control group achieve the same scores in the vocabulary quizzes; and the alternative hypothesis (H1), that the experimental group achieves higher or lower scores than the control group in the vocabulary quizzes. Table 3 shows average scores, t-values, and p-values from the analysis. The mean of the t-values is - 0.45, while the standard deviation of the t-values is 1.22

Table 3. P-values of the control and experimental groups

Quizzes Average (Control) Average (Experimental) T-value P-value Pre-test 60.77 76.58 -2.01 0.06 Unit1 55.08 65.58 -1.38 0.18 Unit2 53.38 52.50 0.12 0.91 Unit3 66.38 59.17 0.99 0.33 Unit4 61.85 61.67 0.02 0.98

Of all the P values, none was less than 0.05, the cutoff level for statistical significance, thus the null hypothesis (H0) could not be rejected. This means that, although the experimental group and control group exhibited differences in their test scores, the differences were not statistically significant and therefore it cannot be concluded that experiment treatment, in any quantitative sense, influenced vocabulary acquisition in this study.

3.4.2 Qualitative results

Qualitative data consisted of gameplay recordings from the experimental group, a post- questionnaire, and interviews.

3.4.2.1 Recordings

An analysis of recordings showed three salient features in the experimental group. First, game development showed students’ unanimous enjoyment of the game yet with differentiated preferences for game themes; second, peer interaction in L1 and L2 was less observed than teacher-student interaction; and third, students were able to use the vocabulary they had learned in previous sessions during in-class activities.

In relation to the first finding, the recordings lend support to previous research on students’ attitudes and motivation in GBL (see for example Liu & Chu, 2010; Mahmoud & Tanni, 2014). The analysis showed that students were highly motivated in all sessions, trying to build their homes, socialize with neighbors, get promoted in their professions and start their own businesses in the simulated world. In this process, they constantly needed to deal with new words and phrases presented by the game and to make decisions. The game offered the students challenges and fulfillment that motivated them to learn more in order to succeed in the game. The coding on game development also showed that each group had its own preference for game themes, and though time devoted to each theme was evenly planned in the experiment design, students tended to spend more time on themes they liked, especially in free gameplay sessions.

In relation to the second finding, contrary to the researcher’s expectation, peer interaction was not sufficiently observed and even if students did talk to each other, such discussion was usually in Japanese. The only exception was found in the pair composed of an overseas student and a Japanese student, as they primarily had to rely on English for communication. Meanwhile, teacher-student interaction, or teacher instruction during gameplay, as was observed when the teacher constantly circulated to check on the students, was much more frequent and efficient in solving students’ problems. On average, interaction in the classroom can be summarized as peer interaction in L2 < peer interaction in L1 < teacher- student interaction in L2. The following are some transcripts that show typical examples of such interactions in the class. The letter S represents students and T, the teacher.

Transcript 1 (originally in Japanese) S1: what should we do now? S2: (pointing to a cross with the word “remove”) remove, this probably means cancel the wall. S1: (after clicking) ah~ yes. 23

Transcript one shows an example of peer interaction aimed at completing a task in the game that also provided an opportunity to learn the new word “remove”. This kind of interaction was effective in acquiring new vocabulary, but was infrequent and mostly carried out in Japanese.

Transcript 2

S1: (choosing a walking style) hahaha this looks like me, nekoze. S2: hahaha, how to say that in English, nekoze? S1: nekoze…hmmm.. I don’t really know. (turning to the teacher) Do you know nekoze in English? T: uhm? S1: Nekoze, you know like this guy, what’ the word for having a round back, having a back structure like this? T: hmmmm. hunchback? S1: Yes hunchback. Yes, I think it’s hunchback. Transcript 2 shows an example where students communicated in their L2 before seeking help from the teacher. Such peer interaction was limited to the pair with an international student.

Transcript 3

S: How can I complete this task? T: Mop up a puddle? You first need to find a puddle. S: Puddle? T: Do you know what a puddle is? S: This? (pointing to a pile of rubbish) T: No, that’s rubbish. A puddle is some water on the ground. For example, if someone just took a bath and he forgot to dry himself before he came out from the tub, there would be a puddle where he stood. S: oh okay. Transcript 4

T: … and prioritize food means to ask the chef to cook her food first. So, prioritize, do you know the word? 24

S: Prioritize, hmmm T: It means to put something on the top of your to-do list (waiting for the student to prioritize food in the game and then explaining the next choice) T: Comp food means to let them eat for free. S: Com..comp, really, free? T: Yeah S: Com.. T: Comp (correcting pronunciation). It means to give away something for free. just like sabisu in Japanese S: Oh I get it.

Transcript 3 and 4 show examples of teacher-student interaction in English, the most frequently occurring form of interaction found in the recordings. It should also be noted that it was usually the teacher who initiated the dialogue when she saw puzzled faces from the students.

In the case of the third finding, analysis of classroom activities revealed that students were able to use the words they had encountered in the game in their presentations. However, while their word choices were suitable for the context, they made grammar mistakes from time to time. The following are transcripts of students’ presentation in class and words in italics are game vocabulary.

Transcript 5

S1: Mikito Toida is the father of this household. (We combined the name of each of us.) He is surely handsome and has warm voice. He is actually aspiring to be a public enemy, related to deviance. He walks in a goofy way, which makes him more like a criminal. On everyday occasion, he wears a jacket, white jeans, and sneakers. I guess you can easily imagine that he would be arrested as soon as you see him! S2: This girl is 20 years old. And she is perky and active. She get upset if she doesn't muscle training. She has an aspiration to be wealthy and to have successful career.

Transcript 5 contains excerpts from students’ introduction to their households. It can be seen students were able to use trait-related words to describe their characters. Words such as deviance, goofy and perky were new to the students and they understood these words with the help of pictures and animations in the game.

Transcript 6

S: Haruka took a bath 8:30 p.m. She became playful thanks to their bubble bath. Then her son Taro started sleeping in Atsuya's bed because his father had already slept in his bed. After that, Haruka was watching TV 10:30 p.m. because she hoped entertainment. Although she watched comedy at first, she changed the channel because she liked romance. She delighted in it.

Transcript 6 also shows an example of how a student used game vocabulary in her presentation. The researcher discovered that as game vocabulary most often appeared as individual words (see Figure 1), students often failed to put newly learned words into grammatical sentences. For example, in the underlined text the student said that her character “hoped entertainment” because the word “entertainment” was not accompanied by any collocations or sentences when she encountered it.

3.4.2.2 Survey results

Survey data showed that motivation-wise, the experimental group was greatly satisfied with the course, rating on average the course above 4 on a spectrum of 1 to 5. Words the students used most to describe the course were “entertaining” and “interesting”. Six students were “very happy”, four students were “somewhat happy” with the low stress environment in the classroom and the role of the teacher as a facilitator, while another said he was neither happy nor unhappy in the class and one said he felt a bit unhappy (see Table 4). When the latter two students were asked why they did not feel happy, they expressed their concerns that “I may end up just enjoying the game” and “although I enjoyed the game, I don't feel so much that my English skills have improved”.

Table 4. The extent to which students were happy with the class

Neither Not happy A bit Somewhat Attitude happy nor Very happy at all unhappy happy unhappy Number of 0 1 1 4 6 students

Surveys further indicated that teacher instruction was the most effective form of classroom intervention in the experiment. The students reported greater likelihood to recall words encountered in both instruction and gameplay, compared to words they came across in gameplay alone. However, they did not think peer interaction helped in improving vocabulary acquisition and suggested that the teacher should always remind or even force them to switch to English in the classroom, though they felt awkward to talk to a Japanese peer in a language other than their mother tongue. This also supported results from the recordings where, in most sessions, some students only spoke English when they were sharing their stories in front of the whole class or when the teacher checked on them. Besides, pair discussion was often less efficient to help the students understand new words. As a student under the pseudonym Marco1 argued in his interview, it was more efficient to simply ask the teacher, because the teacher was “more fluent and uses new words”.

In terms of in-class activities, the students considered presentations interesting and conducive to vocabulary consolidation. Three students also mentioned that such activities enhanced their writing as well, as they would usually prepare a written script. When asked what improvements should be made to current activities, two opinions caught the researcher’s attention. There was one time when the teacher used screen sharing software for the students to show their stories to the whole class, and a participant named Tak in his interview said there should be more of this activity. The second opinion was proposed by Mukko, indicating that the game entailed too many life choices and that therefore “it was probably better to ask all students to follow the same life path”. Meanwhile, the experimental group unanimously agreed that they had learned a lot of words for daily communication, students were primarily concerned with two issues not efficiently addressed by the class: vocabulary retention and translation. A participant with the pseudonym Alison said in her interview that she was not

sure if she could remember the words later, and Ami described her learning experience as “vague”, saying “I don’t know [the corresponding] Japanese translation but know what it means in English”. Examples given by Ami were food names and action verbs. In addition, students had mixed opinions towards game quests and their effectiveness in facilitating vocabulary learning. Some students particularly stressed in their interviews that they needed more guidance on how to complete quests in games.

3.5 Discussion

3.5.1 Quantitative data

As the prior discussion shows, quantitative data in the experiment failed to prove the effectiveness of the experiment treatment in promoting vocabulary acquisition. This failure can be explained from three perspectives: the small sample size and number of tests, unequal gameplay time between the two groups and unclear criteria in evaluating learning outcomes.

The controlled experiment involved altogether 24 participants. The small sample size rendered the experiment data less relevant in a broader statistical context. As Peterson et.al. (2018) observes, large-scale research is very rare in game-based language learning. This was partly due to the short-term nature of most studies and the difficulty in recruiting students in such a non-traditional teaching method. To address this issue, a great deal of qualitative data must be utilized to complement quantitative data. As such, mixed-methods would appear to be the most suitable way to conduct research in this field. In addition, the number of vocabulary tests is limited. The researcher could not generate a solid trend from the tests as the ups and downs in results may be an artifact of this particular project configuration. More tests could have been added, preferably after each session. However, it was difficult to contact students after the course ended, and therefore the study did not include a post-test conducted after a prolonged period. For future studies where vocabulary retention is a major topic, the inclusion of a post-test should be a priority.

Another uncontrollable factor is the discrepancy in gameplay time between the two groups. Although the researcher asked the control group to play the exact same time as the experimental group and checked their playtime according to submitted save files, the out-of- class nature of the play made it impossible to prohibit students of the control group from

playing longer. As such, exposure to game vocabulary may be higher in the control group and thus produced higher scores in their unit vocabulary tests. Instead of limiting play hours, the researcher believes it is more appropriate to allow students in classrooms to play the game in their spare time as well. Further, ten students in the experimental group were absent for one or two sessions, and they were not able to make up for the missed sessions outside the classroom. Therefore, when such students took vocabulary tests, they performed worse than the control group who were able to play the game anytime anywhere. This may also account for the 3 outliers in the experimental group during unit 3 and 4 tests and the fewer peer interactions observed in the recordings.

Third, some students in the experimental group were not sure if they had fully acquired a word and as such their test may be under-scored. When asked about whether she has learned new words through the game, a student expressed her concern regarding a lack of corresponding Japanese translation of words. The following is an excerpt from her interview.

Transcript 7

T: You said in your questionnaire that this course helped with your vocabulary acquisition only “a little bit”. Could you talk more about this in detail? S: Err… I mean, I learnt some words from the playing the game, but I’m not sure I can remember it. T: You mean you are not sure whether you will be able to recall and use it in the future? S: Yes. And some words… I don’t know I learnt some words or not. I just know what it means, not its meaning. T: You mean its Japanese meaning? S: Yes. When I play the game, I know what it means in the game, like if I click a word, my character will do an action. I can describe the word, but I don’t know its Japanese translation, so I’m not sure.

It is clear from the transcript that the student’s perception of vocabulary acquisition complies with a grammar-translation method where she needs to know the exact corresponding Japanese translation of a certain word in order to fully acquire it, and she chose “vaguely know a word” for such words in the tests. This raises another issue as to how to determine whether a

word is “acquired”. Does such acquisition require an accurate mapping of words between L1 and L2, and does it require both receptive and productive recall? For future CALL research in vocabulary acquisition involving this type of game, the criteria for evaluating learning outcomes should be stipulated in detail.

3.5.1.1 Qualitative data

Teacher instruction Teacher instruction was shown to have a positive influence on students’ vocabulary acquisition in this study. Students mentioned in their interviews that the teacher not only provided them with useful information through instruction but also played an untraditional role of a facilitator, which gave them a feeling of closeness and involvement that they had not experienced in other traditional English classes. However, as a major part of teacher instruction in this study included teacher interacting with an individual student or a pair, this requires that the classroom size be strictly controlled if every one of the students is to get amply involved in such interaction. The attention required from the teacher makes teacher-student interaction unsuitable for large-scale classes.

While they were satisfied overall with teacher instruction, students identified two areas for future improvement. First, they suggested that as the game offered numerous life paths for players, vocabulary instruction before gameplay seemed slightly general as the teacher tried to cover all possible paths. Therefore, they believed it may have been better if the teacher had restricted gameplay to just one life path where all pairs would have the same job, open the same store, etc. This suggestion is worth considering especially in a formal instructional context where time is limited. The researcher further suggests additional out-of-class gameplay as a complement to GBL in classrooms. In such an approach, students will be able to focus on the restricted theme during class and can still explore other possibilities the game offers after class. Second, the students were concerned with their vocabulary tests and asked for feedback from the teacher. One student mentioned in her interview, “There were words that I didn’t know in the test and I want to know them after the test”. This was something the researcher did not expect because the tests were originally designed only as a tool to collect quantitative data for the research. In future experiments where vocabulary tests will be employed, feedback on the tests can be added.

Peer interaction

The data shows that peer interaction was not so successful in promoting students’ vocabulary learning in this study. A major reason was the awkwardness the students felt in talking in English, followed by discussion topics on gameplay techniques rather than vocabulary. Some students suggested “the teacher should force us into speaking English in class” while others argued “the teacher should often remind us to speak English”. However, the researcher does not believe either method would work, as they would make the students feel uncomfortable and students are highly likely to return to Japanese when the teacher is gone. Insights from the pair where there was an overseas student and a Japanese native suggests that pairing up students of different L1s may be more effective. However, for most language classrooms around the world such an ideal arrangement is extremely difficult to achieve. Therefore, the researcher does not believe teachers should count on peer interaction in L2 with students of the same L1. However, teachers can surely add more vocabulary-focused game quests, such as learning the name of an item by finding or using it in the game, or learning an action verb by completing the action, so that the students will naturally discuss vocabulary-related questions. Such discussions, even in L1, may promote vocabulary acquisition.

3.5.1.2 In-class activities

In-class activities in the research comprised of mostly presentations; they successfully solicited output from the students and promoted their productive knowledge of game vocabulary. Such activities can surely be integrated into GBL in classrooms. Furthermore, as students prepared scripts before their presentations, in future research and practice teachers can also ask students to submit their scripts and give them feedback, so that students will be able to know the correct grammar and usage of words. Similar to teacher interaction, a major issue with in-class activities is the time limit. If each pair is to present in class, the number of students must be small enough to allow time for teacher instruction and autonomous gameplay.

In addition, more activities should be explored to avoid repeated presentation activities that may lead to a loss of interest on the part of the students. For example, students can work in pairs to dub a short clip of their gameplay videos based on scripts they have written. They can also switch computers from time to time to experience the gameplay of various styles created by peers. Moreover, as one accidental screen-sharing activity attracted their attention,

students asked for more such activities in the future. “Really interesting, everybody was looking at the same thing”. Similarly, in the study by Ebrahimzadeh and Sepideh (2017) which investigated short-and long-term vocabulary retention through a video game, the group of students who watched their peers playing the game outperformed students who learned words through intensive reading. Based on this finding, the researcher believes adding live game streaming into in-class activities may also prove conducive to learning.

3.6 Conclusions and limitations

This study concludes that among the three forms of classroom intervention, teacher instruction and in-class activities were confirmed by qualitative data to be effective in promoting students’ vocabulary acquisition in a GBL classroom. In particular, teacher instruction not only helped students master gaming techniques but also pointed out key words to focus on so that students learned in a more efficiency manner. Teacher-student interaction during gameplay also offered students quick ways to solve their vocabulary issues and create opportunities to communicate in English. Peer interaction, however, failed to facilitate learning as students mostly utilized their L1, and such interaction seldom focused on vocabulary itself. In addition, quantitative data were not able to generate any statistical significance between the experimental and control groups. The findings also suggest that for teacher instruction and in-class activities to play a full part, the size of the classroom should be strictly controlled, so that there will be a balance in time allocation among instruction, activities, and gameplay. Improvements to classroom interventions include designing more vocabulary-centered game quests to promote useful peer interaction in both L1 and L2, adopting varied in-class activities and adding feedback on evaluative tests.

The limitation of the study lies mainly in the recruitment of participants and data collection where the self-report technique dominated. In the recruitment of participants, it would have been better to recruit Japanese participants for both the baseline and control group. However, as this study was not funded, the researcher could only ask an English teacher at the Chinese university to engage her students. Although the English competencies of the control and experimental groups were similar according to self-report data, the different cultural background and gaming experiences may have affected the outcome of the study. The small sample sizes are another issue in participant recruitment. The number of registered students in

the experimental group determined the small size of the study, rendering it less relevant to larger classrooms. In such a case, more qualitative data must be collected to complement quantitative data. This gives rise to another limitation in the study, the reliability of participants’ self-reporting in qualitative data collection. Maxwell and Lopus (1994) argue that students tend to overstate their academic accomplishments. Such systemic reporting biases may well have emerged in this study, particularly when the students were asked to comment on their learning experience. Similar to overstatement, understatement can undermine data reliability as well. For instance, the control and experimental groups showed signs of under-evaluation in vocabulary tests, marking words they actually knew as vague. Apart from limitations in study design, the generalizability of the study is also limited. The study was situated in an English classroom with daily vocabulary as the course goal. Therefore, its results best apply to classes that focus on English for general purposes, rather than for academic purposes. Likewise, the interventions need to be changed if the study is to be applied in courses focusing on writing, listening or other skills. The number of students is another limitation as too many students would render teacher-student interaction and individual presentations impossible.

The researcher, therefore, proposes the following directions for future research: conducting studies with larger sample sizes and diversifying quantitative data, complementing GBL in classrooms with out-of-class gameplay, and in a broader context, investigating the influence of live streams on language learning.

Note:

All participant names used are pseudonyms.

The Second Study: Communicative Competence in a GBLL Classroom

4.1 Introduction

This chapter reports on the second in a series of studies on the integration of games into language classrooms. In the previous study (Wang, 2019), the researcher explored the effects of three particular classroom interventions on students’ receptive vocabulary knowledge with a life-simulation game, The Sims 4. The first study concluded that peer interaction was not effective in facilitating learning in the GBLL classroom, and therefore in the second study, the researcher decided to change peer interaction from mandatory to voluntary, allowing students to allocate more time to other tasks and activities in the class. Apart from receptive vocabulary knowledge, this study delves deeper into the effects of the above game on communicative competence, one form of productive knowledge.

4.2 Background

The term communicative competence was coined by Dell Hymes (1966) after the coming into being of “linguistic competence” (Chomsky, 1965). By observing the communicative activities of children, Hymes (1972) concluded that successful communication requires a speaker to use the language not only correctly, but also appropriately. Based on this observation, he also proposed the four components of communicative competence: linguistic competence, sociolinguistic competence, discourse competence and strategic competence. Later, for the purpose of language testing, Bachman (1990) provided his taxonomy of language competence. According to him, language competence consisted of organizational competence and pragmatic competence. The former was further divided into grammatical competence and textual competence, and the latter into illocutionary competence and sociolinguistic competence. He further summarized communicative competence as “the capacity for implementing, or executing that competence in appropriate, contextualized communicative language use” (Bachman, 1990, p. 84).

4.2.1 Communicative Competence in Language Classrooms

The emergence of such teaching methods as communicative language teaching (CLT) and task- based language teaching (TBLT) in recent decades (see for example Evans, 2013; Farooq, 2015; Long, 2000; Nunan, 1991; Mart, 2018) has fueled the need to improve language learners’ communicative competence through classroom instruction. Many studies in this area have focused on communication strategies, or strategic competence (see Dagarin, 2004; Dörnyei & Dornyei, 2006; Nakatani, 1998); some on linguistic competence (see Newton, 2013); and others on instructional materials and design of communicative activities (see Gilmore, 2011; Nunan, 1989). In these communication-oriented language classrooms, computer-assisted language learning (CALL) technologies also play an important role. Chun (1994) investigated how class discussion through a computer network increased the interactive competence of first- year foreign language learners. Nowrozi (2011) recognized the lack of communicative use of target language both in and outside the classroom and proposed the integration of Computer- Mediated Communication (CMC) into language learning, as it can increase both input (exposure) and output (use) of the target language needed for learners to promote both their linguistic and pragmatic competence. Another study on network-based communication by Lloyd (2012) examined the role social networking sites could play in encouraging both written and oral communication between language learners and native speakers or other speakers of the target language.

4.2.2 Communicative Competence in GBLL

In another strand of CALL, where computer games are integrated into formal instructional settings, researchers are investigating how GBLL is promoting learner’s communicative competence. Peterson (2011) investigated the interaction and attitudes of Japanese English as a foreign language (EFL) learners who participated in gameplay in a massive multiplayer online role-playing game (MMORPG) named “World of Warcraft”. Attention was drawn to the challenging nature of the communication environment and the need for learner training. Reinders and Wattana (2014) studied the effects of digital games on students’ willingness to communicate (WTC) in a Thai university with an online role-playing game named “Ragnarok Online” and concluded that the game significantly improved students’ WTC as well as confidence, reducing their anxiety in communication. With the same game, the two researchers (Reinders & Wattana, 2015) conducted another study on students’ interaction in class. Results 35

showed the game provided students with more interaction opportunities where a wide range of discourse functions were found. With the emergence of Virtual Reality (VR), real-world simulations have become possible providing immersive environments for language learners. Reitz, Sohny and Lochmann (2016) incorporated a generic 3D cooperative VR game into an English as a second language classroom and found that the game promoted students’ communication and confidence in using English.

A closer look at the literature of GBLL and communicative competence reveals the dominance of such themes as learner attitude and interaction, where data are predominantly in the form of researcher observation and student self-reporting and where conclusions unanimously indicate higher motivation and more communication opportunities. However, GBLL studies that utilize well-founded frameworks to quantitatively evaluate communicative competence are rarely found. In quantitative evaluation, rubrics with clear and curriculum- based criteria need to be generated with reference to evaluation frameworks. However, there are many aspects to communicative competence (Malone & Montee, 2010), and it is impossible for a single study to cover all of these aspects. In light of this, the researcher has focused on specific aspects of communication as criteria to both quantitatively and qualitatively investigate the effect of GBLL on students’ communicative competence in language classrooms.

4.3 Methodology

4.3.1 Research Question

Before the research question could be proposed, the specific aspects of communication to be examined in this study must be determined. At the outset, the researcher referred to two notable evaluation frameworks, American Council on the Teaching of Foreign Languages Oral Proficiency Interview (ACTFL OPI) and Stanford Foreign Language Oral Skill Evaluation Matrix (FLOSEM).

ACTFL OPI is a widely applied evaluation framework in oral proficiency and consists of scale-based guidelines on determining the communicative competence of a speaker (ACTFL, n.d.). The guidelines offer holistic descriptions of communicative competence at five scales, distinguished, superior, advanced, intermediate and novice. A closer examination of the

guidelines shows that the scale definitions incorporate a wide range of aspects in communicative competence: vocabulary, syntax, pronunciation, accuracy, spontaneity, fluency, understanding, coherence, cohesion, functions, and situations (Bachman & Savignon, 2006). Meanwhile, FLOSEM (Padilla & Sung, 1999), also a description-based 6-level system, lists in detail the five aspects it focuses on: comprehension, fluency, vocabulary, pronunciation and grammar.

The researcher planned to evaluate students’ communicative competence with student presentations. The presentations would consist of presentational and interpersonal activities. The presentational activity was in the form of storytelling; what happened in the gameplay clips the students had recorded. The recordings were projected onto the classroom wall for everyone to see. The interpersonal communicative activity was in the form of student-student and teacher-student Q&A after the presentational activity. From the previous study, the researcher identified some elements that were vital in ensuring successful storytelling and interaction: quality and attractiveness of stories, presentation of stories in terms of logic, sequencing and speech fluency, intelligibility of the presenter’s pronunciation, audience awareness of the presenter and the presenter’s willingness to negotiate meaning when communication breakdown occurred. With these features profiled against the focuses in the two frameworks, the researcher finalized three main aspects in evaluating communicative competence for the purpose of this study: interaction, fluency and content. Interaction corresponded to audience-awareness and willingness for negotiation of meaning, to spontaneity and understanding in ACTFL OPI, and comprehension in FLOSEM, indicating how much and how well a student interacted with the audience during presentations. Fluency was a relatively broad term and encompassed any components that would affect participants’ unobstructed speech, such as pronunciation, logic, event sequencing and speech fluency, as well as pauses and sentence length. In ACTFL OPI, fluency largely corresponded to syntax, pronunciation, fluency, coherence and cohesion and in FLOSEM, fluency, pronunciation and grammar. Content referred to the quality of presentations, which would depend on the attractiveness of content and the appropriate vocabulary and expressions to deliver them. It corresponded to vocabulary and functions in ACTFL OPI and vocabulary in FLOSEM.

Consequently, the researcher proposes the following research question:

How does classroom-situated GBLL influence students’ communicative competence in terms of interaction, fluency and content?

4.3.2 Participants

This study took place in an English communication course at a university in Japan. The students, and thus the participants, were 11 undergraduates majoring in education, of whom 3 were female and 8 were male. Except for one Korean student, the subjects were all Japanese native speakers. During the first week of a semester, the university allowed window-shopping and students were free to try out various classes. This helped to ensure that all the participants were interested in GBLL and that they participated out of their own will.

A pre-questionnaire (Appendix B) was administered to collect information on a general understanding of participants’ English competence, their gaming experience and their beliefs towards GBLL. The three questions on English learning background revealed that the participants were of varying English levels, with three participants scoring high on internationally recognized tests (TOEIC 990, TOEIC 875 and TOEFL 117) and five reporting no reference scores from any tests. However, according to the guidelines in the Entrance Exam of the university (Exam samples and guidelines for 2019, n.d.), the students are expected to be able to “comprehend abstract readings in both natural sciences and social sciences, extrapolate on logical relationships between sentences, successfully analyze grammatic features, write a 100-word essay with consistency, coherence and clarity, and translate a short paragraph from Japanese to English” (refer to Figure 30 for an original Japanese copy in Appendix E). Although communicative skills were not targeted in the exam, the guidelines indicate that students admitted to the university generally have a B1 to B2 level of English in CEFR (Common European Framework of Reference for Languages: Learning, Teaching, Assessment, n.d.). Except for short visits, only one participant had overseas experience in an English- speaking country: 4.5 years in the United States (TOEFL 117), while the Korean student had been living in Japan for 8 years (TOEIC 990). When asked to comment on their communication skills, their answers largely corresponded to their test scores and overseas experience. See Figure 8 for participants’ self-reporting on communicative competence.

Figure 8. Participants’ self-reporting on communicative competence

In terms of gaming experience, only four participants played games regularly in their spare time and three students said they almost never played games. Moreover, although over half of the students frequently used computers, only two said they would play PC games. The most popular gaming platform among the participants was the mobile phone. It was no wonder that none of the participants reported having played The Sims 4 before. Refer to Figure 9, Figure 10 and Figure 11 for details.

Figure 9. Participants’ gaming experience

Figure 10. Participants’ computer use

Figure 11. Preference for gaming platforms (multiple choices)

When asked their attitudes towards GBLL, participants were reserved. Six students expressed doubts over the effectiveness of GBLL (see Figure 12). Two students commented “This is a totally new type of class I’ve ever had; really excited to take it.” and “I’m looking forward to this class!”

Figure 12. Perceptions towards the usefulness of GBLL

4.3.3 Study Configurations

In this study, the participants were each assigned a computer to play on their own in the classroom. They sat very close to one another, which made peer interaction convenient, but not mandatory. Two participants were put into a pair because there were not enough computers in the class.

The participants took the class once a week and there were altogether 11 weeks in one semester. Each weekly session lasted for 1.5 hours. The sessions were divided into five units according to gameplay themes. Each unit lasted for 2 sessions and the last session was spared for a post-questionnaire and one-on-one interviews. These units consisted of teacher instruction of gameplay and vocabulary (around 100 minutes for the whole class), participants’ autonomous gameplay (around 11 hours for each participant) and Q&A-included presentations (145 minutes for the whole class). Surveys took around 40 minutes for each participant. During instruction, the teacher projected her screen for everyone to see and introduced new themes, gameplay techniques, important words and game quests (tasks to complete in the game). During autonomous gameplay, the participants were required to complete game quests to ensure they were efficiently exposed to language input from the game. Meanwhile, the teacher walked around and interacted with each student, asking if they had questions or how they were doing with the quests. For the purpose of presentations, the participants were asked to record clips or take screenshots of their gameplay. In presentations, participants told stories about what happened in the recordings and screenshots and answered questions from the audience afterwards. Refer toSessional activities

Table 24 in Appendix F for detailed activities in each session.

4.4 Data Collection and Analysis

4.4.1 Data collection instruments

This study adopted mixed methods and data collection instruments included vocabulary tests, recordings and surveys.

4.4.1.1 Recordings

Participants’ presentations were recorded, and the tool used for recording was a cellphone. The recordings consisted of participants’ gameplay-based storytelling and Q&A sessions. The recordings were later used to evaluate participants’ communicative competence. As there was a pair in the class, their presentations were evaluated as a single entity. It should be noted that there were no presentations on the 5th gameplay theme due to the need to conduct one-on-one interview in the 11th week. A total of 145 minutes of presentations on four themes were recorded in this study.

4.4.1.2 Surveys

Surveys included a pre-questionnaire, a post-questionnaire and one-on-one interviews in the last week. The questionnaires were administered online through SurveyMonkey (We're more than survey software, n.d.). The pre-questionnaire focused on the participants’ English background and their attitudes towards GBLL. The post-questionnaire focused on their learning experience and opinions towards the effectiveness of the course in promoting their communicative competence. Refer to Appendix H and Appendix I for the two formats of the questionnaires. A follow-up interview was administered based on the post-questionnaire to acquire more in-depth qualitative information.

4.4.1.3 Vocabulary tests

The study included 6 vocabulary tests, one pre-test, one post-test and four unit tests, all of which consisted 50 words from the game. The pre-test and four unit tests were the same as in study one. After each vocabulary test, the teacher would give correct answers to the students, as requested by students in the first study. A post-test was added in this study that included words and phrases selected by the researcher from the four unit tests, in order to assess whether the students were able to correctly recall vocabulary they previously failed to fully understand or misunderstood. 42

4.4.2 Data analysis

4.4.2.1 Rubric for Evaluating Participants’ Communicative Competence

For each of the three aspects of interaction, fluency and content, four levels on a scale of 0 to 5 are assigned and each level consists of corresponding descriptions in reference to the two evaluation frameworks mentioned above. A rubric (see Appendix G) is thus formed. Two evaluators, the researcher of this study and an outside researcher in the field of GBLL, rated participants’ presentations according to the rubric. The mean score of both evaluators was used as the final score of a participant. The evaluators also supported their ratings qualitatively with comments.

4.4.2.2 Surveys analysis

The quantitative results in questionnaires were analyzed with the embedded tool provided by SurveyMonkey. The result of the pre-questionnaire has already been presented previously in the discussions on participants. The post-questionnaire (Appendix I) consisted of primarily Likert-scale questions on participants’ perceptions of their learning experience and outcomes, and comments were also elicited under each question. The follow-up interview was individualized, with questions based on a participant’s peculiar answers in his or her post- questionnaire. The data was analyzed qualitatively to complement the post-questionnaire and was cross-referenced with results from recording evaluation.

4.4.2.3 Vocabulary tests

The vocabulary tests were evaluated the same way as in the first study: two points for clearly knowing a word/phrase, 1 for vaguely know of a word/phrase and 0 for not knowing the word/phrase at all. The average score of the whole class on the pre- and post-tests and on each theme was calculated.

4.5 Results

4.5.1 Evaluation of Communicative Competence

The two evaluators scored the recordings independently. The results of the class as a whole in each theme-specific unit are presented in Figure 13.

4.5.1.1 Quantitative Results

Figure 13. Boxplot results on interaction, fluency and content

Table 5. Mean scores, standard deviations and median scores of class performance

Interaction Fluency Content Overall Unit Mean Median SD Mean Median SD Mean Median SD Mean Median SD 1 2.5 2 0.71 3.17 3 0.71 3.39 3.5 0.55 3.02 3 0.51 2 3.11 3 0.49 3.25 3 0.59 3.22 3 0.57 3.19 3 0.47 3 2.72 4 0.62 3.22 3 0.62 3.11 3 0.33 3.02 3 0.47 4 2.72 2.5 0.57 3.25 3 0.59 3.00 3 0.61 2.99 3 0.54

Total 2.76 3 NA 3.22 3 NA 3.18 3 NA NA

In the box plot, the circles represent mean scores; and bold horizontal bars, median scores. The boxes along with their whiskers at both ends show the distribution of data in three parts. The bottom edge of the box represents the first quartile (Q1), which is the median score of the lower half of the scores; and the top edge of the box represents the third quartile (Q3), which is the median score of the upper half of the scores. The black square dot outside the boxes in Fluency represents an outlier, or an extreme value, that lies more than one and a half times the length of a box from either end of the box.

It can be seen that apart from a small jump in interaction in unit 2, no clear improvement in any of the three aspects is observed, and performance in content was downgrading slightly. There are many boxes with no whiskers on one end and/or overlapped quartiles and median scores, which indicate high concentrations of participants at certain performance levels. For example, in interaction in unit 1, the whisker at the bottom end is missing and the median score overlaps with the first quartile. It denotes that not only the lowest ¼ but also the lower half of the participants scored 2 on interaction. A box with a missing whisker or overlapped quartile and median indicates a higher standard deviation (as shown in Table 5) and a wider gap of performance among participants. The Overall column of Table 5 shows the ratings of the three aspects combined, fluctuating around 3 from unit to unit. In the Total row, the mean for each aspect in all 4 units was 2.76 for interaction, 3.22 for fluency and 3.18 for content.

4.5.1.2 Qualitative Results

In rating participants’ interaction, both evaluators repeatedly pointed out in comments the lack of interaction during presentations. It was clear from the recordings that literally no participants asked questions during Q&A and it was only the teacher who requested more details or made comments. However, the evaluators did discover attempts from some presenters to interact with the audience by chatting casually before presentations or switching to spontaneous speech when something unexpected happened. In such cases, the audience would also respond, not with words, but with laughter. The following are two examples (words in brackets indicate background information; words in parentheses indicate the correct use of words or phrases).

Excerpt 1

Presenter: The reason why I took this picture is that (clicking open the picture) we had a fire.

Audience: (laughing)

Presenter: Yeah (responding to laughter). Two household [members] died and the reap[er] was there (surprised tone).

Audience: (laughing)

Presenter: After this reap[er] took the souls of my household [members], (switching to another picture in Figure 14) he was watching the TV.

Audience: (laughing)

Presenter: I don’t know what kind of TV show he was watching.

Figure 14. Picture in excerpt 1

In excerpt 1, the presenter shared a sad story in an amusing way. When the presenter said “yeah”, his tone indicated that he was responding to the audience’s laughter as in “yeah, you bet, literally a fire.” When he said, “the reap[er] was there,” he stressed “there” to express his surprise that the reap[er] had suddenly appeared from nowhere. Although he made some mistakes in vocabulary, for example, “household” and “reap” rather than “household members” and “reaper”, he paced his speech on audience’s responses to deliver appropriate humor.

Excerpt 2

Presenter: The bedroom for the child… emmm (searching on the picture) … It’s probably in another picture. Where is it? (mumbling to himself in a low voice) … and… (zooming in the picture and failing to show the room).

Audience: (laughing)

Presenter: I don’t know how to remove the wall so… (the wall blocked the room from a certain angle).

Teacher: There is a point of view that allows you to lower down your walls.

In excerpt 2, the participant had trouble finding a room, and he continued talking to himself while searching for it. The audience was amused by his clumsy action and words. He then explained that the view of the room was blocked by a wall, as a reaction to the laughter from the audience. In both excerpts, the presenters engaged themselves in interaction with the audience by being responsive to audience’s laughter.

Regarding fluency in presentations, the evaluators were satisfied with the clear logic flow of stories. However, unnecessary pauses and incomplete sentences were common issues. Evaluators observed that participants were inclined to use simple and short sentences, where pauses were common between words. For example, “She goes to (pause, 0.5s) inside (pause, 0.5s) the (pause, 0.5s) house (pause, 2s), and (pause, 1s), and talk to other, other person or (pause, 0.5s) witnesses. This seems (pause, 1s) fun.” While this did not affect understanding, it did sound stilted and unnatural. Clauses participants used were mostly adverbial clauses of time and objective clauses, as in “when he got home, he was very tired” and “I think he likes her” but rarely more difficult ones such as relative clauses. Therefore, some presentations sounded like streaks of short sentences amid numerous pauses, and thus speech delivery was greatly compromised. In addition, participants also tended to overuse the conjunction “and” creating never-ending sentences. Aside from the two participants who had overseas experience, the rest of the participants spoke with a Japanese accent. Despite this, most of their speeches were intelligible, except for two participants who had very strong accents.

In content, the evaluators agreed that the stories participants presented were interesting and novel. Laughter was often observed, indicating positive reaction from the audience. Noteworthy stories included abduction by aliens, getting a divorced couple back on good terms and dealing with customers who wouldn’t order in a restaurant. Participants were also able to use original words from the game in storytelling, especially action verbs and item names. For example, a participant used “cloudgaze” from the game when his Sim was lying on the ground watching clouds in the sky; another participant used “vet clinic” in his presentation while he had previously asked the teacher where he could find the “hospital for my pet.” However,

duration was an issue for some presentations. The presentations of three participants in particular were rather short, lasting for only around one minute.

4.5.2 Receptive Vocabulary Knowledge

The individual scores across the six vocabulary tests are shown in Table 6 below (ranked from lowest to highest in terms of individual averages) and the average scores of the class in Figure 15. Boxplots of the vocabulary scores below.

Table 6. Individual scores on six vocabulary tests

No. Pre-test Unit 1 Unit 2 Unit 3 Unit 4 Post-test Mean S1 64 50 35 38 37 62 47.7 S2 93 43 45 54 53 64 58.7 S3 89 70 41 44 41 68 58.8 S4 80 67 53 34 43 87 60.7 S5 84 61 57 43 50 84 63.2 S6 86 67 56 44 63 87 67.2 S7 90 76 59 65 66 88 74.0 S8 93 82 58 61 62 95 75.2 S9 98 52 75 79 72 86 77.0 S10 99 87 75 72 72 98 83.8 S11 98 89 93 81 76 89 87.7 Mean 88.5 67.6 58.8 55.9 57.7 82.5

Figure 15. Boxplots of the vocabulary scores

Both the table and the figure show that the students scored relatively high on the pre- test and post-test, with average scores in the two tests at 88.5 and 82.5. The high average score was already discussed in the first study and was attributed to the low level of difficulty of the pre-test. Then the scores began to decrease from the very first unit and showed only a minor uptick in the fourth unit. As the first five vocabulary tests were the same as the ones in the first study, the researcher compared the experimental group in the first study with the participants in this study. Figure 16 below shows the result.

Figure 16. Comparison between the experimental group in the first study and students in the second study

It can be seen that the two groups of students showed similar decreasing trend in unit tests. However, outliers in the second study were much fewer, only one compared with 6 in the first study. This indicates that the learning outcomes of students in the second study were much closer in the second study. The first study lacked a post-test, but in the post-test here, the average score jumped back to above 80.

In terms of individual data, the average score across all six vocabulary tests of each student ranged from 47.7 to 87.7, a difference of 40 points showing varying learning efficiency of students in the GBLL classroom. It should be noted that the top 2 students, S10 and S11 in Table 6 were the student who had lived in the U.S. for 4.5 years and the Korean student who had been living in Japan for 8 years.

4.5.3 Post-questionnaire

The post-questionnaire (Appendix I) primarily relied on Likert scales to investigate students’ overall ratings of the game, how their communication competence developed in terms of the three aspects, and their opinions towards and overall rating of the course. The results are as follows. 50

Table 7. Participants’ ratings of the game

Scale

Strongly disagree Disagree Neither Agree Strongly agree Average (1) (2) (3) (4) (5) rating Number of Participants It is interesting 0 0 2 6 3 4.1 It gives me higher 0 2 3 4 2 3.5 learning motivation I learnt a lot through 1 1 1 7 1 3.5 the game I learnt very fast 1 3 5 2 0 2.7 through the game It is difficult in terms of 1 5 3 2 0 2.5 gaming skills It is difficult in terms of 1 7 2 0 1 2.4 game language

Table 7 shows participants’ ratings of the game in six criteria on the scale of 1 to 5. Under the scale are rows indicating how many participants offered rating at a certain level. The higher an average rating is, the more strongly participants agreed with a statement. From the table, it can be inferred that the game was interesting and player-friendly in terms of both gaming skills and language for the participants. However, although it gave them higher motivation and learning opportunities, learning through the game was not deemed “fast” or efficient.

Table 8. Participants’ ratings of GBLL activities in improving communicative competence

Content Fluency Interaction Autonomous Gameplay 4.0 2.6 2.8 Teacher-student Interaction 3.5 3.4 3.8 Presentations 3.1 3.1 3.4 Overall 3.53 3.03 3.33

Table 8 shows how much participants rated the GBLL activities in improving their communicative competence from the three aspects. Autonomous gameplay was rated the highest in content, but lowest on fluency and interaction. Teacher-student interaction contributed the most to improvement in both fluency and interaction. Ratings on the role of presentations in the three aspects were close, with interaction being the highest at 3.4. Overall, participants ranked improvement on content, fluency and interaction 3.53, 3.03 and 3.33 respectively.

Table 9. Participants opinions towards GBLL as compared to traditional classes

Opinion Count This is not a teacher-centered class 4 Learning through gameplay is much more interesting 10 I can communicate freely my ideas in the class 0 This class is too easy, not suited to more academic goals 1 Learning through gameplay is slow, not very efficient 3 Others 3

Table 9 shows participants’ opinions towards GBLL against traditional English classes. The opinions were suggested by the researcher, and the participants could choose multiple opinions and/or write down their own. In “others”, three participants wrote their own opinions: “good for learning vocabularies used in daily life, bad for ones in academic writing”; “I felt the game took a lot of time to gain English skills”; and “It was easy and there was not much interaction, but it was very fun and new”.

It is clear their general attitude towards GBLL was positive, with 10 participants considering GBLL more interesting. Four students thought non-teacher-centeredness mattered as well. In addition, there were also some concerns regarding inefficiency, low academic level and lack of interaction.

Table 10. Overall rating of the course

Scale 1 2 3 4 5 Average Number of participants 0 1 2 6 2 3.8

Table 10 shows participants’ overall rating of the course. Eight participants rated the course 4 and above, and only one participant rated it at 2. The average rating was 3.8.

4.5.4 Interview

Based on the results of the above instruments, the researcher conducted interviews to inquire deeper into participants’ learning experience. On interaction, participants agreed that the teacher was the major agent. “I have more chances to speak with the teacher in English in this class.” “I have the opportunity to speak English and interact with the teacher.” On the contrary, peer interaction during presentations was rather limited. As one participant put it, “Oh yeah, I remember you say it a lot ‘any comments or questions?’ in the end, but nobody else is speaking up, so….” When asked whether this was because of disinterest in others’ presentations, most participants disagreed: “Classmates' presentations were interesting. I liked them.” and “The students' presentations were interesting and there were many actions different from what I do in the game.” Only one participant said, “I did not learn much from students' presentations and only some of them are interesting.” One participant further commented, “I don’t know what to ask. I just watch it and see what they are doing.”

In fluency, participants agreed that the course provided them with English speaking opportunities that were rarely found in other classes. “[I] communicate with the teacher when I was in trouble in the game. I don't have chance to speak English with the teacher in other classes.” However, they did not think such opportunities turned into fluency improvement. Reasons were insufficient peer interaction and more importantly, the lack of instruction on how

to speak more fluently. “Not so many questions from the students”. “A little bit fluency and interaction improvement; I need another lesson for fluency.”

In content, all participants believed the game had provided them with ample materials to create stories and with the accurate words to tell them. “I have strong impression of some words. I learnt them from important events,” as one participant put it, “I remember ‘sprinkler’ because my kitchen was on fire. If I bought a sprinkler, there would not be fire.” In response to short durations of presentations, one participant said, “My stories are short, so my presentation is short.” Another participant pointed out that there was not enough time to prepare for presentations in class, and thus her presentations tended to stop where she felt she was not confident or fully prepared.

4.6 Discussion

Results have shown that participants relied heavily on the teacher for interaction during both gameplay and presentations, and that peer interaction was rarely seen. This highlights the important role of the teacher in GBLL and the finding noted previously reaffirms the need to integrate games into formal instructional settings to ensure positive learning outcomes. For peer interaction, it should be noted that the study did not utilize pairs or groups, because the previous study had indicated that pair interaction was not effective and was dominated by the use of Japanese. Although a growing trend in first language (L1) use in second language classrooms is that L1 serves as a cognitive tool to facilitate target language (TL) learning (see Chavez, 2016; Dailey-O’Cain & Liebscher, 2018), researchers do caution such use should be “moderate” (Ghorbani, 2011, p.1658) , “judicious and theoretically principled” (Turnbull & Dailey-O’Cain, 2009, p.2). Therefore, it seems likely that pairing up participants in this study would not have improved their interaction in TL. One thing that can be done for future studies, however, is that during presentations, the teacher can call random students to ask questions and to make comments, changing peer interaction from voluntary to mandatory, so as to ensure ample interaction opportunities.

In terms of content, the participants were exposed to authentic everyday scenarios they were familiar with by playing the game. Authenticity and familiarity with a topic are variables believed to promote participants’ willingness to communicate (WTC) in many studies (see Ayedoun, Hayashi, & Seta, 2015; Lloyd, 2012). Therefore, it appears that the participants felt

less burdened with coming up with presentation topics as the game gave them readily available events to talk about. Participants also recalled the words they had encountered in the game and effectively utilized them into communicative activities. There have been studies that claim the advantage of GBLL is in vocabulary acquisition, but these studies focus largely on receptive vocabulary knowledge (Ashraf, Motlagh, & Salami, 2014; Ghanbaran & Ketabi, 2014). This study goes further to indicate that productive vocabulary knowledge can also be acquired through effective presentational and interactional activities in the classroom, particularly in terms of commonly used words such as household items, actions and facility names. As for the short durations of some presentations, the researcher suggests sparing time for preparation during class, which echoes the voice of one participant who expressed the lack of preparation beforehand. Preparatory activities can include exchange of stories where two participants tell their own stories to each other and present stories of their partners. The researcher believes that this can also increase the opportunities for peer interaction in English, as participants would naturally want their counterparts to tell stories in TL, so that they don’t have to translate the stories on their own. Through such exchange, negotiation of meaning is also likely to occur that would help participants to use proper words and expressions in delivering the content. Where possible, participants may also make up stories or create lines for characters, rather than plainly describing what is happening. In this sense, gameplay dubbing may be challenging and rewarding as well.

Fluency is where both quantitative data and the participants indicated least improvement. A way to interpret the quantitative data may be that only four sessions of presentations were evaluated. It could be possible that as participants engage in more of such activities, there will be more evident improvement on fluency. That apart, the teacher does need to add more instructions in fluency. Pronunciation, basic grammar, the use of filler words or even speaker anxiety may be included in fluency instruction. As far as this study is concerned, the researcher proposes two ways of improving participants’ fluency. The teacher can either send presentation recordings to participants and give feedback to each participant regarding their grammar, sentence construction, pronunciation, etc.; or the teacher can ask the participants to prepare written texts and give feedback on their writings before presentations. The second method is better suited to participants of lower competence who need scaffolding in the written form.

Apart from answering the research question, survey results from the study further support claims made in GBLL literature that computer games give students higher learning motivation. However, it also exposes one of the possible drawbacks: appropriateness for learning academic English. Mayer (2018) conducted a media comparison research to compare the learning outcomes of groups that learnt academic material in a game to the outcomes of those who learnt with conventional media. The result suggested language learning as one of the three promising areas where games may be more effective than conventional media. Despite Mayer’s study, there is little evidence in the literature to support the application of COTS games in language learning at the academic level. As such, serious games, games that have non-entertainment purposes such as education or training (W. L. Johnson, Vilhjálmsson, & Marsella, 2005), may be the next destination where game-based academic language learning is heading. But again, the entertainment nature of games and the rigor of academic learning are bound to pose numerous challenges ahead, such as potentially lower learner motivation as serious content and homework are inevitable, unsatisfactory learning efficiency compared to those of traditional intensive academic courses and difficulties in evaluating academic performance with GBLL.

In terms of acquisition of receptive vocabulary knowledge, due to the lack of a post- test in the first study, the effectiveness of the GBLL classroom in promoting vocabulary acquisition was only partially shown through students’ post-questionnaire and interviews. However, with the post-test in this study, such effectiveness is also proven through quantitative data. The post-test further proves that the relatively low scores in unit vocabulary tests can be attributed to the fact that each theme was different and that the students were simply learning new vocabulary in each theme. When tested on words they performed poorly on previously, the students got higher scores, which suggests the unknown words were actually retained after teacher feedback. In addition, the individual scores also highlight the possibility that students of higher English levels may not benefit the most from the GBLL classroom, as it was likely that they had already known those words before the class. S8 and S11 in Table 6 are cases in point. This again highlights the potential of entertainment-targeted computer games for lower- level language learners and the possible disadvantage of such games in delivering more academic content.

In terms of limitations, this study lacked pre- and post-tests on students’ communicative competence. The researcher could have sent each student a different gameplay clip in the first 56

session and had them present the clips at the beginning of the second session as a pre-test. However, due to time limits it was not possible to conduct a post-test. In addition, data in the form of students’ self-reporting are not always reliable, and that is why the researcher only used it as supplementary qualitative data to offer insights into students’ opinions and perceptions. The small sample size and the lack of a control group is another issue that limits the generalizability of the study. The size of the class where this study took place was relatively limited to a maximum of 13 students, so as to ensure communication opportunities for each student in the classroom. The researcher also found it extremely difficult to recruit students as a control group. As GBLL classroom is a newly emerging trend, it is yet to be accepted into traditional instructional settings to impact greater populations of students. Hopefully, small studies such as this will contribute to the process.

4.7 Conclusions

As the second in a series of GBLL studies, this study explores more deeply the influence of classroom-situated GBLL on students’ communication competence from three aspects: interaction, fluency and content. Both quantitative and qualitative data were collected and analyzed. Quantitative data indicated no clear improvement of participants’ performance in any of the three aspects, while qualitative data showed that the game had provided participants with authentic content and accurate vocabulary for communication and that the teacher played an important role in providing students with interaction opportunities. The study also suggests that in GBLL classrooms, time allocation should be more balanced to allow better student preparation for implementing class activities. Additionally, the teacher has a bigger role to play in improving students’ fluency through instruction. That said, the study further lends support to the integration of games into formal instructional settings to ensure the effective and efficient use of games as language learning opportunities.

For future studies, researchers may approach classroom-integrated GBLL in accordance with curriculum goals, such as listening, writing or even content-language integrated learning. Game selection and classroom activities can also be diversified. Drama games on PC or PS4, for example, are conducive to listening and vocabulary acquisition due to audio and visual input of TL in these games. Combined with activities such as writing, storytelling and pair discussions on decision-making in gameplay, drama games can also

facilitate narrative writing and communication in second language. Out-of-class activities are a relatively unexplored area as well. Online gaming communities such as live stream platforms and fora can also be motivating. Studies that combine both in-class and out-of-class activities represent a further promising area for future research.

The Third Study Part I: Narrative Writing in a GBLL Classroom

5.1 Introduction

This chapter is the first part of the third study and focuses mainly on the influence of classroom-situated GBLL on writing. The third study was designed as such that the written scripts could serve as preparation for speaking activities which the students in the second study found inadequate. To comprehensively evaluate a number of writing samples and also create modified versions of student writing, the researcher decided to conduct a single-participant experiment with one university student. This time, the researcher shifted the classroom background from Japan to China and conducted a pre- questionnaire to 50 Chinese students asking about their writing learning experience at their university. Then the researcher identified key genres they were concerned with and drawbacks in their current curriculums, based on which the research design in this study was formed. A volunteer was chosen from the respondents and participated in a 13 session GBLL class that lasted 2 months in the summer holiday. The students’ writing samples were analyzed both independently and against the revised versions in terms of use of game vocabulary, syntactic complexity and semantic similarity. Classroom intervention in the third study consisted of teacher instruction, teacher-student interaction and writing- scaffolded speaking activities.

5.2 Background

In recent years, studies in game-based language learning (GBLL) has repeatedly suggested the positive role of computer games in promoting second language learners’ motivation (Papastergiou, 2009), vocabulary acquisition (Wang, 2017; Ashraf, Motlagh, & Salami, 2014), interaction (Peterson, 2011), reading (Schmitt, Hurwitz, Sheridan Duel,

& Nichols Linebarger, 2018) and communicative competence (Reitz et al., 2016). An increasing number of GBLL studies are also targeting writing as one potential area where computer games may contribute to learners’ linguistic development. Research has indicated some fundamental connections between games and writing. Moberly (2008), by investigating the workflow and features in game design, argued that new gaming technologies actually require sophisticated rhetorical awareness, promoting reading and writing within the discourses of consumer culture. Colby (2017) specifically pointed out that games provide richly multimodal spaces that incorporate visual, aural, written, spatial, and kinesthetic modes that students can then analyze and explore. Despite efforts to combine computer games with writing, very few studies have investigated the integration of game-based writing (GBW) into formal instructional settings and even fewer have evaluated student writing quality with well-grounded methods and theories. A review of 82 GBLL studies by Cornillie, Thorne and Desmet (2012) showed that theory-based GBLL studies ranked the lowest throughout the period from 1984 to 2010. In fact, if GBW is placed in language classrooms, there is a need to evaluate the effectiveness of the combination of games and the language teacher. In this respect, GBW studies can learn a lot from traditional writing research, which provides valuable insights into a myriad of issues in writing, including genre-based instruction, the role of teacher feedback and evaluation metrics.

Studies on second language writing generally investigate how certain variables can influence the writing quality of students. Under such a research framework, these studies usually contain three elements, variables, indicators of writing quality and evaluation tools. Some of the most popular variables include teacher feedback (Bitchener, Young, & Cameron, 2005; Ferris, Liu, Sinha, & Senna, 2013), genres/text types (Paltridge, 2014a; Qin & Uccelli, 2016), task complexity, planning (Ellis & Yuan, 2004) and multimedia such as computer games (Sheridan & Hart-Davidson, 2008). For indicators of writing quality, researchers have investigated both micro and macro aspects of writing. Micro aspects included lexical sophistication (Kim & Crossley, 2018), grammar (Truscott, 2007) and even one single feature such as the definite article (Bitchener, J., Young, S., & Cameron, D., 2005), while macro aspects consisted of 60

syntactic complexity and coherence/cohesion (Crossley, Kyle, & McNamara, 2016). In terms of evaluation tools, human rating and rating by computer programs/natural language processing (NLP) tools are two predominant means of assessment (see for example, Kim & Crossley, 2018; Yang, Lu, & Weigle, 2015). Human raters can be very flexible, generating results through subjective scoring or through manual coding, while the determination of specific NLP tools will have to consider the variable(s) and indicators of writing quality a study takes up. For example, Beers and Nagy ( 2009) adopted both subjective rating and manual coding of syntactic features including words per clause and clauses per T-unit to examine the relationship of different measures of syntactic complexity with rated writing quality. They concluded that the relationship between syntactic complexity and writing quality was dependent on genres and syntactic complexity measures. In terms of computer raters, Kormos (2011) utilized Coh-Metrix, a text analysis tool that analyzes texts on over 200 measures of cohesion, language, and readability (Graesser, McNamara, Louwerse, & Cai, 2004), to investigate the lexical sophistication and cohesion in narratives produced by upper-intermediate foreign language learners in a bilingual secondary school. Results showed that the major difference between L1 and foreign language writers could be found with relation to lexical variety, sophistication and range.

5.2.1 Computer games and writing in general

Research on the role of computer games in writing largely falls into two strata depending on how “writing” is defined. For some researchers, writing in computer games can refer to any player input into or influence on gameplay, not necessarily in linguistic forms. In this vein, Moberly (2008) argued games consist of highly symbolic constructs that players are required to read and to make meaning of. In addition, players also need to “write” with actions and ultimately to revise their actions in relation to the game symbols. In this sense, “play” can be seen as a form of writing. Connecting this with actual writing, Richard Colby and Rebekah Colby (2008) proposed that if we consider narrative in games, or the “fixed sequence of events” (Juul, 2005, p. 157), as equivalent to the classical rhetorical canons, namely, invention, arrangement, elocution, memory and delivery, we 61

might further see narrative’s counter[part] in computer game theory—play—as the more organic and recursive process of writing later emphasized in composition classrooms. Matthew and Johnson (2008) argue that “gamer-authors” compose texts that serve as examples of public discourse and that gamers can realize their authorial agency by engaging in civic participation within games and online gaming communities.

In the other stratum where writing takes on its traditional connotation as in “writing essays” in language classrooms, researchers are investigating in a more academic manner how computer games can be incorporated into writing instruction. The important elements for writing in the games mentioned above hold enormous educational potential and thus can prompt teachers to recognize new literacies and new pedagogies (Ulrich, Brockbank, Ulrich, & Kryscynski, 2007). Robison (2008) took a close look at the game industry, where game designers explicitly and already express themselves as authors; she thereby illustrated the parallels between writing instruction and game design. Sheri-dan and Hart-Davidson (2008) reported on their experiences with a game specifically designed for a formal educational context—Ink—that addresses exigency and engages students in complex rhetorical situations. Lacasa, Méndez, and Martínez (2008) investigated the differences between formal and informal educational contexts, revealing how the creative power of games helps to forge a relationship between that seeming binary. Proske, Roscoe, & McNamara (2014) conducted a large-scale study with 175 students who were randomly assigned to one of four experimental conditions: game- based, question-based, model-based, and writing-based practice. Results showed that students perceived game-based practice as significantly more interesting and engaging than question-based practice but also underlined the necessity of interconnecting motivational and instructional design when developing practice methods for game-based writing classes.

5.2.2 Computer games, topic familiarity and genre-based writing

In many writing classes, students have little access to the discourse communities that they are writing about or attempting to write within (Colby, 2017), so as David Bartholomae (2005) argued in “Inventing the University”, students’ writing still often takes on 62

decontextualized meaning. In addition, even if students’ writing could be connected with the material and conditions of a community they are writing about, students often have little influence on those conditions (Colby & Colby, 2008). With computer games, students directly influence gameplay through decision-making at various points, and writing in this sense derives from their active involvement. Decision-making and involvement will surely familiarize students with the topics they are writing about. In fact, topic familiarity has been proven to promote writing performance, critical thinking and author expertise (see for example Indah, 2017; Mccrudden & Stenseth, 2016; Salimi & Fatollahnejad, 2012). For example, in shooting games such Fortnite and Player Unknown Battleground (PUGB), players are primarily exposed to intensive input of words related to weaponry, ammunition and military operations. Students playing these games will naturally be expected to have a high level of accuracy in terminology when writing about their gaming experience.

However, a more important and often unnoticed feature of game-based writing is that it facilitates genre-based writing instruction. In systemic functional linguistics, researchers tend to describe genres in terms of broad rhetorical patterns, including descriptions, narratives, arguments and expositions (Hyland, 2018; Paltridge, 2014b). Miller (1984:165) argued that genres ‘serve as keys to understanding how to participate in the actions of a community’ and that the failure to understand genre as social action turns activities such as writing instruction from ‘what should be a practical art of achieving social ends into an act of making texts that fit formal requirements’. The advantages of genre-based writing instruction are significant. Among them, explicit writing objectives based on students’ needs, systematic frameworks for focusing on both language and contents and access to patterns and variations in valued texts are some of the most important (Hyland, 2004). Yasuda (2011) examined how novice foreign language writers develop their genre awareness, linguistic knowledge, and writing competence in a genre-based writing course that incorporates email-writing tasks. In the fifteen-week writing course, 70 Japanese undergraduate students participated in carefully designed genre-based tasks. The results showed that the students made progress in their genre awareness, and that changes in their awareness were apparent in their writing 63

quality. In general, game-based writing in the broader sense can take on two major genres: descriptions and narratives. Descriptions are commonly found in MMORPG games where writing takes place when players type messages to one another, seeking ways to complete game quests. Narratives are almost universal to all games, as students can always write about their gaming experience after gameplay. Nevertheless, no GBLL studies have investigated particularly how computer games would influence genre-based writing, let alone genre-based writing instruction in classroom settings.

5.2.3 Current issues with evaluation of second language writing

Literature in writing evaluation shows a preference towards indicators such as lexical sophistication, syntactic complexity and cohesion, but few studies have approached writing quality from the perspective of semantics. Ellis and Yuan (2004) touched upon semantics when they investigated the effects of planning on fluency, complexity and “accuracy” in second language writing. The lack of attention to semantics may be partly due to the lack of a benchmark against which writing samples can be compared. Guo, Crossley and McNamara (2013), in exploring whether linguistic features can predict TOEFL iBT writing proficiency, found that semantic similarity, or basic text information, was a significant predicator for the integrated writing task, which is basically a recount of a reading passage and a short speech on the same topic. A study by Kormos (2011) in the paragraph noted previously collected data from a small group of L1 writers for comparison at the lexical level, but such data has some drawbacks in evaluating sematic similarity, as L1 writers wrote separately from the students and therefore their writing products could have been vastly different in terms of focus and perspectives. In light of this, the researcher proposes to use students’ original samples and teachers’ revised samples as pairs for comparison. The reason for adopting this approach is that if the revised samples are seen as the optimized versions of the original ones, it is possible to evaluate students’ writing quality through semantic similarity, namely by observing how close the original samples are to the optimized benchmark samples.

In fact, even with syntactic complexity, revised samples may also prove more effective. At this juncture, another issue arises regarding syntactic complexity: does 64

higher syntactic complexity mean higher writing quality? Previous studies have yielded conflicting results (see Beers & Nagy, 2009; Crowhurst 1980a; Jagaiah, 2016; Stewart & Grobe, 1979) and the researcher believes that there is a probability that higher syntactic complexity is not necessarily better, especially when there are more concise alternatives that denote the same meaning. However, with revised samples, it is possible to calculate the distance in syntactic complexity between the samples under investigation and the optimized benchmarks, and the distance, in other words syntactic similarity, can be used as an indicator of writing quality.

5.2.4 The present study

As noted previously, this study is the third in a series of GBLL studies conducted by the same researcher with the PC game The Sims. The first two studies focused on vocabulary acquisition with classroom intervention and communication skills respectively. In this study, the researcher aims to explore the role of the game in student writing development. In summarizing the background above, the researcher identifies two gaps in current GBW literature, the lack of focus on genre in GBW studies and the lack of a benchmark in evaluation that can add semantics into the picture and that can remedy the inconsistent relationship between syntactic complexity and writing quality. As such, the researcher decided to conduct mixed methods research exploring the influence of classroom- incorporated GBW on the writing quality of students in a particular genre through observing how far the students’ original writing samples differ from the revised ones.

5.3 Methodology

5.3.1 Pre-study survey

Before deciding the final research question, the researcher conducted a questionnaire (Appendix H) with 50 students using Surveymonkey (We're more than survey software, n.d.) in a second-tier university in China (there are top-tier, first-tier, second-tier, third- tier universities in China and tertiary educational institutions under the third-tier are called colleges), in order to investigate whether students would be interested in a game-based writing class and to decide the targeted genre of writing and modes of teacher instruction and feedback.

5.3.1.1 Students interests

In the questionnaire, the researcher asked whether students would be interested if their teachers integrated a computer game into the English writing classroom, offer instructions on game language, allow them to play the game in the class and assign them relevant writing tasks. A comment box was added after the questions for students to express the reasons why they thought so. The students’ responses are shown in Table 11 below:

Table 11. Students’ interest towards GBW

Answer choices No. of respondents percentage Very interested 13 26.00% Interested 30 60.00% I don't know 7 14.00% Not interested 0 0.00% Not at all interested 0 0.00% Total 50 100.00%

It can be seen that the majority of students (86%) are interested in the game-based writing class, while 7 students said they did not know if they would be interested. No

students reported disinterest towards a GBLL class. Therefore, on the basis of this feedback data, it may be concluded that a game-based writing class at university level is worth investigation in the Chinese context.

5.3.1.2 Genre of writing

Question 2 in the questionnaire elicited feedback regarding the most common genres the students had been practicing up to the survey. The choices were offered by the researcher according to genre categories mentioned previously and knowledge of English exams in China. An ‘others’ option was available for students to write answers not included in predetermined choices. Table 12 in the following shows the result.

Table 12. Most common writing genres for the students

Answer choices No. of respondents percentage Narrative, recounting past events 11 22.00% Argumentative, expressing your own 16 32.00% opinions and arguments Expositions, explaining how something 6 12.00% works or procedures Correspondence, writing letters 13 26.00% Description, describing current events or 4 8.00% pictures Others 0 0.00% Total 50 100.00%

The result shows argumentative essays and letters were the primary genres the students wrote in, which are actually the two most dominant genres tested in China’s University Entrance Exam and College English Tests (CET). Narrative ranked third at 22%, while description, which bears some similarity to narrative, ranked last at 8%. In a further question regarding whether students searched for background information for writing tasks, their answers were as follows (Table 13): 67

Table 13. Use of background information in writing

Answer choices No. of respondents percentage No, I write with only prompts that come 24 48.00% with writing tasks Yes, I search for relevant information in 20 40.00% Chinese as supplement Yes, I search for relevant information in 5 10.00% English as supplement Others 1 2.00% Total 50 100.00%

Table 13 shows that almost half (48%) of the students said they did not use any background information at all and that they completed writing assignments based solely on given writing prompts. Another 40% of the students said they searched for relevant information in Chinese while only 10% utilized English sources. One student commented in “others” that he/she only cross-checked information online after completing the writing.

Question 6 inquired into students’ preferred areas in enhancing their writing skills, in an effort to understand what is considered most practical and urgent by them. Table 14 shows the result.

Table 14. Most practical areas of writing where students wanted to improve their skills

Answer choices No. of respondents percentage Daily topics/communication 28 56.00% Business writing 5 10.00% Academic writing 8 16.00% Exam-oriented writing 9 18.00% Others 0 0.00%

Total 50 100.00%

The majority of students (28) said they wanted to improve their writing in daily topics or for daily communication purposes. Nine students preferred more exam-oriented practices. Only 8 students expressed preference for academic writing, and even fewer students thought business writing was practical for them.

The researcher decided to focus on a writing genre on which the students were less trained but where the students’ practical concerns in writing can be effectively addressed. Based on the information above, the result was narrative of daily life.

5.3.1.3 Teacher feedback

In determining modes of teacher feedback, the researcher first asked what kinds of feedback the students were receiving from English teachers at their university. Table 15 in the following shows the result.

Table 15. Types of feedback the students received

Answer choices No. of respondents percentage No, I receive no feedback from the 7 14.00% teacher No, but he/she hands out exemplary 11 22.00% samples to us for reference Yes, he/she chooses writing assignments from several students for 28 56.00% review in class Yes, he/she gives individual feedback 4 8.00% on each student's writing assignment Others 0 0.00% Total 50 100.00%

It is clear from the table that the majority of students had teachers who would review some writing assignments in class as group feedback to students. Another 22% received reference samples for them to learn from, and 14% said they received no feedback from the teacher after handing in assignments. Only 4 students received individual feedback from their teachers. Comments given under lack of feedback indicated inadequate class time as the primary reason. Given this result, the researcher decides to explore the influence of individual feedback on students’ writing in a GBW classroom.

5.3.2 Indicators of writing quality

As mentioned previously, studies in second language writing tend to focus on lexical sophistication, syntactic complexity and cohesion, but very few on semantics. However, most studies with cohesion as one of the focuses target argumentative essays (Guo, Crossley & McNamara, 2013; Kim & Crossley, 2018), and the researcher believes semantics is more relevant in examining the clarity and accuracy in depicting stories in narratives. Specifically, the researcher intends to evaluate semantic similarity, with student writing compared with teacher revised writing, as a revised sample represents what is the best-scenario product built upon student work. In terms of syntax, the researcher intends to investigate the issue of syntactic complexity further to focus on syntactic similarity, which is also produced by comparing the syntactic complexity of student writing and that of revised writing. By comparing student writing to revised writing, it’s possible to see the distances between the two in terms of syntax and semantics and therefore writing improvement can be examined by looking at the trend of such distances. Lastly, Given the generally positive feedback on games’ influence on vocabulary acquisition, the researcher wonders to what extent can such receptive lexical knowledge be turned into productive knowledge in narrative writing. Productive use of game vocabulary will also be a focus of this study.

5.3.3 Research questions

In the above context, the researcher proposes the following three research questions: 70

Will classroom-incorporated game-based writing with teacher’s individual feedback:

1. enable the students to productively use game vocabulary in narrative writing? 2. improve the students’ syntax in narrative writing? 3. improve the students’ semantics in narrative writing?

5.3.4 Study design

5.3.4.1 Participant

Considering the time required for face-to-face individual feedback in class, the researcher decided to conduct an exploratory study with one student from the pre-study questionnaire respondents and track the student’s progress over time in the GBW class. The student was a sophomore-to-be majoring in environmental design. In her feedback, she reported having just passed the CET level 4 with a score of 435, which approximately equaled B1 on the Common European Framework of Reference for Languages (Common European Framework of Reference for Languages: Learning, Teaching, Assessment, n.d.). The student wrote mostly argumentative essays before because it was the major genre she had been trained on for exams in China. Her English teacher did not lecture on writing and would only focus on reading materials in her textbooks. However, the teacher did provide feedback to student writing by choosing several student samples for review in class. Before writing, she would search for background information in Chinese and try to translate the ideas into English where necessary. She wanted to improve her academic writing the most and also her daily communication skills. The researcher asked if she was interested in a GBW class that would target narrative writing, she said she was very interested.

Before the experiment started, the researcher administered a pre-test including 50 randomly chosen words in the game vocabulary pool (Wang, 2019). Although this study does not adopt vocabulary tests to examine receptive vocabulary knowledge, the pre-test would still avail the researcher of the student’s familiarity with game vocabulary at the initial stage. The student was required to write whether she knew a word clearly, knew of 71

a word vaguely or did not know a word at all. Two points, one point and zero point were awarded in each case respectively. To ensuring accuracy in self-reporting, the student was also asked to write down the meaning of a word in the first two cases so that the researcher could double-check the answers. The student scored 33 points, with 8 words she clearly knew and 17 words she knew of, while the remaining 25 words were totally unknown to her. This finding indicates that the student was not familiar with the game vocabulary before the GBW class started.

5.3.4.2 Study design

There were 15 sessions in the GBW class and the student used her spare time to complete a writing assignment after each session and handed them in as MS office word files. Each session lasted for 1 hour and 45 minutes, excluding with a 15-minute break after the first hour. Each session included teacher instruction, student gameplay and face-to-face feedback on a writing task from the previous session. Writing feedback and teacher instruction occupied the first hour and the remaining 45 minutes after the break was used for student’s gameplay. The equipment used in the classroom included a desktop computer and an extended monitor. During teacher instruction and writing feedback, the teacher would use the computer and the student could see the teacher’s screen on the extended monitor.

5.3.4.3 Teacher instruction

In each session, the teacher would introduce new gameplay themes to the student, including background information, gameplay techniques and game quests, and have the student watch her play the game on an extended monitor while explaining gameplay techniques and key vocabulary and grammar. For example, in the theme “building a house”, the teacher went through the game guidance on construction with the student, taught the student how to use the building tools and tried to build a basic house. In the meantime, the teacher highlighted important words in the guidance and words related to housing structure and furniture. The game quest was for the student to draw a floor plan of the house she intended to build and to complete the construction in the game accordingly. 72

5.3.4.4 Feedback on writing

Altogether the student wrote 14 samples. Note that in the first session, there was no writing assignment for the teacher to give feedback on. From the second session, the teacher would spend an average of 20 minutes in the first hour on reviewing the student’s writing assignment from the previous session. The feedback was in the form of direct revision based on teacher-student verbal negotiations. Revisions focused on micro aspects such as word choice, collocation and grammar and macro aspects such as writing logic, syntax and semantics. The teacher made modifications in the MS Office word document with the “track changes” feature while the student watched through the extended monitor. Clean versions of revised samples were separately saved.

5.3.4.5 Student gameplay and writing

After the teacher finished instruction on game vocabulary and gameplay techniques, the student would start playing the game on her own with game quests in mind. The teacher would sit in front of the extended monitor and help the student when she encountered problems. During gameplay, the student would use the camera function in the game to record interesting game clips, based on which she wrote stories depicting what had happened in the game. When a session ended, she would send the video file to her cellphone and complete the writing task after class. A list of the samples can be found in Appendix K.

5.3.5 Data collection and analysis tools

5.3.5.1 Data sources

The data sources of this study are as follows: • Original writing samples: writing assignments submitted by the student; • Revised writing samples: writing samples after teacher revision; • Pre-questionnaire: the student’s writing experience, writing class at university and her personal beliefs towards GBW class;

• Pre-test: a test of 50 words in the game to see how familiar the student is with game vocabulary; • Delayed post-test: a test of game vocabulary from teacher feedback, or exclusive to revised writing samples • Post-interview: interview on student’s experience in the GBW class and more qualitative inquiry into results from quantitative data

5.3.5.2 Syntactic Complexity Measures (SCMs) for generating syntactic similarity

Some of the most popular SCMs include mean length of sentence (MLS), mean length of T-unit (MLT), mean length of clause (MLC), clauses per sentence (C/S), clauses per T- unit (C/T), T-unit per sentence (T/S). In choosing the appropriate measures, writing genres, too, play an important role (Beers & Nagy, 2009). The student’s writing in this study were mostly narratives. According to Beers & Nagy (2009), C/S was effective in evaluating the syntactic complexity of narratives. Therefore, the researcher decided the first measure to be C/S. Among the above measures, many are linearly related. For example, MLS / MLT= T/S and C/S / T/S = C/T. With C/S, MLS and T/S, all other measures can be calculated and need not be included. Thus, the researcher finalized the SCMs to be C/S, MLS and T/S in calculating the syntactic complexity of the original and revised samples.

5.3.5.3 Jaccard similarity coefficient

For cross-reference with semantic similarity, the researcher added the calculation of lexical/word-for-word similarity with the Jaccard similarity coefficient. Jaccard similarity coefficient measures similarity between finite sample sets by dividing the size of the intersection by the size of the union of the sample sets (Jaccard, 1901). The greater the coefficient is, the greater the lexical similarity between two sample sets, as is shown in formula (1) and Figure 17 below:

|� ∩ �| |� ∩ �| �(�, �) = = (1) |� ∪ �| |�| + |�| − |� ∩ �|

Figure 17. Principle of Jaccard similarity

5.3.5.4 Analysis tools

Syntactic complexity: L2 Syntactic Complexity Analyzer (L2SCA). L2SCA (Web-based L2 Syntactic Complexity Analyzer, n.d.) is an automated syntactic complexity analysis tool of written English language samples with fourteen different measures proposed in the second language development literature. The analyzer takes a written English language sample in plain text format as input and generates 14 indices of syntactic complexity of the sample (Liu, 2010), including the 3 measures chosen by the researcher.

Semantic similarity: InferSent. InferSent (n.d.) is a sentence embeddings method that provides semantic representations for English sentences. It is trained on natural language inference data and generalizes well to many different tasks (Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A., 2017).

Jaccard similarity coefficient: built-in Python Set Types (n.d.). The Set types calculate the intersection and union of the sets of words, respectively. Then the Jaccard similarity is calculated based on the intersection and union of the sets.

Coding: opening coding scheme on words in delayed post-test

5.4 Data analysis and results

5.4.1 Vocabulary

The researcher used the built-in Python Set Types (n.d.) to create two sets of distinct words (words exclusive of repetition, which means a word that repeatedly appears is counted only once), one for the original samples (List A) and one for the revised samples (List B), and calculated the total number of distinct words in the two sets. Then, the researcher compared the distinct words and in Set A and Set B to see how many distinct words were shared by both sets and what were particular to each set. The results are shown in Figure 18 below.

114 789 247

A: 902 B: 1035

Figure 18. Distinct words in two sets

There were altogether 902 distinct words in Set A and 1035 distinct words in Set B. The intersection between the sets indicates 789 distinct words were shared. With these numbers, it can be deduced that teacher feedback on writing led to 114 distinct words in Set A being discarded and 247 distinct words being added.

5.4.1.1 Productive use of game vocabulary

In terms of the use of game vocabulary, the student was asked to mark words she learnt from gameplay in Set A; then in Set B, the researcher was responsible for identifying words from the game. In this way, it was possible to see how many distinct words from the game the student was able to use and how many were used after teacher feedback. The results are shown in Table 16.

Table 16. Number of game words in two sets

Set No. of distinct words No. of distinct game vocabulary Percentage A 902 78 8.65% B 1035 131 12.7%

From Table 16, it can be seen that teacher feedback led to an increased percentage of game vocabulary in writing samples. With regard to the 53 (131-78) game words missing from the original samples, the researcher observed their features and coded them with three labels according to their salient features: grammar mistake, alternative and correction. Words labelled grammar mistake are the ones that the student actually knew but used with wrong grammatic features. For example, when describing a campfire story, the student wrote “Kristi added log to the campfire”, while it should have been “Kristi added logs to the campfire”. Words labelled alternative are the ones to which there are better alternatives from the game. For example, when describing a gym experience, the student wrote “she walked to him and ran on the running machine” when the word used in the game to refer to “running machine” was “treadmill”. Words labelled correction are the ones where the student made non-typo mistakes in spelling or misunderstood the meaning and use. For example, “he leaves them in this toilet” while the correct one should be “he leaves them in the sink”. When coding was completed, the researcher asked the student to take a delayed post-test with the 53 words 45 days after the end of the class (refer to Appendix L for the post-test).

Verbs in -ing or single third-person present forms and plural nouns were restored to their stems, unless the -ing inflection changed the meaning of the stem (light and lighting) or unless the plural inflection “s” was added to a noun that could otherwise have been a verb or adjective (need and needs, patient and patients). The test required the student to write both the meaning of a word and a sentence with it. The purpose of the test was to see whether the student could accurately use these words after teacher feedback. Results showed that the student was able to write down the meanings of and create sentences with 37 of these words. In the remaining 16 words the student forgot (underlined words in Appendix L), 2 of them are labelled grammar mistake; 11, alternative; and 3, correction. Table 17 below shows the results.

Table 17. Forgotten words in the delayed post-test

Grammar Label Alternative Correction Total Mistake Before post- 15 28 10 53 test After Post- 2 11 3 16 test Forgetting 13.33% 39.29% 30.00% 30.19% rate

From Table 17, it can be observed that the forgetting rates on alternatives and corrections are much higher than that of grammar mistakes. As mentioned above, words labelled grammar mistakes were the ones the student was actually capable of productive recall while words labelled alternative and correction were not within the student’s productive knowledge. This indicates that words the students newly acquired through teacher feedback were less likely to be retained than words the student already knew and used in writing.

5.4.1.2 Syntax

L2SCA yielded data on the three SCMs of both the original and revised samples as follows in Table 18.

Table 18. MLS, C/T and T/S in the original and revised writing samples

Revised Samples Original Samples Sample No. (Benchmarks) MLS C/T T/S MLS C/T T/S 1 9.45 1.25 1.09 11.81 1.29 1.19 2 10.15 1.18 1.41 11.92 1.30 1.42 3 10.25 1.15 1.18 10.83 1.14 1.23 4 12.37 1.60 1.24 12.69 1.48 1.28 5 9.76 1.29 1.08 10.84 1.23 1.13 6 11.25 1.26 1.19 12.42 1.42 1.16 7 13.55 1.47 1.17 15.80 1.51 1.23 8 10.97 1.24 1.23 10.63 1.24 1.28 9 10.94 1.26 1.15 12.73 1.31 1.14 10 12.22 1.19 1.35 12.67 1.19 1.37 11 13.43 1.55 1.26 15.48 1.71 1.22 12 14.00 1.59 1.28 14.32 1.62 1.24 13 13.12 1.42 1.26 12.55 1.28 1.33 14 11.26 1.27 1.18 12.16 1.25 1.19 Average 11.62 1.34 1.22 12.63 1.36 1.24

In terms of comparison between the original and revised samples, it can be seen that all three SCMs in the revised samples are on average higher than those in the original ones, albeit with a very small margin. The researcher plotted comparison between the original and revised samples on the three measures respectively and the results are show in Figure 19, Figure 20 and Figure 21 in the following. 79

10 MLS

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Sample No.

accepted MLS rejected MLS

Figure 19. Comparison on MLS between the two sets of samples

1.5

1 C/T 0.5

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Sample No.

C/T C/T

Figure 20. Comparison on C/T between the two sets of samples

1.6 1.4 1.2 1 0.8 T/S 0.6 0.4 0.2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Sample No.

T/S T/S

Figure 21. Comparison on T/S between the two sets of samples

According to the three figures above, there were times when the original samples were higher than the revised ones in a certain SCM in certain sessions. Further, the researcher calculated the distance between an original and a revised sample by taking the absolute value of their difference. Table 19 below show the result, and Figure 22 shows the trends of such distances.

Table 19. Difference between the two sample sets on the three SCMs

Sample Difference in MLS Difference in C/T Difference in T/S 1 2.35 0.04 0.10 2 1.78 0.11 0.02 3 0.58 0.02 0.05 4 0.32 0.12 0.05 5 1.08 0.06 0.05 6 1.17 0.15 0.03 7 2.25 0.04 0.06 8 0.34 0.00 0.05 9 1.79 0.05 0.02 10 0.45 0.00 0.02 11 2.04 0.16 0.04 12 0.32 0.02 0.04 13 0.57 0.14 0.06 14 0.90 0.02 0.01 Average 1.14 0.07 0.04

0.2 2.5

2 0.15 1.5 0.1 1 MLS

C/T and T/S 0.05 0.5

0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Sample No.

C/T T/S MLS

Figure 22. Trends on the three differences

Table 19 shows that the distances between the original and revised samples are rather small. The figure fails to show a clearly decreasing trend of the distance between original and revised samples on the three measures, which means that the student did not show consistent improvement in terms of syntax. However, figures in the table indicate that the distances between the two sample sets are rather small, with the greatest difference in MLS being 2.3532 in the first writing sample, the greatest difference in C/T being 0.1626 in the 11th writing sample and the greatest difference in T/S being 0.1014 in the first writing sample. On the 5th and 14th feedback, C/T and T/S are almost the same between the two sample sets. A closer look at the revised samples on these two points (Appendix M) reveals that teacher feedback was mostly on micro modifications. All these findings indicate that although the student did not improve, her syntactic complexity was already acceptable given the small differences between the two sample sets.

5.4.1.3 Semantics

Table 20 and Figure 23 below show Jaccard similarity coefficient and semantic similarity between the two sets.

Table 20. Jaccard similarity coefficient and semantic similarity between the original and revised writing samples

Sample Jaccard Similarity Semantic Similarity 1 0.84 0.992 2 0.71 0.988 3 0.76 0.984 4 0.66 0.984 5 0.67 0.986 6 0.68 0.984 7 0.58 0.968 8 0.62 0.976 9 0.60 0.971 10 0.53 0.964 11 0.69 0.975 12 0.70 0.980 13 0.63 0.976 14 0.64 0.983 Average 0.67 0.979

1.00

0.80

0.60

0.40 Similarity 0.20

0.00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Sample No.

Jaccard Similarity Semantic Similarity

Figure 23. Jaccard similarity coefficient and semantic similarity between the original and revised writing samples. 83

From the table and the figure, it can be seen that the Jaccard similarity between the two sets is generally in a decreasing rate, with an average coefficient of 0.666. Despite the decreasing literal similarity, the semantic similarity between the two sets were rather high, approximating to 1 in each pair of original and revised samples. On the 10th point, Jaccard similarity hits the lowest but the semantic similarity stands still high at 0.964. The researcher extracted two excerpts from the 10th original and revised samples for more detailed comparison.

Original excerpt:

The owner told Kristi terrible story about its illness, and it lost in a smelly river. Although, Kristi didn’t know its illness, she did this operation for the cat. After the cat was doing this operation, it felt scared and cried. She did it for a long time, but the cat was still ill.

Revised excerpt:

The owner told Kristi a terrible story about how his cat got ill: it fell down into a smelly river. After several diagnosis sessions, Kristi failed to figure out the illness. However, she decided to operate on the cat anyway, so she took the cat to the operation table. During the operation, the cat felt scared and cried. The operation lasted for a long time, but when it was done, the cat was still not cured.

Comparison shows that the teacher added more information to enhance cohesion of the story, which led to increased number of words in the revised excerpt. Also, the teacher corrected grammar mistakes, such as the lack of articles, and changed misuse of words (“the cat was doing this operation”). Despite the fact that the revised excerpt is of absolutely higher writing quality, the original excerpt was actually able to tell the same story in an understandable and complete manner. This may be why semantic similarity between the two are high despite of the low Jaccard similarity. Teacher feedback did little to improve the student’ semantic ability because gameplay had provided the student with enough information to depict a complete story.

5.4.2 Survey results

In the post-interview, the researcher asked the student about her experience in the classroom and findings from quantitative data. A list of question in the interview can be found in Appendix N. In a recap of her writing experience, she student said the game provided her with plots that could easily be turned into interesting stories in her writing and that connected all the words together in a logical way. She was able to learn and use these words thanks to the context. In terms of teacher feedback, she gave much credit to it in her improvement of syntax and grammatic accuracy. “I used mostly simple sentences in my writing before, and I thought that only one tense is allowed in an essay. The teacher helped me diversified my sentence structures and correct my grammar.” Examples given by the student were increased use of relative clauses and of present and past participles as adverbial phrases, and various tenses to indicate event sequence. In terms of difficulties, her writing process involved idea-forming in Chinese and then translation into English in her mind. During the translation process, she constantly encountered difficulties in finding accurate and corresponding words and expressions in English. However, with game vocabulary, she did not need translation and could directly use them in writing. The only problem was that she sometimes memorized or took notes of words and expressions with wrong spellings, for example, “map” instead of “mop”. When it comes to the forgotten words in game vocabulary, the student said she forgot many words from teacher feedback in the post-test because words the teacher used were more difficult and advanced, which meant she had difficulty remembering words of higher registers. For example, the student wrote “Kristi did not care her friend and forgot her” while the teacher changed it into “Kristi was so obsessed [with the snow] that she forgot her new friend”. The word “obsessed” was considered more difficult than “not care” by the student. When asked about the comparison on traditional writing and game-based writing classes, she said GBW was advantageous in that it offered a scenario where new words were presented in a meaningful and logical way, while traditional classes are more systematic in that they do not require students to explore the teaching points by themselves.

5.5 Discussion

5.5.1 Lexicon

Both quantitative and qualitative results indicated the GBW class had positive effects on the student’s productive use of game vocabulary. Two issues also appeared: how to ensure words in the game are remembered in the correct form and how to increase the retainment rate of higher-register words in teacher feedback. No GBLL studies have acknowledged the first question, and the researcher believes this issue is particular to GBLL learning because the linguistic elements appear and disappear quickly on a screen as the game develops. In traditional classrooms where books are available, students can always turn to printed information for help. To address this issue, the researcher believes teacher feedback on individual writing is absolutely essential. If possible, students can also share their notes with peers for review and enrichment of their game vocabulary. As for the second issue, there are previous studies conducted in conventional language classrooms indicating that teacher feedback was not effective in improving students’ writing ability. Truscott (2007) evaluated and synthesized research on whether error correction affects learners’ ability to write accurately. He found that in existing research, the best estimate was that correction had a small negative effect on learners' ability to write accurately, and that if correction indeed had any actual positive effects, they were very small. Another study by Ferris, Liu, Sinha, & Senna (2013) on written corrective feedback indicated that formal knowledge of language rules played a limited and sometimes even counterproductive role in their self-editing and composing. Based on the student’s interview, the researcher believes that the reason teacher feedback fails to improve students’ writing ability in many studies may be that such feedback is beyond the students’ current mastery or contains brand-new linguistic rules students need to familiarize themselves with before accurately applying such rules in writing. In this study, for example, the student forgot those higher-register words from the teacher while clearly remembered how to use more complex sentence structures, because she did not have a chance to use those words again after teacher feedback but was able to write many

sentences with a particular structure such as relative clauses. As such, more student practice with information from teacher feedback may increase student intake of feedback.

5.5.1.1 Saturation point for productive vocabulary in writing

Apart from game vocabulary, researchers can also examine and compare lexical sophistication between original and revised samples as a way to track students’ progress. Perhaps more relevantly, teachers would naturally want to know the appropriate time span for implementing game-based writing in classes. In the following discussion, the researcher outlines a way to calculate a point where GBW will no longer contribute to the use of new words with data from this study as an example.

With the original writing samples, the researcher also built a dynamic corpus, with each original writing sample being added one after another upon submission by the student. With 14 successive additions, the research calculated the accumulated number of distinct words in the dynamic corpus (C) upon each submission. For example, when the first sample was submitted, the researcher calculated the distinct words in the first sample

(C1); when the second sample was submitted, the number of distinct words in the first and second samples (C2) was calculated; when the last sample was submitted, the number of distinct words in all 14 samples (C14) was calculated. In this way, the researcher was able to know how many new distinct words the student used in the ith writing sample by subtracting Ci-1 from Ci. By dividing the number of new distinct words by the number of distinct words in the corresponding writing sample, the rate of new distinct words can be obtained, as is shown in formula (2):

�� − �� = (� ≥ �) (2) ��

In the formula, R1 refers to the rate of new distinct words in the ith writing sample;

Ci refers to the number of distinct words in the ith corpus; and Ni refers to the number of distinct words in the ith writing sample. “i” is set to be no less than 2 because all distinct words in the first writing sample are new, rendering R1 =1 all the time.

The result is shown in Table 21 below.

Table 21. Rate of distinct words in each original sample

Accumulated new distinct rate of new distinct words Sample distinct words words distinct words

(Ni)

(Ci) (Ci-Ci-1) (Ri) 1 145 145 145 1 2 239 94 131 0.72 3 304 65 127 0.51 4 406 102 180 0.57 5 464 58 156 0.37 6 523 59 145 0.41 7 573 50 163 0.31 8 627 54 163 0.33 9 669 42 168 0.25 10 707 38 126 0.30 11 736 29 145 0.20 12 792 56 184 0.30 13 844 52 188 0.28 14 902 58 176 0.33 Average N/A N/A 157 N/A

1.2

0.8

0.6

0.4

0.2 Rate of new distinct words

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Sample No.

Figure 24. Rate of new distinct words in a writing sample (Ri)

From Table 21 and Figure 24, it can be seen that as her gameplay developed, the student naturally added new words into writing samples, but the rate of new distinct words in each sample gradually decreased. In theory, a hypothesis holds true that due to the exhaustive themes and vocabulary in the enclosed game environment, the student will stop using new words once the number of writing assignments reaches a “saturation point”, where the rate of new distinct words will be zero or near-zero. Though Figure 24 does not show where the saturation point will be, prediction is possible with a function whose plot would match the data trend and the hypothesis. The hypothesis that the rate will be nearing zero when x nears infinity means the function will be an exponential one. By using the embedded trendline function of MS Excel, an exponential function is obtained as follows:

� = ��.�� (3) The plot of the function f(x) is as follows in Figure 25.

1.2

0.8

0.6

0.4 y = e-0.119x 0.2 Rate of new distinct words 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 Sample No.

Figure 25. Trendline for rates of new words in writing samples

As the saturation point is assumed to be the time at which the student will no longer add new distinct words into her writing, the point before the saturation point, defined as the P point here, will have at least 1 new distinct new word in the sample.

Divide 1 by distinct words in the Pth sample (Np), the minimum rate of new distinct words will be obtained. The researcher here uses the average of N (Navg=157) in Table 21 to -0.119x predict Np, rounded to 1. By solving the x in e =1/157, the result 62 (rounded to 1) is achieved. That is to say, the student will stop adding new distinct words into her writing in the 63th writing sample. Of course, having students write 63 samples in a class is not practical, and especially in the case of on online games with open gaming environments, the saturation point will appear much later. However, with this method, teachers can conduct their own studies and set reasonable goals (for example, setting 2 or even more new distinct words rather than 1 before the saturation point) for future students.

5.5.2 Syntax

As is shown by quantitative results, the subject showed a high level of syntactic complexity as that in the samples revised by the teacher. There are two possible explanations for this result. First, the teacher’s revision focused on wording, instead of sentence structure that has a greater influence on syntax. Second, the teacher did revise sentence structures, but such revisions at different locations offset one another in a writing 90

sample, rendering the total unchanged. A closer look at the 10th samples (Appendix O) where the three SCMs are very close between the two set reveals that both reasons attributed to the close syntactic similarity. For example, the teacher changed the student’s second sentence “She bought a pet house, because she loves pet” into “As she loves pets, she decided to buy a vet clinic” for better logic connection with the previous sentence and for higher accuracy of word use. However, the T/S and C/T are unchanged and even the length of sentence after change, 11, is very similar to the original one, nine. In the last paragraph of the student’s writing sample, which consists of only one sentence, the T/S and C/T are three and two respectively (three independent sentences linked by one “but” and one “and”; one main clause and one adverbial clause introduced by “because” in the last T-unit). The teacher changed this sentence into two separate ones, with the original measures reduced to the average of those of the two separate sentence, T/S=(2+1)/2=1.5 and C/T=(1+1+2)/3=1.33. Revisions that reduce the measures like this are likely to have offset the teacher’s effort to add more complex sentence structures. Therefore, the three measures will still stay very similar after revision overall. Similarly, higher-quality writing may show a syntactic variety where T/S, C/S, MLS can be higher in certain sentences and lower in others.

In addition, if we look closely at the syntactic complexity of the original sample, we can find that none of the measures showed a consistent trend, and that ups and downs were interwoven. This lends support to previous studies that question the use of syntactic complexity at the text level to predict writing quality (Hirano, 1988; Qin & Uccelli, 2016). To approach this statistically with T/S as an example, the minimum of T/S in every random sentence will be no less than 1 (one independent sentence without any “and”, “or”, “but”, etc. to connect other independent sentences). Then it would also be strange for every sentence to have a T/S of 3 (three independent sentences linked by “and”, “or”, “but”, etc.). As such, for a normal writing sample, its T/S at the text level is expected to be somewhere between 1 and 2, and studies investigating T/S have supported the researcher’s assumption with numbers in this range as well (see for example Frear & Bitchener, 2015; Lu & Ai, 2015). At this point, a question emerges: if T/S for all writing samples fall between 1 and 2, how much do digits after the decimal matter? And how 91

large can a difference, such as the difference between 1.1 and 1.2, be deemed statistically significant to a certain genre? Due to the above two factors, the researcher believes syntactic complexity is fundamentally an invalid indicator of writing quality, but if syntactic similarity between original and revised samples can be calculated at the sentence level, by adding up the absolute value of the difference of each sentence pair, the offsetting problem will be effectively addressed.

5.5.3 Semantics

The high semantic similarity between the two sample sets showed that the student was able to convey her stories in the full, though there were errors in wording and grammar. This further indicates that computer games, with their visual, aural, written, spatial, and kinesthetic modes, can facilitate greater accuracy in student writing. In addition, the results of this study also support the researcher’s claim that semantic similarity at text level can be used as an indicator of information accuracy, as part of writing quality. However, more indicators are required to complement semantic similarity in terms of grammar, syntax and register/lexical sophistication. Another possible reason for the high semantic similarity between the two sample sets is that the teacher’s revisions were based on the original samples. In future writing studies, if the revised sample set are changed into independent writing by the teacher, the results would be different. But again, as the researcher noted previously in reference to work by Kormos (2011), how to ensure that students and the teacher are writing about the same thing from the same perspectives will be extremely difficult, especially in argumentative essays. This method may be more applicable to description where no subjective opinions are solicited.

5.6 Limitations of the study

The most obvious limitation of the study is the small sample size. The researcher intended to investigate face-to-face individual feedback to student writing in class, and in this case, time devoted to feedback is linearly related to the number of students. In order to balance the time for feedback, instruction and gameplay, the researcher could only work with one

student in this exploratory study. That is also the most common reason students in the post-study survey mentioned regarding why their teacher barely gave individual feedback—there was simply not enough time. In future studies, teacher-practitioners may consider individual feedback on the written form during after-class hours and presenting the most salient issues discovered in student writing during class. In addition, student debriefing through online platforms by the instructor as well as to peers can also be beneficial (Crookall, 2010, 2014).

The pre-study questionnaire was administered to students of the same university. Although these students had different English teachers, the results may still be biased as these teachers could have been influenced by the same teaching ideology and curriculum determined by the university. Besides, students in the same university may have similar English levels, as is seen from students’ test scores in the pre-study survey. Thus, results concerning students’ perspectives or opinions may not apply to students at other universities of different tiers. Last, student self-reporting can be inherently biased, as sometimes they would say what the teacher wants to hear.

5.7 Conclusion and Future Directions

This mixed methods study looks at how a game-based writing class influences student narrative writing in terms of lexical use, syntax and semantics through the student’s productive use of game vocabulary, syntactic similarity and semantic similarity. The student’s original writing samples and samples revised by the teacher were compared with the latter as benchmark to generate similarity data. Evaluation relied on both NLP tools and human coding for text analysis and interviews with the student were also included. The study concluded that the game alone provided students with meaningful and logical information to compose narratives; that the combination of teacher instruction and the student’s gameplay facilitated the student’s productive use of game vocabulary; and that teacher feedback was less efficient in vocabulary retainment but was effective in improving the student’s syntax. Further, the study rejects the use of syntactic complexity at the text level as an indicator of writing quality due to the inherent offsetting issue

among different sentences in a text, and instead proposes syntactic similarity at the sentence level for evaluation students’ writing samples against benchmark samples.

In future research involving GBW classes, the researcher also proposes the prediction of a lexical saturation point with rates of new distinct words in samples, which can be used as reference for class duration planning. Where teachers find it difficult to implement GBW due to time limits, videos from game streamers and uploaders or game walkthroughs can also be used during class time for analysis of vocabulary, grammar and sentence structure. This configuration will be very similar to studies on the influence of multimedia on student writing, including videos, audios, infographics, etc., which are already proven effective (see for example Wang & Dalsky, 2017). What is unique about using game videos, however, is that students can be encouraged to play the game after class to complement class hours as well as to maintain high learning motivation and engagement.

The Third Study Part II: Back to Communicative Competence

6.1 Introduction

This chapter is the second part of the 3rd study, where the same student participated in a communication-oriented class with communication activities based on her previous game-based writing and with teacher feedback on how to improve her communicative competence. This chapter builds on the second study on communicative competence and further includes writing as scaffolding for speaking in the GBLL classroom. During speaking activities in this part, more speaking sessions were held and more interaction opportunities were offered to the student. In the first two studies and especially the second one, the researcher found that the lack of interaction was a salient issue in communication activities and that the lack of preparation and teacher feedback on means of improvement were the major reasons for limited fluency development. Therefore, in this study, the researcher decided to target these weaknesses in the previous study and evaluate how the student’s communicative competence would develop with rich interaction opportunities and writing as scaffolding in speaking activities.

6.2 Background

6.2.1 Relationship between writing and speaking in second language classrooms

In second language classroom research, there have been ample studies dedicated to the relationship between writing and speaking development of students. Most studies concluded that speaking could facilitate writing development (see for example Halpern, Vann, 1982; Hubert, 2011). As early as in 1982, Vann (1982) focused on students who were non-native speakers of English and proposed a model for the development of students’ writing skills, a model which resembled the relationships between children’s 95

oral and written language development. Further, she offered an alternative to the static notion of oral interference as an explanation for the writing problems of ESL students. Instructional strategies for bridging the gap between the spoken and written communication skills of ESL students were also discussed. In the Introduction of the book Connecting Speaking & Writing in Second Language Writing Instruction, Weissberg (2006) initiated the reader into his central argument that second language writing is best acquired through a dialogic classroom model. One of his key assumptions was that “social interaction provides an ideal context for mastering complex cognitive skills like writing” (p. 3). In “From Talking to Writing” of the same book, Weissberg put forward developmental and sociocultural theories, as well as evidence from empirical studies, to support this claim that social interaction should be a basic part of the L2 writing classroom.

Researchers who concluded that speaking facilitates writing in similar studies basically based their assumptions on children’s L1 acquisition where they start to speak long before they begin to write. Writing in a language is an artificial act that cannot be achieved without training and guidance and therefore, it is natural for children to speak better than they write for a certain period of time (Baba, Takemoto, & Yokochi, 2013). Kroll (1981), for example, compared younger (3rd and 4th graders) and older (6th graders) children’s speaking and writing performances when they explained a board game in the two modalities. He concluded that the younger children produced more information when speaking than when writing while the older ones produced a similar amount of information in both modalities.

However, very few studies have looked at the synergy between speaking and writing for L2 adults (Belcher & Hirvela, 2008). It is obvious that Kroll’s theory for L1 children cannot be directly applied to L2 adults (Weissberg, 2006; Williams, 2008). Cognitively, L2 learners “learn” instead of “acquiring” a language, which suggests accumulation of grammar rules and vocabulary is the most common way to mastering a language. In addition, adults are mostly likely to receive the input of grammar rules and vocabulary from textbooks, or in the written form. As a matter of fact, many L2 learners, especially those in China and Japan, are exposed to more practices of writing rather than speaking and therefore have higher writing competence than speaking, as indicated by 96

data from the official website of TOEFL (Test and Score Dara Summary for the TOEFL iBT Tests, n.d.). Weissberg (2000) investigated how five adult Spanish learners of English developed syntax in their speech and writing over one semester. The five learners engaged in various speaking and writing tasks, and the syntactic features in their speech and writing were analyzed in terms of accuracy and syntactic innovation. Weissberg found that syntactic accuracy in writing developed to a greater extent than in speech over the semester and that more than half of the new syntactic forms appeared first in writing while about 20% appeared first in speech (the others appeared in both simultaneously). These findings suggest that L2 learners’ development of speaking proficiency does not always precede that of writing proficiency and that the development of writing proficiency may facilitate and promote that of speaking proficiency.

6.2.2 Interaction in second language classrooms

According to Brown (2007), interaction is the collaborative exchange of thoughts, feelings, or ideas between two or more people, resulting in a reciprocal effect on each other. Tracing the background for an interactional theory of language, Piaget & Inhelder (1969), stated that language represents the knowledge a child has acquired by means of physical interaction with the setting around him/her. Vygotsky (1978) claimed that language develops from social interaction for communication purposes. Moreover, based on the psychological approach to learning, other authors as Hatch (1992), Pica (1994) and Long (1983) in Lightbown & Spada (2006), have claimed that conversational interaction helps second language acquisition. Ellis (1980) speculates that second language learners obtain more practice in a foreign language and are motivated to engage in further communication when they have greater opportunities to speak the language and achieve successful communication.

Inside the EFL classroom and for the past decades, interaction has been the object of many theories in language teaching and learning such as the Communicative Approach, Communicative Language Teaching (CLT), Content-Based Instruction and so forth. For many language learners, the classroom is the primary arena where they receive intensive thought-provoking input, practice the language, and reflect on their use of it. Providing 97

learners with activities that nurture this exploration and allow for interaction is important for their language development and for preparing learners to use the language successfully when they leave the class environment (Allwright, 1984).

In terms of interaction participants, interaction in second language classrooms mainly includes two types: teacher-student interaction and student-student interaction (peer interaction). Woolfolk (2004) emphasized the importance of learners’ interaction with their peers in order to validate their thinking, to be questioned, to receive feedback, and to observe how others work out problems. Teacher-student interaction, according to Tsui (1995), manifests primarily through feedback in the way teachers make evaluations and give comments on students’ performance in order to repair their utterances or to acknowledge the information given by the students. Lyster and Ranta (1997) cited in Lightbown & Spada (2006) and Crookes & Chaudron (1991) cited in Celce-Murcia (2001) highlighted the types of feedback as follows: explicit correction or the teacher providing the correct form of what was incorrect; recasts or the teacher reformulating all or part of students’ utterances minus the error; clarification request or the teacher indicating the misunderstanding of a students’ utterance; metalinguistic feedback or the teacher providing information, comments or questions about how to form the students’ utterance without giving the correct form; elicitation or the teacher using three techniques to correct students (elicit completion of the utterance).

Vigotsky (1978) asserted that the real importance of interaction is the one which focuses on the participation of both the learner and the interlocutor emphasizing the relevance of modified input. On this matter, Ellis (1986) stated that motherese, a type of modified input in a natural setting with a mother and her child, or language adjustments a mother uses for making language clear, resembles a mechanism known as foreigner talk employed with some modifications in the classrooms. According to Ellis, in foreigner talk the teacher or the native speakers take the role of a mother and her child when communicating with the students, using simpler sentences and vocabulary, easy grammatical items, imperative commands and more clarification requests. Ellis also observed these adjustments and classifies them into three levels of modified input: pronunciation, grammar, and lexis. Tsui (1995: 55) explained these levels as follows: “In 98

terms of phonology, teachers tend to slow down their speech rate, and use less reduced vowels, fewer contractions, more standard pronunciation and more exaggerated articulation. In terms of syntax, teachers tend to use better-form and shorter sentences, and fewer subordinate and conditional clauses. In terms of vocabulary, teacher talk is more basic, with fewer colloquial expressions, more concrete and proper nouns, and fewer indefinite pronouns”. When students cope with the learning of a foreign language, the adjustments performed by the teacher help them to understand better the target language.

6.2.3 The present study

The second study on communicative competence concluded that the GBLL classroom lacked peer interaction due to cultural issues, and that the participants did not prepare enough for the presentational activity in the classroom. As the lack of peer interaction was repeatedly observed in both the first and second studies despite the teacher’s effort to encourage such interaction, in this study, the researcher decided to engage three teachers in the communicative activity, rather than students. In this way, the teachers can assume both the role of more capable peers interacting with the student and of teachers giving feedback and instruction regarding the student’s performance. For preparation, the writing samples in first part of study will serve as scripts for the student to better prepare for the communicative activity. As such, the research question of the second part of the third study is as follows:

How will the student’s communicative competence develop with rich interaction opportunities and writing scaffolding?

6.3 Methodology

6.3.1 Study design

The study is the second part of the study in Chapter 5. After the student completed her game-based writing classes, the curriculum focus shifted to communication skills and the student was engaged in speaking activities based on her writing in the first part of the

study. The purpose is to investigate how writing-scaffolded speaking activities with rich interaction opportunities would influence the student’s communicative competence in a GBLL classroom.

There are altogether 13 speaking sessions that corresponded to the 13 writing samples the student wrote and the gameplay clips the student used as materials for writing were used for speaking as well. Before each speaking session, the student would prepare on her own with her writing as speech scripts. The speaking activity consisted of both presentational and interactional tasks, with the former in the form of the student’s video- based storytelling and the latter in the form of student-teacher Q&A. After the speaking activity, the teachers would give feedback to the student on how to improve her communicative competence as a form of instruction. The whole speaking sessions were recorded.

Before the class started, the researcher conducted a pre-test on the student and a post-test was conducted one week after the study. In both the pre-test and post-test, the student was shown a picture and was asked to talk about what she thought was happening in the picture (Appendix P). The teacher then asked the student some questions. The pre- test and post-test were evaluated by the same three teachers.

6.3.2 Data collection and analysis

Data collection and analysis in this study are three-phased. The first phase consisted of quantitative data in the form of teachers’ ratings of the student’s communicative competence (including the pre-test), and qualitative data including the teacher’s notes, remarks and recordings of the speaking sessions. After the student had completed the communicative activity, the three teachers would independently rate the student’s performance according to a rubric of three metrics, interaction, fluency and content (Appendix Q), which was adapted from the one used in the second study in Chapter 3. The difference between the two rubrics is that the one used in this study is scaled from 0 to 10, as opposed to 0 to 5 in the previous one. Remarks and notes were provided by the teachers as rationales for ratings. The ratings were kept unknown to the student. Their

100

average ratings were used as results and the overall score on a certain session was the average of three metrics.

In the second phase, based on results in the first phase, the researcher adopted speech rate to provide more data triangulation on the metric of fluency in the rubric, as fluency manifested a different development trend from the other two metrics. Speech rate is calculated by dividing the number of words in the student’s presentation by the duration of the presentation. The number of words included all complete utterance of words, with repeated utterances such as “he, he, he” counted as three words. The researcher counted the numbers twice and the average of the two were used as the final number of words for a presentation. As the researcher also recorded speaking sessions, the timeline on the video files provided the duration of each presentation. The speech rate for a certain session can thus be determined as the following formula:

��1 + ��2 2 Speech rate = ��

In the third phase, the post-test was administered and an interview was conducted to solicit more qualitative data on her attitudes and perspectives.

6.4 Result

6.4.1 Rubric-based teacher evaluation

6.4.1.1 Pre-test

In the pre-test, where the student was asked to describe a picture presented to her, she was rated 2.33 on interaction, 2.67 on fluency and 3.00 on content by the three teachers on a scale of 0-10. The teachers remarked that there were very long pauses in her speech that indicated her unfamiliarity with the content in the picture. In addition, she was not able to react effectively to questions and comments from the teachers. When communication breakdown occurred, whether due to unknown words used by the teachers or questions

101

she did not know how to answer, she would usually show a puzzled or upset face, looking up to the ceiling, rather than negotiating with the teachers.

6.4.1.2 Writing-scaffolded interaction-rich speaking activity

The evaluation results for the 13 sessions of speaking activities in the GBLL classroom are as follows in Table 22, Figure 26 and Figure 27.

Table 22. Teacher evaluation on the student’s communicative competence during the experiment

Session 1 2 3 4 5 6 7 8 9 10 11 12 13

Interaction 3.33 3.33 4.67 4.33 4.33 5 4.67 6.00 6.67 6.00 6.00 6.67 7.00

Fluency 4 4.67 5 5.33 5 4.67 5.33 5.33 5.67 5.00 5.00 6.00 4.67

Content 5.33 5 5.33 5 4.67 5 5.00 7.00 7.17 6.17 6.00 7.00 6.33

Overall 4.22 4.33 5.00 4.89 4.67 4.89 5.00 6.11 6.50 5.72 5.67 6.56 6.00

7.5

6.5

5.5

Rating 5

4.5

3.5

3 1 2 3 4 5 6 7 8 9 10 11 12 13 Session No.

Interaction Fluency Content

Figure 26. Teacher evaluation on the student’s communicative competence during the experiment 102

7.00 6.50 6.00 5.50 5.00

Rating 4.50 4.00 3.50 3.00 1 2 3 4 5 6 7 8 9 10 11 12 13 Session No.

Figure 27. Teacher evaluation of the student’s communicative competence overall

Generally speaking, the student showed improvement in her overall communicative competence, as indicated by Figure 27 above.

In Figure 26, the three metrics in the rubric generally follow an increasing trend, albeit with ups and downs. Among them, interaction sees the greatest improvement, from less than 3.33 in the first session to 7.00 in the last session. Content ranks second, from 5.33 to 6.33, while improvement on fluency is less satisfactory, from 4 to 4.67. Overall, the rating on student’s communicative competence increased from 4.22 to 6.00.

Three features can be identified in the table and figures above: very low interactional skills at the beginning, very steep improvement in session 8 on both interaction and content and limited improvement on fluency. To understand these features, qualitative remarks from the three teachers are very helpful.

In terms of low interactional skills at the beginning, the teachers agreed that the student was simply reading her scripts in the first and second sessions. In her reading aloud, she was not even able to read clearly and smoothly, which made it difficult for the teachers to identify her sentence structures. Besides, her poor pronunciation provided a hinderance to understanding. In addition, the student lacked audience-awareness. Her presentations usually started right away with reading her scripts, without any opening remarks or introduction such as “today I’m going to tell you a story about…” However, from the third session and with the teachers’ repeated feedback, the student dropped the 103

scripts and added introduction to her presentation, which explains the uptick on interaction in the third session. In the last session, the teachers all commented that the student was talking to them in a very natural way as if the presentation was no longer a task for her. She was eager to tell the teachers what happened and responded to their questions with wit and humor. For example, when one of her characters died in the game, she said “although Kristi is sad after Jaden’s death, the good thing is she can find a new boyfriend, or boyfriends.”

As for the upsurge on both interaction and content in the 8th sessions, the teachers remarked “not much eye contact but much better from last time and the student negotiated meaning with the teachers” and “the student smiled towards the audience and added gestures to communicate more naturally”. Also, in the 8th story, one of the characters in the game died and it came as a shock to the audience. It was very eye-catching and the student depicted it vividly in detail. Therefore, the ratings on content were rather high.

In terms of improvement in fluency, the three teachers observed that the student’s pronunciation and speech tempo were the major challenges for her. In the first session, the student’s speech was barely intelligible and the teachers could only understand her with a script of her speech. Also, the student had some deep-rooted issues pronouncing certain vowels and consonants. She would pronounce /e/ into /ai/, and therefore “air” sounded like “ire” and “death” like “die-th”. The consonant /n/ was also difficult due to influence from her local dialect which does not distinguish /n/ from /l/. The teachers corrected such pronunciations many times, but the student would still make the same mistake again and again. Apart from these patterned mistakes, she would also pronounce words into other totally irrelevant words, for example, “hell” into “hole” and “sharply” into “shopping”. The teachers suggested she try to speak more slowly so that she could pronounce each syllable clearly, and she did make some small progresses. In terms of speech tempo, the student tended to repeat a word or phrase several times before she came up with the next word. For example, she would say “so, she, she, she, she went to sleep…”. Perhaps most importantly, the teachers all pointed out to the student’s unnatural pauses. The student made many pauses in between words and sentences and this had affected her fluency in two ways. First, long pauses cut her speech into fragmented sentences that 104

compromised her speech fluency. Second, it was difficult to identify the syntax of her speech which then undermined comprehension by the audience. In face of these two challenges, the student made only minor progress.

6.4.2 Speech rate

As human rating failed to indicate consistent improvement on fluency, the researcher decided to use a purely quantitative data to validate results derived from human ratings. Table 23 below shows the speech rate of the students throughout the experiment.

Table 23. Speech rates of the students in the 13 sessions

Number of words Duration Speech rate Session 1st 2nd (minutes) final (words/minute) counting counting 1 5.63 334 344 339 60.18 2 6.50 312 333 322.5 49.62 3 5.35 310 327 318.5 59.53 4 5.88 382 366 374 63.57 5 8.25 402 382 392 47.52 6 10.83 491 519 505 46.62 7 12.22 537 531 534 43.71 8 7.35 380 368 374 50.88 9 11.50 477 438 457.5 39.78 10 9.75 422 420 421 43.18 11 12.12 497 511 504 41.60 12 22.72 856 844 850 37.42 13 15.83 608 590 599 37.83 Average 10.30 NA NA 462.42 47.80

In the Duration column, it can be seen that the student’s speech was gradually extending with more interesting stories observed by the teachers in human ratings. It

105

should be noted that the durations also included time devoted to video playing, because it was impossible to separate the videos from the student’s speech. As such, there may be outliers in the data where time devoted to playing the video in a certain session was much higher than in other sessions. To validate and exclude outliers, the researcher plotted the data with presentation time as the x-axis and number of words as the y-axis (see Figure 28). Theoretically speaking, the number of words in a presentation should be linearly related to the duration of the presentation and therefore can be approximately represented by a “y=a*x” function (if all data fit perfectly into the function, the “a” in the function will be equal to the average speech rate). Based on this assumption, a linear regression analysis was performed using the built-in functionality of Microsoft Excel. As shown in Figure 28, a dotted regression line was plotted denoted by “y=42.089x”.

1200

1000 y = 42.089x 800

600

400

200 Number of words in the speech 0 0.00 5.00 10.00 15.00 20.00 25.00 Duration of the speech (in minutes)

Figure 28. Linear regression on speech rates

To test whether outliers are included in this dataset, the researcher calculated the distance of all the data points to the regression line, and checked whether the data are in the range of [� − �(� − �), � + �(� − �)] where � is the first quartile of the dataset, � is the third quartile of the dataset, and � = 1.5, a common constant used to detect outliers. It turns out that all the numbers are in this range, which means that there are no outliers and data in the Speech Rate column are all valid. That said, a plot of speech rate can thus be made which is shown in Figure 29 below. 106

70.00 60.00 50.00 40.00 30.00 20.00 10.00 0.00

Speech rate (words/minute) rate Speech 1 2 3 4 5 6 7 8 9 10 11 12 13 Session No.

Figure 29. Trendline of the student’s speech rates

Figure 29 shows a generally declining trend in the student’s speech rate. The result supports teachers’ ratings and observations that the student’s improvement in fluency was quite limited. The average speech rate, 47.80 words per minute, indicates that the student on average uttered 47.8 words in one minute of presentation in this GBLL class.

6.4.2.1 Post-test

In the post test, the teachers rated the student 7.33 on interaction, 5.00 on fluency and 5.33 on content. Compared with the last 3 sessions of the speaking activity, the student stabilized her performance on interaction, which means that she could maintain a similar level of interaction in tasks other than the one in the GBLL classroom. Fluency was relatively unchanged, while content dropped by a small margin. The teachers observed that as the student did not prepare a written script and the picture gave only part of the information needed for the post-test, the student seemed to stuggle on main ideas and speaking logic. Compared with the pre-test, however, it can be concluded that the student improved by a large margin in all three metrics.

6.4.2.2 Interview

Based on results above, the researcher asked the student questions specifically targeted at her interaction, fluency and attitudes towards the GBLL class. The interview was conducted in Chinese so that the student would better express herself. The student said that before she took the class, she was afraid to talk in English because most often she 107

could not find appropriate ways to express her ideas. However, she had a strong intention to improve her daily communication skill. As none of the English courses at her university focused on communication, she thought the GBLL class was one of a kind in promoting both learner motivation and skills in English communication. Through the GBLL class, she gradually overcame her fear of making mistakes in speaking and could talk more naturally with the teachers. At the beginning, she would constantly refer to her writing because she was not confident enough. Later, she discovered that her writing was actually a hinderance to her speech, because she always intended to recite perfect sentences. She forced herself to drop the scripts and found that spontaneous speech with writing as background scaffolding was much better. The teachers also discovered that as she stopping relying on scripts, there were times when she added more improvised content, such as trying to imitate a character in the game by singing a song. In addition, she also mentioned the communication strategies as important takeaways from the teachers. She learnt how to make requests for clarification, instead of standing there only showing a puzzled expression. The later sessions involved light-hearted story-sharing and conversations with the teachers.

In terms of fluency, the student admitted that her pronunciation was poor, and sometimes she could not put her tongue into the right position for a sound. Twenty years of local dialect had also affected her pronunciation immensely. As she barely practiced speaking before, this class meant a good start for her. As a result, she was not concerned with the absence of short-term effects on her fluency. She was positive that with more practice, she would improve slowly but surely in the future.

When asked about her attitude towards the GBLL class and the challenges of applying such a class into her university curriculum, the student said she liked the class arrangements very much and wanted to see more of it. However, she did not think it was possible to have such classes at her university, where English as general education subject was taught in large-size classes of around 100 students. With so many students, it remained unsure whether all students would like to learn from games and some would easily lose their concentration in a playful classroom atmosphere.

108

6.5 Discussion

6.5.1 Research question

Interaction helped the student to become more confident and natural in her speech and conversation with the teachers, as indicated by the interaction metric in the evaluation rubric. Besides, through interaction, the student was also able to take advice and feedback from the teachers for improvement. This had positively impacted the student’s pronunciation in fluency and vocabulary in content. However, it should be noted that the student interacted with more capable interlocutors. In most second language classrooms, it is not possible for the teacher to interact with every student as in this study. Therefore, effects on communicative competence would be lower with peer interaction.

The writing samples from the first part of the study was useful for the student during preparation, but was counter-productive when the student was engaging in speaking activities. Therefore, the researcher believes writing can be very useful for presentations, but information should be internalized to ensure natural communication.

6.5.2 Speech rate

On average, the student had a speech rate of 47.8 words per minute. However, this does not mean the student’s speech rate would stay approximately to the average if the class continued. Theoretically speaking, with classroom interventions, a student’s speech rate would manifest changes, be they ups or downs, during early stages before it stabilizes at a certain point. In this class, the student’s speech rate was downgrading at the beginning, possibly because she was reading scripts in the first several sessions and she gradually added more audience-awareness in later sessions. Although the plot seems to stabilize at the 12th and 13th session, it cannot be concluded that the stabilized speech rate would be around that level. More sessions will be needed to generate a reliable result in terms of this issue. The researcher here proposes longitudinal studies to predict and validate students’ stabilized speech rates. In addition, as the teachers observed, the student uttered some words or phrases repeatedly during presentations. If the student continues to

109

improve her fluency and reduced such repeated words, she will be able to fit in more meaningful words under the same speech rate, which means the same speech rate can represent different communication efficiencies.

6.6 Conclusions

The study looks at how rich interaction rarely found in ESL classrooms in China and writing as scaffolding would influence a student’s communicative competence through a series of speaking activities in the GBLL classroom. The study concludes that rich interaction, especially interaction with more capable interlocuters, is conducive to student’s communicative competence. It also shows that writing can be both helpful and counterproductive in that the student constantly intended to recite the writing scripts. Meanwhile for fluency, it may take much longer before the student shows conspicuous improvement, as indicated both by the teachers’ ratings and the student’s speech rates.

In terms of limitations, as the whole study is a single-participant experiment, the results apply only to students of very similar English levels and learning backgrounds. In addition, the researcher used speech rate to further explore why fluency showed only minor progress, but there were no such purely objective quantitative data to supplement the other two metrics of interaction and content where the student showed much progress. It appears that based on the finding of this study, coding of communication strategies may be useful in supporting improvement in interaction.

In future research, researchers may target studies of longer terms and/or larger scales. In addition, with different game genres of various language input features, studies investigating less commonly seen themes such as listening and reading comprehension may yield important findings, particularly when GBLL classrooms are combined with out-of-class activities such as live streaming and community discussions on game fora.

110

Conclusions, Limitations and Future Research

7.1 Conclusions

The conclusions drawn from the series of studies presented in the thesis can be summarized into two major areas: why and how games should be integrated into language classrooms; and what aspects of language skills can benefit from a GBLL classroom.

Results from the studies indicate that when games are integrated into language classrooms, students showed better learning outcomes while maintaining high motivation. In securing fruitful learning results from playing games, a classroom environment provides students with necessary guidance in face of linguistic input scattered around the gaming environment, so that they can identify the learning focuses, i.e., the curriculum goals. Teachers play an important role in the classroom by filtering useful information and giving instructions and timely feedback to keep the students on track. In this way, games can be seen as a teaching tool, similar to other teaching materials such as books or videos, but with a high level of interaction and engagement.

While classroom integration is necessary for games to yield better learning results, it should be noted that not all language classes need to involve games in teaching. Qualitative data from the studies suggest that the simulation game investigated in this research was conducive to learning daily English, in both writing and speaking, but the game may not be as useful when integrated into a more academic course, for example, scientific writing. In fact, the majority of COTS games are not targeted towards an academic audience and therefore teachers should carefully decide whether or not to utilize games in their classes. When it comes to academic content, students may even prefer to learn through traditional methods, such as reading research articles and writing reports, rather than through playing games, even “serious” games designed specifically for educational purposes.

When integrating games into classrooms, teachers need to carefully design plans and interventions. The studies conclude that not all interventions in traditional classrooms, 111

for example, peer interaction, are useful in GBLL classrooms. Classroom activities suited to particular curriculum goals should be put in place to evaluate learning results and/or help students use what they have learnt. Gameplay tasks, on the other hand, will be useful in reminding the students to stay focused on certain targets instead of roaming aimlessly around in the game environment. Another intervention that looks promising is supplementary out-of-class activities, such as gameplay after class and engagement in a gaming community. As van Gog (2013) notes, games can be used out of class to effectively extend the school day and increase time on task, which has been shown to be a key determinant of learning. Mc Neil (2018), in arguing the importance of the linguistic input from gaming communities, claims that without knowledge of what games are and what gaming entails, teachers may focus on in-game texts only and not consider how interactions with other gamers and texts outside of games afford possibilities for L2 learning.

7.2 Limitations

The limitations of the research can be summarized into two major categories in terms of study design, data collection and analysis. In terms of study design, the limited number of participants represents a limitation of the research. In the second study, in particular, no control group was involved due to the difficulty in recruiting volunteer participants and also the ethical issue involved in recruiting students to play games in a classroom environment without any intervention. In the third study, the detailed analysis of writing samples and presentations was so time-consuming that the researcher was only able to focus on one student. Therefore, the generalizability of the studies is limited to classes of similar configurations and students of similar competences. In addition, all three studies in this research lasted for no more than half a year. According to findings in the third study, it may take more over 60 sessions for the student to be proficient in writing narratives and even more in daily communication, which suggests that studies of longer durations, for example, over a year, are needed to confirm that students can actually achieve a high level of proficiency through GBLL.

112

For data collection and analysis, the mixed-method studies included student self- reporting as part of qualitative data. The validity of such data may be questionable as participants may have adjusted their responses to include favorable comments towards the studies and the researcher. Besides, self-reporting also served as the basis for some quantitative data in the studies (for example, students’ ratings), and therefore there is a possibility that the quantitative data may have been biased as well. In fact, evaluators’ ratings of student performance on communication activities may have been prone to personal biases as well. In this context, the researcher believes the objectivity or the minimization of biases in both quantitative and qualitative is a must for studies to generate reliable results. Coding, for example, is a good way to turn data into quantifiable metrics while still offering qualitative support. More utilization of NLP tools may also be helpful, such as the tools used to analyze syntactic complexity and semantic similarity. In evaluating students’ pronunciation, for a further example, researchers can make use of speech-to-text tools to transcribe students’ speech into texts to evaluate how many of the words are intelligible, rather than determining intelligibility against the researchers’ personal standards. The use of NLP tools will also help scale up research in the future given the batch processing capability offered by such tools.

7.3 Future Research

Due to limitations of the selected game, the studies were not able to further provide evidence of games facilitating reading or listening skills. However, with a myriad of COTS games available in various genres, the researcher believes all four skills, i.e., reading, listening, speaking and writing, can be enhanced with specific types of linguistic input from games. Drama games, for example, may promote reading skills as players constantly need to make quick decisions based on texts on the screen. Massive multiplayer online games where audio chat is available for gameplay, such as in the battel royale game PlayerUnknown’s Battleground, are likely to enhance both listening and speaking, as team communication is vital to player survival in the game.

113

In addition, researchers may consider expanding research into virtual words that also offer great educational potential. A recent research review by Peterson, Wang and Mirzaei (2018) revealed that only five case studies (Canto, de Graff, & Jauregi, 2014; Chen, 2016; Deutschmann & Panichi, 2009; Kruk, 2015; Wigham & Chanier, 2013) have investigated integration of virtual worlds into regular courses. Therefore, the integration of virtual worlds into classrooms is worth investigating as well. Also, as mentioned above, GBLL studies may consider the inclusion of gaming communities to expand linguistic input in out-of-class environments. Pivec (2009), an advocate for games in classrooms, proposed that the meta-game surrounding the game is of more value than the game itself and suggest how games can be used effectively at all levels of education. Notable communities such as discussion fora and social networking sites include Twitter, Reddit or even comments under Youtube videos. Moreover, as examples of the meta-game contents, game streaming platforms such as Twitch and walkthrough/tutorial videos uploaded by players are also promising for improving listening skills. Therefore, games and gaming communities boast great potential for language learning and when properly integrated into language classrooms, students and teachers will surely benefit from the dynamic and innovative teaching tool of games.

114

References

Abrams, S., & Walsh, S. (2014). Gamified vocabulary: Online resources and enriched language learning. Journal of Adolescent & Adult Literacy, 58(1), 49-58. doi:10.1002/ jaal.315 ACTFL. (n.d.). Retrieved from https://www.actfl.org/publications/guidelines-and- manuals/actfl-proficiency-guidelines-2012/english/speaking Aghlara, L., & Tamjid, N. H. (2011). The effect of digital games on Iranian children's vocabulary retention in foreign language acquisition. Procedia-Social and Behavioral Sciences, 29, 552-560. Alyaz, Y., Spaniel-Weise, D., & Gursoy, E. (2017). A study on using serious games in teaching German as a foreign language. Journal of Education and Learning, 6(3), 250-264. Anderson, T. A., Reynolds, B. L., Yeh, X. P., & Huang, G. Z. (2008). Video games in the English as a foreign language classroom. Proceedings of the 2008 Second IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning, Banff, Canada, (pp. 188-192). IEEE. Ashraf, H., Motlagh, F. G., & Salami, M. (2014). The impact of online games on learning English vocabulary by Iranian (low-intermediate) EFL learners. Procedia-Social and Behavioral Sciences, 98(2014), 286-291. https://doi.org/10.1016/j.sbspro.2014.03.418 Bachen, C. M., Hernández-Ramos, P. F., & Raphael, C. (2012). Simulating REAL LIVES: Promoting global empathy and interest in learning through simulation games. Simulation & Gaming, 43(4), 437-460. Baek, Y. K. (2008). What hinders teachers in using computer and video games in the classroom? Exploring factors inhibiting the uptake of computer and video games. Cyberpsychology and Behavior. https://doi.org/10.1089/cpb.2008.0127 Bandicam. (n.d.). Retrieved from https://www.bandicam.com/ Belcher, D., & Hirvela, A. (Eds.). (2008). The oral-literate connection: Perspectives on L2 speaking, writing, and other media interactions. Ann Arbor: The University of Michigan Press. 115

Berns, A., Gonzalez-Pardo, A., & Camacho, D. (2013). Game-like language learning in 3-D virtual environments. Computers & Education, 60(1), 210-220. Bork, A. (1981). Learning with computers. Bedford, MA: Digital Equipment Corporation. Bredemeier, M. E., & Greenblat, C. S. (1981). The educational effectiveness of simulation games: A synthesis of findings. Simulation & Games, 12(3), 307-332. Bush, J. C. (2015). The impact of classroom games on the acquisition of second language grammar. Language in Focus, 1(2), 17-29. Calvo-Ferrer, J.R. (2017). Educational games as stand-alone learning tools and their motivational effect on L2 vocabulary acquisition and perceived learning gains. British Journal of Educational Technology, 48 (2), 264-278. Canto, S., de Graaff, R., & Jauregi, K. (2014). Collaborative tasks for negotiation of intercultural meaning in virtual worlds and video-web communication. In M. Gonzalez-Lloret, & L. Ortega (Eds.), Technology and tasks: Exploring technology- mediated TBLT (pp. 183-212). Washington, DC: Georgetown University Press. Carrier, M. (1991). Simulations in English language teaching: A cooperative approach. Simulation & Gaming, 22(2), 224-233. Chapelle, C. A. (2001). Computer applications in second language acquisition. Cambridge: Cambridge University Press. Charsky, D., & Ressler, W. (2011). “Games are made for fun”: Lessons on the effects of concept maps in the classroom use of computer games. Computers and Education, 56(3), 604–615. https://doi.org/10.1016/j.compedu.2010.10.001 Chen, J. C. C. C. (2016). The crossroads of English language learners, task-based instruction, and 3D multi-user virtual learning in Second Life. Computers & Education, 102, 152-171. Chik, A. (2011). Digital gaming and social networking: English teachers’ perceptions, attitudes and experiences. Pedagogies, 6(2), 154-166. https://doi.org/10.1080/1554480X.2011.554625 Coady, J. (1993). Research on ESL/EFL vocabulary acquisition: Putting it in context. In T. Huckin, M. Haynes, & J. Coady (Eds). Second language reading and vocabulary learning (pp.3-23). Norwood, NJ: Ablex Publishing.

116

Colby, R. S. (2017). Game-based Pedagogy in the Writing Classroom. Computers and Composition, 43(2017), 55-72. https://doi.org/10.1016/j.compcom.2016.11.002 Colby, R. S., & Colby, R. (2008). A Pedagogy of Play: Integrating Computer Games into the Writing Classroom. Computers and Composition, 25(3), 300-312. https://doi.org/10.1016/j.compcom.2008.04.005 Coleman, D. W. (2002). On foot in SIM CITY: Using SIM COPTER as the basis for an ESL writing assignment. Simulation & Gaming, 33(2), 217-230. Common European Framework of Reference for Languages: Learning, Teaching, Assessment. (n.d.). Retrieved from https://www.coe.int/en/web/common- european-framework-reference-languages Cornillie, F., Thorne, S. L., & Desmet, P. (2012). ReCALL special issue: Digital games for language learning: Challenges and opportunities. ReCALL, 24(3), 243–256. https://doi.org/10.1017/S0958344012000134 Crabtree, B., & Miller, W. (1999). Using codes and code manuals: A template organizing style of interpretation. In B. F. Crabtree & W. L. Miller (Eds.), Doing qualitative research (2nd ed.) (pp. 163-177). Thousand Oaks, CA: Sage Publications. Crook, C. (1996) Schools of the future. In Gill, T. (Ed.). Electronic Children: How children are responding to the information revolution (pp.75–88). London: National Children’s Bureau. Crookall, D. (2010). Serious games, debriefing, and simulation/gaming as a discipline. Simulation and Gaming, 41(6), 898–920. https://doi.org/10.1177/1046878110390784 Crookall, D. (2014). Engaging (in) Gameplay and (in) Debriefing. Simulation and Gaming, 45(4-5), 416–427. https://doi.org/10.1177/1046878114559879 DeHaan, J. W. (2005). Acquisition of Japanese as a foreign language through a baseball video game. Foreign Language Annals, 38(2), 278-282. Deutschmann, M., & Panichi, L. (2009). Talking into empty space? Signaling involvement in a virtual language classroom in Second Life. Language Awareness, 18(3-4), 310-328.

117

Dickey, M. D. (2007). Game design and learning: A conjectural analysis of how massively multiple online role-playing games (MMORPGs) foster intrinsic motivation. Educational Technology Research and Development, 55(3), 253-273. Driskell, J. E., & Dwyer, D. J. (1984). Microcomputer videogame based training. Educational technology, 24(2), 11-16. Ebrahimzadeh, M., & Sepideh, A. (2017). The effect of the digital video games on ELF students’ language learning motivation. Teaching English with Technology, 17(2), 87-112. Egenfeldt-Nielsen, S. (2005). Beyond edutainment: Exploring the educational potential of computer games. Unpublished Ph.D. thesis, IT-University of Copenhagen, Copenhagen. Fletcher, J. D., Tobias, S., & Wisher, R. A. (2007). Learning anytime, anywhere: Advanced distributed learning and the changing face of education. Educational Researcher, 36(2), 96-102. Fraser, M. W., & Galinsky, M. J. (2010). Steps in intervention research: Designing and developing social programs. Research on Social Work Practice, 20(5), 459-466. Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33(4), 441-467. Gee, J. P. (2007). What video games have to teach us about learning and literacy (2nd ed.). New York: St. Martin’s. Ghanbaran, S., & Ketabi, S. (2014). Multimedia games and vocabulary learning. Theory and Practice in Language Studies, 4(3), 489-496. https://doi.org/10.4304/tpls.4.3.489-496 Godwin-Jones, R. (2014). Games in language learning: Opportunities and challenges. Language Learning and Technology, 18(2), 9-19. Goodyear, P. & Tait, K. (1992). Learning with computer-based simulations: tutoring and student modelling requirements for an intelligent learning advisor. In M. Carretero, M. Pope, R. Simons, & J. Pozo (Eds.), Learning and instruction (Vol. 3, pp. 463– 481), Oxford: Pergamon Press.

118

Goodyear, P. (1992). The provision of tutorial support for learning with computer-based simulations. In E. de Corte, M. Linn, H. Mandl, & L. Verschaffel (Eds.), Computer- based learning environments and problem solving (pp. 391-409), Berlin: Springer Verlag. Gredler, M. E. (1996). 17. Educational games and simulations: A technology in search of a (research) paradigm. Technology, 39, 521-540. Healey, D. (1999) Classroom practice: Communicative skill-building tasks in CALL environments. In J. Egbert & E. Hanson-Smith (Eds.). CALL environments: Research, practice, and critical issues (pp. 116–136). Alexandria, VA: TESOL. Hirano, K. (1988). Research on T-unit measures in ESL. Journal of Child Language Acquisition and Development, 8(2), 14–15. Hwang, G. J., Hsu, T. C., Lai, C. L., & Hsueh, C. J. (2017). Interaction of problem-based gaming and learning anxiety in language students' English listening performance and progressive behavioral patterns. Computers & Education, 106, 26-42. James, K. K., & Mayer, R. E. (2019). Learning a second language by playing a game. Applied Cognitive Psychology, 33(4), 669–674. https://doi.org/10.1002/acp.3492 Jean, G., & Simard, D. (2011). Grammar teaching and learning in L2: Necessary, but boring?. Foreign Language Annals, 44(3), 467-494. Jones, C. and Fortescue, S. (1987) Using computers in the language classroom. New York: Longman, Inc. Kiili, K. (2005). Digital game-based learning: Towards an experiential gaming model. The Internet and Higher Education, 8(1), 13-24. Klauer, K. J. (1985). Framework for a theory of teaching. Teaching and Teacher Education, 1(1), 5-17. Kruk, M. (2015). Practicing the English present simple tense in Active Worlds. International Journal of Computer-Assisted Language Learning and Teaching, 5(4), 52-65. Leutner, D. (1993). Guided discovery learning with computer-based simulation games: Effects of adaptive and non-adaptive instructional support. Learning and Instruction, 3(2), 113-132.

119

Liu, T. Y., & Chu, Y. L. (2010). Using ubiquitous games in an English listening and speaking course: Impact on learning outcomes and motivation. Computers & Education, 55(2), 630-643. Mahmoud, A. A. A., & Tanni, Z. A. (2014). Using games to promote students' motivation towards learning English. Journal of Al-Quds Open University for Educational and Psychological Research and Studies, 336(2320), 1-21. Malone, T. W. (1981). What makes things fun to learn? A study of intrinsically motivating computer games. Pipeline, 6(2), 50. Maxwell, N. L., & Lopus, J. S. (1994). The Lake Wobegon effect in student self-reported data. The American Economic Review, 84(2), 201-205. Mayer, R. E. (2016). What Should Be the Role of Computer Games in Education? Policy Insights from the Behavioral and Brain Sciences, 3(1), 20–26. https://doi.org/10.1177/2372732215621311 Mayer, R. E. (2018). Computer Games in Education. Annual Review of Psychology. 70(1), 531-549. https://doi.org/10.1146/annurev-psych-010418-102744 McNeil, L. (2018). Introducing teachers to digital game–enhanced pedagogy: Successes and challenges in a graduate course. TESOL Journal, 9(3), 580–584. https://doi.org/10.1002/tesj.383 Miller, M., & Hegelheimer, V. (2006). The SIMs meet ESL Incorporating authentic computer simulation games into the language classroom. Interactive Technology and Smart Education, 3(4), 311-328. Molin, G. (2017). The role of the teacher in game-based learning: A review and outlook. In M. Ma, & A. Oikonomou (Eds.), Serious Games and Edutainment Applications: Volume II (pp. 649-674). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-51645-5_28 Nagahashi, T. L. (2007). Techniques for reducing foreign language anxiety: Results of a successful intervention study. Akita University Annual Report on Basic Education, 9, 53-60. Number of Gamers Worldwide 2021. (2019). Retrieved from https://www.statista.com/statistics/748044/number-video-gamers-world/

120

Outlier (n.d.). Retrieved from https://www.wikiwand.com/en/Outlier Papastergiou, M. (2009). Digital game-based learning in high school computer science education: Impact on educational effectiveness and student motivation. Computers & Education, 52(1), 1-12. Peterson, M. (2010). Computerized games and simulations in computer-assisted language learning: A meta-analysis of research. Simulation & Gaming, 41(1), 72-93. Peterson, M. (2012). Learner interaction in a massively multiplayer online role playing game (MMORPG): A sociocultural discourse analysis. ReCALL, 24(3), 361-380. Peterson, M. (2016). Virtual worlds and language learning. In F. Farr & L. Murray (Eds.), Routledge handbook of language learning and technology (pp.308-319). Oxford: Routledge. Peterson, M. (2017). Introduction. In M. Peterson (Ed.), Digital language learning and teaching: Critical and primary sources, Vol. IV. New developments in computer assisted language learning (pp.1-18). London: Bloomsbury. Peterson, M., Wang, Q., & Mirzaei, M.S. (2018). The use of network-based virtual worlds in second language education: A research review. In M. Kruk (Ed.), Assessing the effectiveness of virtual technologies in foreign and second language instruction (pp.1-17). Pennsylvania: IGI Global. Pivec, P. (2009). Game-based Learning or Game-based Teaching? Learning, (July), 1– 24. Purushotma, R. (2005). Commentary: you're not studying, you're just…. Language Learning & Technology, 9(1), 80. Qin, W., & Uccelli, P. (2016). Same language, different functions: A cross-genre analysis of Chinese EFL learners’ writing performance. Journal of Second Language Writing. https://doi.org/10.1016/j.jslw.2016.06.001 Ranalli, J. (2008). Learning English with The Sims: exploiting authentic computer simulation games for L2 learning. Computer Assisted Language Learning, 21(5), 441-455.

121

Randi, J., & Corno, L. (1997). Teachers as innovators. In B. Biddle, T. Good, & I. Goodson (Eds.) International handbook of teachers and teaching (pp. 1163-1221). Dordrecht: Springer. Reinders, H. & Wattana, S. (2015) Affect and willingness to communicate in digital game-based learning. ReCALL, 27(1), 38-57. Reitz, L., Sohny, A., & Lochmann, G. (2016). VR-based gamification of communication training and oral examination in a second language. International Journal of Game- Based Learning. 6(2), 46-61. https://doi.org/10.4018/IJGBL.2016040104 Sadler, R. (2012). Virtual worlds for language learning: From theory to practice. Bern: Peter Lang. Saito, K. (2012). Effects of instruction on L2 pronunciation development: A synthesis of 15 quasi-experimental intervention studies. TESOL Quarterly, 46(4), 842-854. Schreier, M. (2012). Qualitative content analysis in practice. Thousand Oaks, CA: Sage. Schwienhorst, K. (2002). The state of VR: A meta-analysis of virtual reality tools in second language acquisition. Computer Assisted Language Learning, 15(3), 221- 239. Sheridan, D. M., & Hart-Davidson, W. (2008). Just for fun: Writing and literacy learning as forms of play. Computers and Composition, 25(3), 323-340. Sitzmann, T. (2011). A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64(2), 489-528. Squire, K. (2005). Changing the game: What happens when video games enter the classroom?. Innovate: Journal of Online Education, 1(6), Article 5. Squire, K., & Barab, S. A. (2004). Replaying history: Learning world history through playing Civilization III. Bloomington, IN: Indiana University. Statista. (n.d.). Retrieved from www.statista.com Sylvén, L. K., & Sundqvist, P. (2012). Gaming as extramural English L2 learning and L2 proficiency among young learners. ReCALL, 24(3), 302-321. Test and Score Dara Summary for the TOEFL iBT Tests Test and Score Dara Summary for the TOEFL iBT Tests https://www.ets.org/toefl/score_data_summary

122

The Best Life Simulation Games of All Time. (n.d.). Retrieved from https://www.ranker.com/list/the-best-life-simulation-games/reference The Sims 4. (n.d.). Retrieved from https://www.ea.com/games/the-sims/the-sims-4 Thurman, R.A. (1993) Instructional simulation from a cognitive psychology viewpoint. Educational Technology Research and Development, 41(4), 75–89. Turnbull, M., & Dailey-O’Cain, J. (2009). Introduction. In M. Turnbull & J. Dailey- O’Cain (Eds.), First Language Use in Second and Foreign Language Learning (pp. 1–14). Bristol, Blue Ridge Summit: Multilingual Matters. https://doi.org/10.21832/9781847691972-003 Tüzün, H., Yilmaz-Soylu, M., Karakuş, T., Inal, Y., & Kizilkaya, G. (2009). The effects of computer games on primary school students’ achievement and motivation in geography learning. Computers and Education, 52(1), 68–77. https://doi.org/10.1016/j.compedu.2008.06.008 Van Eck, R. (2006). The effect of contextual pedagogical advisement and competition on middle-school students’ attitude toward mathematics using a computer-based simulation game. Journal of Computers in Mathematics and Science Teaching, 25(2), 165-195. Van Eck, R., & Dempsey, J. (2002). The effect of competition and contextualized advisement on the transfer of mathematics skills a computer-based instructional simulation game. Educational Technology Research and Development, 50(3), 23- 41. van Gog, T. (2013). Time on task. In Hatte, J., Anderman, E. M. (Eds.), International guide to student achievement (pp. 432-433). New York, NY: Routledge. Vann, R. (1982). Bridging the gap between oral and written communication in EFL. In B. Kroll and R. Vann (Eds.) Exploring speaking-writing relationships: Connections and contrasts (pp.165-194). Urbana, Illinois: National Council of Teachers of English. VocabularyProfile English. (n.d.). Retrieved from https://www.lextutor.ca/vp/eng/ Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Massachusetts: Harvard University Press.

123

Wager, W., & Gagné, R. M. (1988). Designing computer-aided instruction. In D. H. Jonassen (Ed.), Instructional designs for microcomputer courseware, (pp. 35-60). Hillsdale, NJ: Lawrence Erlbaum Associates. Wang, Q. & Dalsky, D. (2017). Students and teachers co-researching difficulties with vocabulary in academic writing: A case study of Exploratory Practice. In T. Stewart (Ed.). TESOL voices: Insider accounts of classroom life—Higher Education (pp. 123-129). Alexandria, VA: TESOL Press. Wang, Q. (2019). Classroom intervention for integrating simulation games into language classrooms: An exploratory study with the SIMs 4. CALL-EJ, 20(2), 101-128. Wang, Q. (in press). The role of classroom-situated game-based language learning in promoting students’ communicative competence. International Journal of Computer-Assisted Language Learning and Teaching (IJCALLT). Weissberg, B. (2000). Developmental relationships in the acquisition of English syntax: writing vs. speech. Learning and Instruction, 10(1), 37-53. Weissberg, R. (2006). Connecting speaking & writing in second language writing instruction. Ann Arbor: The University of Michigan Press. We're more than survey software. (n.d.). Retrieved from https://www.surveymonkey.com/ Wigham, C. R., & Chanier, T. (2013). A study of verbal and nonverbal communication in second life—the ARCHI21 experience. ReCALL, 25(1), 63-84. Williams, J. (2008). The speaking-writing connection in second language and academic literacy development. In D. Belcher & A. Hirvela (Eds.), The oral-literate connection: Perspectives on L2 speaking, writing, and other media interactions (pp. 10-25). Ann Arbor: The University of Michigan Press. Wouters, P., & Van Oostendorp, H. (2013). A meta-analytic review of the role of instructional support in game-based learning. Computers and Education, 60(1), 412-425. https://doi.org/10.1016/j.compedu.2012.07.018 Wu, C. J., Chen, G. D., & Huang, C. W. (2014). Using digital board games for genuine communication in EFL classrooms. Educational Technology Research and Development, 62(2), 209-226.

124

Yudintseva, A. (2015). Game-enhanced second language vocabulary acquisition strategies: A systematic review. Open Journal of Social Sciences, 3(10), 101.

125

Sessional themes and activities for both groups

Tests and activities Session Themes Gameplay quests Control Experimental group group Pre- Getting to know the Pre-questionnaire questionnaire game and deciding 1 Theme 1: and pre-test of game and pre-test household members in my vocabulary of game the game household vocabulary Creating characters, or Introducing 2 NA Sims household members Presenting family 3 Building houses NA members in detail Vocabulary test 1: Living in the simulated my household Vocabulary 4 Theme 2: world test 1 my life Presenting homes Storytelling what Making friends and 5 happened in the NA going on vacations family Vocabulary test 2: Theme 3: Finding a job and my life Vocabulary 6&7 my job getting promoted test 2 Sharing vocabulary Vocabulary test 3: Vocabulary 8 Opening a retail store my job test 3

126

Storytelling what Theme 4: happened in your my work business 9 Opening a restaurant Presenting stores NA Freely Exploring the Presenting 10 NA world restaurants Vocabulary test 4: Free Vocabulary my business gameplay Freely Exploring the test 4 11 world Post-questionnaire

and interview

127

The pre-questionnaire

1. Please write down your name and email address here. 2. Do you have a recent TOEFL, TOEIC, IELTS or EIKEN score? If so write it below. 3. Have you lived/travelled overseas? If so, write down where you have been and how long you stayed there. 4. How would you describe your daily English?

• I can barely communicate in English. • I can somehow understand and make myself understood in English. • I can communicate relatively smoothly in English with occasional mistakes. • I can communicate freely in English. • Others: 5. How would you describe your computer experience?

• I'm a very experienced computer user. • I often use computers. • I seldom use computers. • I almost never use computers. • Others: 6. How would you describe your gaming experience?

• I am a game addict. • I often play games in my spare time. • I seldom play games. • I almost never play games. • Others: 7. On what kind of platform do you prefer to play games?

• Mobile phones • PCs • Consoles (PS4, Xbox, etc.) 128

• Handheld consoles (PSV, Switch, etc.) • Others: 8. Have you played the Sims before? If so write down how long you have played it in the box. 9. Do you think that the game-based class sessions will help improve your vocabulary and communication skills? Please explain why in the comment box.

• Yes. • No. • I'm not sure. Why do you think so? 10. Write down any questions or any comments you have for this course.

129

The post-questionnaire

1. To what extent do you think you are happy with the class? Please briefly state your reasons as well.

Not happy at all A bit unhappy Not happy nor unhappy Somewhat happy Very happy

Others:

2. This is a communication class. To what extent do you think this class has improved your English communication skills? And in what ways?

Not at all A little bit Somewhat Very much

Others:

3. To what extent do you think this class has enriched your vocabulary? How did you learn these words?

Not at all A little bit Somewhat By a large margin

Others:

4. Are there any other skills you think this class has helped you improve? And why? 5. What do you think of this class when compared with other traditional textbook-based English classes? You can state both advantages and disadvantages. 6. Do you have any suggestions to this class? Anything is okay. 7. As we are using this data for research, please write down a nickname for yourself in case this research is going to be published. Thank you very much!

130

List of words in the vocabulary pre-test

achieve habit advanced historic advocate humble aim insult amusement journey anticipate manipulate apology material automate minimal band offer bill overlap broadcast predict charm programming cheer property conceive recover confidence regular convenience reject curious relieve decorative stress design sweat domestic tax dull trigger employ unique extract various flexible whisper fundamental wicked

131

Requirements from the English entrance exam

Figure 30. Entrance exam guidelines for English 132

Sessional activities

Table 24. Sessional activities

Session Theme Gameplay Quests Tests and Activities Get to know the game Pre-questionnaire; and decide household Instruction on how to members in the game; 1: My household create households and 1-2 Create characters, or construct houses and on

Sims; key vocabulary;

Build houses. Autonomous gameplay Live in the simulated Presentations on family world; members and houses;

2: My life Upgrade one of the Instruction on daily life 3-4 life skills to level 3; activities and key

Make friends and go vocabulary; on vacations and Autonomous gameplay. dates. Presentations on daily life and social events; Find a profession and 5-6 3: My job get promoted to the Instruction on careers highest rank. and key vocabulary;

Autonomous gameplay. Open a retail store, a Presentations on restaurant or a vet 7-8 4: My business workplace events; clinic;

133

Upgrade the business Instructions on opening to level 3. up businesses and key vocabulary;

Autonomous gameplay. Presentations on Experience at least businesses; one season featured 5: seasons and in the game; Instruction on seasons 9-10 holidays and holidays pack and Complete holiday on key vocabulary. quests in a season. Autonomous gameplay. Freely Explore the Post-questionnaire and 11 Free gameplay game. interview.

134

Evaluation rubric for presentation recordings

Interaction Score High audience-awareness; frequent interaction with peers or the 4-5 teacher; natural reaction to questions and comments. Switching to notes at some point; some interaction with peers or the teacher; able to react to questions or comments with the teacher’s 2.5-3.5 help. Reciting or reading notes most of the time; limited interaction with peers or the teacher; fragmented utterances as answers to questions or 1-2 comments. Reciting or reading notes all the time; no interaction with peers or the 0-0.5 teacher; unable to react to questions or comments. Fluency Natural pauses; excellent intelligibility; complete and grammatically 4-5 correct sentences with a diverse range of sentence structures. Occasional long pauses; good intelligibility; complete sentences with occasional grammatical mistakes; relatively diverse sentence 2.5-3.5 structures Frequent long pauses; intelligible pronunciation; complete sentences 1-2 with frequent grammatical mistakes; limited sentence structures. Overwhelming long pauses and filler words; poor intelligibility; fragmented sentences with frequent grammatical mistakes; 0-0.5 monotonously simple sentence structures. Content Interesting and unique events or made-up stories; Well-sequenced and logical storytelling; proper vocabulary and expressions to depict 4-5 stories. Attractive events or made-up stories; Natural flow of stories; occasional misuse of words or expressions that do not affect 2.5-3.5 understanding. 135

Normal events or made-up stories; Abrupt transitions between events in stories; misuse of words or expressions that affect understanding 1-2 from time to time. Trivial events or made-up stories; Unclear event sequence; frequent 0-0.5 misuse of words or expressions that greatly affect understanding.

136

The pre-questionnaire

Pre-questionnaire

Basic Information

1. Please write down your name, student ID number and email address here.

English Background

2. Do you have a recent TOEFL, TOEIC, IELTS or EIKEN score? If so, write it below. 3. Have you lived/travelled overseas? If so, write down where you have been and how long you have stayed there. 4. How would you describe your daily communication in English? • I can barely communicate in English. • I can somehow understand and make myself understood in English. • I can communicate relatively smoothly in English with occasional mistakes. • I can communicate freely in English. • Other (please specify)

Gaming Experience

5. How would you describe your computer experience? • I'm a very experienced computer user. • I often use computers. • I seldom use computers. • I almost never use computers. • Other (please specify) 6. How would you describe your gaming experience? • I am a game addict. • I often play games in my spare time. • I seldom play games. • I almost never play games.

137

• Other (please specify) 7. What kind of platform do you prefer to play games? (multiple choices) § Mobile phones § PCs § Consoles (PS4, Xbox, etc.) § Handheld consoles (PSV, Switch, etc.) § Other (please specify) 8. Have you played the Sims 4 before? If so, write how long you have played it in the box. • Yes • No

Attitudes toward GBLL

9. Do you think game-based language learning in the classroom will help improve your vocabulary and communication skills? Please explain why in the comment box. • Yes. • No. • I'm not sure. Why:

10. Write down any questions or comments you have for this course.

138

The post-questionnaire

Post-questionnaire

Dear students, as our course is coming to an end, we would like you to complete a short questionnaire about your learning experience in this non-traditional English classroom. We have been upgrading the course year-on-year with valuable feedback from students and we wish to continue exploring English courses that students really enjoy. The questionnaire is not linked in any way to your credit or grades. Please feel free to express your opinions. Thank you!

1. Please write your name and a pseudonym you wished to be called by in an academic paper to be written based on our course. 2. How would you rate the game SIMs4 in the following aspects?

Strongly disagree Disagree Neither Agree Strongly agree

• It is interesting Comments if any

• It gives me higher learning motivation Comments if any

• I learnt a lot through the game Comments if any

• I learnt very fast through the game Comments if any

• It is difficult in terms of gaming skills Comments if any

• It is difficult in terms of game language Comments if any

139

3. How would you rate the effectiveness of autonomous gameplay (playing the game by yourself) in improving your English communication skills from the following three aspects?

1 2 3 4 5

• Interaction Comments if any

• Fluency Comments if any

• Content Comments if any

4. How would you rate the effectiveness of teacher-student interaction in improving your English communication skills from the following three aspects?

1 2 3 4 5

• Interaction Comments if any

• Fluency Comments if any

• Content Comments if any

5. How would you rate the effectiveness of presentations in improving your English communication skills from the following three aspects?

1 2 3 4 5

• Interaction Comments if any 140

• Fluency Comments if any

• Content Comments if any

6. What do you think of this class when compared with pure gameplay? The following are some hints. § The game is less interesting when it is played in the classroom. § I think I learn more in this class than just playing the game outside the classroom § Your opinion: 7. What do you think of this class when compared with other traditional textbook-based English classes? You can state both advantages and disadvantages. The following are some hints. § This is not a teacher-centered class § Learning through gameplay is much more interesting § I can communicate freely my ideas in the class § This class is too easy, not suited to more academic goals § Learning through gameplay is slow, not very efficient § Your opinion: 8. Are there any other skills you think this course has helped you improve? And why? § Listening § Writing § Reading § Others and reasons (please specify) 9. What is your overall rating of this course from 1 to 5?

Comments (if any)

10. Do you have any suggestions for this course or anything to say?

141

The pre-study survey

Pre-study questionnaire (original in Chinese)

1. Please write down your most recent score from a nationally or internationally recognized English exam（CET4, 6; TOEIC, TOEFL, IELTS, etc.; University Entrance Exam with province and year） 2. What writing genre have you been mostly trained on up to now? • Narrative, recounting past events • Argumentative, expressing your own opinions and arguments • Expositions, explaining how something works or procedures • Correspondence, writing letters • Description, describing current events or pictures • Others: 3. Will you search for background information/source texts before writing? • No, I write with only prompts that come with writing tasks • Yes, I search for relevant information in Chinese as supplement • Yes, I search for relevant information in English as supplement • Others: 4. Does your English teacher at university give clear instructions on writing? Briefly state your reasons. • No • Yes 5. Does your English teacher at university give you feedback on writing assignments? • No, I receive no feedback from the teacher • No, but he/she hands out exemplary samples to us for reference • Yes, he/she chooses writing assignments from several students for review in class • Yes, he/she gives individual feedback on each student's writing assignment • Others 6. Which of the following areas do you most prefer to develop writing skills?

142

• Daily topics/communication • Business writing • Academic writing • Exam-oriented writing • Others 7. Will you be interested if your teacher integrates a life-simulation game into your English classroom, offers instructions on vocabulary, asks you to complete relevant writing tasks and have some students present their stories in class? Briefly state your reasons. • Very interested • Interested • I don't know • Not interested • Not at all interested

143

List of student writing samples

List of student writing samples and their themes

Sample No. Title Theme Introducing two family 1 Household introduction members, Kristi and Jaden Describing a normal day in 2 A routine day the family Introducing the house and 3 Kristi’s and Jaden’s rooms separate rooms for family members Two family members 4 A funny night spending a night at a local bar Kristi and her friend 5 A special gym exercising at a local gym Two family members going 6 An embarrassing trip on vacation in the woods Jaden took Kristi out on a 7 A bad date date which turned out rather awkward Jaden was tragically frozen 8 Jaden’s death to death in front of the house The soul of Jaden came back 9 Surprise to haunt the house Kristi started a vet clinic in 10 Kristi’s vet clinic the neighborhood and treated animal patients 11 A day wasted Kristi slept all day at work 12 Christmas day Christmas celebration

144

Kristi married the spirit of 13 Kristi and Jaden’s wedding Jaden and the newly-weds held a wedding reception The couple built a new Kristi and Jaden’s new 14 house and adopted pets and house kids

145

The vocabulary post-test

decorative clinic roasting lighting log wear bathroom diagnosis flooring lamp needs office salad stall treadmill adult alien amused bedding bucket ceiling client cure customer decoration embarrassingly emotions focused fridge household innovative knot mature mop moustache moves neighborhood obsessed offer patients plate professional rented salary shifts sibling sink skirt slapping sportswear studio uniform vet

146

Writing samples with revision: #5 and #14

Revised Sample #5

147

Revised sample #14

148

The post-interview

List of questions in post-interview

1. Generally speaking, what kind of influence do you think the game had on your writing? 2. Do you think teacher feedback/revision on your writing samples effective? If so, in what aspects specifically? 3. Did you encounter any writing difficulty in this course? 4. What did you get from the course regarding narrative writing? 5. What do you think of this GBW class as compared with traditional writing classes? 6. What challenges do you foresee if this GBW class is to be integrated into the curriculum of Chinese universities? 7. Regarding the vocabulary post-test, it turns out you forgot around 40% of the words you learnt from teacher feedback. However, you forgot only 2 words you had previously been able to use in your writing, well, with wrong grammar though. Can you explain why this is so?

149

The 10th samples (original and revised)

The 10th samples

Original sample

Kristi’s Pet House

Kristi is a rich lady, and she has enough money to buy a store. She bought a pet house, because she loves pet.

Every day, she inspected the house and looked around. Then, she communicated with client and took care of their pets. A moment later, Kristi felt hungry and took a break with eating hamburger. Just then, a lady entered to the house and asked Kristi some question about pets. Kristi began to treat a white cat due to a lot of pet patient. She walked to examination table with the cat, and it stood on the table. The cat was looked terrible, so Kristi comforted it. Its owner looked at it in a sad mood. The owner told Kristi terrible story about its illness, and it lost in a smelly river. Although, Kristi didn’t know its illness, she did this operation for the cat. After the cat was doing this operation, it felt scared and cried. She did it for a long time, but the cat was still ill.

Kristi was sorry and continued to walk back the table to examine the cat. Kristi found its illness by observing its fur. Then she looked at its mouth. It took her a long time, so Kristi felt tired. When Kristi treated the cat, its owner was waiting it, so he lost his patience. Finally, Kristi used medicine to treat the cat. The cat got recovery and its owner was delighted. He paid her some money and return home in a good mood.

The house was still opened, but Kristi came back and didn’t exchange her clothes because she needed to sleep.

150

Revised sample

Kristi’s Vet Clinic

Kristi is a rich lady, and she has enough money to buy a store. As she loves pets, she decided to buy a vet clinic.

Every day, she starts her day at the vet clinic with inspection around the house. Today she has several customers. she communicated with clients and took care of their pets. A moment later, Kristi felt hungry and took a break to eat a hamburger. Just then, a lady entered her office and asked Kristi some questions about pets. Kristi answered her questions but soon began to treat a white cat as there were a lot of animal patients. She walked to an examination table with the cat, and the cat stood on the table. It looked terrible, so Kristi comforted it. Its owner looked at it in a sad mood. The owner told Kristi a terrible story about how his cat got ill: it fell down into a smelly river. After several diagnosis sessions, Kristi failed to figure out the illness. However, she decided to operate on the cat anyway, so she took the cat to the operation table. During the operation, the cat felt scared and cried. The operation lasted for a long time, but when it was done, the cat was still not cured.

Kristi was sorry and took the cat back the table for examination. Kristi tried to observe its fur, and then she looked at its mouth. The diagnosis took her a long time, so Kristi felt tired. When Kristi was examining the cat, its owner was waiting beside them, and he started to lose his patience. Finally, Kristi used some medicine to treat the cat. The cat somehow recovered and its owner was delighted. He paid her some money and return home with the cat in a good mood.

The clinic was still open, but Kristi decided to come home for a rest. She was so tired that she didn’t even change her vet uniform. What a day! She surely needs to improve her vet skills.

151

Pictures used in the pre-test and post-test

152

Evaluation rubric for presentations

Data: Rater:

Interaction Score Remarks High audience-awareness; frequent interaction with peers or the teacher; 9-10 natural reaction to questions and comments. Switching attention to notes at some point; some interaction with peers or the teacher; 6-8 able to react to questions or comments with the teacher’s help. Reciting or reading notes most of the time; limited interaction with peers or the 3-5 teacher; fragmented utterances as answers to questions or comments. Reciting or reading notes all the time; no interaction with peers or the teacher; 1-2 unable to react to questions or comments. Fluency Natural pauses; excellent intelligibility; complete and grammatically correct 9-10 sentences with a diverse range of sentence structures. Occasional long pauses; good intelligibility; complete sentences with occasional 6-8 grammatical mistakes; relatively diverse sentence structures Frequent long pauses; intelligible 3-5 pronunciation; complete sentences with

153

frequent grammatical mistakes; limited sentence structures. Overwhelming long pauses and filler words; poor intelligibility; fragmented sentences with frequent grammatical 1-2 mistakes; monotonously simple sentence structures. Content Interesting and unique events or made-up stories; Well-sequenced and logical 9-10 storytelling; proper vocabulary and expressions to depict stories. Attractive events or made-up stories; Natural flow of stories; occasional misuse 6-8 of words or expressions that do not affect understanding. Normal events or made-up stories; Abrupt transitions between events in stories; 3-5 misuse of words or expressions that affect understanding from time to time. Trivial events or made-up stories; Unclear event sequence; frequent misuse of words 1-2 or expressions that greatly affect understanding.

154

Notes:

i The Sims 4 and screenshots of it are licensed property of Electronic Arts, Inc.

155