Challenging Learners in Their Individual Zone of Proximal Development Using Pedagogic Developmental Benchmarks of Syntactic Complexity Xiaobin Chen and Detmar Meurers LEAD Graduate School and Research Network Department of Linguistics Eberhard Karls Universitat¨ Tubingen,¨ Germany fxiaobin.chen,[email protected] Abstract and identify reading material individually chal- lenging learners, essentially instantiating the next This paper introduces an Intelligent Com- stage of acquisition as captured by Krashen’s con- puter Assisted Language Learning system cept of i+1 (Krashen, 1981) or relatedly, but em- designed to provide reading input for lan- phasizing the social perspective, Vygotsky’s Zone guage learners based on the syntactic com- of Proximal Development (ZPD; Vygotsky, 1976). plexity of their language production. The In terms of structure of the paper, we first locate system analyzes the linguistic complexity our approach in terms of the Complexity, Accu- of texts produced by the user and of texts racy, and Fluency (CAF) framework in SLA re- in a pedagogic target language corpus to search. Then we review approaches adopted by identify texts that are well-suited to foster earlier studies in developmental complexity re- acquisition. These texts provide develop- search, including problems they pose for a peda- mental benchmarks offering an individu- gogical approach aimed at offering developmental ally tailored language challenge, making benchmarks. We propose and justify a solution, ideas such as Krashen’s i+1 or Vygotsky’s before presenting the architecture and functional- Zone of Proximal Development concrete ity of the SyB system. and empirically explorable in terms of a broad range of complexity measures in all 2 Development of Syntactic Complexity dimensions of linguistic modeling. The three-part model of development distinguish- ing Complexity, Accuracy, and Fluency has gained 1 Introduction significant popularity among SLA researchers The analysis of linguistic complexity is a promi- (Wolfe-Quintero et al., 1998; Skehan, 2009; nent endeavor in Second Language Acquisition Housen et al., 2009; Bulte´ and Housen, 2012) (SLA) where Natural Language Processing (NLP) since it was first delineated by Skehan (1989). It technologies are increasingly applied in a way provides SLA researchers with a systematic and broadening the empirical foundation. Automatic quantitative approach to development. Among complexity analysis tools such as CohMetrix (Mc- the CAF triplet, complexity arguably is the most Namara et al., 2014), the L2 Syntactic Complexity researched and most “complex” due to its poly- Analyzer (Lu, 2010), and the Common Text Anal- semous and multidimensional nature (Bulte´ and ysis Platform (Chen and Meurers, 2016) support Housen, 2012; Vyatkina et al., 2015). Complex- studies analyzing interlanguage development (Lu, ity in the SLA literature has been used to refer to 2011; Lu and Ai, 2015; Mazgutova and Kormos, task, cognitive, or linguistic complexity (Housen 2015), performance evaluation (Yang et al., 2015; et al., 2009). In the present paper, we investigate Taguchi et al., 2013), and readability assessment complexity from a linguistic perspective, where it (Vajjala and Meurers, 2012; Nelson et al., 2012). is concisely characterized by Ellis (2003) as “the In this paper, we introduce a new system extent to which language produced in performing called Syntactic Benchmark (SyB) that utilizes a task is elaborate and varied”. While the lin- NLP to create syntactic complexity benchmarks guistic complexity construct consists of a range of sub-constructs at all levels of linguistic modeling, This work is licensed under a Creative Commons Attribu- tion 4.0 International License. License details: http:// such as lexical, morphological, syntactic, seman- creativecommons.org/licenses/by/4.0 tic, pragmatic and discourse (Lu, 2010; Lu, 2011; Lu and Ai, 2015; Ortega, 2015; Mazgutova and ity from a developmental perspective is of far- Kormos, 2015; Jarvis, 2013; Kyle and Crossley, reaching relevance and applicability. 2015), the focus in this paper is on syntactic com- plexity. 2.1 Development of Syntactic Complexity in In line with Ellis’s (2003) definition of linguis- Learner Corpora tic complexity, Ortega (2003) characterized syn- A number of longitudinal and cross-sectional stud- tactic complexity as the range of syntactic struc- ies have been conducted to investigate the rela- tures and the elaborateness or degree of sophistica- tionship between syntactic complexity and learner tion of those structures in the language production, proficiency, aimed at finding (i) the most informa- which we adopt as the operational definition in this tive complexity measures across proficiency lev- paper. The uses of syntactic complexity analysis in els (Lu, 2011; Ferris, 1994; Ishikawa, 1995), (ii) SLA research include (i) gauging proficiency, (ii) the patterns of development for different syntac- assessing production quality, and (iii) benchmark- tic measures (Bardovi-Harlig and Bofman, 1989; ing development (Ortega, 2012; Lu and Ai, 2015). Henry, 1996; Larsen-Freeman, 1978; Lu, 2011), The development of syntactic complexity in or (iii) discovering a developmental trajectory of language produced by learners is closely related syntactic complexity from the learner production to the learner’s proficiency development. While (Ortega, 2000; Ortega, 2003; Vyatkina, 2013; Vy- the goal of language acquisition is not as such atkina et al., 2015). to produce complex language, advanced learners With a few exceptions (Vyatkina, 2013; Tono, usually demonstrate the ability to understand and 2004), one thing these studies have in common produce more complex language. With increasing is that they analyze the syntactic complexity de- proficiency, the learners are expanding their syn- velopment of learners based on their production. tactic repertoire and capacity to use a wider range This seems natural since it investigates complex- of linguistic resources offered by the given gram- ity development by analyzing the production of mar (Ortega, 2015), thus producing “progressively the developing entity, i.e., the learners. In prin- more elaborate language” and “greater variety of ciple, a longitudinal learner corpus with a contin- syntactic patterning”, constituting development in uous record of productions from individual learn- syntactic complexity (Foster and Skehan, 1996). ers over time would seem to enable us to deter- As a result, syntactic complexity is often used to mine the developmental trajectory and linguistic determine proficiency or assess performance in the complexity benchmarks. However, this approach target language (Larsen-Freeman, 1978; Ortega, encounters some challenges that make it subopti- 2003; Ortega, 2012; Vyatkina et al., 2015; Wolfe- mal for determining developmental benchmarks in Quintero et al., 1998; Lu, 2011; Taguchi et al., practice. 2013; Yang et al., 2015; Sotillo, 2000). First, the approach is dependent on learner cor- Besides the practical side of performance as- pora varying significantly on a number of param- sessment and placement, in SLA research the de- eters such as the learners’ background, the tasks velopmental perspective is considered to be “at eliciting the production, and the instructional set- the core of the phenomenon of L2 syntactic com- tings, etc. Significant effects of such factors on the plexity” (Ortega, 2015). However, it is also the syntactic complexity of learner writing have been least addressed and understood phenomenon of identified in a number of studies (Ellis and Yuan, syntactic complexity in SLA research (Vyatkina 2004; Lu, 2011; Ortega, 2003; Sotillo, 2000; Way et al., 2015; Ortega, 2012). Understanding the et al., 2000; Yang et al., 2015; Alexopoulou et al., development of syntactic complexity would en- 2017). Consequently, the developmental patterns able SLA researchers to determine trajectories of or benchmarks constructed from different learner the learners’ development and set benchmarks for corpora, elicited using different tasks, etc. are certain time points or across a given time span. likely to vary or even contradict each other. For ex- On the practical side, such work could help lan- ample, the correlation between subordination fre- guage teachers select or design appropriate learn- quency and proficiency level have been found to ing materials, and it can provide a reference frame be positive (Aarts and Granger, 1998; Granger and for testing the effectiveness of instructional inter- Rayson, 1998; Grant and Ginther, 2000), negative ventions. Hence researching syntactic complex- (Lu, 2011; Reid, 1992), or uncorrelated (Ferris, 1994; Kormos, 2011). It is difficult to build on of limited practical use for proficiency placement such conflicting findings in practice. or performance assessment. Naturally this does not mean that research into developmental patterns Second, the NLP tools used for the automatic based on learner corpora is not important or rel- complexity analysis do not work equally well evant for SLA. On the contrary, the dynamic and when applied to the language produced by learners adaptive nature of language acquisition means that at varied proficiency levels. Complexity analysis it is challenging and interesting to approach lan- is currently performed using tools developed for guage development in a way accounting for in- different analysis needs (McNamara et al., 2014; dividual differences (Larsen-Freeman, 2006; Ver- Lu, 2010; Kyle and Crossley, 2015; Chen and spoor et