Automatic Analysis of Linguistic Complexity and Its Application in Language Learning Research Xiaobin Chen Advisors Prof. Dr. Detmar Meurers Prof. Dr. Katharina Scheiter A dissertation presented for the degree of Doctor of Philosophy LEAD Graduate School & Research Network Seminar f¨urSprachwissenschaft Eberhard Karls Universit¨atT¨ubingen Germany December 27, 2018 Automatic Analysis of Linguistic Complexity and Its Application in Language Learning Research Dissertation zur Erlangung des akademischen Grades Doktor der Philosophie in der Philosophischen Fakult¨at der Eberhard Karls Universit¨atT¨ubingen vorgelegt von Xiaobin Chen aus Guangdong, China 2018 Gedrukt mit Genehmigung der Philosophischen Fakult¨at der Eberhard Karls Universit¨atT¨ubingen Dekan(in): Prof. Dr. J¨urgenLeonhardt Hauptberichterstatter(in): Prof. Dr. Detmar Meurers Mitberichterstatter(in): Prof. Dr. Katharina Scheiter Tag der m¨undlichen Pr¨ufung:14.12.2018 T¨ubingen To my wife Chang Wei, and our children Danning and Yuanning. Declarations Erkl¨arungbzgl. anderer Promotionsverfahren Hiermit erkl¨areich, dass ich mich keinen anderen Promotionsverfahren oder ent- sprechenden Pr¨ufungsverfahren unterzogen habe. Erkli¨arungbzgl. Verwendung von Hilfsmitteln Ich erkl¨arehiermit, dass ich die zur Promotion eingereichte Arbeit mit dem Ti- tel: Automatic Analysis of Linguistic Complexity and Its Application in Language " Learning Research\ selbst¨andigverfasst, nur die angegebenen Quellen und Hilfsmit- tel benutzt und w¨ortlich oder inhaltlich ¨ubernommene Stellen als solche gekennze- ichnet habe. Ich versichere an Eides statt, dass diese Angaben wahr sind und dass ich nichts verschwiegen habe. Mir ist bekannt, dass die falsche Abgabe einer Ver- sicherung an Eides statt mit Freiheitsstrafe bis zu drei Jahren oder mit Geldstrafe bestraft wird. Erkl¨arungbzgl. kommerzieller Unterst¨utzung Hiermit erkl¨areich, dass die Gelegenheit zum vorliegenden Promotionsverfahren nicht kommerziell vermittelt wurde. Ich habe die Dissertation ohne kommerzielle Hilfe von Dritten verfasst und keine Organisation eingeschalten, die gegen Entgelt Betreuer f¨urdie Anfertigung von Dissertationen gesucht hat oder die f¨urmich die mir obliegenden Pr¨ufungsleistungenganz oder teilweise erledigt hat. Hiermit best¨atigeich, dass es mir die Rechtsfolge der Inanspruchnahme eines gewerb- lichen Promotionsvermittlers und die Rechtsfolge bei Unwahrhaftigkeiten in dieser Erkl¨arung(Ausschluss der Annahme als Doktorand, Ausschluss der Zulassung zum Promotionsverfahren, Abbruch des Promotionsverfahrens und R¨ucknahme des er- langten Grades wegen T¨auschung gem¨aß x 21) bekannt ist. Erkl¨arungbzgl. Verurteilungen Hiermit erkl¨areich wahrheitsgem¨aßund vollst¨anding,dass keine wissenschaftsbe- zogene strafrechtliche Verurteilungen, Disziplinarmaßnahmen und anh¨angigeDiszi- plinarverfahren gegen mich laufen. Xiaobin Chen December 27, 2018, T¨ubingen i ii Acknowledgments This dissertation would not have been possible without the considerable support and careful supervision from my supervisor Prof. Detmar Meurers, who granted me maximum academic freedom to pursue the interesting topics in this research and provided me with numerous invaluable advice on various issues along the way. His great passion and serious attitude towards scientific inquiry have been and will always be an aspiration of my research career. A million thanks to Detmar. I owe my deepest gratitude to my family who have always been supportive to my work. I would like to thank my parents, Chen Huihua and Cai Huihua, for bearing with the deprivation of the happiness they had been enjoying living with their favorite grand-daughter before I decided to move far away to Germany to do my PhD with the family. I am heartily thankful to my wife Chang Wei who gave up a decent job in a Chinese university to become a full-time housewife so as to support my research. Because of her great sacrifice and goodwill, I am able to come to the warmth of a well managed home with two happy children and a delicious meal everyday after a hard day's work. It is really the greatest comfort in the whole world. My gratitude also goes to my parents-in-law Chang Jingmin and Zhang Yanqing for every support they provided us with. Life would have been harder without their generosity and trust. I am also indebted to many of my advisors, colleagues, and friends, in particular Patrick Rebuschat, Katharina Scheiter, Michael Grosz, Bj¨ornRudzewitz, Maria Chinkina, Sim´onRuiz, Sabrina Galasso, and Zarah Weiß among a lot others. I am forever grateful to the interesting discussions with them on issues related or unrelated to research at the lunch breaks and on the other occasions. The insights I gained from these wonderful colleagues are what makes me develop into a competent researcher and critical thinker. Last but by no means the least, I would like to thank the LEAD Graduate School & Research Network of the University of T¨ubingen,who funded my PhD and pro- vided me with a wonderful interdisciplinary research environment, as well as gener- iii ous fundings for conferences and intramural projects to make my research possible. My special thanks go to Mareike Bierlich, Sophie Freitag, Katharina Lichtenberger, and the other members of the Scientific Coordination team and the steering board of LEAD for their support and management of the projects leading to this dissertation. iv Funding This research was funded by the LEAD Graduate School & Research Network [GSC1028], a project of the Excellence Initiative of the German federal and state governments. v vi Abstract The construct of complexity, together with accuracy and fluency have become the central foci of language learning research in recent years. This dissertation focuses on complexity, a multidimensional construct that has its own working mechanism, cognitive and psycholinguistic processes, and developmental dynamics. Six studies revolving around complexity, including its conceptualization, automatic measure- ment, and application in language acquisition research are reported. The basis of these studies is the automatic multidimensional analysis of linguistic complexity, which was implemented into a Web platform called Common Text Anal- ysis Platform by making use of state-of-the-art Natural Language Processing (NLP) technologies . The system provides a rich set of complexity measures that are easily accessible by normal users and supports collaborative development of complexity feature extractors. An application study zooming into characterizing the text-level readability with the word-level feature of lexical frequency is reported next. It was found that the lexical complexity measure of word frequency was highly predictive of text read- ability. Another application study focuses on investigating the developmental in- terrelationship between complexity and accuracy, an issue that conflicting theories and research results have been reported. Our findings support the simultaneous development account. The other few studies are about applying automatic complexity analysis to promote language development, which involves analyzing both learning input and learner production, as well as linking the two spaces. We first proposed and val- idated the approach to link input and production with complexity feature vector distances. Then the ICALL system SyB implementing the approach was developed and demonstrated. An effective test of the system was conducted with a random- ized control experiment that tested the effects of different levels of input challenge on L2 development. Results of the experiment supported the comprehensible input hypothesis in Second Language Acquisition (SLA) and provided an automatizable operationalization of the theory. vii The series of studies in this dissertation demonstrates how language learning research can benefit from NLP technologies. On the other hand, it also demonstrates how these technologies can be applied to build practical language learning systems based on solid theoretical and research foundations in SLA. viii Publications Part of the contents of this dissertation is based on the following publications and submitted manuscripts. • Chen, X., Meurers, D., and Rebuschat, P. (Submitted-a). Investigating Krashen's i + 1 : An experimental ICALL study on the development of L2 complexity. • Chen, X., Weiß, Z., and Meurers, D. (Submitted-b). Is there a developmental trade-off between complexity and accuracy in L1 and L2 acquisition?. • Chen, X. and Meurers, D. (2018a). Linking text readability and learner profi- ciency using linguistic complexity feature vector distance. Computer Assisted Language Learning, In press. • Chen, X. and Meurers, D. (2018b). Word frequency and readability: Predicting the text-level readability with a lexical-level attribute. Journal of Research in Reading, 41(3):486{510. • Chen, X. and Meurers, D. (2017a). Challenging learners in their individual Zone of Proximal Development using pedagogic developmental benchmarks of syntactic complexity. In Proceedings of the Joint 6th Workshop on NLP for Computer Assisted Language Learning and 2nd Workshop on NLP for Re- search on Language Acquisition at NoDaLiDa, pages 8{17, Gothenburg, Swe- den, 22nd May. Link¨opingUniversity Electronic Press, Link¨opingsuniversitet. • Chen, X. and Meurers, D. (2016b). CTAP: A web-based tool supporting au- tomatic complexity analysis. In Proceedings of the Workshop on Computa- tional Linguistics for Linguistic Complexity at COLING, pages 113{119, Os- aka, Japan, 11th December. The International Committee on