Thresholds, Text Coverage, Vocabulary Size, and Reading Comprehension in Applied Linguistics
Total Page:16
File Type:pdf, Size:1020Kb
Thresholds, Text Coverage, Vocabulary Size, and Reading Comprehension in Applied Linguistics by Myq Larson A thesis submitted to the Victoria University of Wellington in fulfilment of the requirements for the degree of Doctor of Philosophy in Applied Linguistics. Victoria University of Wellington 2017 © 2017 — Myq Larson All rights reserved. Abstract The inextricable link between vocabulary knowledge and reading comprehension isin- controvertible. However, questions remain regarding the nature of the interaction. One question which remains unresolved is whether there is an optimum text coverage, or ratio of known to unknown words in a text, such that any deleterious effects of the unknown words on reading comprehension are minimised. A related question is what vocabulary size would a reader need to have in order to achieve the optimum text cov- erage for a given text or class of texts. This thesis addresses these questions in three ways. First, a replication and expan- sion of a key study (Hu & Nation, 2000)1 was performed. In that study, 98% text coverage was found to be optimal for adequate reading comprehension of short fiction texts when reading for pleasure. To replicate that study, equivalent measures of reading compre- hension were collected from a more homogeneous group of participants at a university in northern Thailand (n = 138), under stricter conditions and random assignment to one of three text coverage conditions, to verify the generalisability of the results. The ori- ginal study was also expanded by measuring reader characteristics thought to contribute to reading comprehension, such as vocabulary size, l1 and l2 literacy, and reading atti- tudes, in an effort to improve the explainable reading comprehension variance. In order to more accurately calculate the text coverage a reader experiences for a particular text, both the vocabulary profile of the text and the vocabulary size ofthe reader must be known as precisely as possible. Therefore, to contribute to the question of vocabulary size, changes such as measuring item completion time and varying the order of item presentation were made to the VST (P. Nation & Beglar, 2007) to improve its sensitivity and accuracy. This may ultimately lead to increased precision when using text coverage to predict reading comprehension. Finally, l2 English vocabulary size norms were established to supplement the dia- gnostic usefulness of the VST. Data were collected through an online version of the VST 1Hu and Nation (2000) is occasionally cited as Hsueh-Chao & Nation (e.g. in Brantmeier, 2004) due to a typographical error in the original publication. iii iv ABSTRACT created for this thesis from primarily self-selected participants (n ≈ 1:31 × 105) located in countries (n ≈ 100) around the world representing several l1 and age groups. Analysis of the data collected for this thesis suggest that text coverage explains much less reading comprehension variance than previously reported while vocabulary size may be a more powerful predictor. An internal replication of Hu and Nation (2000) found errors in the calculation of optimum text coverage and in the reported size of the effect on reading comprehension. A critical review of the theoretical foundations of the text coverage model of reading comprehension found serious flaws in construct operationalisation and research design. Due to these flaws, most research which has purported to measure the effect of text coverage on reading comprehension actually measured the effect of an intervening variable: readers’ vocabulary size. Vocabulary size norms derived from data collected through an online version of the VST appear to be reliable and representative. Varying item presentation order appears to increase test sensitivity. Despite a moderate effect for l1 English users, item completion time does not seem to account for any variance in vocabulary size scores for l2 English learners Based on the finding that vocabulary size may explain both reading comprehension and text coverage, the putative power of text coverage to predict reading comprehen- sion is challenged. However, an alternative measure which may offer greater power to predict reading comprehension, the VST, has been modified and made available online. This version of the VST may provide greater sensitivity and ease of use than the offline, paper-based version. Dedication For Professor Thomas Michael Cobb, for openly sharing with the world a bountiful array of hand-crafted, web-based resources which have long been a source of inspiration. And for taking the time to thoroughly and vigorously critique this thesis. v vi DEDICATION Acknowledgements This thesis would not have been possible without various forms of support, advice, and encouragement graciously provided by the following people, organisations, pro- grammes, and things: General • Paul Nation • 邱麗靜 (/ʈʂʰīɤʊ lì.ʈʂìŋ/, Li-ching Chiu) • ธัญญาภรณ์ ผ่องผิว (/tʰan.jaː.pʰɔn pʰɔ̀ːŋ.pʰǐu/, Tanyapon Phongphio) Resources and finances • PhD Scholarship awarded by Victoria University of Wellington • Faculty Research Grants awarded by the Faculty of Humanities and Social Sciences • Innumerable resources and services donated by Paul Nation Data • 138 anonymous participants in Thailand • ≈ 1:31 × 105 anonymous participants around the world Technical • Patrick Hindmarsh • Members of SWEN 302: – Cameron Gray – Josephine Hall – Patrick Hindmarsh – Nicholas Lim – Daniel Park – Felix (Fuci) Shi – Philip Walkerdine – Henry Williams • the authors of and contributors to the software listed in Appendix I (page 237) vii viii ACKNOWLEDGEMENTS Morals, ethics, and motivation • Atheist Community of Austin • Richard Dawkins • Daniel Dennett • Neil deGrasse Tyson • Sam Harris • Christopher Hitchens (1949–2011) • Paul Nation • James Randi • Carl Sagan (1934–1996) • Michael Shermer • Peter Singer Miscellaneous • Every researcher and author cited in this thesis • Many other individuals and organisations too numerous to itemise • Coffee (lots) Preface Citations One convention in this thesis may appear, at first glance, to be an error. Careful readers are hereby warned that the citation format in this thesis strictly follows the “Publication Manual of the American Psychological Association” (American Psychological Associ- ation, 2010). The consequence of this policy is that the names of some authors appear to be inconsistently cited. Fear not; there is method in the madness. Inconsistencies in self-attribution on the part of some authors in their publications is the cause.The “Publication Manual of the American Psychological Association” provides no latitude to unilaterally normalise citations and doing so would run afoul of the express purpose of a citation: to provide enough accurate and complete information for the reader to retrieve a copy of the cited source (see p. 180). A long history of carelessness in report- ing research, detailed in chapter 5 (page 111), gives further justification for presenting authors’ names in the exact format as they appear in publication. Nevertheless, it useful to protect the sanity of the mindful reader by briefly docu- menting here the list of authors who, despite variations in their names, are actually the same persons: I. S. Paul Nation: Hirsh and Nation (1992), Hu and Nation (2000), Laufer and Nation (2001), Liu and Nation (1985), I. S. P. Nation (2006), I. S. P. Nation (2001), P. Nation (1983), P. Nation (1993), P. Nation and Beglar (2007), P. Nation and Coady (1988), P. Nation and Ming-Tzu (1999), Nguyen and Nation (2011) Charles A. Perfetti: C. Perfetti (2007), C. A. Perfetti and Hogaboam (1975) Regrettably, there is but one exception to this rule. The first author of HuandNation (2000) is listed as Marcella Hu Hsueh-chao in print. Almost every secondary source which cites this study lists the authors as Hu and Nation, Brantmeier (2004) being a notable and admirable exception, so to not follow suit would result in greater confusion. ix x PREFACE This minor violation of correcting the typographical error in the original publication still adheres to goal of helping the reader retrieve the original source. This exception is further supported by examining the unpublished Master’s thesis (Hu, 1999) on which the published article is based. The author of that work is listed as Hsueh-chao Marcella Hu. Non-English text Another idiosyncrasy which may surprise observant readers is the use of languages other than English when it does not appear to be strictly necessary. This is done for three reasons. The first is practical. Often times there are several methods oftranslit- erating other languages into the Latin-based English orthography. Sometimes, none of the available methods are ideal due to the lack of direct correspondence between the other language and English.2 Therefore, writing languages in their own orthography is the most accurate representation possible. On a deeper level, another reason to use native orthographies is to show respect to the study participants and their l1s. Wherever possible, non-English words are written in their respective orthographies followed by both a phonetic transcription for readers who may want to know the pronunciation and an English translation to provide absolute clarity. Finally, this thesis deals with reading in a l2. Using non-English orthographies explicitly demonstrates to the reader how orthographic distance from a reader’s l1 can present a unique challenge when learning to read in some languages. When en- countered, readers are invited to linger on these examples while