MEASURED LANGUAGE The Georgetown University Round Table on Languages and Linguistics Series Selected Titles Discourse 2.0: Language and New Media DEBORAH TANNEN AND ANNA MARIE TRESTER, EDITORS Educating for Advanced Foreign Language Capacities: Constructs, Curriculum, Instruction, Assessment HEIDI BYRNES, HEATHER D. WEGER-GUNTHARP, AND KATHERINE A. SPRANG, EDITORS Implicit and Explicit Language Learning: Conditions, Processes, and Knowledge in SLA and Bilingualism CRISTINA SANZ AND RONALD P. LEOW, EDITORS Language in Use: Cognitive and Discourse Perspectives on Language and Language Learning ANDREA E. TYLER, MARI TAKADA, YIYOUNG KIM, AND DIANA MARINOVA, EDITORS Linguistics, Language, and the Professions: Education, Journalism, Law, Medicine, and Technology JAMES E. ALATIS, HEIDI E. HAMILTON, AND AI-HUI TAN, EDITORS Telling Stories: Language, Narrative, and Social Life DEBORAH SCHIFFRIN, ANNA DE FINA, AND ANASTASIA NYLUND, EDITORS MEASURED LANGUAGE Quantitative Approaches to Acquisition, Assessment, and Variation Jeffrey Connor-Linton and Luke Wander Amoroso, Editors GEORGETOWN UNIVERSITY PRESS Washington, DC Georgetown University Press, Washington, D.C. www.press.georgetown.edu © 2014 by Georgetown University Press. All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and recording, or by any information storage and retrieval system, without permission in writing from the publisher. Library of Congress Cataloging-in-Publication Data Measured language : quantitative studies of acquisition, assessment, and variation / Jeffrey Connor-Linton and Luke Wander Amoroso, Editors. pages cm. — (Georgetown University Round Table On Languages And Linguistics Series) Includes bibliographical references and index. ISBN 978-1-62616-037-8 (pbk. : alk. paper) 1. Linguistics—Methodology. 2. Linguistics—Statistical methods. 3. Language and languages—Versification. 4. Computational linguistics. I. Connor-Linton, Jeff, editor. II. Amoroso, Luke Wander, editor. P126.M39 2014 410.72’7—dc23 2013024841 This book is printed on acid-free paper meeting the requirements of the American National Standard for Permanence in Paper for Printed Library Materials. 15 14 9 8 7 6 5 4 3 2 First printing Printed in the United States of America ■ Contents Acknowledgments xi Introduction xiii ■ Jeff Connor-Linton and Luke Wander Amoroso, Georgetown University 1. The Ubiquitous Oral versus Literate Dimension: A Survey of Multidimensional Studies 1 ■ Douglas Biber, Northern Arizona University 2. When Ethnicity Isn’t Just about Ethnicity 21 ■ Penelope Eckert, Stanford University 3. Does Language Zipf Right Along? Investigating Robustness in the Latent Structures of Usage and Acquisition 33 ■ Nick C. Ellis, University of Michigan; Matthew Brook O’Donnell, University of Michigan; Ute Römer, Georgia State University 4. Subjectivity and Efficiency in Language Assessment: Explorations of a Compensatory Rating Approach 51 ■ Steven J. Ross, University of Maryland 5. Subgrouping in Nusa Tenggara: The Case of Bima-Sumba 63 ■ Emily Gasser, Yale University 6. Young Learners’ Storytelling in Their First and Foreign Languages 79 ■ Yuko Goto Butler and Wei Zeng, University of Pennsylvania 7. Measuring Quechua to Spanish Cross-Linguistic Influence 95 ■ Marilyn S. Manley, Rowan University 8. Speedup versus Automatization: What Role Does Learner Proficiency Play? 111 ■ Jessica G. Cox and Anne M. Calderón, Georgetown University 9. Frequency Effects, Learning Conditions, and the Development of Implicit and Explicit Lexical Knowledge 125 ■ Phillip Hamrick, Georgetown University; Patrick Rebuschat, Lancaster University vi Contents 10. The Differential Role of Language Analytic Ability in Two Distinct Learning Conditions 141 ■ Nadia Mifka Profozic, University of Zadar, Croatia 11. U-Shaped Development: Definition, Exploration, and Falsifiable Hypotheses 155 ■ Hiroyuki Oshita, Ohio University 12. Using Simulated Speech to Assess Japanese Learner Oral Proficiency 171 ■ Hitokazu Matsushita and Deryle Lonsdale, Brigham Young University 13. Keys to College: Tracking English Language Proficiency and IELTS Test Scores in an International Undergraduate Conditional Admission Program in the United States 183 ■ Reese M. Heitner, Barbara J. Hoekje, and Patrick L. Braciszewski, Drexel University 14. How Does Foreign Language Proficiency Change over Time? Results of Data Mining Official Test Records 199 ■ Amber Bloomfield, Steven Ross, Megan Masters, Kassandra Gynther, and Stephen O’Connell, University of Maryland 15. The Development of Complexity in a Learner Corpus of German 213 ■ Colleen Neary-Sundquist, Purdue University Index 229 ■ Illustrations Figures 1.1 Mean Scores of Registers along Dimension 1: Involved vs. Informational Production 8 1.2 Mean Scores for Registers along Dimension 2: Narrative vs. Nonnarrative Discourse 10 2.1 Negative Concord at Belten High (p=.000) 22 2.2 Negative Concord by Gender and Social Category 22 2.3 Negative Concord by Subgroups of Jocks and Burnouts 23 2.4 The Coastal California Shift 25 2.5 The Fields Pattern: Rachel’s TRAP 29 2.6 The Steps Pattern: Selena’s TRAP 29 2.7 Linda’s TRAP in Sixth and Seventh Grades 30 3.1 BNC Verb Type Distribution for ‘V across n’ and for ‘V n n’ 42 3.2 A Semantic Network for ‘V across n’ from the BNC Using WordNet as a Base 44 4.1 OPI Task Difficulties 53 4.2 Probability of Rating Difference by Rater Severity 55 4.3 Risk of Not Improving over Time Attributable to Severity Differences 58 4.4 Martingale Residuals for Rater Severity Differences 59 4.5 Rater Co-Calibration Scheme 60 4.6 Risk of Not Improving over Time after Rating Co-Calibration 61 5.1 Five Subgrouping Hypotheses 64 5.2 Language Map 67 5.3 Unrooted Distance-Based Network 72 5.4 Stochastic Dollo Maximum Clade Credibility Tree 74 6.1 Structural Coherence in L1 and L2 and FL Fourth Graders (N = 32) 86 6.2 Structural Coherence in L1 and FL Sixth Graders (N = 32) 86 6.3 Structural Coherence in L1 and FL Eightn Graders (N = 32) 87 6.4 Structural Coherence in FL and PPVT in FL 89 6.5 Number of Idea Units Fourth Grade (N = 16) 90 6.6 Number of Idea Units Sixth Grade (N = 16) 90 vii viii Illustrations 6.7 Number of Idea Units Eighth Grade (N = 16) 91 7.1 Number of Participants Using the Thirty-One Features 101 7.2 Cross-Linguistic Feature Implicational Scale 102 9.1 Simple Screenshot Sequence from the Exposure Phase 131 9.2 Accuracy at Test by Input Frequency When Using Implict and Explicit Knowledge 135 11.1 Group Differences in Judgment of Active and Passive Sentences 163 12.1 Generated SS Scores for Students by Level across SOPI Tasks 179 12.2 Means of Generated SS Scores by Student Level across SOPI Tasks 179 12.3 Differences in Generated SS Scores across Class Level 180 12.4 Distribution of Generated SS Scores across All Items 181 13.1 Demographic Data of Gateway Participants 187 13.2 Average IELTS Scores of Matriculated Students Grouped by Test Date 188 13.3 Average L/R/W/S IELTS Scores of Matriculated Students by Test Date 189 13.4 Average and Range of IELTS Scores of Full Cohort Grouped by ELC Levels 2-6 189 13.5 Average and Range of Gateway CoAS GPAs for Matriculated Students 190 14.1 Latent Growth Model Fit to the 1–4 Listening Test Administrations in the Dataset 204 14.2 Latent Growth Model Fit to the 1–4 DLPT Reading Test Administrations 205 14.3 The Latent Growth Model Fit to the 2–4 Speaking Test Administrations 206 14.4 Retention of Listening Skills between First and Most Recent Test 208 14.5 Retention of Reading Skills between the First and Most Recent Test 209 14.6 Retention of Speaking Skills between Second and Most Recent Test 209 14.7 Retention of Speaking Skills between Second and Most Recent Speaking Test by Initial Proficiency Level 210 15.1 Subordination Rate for Speaking vs. Writing 224 15.2 Coordination Rate for Speaking vs. Writing 224 15.3 Mean Length Clause for Speaking vs. Writing 225 Tables 1.1. MD studies of English discourse domains 12 1.2. MD studies of discourse domains of other languages 14 5.1. Languages analyzed 68 5.2. Sample data (cognate judgments) 70 5.3. Sample data for “old”, with coding converted to binary 71 Illustrations ix 6.1. Descriptive statistics (means and standard deviations) of structural coherence scores in L1 and FL 85 6.2. Regression analyses predicting the story grammar scores in FL from those in L1 85 6.3. Correlations among variables 88 7.1. Participant characteristics 97 7.2. Classification of thirty-one cross-linguistic features 98 8.1. Participant characteristics by proficiency level 118 8.2. Mean RT (ms), SDRT, CVRT, and error rate for L1 and L2 (standard deviations in parentheses) 121 9.1. Ambiguous and unambiguous target items and their referents 130 9.2. Accuracy and proportions (%) across source attributions 135 10.1 Descriptive statistics for language analysis test 146 10.2 Descriptive statistics for test scores in oral production 147 10.3 Descriptive statistics for test scores in written production 148 10.4 Between group differences on each posttest: RE and CR group 148 10.5 Correlations between gains and LAT scores 149 12.1 Simulated speech test items selected for administration and analysis 174 12.2 Features used for ASR-based fluency analysis 176 12.3 Classification accuracy for k-NN learning of SS test items 177 12.4 Prediction accuracy rate (%) for machine learning of SS test items using decision trees 178 12.5 Factors and their calculation for SS score generation 178 13.1 AY 2010-2011 Gateway Curriculum 186