The empirical base of linguistics Grammaticality judgments and linguistic methodology Carson T. Schütze language Classics in Linguistics 2 science press Classics in Linguistics Chief Editors: Martin Haspelmath, Stefan Müller In this series: 1. Lehmann, Christian. Thoughts on grammaticalization 2. Schütze, Carson T. The empirical base of linguistics: Grammaticality judgments and linguistic methodology 3. Bickerton, Derek. Roots of language ISSN: 2366-374X The empirical base of linguistics Grammaticality judgments and linguistic methodology Carson T. Schütze language science press Carson T. Schütze. 2016. The empirical base of linguistics: Grammaticality judgments and linguistic methodology (Classics in Linguistics 2). Berlin: Language Science Press. This title can be downloaded at: http://langsci-press.org/catalog/book/89 © 2016, Carson T. Schütze Published under the Creative Commons Attribution 4.0 Licence (CC BY 4.0): http://creativecommons.org/licenses/by/4.0/ ISBN: 978-3-946234-02-9 (Digital) 978-3-946234-03-6 (Hardcover) 978-3-946234-04-3 (Softcover) 978-1-523743-32-2 (Softcover US) ISSN: 2366-374X DOI:10.17169/langsci.b89.100 Cover and concept of design: Ulrike Harbort Typesetting: Felix Kopecky, Sebastian Nordhoff, Carson T. Schütze Fonts: Linux Libertine, Arimo, DejaVu Sans Mono Typesetting software:Ǝ X LATEX Language Science Press Habelschwerdter Allee 45 14195 Berlin, Germany langsci-press.org Storage and cataloguing done by FU Berlin Language Science Press has no responsibility for the persistence or accuracy of URLs for external or third-party Internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate. For my mother, Dorly Schütze, and the memory of my father, Ted Schütze It is simultaneously the greatest virtue and failing of linguistic theory that sequence acceptability judgments are used as the basic data. (Bever 1970b) Contents Preface (2016) xi Preface (1996) xvii Acknowledgments (1996) xix 1 Introduction 1 1.1 Goals ................................. 1 1.2 Approach ............................... 7 1.3 Motivation: Whither Linguistics? ................. 10 1.4 A Working Hypothesis ....................... 13 1.5 Scope and Organization ....................... 15 2 Definitions and Historical Background 19 2.1 Introduction ............................. 19 2.2 A Short History of Grammaticality ................ 20 2.3 The Use of Judgment Data in Linguistic Theory .......... 36 2.3.1 Introduction ......................... 36 2.3.2 The Dangers of Unsystematic Data Collection ...... 37 2.3.3 A Case Study in the Use of Subtle Judgments ...... 41 2.3.4 The Interpretation of the Annotations and Degrees of Bad- ness ............................. 44 2.4 Introspection, Intuition, and Judgment .............. 48 2.5 Conclusion .............................. 52 3 Judging Grammaticality 55 3.1 Introduction ............................. 55 3.2 Tasks that Access Grammaticality ................. 56 3.3 The Nature of Graded Judgments .................. 62 3.3.1 Is Grammaticality Dichotomous? ............. 62 3.3.2 Experiments on Chomsky’s Three Levels of Deviance .. 70 3.3.3 Other Experiments ..................... 74 Contents 3.3.4 Ratings, Rankings, and Consistency ............ 77 3.4 The Judgment Process ........................ 81 3.5 The Interpretation of Judgments with Respect to Competence . 88 3.6 Conclusion .............................. 96 4 Subject-Related Factors in Grammaticality Judgments 97 4.1 Introduction ............................. 97 4.2 Individual Differences: Three Representative Studies ....... 98 4.3 Organismic Factors ......................... 106 4.3.1 Field Dependence ...................... 106 4.3.2 Handedness ......................... 107 4.3.3 Other Organismic Factors ................. 109 4.4 Experiential Factors ......................... 112 4.4.1 Linguistic Training ..................... 112 4.4.2 Literacy and Education ................... 120 4.4.3 Other Experiential Factors ................. 124 4.5 Conclusion .............................. 126 5 Task-Related Factors in Grammaticality Judgments 127 5.1 Introduction ............................. 127 5.2 Procedural Factors .......................... 129 5.2.1 Instructions ......................... 129 5.2.2 Order of Presentation ................... 132 5.2.3 Repetition .......................... 133 5.2.4 Mental State ......................... 138 5.2.5 Judgment Strategy ..................... 142 5.2.6 Modality and Register ................... 144 5.2.7 Speed of Judgment ..................... 146 5.3 Stimulus Factors ........................... 148 5.3.1 Context ........................... 148 5.3.2 Meaning ........................... 157 5.3.3 Parsability .......................... 160 5.3.4 Frequency .......................... 161 5.3.5 Lexical Content ....................... 162 5.3.6 Morphology and Spelling ................. 164 5.3.7 Rhetorical Structure .................... 164 5.4 Conclusion .............................. 165 viii Contents 6 Theoretical and Methodological Implications 167 6.1 Introduction ............................. 167 6.2 Modeling Grammaticality Judgments ............... 168 6.2.1 Previous Work ....................... 168 6.2.2 The Outlines of a Preliminary Model ........... 169 6.2.3 Applications of the Model ................. 177 6.3 Methodological Proposals ...................... 180 6.3.1 Materials .......................... 180 6.3.2 Procedure .......................... 183 6.3.3 Analysis and Interpretation of Results .......... 191 6.4 Conclusion .............................. 197 7 Looking Back and Looking Ahead 199 7.1 Introduction ............................. 199 7.2 Directions for Further Research .................. 201 7.3 The Future in Linguistics ...................... 206 References 209 Indexes 229 Name Index ................................. 229 Subject Index ................................ 235 ix Preface (2016) Since the original version of this book (University of Chicago Press, 1996) went out of print in the 2000s, I have continued to receive inquiries from people asking how they can obtain a copy. I am therefore thrilled that Language Science Press has offered to make the title available again, as part of their Classics in Linguistics series. I would like to thank series editors Stefan Müller and Martin Haspelmath, as well as Sebastian Nordhoff and Felix Kopecky, for their help in making this happen. The content of this new printing is identical to the first printing, with thefol- lowing exceptions: • I have altered the wording in a few places where I found it insufficiently clear or terminologically outdated; • my uses of the term informant(s) have been replaced with consultant(s) or speaker(s), in keeping with current practice (of course, the former term still appears in some quoted passages); • I have updated the reference information for a couple of works that had not been published at the time of the original printing, particularly Cowart (1997); • the original index has been split into name and subject indexes, and both are now more comprehensive. In terms of presentation, the following things have changed: • the format of citations and references has been adapted to LangSci house style, as have other minor typographical choices; • full given names have been added to references whenever available; • since the text has been freshly typeset, the page numbers do not match those of the original printing; however, the (sub)section numbers are un- changed: I suggest using those if it is necessary to specify a location within a chapter. Example numbers are also unchanged. Preface (2016) Importantly, I have not attempted to update the content in light of subsequent relevant research, since this would undoubtedly have compelled me to try to write a whole new book. Of course, linguistics and psycholinguistics have chang- ed a great deal in the 20 years since I completed the original manuscript; e.g., “theoretical” linguistics has notably become more “experimental.” Also, some of my own views on the issues have evolved over those two decades. There are passages in the book that I would have omitted or altered, if I had allowed myself to make any substantive revisions. Instead, I have chosen to restrict all follow- up discussion to this preface. In what follows I try to point readers to works that should allow them to “get up to speed” on intervening developments. For collections that are comprised mainly of papers on topics that are impor- tant in the book, see McNair et al. (1996), Penke & Rosenbach (2004), Kepser & Reis (2005), Borsley (2005), Featherston (2007) and replies in the same journal issue, Featherston & Sternefeld (2007), Featherston & Winkler (2009), and Win- kler & Featherston (2009). My more recent views can be found in the following surveys: Schütze (2006; 2011) and Schütze & Sprouse (2013). There have been (at least) four major developments involving the empirical base of linguistics that anyone interested in the topic should be aware of. 1. The adaptation of the magnitude estimation task from psychophysics to judgment collection (Bard, Robertson & Sorace 1996). This was touted as having numerous potential advantages over the traditional Likert scale task, most or all of which have been subsequently refuted (see Weskott & Fanselow 2011 and Sprouse, Schütze & Almeida 2013). 2. The use of World Wide Web searches to establish attestation, and inferac- ceptability, of certain sentence/construction