A Quantitative Approach to Linguistic Model Validation Ph.D. Thesis by LEE PICK PANG INSTITUTE OF EDUCATION UNIVERSITY OF LONDON 1989 DECLARATION I hereby declare that this thesis represents my own work and that it has not been previously submitted to this university or other institutions in application for a degree, diploma or other qualification. Table of Contents Table of Contents Abstract Acknowledgements iii Initial Influences, Objectives & The Methodology of The Study 1.1 Introduction 1 1.2 Current state of linguistics and applied linguistics . 1 1.3 The domain of the research 3 1.4 Objectives of the present research project . 4 1.5 Delimitations of the study 5 1.5.1 Text and discourse 5 1.5.2 Text structure 7 1.5.3 The reading process 8 1.5.4 Scope of the Study . 8 Alternative Models of Text Analysis 2.1 Introduction . .. .10 2.2 The Tagmemic approach . 11 2.2.1 Overview . .11 2.2.2 Meyer (1975) . 14 2.2.3 Clements (1979) . 15 2.3 Schema Theory .. 16 2.3.1 Overview .. 16 2.3.2 Schema 17 2.3.3 Script . .18 2.3.4 Frame . 18 2.3.5 Observations .19 Table of Contents 2.4 Text Linguistics 20 2.4.1 van Dijk 20 2.4.2 de Beaugrande & Dressler 21 2.4.3 Specimen text analysis 22 2.4.4 Observations 23 2.5 Text Signalling Approach . ..... 24 2.5.1 Clause relation & signalling devices . 24 2.5.2 Discourse pattern 26 The Text Analysis Approach Adopted 3.1 Introduction 28 3.2 Criteria for choice 29 3.2.1 Being text-based . .. 29 3.2.2 Being codable . .. 29 3.2.3 Surface textual features . 30 3.3 Evaluation of the text analysis approaches 30 3.3.1 The Schema approach . 30 3.3.2 Text Linguistics . .. 32 3.3.3 The Tagmemic approach . 33 3.3.4 Text Signalling approach 34 3.4 Conclusion .... 34 Statistical Discussion: An Introduction to the Research Methods 4.1 Introduction 36 4.2 The nature of the data to be collected and the method of analysis 36 4.2.1 Scientific research 36 4.2.2 Quantitative and qualitative research 38 4.3 Sources of behavioural data 44 Table of Contents 4.4 Relevance to the present research . 46 4.5 Repertory Grid Analysis • .... 47 4.5.1 Personal Construct Psychology and Repertory Grid Analysis 47 4.5.2 Characteristics of Repertory Grid Analysis . 48 4.6 The general characteristics of Repertory Grid Analysis . 50 4.6.1 General outline . 50 4.6.2 Repertory Grid Analysis and linguistics . 51 4.6.3 Identification of the Elements 51 4.6.4 Specification of the Constructs 52 4.6.5 Configuration of the constructs 53 4.7 Comparison of repertory grid analysis methods 53 4.7.1 Introduction . 53 4.7.2 Repertory grid analysis methods 54 4.7.3 Theoretical and statistical considerations 56 4.8 An Empirical Investigation . 59 4.8.1 The text ..... 59 4.8.2 Methods of analysis . 59 4.9 Generating the Theoretical Grid . 59 4.9.1 The constructs . 59 4.9.2 The grid .. 60 4.10 Analysis Using FOCUS . 60 4.10.1 Cluster Analysis . 60 4.10.2 Data display . 62 4.10.3 The Spaced Focus grid . 62 4.10.4 Comments on FOCUS . 65 4.11 Analysis using INGRID . 66 4.11.1 Principal component analysis . 66 4.11.2 Observations . .69 4.12 Conclusion 70 Table of Contents Implementing Repertory Grid Analysis 5.1 Introduction 72 5.2 The implementation of repertory grid analysis 72 5.2.1 Elicitation of the grid 73 5.2.1.a Identification of elements 73 5.2.1.b Specification of constructs 73 5.2.2 Repertory grid analysis using INGRID 74 5.2.3 INGRID 75 5.2.4 The other programs 76 5.3 The Pre-pilot study 77 5.3.1 Introduction 77 5.3.2 The Study 78 5.3.2.a The Experimental Subjects 78 5.3.2.b The Grid . 78 5.3.2.c The Experiment 80 5.3.2.d The Results 81 5.3.3 Observations 82 5.4 The Pilot Study .... 84 5.4.1 Objectives . . 84 5.4.2 The design . . 84 5.4.3 The sample . . 85 5.4.4 The texts . .... 85 5.4.5 The constructs ... 86 5.4.6 The theoretical grids . 87 5.4.7 The triads . . 88 5.4.8 The experiment ... 88 5.4.8.a Organizing the experimental sessions . 88 5.4.8.b Conducting the experimental sessions . 88 5.4.8.c Analysing the grids .. 89 Table of Contents 5.4.9 Results 89 5.4.9.a Aim of this section . 89 5.4.9.b Construct elicitation through triads . 89 5.4.9.c Comparison with the theoretical grid . 92 5.4.10 Observations 96 5.4.10.a Target element identifications 96 5.4.10.b Comparison with the theoretical grid 97 5.4.10.c Analysis of individual subject grids 97 5.4.11 Conclusion 98 The Experiment and Further Research Design Issues 6.1 Introduction 99 6.2 The Experimental Procedure 99 6.2.1 The population and the sample 99 6.2.2 Grid elicitation . .. 101 6.2.3 The analysis design 102 6.2.3.a The overall approach • 102 6.2.3.b Quantitative analysis • 103 6.2.3.c Qualitative analysis 104 6.2.4 The experimental sessions 105 6.3 The Analysis 107 6.3.1 Introduction . 107 6.3.2 Repertory Grid Analysis 107 6.3.3 Quantitative Analysis 109 6.3.4 Qualitative Analysis 110 Table of Contents RESULTS I - Individual Perception 7.1 Introduction 114 7.1.1 Scope of the presentation 114 7.1.2 Repertory grid analysis 115 7.2 The INGRID output .115 7.3 The Theoretical Grids .116 7.3.1 The Helicopter text • 116 7.3.1.a Overview of the statistics . 116 7.3.1.b Sums of squares . 118 7.3.1.c Descriptive statistics . 119 7.3.1.d Principal component analysis . 120 7.3.1.e Issues relating to principal component analysis . 122 7.3.1.f The Bartlett test . 124 7.3.1.g Principal component analysis for the Helicopter text 125 7.3.1.h Observations . 126 7.3.2 The Computer text 128 7.3.2.a Observations . 133 7.4 Case studies of subject grids .... 136 7.4.1 Introduction ...... 136 7.4.2 Case 1 . 137 7.4.3 Case 2 . 145 7.4.4 Case 3 . 153 7.4.5 Case 4 . 158 7.4.6 Case 5 . 164 7.4.7 Observations ..... 169 Table of Contents 7.5 Qualitative Analysis 171 7.5.1 Introduction 171 7.5.2 Case 1 173 7.5.3 Case 2 174 7.5.4 Case 3 . 175 7.5.5 Case 4 .... 176 7.5.6 Case 5 177 7.5.7 Observations 179 RESULTS 2 - Theoretical & Subject Grid Agreement 8.1 Introduction 182 8.2 Theoretical grid and subject grid agreement 183 8.2.1 Matching the constructs 183 8.2.2 Indices of agreement between constructs Correlation and Angular distances . 184 8.2.3 Statistics on construct correlations & angular distances Introductory remarks . 185 8.2.4 Descriptive statistics . 187 8.2.5 The extent of theory-subject agreement . 190 8.3 Subject consensus . 193 8.3.1 Introduction . 193 8.3.2 Overall results . 194 8.3.3 The Helicopter text .. 196 8.3.4 The Computer text . 201 8.3.5 Observations .... 205 Table of Contents 8.4 ANOVA 206 8.4.1 Introduction 206 8.4.2 The Results 208 8.4.2.a Overall results 208 8.4.2.b Detailed results for main effects 210 8.4.2.c Interaction effects 211 8.4.3 Observations 216 8.5 Observations on theory-subject agreement 216 Conclusions & Implications for Future Research 9.1 Introduction 218 9.2 The methodological investigation 219 9.2.1 Introduction 219 9.2.2 Review of the field of text analysis and the choice of the linguistic model . 220 9.2.3 Discussion and choice of an analytic technique . 221 9.2.4 The pilot studies . .. 222 9.2.4.a The pre-pilot study . 222 9.2.4.b The pilot study . 223 9.2.5 The main study . 224 9.3 Principal findings 226 9.3.1 Introduction 226 9.3.2 Repertory grid analysis 226 9.3.2.a the choice of elements and constructs 227 9.3.2.b The construct elicitation technique 228 9.3.2.c Principal component analysis 229 Table of Contents 9.3.3 The Winter-Hoey model . 231 9.3.3.a What has been validated . 231 9.3.3.b Problems with the Winter-Hoey model . 233 9.3.3.c To what extent is the Winter-Hoey model not validated? . 234 9.3.3.d Conditions for adequacy of linguistic model validation: . 235 9.3.4 Some special issues 236 9.3.4.a Text patterning and text signalling constructs 236 9.3.4.b The text structure as analysed by a quantitative method . 237 9.3.4.c Observations . 238 9.4 Implications for future research . 239 9.4.1 Introduction . 239 9.4.2 Repertory grid analysis . 239 9.4.2.a Element and construct specification . 240 9.4.2.b Construct elicitation technique . 241 9.4.2.c Principal component analysis . 242 9.4.3 The Winter-Hoey model .