The Role of Syllable Structure in Verbal Short-Term Memory Tom Hartley University College London September 20,1995 THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY 1 Abstract Remembering the sound of a new word when it is first encountered is an important skill which plays a critical role in the development of vocabulary (Gathercole & Bad- deley, 1989), yet the mechanismsunderlying this form of verbal short-term memory are not well understood. Errors in the repetition and serial recall of nonwords indi- cate that structural properties of the syllable are represented in short-term memory, but existing accounts of serial learning and recall do not incorporate any representa- tion of linguistic structure. Models of speech production implicate syllable structure in the representation of phonological form, but do not explain how such represen- tations are acquired. This thesis draws together theories of speech production and serial memory to develop a computational model of nonword repetition based on the novel idea that short-term memory for the serial order of a sequence of speech sounds is constrained by a syllabic template. The results of simulations using the model are presented and compared with experimental findings concerning short-term memory for nonwords. The interaction of short- and long-term phonological memory systems and the aquisition of vocabulary are discussed in terms of the model. The model is evaluated in comparison with other contemporary theories. 2 Acknowledgements The research described in this thesis was supported by a grant from the SERC. Rebecca Treiman and Sue Gathercole generously provided supplementary information and data from some of their experimental studies which helped enormously in developing the model described in chapter 5 I would like to thank my supervisor, George Houghton, for helping me find a subject worth investigating, for giving me the freedom to explore it and for providing encouragement and inspiration. Many others at UCL helped me in one way or another, by providing inspiration, advice and support. I would particularly like to thank Stevie Sackin, David Glasspool, Paul Hibbard, John Draper, Neil Burgess, Willie Curran, Colin Clifford, Keith Langley, Rick Cooper, and Jonny Farringdon. There are many people whose humour and friendship helped me to complete this project: Tom Manly, my brother Olly, James Taylor, Jenny Porter and The Crayons all helped me carry on. Special thanks are due to Sophie Scott, who is a great friend. She encouraged me to come to UCL in the first place, set a good example, helped me get my act together and made working here alot of fun. Without Rachel Harris, I could not have even begun this thesis. I cannot thank her enough for the constant love and support she has given me. I would also like to thank all my family especially my mum, Di Beattie, my dad, Ian Hartley, and my stepmother Jennifer Hartley for all the encouragement they have given me. This thesis is for my grandparents. Thankyou. 3 Contents 1 Introduction 18 I General introduction 18 ........................... 2 Short-term 19 memory ............................ 2.1 Functions 21 of short-term memory ................. 2.2 The 24 role of speech in short-term memory ............ 2.3 Serial 25 order and short-term memory ............... 2.4 Errors in 26 speech and short-term memory ............ 3 Overview 28 .................................. 2 The problem of serial order: general issues and models 29 1 'Buffers ................................... 31 Associative 33 chaining ........................... 2.1 Representing items 34 repeated and multiple sequences ...... 4 CONTENTS 2.2 Coarticulation Anticipation and ................. 37 2.3 More 'chaining' recent models .................. 39 3 Parallel models of serial order ...................... 40 3.1 1-1 models ............................. 43 P-I models ................................. 51 4.1 Competitive 54 queuing ....................... 5 Summary Conclusions 63 and ........................ 3 Empirical background: serial order and memory for speech 64 1 Stimulus familiarity 64 and recall ...................... 2 Phonological 72 constraints on recall .................... 3 Constraints 74 on speech errors ....................... 3.1 Errors in 75 spontaneous speech .................. 3.2 Experimentally 79 elicited slips ................... Summary 80 ................................. 4 Modelling constraints on phonological retrieval: a cyclical syllable template 82 Phonological STM, CQ 83 slips, and models ................ 5 CONTENTS 2 PsYcholinguisticmodels of phonologicalretrieval ............ 90 3 General constraints on syllabic phonology ................ 95 A template syllabic ............................ 98 4.1 Parsing the syllable ........................ 98 4.2 Parsing continuous 'speech .................... 100 5A new model of short-term memory 103 I Description the of model .................. 103 1.1 Architecture ............................ 104 1.2 Learning Recall 107 and ....................... 1.3 Recall ill ............................... Simulations 115 ................................ Quantitative 116 measuresof performance ............. 2.2 Qualitative 118 analyses ....................... 2.3 Modelling damaged /developing 126 a store ............. 2.4 Error 129 mechanisms ......................... 9.5 Syllable 136 structure and errors ................... 2.6 Testable 139 predictions ........................ 6 CONTENTS 6 Future directions and conclusions 141 I Extending the the lexicon 141 model: short-term memory and ...... The in to model relation other work ................... 146 2.1 Models 147 of serial order and short-term memory ......... 2.2 Models 150 of speech production ................... 2.3 Syllable 151 structure ......................... 3 Conclusions 153 ................................ A Coding used in phonological simulations 155 I Vowels 156 ................................... 2 Consonants 157 ................................ B Technical appendix to chapter 5 158 1 Activation 158 ....................... 2 Learning 159 .................................. Recall 160 .............. ................... 4 Rate 163 of presentation and recall . ................... C Exception words 165 7 CONTENTS D Dynamics of the syllabic template 169 Dynamics the template 169 I of syllabic .................... 1.1 Capturing in 170 syllable structure a recurrent network ...... 1.2 Determining 175 syllabic phase .................... Formal 182 2 specification of the syllabic phase algorithm .......... Bibliography 185 8 List of Figures 2.1 Representation of a sequence in the form of a chain of item-item associations................................. 34 2.2 a) Associative chaining has difficulty in acccounting for the repre- sentation of sequences containing repeats. It is not clear which of the associative links from "E" should befollowed. b) If a single associative structure is to represent multiple sequences, there is no mechanism to select those links which must be followed to generate a particular se- quence. Here phonemic representations of cat, tack and act would in- terfere with one another (different types of arrows are usedto show the links needed for each word). c) and d) Wickelgren's context-specific coding overcomes these difficuffies, but only at the expense of using different tokens to the type 35 enbrely represent instances of same .... 2.3 Structure of part of a typical neural network. A node 1 has a number of afferent connections. The weighted sum of these inputs (i, = Eiwijoi) determine Js (aj (Ij)) The i.s used to activation =f . node's output is a function (g) of its activation. Each node's efferent connections are weighted, and weights can be changed systematically so that the learns to to inputs 42 network produce an adaptive response its ...... 9 LIST OF FIGURES Estes' (1972) for 2.4 scheme the representation of serially ordered ac- For (e. tions. each sequence g., "CHA, 1,N"), there is a node which activates its nodes standing for its constituent items via excitatory links to each (solid arrows). Item nodes representing actions later in the sequencereceive inhibitory input (dashed arrows) from nodes representing earlier items - for clarity, only the lateral connections from "H" are shown here; the later an item is in the list, the more items precede it. Items nearer the end of the sequencereceive most inhibition, thus when the item nodes are activated in parallel by input from the 45 sequencenodes, an activation gradient is set up........ 2.5 Jordan (1986) and Elman (1990) have put forward similar schemes for the learning and representation of serial order in connectionist networks. In each case fixed recurrent connections (solid arrows) pro- vide a tZme-varying context at the level of the input units. Modifiable feedforward connections (dashed arrows) are adjusted using supervised learning 47 algorithms ............................. 2.6 Grossbergl'S (1978) model uses weights of varying strength between sequence and item nodes to set up an activation gradient, which is to behaviour 51 used control serial ...................... 2.7 Sequences coded using Grossberg's scheme (1978) do not interfere in the way that sequences coded as associative chains do. Here, for ex- ample, the weights used to represent the sequences "A, CT", "T, R, A) P" "P, A, T" do 53 and not conflict with one another ......... 10 LIST OF FIGURES 2.8 In (CQ) competitive queuing models, a time-varying control signal (or is to context) used