Statistical Language Learning: Learning-Oriented Theories Is That Mechanisms and Constraints Such Accounts Seem at Odds with One of the Central Observations 1 Jenny R

Home , Language acquisition, Language development

110 VOLUME 12, NUMBER 4, AUGUST 2003

most important arguments against Statistical Language Learning: learning-oriented theories is that Mechanisms and Constraints such accounts seem at odds with one of the central observations 1 Jenny R. Saffran about human languages. The lin- Department of Psychology and Waisman Center, University of Wisconsin–Madison, guistic systems of the world, de- Madison, Wisconsin spite surface differences, share deep similarities, and vary in non- arbitrary ways. Theories of language Abstract guage, it seems improbable that chil- acquisition that focus primarily on What types of mechanisms dren could ever discern its structure. preexisting knowledge of language underlie the acquisition of hu- The process of acquiring such a sys- do provide an elegant explanation man language? Recent evidence tem is likely to be nearly as com- for cross-linguistic similarities. suggests that learners, includ- plex as the system itself, so it is not Such theories, which are exempli- ing infants, can use statistical surprising that the mechanisms un- fied by the seminal work of Noam properties of linguistic input to derlying language acquisition are a Chomsky, suggest that linguistic uni- discover structure, including matter of long-standing debate. One versals are prespecified in the child’s sound patterns, words, and the of the central focuses of this debate linguistic endowment, and do not re- beginnings of grammar. These concerns the innate and environ- quire learning. Such accounts gener- abilities appear to be both pow- mental contributions to the lan- ate predictions about the types of erful and constrained, such that guage-acquisition process, and the patterns that should be observed some statistical patterns are degree to which these components cross-linguistically, and lead to im- more readily detected and used draw on information and abilities portant claims regarding the evolu- than others. Implications for the that are also relevant to other do- tion of a language capacity that in- structure of human languages mains of learning. cludes innate knowledge of this kind are discussed. In particular, there is a funda- (e.g., Pinker & Bloom, 1990). mental tension between theories of Can learning-oriented theories language acquisition in which also account for the existence of Keywords learning plays a central role and language universals? The answer to language acquisition; statistical theories in which learning is rele- this question is the object of current learning; infants gated to the sidelines. A strength research. The constrained statistical of learning-oriented theories is that learning framework suggests that they exploit the growing wealth of learning is central to language acqui- Imagine that you are faced with evidence suggesting that young sition, and that the specific nature of the following challenge: You must humans possess powerful learning language learning explains similar- discover the underlying structure mechanisms. For example, infants ities across languages. The crucial of an immense system that contains can rapidly capitalize on the statis- point is that learning is constrained; tens of thousands of pieces, all gen- tical properties of their language learners are not open-minded, and erated by combining a small set of environments, including the distri- calculate some statistics more elements in various ways. These butions of sounds in words and the readily than others. Of particular in- pieces, in turn, can be combined in orders of word types in sentences, terest are those constraints on learn- an infinite number of ways, although to discover important components ing that correspond to cross-linguis- only a subset of those combinations of language structure. Infants can tic similarities (e.g., Newport & is actually correct. However, the sub- track such statistics, for example, Aslin, 2000). According to this set that is correct is itself infinite. to discover speech categories (e.g., framework, the similarities across Somehow you must rapidly figure native-language consonants; see, e.g., languages are indeed nonacciden- out the structure of this system so Maye, Werker, & Gerken, 2002), tal, as suggested by the Chomskian that you can use it appropriately word boundaries (e.g., Saffran, Aslin, framework—but they are not the early in your childhood. & Newport, 1996), and rudimentary result of innate linguistic knowl- This system, of course, is human syntax (e.g., Gomez & Gerken, edge. Instead, human languages language. The elements are the 1999; Saffran & Wilson, 2003). have been shaped by human learn- sounds of language, and the larger However, theories of language ing mechanisms (along with con- pieces are the words, which in turn acquisition in which learning plays straints on human perception, pro- combine to form sentences. Given a central role are vulnerable to a cessing, and speech production), and the richness and complexity of lan- number of criticisms. One of the aspects of language that enhance

Published by Blackwell Publishing Inc.

CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE 111 learnability are more likely to per- is a challenging problem for infants words, we exposed adults, first sist in linguistic structure than acquiring their first language, for graders, and 8-month-olds to spo- those that do not. Thus, according speakers do not mark word bound- ken nonsense languages in which to this view, the similarities across aries with pauses, as shown in Fig- the only cues to word boundaries languages are not due to innate ure 1. Instead, infants must deter- were the statistical properties of knowledge, as is traditionally mine where one word ends and the the syllable sequences (e.g., Saffran claimed, but rather are the result of next begins without access to obvi- et al., 1996). Listeners briefly heard constraints on learning. Further, if ous acoustic cues. This process re- a continuous sequence of syllables human languages were (and con- quires learning because children containing multisyllabic words tinue to be) shaped by constraints cannot innately know that, for exam- from one of the languages (e.g., on human learning mechanisms, it ple, pretty and baby are words, golabupabikututibubabupugola- seems likely that these mechanisms whereas tyba (spanning the bound- bubabupu. . . ). We then tested our and their constraints were not tai- ary between pretty and baby) is not. participants to determine whether lored solely for language acquisi- One source of information that they could discriminate the words tion. Instead, learning in nonlinguis- may contribute to the discovery of from the language from sequences tic domains should be similarly word boundaries is the statistical spanning word boundaries. For ex- constrained, as seems to be the structure of the language in the in- ample, we compared performance case. fant’s environment. In English, the on words like golabu and pabiku A better understanding of these syllable pre precedes a small set of with performance on sequences like constraints may lead to new con- syllables, including ty, tend, and cedes; bupabi, which spanned the bound- nections between theories focused in the stream of speech, the probabil- ary between words. To succeed at on nature and theories focused on ity that pre is followed by ty is thus this task, listeners would have had nurture. Constrained learning mech- quite high (roughly 80% in speech to track the statistical properties of anisms require both particular expe- to young infants). However, because the input. Our results confirmed that riences to drive learning and pre- the syllable ty occurs word finally, it human learners, including infants, existing structures to capture and can be followed by any syllable that can indeed use statistics to find manipulate those experiences. can begin an English word. Thus, word boundaries. Moreover, this the probability that ty is followed ability is not confined to humans: by ba, as in pretty baby, is extremely Cotton-top tamarins, a New World low (roughly 0.03% in speech to monkey species, can also track sta- LEARNING THE SOUNDS young infants). This difference in se- tistics to discover word boundaries OF WORDS quential probabilities is a clue that (Hauser, Newport, & Aslin, 2001). pretty is a word, and tyba is not. One question immediately raised In order to investigate the nature More generally, given the statistical by these results is the degree to of infants’ learning mechanisms, my properties of the input language, which statistical learning is limited colleagues and I began by studying the ability to track sequential prob- to languagelike stimuli. A growing an aspect of language that we knew abilities would be an extremely body of results suggests that se- must certainly be learned: word useful tool for infant learners. quential statistical learning is quite segmentation, or the boundaries be- To explore whether humans can general. For example, infants can tween words in fluent speech. This use statistical learning to segment track sequences of tones, discovering “tone-word boundaries” via statistical cues (e.g., Saffran, Johnson, As- lin, & Newport, 1999), and can learn statistically defined visual patterns (e.g., Fiser & Aslin, 2002; Kirkham, Slemmer, & Johnson, 2002); work in progress is extending these results to the domain of events in human action sequences. Given that the ability to discover Fig. 1. A speech waveform of the sentence “Where are the silences between words?” units via their statistical coherence The height of the bars indicates loudness, and the x-axis is time. This example illus- is not confined to language (or to hu- trates the lack of consistent silences between word boundaries in fluent speech. The vertical gray lines represent quiet points in the speech stream, some of which do not mans), one might wonder whether correspond to word boundaries. Some sounds are represented twice in the transcrip- the statistical learning results actu- tion below the waveform because of their continued persistence over time. ally pertain to language at all. That

112 VOLUME 12, NUMBER 4, AUGUST 2003 is, do infants actually use statistical world’s languages, but fail to learn tial structures better when they are learning mechanisms in real-world regularities that are inconsistent with organized into subunits such as language acquisition? One way to natural language structure. For ex- phrases than when they are not? We address this question is to ask what ample, infants rapidly learn new identified a statistical cue to phrasal infants are actually learning in our phonotactic regularities involving units, predictive dependencies (e.g., segmentation task. Are they learn- generalizations across sounds that the presence of a word like the or a ing statistics? Or are they using share a phonetic feature, while fail- predicts a noun somewhere down- statistics to learn language? Our re- ing to learn regularities that disre- stream; the presence of a preposition sults suggest that when infants be- gard such features. Thus, it is easier predicts a noun phrase somewhere ing raised in English-speaking en- for infants to learn a set of patterns downstream), and determined that vironments have segmented the that group together /p/, /t/, and learners can use this kind of cue sound strings, they treat these non- /k/, which are all voiceless, and to locate phrase boundaries (Saf- sensical patterns as English words that group together /b/, /d/, and fran, 2001a). (Saffran, 2001b). Statistical lan- /g/, which are all voiced, than to In a direct test of the theory that guage learning in the laboratory learn a pattern that groups together predictive dependencies enhance thus appears to be integrated with /d/, /p/, and /k/, but does not learnability, we compared the acqui- other aspects of language acquisi- apply to /t/.2 Studies of this sort sition of two nonsense languages, tion. Related results suggest that 12- may provide explanations for why one with predictive dependencies as month-olds can first segment novel languages show the types of sound a cue to phrase structure, and one words and then discover syntactic patterning that they do; sound lacking predictive dependencies regularities relating the new words— structures that are hard for infants (e.g., words like the could occur ei- all within the same set of input. This to learn may be unlikely to recur ther with or without a noun, and a would not be possible if the infants across the languages of the world. noun could occur either with or formed mental representations only without words like the; neither type of the sequential probabilities relat- of word predicted the presence of ing individual syllables, and no the other). We found better language word-level representations (Saffran STATISTICAL LEARNING learning in listeners exposed to lan- & Wilson, 2003). These findings point AND SYNTAX guages containing predictive depen- to a constraint on statistical language dencies than in listeners exposed to learning: The mental representa- Issues about learning versus in- languages lacking predictive depen- tions produced by this process are nate knowledge are most promi- dencies (Saffran, 2002). Interestingly, not just sets of syllables linked by nent in the area of syntax. How the same constraint on learning statistics, but new units that are could learning-oriented theories ac- emerged in tasks using nonlinguis- available to serve as the input to count for the acquisition of abstract tic materials (e.g., computer alert subsequent learning processes. structure (e.g., phrase boundaries) sounds and simultaneously pre- Similarly, it is possible to exam- not obviously mirrored in the sur- sented shape arrays). These results ine constraints on learning that face statistics of the input? Unlike support the claim that learning might affect the acquisition of the accounts centered on innate linguis- mechanisms not specifically de- sound structure of human languages. tic knowledge, most learning-ori- signed for language learning may The types of sound patterns that in- ented theories do not provide a have shaped the structure of human fants learn most readily may be more transparent explanation for the languages. prevalent in languages than are ubiquity of particular structures sound patterns that are not learn- cross-linguistically. One approach to able by infants. We tested this hy- these issues is to ask whether some pothesis by asking whether infants nearly universal structural aspects DIRECTIONS FOR find some phonotactic regularities of human languages may result FUTURE RESEARCH (restrictions on where particular from constraints on human learning sounds can occur; e.g., /fs/ can oc- (e.g., Morgan, Meier, & Newport, Results to date demonstrate that cur at the end, but not the beginning, 1987). To test this hypothesis, we human language learners possess of syllables in English) easier to ac- asked whether one such aspect of powerful statistical learning capac- quire than others (Saffran & Thies- syntax, phrase structure (groupings ities. These mechanisms are con- sen, 2003). The results suggest that of types of words together into sub- strained at multiple levels; there are infants readily acquire novel regu- units, such as noun phrases and verb limits on what information serves larities that are consistent with the phrases), results from a constraint on as input, which computations are types of patterns found in the learning: Do humans learn sequen- performed over that input, and the

Published by Blackwell Publishing Inc.

CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE 113 structure of the representations that abstract knowledge), and about patterns accessible to human learners emerge as output. To more fully un- whether learning can still be con- help in disentangling the complexi- derstand the contribution of statisti- sidered statistical when the input to ties of this system? Although the cal learning to language acquisition, learning is abstract. Debates re- answer to this question remains un- it is necessary to assess the degree to garding the proper place for statis- known, it is possible that a combi- which statistical learning provides tical learning in theories of language nation of inherent constraints on the explanatory power given the com- acquisition cannot be resolved in ad- types of patterns acquired by learn- plexities of the acquisition process. vance of the data. For example, al- ers, and the use of output from one For example, how does statisti- though one can distinguish be- level of learning as input to the next, cal learning interact with other as- tween statistical versus rule-based may help to explain why something pects of language acquisition? One learning mechanisms, and statisti- so complex is mastered readily by way we are addressing this ques- cal versus rule-based knowledge, the the human mind. Human learning tion is by investigating how infants data are not yet available to deter- mechanisms may themselves have weight statistical cues relative to mine whether statistical learning it- played a prominent role in shaping other cues to word segmentation self renders rule-based knowledge the structure of human languages. early in life. The results of such stud- structures, and whether abstract ies provide an important window knowledge can be probabilistic. into the ways in which statistical Significant empirical advances will Recommended Reading learning may help infant learners be required to disentangle these and Gómez, R.L., & Gerken, L.A. (2000). to determine the relevance of the other competing theoretical distinc- Infant artificial language learning many cues inherent in language in- tions. and language acquisition. Trends put. Similarly, we are studying Finally, cross-species investiga- in Cognitive Sciences, 4, 178–186. how statistics meet up with mean- tions may be particularly informa- Hauser, M.D., Chomsky, N., & Fitch, ing in the world (e.g., are statisti- tive with respect to the relationship W.T. (2002). The faculty of language: What is it, who has it, and cally defined “words” easier to learn between statistical learning and hu- how did it evolve? Science, 298, as labels for novel objects than sound man language. Current research is 1569–1579. sequences spanning word bound- identifying species differences in the Pena, M., Bonatti, L.L., Nespor, M., & aries?), and how infants in bilingual deployment of statistical learning Mehler, J. (2002). Signal-driven environments cope with multiple mechanisms (e.g., Newport & As- computations in speech process- ing. Science, 298, 604–607. sets of statistics. Studying the inter- lin, 2000). To the extent that nonhu- Seidenberg, M.S., MacDonald, M.C., section between statistical learning mans and humans track different & Saffran, J.R. (2002). Does gram- and the rest of language learning statistics, or track statistics over dif- mar start where statistics stop? may provide new insights into how ferent perceptual units, learning Science, 298, 553–554. various nonstatistical aspects of lan- mechanisms that do not initially ap- guage are acquired. Moreover, a pear to be human-specific may ac- clearer picture of the learning mech- tually render human-specific out- Acknowledgments—The preparation of anisms used successfully by typical comes. Alternatively, the overlap this manuscript was supported by grants from the National Institutes of Health language learners may increase re- between the learning mechanisms (HD37466) and National Science Founda- searchers’ understanding of the available across species may sug- tion (BCS-9983630). I thank Martha Ali- types of processes that go awry gest that differences in statistical bali, Erin McMullen, Seth Pollak, Erik Thiessen, and Kim Zinski for comments when children do not acquire lan- learning cannot account for cross- on a previous version of this manuscript. guage as readily as their peers. species differences in language- It is also critical to determine learning capacities. which statistics are available to young learners and whether those Notes statistics are actually relevant to natural language structure. Researchers CONCLUSION 1. Address correspondence to Jenny R. Saffran, Department of Psychology, do not agree on the role that statis- University of Wisconsin–Madison, Mad- tical learning should play in acqui- It is clear that human language ison, WI 53706; e-mail: jsaffran@ sition theories. For example, they is a system of mind-boggling com- wisc.edu. disagree about when learning is plexity. At the same time, the use 2. Voicing refers to the timing of vi- best described as statistically based of statistical cues may help learners bration of the vocal cords. Compared with voiceless consonants, voiced con- as opposed to rule based (i.e., uti- to discover some of the patterns sonants have a shorter lag time between lizing mechanisms that operate lurking in language input. To what the initial noise burst of the consonant over algebraic variables to discover extent might the kinds of statistical and the subsequent vocal cord vibrations.

114 VOLUME 12, NUMBER 4, AUGUST 2003

References fect phonetic discrimination. Cognition, 82, Saffran, J.R. (2001b). Words in a sea of sounds: The B101–B111. output of statistical learning. Cognition, 81, 149–169. Fiser, J., & Aslin, R.N. (2002). Statistical learning of Morgan, J.L., Meier, R.P., & Newport, E.L. (1987). new visual feature combinations by infants. Structural packaging in the input to language Saffran, J.R. (2002). Constraints on statistical lan- Proceedings of the National Academy of Sciences, learning: Contributions of intonational and guage learning. Journal of Memory and Lan- USA, 99, 15822–15826. morphological marking of phrases to the ac- guage, 47, 172–196. quisition of language. Cognitive Psychology, 19, Gomez, R.L., & Gerken, L. (1999). Artificial gram- 498–550. Saffran, J.R., Aslin, R.N., & Newport, E.L. (1996). mar learning by 1-year-olds leads to specific Newport, E.L., & Aslin, R.N. (2000). Innately con- Statistical learning by 8-month-old infants. Sci- and abstract knowledge. Cognition, 70, 109– strained learning: Blending old and new ap- ence, 274, 1926–1928. 135. proaches to language acquisition. In S.C. Saffran, J.R., Johnson, E.K., Aslin, R.N., & New- Hauser, M., Newport, E.L., & Aslin, R.N. (2001). Howell, S.A. Fish, & T. Keith-Lucas (Eds.), Pro- port, E.L. (1999). Statistical learning of tone Segmentation of the speech stream in a non- ceedings of the 24th Boston University Conference sequences by human infants and adults. Cogni- human primate: Statistical learning in cotton- on Language Development (pp. 1–21). Somer- tion, 70, 27–52. top tamarins. Cognition, 78, B41–B52. ville, MA: Cascadilla Press. Kirkham, N.Z., Slemmer, J.A., & Johnson, S.P. Pinker, S., & Bloom, P. (1990). Natural language Saffran, J.R., & Thiessen, E.D. (2003). Pattern in- (2002). Visual statistical learning in infancy: and natural selection. Behavioral and Brain Sci- duction by infant language learners. Develop- Evidence of a domain general learning mecha- ences, 13, 707–784. mental Psychology, 39, 484–494. nism. Cognition, 83, B35–B42. Saffran, J.R. (2001a). The use of predictive depen- Saffran, J.R., & Wilson, D.P. (2003). From syllables Maye, J., Werker, J.F., & Gerken, L. (2002). Infant dencies in language learning. Journal of Mem- to syntax: Multi-level statistical learning by 12- sensitivity to distributional information can af- ory and Language, 44, 493–515. month-old infants. Infancy, 4, 273–284.

We define a symbolic artifact as The Origins of Pictorial Competence something that someone intends to Judy S. DeLoache,1 Sophia L. Pierroutsakos, and David H. Uttal stand for something other than itself (DeLoache, 1995). Thus, virtu- Department of Psychology, University of Virginia, Charlottesville, Virginia (J.S.D.); ally anything can serve as a sym- Department of Psychology, Furman University, Greenville, South Carolina (S.L.P.); and Department of Psychology, Northwestern University, Evanston, Illinois bol, and virtually any concept that (D.H.U.) one has can be symbolized, but the symbol is always different in some way from that which it represents. What makes something symbolic is Abstract symbolic species” (Deacon, 1997), and symbolization is the “most human intention; an entity be- Pictorial competence, which characteristic mental trait of [hu- comes a symbol only as the result refers to the many factors in- mans]” (Langer, 1942, p. 72). Just as of a person using it to denote or re- volved in perceiving, interpret- the emergence of the symbolic ca- fer to something. ing, understanding, and using pacity in the course of evolution ir- pictures, develops gradually revocably transformed the human over the first few years of life. species, so too does the develop- Although experience is not re- ment of symbolic functioning THE CHALLENGE OF quired for accurate perception transform young children. The ca- DUAL REPRESENTATION of pictures, it is necessary for pacity for symbolization vastly ex- understanding the nature of pands their intellectual horizons, pictures. Infants initially re- Although mastering symbols is liberating them from the con- spond to depicted objects as if a universal task, it is not an easy straints of time and space and en- they were real objects, and tod- one. A formidable challenge to abling them to acquire information dlers are remarkably insensi- young children in developing com- about reality without directly expe- tive to picture orientation. Only petence with symbols stems from riencing it. gradually do young children the inherently dual nature of sym- All children growing up any- figure out the nature of pictures bols; every symbolic artifact is an where in the world must master a and how they are used. object in and of itself, and at the wide variety of symbol systems same time it also stands for some- Keywords and symbolic artifacts for full par- thing other than itself. To under- ticipation in their society. Our re- symbolic development; pic- stand and use a symbol, dual repre- ture perception search has focused on how young sentation is necessary—one must children begin to understand and mentally represent both facets of exploit the informational potential the symbol’s dual reality, both its As philosophers, new and old, of various symbolic objects, includ- concrete characteristics and its ab- have emphasized, humans are “the ing models, maps, and pictures. stract relation to what it stands for

Published by Blackwell Publishing Inc.