Running head: SURPRISE AS VICARIOUS PREDICTION ERROR

Surprisingly unsurprising! Infants’ looking time to improbable events is modulated by

others’ expressions of surprise

In preparation

Yang Wu

Hyowon Gweon

Department of Psychology, Stanford University

Keywords: cognitive development; social cognition; surprise; prediction error; emotion, statistical

inference; looking time

Address for correspondence:

Corresponding Author Name: Yang Wu

Department of Psychology

Stanford University

450 Serra Mall, Jordan Hall 280

Stanford, CA 94305

E-mail: [email protected]

1 Abstract

Research in diverse disciplines suggests that agents’ own prediction error enhances their learning. Yet, human learners also possess powerful capacities to learn from others. Here we ask whether infants can use others’ expressions of surprise as vicarious prediction error signals to infer hidden states of the world. First, we conceptually replicated Xu & Garcia (2008), showing that infants (12.0-17.9 months) looked longer at improbable than probable sampling outcomes (Experiment 1). Then we added emotional cues to the design (Experiment 2). Before revealing an outcome to an infant, the experimenter looked at the outcome and showed either an unsurprised or surprised emotional expression. While infants still looked longer at the improbable than the probable outcome following the experimenter’s unsurprised emotional expression, this trend was reversed following the experimenter’s surprised expression. Thus, although for decades infants’ looking time has been assumed to be longer for events that are unexpected or surprising, these results suggest that infants’ inferential abilities in social contexts are powerful enough to make surprising events “surprisingly unsurprising.”

2 Human learners accomplish remarkable achievements in their first years of life – understanding minds, numbers, and languages, to name a few (Carey, 2009; Gopnik, 2012). How children learn has inspired research in diverse scientific domains, and has been intensively studied for several decades. One challenge for learning, noted by this literature, is that our learning environment is often complex, dynamic and full of information. To make learning efficient, or even possible at all, learners must understand when to attend to some information, when to ignore it, and from whom to learn.

One of the most powerful proposals for how learning occurs states that agents identify opportunities for learning based on prediction error (e.g., Kulkarni, Narasimhan, Saeedi, & Tenenbaum,

2016; Schultz, 2016; Silver et al., 2018). When there is a discrepancy between what an agent expects to happen and the actual outcome, the agent uses the discrepancy to modify its behavior. The neural and computational bases of such error-driven learning have been extensively studied in both animal models and human adults (see Schultz, 2016 for a review). This learning mechanism has also been a central notion of theoretical models in various domains, including reinforcement learning, perceptual inference, cognition, and decision-making (see Den Ouden, Kok, & De Lange, 2012).

This type of error-driven, surprise-induced learning has also been found in human infants and children. When infants and children observe an event that is inconsistent with their predictions or expectations, they show a range of behavioral, neural and physiological changes. The most well-known behavior is that infants look longer at an event that violates their expectation than an event that does not.

Accordingly, infants’ looking time to expected vs. unexpected events has been used in developmental research as a popular dependent measure (i.e., violation-of-expectation paradigm). Infants also show changes in pupil dilation (Jackson & Sirois, 2009), cerebral blood flow (Wilcox, Bortfeld, Woods, Wruck,

& Boas, 2005) and brain electrical activity (Berger, Tzur, & Posner, 2006) following unexpected events.

Critically, infants and older children also explore more when events are unexpected, and show enhanced

3 learning following surprising events (e.g., Bonawitz, van Schijndel, Friel, & Schulz, 2012; Chandler &

Lalonde, 1994; Phelps & Woolley, 1994; Schulz, 2012; Stahl & Feigenson, 2018).

Nonetheless, we often miss out on a lot of surprising events. Sometimes we fail to pay attention or lack visual access to those events. Critically, when one lacks the relevant world knowledge, one might fail to find an event surprising even when it is in principle surprising; for instance, if you know little about the weather in Lima (where it almost never rains), you won’t be surprised when it rains in Lima. Young learners, in particular, may often fail to detect surprising events. Despite their sophisticated, early- emerging knowledge of the physical and social world (e.g., core principles of physical objects; Spelke,

Breinlinger, Macomber, & Jacobson, 1992), events that violate this knowledge are quite rare (e.g., a car floating in the air), making many potential opportunities for learning go unnoticed. Thus, although surprise-induced learning may be one fundamental mechanism that explains how agents learn broadly, it does not explain some critical aspects of human learning.

However, learning does not occur in isolation; young learners benefit from others who already know much about the world (Csibra & Gergely, 2009; Boyd & Richerson, 1988). Here we propose that young children may not only learn from their own prediction error, but also use others’ expressions of surprise as vicarious prediction error signals to identify special opportunities for learning. Such cues are abundant in children’s everyday life: Parents, caregivers, and educators do not only express various emotional expressions when they interact with young children, but also tend to exaggerate them for pedagogical and communicative purposes. In particular, expressions of surprise are often used to attract children’s attention to an interesting (and potentially informative) event even when it may not necessarily be surprising for adults.

This proposal is supported by the literature on early emotion understanding. From infancy on, children show remarkably early-developing sensitivity to others’ emotional expressions, and even represent others’ emotions as caused by events in the world and others’ goals, desires, and beliefs (see

4 Reschke, Walle, & Dukes, 2017 and Wellman, 2014 for reviews). Infants and children can also perform backward inferences, using observed emotional expressions to reason backward about their unobserved causes in the environment (Wu, Muentener, & Schulz, 2017) and hidden mental states of other people

(Wu & Schulz, 2018). These studies suggest that even young children have a sophisticated understanding of others’ emotional expressions, and use those emotional expressions as powerful sources of information to guide their understanding of the world.

Research on the emotional expression of surprise, however, has primarily studied this emotional expression as an affective consequence of one’s false beliefs (e.g., Hadwin & Perner, 1991; Ruffman &

Keenan, 1996; Harris, 1989; Wellman & Banerjee, 1991; Scott, 2017). Additionally, a recent study found that 7-year-olds expect others to be surprised in response to a low-probability event (Doan, Friedman, &

Denison, 2018). This work provides additional support that young children can represent others’ emotions as jointly caused by their internal mental states and external world states. However, a critical question about the relevance of surprise for learning remains open: Do young children use others’ expressions of surprise as signals of prediction error and use them to guide their own inferences about the world?

To answer this question, we ask whether infants (12- to 17-month-olds) can use others’ expressions of surprise to form some expectations about an unobserved outcome. When an agent looks surprised after seeing the outcome of an event (but the outcome is not yet visible to the infant), it provides important information about the nature of the hidden outcome: it is unexpected. Thus, infants would therefore expect an unexpected outcome and look longer when the outcome is expected than unexpected. By contrast, if the agent looks unsurprised, this suggests that the hidden outcome is consistent with what she expected, leading to the usual pattern of looking time (i.e., longer when the outcome is unexpected than when it is expected; Aslin, 2007). Thus, our primary interest here is an interaction effect between the agent’s expression (Surprise vs. No Surprise) and the probability of the outcome (Probable vs. Improbable) in infants’ looking time.

5 We test these predictions using events adapted from a well-established prior study (Xu & Garcia,

2008). This study has shown that 8-month-olds expect randomly drawn samples to be representative of the population; when an agent randomly draws ping-pong balls from a box that contains mostly white and just a few red ones, infants look longer when the sample is mostly red (an improbable outcome) than when it is mostly white (a probable outcome). In our study, we add emotion information to the paradigm; before the agent reveals the outcome to the infant, she first looks at it herself and makes either a surprised or an unsurprised emotional response to it. Then she reveals the outcome to the infant. We predict that while previewing the agent’s unsurprised expression would result in a similar pattern of looking time as the original study, observing a surprised emotional expression would reverse the pattern such that infants would look longer when the revealed outcome is expected than unexpected.

We choose the improbable events in Xu & Garcia (2008) for our initial investigation, rather than impossible events that violate object knowledge (Spelke et al., 2012), in order to maximize the chance of detecting the effect of surprise. Our intuition is that improbable events would be more sensitive than impossible events for this purpose, because both probable and improbable outcomes are in our hypothesis space and they only differ in probability. For example, when someone randomly draws a ball from a box that contains mostly red balls and only a few white ones, it is possible for her to get either a red ball or a white ball, although a red ball is more probable than a white ball. If we observe that she is surprised by what she got, we may form a specific expectation of the outcome: she may have gotten a white ball. By contrast, impossible outcomes are not always in our hypothesis space. When someone is surprised by a hidden impossible outcome (e.g., a car goes through a solid wall), although we may still have an abstract representation of the outcome (i.e., it is surprising), it is often unlikely that we have the specific expectation of what actually happened simply based on others’ surprised emotional expressions. For this reason, we expect that others’ surprised emotional expressions have a stronger effect on infants’ looking

6 time to improbable events than impossible events, and as our initial investigation, we use improbable events to maximize the chance of detecting the effect.

Last, although the understanding of intuitive statistics has been shown in infants within their first year of life (Denison, Reed, & Xu, 2013; Xu & Garcia, 2008), this study focuses on infants aged 12 to 17 months for several reasons. First, we expect the integration of emotion understanding and event probability to manifest later than an understanding of event probability alone. Second, infants aged 12 to

17 months have been shown to be able to discriminate diverse positive emotional vocalizations and map them onto their probable causes in the environment (Wu et al., 2017). Third, our study requires attending to a few different events and may have working memory demands that are too high for those under 12 months of age. Additionally, a recent study shows that by 20 months, toddlers can predict others’ surprised expressions based on their false beliefs (Scott, 2017). Since our study uses a task simpler than a false belief task, we set the age range to be lower than 20 months and stay the span of 6 months: 12.0 – 17.9 months.

In Experiment 1, we conceptually replicate Xu & Garcia’s study (2008) to ensure that our stimuli and procedure can elicit similar results as prior work: infants look longer at improbable than probable outcomes. In Experiment 2, we add emotion information to the design to look at the effect of others’ surprised emotional expressions. We predict an interaction between emotion (Surprise vs. No Surprise) and outcome probability (Probable vs. Improbable): infants will still look longer at an improbable than probable outcome following others’ unsurprised emotional expressions but will show a reversed pattern of looking time following others’ surprise.

Experiment 1

Method

Participants

7 We estimated the effect size from Xu & Garcia (2008, Experiments 1, 2, 4, and 5; average Cohen’s d = .55) to compute the required sample size. The power analysis suggested that we needed N = 28 to reach 80% power. We thus recruited 28 infants between 12.0 and 17.9 months (mean: 15.5) from a local museum. Our a priori exclusion criteria were that infants would be dropped and replaced if over half test trials were not usable due to fussiness, parental or sibling interference, experimenter error, or infant looking time over 3 standard deviations of the mean. Two infants were dropped and replaced because of fussiness (n = 1) and parent/sibling interference (n = 1).

Materials

We made two boxes (30 cm X 24 cm X 30 cm) with carton board. One box was used for the familiarization trials and the other for the test trials. The front and back sides of the box were transparent.

Following Xu & Garcia (2008), we divided the inside of each box into three compartments, and filled each compartment with mixed red and white ping pong balls. The experimenter always pretended to draw balls from the middle compartment through a top opening of the box, but it appeared that she drew balls from a big box filled with ping-pong balls. For the familiarization box, both the front and back compartments were filled with 50% white and 50% red ping-pong balls and the box appeared to contain half red and half white balls when seen from both sides. For the test box, the front compartment was filled with 90% white and 10% red ping-pong balls and the reverse for the back compartment such that the box appeared to contain mostly white balls when seen from one side, and appeared to contain mostly red balls when seen from the other side.

To make sure that every time the right ball was sampled and the infant could not see the color of the sampled ball until the experimenter revealed it to them, we decided that the experimenter would only pretend to draw a ball and put it inside a container but in reality a ball had already been placed inside the container in advance. For this purpose, we made six small opaque containers with carton board. Each container had a top opening so that the experimenter could pretend to drop a ball inside. The front side of

8 each container could be opened to reveal what was inside. Inside each box we attached either a red or white ball. When the experimenter pretended to drop a ball (but actually dropped nothing) and then revealed the ball that had been placed inside in advance, it appeared that the experimenter dropped that ball inside. Additionally, we pre-recorded a ball-dropping sound and placed a small speaker that could play that sound inside the container; the experimenter could trigger the ball-dropping sound by pressing a button on the back of the container when she pretended to drop a ball. Naïve adults believed that the experimenter actually drew a ball from inside the big box and placed it inside the container.

A piece of black felt was used to cover the big box before each trial began. A webcam was used to monitor the infants’ looking during each test phase. A laptop and a mouse were used to code infants’ looking time online.

Procedure

Each infant was tested in a private room inside a local museum. The infant sat on the parent’s lap, approximately 1.5 meters away from the experimenter. The experimenter first greeted the infant and said:

“Let me show you something!” Then she started the familiarization phase.

There were two trials in the familiarization phase. On each trial, the experimenter took out a big box covered by a piece of black felt as well as a small opaque container. She removed the felt from the box and said: “Look!” The box appeared to contain 50% red and 50% white ping-pong balls based on its transparent front. The experimenter shook the box and turned her head to its front to indicate that she knew the population of balls inside. Then she said: “Let me take a ball! Look at me!” She pointed to her eyes and then covered her eyes with one hand while reaching into the box with her other hand to randomly draw a ball. She pretended to take a ball out of the box and then put it inside the container while explaining:

“I got one and I put it inside here.” In reality, she did not draw any ball and there was a ball already placed in the container. She triggered a ball dropping sound by pressing a secret button on the back of the container when she pretended to drop the ball. Then she opened her eyes and looked at the infant. After a

9 brief pause, she revealed the ball inside the container by opening its front cover. She said: “Look what I got!” It was a red ball on one trial and a white ball on the other (order counterbalanced). Then she looked away for five seconds.

There were four trials in the test phase. Each test trial was similar to the familiarization trial except that the box contained 90% red balls and 10% white balls or vice versa (order alternated across trials and counterbalanced across participants) such that one outcome was expected (i.e., the ball drawn from the box is the majority color) or unexpected (i.e., it is the minority color; color counterbalanced). At the end of each trial, the experimenter looked away until the infant looked away from the stage for two consecutive seconds. The same experimenter made this online judgment; while she was looking away, she could see the infant’s looking from a hidden laptop connected to a webcam that monitored the infant’s behavior.

She secretly coded the infant’s looking with a mouse under the table. Later, a coder blind to the hypothesis and conditions of the study coded the infant’s looking offline from videotapes; the inter-coder reliability was high, but we used the blind-coded data for further analyses.

Results and discussion

Using the Likelihood Ratio test, we selected a linear mixed effect model that best fitted our data.

The best-fit model had Outcome Probability (Probable vs. Improbable) and Trial Order (1, 2, 3, or 4) as fixed factors and Subject as a random factor. There was an effect of Trail Order (F(3, 71) = 3.90, p = .012), suggesting that infants’ looking time decreased over the test phase. Replicating previous work, infants looked longer at the improbable outcome than the probable outcome (F(1, 71) = 2.98, p = .089; See Figure

2B), suggesting that our stimuli and paradigm can elicit similar results as prior work. Note that the improbable sample in our study was not as extremely improbable as the original study (e.g., 1 red ball from a 10% red and 90% white box is not as improbable as 1 white and 4 red balls from the same box).

While this might have led to a somewhat smaller effect size than the original study, the use of a single sample was a critical design decision to constrain the number of possible outcomes to just 2 (either red or

10 white); our main prediction (i.e., an interaction between emotion and outcome probability) does not hinge on the number of balls sampled nor the original effect size; it depends on the combination of an improbable/probable outcome and the experimenter’s surprised/unsurprised look.

In our next experiment, we add emotion information to the design to test if emotional cues can change infants’ representation of a hidden outcome. After the experimenter puts the sampled ball in the container, she orients the front side of the container to herself and reveals the ball inside so that the ball is visible to her but not to the infant. She makes either a surprised or an unsurprised emotional expression to the ball inside. Then the outcome is revealed to the infant. We predict an interaction between the experimenter’s emotional expression and the probability of the outcome: infants will look longer at the improbable than probable outcome in the No Surprise condition, and critically, show the opposite pattern in the Surprise condition.

Experiment 2

Method

Participants

Since this was the initial investigation of the effect of others’ surprise on infants’ expectation of a hidden outcome, we did not have the necessary grounds for estimating effect size. Note that the primary effect we are interested in here is the interaction between Emotion (surprise vs. no surprise) and Event

Probability. The effect size found in Experiment 1 (i.e., the main effect of event probability when there was no emotional cue) was thus uninformative about the effect size of interest here. As a first step however, we chose the same sample size as Experiment 1 (N = 28, aged 12.0 – 17.9 months, which was larger than most of the infant studies in literature) to ensure sufficient power to detect the effect of interest.

We thus recruited 28 infants between 12.0 and 17.9 months (mean: 15.4) from the same local museum.

Following our a priori exclusion criteria, five infants were dropped and replaced due to fussiness (n = 1), parent/sibling interference (n = 2), experimenter error (n = 1), and looking time over 3 standard deviations

11 of the mean (n = 1). (Additionally, due to miscommunication, the experimenter turned on a noisy fan in the testing room for four consecutive testing days and all infants (n = 12) tested during those days were dropped.)

Materials

We used the same materials as Experiment 1 with one exception. We added an extra layer to the front cover of each container in order to make the experimenter blind to the condition when she made an emotional expression. That is, she could pretend to open the front cover by opening only one of the two layers such that she was blind to the color of the ball when she made an emotional expression (but to the infant, it appeared that the experimenter could see the ball. The experimenter could open both layers altogether when she revealed the ball to the infant.

Procedure

The procedure was almost identical to Experiment 1 with one exception. On each test trial, after the experimenter put the sampled ball inside the container, she oriented the front side of the container towards herself, revealed its content (to herself but not to the infant), and expressed either a surprised or an unsurprised happy expression (order counterbalanced). In reality, the experimenter only opened one of the two layers of the front cover and was blind to the color of the ball when she made the expression. Then the experimenter turned the front side of the container back to the child and revealed the ball to the child by opening both layers of the front cover.

Results and discussion

Using the Likelihood Ratio test, we selected a linear mixed effect model that best fitted our data.

The best-fit model had Outcome Probability (Probable vs. Improbable), Emotion (Surprise vs. No

Surprise), Trial Order (1, 2, 3, or 4), and their interactions as fixed factors and Subject as a random factor.

Our primary interest, the interaction between Emotion and Outcome Probability was significant (F(1, 57)

12 = 7.38, p = .009). See Figure 2B. Post hoc analyses suggest that consistent with Experiment 1, there was a trend in the No Surprise conditions that the infants looked longer at the improbable than the probable outcome (t(19) = 1.92, p = .070; paired t-test); for the Surprise conditions, such difference disappeared

(t(23) = -.79, p = .438).

There was also a three-way interaction among Emotion, Outcome Probability, and Trial Order

(F(3, 57) = 4.58, p = .006), suggesting that the interaction between Emotion and Outcome Probability differed across trials. There was also a main effect of Trial Order (F(3, 57) = 6.91, p = .001). No other main effects or interactions were found (all Fs < 2.45, all ps > .123). Because of the three-way interaction, we looked at each trial separately. The predicted interaction was primarily on the first test trial (F(1, 21)

= 12.85, p = .002) but not on the following trials (all Fs < 1.90, all ps > .183). See Figure 2C. For the No

Surprise conditions, there was a trend that the infants looked longer at the improbable than probable outcome (t(3.09) = 2.40, p = .094; Welch Two Sample t-test); for the Surprise conditions, there was a flipped trend (t(13) = -2.11, p = .055). Although preliminary, these results provide suggestive evidence that infants as young as 12-17 months use others’ expressions of surprise as signals of prediction error and use them to guide their inferences about the world.

General Discussion

Here we investigated whether 12- to 17-month-old infants can use others’ expressions of surprise as vicarious prediction error signals to guide their understanding of the world. We looked at this by adapting a well-established prior study on intuitive statistical reasoning that used a violation-of- expectation paradigm (Xu & Garcia, 2008). In Experiment 1, we conceptually replicated their finding albeit with a weaker effect size, establishing that infants look longer at an improbable than probable outcome in our task. Then we added the critical emotion information: In Experiment 2, before infants saw an outcome, they first observed an experimenter’s emotional response – either a surprised or an unsurprised emotional expression – to the outcome (which was revealed to the infants only after the

13 experimenter’s expression). Consistent with our key prediction, we found a significant interaction between the experimenter’s emotional expression and the probability of the outcome, suggesting that observing others’ emotional expressions changed the infants’ representation of the hidden outcome. This interaction effect was the largest on the first test trial, and post hoc analyses confirmed the direction of interaction:

There was a trend that infants looked longer at the improbable than probable outcome following the experimenter’s unsurprised emotional expression, but a reverse pattern following the experimenter’s surprise.

This study provides important insights into the role of prediction error in learning. There has been a long history in diverse disciplines that emphasizes the role of prediction error in learning. However, all this work has looked at how agents learn when they themselves are surprised. Our study shows that humans also learn when others are surprised. They use others’ expressions of surprise as vicarious prediction error signals and use them to make inferences about the unknown world. Such ability is especially critical for young learners. Young children have limited knowledge of the world and may often fail to detect surprising events when they do not have the relevant world knowledge to do so. Nonetheless, our study suggests that children possess powerful capacities to learn from other people, and use others’ emotional expressions as rich sources of information to gain information about the world.

These results are broadly consistent with previous work showing that infants and children can recover rich unobserved information (e.g., hidden events in the world and internal mental states) from observed emotional cues (Repacholi & Gopnik, 1997; Wu, Haque, & Schulz, 2018; Wu, Muentener, &

Schulz, 2017; Wu & Schulz, 2017, 2018). Such work however, has primarily focused on positive and negative emotional expressions. Our study suggests that surprise has a unique place in the space of emotions; rather than reflecting valence, surprise reflects a violation of expectation, and even infants readily use others’ expressions of surprise to expect an unexpected event in the world. This study thus moves towards a more comprehensive investigation of emotion understanding in early childhood.

14 Yet, more work is needed to establish the robustness of our results. Note that the predicted pattern of results was observed only on the first trial. Although we suspected that the effect would decrease with repeated trials, we did not have an a priori hypothesis about which trials would show the predicted effect.

Although our key prediction – interaction between surprise and probability of test event – was significant across all trials, the strongest test of our prediction is a complete reversal in looking time that we observed only on the first trial. We are planning a preregistered replication of this effect by using a larger sample size estimated from the current results.

Another way to conceptually replicate our results is to find the same effect using different kinds of events. The current study looks at events that vary in probability, capitalizing on prior work on intuitive statistics (i.e., Xu & Garcia, 2008). Future work can look at another type of surprising events: impossible events (e.g., violation of core knowledge in physics; Spelke et al., 1992). Such events have been shown to trigger infants’ exploration and learning (Stahl & Feigenson, 2015); conceptually replicating the effect of others’ surprise using impossible events will add an important step into understanding the role of prediction error in early learning, both directly and vicariously experienced.

Although much future work awaits us, these results demonstrate a striking effect of others’ surprise on infant’s looking time. For decades, infants’ looking time has been assumed to be longer for events that are unexpected or surprising. Our results, however, show that infants’ inferential abilities in social contexts are powerful enough to make surprising events “surprisingly unsurprising.”

15 References

Aslin, R. N. (2007). What's in a look?. Developmental science, 10(1), 48-53.

Berger, A., Tzur, G., & Posner, M. I. (2006). Infant brains detect arithmetic errors. Proceedings of the

National Academy of Sciences, 103(33), 12649-12653.

Bonawitz, E. B., van Schijndel, T. J., Friel, D., & Schulz, L. (2012). Balancing theories and evidence in

children’s exploration, explanations, and learning. Cognitive Psychology, 64, 215–234.

Boyd, R., & Richerson, P. J. (1988). Culture and the evolutionary process. University of Chicago press.

Carey, S. (2009). The Origin of Concepts: Oxford Series in Cognitive Development.

Chandler, M. J., & Lalonde, C. E. (1994). Surprising, miraculous, and magical turns of events. British

Journal of Developmental Psychology, 12, 83–95.

Csibra, G., & Gergely, G. (2009). Natural pedagogy. Trends in Cognitive Sciences, 13(4), 148–153. den Ouden, H. E. M., Friston, K. J., Daw, N. D., McIntosh, A. R., & Stephan, K. E. (2009). A dual role

for prediction error in associative learning. Cerebral Cortex, 19, 1175–1185.

Den Ouden, H. E., Kok, P., & De Lange, F. P. (2012). How prediction errors shape perception, attention,

and motivation. Frontiers in psychology, 3, 548.

Denison, S., Reed, C., & Xu, F. (2013). The emergence of probabilistic reasoning in very young infants:

Evidence from 4.5-and 6-month-olds. Developmental Psychology, 49(2), 243.

Doan, T., Friedman, O., & Denison, S. (2018). Beyond belief: The probability-based notion of surprise in

children. Emotion, 1–12.

Gopnik, A. (2012). Scientific thinking in young children: Theoretical advances, empirical research, and

policy implications. Science, 337(6102), 1623-1627.

Hadwin, J., & Perner, J. (1991). Pleased and surprised: Children's cognitive theory of emotion. British

Journal of Developmental Psychology, 9(2), 215–234.

Harris, P. (1989). Children and emotion: the development ofpsychological understanding.

16 Jackson, I., & Sirois, S. (2009). Infant cognition: going full factorial with pupil dilation. Developmental

science, 12(4), 670-679.

Kulkarni, T. D., Narasimhan, K., Saeedi, A., & Tenenbaum, J. (2016). Hierarchical deep reinforcement

learning: Integrating temporal abstraction and intrinsic motivation. In Advances in neural

information processing systems (pp. 3675-3683).

Phelps, K. E., & Woolley, J. D. (1994). The form and function of children’s magical beliefs.

Developmental Psychology, 30, 385–394.

Repacholi, B. M., & Gopnik, A. (1997). Early reasoning about desires: evidence from 14-and 18-month-

olds. Developmental psychology, 33(1), 12.

Reschke, P. J., Walle, E. A., & Dukes, D. (2017). Interpersonal development in infancy: The

interconnectedness of emotion understanding and social cognition. Child Development

Perspectives.

Ruffman, T., & Keenan, T. R. (1996). The belief-based emotion of suprise: The case for a lag in

understanding relative to false belief. Developmental Psychology, 32(1), 40-49.

Schultz, W. (2016). Dopamine reward prediction-error signalling: a two-component response. Nature

Reviews Neuroscience, 17(3), 183-195.

Schulz, L. (2012). The origins of inquiry: inductive inference and exploration in early childhood. Trends

in Cognitive Sciences, 16(7), 382–389.

Scott, R. M. (2017). Surprise! 20-month-old infants understand the emotional consequences of false

beliefs. Cognition, 159, 33-47.

Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran,

D., Graepel, T., Lillicrap T., Simonnyan, K., & Hassabis, D. (2018). A general reinforcement

learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419), 1140–

1144.

17 Spelke, E., Breinlinger, K., Macomber, J., & Jacobson, K. (1992). Origins of knowledge. Psychological

Review, 99(4), 605–632.

Stahl, A. E., & Feigenson, L. (2018). Violations of Core Knowledge Shape Early Learning. Topics in

cognitive science.

Wellman, H. M. (2014). Making minds: How theory of mind develops. Oxford University Press.

Wellman, H. M., & Banerjee, M. (1991). Mind and emotion: Children's understanding of the emotional

consequences of beliefs and desires. British Journal of Developmental Psychology, 9(2), 191-214.

Wilcox, T., Bortfeld, H., Woods, R., Wruck, E., & Boas, D. A. (2005). Using near-infrared spectroscopy

to assess neural activation during object processing in infants. Journal of biomedical optics, 10(1),

011010.

Wu, Y., Haque, J. A., & Schulz, L. E. (2018). Children can use others' emotional expressions to infer their

knowledge and predict their behaviors in classic false belief tasks. In Proceedings of the 40th

Annual Meeting of the Cognitive Science Society (pp. 1193-1198).

Wu, Y., Muentener, P., & Schulz, L. E. (2017). One- to four-year-olds connect diverse positive emotional

vocalizations to their probable causes. Proceedings of the National Academy of Sciences of the

United States of America, 114(45), 11896–11901.

Wu, Y. & Schulz, L. E. (2017). What do you really think? Children’s ability to infer others’ desires when

emotional expressions change between social and nonsocial contexts. In Proceedings of the 39th

Annual Meeting of the Cognitive Science Society (pp. 1363-1368).

Wu, Y. & Schulz, L. (2018). Inferring beliefs and desires from emotional reactions to anticipated and

observed events. Child Development, 89(2), 649-662.

Xu, F., & Garcia, V. (2008). Intuitive statistics by 8-month-old infants. Proceedings of the National

Academy of Sciences, 105(13), 5012–5015.

18 Figure 1. Procedures for Experiments 1 and 2. The Emotion phase highlighted in red was only included in Experiment 2.

19 Figure 2. Results of Experiment 1 (A) and Experiment 2 (B and C). Error bars indicate SE.

A Experiment 1 Replication B Experiment 2 C Experiment 2 of Xu & Garcia (2008) All Four Trials First Trial Second Trial Third Trial Fourth Trial

30 30 30 30 30 30

20 20 Probability 20 20 20 20 Probable Improbable

10 10 10 10 10 10 Looking Time (s) Looking Time (s) Looking Time (s) Looking Time (s) Looking Time (s) Looking Time (s)

0 0 0 0 0 0 Probable Improbable Event Probability Surprise No Surprise Surprise No Surprise Surprise No Surprise Surprise No Surprise Surprise No Surprise Emotion Emotion Emotion Emotion Emotion

20