sign in sign up
<<
Home , Aesthetic emotions, Affect theory, Amusement, Contempt, Desire, Disgust

Downloaded by guest on October 3, 2021 www.pnas.org/cgi/doi/10.1073/pnas.1702247114 emotional of science the advance (https://s3-us-west-1.amazonaws.com/ to . available elicit made emo- are they the emogifs/map.html) of map states interactive In an and tional space. videos categorical of library high-dimensional our addition, complex, emotional Reported a dominance. occupy and states as affective to such in appreciation variation dimensions smooth aesthetic to to correspond others—that and , reported , to of horror to distribution emotion—from of to the gradients uncover analyzing we fuzzy states By are categorical emotional discrete. emotion by of than captured categories rather best between boundaries space the semantic rep- labels, valence are a (e.g., within emotional resented dimensions reported experience Although affective subjective such ). of measured and labels reports commonly categorical capture expe- than that better emotional revealed reported as of analyses varieties Further videos distinct dimensional the rience. 27 that and find elicit discrete we expe- reliably by methods, reported self-report structured which Across to are geometries. extent emotion the of expe- and riences emotional date reported to of studied array short riences richest 2,185 the by frame- elicited examining conceptual states videos, emotional a reported introduce analyze we to work emotion—have the Here of debate. is, space intense semantic categories—that sparked semantic the emotion of of organization between thousands, geometric boundaries states not emotional the reported if of and distribution hundreds, the about with 2017) Claims 9, rep- terms. part, February people review that for in experiences (received 2017 subjective resent, 7, in August centered approved and are NY, York, New University, York New LeDoux, E. Joseph by Edited a Cowen S. Alan gradients continuous emotion by of bridged categories distinct 27 captures Self-report any gny fot n teto ifrnit eot femo- of reports cer- differentiate dominance, and as more effort, , other, such emo- tainty, how specific dimensions documented of has affective occurrence inquiry context-directed the of line for related account situations a To or tions, 21). objects 20, specific 18, toward disconnected felt (14, These raw, emotions emotions. to describe opposed specific to as expe- to thought affective moods all are of diffuse have dimensions core finite more emotion the from at a of lie riences, to identify experiences posited and are reported to dimensions—valence arousal—that affective to expe- Efforts two yielded central certain consistently 19). most axes make (18, that of states emotional experience, affective feel emotional core reported riences the of on space semantic focusing the of sions dif- individual well (17). as emotion emotion-related in (16), characterizing ferences physiology in and signaling, to cognition, experi- crucial that expe- the is emotional (1–10), Given reported riences emotion of of 15). space non semantic (14, the qua of sine understanding occur the considered they often of is which variety ence types in rich the a by situations characterized to of readily refer if most that (11–13) hundreds, states terms includes emotional semantic transient that of their space thousands, represent semantic not a People within (1–10). experiences language with resent C eklySca neato aoaoy eateto scooy nvriyo aiona ekly A94720 CA Berkeley, California, of University Psychology, of Department Laboratory, Interaction Social Berkeley n ieo hoiighsdcmne h neligdimen- underlying the documented has theorizing of line One nrlt h cec feoini h rnil htemo- that rep- principle people that the experiences is subjective in emotion centered of are tions science the to entral a,1 n ahrKeltner Dacher and | eatcspace semantic | dimensions a | iceeemotion discrete 1073/pnas.1702247114/-/DCSupplemental at online supporting contains article This 1 aadpsto:A neatv a eae oti td saalbeat available is study this to related map west-1.amazonaws.com/emogifs/map.html interactive An deposition: Data Submission. Direct PNAS a is article This . of conflict no declare authors The A.S.C. paper. and the data; wrote analyzed D.K. A.S.C. and tools; reagents/analytic new contributed A.S.C. research; result, a As situations. varied gra- to response and space in clusters topological relatedness distinct of general both expe- dients involve more emotional simultaneously a reported could within that how organized explore are openly riences to have unable claims theoretical been these testing emotional mod- to been approaches reported has Statistical space est. semantic how a within organized understanding are experiences in progress empirical the emotion. of shaped by study processes—have experience—and, emotional emotion-related cap- their other categories on implication, claims report emotion people Clearly, and how 40–49). dimensions ture 35, affective of 32, specific hundreds 27–29, of that 25, focus (16, the studies discretely been (14). empirical families, have situations emotion categories, similar such into in partitioned dimensions, occur irritation, affective that as with such (39) As such states label, related and prototypical A closely 15, a , contain 38). by to and 37, go “,” 6 (16, may as states family, from emotional emotion accounts or all theoretical cluster, of distribution in the ranging num- describe limited clusters, a that of space. posit describe semantic ber theories , presupposed emotion basic and a precisely, within fear, More states awe, of as clusters such discrete categories, emotion cific (25–36). of activity hundreds phys- combi- behavior, of and Varying to focus iology, experience 22–24). the emotional 19, reported been linking 14, have studies dimensions (1, such anger of and as nations such arousal, or and fear, valence and similar of experiences tional uhrcnrbtos ...adDK eindrsac;ASC n ..performed D.K. and A.S.C. research; designed D.K. and A.S.C. contributions: Author owo orsodnesol eadesd mi:[email protected]. Email: addressed. be should correspondence whom To prxmto fagoercsrcueo eotdemotional an reported experience. comprise of structure results geometric Our a of theories. approximation gra- discrete smooth and to by contrary coherent linked dients, are are a categories in categories many appraisals fashion, Although powerful dimensional arousal. organize dimen- and to affective valence found measured as widely “amuse- such of as sions ratings such by categories than by ment” captured Reported better experience. is emotional experience reported of evocative varieties uncov- emotionally 27 we content, ered 2,185 situational varying richly by with videos elicited short states reports emotional analyze to of methods statistical the Using shaped geo- emotion. have of are space study semantic experiences a within emotional organized reported metrically how about Claims Significance ept h evsv nuneo hs hoeia approaches, theoretical these of influence pervasive the Despite spe- how details experience emotional to approach second A . . www.pnas.org/lookup/suppl/doi:10. NSEryEdition Early PNAS https://s3-us- | f10 of 1

PSYCHOLOGICAL AND PNAS PLUS COGNITIVE SCIENCES little is known about both the boundaries between emotion cat- experience. These varieties of reported experience can be com- egories and their arrangement within a larger semantic space of bined in ways that account for both individual emotion terms emotion, despite these issues’ controversial areas of con- and the collections of terms that comprise reported emotional trasting claims (2, 3, 20, 37, 50). Further, the states that have experiences. In other words, every term and reported experience been studied have been limited in scope, often encompassing corresponds, mathematically, to a single point within the seman- only 6 to 12 categories (46) (although see ref. 24), in large part tic space, determined by a linear combination of the semantic for methodological reasons. Widely used self-report measures, dimensions that define the space. For example, a semantic space such as the Positive and Negative Schedule, capture 10 could have a semantic dimension directly corresponding to the to 12 emotions (51–53). The same is true of widely used affect- term “,” with other terms such as “excitement” and “elation” eliciting stimuli, such as the International Affective Picture Sys- also positioned at various points along this dimension. Alterna- tem (54), and the Gross and Levenson films (55). As a result, tively, the terms joy, excitement, and elation could all correspond the array of emotional states captured in past studies is too to points obtained by applying suitable weights to other seman- narrow to generalize, a priori, to the rich variety of emotional tic dimensions, perhaps including “awe” and . Because this experiences that people deem distinct, including the expanding analysis entails that all reported emotional experiences are lin- array of emotion categories discovered to correspond to dis- ear combinations of the dimensions of a semantic space, it fol- tinct behaviors (16, 56). Also, many claims are founded upon lows that a semantic space can be derived by applying linear studies that apply multivariate techniques such as factor anal- dimensionality reduction techniques to self-report judgments of ysis to self-reports of contemporaneous or recalled emotional emotional experiences. However, accurately deriving a seman- experiences (24, 53, 57). Such studies have accounted for cor- tic space of reported emotional experience may require larger, relations between items (e.g., between valence and awe) but not more diverse samples of experiences than have been typical of the degree to which people agree on individual items that may be past factor analytic studies of the dimensions of reported emo- independent (e.g., reliable judgments of awe that do not simply tional experience (67). reflect positive valence). Further, past multivariate approaches Guided by this theorizing, we set out to use modern large- have relied on heuristic methods that do not generate P values, scale statistical methods and a large, diverse dataset to confidence intervals, or posterior probabilities in estimating how interrogate the semantic space of reported emotional experience many dimensions are needed to account for reported emotional elicited by dynamic visual stimuli. We first gathered the widest experiences (58). array of emotionally evocative stimuli ever studied: 2,185 short This investigation introduces a mathematically based concep- video clips depicting a range of emotional situations. The videos tual framework and empirical approach to characterizing the were gathered by querying search engines and content aggrega- varieties of experience captured by emotional self-report, the tion websites with contextual phrases targeting 34 emotion cat- most widely used measure of experience (1–10). By examin- egories, such as “close call” (targeting relief) and “mushroom ing emotional experiences reported in response to the widest cloud” (targeting awe). The 34 emotion categories were derived array of psychologically significant situations—powerfully evoca- from emotion taxonomies of prominent theorists (for a summary tive video clips—ever studied, we provide answers to the fol- see ref. 16); recent studies of positive emotions such as awe, joy, lowing questions. How many distinct varieties of emotion do love, , and excitement (68–70); Darwin’s observations of people reliably report experiencing across distinct situations? Is emotional states such as , , and reported emotional experience better understood in terms of (71, 72); findings of states found to reliably occur in daily inter- categories, such as amusement and awe, or in terms of widely actions, such as , awkwardness, and calmness (73); and measured affective dimensions, such as valence and effort? Do conceptualizations of nuanced differences between states such as boundaries between emotion categories such as amusement and fear, anxiety, and horror (74) (see Table S1 for a list of states and awe correspond to discrete jumps or smooth transitions in how their theoretical origins). The videos on average lasted about 5 s emotions are reported to be experienced (14, 37, 59)? and portrayed an exceptionally wide range of psychologically sig- We address these conceptual issues by focusing on self-reports nificant situations, including births and babies, weddings and pro- of emotion terms (e.g., anger or “love”), given that self-reports posals, and , and , endearing ani- are currently the most accessible measure of subjective experi- mals, whales and elephants, and , natural ence (1, 9). Self-reports carry information about a person’s inter- and wonders, natural disasters, explosions and warfare, and nal state (1, 9) and correlate with other psychological processes vomit, physical pratfalls, sexual acts, respected and hated celebri- that are thought to be emotion-related, such as expressive behav- ties, nostalgic films, awkward handshakes, delicious food, dance, ior (60–63). Nonetheless, it is important to note that the meaning sports, accidents and close calls, surgeries, risky stunts, soldiers of emotional self-report is subject to ambiguities, including basic returning home, and many others. indeterminacies of linguistic reference (64), difficulties in captur- We presented the emotionally evocative videos from this ing certain subjective phenomena in words [e.g., moods, mem- library to participants on Amazon Mechanical Turk to obtain ories, inchoate physical sensations, and automatic appraisals repeated (9–17) judgments of emotional states elicited by each (65)], differences in the granularity of emotional awareness and video. More specifically, participants provided one of three kinds expression (66), and influences on language use of culture, gen- of reports of emotional experience in response to a random sam- der, and social class. The subjective experience of emotion is pling of videos. One group offered free response interpretations shaped by many complex processes that self-report measures of their emotional response to each of 30 videos, thus allowing only partially capture; self-report is not a direct readout of us to ascertain which categories of emotion spontaneously arise experience. in people’s relatively unconstrained reports of subjective expe- Heedful of these considerations, in our theorizing we begin rience (75). A second group of participants rated each of 30 from the broad assumption that self-reported emotional experi- videos in terms of the degree to which it made them feel the 34 ences correspond to points within a semantic space. Such a space emotion categories of interest, allowing us to interrogate in finer is characterized by its dimensionality—the number of indepen- detail the structure of the reported emotional states correspond- dent directions in the space—and the distribution of all emo- ing to this set of categories. A final group of participants rated tional experiences that people can report along these dimen- each of 12 videos they viewed in terms of its placement along 14 sions. Conceptually speaking, each dimension of this semantic widely measured scales of affective dimensions, which, in varying space corresponds to a distinct variety of reported emotional combinations, are frequently used to measure self-reported

2 of 10 | www.pnas.org/cgi/doi/10.1073/pnas.1702247114 Cowen and Keltner Downloaded by guest on October 3, 2021 Downloaded by guest on October 3, 2021 ote215vdo.S a,ti ol ugs httecate- the tends SH-CCA that fact, vari- (In suggest distinct experience. would semantically emotional 24–26 reported this of least far, eties at So capture reliabil- response ratings videos. the in gorical 2,185 experience explain emotional the to of to required reports combina- participants’ were of linear categories) ity 24–26 the (i.e., of experience tions emotional ( reported 24 of between of (P that scenes 26 found pastoral and we (e.g., evoked SH-CCA others appreciation be Using by aesthetic differentially nature). evoked while may be explosions) differentially awe (e.g., may which stimuli item, to individual some dis- extent each by without of the ratings appreciation,” as in “aesthetic such variance and reliable between words, the awe carding (correlations) other as In variance such studies). shared items, simulated for 2,312 accounts in SH-CCA validation its and Methods see judgments, judgments; dimensional free-response nine-point 276,696 19,710 and multiple- judgments, (27,660 categorical judgments individual choice procedures 324,066 of these total told, a All yielded 77). 76, (24–36, experience emotional oe n Keltner and Cowen reliable independent, (see raters of ratings, across Information judgments the number category Supporting of the in half variance using of of other compared dimensions estimate from are half obtained an simultaneously from averages yielding items the obtained ratings, all to averages the correlated of CCA of the are ratings half awe) the SH-CCA, ratings other (e.g., of In the the item stimuli. from of single obtained across half a averages for from the clip obtained with video averages each anal- the reliability of which split-half (SH- of in generalization analysis ysis, a correlations is canonical SH-CCA split-half devised struc- CCA). we called categories experience, method distinct emotional a of semantically reports many participants’ tured how Experience. Emotional examine Reported of To Varieties Distinct that 27 for categories Evidence 34 the using reported investigation. emotional this be guided distinct can of number that the dif- experiences by reveal rated would Determining reliably observers were (78). dimensions ferent distinct category these each of many for how but loadings uncorrelated components continuous linearly principal are have that deriving ratings—dimensions by of linear the over resolved grouping from concern be conceptual This cat- can a triumph. one and to dependence if adoration equivalent example, as was for such case, joy, others, the as indepen- be linearly such also are egory, could all latter not are The that categories dent. some generally, that more possibility or, the these synonyms, open However, of fit- emotion. leave categories as of also reported experience 34 findings reliably the all are for one they that descriptions that ting least show in at meaningful results are to These emotion response S1A). in (Fig. rates video above-chance reported be data). significantly to our found at were in categories with observed emotion judgment 34 randomly category all Importantly, choosing of being rates raters level base same simulated (chance the (FDR) from 54% rate one obtained averaging discovery least 27%, [false concordance at raters with for across <0.05], concordance emotion 75% significant that of found elicited category We videos choices. same the 34 the eliciting the of as among we video emotion concordance, given of By a category categories. judging emotion judgment raters 34 multiple in the mean of concordance reliably assessed each significant of we videos elicited rates experiences, videos the emotional many whether distinct how distinct of of test reports reports To elicited elicit experiences? videos distinct emotional Did question: liminary Elicitation. Emotion Results gathered). ings and < alsS1 Tables .)saitclysgicn eatcdimensions semantic significant statistically 0.1) rtclt u ocpuledao sapre- a is endeavor conceptual our to Critical and and S2 i.S2 Fig. o oeifraino h - the on information more for o eal ftemethod the of details for aeil and Materials P < 0.05) aespriiat sdi eotn ntereprec in experience their on response (see free reporting videos the in the and used viewing ratings participants also category shared labels we emotion reliably were (75), the variance self-reports of between dimensions emotion many of how assessed specificity apparent the see dimensionality; of S2.) estimates Fig. conservative overly produce to nttlwr eibysprbei enn ihrsetto (In respect videos. with the by meaning elicited in states separable emotional dimensions—27 the reliably semantic separate total—were on in variance. loading reliable much those contribute on However, to enough loadings consistently insufficiently strong not used were have or they envy) not (e.g., perhaps dimensions do semantic manner, were categories any dependent they where linearly synonyms; dimension approximately as semantic different an dimen- same Where in semantic the category. used distinct on each a coload that on categories given maximally videos, loaded 2,185 emo- sion experiences the of reported by the categories describing elicited 27 in meaning approximately distinct had that tion show Essentially, findings dimensions. had semantic these which any envy, on and loadings negligible and only ; experiences and on empathic coloaded both which of sympathy, anger; on of coloaded experiences experiences which , on fact, and coloaded In admiration; which of awe. distinct triumph, to and as mapped pride not such categories dimensions: emotion categories, number seven single small only were a on there dimen- on loadings semantic inter- load the have of easily to many sions tend more rotation, factor each are After categories. but of they same Factor that components the 1. in span principal Fig. pretable that the in dimensions as semantic to shown dimensions of as space rotation 27 set categories, a factor the 34 yields applied the rotation extract then on to category loadings variance, their the PCA most within used the dimensions explaining first 27 To the we experiences? of judgments, emotional meaning reported the extract to correspond emotional umented reported of varieties distinct semantically experience. 27 to for validity up convergent and as categorical ratings—serves the judgment response between free CCA and cat- ratings the pat- within judgment datasets—SH-CCA independent egorical and linearly con- methods significant two near across the of terns The number of experience. the emotional in 27 vergence of identified varieties interpretations distinct same free-response of reports and shared response choice of (P free we and experience patterns analysis, categorical emotional independent this the variables linearly In between of which variance other. significant sets CCA, each 27 two with using task found of correlate so response each maximally did within free that We combinations the reli- linear “shock”). in also finds we vs. were terms horror) “suspense” words, distinct vs. captured (e.g., other with fear emotion associated In (e.g., of ratings ably terms). varieties categorical response the distinct by free many of how determined use of quency o n er n etei prcaino euyadfeelings and hor- beauty (80), or appreciation (81). and aesthetic awe of interest and (79), fear, desire and roman- ror sexual between and distinctions the love as tic such spe- claims, more theoretical to relevant cific but states nuanced (68–70), between excitement differences as for such also emotion, understud- traditionally positive for of see evidence varieties find summary ied we a do (for only date Not 16). to ref. theories experience emotion reported of by varieties anticipated space distinct than semantic in self- a richer far demonstrate emotion emotions videos from of short to derive response we in for dimensions report criteria 27 stricter under The differed understand significance.) have to may dimensions 26 dimensions and the 25, how 24, with analysis this repeat oadescnen htfre hiemtosmyinflate may methods choice forced that concerns address To o ote2 itntsmni iesosw aedoc- have we dimensions semantic distinct 27 the do How i.S1C Fig. < .1,maigpol’ multiple- people’s meaning 0.01), o ersnaino fre- of representation for NSEryEdition Early PNAS i.S3 Fig. ealso we | f10 of 3

PSYCHOLOGICAL AND PNAS PLUS COGNITIVE SCIENCES videos that evoked similar reported emotional experiences. To plot the 27 varieties of emotion elicited by the videos within this 2D space we use a chromatic map, in which each video is colored uniquely according to the specific varieties of reported emotional experience that it elicited. Specifically, the letters correspond- ing to each video are colored using a weighted interpolation of the colors corresponding to each of the semantic dimensions on which they loaded positively. Thus, smooth gradients between these semantically distinct varieties of reported experience corre- spond to smooth transitions in color. This analysis reveals a com- plex distribution of reported emotional experiences that is nei- ther simply clustered nor simply uniform. Inspection of Fig. 2A does reveal certain clusters of emotional experience, for exam- ple, those of craving (desire), and romantic love, and . At the same time, many categories of emotional experience share smooth gradients with other semantically dis- tinct categories, forming smooth transitions between particular varieties of reported emotional experience. For example, the videos are distributed along smooth gradients from anxiety to fear to horror to disgust, calmness to aesthetic appreciation to awe, and adoration to amusement to awkwardness, among oth- ers. Adjacent semantic dimensions along these gradients, such as anxiety and fear, were elicited by an overlapping set of videos, A Fig. 1. Factor analysis loadings on 27 dimensions of variance within the cat- corroborating the shape of the distribution revealed by Fig. 2 . egorical responses. Statistical analyses revealed that categorical judgments These results reveal that the boundaries between many distinct reliably captured up to 27 separable dimensions of variance, each corre- emotion categories are fuzzy rather than discrete. sponding to a semantically distinct variety of reported emotional experi- In more fine-grained analyses, we find that these fuzzy bound- ence. Here, the first 27 principal components of variance within the categor- aries are highly specific to particular pairs of distinct categories. ical judgments, extracted using principal components analysis (PCA), have As shown in Fig. 2B, anxiety and fear (F and Q) were elicited been rotated into more interpretable dimensions using varimax rotation, by many of the same videos (75 times in total), as were fear which finds dimensions that load on relatively few categories. Categories and horror (Q and R; 55 times), yet anxiety and horror were without maximal loadings on any dimensions (contempt, disappointment, elicited by few of the same videos (just eight times). Further envy, guilt, relief, sympathy, and triumph) were either not judged reliably or were taken as roughly linearly dependent with other more frequently used inspection of Fig. 2B reveals that emotion categories mapped to categories during dimensionality reduction. Categories loading on separate distant locations within the t-SNE space, such as awe and dis- dimensions were reliably separable in meaning with respect to the emo- gust, were seldom elicited by any of the same videos. These find- tional states elicited by the videos. The dimensions we derive from emotion ings converge with that emotion categories “cut nature self-report in response to short videos demonstrate a complexity of emo- at its joints” (20), but fail to support the opposite view that tion structure beyond what has been proposed in most emotion theories reported emotional experiences are defined by entirely indepen- to date, reliably differentiating emotional states as nuanced as aesthetic dent dimensions (82). Based on the distribution of emotional appreciation (i.e., feelings of beauty and awe). states elicited by thousands of videos along 27 semantic dimen- sions, we can infer that the majority of categories of emotion share fuzzy boundaries with one or two other distinct categories, The Distribution of Reported Emotional Experience: “Discrete” Cat- forming conceptually related chains of reported experiences, egories Are Bridged by Continuous Gradients of Meaning. Under- such as that from calmness to aesthetic appreciation to awe. To standing the semantic space of emotion requires examin- illustrate these findings and their conceptual implications, we ing not only the semantic dimensions of reported emotional provide a fully interactive version of Fig. 2A (https://s3-us-west- experience—that is, what distinct varieties of emotional experi- 1.amazonaws.com/emogifs/map.html) in which each video is dis- ence do people report?—but also the distribution of states along played when its position in the map is hovered over with the these dimensions. Such a line of inquiry is germane to an endur- cursor. Inspection of this map confirms qualitatively that within ing question: How can distinct varieties of emotional experience the 27-dimensional semantic space of reported emotional expe- be combined or blended together? Discrete emotion theorists riences, most states occupy continuous gradients as opposed to predict the shape of the distribution to approximate a number discrete clusters. of distinct clusters. Another possibility suggests that emotional states are more evenly distributed along affective dimensions Categorical Labels Explain More Variance in Reported Emotional such as valence and arousal (82). While it is difficult to visual- Experience than Proposed Affective Dimensions. We next com- ize a 27-dimensional point cloud, we can use modern data visu- pared the categorical structure of reported emotional experience alization techniques to interrogate how emotional responses to to the information carried by the 14 affective dimension judg- the 2,185 videos are distributed along the 27 semantic dimen- ments—approach, arousal, attention, , commitment, sions of emotional experience documented in the previous anal- control, dominance, effort, fairness, identity, obstruction, safety, ysis. In Fig. 2A we use a method called t-SNE (83). This method upswing (improvement of conditions), and valence. Some theo- projects high-dimensional data—the 27 dimensions of reported rists have suggested that categorical representations of emotion emotional experience we have uncovered—onto two nonlinear are explained largely by position within a space formed by par- axes, such that the local distances between data points are accu- ticular combinations of these affective dimensions (18, 24, 77, rately preserved while more distinct data points are separated by 82, 84). This claim has led to many studies measuring emotional longer, more approximate, distances. experience using some combination of affective dimensions (25– In Fig. 2A we apply t-SNE to map the 2,185 videos along all 35). To test whether the categorical judgments of emotional 27 semantically distinct varieties of emotional experience, result- states were a function of the affective dimension judgments, ing in a 2D space in which each video is surrounded by other we examined whether the affective dimension judgments could

4 of 10 | www.pnas.org/cgi/doi/10.1073/pnas.1702247114 Cowen and Keltner Downloaded by guest on October 3, 2021 Downloaded by guest on October 3, 2021 saogtetre(u f60 o-egtdfe epnetrs togyvldtn h aeoia aig smaue fsbetv experience. subjective of measures as category Keltner ratings anxiety loading and categorical yet highest Cowen the times), the 55 validating dimensions, R; strongly 22/27 and terms, For (Q response horror videos. free and across top-weighted fear dimensions, 600) were (C judgment of as categorical (out times). total), three the in eight the times onto (just among (75 regressed videos videos is were same same judgments the the response of by free many each elicited by The on elicited seldom video were were each Q) horror and of at (F and loadings map fear individual and interactive of anxiety experiences, significance the example, ( The emotional For see distribution (FDR dimension. reported null (also significantly judgment a loaded of categorical of disgust videos simulation categorical varieties each to which via the on distinct determined horror of video was among to each each judgment dimension gradients judgment of categorical to fear categorical loading coloadings reveals corresponding to highest significant colors map anxiety their of unique resulting to Number from the correspond The (B) of gradients that positively. interpolation the letters loaded weighted as as a they axes load- such using generating these which dimensions, colored along on judgment plotted are categorical dimensions are and 27 videos judgment the alphabetically) data individual on broken distinct videos The stochastic more ties 2,185 axes. separating t-distributed (with the two while of experience. dimension on points loadings emotional data video the reported between to each distances applied of of was local space ings distances, preserves categorical approximate, accurately 27-dimensional more that longer, a method by visualization within points data videos a 2,185 (t-SNE), to embedding responses neighbor emotional average of map 2. Fig. h tutr frpre mtoa xeine mohgainsaog2 eatclydsic aeoia ugetdimensions. judgment categorical distinct semantically 27 among gradients Smooth experience: emotional reported of structure The .5 npiso w aeoia ugetdmnin.Teerslsvldt h mto rdet bevdin observed gradients emotion the validate results These dimensions. judgment categorical two of pairs on <0.05) o rersos em soitdwt ahctgrcljdmn dimension. judgment categorical each with associated terms response free Top ) .W hncutdtenme fisacsin instances of number the counted then We Information ). Supporting https://s3-us-west-1.amazonaws.com/emogifs/map.html). NSEryEdition Early PNAS chromatic A (A) | f10 of 5 A.

PSYCHOLOGICAL AND PNAS PLUS COGNITIVE SCIENCES better explain each of the categorical judgment dimensions, or abrupt changes in associated affective dimensions. To do so, we vice versa. That is, do affective dimension judgments of the regressed from the categorical ratings to the affective dimensions degree to which each video elicits displeasure, arousal, submis- across videos that elicited each of two different emotions to a siveness, and so on, better explain a person’s reports of fear, or significant extent. In these analyses, we found that the smooth does labeling an experience as fear better explain the person’s boundaries documented between particular categories reflected reports of these well-validated affective dimensions? We com- smooth variations in affective dimensions such as arousal and pared these possibilities using cross-validated regression tech- commitment to an individual (Fig. S4). That the affective dimen- niques. With both linear and nonlinear regression (Fig. 3) we sions vary smoothly across gradients between categories calls into found that the affective dimension judgments explained at most question the notion from basic emotion theories that prototypi- 61% of the explainable variance in the categorical judgment cal patterned responses are similar across all instances of a given dimensions, whereas the categorical judgment dimensions consis- category. While each category is associated with a specific pat- tently explained 78% of the explainable variance in the affective tern of affective dimension ratings, these ratings do not shift dimension judgments (Supporting Information). That the categor- abruptly across categories; rather, they vary smoothly along the ical judgment dimensions explained the affective dimension judg- gradients associated with each emotion category. ments substantially better than the reverse (P < 10−6, bootstrap test, both linear and nonlinear regression) suggests that the cat- Unifying Factors of the Meaning of Reported Emotional Experi- egories have more semantic value than the affective dimensions ences. A central claim across emotion theories is that emotional in explicating people’s reports of emotional experience elicited experiences are defined by factors that reflect the coalescence by short videos—that they capture most of what the 14 affective of affective dimensions and categorical labels of current experi- dimensions capture, and more, with respect to the emotions peo- ences (18, 22, 24, 86, 87). To examine the relationship between ple reliably report experiencing in response to evocative videos. the affective dimensions and categories, we ascertained how affec- tive dimensions covary with categorical judgment dimensions of Smooth Category Gradients Correspond to Smooth Differences in elicited emotion (24, 84). We did so by applying CCA between Reported Emotional Experience. The above analyses indicate that the 14 affective dimensions and 27 categorical judgment dimen- judgments of the affective dimensions were largely explained by sions associated with each video (Fig. 4). Here, CCA extracted the categories. How, then, do the affective dimensions vary as linear combinations of affective dimension judgments that corre- a function of the categorical judgments? For instance, do the lated maximally with linear combinations of categorical judgment affective dimensions vary smoothly across category gradients, or dimensions. This analysis yielded 13 significant canonical variates are there sharp jumps in affective dimensions at points where (P < 0.01), shared dimensions of variance between participants’ the most frequently reported category of emotion changes? self-reports of affective dimensions and categorical judgments of The presence of sharp jumps in associated affective dimensions their reported emotional experiences (Fig. S5). Each canonical between neighboring categories would support basic emotion variate might be thought of as a unifying factor, or central com- theories (85), which predict that basic emotions are associated ponent, of the meaning of reported emotional experiences, with with prototypical patterned responses (e.g., subjective, behav- both a categorical and affective dimension loading. ioral, and physiological) that are similar across all instances of Close inspection of Fig. S5 reveals how affective dimensions a given category but different, in discrete fashion, from the pat- relate to categories of emotional experience (for similar analy- terned responses of other emotions (22, 38, 39). In light of sis see ref. 24). For instance, the first canonical variate corre- these conceptual claims, we tested whether gradients between sponds almost exclusively to valence, the same initial dimension distinct reported emotional experiences reflected smooth or uncovered in nearly all factor analytic studies of reported emo- tional experience (18), and differentiates experiences of highly positive states (calmness and joy) from the most negative states (horror and empathic pain). More social affective dimensions such as feelings of dominance and approach likewise drive reported emotional experience. For example, judgments of approach and dominance, variates 4 and 6 in Fig. S5, account for variation in reports of anger (76, 88, 89). Judgments of commit- ment account for variation in reports of emotional experiences such as sadness and adoration that are closely related to attach- ment processes (90). To visualize each canonical variate, we project both its cate- gorical and affective dimension loadings onto our t-SNE map. In Fig. 4 B–E, loadings of the first six canonical variates—both cat- egorical (B and D) and affective dimension (C and E)—are pro- jected as colors onto the t-SNE map. Each color channel (red, green, and blue) in Fig. 4 B–E corresponds to a canonical vari- ate. Hence, the extent of similarity in color between B and C as Fig. 3. Variance explained by the categorical judgments in the affective well as between D and E indicate the extent to which linear com- dimension judgments, and vice versa. The categorical judgment dimen- binations of category judgments (B and D) are similar in meaning sions explain significantly more variance in the affective dimension judg- to particular linear combinations of affective dimensions (C and ments than vice versa (P < 10−6, bootstrap test). These findings hold when E). The maps allow us to further interpret some of the gradi- using both linear regression with ordinary least squares (Left) and nonlinear ents observed along the 27 varieties of reported emotional expe- regression with k-nearest neighbors (Middle). This suggests that the cate- riences represented in Figs. 1 and 2A. For example, the gradient gories have the most value in explicating reported emotional experiences from disgust to sadness is associated with a relative increase in elicited by short videos. (Explained variance was calculated using leave-one- out-cross-validation and then divided by the estimated explainable variance. commitment to an individual. For this analysis, we used nine ratings per video. For k-nearest neighbors, we Discussion tested k from 1 to 50 and show the results from choosing the optimal k, i.e., the one that resulted in the greatest average explained variance for each Central to the science of emotion are claims about the seman- prediction. See also Supporting Information. tic space of reported emotional experience: How many distinct

6 of 10 | www.pnas.org/cgi/doi/10.1073/pnas.1702247114 Cowen and Keltner Downloaded by guest on October 3, 2021 Downloaded by guest on October 3, 2021 uget o hs he iesoso mtoa xeine aeso ahmprflc h obnto flaig ntetredmnin htgive that dimensions dimensional three and the categorical on also the loadings (see between of dimension information combination affective shared the in of reflect differences degree map smooth the each to illustrates on correspond (D F Labels color. and experience. each E emotional to between of colors rise dimensions in three Similarity these map. for each judgments for (see titles dimensions the (P in affective significant given be the to on found (C coefficients were variates its dimensions affective interpreting the by and variate dimensions judgment categorical the between 4. Fig. oe n Keltner and Cowen dis- richer of a space involves English The of in (56). experiences upwards signals emotional reported that nonverbal tinct distinct suggesting have literature states signaling 20 associ- emotion developments reliably recent the with are in converges experience situations distinct that reported with finding of ated Our experience varieties to videos. of distinct response short varieties 27 in evocative reported 27 emotionally reliably 2,000 to emotion over up of categories for experi- the evidence of from obtain categories we reported to ence, of methods dimensionality statistical the Using determine experience. emotional reported of self-report. of of indeterminacies heedful and videos, ambiguities short the evocative different emotionally three the 2,185 using to responses, methods, interrogate emotional partic- their To had on we report experience, ipants experience. emotional semantic reported reported of space a semantic of to within corresponding varieties dimensions points distinct semantic as emotional along characterized of distributed space, reports be that can positing experience framework mathematical a dimensions categorization? affective emotion proposed of to relate judgments that factors What vs. unifying arousal? and the state, valence are as the experience such of affect emotional of labeling dimensions proposed reported categorical the is in extent rooted emotional what are reported To What of varieties experiencing? experience? distinct report between people boundaries do the emotion of varieties u rtfidn ocrstercns ftesmni space semantic the of richness the concerns finding first Our from emerge investigation present the from findings The aoia orltosaayi ewe h aeoia ugetdmnin n h fetv iesos ( dimensions. affective the and dimensions judgment categorical the between analysis correlations Canonical o h rttrecnnclcreain,poetda e,gen n leclrcanl notetSEmpfo i.2 Fig. from map t-SNE the onto channels color blue and green, red, as projected correlations, canonical three first the for ) and E Like ) B and u o h orhtruhsxhcnnclcreain.Altogether, correlations. canonical sixth through fourth the for but C, i.S4 Fig. o nlsso rdetsmoothness). gradient of analysis for i.S5 Fig. ey er orr n igs;adanme fohremotion other of number a and disgust; and horror, anx- linking appreciation; fear, gradients aesthetic iety, observe and we awe, where admiration, 2A, of Fig. mean- experiences in see affective can in cat- of one gradients other as categories smooth ing, one to least most correspond at stimulus, These with egory. on single gradients continuous a constraints share to be simul- emotion reported response may be in can gradients. there experiences taneously continuous emotional that of and suggest varieties which findings clusters our discrete While both framework, statistical identify open-ended within an can states using distributed emotional space elicited evenly dimensional a of more distribution which are approach, the Our emotion they interrogates (82). or dimensions that independent clusters more or distinct across emotional 37–39) of that (16, number either ways: limited families two a of emo- occupy emo- one of in states distribution of shaped the is structure that states suggested the tional have regarding theorists matter Past debate another tion. theoretical another, one to to relation central in distributed are ences shape may of that varieties experience behavior. rich emotional the Nev- reported reveals incorporated. reliably investigation be present to emotional need the will stimuli, of ertheless, cultures of modalities other types other and other taxonomy self-report, response, of definitive to studies the approaches is which no other this for By that states, (37). claim field emotional to the of mean in we earlier do considered means than states of variety o h ofcet) (B coefficients). the for u etfidn ocrshwrpre mtoa experi- emotional reported how concerns finding next Our and < C B–E .Ctgrclvrae ( variates Categorical ). .1.W sindlbl oec canonical each to labels assigned We 0.01). lutaeta h aeoia gradients categorical the that illustrate h rt1 aoia correlations canonical 13 first The A) NSEryEdition Early PNAS oo eed are legends Color A. n dimensional and B) | f10 of 7

PSYCHOLOGICAL AND PNAS PLUS COGNITIVE SCIENCES categories. These findings suggest a far more complex distribu- tent with findings by Kragel and LaBar (96) that representa- tion of emotional states than the clustered or more uniform dis- tions of subjective emotional experiences are found in high- tributions hinted at in discrete and dimensional theories (16, level brain regions (e.g., ). Our findings 37–39, 82). These findings also raise intriguing questions war- raise intriguing questions about how such brain regions might ranting further research. For example, the smooth gradients of encode the dozens of distinct varieties of emotion that we have affective meaning we document may account for how people uncovered. transition from one experience to the next (e.g., from admira- It is worth noting important limitations of the present investi- tion to awe; see ref. 91), and for mixed emotional experiences gation. As in so many studies of emotional experience, the con- (92, 93). clusions we might draw depend on the degree of correspondence Finally, our findings speak to the question of how people con- between self-reports and subjective experiences. As noted ear- ceptualize their emotional experiences in semantic terms. When lier, some aspects of emotional experience may elude self-report, participants were asked to judge their emotional state by choos- and there are other potential determinants of self-report besides ing from a list of 34 categories or by placing their experiences emotional experience. On this latter point, we note that reported along 14 different dimensional scales of affective appraisal and emotional experience may reflect a combination of three con- motivation, the categorical judgments more powerfully explained ceptually distinct phenomena: (i) emotional experience itself; (ii) variance in the affective dimension judgments than vice versa cognitive and perceptual experiences that may not in their own (Fig. 3). Categorical labels organize affective dimensions in a right be considered emotional, but may nevertheless color how coherent and powerful fashion. It is important to recognize that an emotional experience is labeled (18); or (iii) of the most current constructivist and appraisal theories seldom affective , that is, the emotional experience that a situ- propose that specific dimensions offer an exhaustive descrip- ation could potentially cause or should cause according to cul- tion of reported emotional experience (94). Nevertheless, hun- tural norms of emotional experience (18, 97). Future research dreds of studies of emotion-related behavioral, cognitive, phys- will need to systematically examine such processes, to the extent iological, and neural effects have focused on measurements of possible, to further characterize the semantic space of emotional valence, arousal, and other specific affective dimensions (24–36, experience. 76, 77). The present findings suggest that reported emotional It is also important to mention that we have focused on com- experience is more precisely conceptualized in terms of cate- monalities in the emotions people report experiencing in each gories more often put forward by discrete emotion theories (16, situation; individuals also differ, often in striking fashion, in their 37–39), although, contrary to discrete theories, we find that the reports of emotional responses to a given situation (98, 99). How- boundaries between emotion categories are fuzzy rather than ever, the results of an additional analysis we performed suggest discrete in nature. that differences such as gender, age, social class, and personal- Our findings dovetail with recent findings related to the neu- ity factors explained at most a small proportion of the variance ral representation of emotional experience. Most notably, Kragel in reported emotional experience compared with commonalities and LaBar (95) decoded emotional experiences from concomi- across participants (Fig. S6). Nevertheless, it will be crucial for tant fMRI of the human brain, documenting that distributed future studies to examine how such culture-related sources of patterns of brain activity distinguish among discrete emotions. variation in emotion self-report shape the structure of semantic Specifically, experiences of amusement, anger, , spaces of emotional experience. fear, sadness, and surprise could be discriminated with above- Granting these limitations and caveats, our results reveal how chance accuracy based on patterns of brain activation, even when emotion concepts are reliably structured in their association with classifiers were trained and tested on emotional states elicited by distinct situations. These findings have generative implications separate modalities of stimuli (film and ). In light of these for studies that relate reported experience to behavior, physi- findings, the results of the present investigation raise the intrigu- ology, and individual differences (14). With respect to neuro- ing possibility that distinct patterns of neural activation might physiology, for example, hundreds of studies of the brain regions distinguish among a much broader array of states than have been activated during reported emotional experiences have focused investigated so far, such as the many positive states we have doc- on valence and arousal or the six basic emotions (25, 26, 28, 29, umented here (e.g., aesthetic appreciation and awe) and may 34, 35, 42, 44, 47–49), leaving out the many other varieties of reflect continuous gradients rather than discrete categories. New reported emotional experience that we find reliably occur in dis- fMRI modeling approaches could fruitfully be combined with the tinct situations and that could potentially be represented in dis- emotion elicitation and multidimensional reliability estimation tinct brain activity patterns. techniques introduced here to determine the number of distinct Questions about the structure of reported emotional experi- varieties of emotion that are represented by distinct patterns of ences are foundational to the science of emotion. Answers to neural activity. such questions bear upon the most central theoretical claims in The present findings regarding the structure of self-reported the field. Our conceptualization of how emotional self-reports emotional experiences may also inform recent theoretical are situated within a semantic space and our geometric analytic efforts to explicate how such experiences are generated. For techniques have yielded more nuanced, complex answers than is example, in the higher-order theory of emotional conscious- typical in the theorizing that has sparked such intense debate. ness recently put forward by LeDoux and Brown (15), it Reported emotional experiences inhabit at least 27 dimensions is posited that emotional experiences are introspective states associated with reliably distinct situations and are distributed defined by schematic representations of psychologically signif- along continuous gradients between particular emotion cate- icant situations, and emotion terms are symbolic representa- gories within this space. With analytic methods, and by studying tions of these states. Cast within this theorizing, the reliabil- the widest array of emotions and elicitors to date, we have uncov- ity that we observed across participants in their reported emo- ered an approximation of a geometric structure of reported emo- tional responses to particular situations—viewing short videos— tional experience. emerges from commonalities in the symbolic structures partici- It will be important to extend these methods and findings to pants relied upon to label their emotional reactions to the evoca- studies of other emotion elicitors, such as music, daily activities, tive stimuli. The higher-order theory of emotional conscious- and social interactions. It will also be critical to ascertain geo- ness also predicts that emotional states are represented in parts metric structures of reported emotional experience within other of the brain responsible for higher-order cognition (15), consis- cultures and their languages, given that here we have only studied

8 of 10 | www.pnas.org/cgi/doi/10.1073/pnas.1702247114 Cowen and Keltner Downloaded by guest on October 3, 2021 Downloaded by guest on October 3, 2021 1 or ,e l 21)Nrso aec,aosl oiac,adaeo acquisition of age and dominance, arousal, valence, of Norms (2013) al. et A, Moors 31. Approaches, studies: emotion and music of review the A during (2013) arousal JK and Vuoskoski T, valence of Eerola subserving perception systems and 27. Neural experience (2010) the al. in et language T, of Colibazzi role 26. The (2016) al. et JA, Brooks 25. emotion. of theory cognitive-motivational-relational a on Progress (1991) RS Lazarus 22. 0 as B oisnM 20)Maue feoin review. A emotion: of Measures (2009) MD Robinson IB, Mauss 30. pos- of basis brain The (2016) LF Barrett J, Weber Con- TD, Wager brain: AB, Satpute the KA, onto Lindquist emotions 29. dimensional and discrete Mapping (2012) S Hamann 28. emotion. in appraisal cognitive of Patterns (1985) PC Ellsworth CA, emotions. Smith discrete 24. of determinants Appraisal (1991) IJ Roseman 23. way. right the in experience emotional into feelings Getting (2009) kinds? P natural Goldie emotions Are 21. (2006) component LF the Barrett for Evidence 20. emotion: of architecture dynamic The (2009) emotion. KR of Scherer construction psychological 19. the and affect Core (2003) JA Russell 18. 7 oebr L(98 eeso nlssadteognzto faffect. of organization the and analysis of Levels (1998) EL Rosenberg 17. 6 ete ,Lre S(00 Emotion. (2010) JS . Lerner emotional D, of Keltner theory higher-order 16. emotion. A of (2017) R model Brown occ JE, the LeDoux in construction 15. Psychological (2013) A Ortony GL, explo- Further Clore : Emotion 14. (1987) C O’connor D, pair. Kirson neatly J, don’t Schwartz P, experiences Shaver and names 13. emotion Why (2005) M Silver J, emotions. Sabini of categorization 12. the and Culture (1991) JA Russell 11. K 10. oe n Keltner and Cowen prvdb h ntttoa eiwBada h nvriyo California, of were See University . mation procedures informed the the experimental their at gave in The participants Board All y). part Review Berkeley. 36 Institutional took the = participants by age US approved mean English-speaking Mechani- females, 853 Amazon (403 of using study total obtained A were Turk. videos cal the of judgments Emotion Methods geo- and Materials a within arranged are how states space. of emotional metric understanding how an toward are shift emotions to again nonverbal once and expression, response, system nervous central be physio- response, peripheral logical could emotion-related of here studies developed to applied methods fruitfully The English (100). using concepts participants emotion US by reported experience emotional .Rbno D lr L(02 eifadfeig vdnefra cesblt model accessibility an for Evidence : and (2002) GL Clore MD, consciousness. Robinson and 9. Emotion (2007) and R Adolphs in N, feelings Tsuchiya emotional emo- 8. Core about consciousness: talk Affective we when (2005) about J talk Panksepp we What 7. (2016) C Alberini L, Phelps J, LeDoux 6. experience. hedonic and Emotions (2003) NH Frijda 5. world. the of knowledge and feeling, Emotion, (2004) P Goldie 4. .IadC 20)Eoinfeig tmfo vlto n erbooia develop- neurobiological and from stem feelings Emotion (2007) CE . Izard new a and 3. schemas, emotion kinds, natural emotion. emotions, of Basic (2007) experience CE The Izard (2007) JJ 2. Gross KN, Ochsner B, Mesquita LF, Barrett 1. o ,0 uc words. Dutch 4,300 for 237. stimuli. and models, emotion emotions. induced of experience meta-analysis. neuroimaging A emotion: Psychol Psychol Am 1:232–239. tv n eaieafc:Eiec rmamt-nlsso h ua neuroimaging human the of meta-analysis a literature. from Evidence affect: negative and itive consensus. and troversies 200. model. process Rev 2:247–270. ibr T ide Wly e York). New (Wiley, G Lindzey DT, Gilbert self-report. emotional of 167. . tions. York). New Psychology Press, nic Univ (Oxford RC Solomon ed Emotions, Oxford). on Philosophers Contemporary ing: alAa c USA Sci Acad Natl Rev Emot approach. prototype a of ration Inq chol 450. 2007. al., et Barrett for Corrections Sci acts: conceptual from not ment, Sci Psychol Perspect Psychol Rev vce (2003) Z ovecses ¨ 2:404–405. 110:145–172. o details. for Cell 48:813–838. 16:1–10. osiu Cogn Conscious 5:335–343. 167:1443–1445. ee Cortex Cereb 46:819–834. 58:373–403. d anmnD cwr ,Dee RselSg Foundation, (Russell E Diener N, Schwarz D, Kahneman eds , onEmot Cogn eahradEmotion and Metaphor 114:E2016–E2025. 2:260–280. ea e Methods Res Behav 26:1910–1922. 14:30–80. sco Bull Psychol rnsCg Sci Cogn Trends 23:1307–1351. ui Percept Music esScPsychol Soc Pers J Emotion 128:934–960. adoko oilPsychology Social of Handbook o onAfc Neurosci Affect Cogn Soc 16:458–466. CmrdeUi rs,Cmrde UK). Cambridge, Press, Univ (Cambridge 10:377–389. 45:169–177. 30:307–340. es sco Sci Psychol Persp 52:1061–1086. elBig onain fHedo- of Foundations Well-Being: rnsCg Sci Cogn Trends sco Bull Psychol 1:28–58. hnigAotFeel- About Thinking uprigInfor- Supporting onEmot Cogn onEmot Cogn 12:169–183. esetPsychol Perspect e e Psychol Gen Rev d ik ST, Fiske eds , esSoc Pers J mtRev Emot 110:426– 23:209– 11:158– Psychol 5:161– Proc Ann Psy- 6 ec C lrsS,BnhS 21)Dsrt mtospeitcagsi cogni- in changes predict emotions Discrete (2011) SW Bench SA, Flores HC, Lench 46. and social in emotion of expression and Suppression (2017) C Hunt E, Chervonsky emotions 41. discrete of influence The (2011) V Kligyte EP, Waples S, basic. Connelly AD, are Angie emotions 40. All (1994) P basic. Ekman emotions calling 39. by meant is What (2011) D Cordaro P, Ekman 38. 8 hnK,WgrT alrS,Lbro 20)Fntoa eraaoyo emotion: of neuroanatomy Functional (2002) I Liberzon SF, Taylor T, emo- Wager of KL, neuroanatomy Phan Functional 48. (2003) AD Lawrence I, Nimmo-Smith FC, Murphy 47. review. A emotion: in activity system nervous Autonomic (2010) estimation SD sadness: likelihood Kreibig activation and An 45. mapping: fear Affective just (2015) JL Not Robinson LA, (2012) Kirby R 44. Palermo E, McKone R, O’Kearney A, activa- Dawel of 43. Predictors (2008) CH Fu AS, David MJ, Brammer SG, Costafreda 42. about. agree emotion study An who processing: scientists picture What Affective (2016) (2008) P J Ekman Polich 37. H, Sequeira S, Nordin JK, Olofsson (2008) 36. al. et TD, Wager 35. of lateralization and gender, Valence, (2003) SF Taylor I, Liberzon KL, Phan TD, Wager chal- 34. emotion audio/visual international first 2011–The Avec (2011) al. and et emotion B, Schuller measuring and 33. Inducing (2014) LF Barrett KA, Lindquist KS, Quigley 32. 9 art F usl A(2014) JA Russell LF, Barrett guide. user’s A 59. analysis: factor Exploratory (1993) T Cox affect. E, of Ferguson circumplex A 58. (1980) J Russell 57. international the from images on data category Emotional (2005) al. et measures JA, brief Mikels of validation 54. and Development (1988) A Tellegen LA, question- Clark emotions D, discrete Watson The 53. (2016) E Harmon-Jones B, Bastian and C, flourishing assess Harmon-Jones to scales 52. Short measures: well-being New (2010) al. et E, Diener con- appraisal-based 51. How affects: of psychology the Neurologizing (2007) J Panksepp basic of correlates 50. neural discrete for support Neuroimaging (2010) S Hamann K, Vytal 49. 6 ete ,CraoD 21)Udrtnigmlioa mtoa expressions: emotional multimodal Understanding (2015) and DT self-report, Cordaro Physiology, D, suppression: Keltner Emotional 56. (1993) RW Levenson JJ, Gross 55. h onTmltnFoundation. Templeton John the ACKNOWLEDGMENTS. the within videos that dragged. Note and database. clicked our be can within map filename its as “.mp4,” by serves followed which, also tag, numeric unique extended video’s an the for reveal video will each period to corresponding both number In the content). over floating graphic the maps, extremely viewing contains in again, in discretion which, “uncensored” careful map, word exercise uncensored the please with (although “map” URL word previous the the replacing by read- 18+ to age available also of is ers map chromatic uncensored an However, map.html). (https://s3-us-west-1.amazonaws.com/emogifs/ map paper the chromatic in the elsewhere in linked blurred sexual are content and/or graphic , highly violence, with Videos graphic content. highly contain which of the many viewing videos, in discretion exercise Please goo.gl/forms/XErJw9sBeyuOyp5Q2. h ,8 iesadterma aig a erqetdhere: requested be can ratings mean their and videos 2,185 The in uget xeine eair n hsooy eaaayi fexperimen- of elicitations. meta-analysis emotion A tal physiology: and behavior, experience, judgment, tion, chol meta-analysis. A outcomes: interpersonal review. meta-analytic A 1422. decision-making: and on 15–19. pp York), New Press, Univ (Oxford R Davidson 370. 11:31–34. eaaayi feoinatvto tde nptadfmri. and pet in studies activation emotion of meta-analysis A meta-analysis. A tions: meta-analysis. (ALE) vocal and facial for psychopathy. deficits in recognition expressions emotion pervasive of evidence Meta-analytic fmri and pet 385 of meta-analysis A stimuli: emotional studies. of processing the during tion findings. erp of review integrative 249–271. pp 3, Vol York), New (Guilford, LF Barrett JM, Jones neuroimag- from findings of meta-analysis A emotion: ing. in brain functional 415–424. pp Berlin), (Springer, lenge. Psychology ality secrets. and tricks, Tips, affect: e York). New 1:84–94. socrates.berkeley.edu/∼keltner/publications/keltner&Cordaro%202016.pdf. at able system. picture affective scales. panas The affect: negative and positive of emotions. self-reported state measuring for tool new A naire: feelings. negative and positive coexist. can theory emotion basic and structivism meta-analysis. voxel-based A emotions: 348. eetavne nbsceointheory. emotion basic in advances Recent behavior. expressive Neuroimage 84:394–421. nentoa ofrneo fetv optn n nelgn Interaction Intelligent and Computing Affective on Conference International ri e Rev Res Brain dJd J(abig nvPes e ok,p 220–252. pp York), New Press, Univ (Cambridge CJ Judd ed , 19:513–531. 10.1016/j.bandc.2015.04.006. Cogn, Brain esScPsychol Soc Pers J 58:57–70. onAfc ea Neurosci Behav Affect Cogn hsrsac a upre npr yagatfrom grant a by part in supported was research This sco Bull Psychol ea e Methods Res Behav d ei ,Haviland- M, Lewis eds Emotion, of Neuroimaging The ersiBoea Rev Biobehav Neurosci h scooia osrcino Emotion of Construction Psychological The adoko eerhMtosi oiladPerson- and Social in Methods Research of Handbook o ni Res Indic Soc ilPsychol Biol 137:834–855. 64:970–986. esScPsychol Soc Pers J onNeurosci Cogn J mto Researcher Emotion Emotion d ka P, Ekman eds Emotion, of Nature The 97:143–156. 37:626–630. 77:247–265. esScPsychol Soc Pers J es sco Sci Psychol Persp 17:669–683. 36:2288–2304. 3:207–233. NSEryEdition Early PNAS 22:2864–2885. 36:1152–1168. dSaatn .Avail- A. Scarantino ed , LSOne PLoS onEmot Cogn Neuroimage 2:281–296. 54:1063–1070. n eetAssess Select J Int es sco Sci Psychol Persp mtRev Emot 11:83–111. | (Guilford, 25:1393– ilPsy- Biol https:// 16:331– f10 of 9 3:364–

PSYCHOLOGICAL AND PNAS PLUS COGNITIVE SCIENCES 60. Ekman P, Freisen WV, Ancoli S (1980) Facial signs of emotional experience. J Personal 95. Kragel PA, LaBar KS (2015) Multivariate neural biomarkers of emotional states are Soc Psychol 39:1125–1134. categorically distinct. Soc Cogn Affect Neurosci 10:1437–1448. 61. Brown S, Schwartz G (1980) Relationships between facial electromyography and sub- 96. Kragel PA, LaBar KS (2016) Decoding the nature of emotion in the brain. Trends Cogn jective experience during affective imagery. Biol Psychol 11:49–62. Sci 20:444–455. 62. Mauss IB, Levenson RW, McCarter L, Wilhelm FH, Gross JJ (2005) The tie that 97. Russell JA (2017) Mixed emotions viewed from the psychological constructionist per- binds? coherence among emotion experience, behavior, and physiology. Emotion 5: spective. Emot Rev 9:111–117. 175–190. 98. Siemer M, Mauss I, Gross JJ (2007) Same situation–different emotions: How appraisals 63. Matsumoto D, Keltner D, Shiota MN, O’Sullivan M, Frank M (2008) Facial expressions shape our emotions. Emotion 7:592–600. of emotion. Handbook of Emotions, eds Lewis M, Haviland-Jones JM, Barrett LF (Guil- 99. Barrett LF (2004) Feelings or words? understanding the content in self-report ratings ford, New York), Vol 3, pp 211–234. of experienced emotion. J Pers Soc Psychol 87:266–281. 64. Quine WV (1970) On the reasons for indeterminacy of translation. J Philos 67:178– 100. Mesquita B, Frijda NH, Scherer KR (1997) Culture and emotion. Handbook of Cross- 183. Cultural Psychology, eds Berry JW, Dasen PR, Saraswathi TS (Oxford Univ Press, 65. Moors A (2010) Automatic constructive appraisal as a candidate cause of emotion. Oxford), Vol 2, pp 255–297. Emot Rev 2:139–156. 101. Shaver PR, Morgan HJ, Wu S (1996) Is love a “basic” emotion? Personal Relationships 66. Lindquist KA, Barrett LF (2008) Emotional complexity. Handbook of Emotions, eds 3:81–96. Lewis M, Haviland-Jones JM, Barrett LF (Guilford, New York), pp 513–530. 102. Silvia PJ (2005) Cognitive appraisals and interest in visual art: Exploring an appraisal 67. MacCallum RC, Widaman KF, Zhang S, Hong S (1999) Sample size in factor analysis. theory of . Empirical Studies 23:119–133. Psychol Meth 4:84–99. 103. Ruch W (1993) Exhilaration and humor. Handbook of Emotions, eds Lewis M, 68. Shiota MN, Neufeld SL, Yeung WH, Moser SE, Perea EF (2011) Feeling good: Auto- Haviland JM (Guilford, New York), pp 605–616. nomic nervous system responding in five positive emotions. Emotion 11:1368–1378. 104. Rozin P, Lowery L, Imada S, Haidt J (1999) The CAD triad hypothesis: A mapping 69. Shiota MN, et al. Beyond : Toward a science of discrete positive emotions. between three (contempt, anger, disgust) and three moral codes Am Psychol, in press. (community, autonomy, divinity). J Pers Soc Psychol 76:574–586. 70. Fredrickson BL (1998) What good are positive emotions? Rev Gen Psychol 2:300–319. 105. Perkins AM, Inchley-Mort SL, Pickering AD, Corr PJ, Burgess AP (2012) A facial expres- 71. Darwin C (1872/1998) The Expression of the Emotions in Man and Animals (Oxford sion for anxiety. J Pers Soc Psychol 102:910–924. Univ Press, New York). 106. Spielberger CD (1972) Anxiety as an emotional state. Anxiety: Current Trends in The- 72. Keltner D (2009) Born to be Good (Norton, New York). ory and Research, ed Spielberger CD (Academic, New York), Vol 1, pp 23–49. 73. Rozin P, Cohen AB (2003) High frequency of facial expressions corresponding to con- 107. Keltner D, Tracy J, Sauter DA, Cordaro DC, McNeil G (2016) Expression of emotion. fusion, concentration, and in an analysis of naturally occurring facial expres- Handbook of Emotions, eds Barrett LF, Lewis M, Haviland JM (Guilford, New York), sions of Americans. Emotion 3:68–75. 4th Ed, pp 467–482. 74. Reiss S (1991) Expectancy model of fear, anxiety,and . Clin Psychol Rev 11:141– 108. Berridge KC, Robinson TE (2003) Parsing reward. Trends Neurosci 26:507–513. 153. 109. Zeelenberg M, et al. (1998) Emotional reactions to the outcomes of decisions: The 75. Russell JA (1994) Is there universal recognition of emotion from facial expressions? A role of counterfactual thought in the experience of and disappointment. Orga- review of the cross-cultural studies. Psychol Bull 115:102–141. nizational Behav Human Decision Processes 75:117–141. 76. Mehrabian A, Russell J (1974) An Approach to Environmental Psychology (MIT Press, 110. Rozin P, Fallon AE (1987) A perspective on disgust. Psychol Rev 94:23–41. Cambridge, MA). 111. Singer T, et al. (2004) for pain involves the affective but not sensory compo- 77. Osgood CE (1966) Dimensionality of the semantic space for communication via facial nents of pain. Science 303:1157–1162. expressions. Scand J Psychol 7:1–30. 112. Valentine CW (2015) The Experimental Psychology of Beauty (Routledge, 78. Hotelling H (1933) Analysis of a complex of statistical variables into principal compo- Abingdon, UK). nents. J Educ Psychol 24:417–441. 113. Van de Ven N, Zeelenberg M, Pieters R (2009) Leveling up and down: The experiences 79. Diamond LM (2003) What does sexual orientation orient? a biobehavioral model dis- of benign and malicious envy. Emotion 9:419–429. tinguishing romantic love and sexual desire. Psychol Rev 110:173–192. 114. Ohman¨ A (1986) Face the beast and fear the face: and social as proto- 80. Reeve J (1993) The face of interest. Motiv Emot 17:353–375. types for evolutionary analyses of emotion. Psychophysiology 23:123–145. 81. Keltner D, Haidt J (2003)Approaching awe, a moral, spiritual and aesthetic emotion. 115. Baumeister RF, Stillwell AM, Heatherton TF (1994) Guilt: An interpersonal approach. Cogn Emot 17:297–314. Psychol Bull 115:243–267. 82. Posner J, Russell JA, Peterson BS (2005) The circumplex model of affect: An integrative 116. Tamborini R, Stiff J, Heidel C (1990) Reacting to graphic horror: A model of empathy approach to affective , cognitive development, and psychopathology. and emotional behavior. Commun Res 17:616–640. Dev Psychopathol 17:715–734. 117. Reeve J (1989) The interest–enjoyment distinction in intrinsic motivation. Motivation 83. Maaten LVD, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9:2579– Emotion 13:83–103. 2605. 118. Silvia PJ (2005) What is interesting? Exploring the appraisal structure of interest. Emo- 84. Ellsworth PC (2014) Basic emotions and the rocks of New Hampshire. Emot Rev 6: tion 5:89–102. 21–26. 119. Wildschut T, Sedikides C, Arndt J, Routledge C (2006) Nostalgia: Content, triggers, 85. Etcoff NL, Magee JJ (1992) Categorical perception of facial expressions. Cognition functions. J Personality Soc Psychol 91:975–993. 44:227–240. 120. Tracy JL, Robins RW (2007) The psychological structure of pride: A tale of two facets. 86. Frijda NH (1987) The Emotions (Cambridge Univ Press, Cambridge, UK). J Pers Soc Psychol 92:506–525. 87. Lerner JS, Keltner D (2001) Fear, anger, and risk. J Pers Soc Psychol 81:146–159. 121. Tomkins SS (1984) . Approaches to Emotion, eds Scherer KR, Ekman P 88. Carver CS, Harmon-Jones E (2009) Anger is an approach-related affect: Evidence and (Psychology, Hove, UK), pp 163–195. implications. Psychol Bull 135:183–204. 122. Bonanno GA (2010) The Other Side of Sadness: What the New Science of Bereavement 89. Davidson RJ (1993) Parsing affective space: Perspectives from neuropsychology and Tells Us About Life After Loss (Basic Books, New York). psychophysiology. Neuropsychology 7:464–475. 123. Diener ED, Emmons RA, Larsen RJ, Griffin S (1985) The satisfaction with life scale. J 90. Fraley RC, Shaver PR (2000) Adult romantic attachment: Theoretical develop- Pers Assess 49:71–75. ments, emerging controversies, and unanswered questions. Rev Gen Psychol 4:132– 124. Goetz JL, Keltner D, Simon-Thomas E (2010) : An evolutionary analysis 154. and empirical review. Psychol Bull 136:351–374. 91. Ellsworth PC (1991) Some implications of cognitive appraisal theories of emotion. 125. Tracy JL, Matsumoto D (2008) The spontaneous expression of pride and : Evi- International Review of Studies on Emotion, ed Strongman KT (Wiley, New York), Vol dence for biologically innate nonverbal displays. Proc Natl Acad Sci USA 105:11655– 1, pp 143–161. 11660. 92. Berrios R, Totterdell P, Kellett S (2015) Eliciting mixed emotions: A meta-analysis com- 126. Frijda NH, Kuipers P, Ter Schure E (1989) Relations among emotion, appraisal, and paring models, types, and measures. Front Psychol 6:428. emotional readiness. J Pers Soc Psychol 57:212–228. 93. Larsen JT, McGraw AP, Cacioppo JT (2001) Can people feel happy and sad at the same 127. Smith ER, Mackie DM (2008) Intergroup emotions. Handbook of Emotions, eds time? J Pers Soc Psychol 81:684–696. Lewis M, Haviland-Jones JM, Barrett LF (Guilford, New York), 3rd Ed, pp 428–439. 94. Barrett LF (2013) Psychological construction: The Darwinian approach to the science 128. Smith CA, Lazarus RS (1990) Emotion and adaptation. Handbook of Personality: The- of emotion. Emot Rev 5(4):379–389. ory and Research, ed Pervin LA (Guilford, New York), pp 609–637.

10 of 10 | www.pnas.org/cgi/doi/10.1073/pnas.1702247114 Cowen and Keltner Downloaded by guest on October 3, 2021