What music makes us feel: At least 13 dimensions organize subjective experiences associated with music across different cultures

Alan S. Cowena,1,2, Xia Fangb,c,1, Disa Sauterb, and Dacher Keltnera

aDepartment of Psychology, University of California, Berkeley, CA 94720; bDepartment of Psychology, University of Amsterdam, 1001 NK Amsterdam, The Netherlands; and cDepartment of Psychology, York University, Toronto, ON M3J 1P3, Canada

Edited by Dale Purves, Duke University, Durham, NC, and approved December 9, 2019 (received for review June 25, 2019) What is the nature of the feelings evoked by music? We investi- people hear a moving or ebullient piece of music, do particular gated how people represent the subjective experiences associated feelings—e.g., “sad,”“fearful,”“dreamy”—constitute the foun- with Western and Chinese music and the form in which these dation of their experience? Or are such feelings constructed representational processes are preserved across different cultural from more general affective features such as valence (pleasant vs. groups. US (n = 1,591) and Chinese (n = 1,258) participants listened unpleasant) and arousal (9, 11–15)? The answers to these to 2,168 music samples and reported on the specific feelings (e.g., questions not only illuminate the nature of how music elicits “ ”“ ” angry, dreamy ) or broad affective features (e.g., valence, subjective experiences, but also can inform claims about how the arousal) that they made individuals feel. Using large-scale statisti- brain represents our feelings (8, 15), how infants learn to rec- cal tools, we uncovered 13 distinct types of subjective experience ognize feelings in themselves and others (16), the extent to which associated with music in both cultures. Specific feelings such as representations of subjective experience are universal across “triumphant” were better preserved across the 2 cultures than cultures (4–7), and the etiology of affective disorders (17, 18). levels of valence and arousal, contrasting with theoretical claims that The dimensionality of subjective experience concerns the range valence and arousal are building blocks of subjective experience. This held true even for music selected on the basis of its valence and of feelings that people experience. How many distinct feelings do arousal levels and for traditional Chinese music. Furthermore, the we experience in response to music? Which of the feelings asso- feelings associated with music were found to occupy continuous ciated with music are dependent on the cultural background of the gradients, contradicting discrete emotion theories. Our findings, vi- listener, and which are consistent across cultural groups? sualized within an interactive map (https://www.ocf.berkeley.edu/ Finally, there is the question of distribution: How are the ∼acowen/music.html) reveal a complex, high-dimensional space of feelings that music evokes distributed: Are they discrete clusters subjective experience associated with music in multiple cultures. of states, or can they be blended together (11, 19)? These findings can inform inquiries ranging from the etiology of Together, the conceptualization, dimensionality, and distribu- affective disorders to the neurological basis of emotion. tion of subjective experience capture what we refer to as a se- mantic space of subjective experience. Studies drawing upon this emotion | affect | music | culture | semantic space conceptual framework have renewed our understanding of the feelings elicited by videos of real-world events (12) and attributed entral to the meaning of music are the subjective experiences to facial expressions and vocal utterances (10, 20, 21). Cthat it evokes (1–3). Performers across cultures (4, 5) can reliably convey intense feelings with songs and instruments of Significance different kinds and often do so by relying on acoustic features and associated percepts—pitch, loudness, pace—characteristic Do our subjective experiences when listening to music show of the human vocal expression of emotion (6) and of speech (7). evidence of universality? And if so, what is the nature of these Music also modulates activity in brain regions implicated in experiences? With data-driven methodological and statistical emotion-related processing (8). approaches, we examined the feelings evoked by 2,168 music What is not well understood is how music evokes feelings in excerpts in the United States and China. We uncovered 13 dis- listeners. What is the taxonomic structure of the subjective ex- tinct types of experiences that people across 2 different cul- periences that music evokes? How do people represent their tures report in listening to music of different kinds. Categories experiences when hearing music? To what extent are these such as “awe” drive the experience of music more so than processes consistent or variable across cultural groups? The broad affective features like valence. However, emotions that present investigation addresses these questions, guided by a scientists have long treated as discrete can be blended to- theoretical approach to subjective experience (9) and empirical gether. Our results provide answers to long-standing questions and quantitative approaches (10). about the nature of the subjective experiences associated with music. A Semantic Space Approach to Subjective Experience Music can evoke intense subjective experiences that people Author contributions: A.S.C., D.S., and D.K. designed research; A.S.C. and X.F. performed represent with concepts such as “awe,”“joy,” and “anger” (1–3). research; A.S.C. contributed new reagents/analytic tools; A.S.C. analyzed data; and A.S.C., Within any realm—social interactions, the appreciation of art, or X.F., D.S., and D.K. wrote the paper. the daily fluctuations of mood, for example—subjective experi- The authors declare no competing interest. ences are varied, blended, and complex (11). To account for such This article is a PNAS Direct Submission. complexity, we have offered a semantic space approach that Published under the PNAS license. structures subjective experiences in terms of 3 properties: their 1A.S.C. and X.F. contributed equally to this work. conceptualization, dimensionality, and distribution. 2To whom correspondence may be addressed. Email: [email protected]. The conceptualization of subjective experience concerns the This article contains supporting information online at https://www.pnas.org/lookup/suppl/ nature of the concepts that most precisely characterize our doi:10.1073/pnas.1910704117/-/DCSupplemental. feelings. This question is central to affective science (11). When First published January 6, 2020.

1924–1934 | PNAS | January 28, 2020 | vol. 117 | no. 4 www.pnas.org/cgi/doi/10.1073/pnas.1910704117 Downloaded by guest on September 26, 2021 A Methodological Approach to Subjective Experience people report feeling chills, shivers down their spine, laughter, Characterizing the semantic space of any response modality re- tears, goose bumps, lumps in their throat, and rushes of adren- quires a departure from past approaches. The vast majority of aline (23–27). However, we have yet to fully understand the research on subjective experience has focused on a small number different varieties of subjective experience associated with dif- of emotions (often just 4 to 6) (11). These experiences are typ- ferent kinds of music [or the distinctions between experienced ically elicited using a limited number of stimuli—often only 1 or 2— and perceived feelings in music, which are difficult to capture preselected to elicit more prototypical emotional responses, (1, 28); see Discussion for implications for the present study]. Well- which are measured by asking participants to choose among 4 to replicated studies have found that people within a cultural group 6 emotion categories or to report ratings of valence and arousal can, with 70% or greater accuracy, label the feelings associated (and on occasion other appraisals) (11, 18). These methods pre- with small assortments of musical selections with 6 or fewer — “ ”“ ”“ ” clude advances in understanding how people conceptualize sub- emotion categories most notably, anger, happiness, fear, “ ”“ ” “ ” — jective experience in terms of both categories and affective sadness, surprise, and tenderness (1) or along scales of features (conceptualization), the range of feelings people experi- valence and arousal (29). ence (dimensionality), and the nature of the boundaries between Less is understood, however, about the broader structure of specific feelings (distribution) (9–12). subjective experiences associated with music. Many studies have With respect to the conceptualization of subjective experience, contrasted models that sort the feelings associated with music the use of small stimulus sets precludes comparing competing into 6 discrete emotion categories with models that represent models of experience—for example, whether it is driven by spe- these feelings as levels of valence and arousal (29, 30). However, cific feelings such as “amusement” or broad affective features such because these are 2 among many possible ways that the feelings as valence. This is because with few stimuli that vary in many ways, associated with music may be structured (11), the focus on these many alternative conceptualizations become equivalent (that is, 2 models has precluded a richer understanding of the concep- tualization, dimensionality, and distribution of the subjective competing model features become collinear) (22). For example, experiences associated with music. categories such as “amusing” and more general features such as One exception to the focus on 6 emotions or valence and “valence” become statistically inseparable when studying just 1 or arousal is a study by Zentner et al. (3), in which factor analysis 2 positive emotion stimuli. was applied to participants’ self-reports of how often they had With respect to the dimensionality of subjective experience, felt 89 feelings in their past experiences of listening to music the focus on a narrow range of states—usually 6—precludes from their preferred genre. This study identified 9 potentially

examining how subjective responses may vary along a much PSYCHOLOGICAL AND COGNITIVE SCIENCES distinct dimensions of subjective experience. However, the wider array of dimensions (9–12). Does music only evoke the methods used in this study were retrospective and consequently 6 emotions that are most widely studied? Or does music evoke a subject to memory biases—for example, the potential confusion more complex array of feelings? between the feelings associated with music and those associated With respect to the distribution of subjective experience, the with the contexts in which the music was heard. It was also study of only prototypical exemplars of particular emotions makes subject to the limitations of factor analysis, which does not it difficult to account for potential blends in subjective experience – consider the reliability with which responses are associated with (9, 11, 12), which are likely to be associated with music (1 3, 10). individual stimuli (SI Appendix, SI Discussion and Movie S1). Understanding the nature of subjective experience therefore This raises the question of how many kinds of subjective expe- requires collecting responses to a vast array of rich and diverse rience can reliably be associated with actual music excerpts in stimuli, using multiple-response formats, targeting a wide variety controlled or randomized contexts. of potentially distinct feelings, and including more than just Moreover, a limited number of studies have examined the prototypical elicitors or exemplars associated with each feeling. feelings associated with music in multiple cultures (5), which is A recent investigation guided by these considerations analyzed critical to endeavors to understand potential universals, and their ’ US participants responses to thousands of evocative videos of likely neurophysiological underpinnings, in responses to music real-world scenes including landscapes, artwork, sports, animals, (31). One study focused on 3 emotion categories (happiness, and injuries. With data-driven statistical techniques, it was found sadness, and fear) and found that they were associated with that self-report captures over 2 dozen dimensions, or kinds of similar music in Westerners and participants from a remote subjective experience evoked by videos (12). The taxonomy of African population, the Mafa of the Mandara Mountains in subjective experience is far richer than anticipated by traditional Cameroon (32). In contrast, more recent studies have uncovered theories of emotion (11). However, because videos impart de- cultural differences in feelings of pleasantness versus un- tailed semantic information (e.g., images of baby faces, spiders, pleasantness associated with music (33, 34). For example, the or nature scenes), the extent to which participants’ responses Tsimané of the Amazon rainforest do not perceive dissonance as represented distinct subjective experiences versus distinct clas- unpleasant (34). sifications of semantic content is unclear. Moreover, the study of Overall, then, we have a limited understanding of the structure subjective experience within a single culture leaves open the that people across cultures rely on to represent the feelings as- question of the extent to which the conceptualization, di- sociated with music. To what extent do categories or broader mensionality, and distribution of subjective experience is pre- affective features capture cultural universals in the feelings as- served across cultures. In the present study, we move beyond sociated with music? What is the range of feelings that music earlier work by examining the subjective experiences elicited by conveys? Are the boundaries between the feelings associated instrumental music, which is largely free of semantic content with music discrete or continuous? Together, the conceptuali- (although not necessarily of learned semantic associations, as we zation, dimensionality, and distribution of feelings associated will discuss), and by comparing reports of subjective experience with music comprise a taxonomy of its experiential effects, the across the United States and China. We do so to understand foundation for investigations of how and why music moves us. more abstract representations of subjective experience and how they may be preserved across multiple cultures. Experiment 1. Uncovering the Feelings Associated with Music The Feelings Associated with Music In our primary investigation, we derive a semantic space of Although it is abstract and nonrepresentational, instrumental subjective experience reliably associated with a broad range of music evokes rich subjective experiences. In response to music, Western music across 2 rather different cultures. We do so by

Cowen et al. PNAS | January 28, 2020 | vol. 117 | no. 4 | 1925 Downloaded by guest on September 26, 2021 examining several hundred thousand judgments from partici- and corresponding references.) Participants judged each music pants in the United States (n = 1,011) and China (n = 895) (for sample on 9-point Likert scales (1 = negative levels/none of the discussion of cultural differences, see SI Appendix, SI Discussion, quality, 5 = neutral/moderate levels, 9 = extremely high levels). and refs. 35 and 36). Each participant heard subsets of Based on past estimates of reliability of observer judgment, for 1,841 nonlyrical music samples that were gathered in an open- each music sample we collected an average of 12 responses from ended format. We applied data-driven statistical modeling separate participants in each of the 2 response formats in each techniques to these judgments to characterize the semantic space culture. This amounted to a total of 375,230 judgments of all of subjective experience associated with music in each culture. music samples (59,277 forced-choice categorical judgments and By comparing the feelings associated with music in participants 315,953 9-point scale judgments; see SI Appendix, Methods S2). from China and the United States, we can ascertain the extent to which the conceptualization, dimensionality, and distribution of Results subjective experiences are preserved across these 2 different Overview. Guided by our semantic space framework (9), past cultures. validated methods, and a central design feature of this in- At stake in the study of the semantic space of subjective ex- vestigation—data gathered from the United States and China, perience are answers to questions of central theoretical import nations with cultures differing in their norms and values [e.g., (9, 11). In terms of the conceptualization of subjective experi- individualism vs. collectivism, low vs. high power distance, low vs. ence, with what consistency across cultures are responses to high long-term orientation, and high vs. low indulgence (35)]— music represented using specific categories of subjective expe- our data analysis proceeded as follows. First, to explore issues of rience, such as awe and fear, and information about broad af- conceptualization, we examined which is better preserved across fective features, such as valence and arousal? With respect to the the 2 cultural groups—categories of subjective experience (e.g., dimensionality of subjective experience, how many distinct va- “angry,” dreamy”) or affective features (valence, arousal, etc.)— rieties of feelings associated with music are preserved across and which is more potent in explaining variance in judgments cultures? In terms of the distribution of subjective experiences from the other cultural group (10). To address the dimension- across cultures, do feelings occupy discrete clusters—families of ality of the subjective experiences associated with music, we re- related states such as anxiety and fear—or do they lie along lied on statistical techniques that uncover how many distinct continuous gradients? dimensions are required to account for cross-cultural similarities To answer these questions, we collected judgments from par- in judgments of subjective experience. Finally, with visualization ticipants in the United States and China employing the richest techniques, we explored the distribution of feelings associated stimulus set of Western music samples ever studied, both in with music within a high-dimensional space. terms of the variety of different feeling states and affective fea- tures conveyed and in terms of the diversity of exemplars asso- Mapping Cultural Similarities in the Subjective Experiences Associated ciated with each feeling, ideal for deriving a semantic space of with Music. To assess cross-cultural similarity in subjective subjective experience. experience, past studies have most typically ascertained Participants from the United States were recruited on Amazon whether participants’ judgments match experimenter expecta- Mechanical Turk. Participants from China were recruited from a tions. Instead, we operationalized the reliability of subjective multi-institutional participant pool by X.F. To collect a library of experience in terms of interrater agreement, a data-driven ap- music, 111 US participants (41 women, mean age = 32) were each proach (see SI Appendix, SI Discussion, for detailed rationale). asked to contribute 5-s music samples conveying each of a wide We first analyzed the combined data from Chinese and US range of subjective experiences drawn from studies of the feelings participants to verify that the 1,841 music samples were reliably associated with music and the voice (1–3, 20). The categories were associated with different feelings. In this analysis, we found that “amusing,”“angry,”“annoying,”“anxious/tense,”“awe-inspiring/ many different categories of subjective experience were associ- amazing,”“beautiful,”“bittersweet,”“calm/relaxing/serene,” ated with music with a moderate degree of reliability. All but one “compassionate/sympathetic,”“dreamy,”“eerie/mysterious,” category of subjective experience (“entrancing”)wereevoked “energizing/pump-up,”“entrancing,”“erotic/desirous,”“euphoric/ at significant rates by a number of music samples ranging from ecstatic,”“exciting,”“goose bumps,”“indignant/defiant,”“joyful/ 21 (“painful”)to360(“calm/relaxing/serene”)(q < 0.05; cheerful,”“nauseating/revolting,”“painful,”“proud/strong,”“ro- Monte Carlo simulation using empirical base rates; see SI mantic/loving,”“sad/depressing,”“scary/fearful,”“tender/long- Appendix, Methods S3). Indeed, the vast majority (93.3%) of the ing,”“transcendent/mystical,” and “triumphant/heroic.” These 1,841 music samples evoked at least one category of subjective methods enabled us to investigate a broad range of feelings as- experience at a significant rate (q < 0.05; see SI Appendix,Fig. sociated with music. (See SI Appendix, Methods S1,fordetailsand S1, for distribution of categorical and dimensional ratings). On SI Appendix,TableS2, for references for each category and average, 42.3% of raters from China and the United States chose Chinese translations.) the most agreed-upon category of subjective experience for each After assembling a diverse library of 1,841 music samples, we music sample (chance level = 23.1%, Monte Carlo simulation of recruited participants from the United States (n = 1,011, all category judgments). While interrater agreement rates varied, 527 women, mean age = 35.5) and China (n = 895, 556 women, some music samples were labeled very consistently with cate- mean age = 30.4) to judge the music samples in 1 of 2 response gories including “energizing/pump-up” (up to 89% of raters), formats (see SI Appendix, Fig. S1 for depiction of surveys). One “triumphant/heroic” (83%), “amusing” (79%), “annoying” group of participants was asked to choose the feelings evoked by (79%), “scary/fearful” (76%), “joyful/cheerful” (74%), “calm/ each music sample from the 28 categories listed above. relaxing/serene” (74%), “eerie/mysterious” (69%), and “romantic/ A second group of participants rated each music sample on loving” (67%). The affective features of the music samples also 11 scales measuring broad affective features. The scales were varied significantly, with the magnitude of Cohen’s d for each of culled from dimensional and componential theoretical accounts the affective scale ratings exceeding 1.2 ( q < 0.05; Monte Carlo of appraisal processes proposed to underlie emotional experi- simulation; see SI Appendix, Methods S3) for between 12 (cer- ence and expression (13, 37), including arousal, attention, cer- tainty) and 95 (valence) different music samples. tainty, commitment, dominance, enjoyment, familiarity, identity, obstruction, safety, and valence. (Note that these labels are Is the Experience of Specific Feelings or Broad Affective Features shorthand for the questions to which raters responded. See SI Better Preserved across Cultures? Next, we compared the sub- Appendix, Methods S2 and Table S2, for wording of each question jective experiences associated with music across the 2 cultures.

1926 | www.pnas.org/cgi/doi/10.1073/pnas.1910704117 Cowen et al. Downloaded by guest on September 26, 2021 In doing so, we sought to ascertain whether judgments of specific 1 A .9 categories of subjective experience or affective features were .8 .7 better preserved. In past studies, cross-cultural similarity has .6 .5 typically been ascertained by comparing the rates with which .4 members of different cultures label a target stimulus—a facial .3 — .2 expression, vocalization, or less typically, a musical segment US-China Corr. .1 with the same term. This approach does not capture whether 0 members of each culture also use the same concepts (either Angry rousal Safety alence Painful Identity ttention Dreamy A V Exciting Beautiful Amusing Certainty categories or scale ratings) to label nontarget stimuli in a similar A Annoying Familiarity Enjoyment Entrancing Dominance Bittersweet Obstruction

fashion. For example, if 60% of US participants rate a music Commitment Scary, fearful Scary, Goosebumps Transcendent Proud, strong Nauseating, rev. Anxious, tense Tender, longing Tender, “ ” “ ” Joyful, cheerful Indignant, defiant Erotic, desirous Euphoric, ecstatic sample as joyful and 40% as triumphant, the interrater Sad, depressing Romantic, loving Eerie, mysterious Awe-inspiring, amaz. Awe-inspiring, Compassionate, symp. Triumphant, heroic Triumphant, Calm, relaxing, serene agreement between US and Chinese raters could never exceed Energizing, pump-up 60% (the interrater agreement among US raters alone) even if 1 * * the same proportions of Chinese participants respond with each B .8 US Chinese .6 category. Hence, data incorporating the full distribution of re- category category .4 .2 XYZ sponses in each culture are critical to fully understanding cultural multiplied judgments judgments 0 similarities in the feelings associated with stimuli of different equals Valence approx. 1 * kinds. .8 * Given this concern, we took a different analytic approach than r correlates US Chinese .6 category category .4 is typical in the field: for each category of experience and af- .2 XYZ to valence to valence 0 fective scale, we correlated the mean judgments of US raters weights weights Arousal with those of Chinese raters across all 1,841 music samples. This US Chinese analysis reveals the extent to which US and Chinese participants valence valence use the categories and judgment scales in a similar fashion when judgments YXUS Chinese Zjudgments labeling the subjective experiences associated with the 1,841 predicted .88 valence .75 valence .90 predicted from Chinese judgments judgments from US music samples. To control for within-culture variations in the use categories categories of the categories and affective scales, we then divided this value by the within-culture explainable variances of these mean judg- ments (see SI Appendix, Methods S4 and S5, for elaborated US Chinese PSYCHOLOGICAL AND COGNITIVE SCIENCES arousal arousal rationale and further details). Dividing by the explainable var- judgments judgments iances results in an estimate of the correlation that would emerge YXUS Chinese Z predicted .97 arousal .80 arousal .91 predicted if we averaged an infinite number of ratings of each music from Chinese judgments judgments from US sample in each culture. We refer to this estimate as a signal categories categories correlation: it captures the degree of similarity between cultures in the association of each specific feeling and affective feature Fig. 1. (A) Correlations in feelings associated with music across 2 distinct with music (see SI Appendix, Fig. S3, for demonstration that cultures. The cross-cultural signal correlation for each category (orange bars) these methods are effective using simulated data). The cross- and affective scale (green bars) captures the degree to which each judgment cultural signal correlations in judgments of each category and is preserved across Chinese and US listeners across all 1,841 music samples. The cross-cultural signal correlation is calculated by correlating the mean affective scale are shown in Fig. 1A. “ ” responses by Chinese participants with the mean responses by US partici- If feelings such as amusement are constructed in part from pants and dividing by the explainable variance in responses from each cul- more basic ingredients of valence and arousal, one would expect ture. Error bars represent SE. (See SI Appendix, Methods S4 and S5,for greater convergence across cultures in how these broad affective details of explainable variance and SE estimation; SI Appendix, Fig. S3,for features are associated with music than in how the feelings pu- confirmation that these results accurately recover population-level correla- tatively constructed from these affective features are associated tions; and SI Appendix, Fig. S4, for similar results using Spearman correla- with music (13, 14, 38). Inspection of cross-cultural correlations tions and/or binary affective scale ratings.) (B) Categories of subjective for judgments of each affective feature and category of subjective experience account for preservation of affective features across cultures. experience reveals otherwise (Fig. 1A). With cross-cultural signal Category judgments are used to predict affective feature judgments within each culture using ordinary least squares regression. Then, regression correlations exceeding 0.76, 18 categories of subjective experi- weights are multiplied by the category judgments from the other culture to ence were better preserved across China and the United States predict affective feature judgments in the first culture. Signal correlations than valence (r = 0.75). Furthermore, cross-cultural signal between the Chinese and US participants predicted and actual valence/ correlations were significantly greater for joyful/cheerful and arousal judgments are given in the small red/blue lines and circles, signified triumphant/heroic [P ≤ 0.002, bootstrap test; q < 0.05, Benjamini– by adjacent letters, and plotted on the Upper Right. Category judgments Hochberg false discovery rate (FDR) correction (39) (SI Appen- from each culture are better than valence/arousal judgments from each dix, Methods S5)] and for amusing, compassionate/sympathetic, culture at predicting valence (*P = 0.002 [Y], 0.008 [Z]; bootstrap test, 2-tailed) = energizing/pump-up, exciting, sad/depressing, and scary/fearful and arousal (*P 0.012 [Y], 0.024 [Z]) judgments from the other culture. These (P ≤ 0.024; q < 0.1) than for valence. Correlations for 16 cate- results are consistent with the hypothesis that specific feelings are elicited by music, and then subsequently used to construct valence/arousal judgments in a gories of subjective experiences were also better preserved across = more culture-specific process of inference. (See SI Appendix,Fig.S5,forcon- the 2 cultures than that for arousal (r 0.81), although the sistent results with other affective feature judgments.) differences were not statistically significant for individual cate- gories. The finding that experiences of many categories of sub- jective experience are better preserved across the 2 cultures than about the conceptualization of subjective experience: perhaps that of valence—sometimes considered a “basic building block affective features such as valence and arousal are in fact psy- of emotional life” (38) —contrasts with the claim that experi- ences of specific feelings derive from broader affective features chologically constructed from more specific feelings. In other (13, 14, 38). words, perhaps subjective experiences are best represented in That reported experiences of many categories of subjective terms of more specific feelings such as and “anger” and “tri- experiences were more consistent across the 2 cultures than umph,” and levels of valence, arousal, and other affective those of broader affective features raises an intriguing question features emerge as higher-order evaluations of these basic

Cowen et al. PNAS | January 28, 2020 | vol. 117 | no. 4 | 1927 Downloaded by guest on September 26, 2021 experiences. If this were the case, then the added processing across 2 datasets measuring the same attributes (US and Chinese required to infer valence and arousal could heighten differences judgment data). The resulting components are ordered in terms across cultural groups. As a result, reports of specific feelings, as of their level of positive covariance across the 2 datasets (cul- opposed to affective features, from a given cultural group would tures). (See SI Appendix, Methods S7, for further details and be more accurate in predicting reports of the same affective discussion.) features from the other cultural group. Collecting many in- Given that we found that categories of subjective experience dependent ratings of both categories of subjective experience explained the cross-cultural preservation of the affective fea- and affective features such as valence and arousal across tures, we applied PPCA to the US and Chinese category judg- 1,841 music samples, as we have done, allowed for a rigorous test ments of the 28 categories of subjective experience associated of this possibility. with the 1,841 music samples to determine the number of in- To test this hypothesis concerning the primacy of specific dependent dimensions, or kinds, of subjective experience that feelings and broader affective features in subjective experiences, were experienced in both cultures. We applied PPCA in a leave- we used linear regression analyses to derive cross-cultural signal one-rater-out fashion to determine the statistical significance correlations in the mapping between category and affective scale of each component. (See SI Appendix, Methods S7,forfurther judgments of the music samples (SI Appendix, Methods S6). details.) These analyses ascertain whether ratings of specific feelings are As we show in Fig. 2, PPCA revealed that 13 distinct semantic stronger predictors of broad affective feature judgments across dimensions of subjective experiences were associated with music the 2 cultures or vice versa. In keeping with the idea that judg- and significantly preserved across the US and China participants’ ments of music in terms of broad affective features derive from q < q < the cross-culturally preserved experience of more specific feeling judgments [ 0.001 across all held-out raters, 0.05 across states, we find that category judgments consistently predict af- fective feature judgments from the other culture as robustly as, or more robustly than, affective feature judgments themselves. We can see this in Fig. 1B, which illustrates how category judg- ments from one culture better predict judgments of valence (blue bars labeled Y and Z) and arousal (red bars labeled Y and Z) from the other culture than the valence and arousal judgments themselves (blue and red bars labeled X). Results are similar for other affective features such as dominance, as shown in the bar graphs at the top of SI Appendix, Fig. S5. By contrast, the af- fective feature judgments from each country generally do a poorer job of predicting the category judgments from the other culture, as indicated in the bar graph at the bottom of SI Ap- pendix, Fig. S5. Based on these results, it is more plausible that judgments of general affective features (valence, arousal, etc.) are psychologically constructed from experiences that are cap- tured by specific feelings (“amusement,”“fear,” etc.) than vice versa in the representation of subjective experience conveyed by music.

Music Is Associated with at Least 13 Distinct Kinds of Subjective Experiences across 2 Cultures. Thus far we have documented that all but one of the categories of subjective experience that we examined were associated at above-chance levels of reliability with at least 21 music samples and that the feelings that partic- Fig. 2. Thirteen distinct feelings associated with music are preserved across ipants reported upon hearing music were captured more reliably cultures. (Top Left) Proportion of variance explained within the US and Chinese ratings by the 29 PPCs of the mean categorical ratings of 28 emo- across the 2 cultures by these categories than by broad affective tions across cultures. (Bottom Left) PPCA finds dimensions of maximal co- features (e.g., valence and arousal). These findings set the stage variance across 2 cultures. Here, the covariance of each PPC across cultures, for our next question: How many distinct varieties of subjective scaled by total positive covariance, is plotted alongside the cross-cultural test experience were consistently evoked across the 2 cultures: That correlation derived from a cross-validation analysis (see SI Appendix, Meth- is, what is the dimensionality of the cross-cultural semantic space ods S7, for details). We can see that the PPCs are ordered in terms of of subjective experience associated with music? For example, we descending within-sample covariance explained. While it is useful to order have not yet ruled out whether some categories were redundant. the components in terms of the proportion of the shared variance across For example, there may have been interrater reliability in la- cultures that they each explain, we are ultimately interested in the out-of- beling music with categories such as “awe” and “fear,” but per- sample test correlation, which measures how similarly the 2 cultures judged music samples along each dimension. The test correlation was significant for haps these categories were actually associated with precisely the 13 PPCs [q < 0.001 across all held-out raters, q < 0.05 across held-out raters same music samples. If we reduce the US and Chinese judgments from each country individually, ForwardStop sequential FDR-corrected (40) to a more limited number of dimensions, what is the minimum one-tailed Wilcoxon signed-rank test (41)]. Given the complexity of the cross- number necessary to account for commonalities in reported validation analysis, these correlations are not adjusted for explainable vari- subjective experience in response to music across cultures? To ance, and it is safe to assume that the population-level correlations are address this question, we use a recently developed analytic substantially higher. (Right) By applying factor rotation (varimax) to the approach. 13 significant PPCs, we found 13 preserved varieties of subjective experience “ ”“ ” Namely, to compute the total number of distinct varieties of that each load maximally on a distinct category: amusing, annoying, “anxious/tense,”“beautiful,”“calm/relaxing/serene,”“dreamy,”“energizing/ subjective experience that were significantly preserved across the pump-up,”“erotic/desirous,”“indignant/defiant,”“joyful/cheerful,”“sad/ US and Chinese judgments of the music samples, we used a depressing,”“scary/fearful,” and “triumphant/heroic.” In subsequent method that we call principal preserved component analysis figures, we will refer to each component using the category of subjective (PPCA) (10). PPCA extracts linear combinations of attributes experience that it loads on maximally. Each component is also assigned a (here, judgments of subjective experience) that maximally covary letter,shownonthex-axis.

1928 | www.pnas.org/cgi/doi/10.1073/pnas.1910704117 Cowen et al. Downloaded by guest on September 26, 2021 held-out raters from each country individually, ForwardStop

sequential FDR-corrected (40) one-tailed Wilcoxon signed-rank All Raters DE E. Calm, E E DDEEEEEE E DEE EEE relaxing, E D EEE E test (41)]. We thus find that music conveys 13 distinct varieties of EFE EE E E E D. Beautiful FFEEE EEEEEEEEE E E serene EEEEE E E EEE EK EE E E E EE EE EE E subjective experience across the 2 cultures. (In Fig. 2, the Upper D EED E EEE EE EE D DE DKE E DD E E KE E E DE E EEEEE E EDK D DD EEEEEEEE E E EKK K E Left chart shows the proportion of variance explained by each H. Erotic, DE D E E E E E E E E DE EEE E D KK E E DD DE E E EEEE E KKK EEEEE EEE E E D EEEEE E DEKKK KE KE dimension [PPC] uncovered by PPCA in data from each culture; desirous H H EEE D E EEE EEK KK K H HHH E EE EE DEED EE EEEE KE K EKEE DK EK H HEHH DE DDDE DD EE KK K HH HHE ED E EEEEEK EKKD K H H E D D E EK K KKK the Bottom Left chart reveals the proportion of preserved co- HHHH E HBE E E E DDD D EEE E KE KKK H H E DDDE DDD E E KKKKKKK K. Sad, J. Joyful, HH HHH E DD DE KKK H HH H EE D DDE E EDD K K KK variance for each dimension, as well as the corresponding cor- HHH HH L E D D D H D depressing A. cheerful A H E E EDDDE HD K E K HAHE H HH HEEHK D DDKDK E KK K JA HH H HED D ED D E K C K K K relation and its significance.) Amusing A JJ H H E H E D D MD EE KK KK AAA J AA JA J JA EJAE HH EAEE DE EE E MKDK K AAAAA A JJ J GJ J J JE EJEG HED EE EF K KKKKK A AAAJ J J J JJJ HEH E KD D DFE EKKK KKKKKK LK BB AJ J J JJJ J J GH J EEE ED J D E K KD K GB A A J AJ JJ AJ J JJ JE G D EF D D EF KH M GAJ A A J JJJJ J J EE HF DF DD F K K G AGGJ J J J JJ JJ D DJDJ JJ E DJ D HF FE FFFD E FKK A GJ JJJ JJ DJD FEE F FEF F FE KK What Are the Feelings Associated with Music? To find the 13 di- G G GG JJ JJ J J EF F F FEK F. Dreamy J G GGJ GJ J J J JJJ J G J JFD F FF EE F F GG J J J J H JJ F FFF F FFF EF FE G GG JG J JJ J G FFD DH F FF F FF F FD DM mensions, or patterns, of subjective experience within the cate- G GGGGGG JGJ J F F FF F FFF F DD D G GGG G JJ G DM J F FFF FFF D D M G G G J G GG J G FFF FF DMD G GG GG G G G GGJ J MG L D FF FFFFF BD MDM D G G GGG MMM C F F F F FMMMMMM gorical judgments that were preserved across participants from G GGGG G GG G M G AF F F FE FF DM M GG GGGG G G AG J G GGMG F H F FFFFFFFMM M G G GG GGGG G GG G M F B F F F G G G GGG GJ HHHFF FFFF FF M the United States and China, we applied factor rotation (vari- G GG G G AG H FF F FF GGG G G G G G GG AA H H F F KF CE F LF M. Triumphant, G GG G G G GG G G J G G HH C F FF HL FF L GG GGGG G GGGG G GG G GF H C F FFFF FF max) to the 13 significant components extracted using PPCA. GG GGG GGGG GGGG GGG GB G FFFFFFF FF K heroic G GGGGGGG GGBBB GGG GB K F F FF FKL K G GG GGGGGGG G G BC H M L L F FL K K GG GGG G GGG G G G B G C L L L CCL M G GGG GG G G GGG B L L L KL FF CC M MMM M Here, factor rotation extracts a simplified representation of the G G G G GG GG BGG C B CCB K FLFF FF C M G. G GGGG G B GG GC GB I C B KL LL LFF FF MMM M M G G G IB IG G C B BC LF L MMM M MM G GG GG GMGGG GGGCC CG B B BB L L LCL L D M M MM MM M space by attempting to find dimensions constituted of only a few Energizing, G I G G IG BB B BF B L C C C MM MMM MMMMM GGGGG GGG G GG BB BA A B LL LCC CCM MMMM M MM M MMMM G GG GG I I BIIIII I GI BB B B F C MM MMMM M M G I GG GI I B I B B BB FF L L C CCCC M MMM MMM MMMM pump-up G GG G I GGGGB G BBBBB B L LCL L LC C L CM M MMMM MMMM MMM categories each. After factor rotation, we find that each of the G G GGI G I II GI GMC G C B BBBBB BBB L LLLLL C L MMMMMM M MMMMMMMMM G G GGGI II I MG BBBCB LLCCLLLL LLL C MM MMMMM MMMMMMM GG G G III GI GM MM L BBB LL LLL LL LL MMM MMM M MGGGI G CC C BBB LLLLLL LLLLL LL M C MM MMM M M 13 resulting dimensions (PPCs) loaded maximally on a distinct I I GGG BB LLLLLLLLLL L LMMM MMM M C C LL L L M G M B. Annoying feeling reported in response to the music samples (Fig. 2). Thus, I. Indignant, L. Scary, C. Anxious, the dimensions can be interpreted as 13 distinct varieties defiant fearful tense of subjective experience: “amusing,”“annoying,”“anxious/

DE DEE E E ”“ ”“ ”“ ”“ DEEEEEE E DDEEEEEE E D E E tense, beautiful, calm/relaxing/serene, dreamy, energizing/ DEE EEEEE E DDE EEEEE FEE D EEE E FE E EE E EEE EE EEEEEEE E E China E E EEEEEEEEEE E E USA FFE EE FFEEE E E E EEEEEE E E EK EEEEE E E EE EK EE E E EE EE EEE EE EDE EEEEEEE EE E ED EED E EEE EE EE D EED EE EE EE ”“ ”“ ”“ DD DKE DD E KE E DDEDE DKE E DDE E KE E E DEE EEEEE E D E EEEEEE E EDK pump-up, erotic/desirous, indignant/defiant, joyful/cheer- D DD EEEEE EEEE EE E KKK D DD EEEEEEEEE E EKK K E DE D EEEEE EE E EK E DE DD EE E E K E E E E DE EEEE E DKKK E E D DEE EEEEE E D KKK EE E DD DE E EE E E D KKK E E EEE EEE D E EEE EEE DKKKKKE E EEEEE EEEE E DD EEEEEE E EEKK KKKKE E E EEEE E DD EEE EEEE EEKK K KKE H HH E EEEE ED E EEEEEEKE EK KK H HHHH E EE E DED EE EEEE KE EK EKKEED EK ”“ ”“ ” “ ” HHHHHHH DE DDDEE DED E E EKEEDK EK HH HEHH DE DDDEE EDD EE KK KK E D E E E E K KKD H H ED E EEE EK KKD ful, sad/depressing, scary/fearful, and triumphant/heroic. HH HHE EDD D E E K EKKKDK HH HHE E D D E E E E KDKK H H HE E E DD EE E KEK KKKK HH H E H E E E DD EEE KEK KKK HHHH HBE E E DDDDE DD E E E KKKK H H BE E DDDDE DD EE E KKK KK HHHH E DDDD DE E K KKKK HHHHH E DDDD DE E KKKK HHH H H E D DE ED K K KKKK H HH H EE D DDE E ED K K KK H HHHH H L EE D D D EH D HHH H L E D D E D H D A H H E E EDDDE HD K E K HA H DE E EDDED HD K E K HAHE HHH HEEHK E D E DDKDKE KK KK AHE HHH HEEHK E E DDKKE K C KK KK We can infer that these 13 dimensions correspond to qualities of A JJJA HH H HD D DMD E K C K K K A JJJA HH H HD D DMD E K K A H H EH E D D D EE K KK J AA A A H H HEH E D D D EE E D K KK AAA JJJ AA JA J JA EJAE E HH EAEE DE EEE MKDK K AAAAA A JJ A JGJJ J EJAE E H EAEE E EE MKKKKK AAAAA JA J J GJJ J JE JEG HEDE E E EKEF KKKKKKKL A AA J JJ JJ JJJJE JEGHHEDE E D DFE EKEF K KKKKKLK BB AAAA JJ J J JJ JJJJ J GHHEJ E ED KD D JDF K K KKKKKKK K BB AAAJ JJ J JJ JJ J GHEJ EEE ED KD J DKEK KKKKK K B JJ JJ J GD EE D EFD EK K D K GB AA J J JJ AJ J J JE GD E D D EF KH D GAG AAA AJ J JJ AJ JJ J J E E EFF DFD F H K MK GAJ A A J JJJJ J J E EFF DF DD F K MK J AGG J JJJJJJJ DJDJ J E DJD HHF FDFDFD E FKK GAGGJ JJJ JJJ JJ D DJDJ JJ E DJD HHF FE FFFD E FKK music that evoke similar feelings in China and the United States. AG GJJJJJJ J JJ DJD J EE FEFEFF FE KK A GGJ JJ JJ JD F FEE FEFF FE KKK G G J J J EF F F EK G G G JJ JJ J J E F F FE JG GGJ GJ J J JJJJJ JJ G J JFD FFF EFFF JG GGJ GJ J J JJJJ J GJ JF FJFD F FF EEF F GG J J JJJ J H JJ F FFF FFFFF EFEFE GG J J J J H G J F F DH FFFF EFFEFF G GG JGGJGJJ JJ G FFD DH F FF FFFF F FD DM G GG JGGJGJJ JJ FFD F FFF F F FDD DM G GG G JGJ J JF F F FFFF FFFF F DDD MD GGG G GG JGJ JJ JF F FF F FFFF F DD D MD G GGG GG J G DM FFFFFFF DDD G G GG GJ G J G DM FFFFFFF DD GGGG GG J GG GGGJJ J MG D FFFF FFFF BD MMMDD GGGGG GGG GG GGGJ J MG L D FF FFFF BD MDMMD Note that some dimensions also involve secondary or tertiary G GG G G G GGG MMM L F FFF F FMDMMM GG GGG G GGG MMM C F F F F FMMMMM GGGG G GG G MG C AF F F F FF DMM G GGG GG MG AF F F FE FF DM M GGGGGG G G AG J G G MG F H A F FE FFFFFM M GGGGGGGGG G AG J G GGMMG F BFF H F FFFFFFMM G G G GGGG G GG G GM F B F F FFM GGG G GGGG G GG G F FF F G GG GGG GJ HHHFF FFF FF M G GGG GGG GJ G HHHFF FFF FF M G G G G G F PSYCHOLOGICAL AND COGNITIVE SCIENCES GGGG GG G AAAG HHH F EFF F FF GGGGG GG GG AAA HHH FF CEFF FFLF — “ ” GG G G GGG G G JG G H C F FF KF HC FF FLL G G G G G GG G G J G G HH C F F KF H FF L G GGGG G G GGG G G G HH FF L F F GG GGG G GGGG GG GGF H C F L FFF F categories for example, the indignant/defiant dimension also GGG GGGGGGG GGG GG GG G GF C F FF K GG GGGGGGGG GGGG GGG GGGB G FFFF FF F K GG GG GGGGG GGBB GGGGGB G K FFFFF FFF K K G GGGGGG GGBBB GGG GB K FFF FF FKL K G GGGGGGGGGGG B G G B BC HM L FLF FL F LK G GG GGGGGGG G G BC HM L L F FL K K GG GGG GGGG G GG BG C L L L F CCL K M G GGG GGGGG G GG BGGG C L L L FF CC M M G GGGGGGG GG G GGG B L CL L KL FFL C MM M G GGGGGGG G G G B L CCLB L KL FLF C C MM M G G G G G G B BGG C IB CB K L FFF FFF C M M G G G G B B G GC GB IB KL LL F FFFF MMM M “ ”— G GGGGG IG GG GC GB B C B KL L LLF F MMMM M M G G GGGGG IG GG C B C B K L LL MMM MM G G IB GGGC B C LF C M M MM G GG G M G IB GGGCC B C LF C L M M M loads on anger indicating that these categories were used G GG GGGMGGG GCC CG B B BB B L L LCL L D M M MMMMMMMM GG GI GGGG G C CG BB B BB B LBL LCL C D M MM MMMMMMMMMM G I G G I BB B F B L CC C MM MM MMMM GG G GGGB I B F L L CC C M M MMMM GGGGGGGGG GG BIB G BA B A B LL LC CM CMMMM MMM M MMM GGG GG GG I IIIBI G BA B A F B L LC CM CMMMM MMMMM MMM G GIG G GI I BIIII I I IB B BBF F L CC MMM MMMM M IGGG GII IB II I IB B BBB FF L CC MMMMM MMM MMMM G I G I IGGGIGB B BBB BBB L FLLC L C CCCL CMM MMM MMMMMM G I GGGI GGGGB BG BBBB BB L LLC L LC CCCL CMM MMMM MMMMMMMM G GGIGGGG GI MCGG C BBBBB BBB LLLLLLLLL L MMMMMMMMMMMMMMMMM G GGGI G I IIGI GMC G C B BBBBB BBB LLLLLLL C L MMMMMMMM MMMMMMMM G G GG GGGIII III MGG B BBBCB LLCCLLLL LLLC C MMMMMMMMMMMMMMM G G GGGIII II MG BBBCB LLCCLLLL LLL LC MMMMMMM MMMMMMMM GG G G IIII GI GMMML BBB LLLLLL LL MMMM MMM M GG G GG I III G GMMMCL BBB LL LLL LLLLLLLLM MMMMMMMMM similarly to label music samples. These findings replicate past MGGGI G CC C BBB LLLLLLLLLLLLLLM C MML MMM MMM MGGG GG C C BBB L LLLLLLLLLLL C L M MM MM I I GG B L LLLL L MM MMM M I I G LLLLLLL L MM M G C CLLL L L M G C CLL L M studies’ findings that several dimensions of subjective experience M M can be evoked across cultures by music [“calmness,”“energy,” “ ”“ ”“ ” “ ” Fig. 3. Visualizing the 13-dimensional structure of subjective experience fear, joy, sadness, and tension (29)], but also uncover evoked by music reveals smooth gradients between specific feelings in both other dimensions that had not previously been demonstrated cultures. To visualize the feelings associated with music, maps of average to be associated with distinct qualities of music, including category judgments of the 1,841 music samples within a 13-dimensional “amusement,”“annoyance,”“defiance,”“desire,”“dreaminess,” categorical space of subjective experience were generated. t-SNE, a data and “triumph.” visualization method that accurately preserves local distances between data points while separating more distinct data points by longer, more approxi- Are the Feelings Associated with Music Distributed along Gradients of mate, distances (42), was applied to the concatenated US and Chinese scores Subjective Experience? Having thus far examined the dimension- of the 1,841 music samples on the 13 categorical judgment dimensions, ality and conceptualization of the semantic space of subjective generating coordinates of each music sample on 2 axes (this does not mean the data are in fact 2D; see SI Appendix, SI Discussion). The individual music experience associated with music, we sought to establish how the samples are plotted along these axes as letters that correspond to their feelings associated with music are distributed. Do they lie within highest-loading judgment dimension (with ties broken alphabetically) and discrete clusters, as predicted by basic emotion theories (42), or are colored using a weighted average of colors corresponding to their scores along continuous gradients between categories of subjective ex- on the 13 dimensions (see SI Appendix, Methods S8, for details). The perience, as recently documented in our investigation of the resulting map reveals gradients in subjective experience, such as a gradient feelings associated with videos (12)? As can be seen in Fig. 3, the from anxious to triumphant music. For an interactive version of this map, feelings associated with music lie along continuous gradients visit https://www.ocf.berkeley.edu/∼acowen/musicval.html. Maps colored between categories of subjective experience. These gradients using projections of the mean US judgments only (Top Right) and Chinese between different categories of subjective experience are evident judgments only (Bottom Right) on the same 13 dimensions demonstrate that reported experiences of the 13 distinct feelings associated with music and when we visualize smooth variations in the categorical judgment smooth gradients between them is largely preserved across the 2 cultures. profiles of the 1,841 music samples using a method called t-distributed stochastic neighbor embedding (t-SNE) (42). t-SNE projects data into a 2-dimensional (2D) space that largely pre- jective experience in dysfunction, neurophysiology, and other serves the local distances between data points. In Fig. 3, t-SNE is important areas of research. For example, if subjective experi- used to visualize the smooth gradients between feelings associ- ences could be sorted into discrete categories, then diagnoses of ated with music, represented in different colors, and the extent mood disorders might rely on how much time is spent experi- to which they are preserved across cultures (see SI Appendix, encing each feeling, one at a time; however, if subjective expe- Methods S8, for details). To allow further exploration of the riences form continuous gradients, it could be more effective to feelings associated with music and the smooth gradients between them, we also provide an online, interactive version of Fig. 3 in parameterize feelings along multiple continuous scales. It is which each music sample can be played while viewing its cate- crucial, then, to understand whether the smooth gradients that gorical and affective scale ratings: https://www.ocf.berkeley.edu/ we observe in the map actually correspond to smooth variations ∼acowen/music.html. in the feelings associated with the music samples in individual Understanding whether subjective experiences occupy discrete listeners. clusters or continuous gradients informs the classification of To answer this question, we analyzed the degree to which stimuli and response, preliminary to studying the role of sub- reports of valence, arousal, and other affective features steadily

Cowen et al. PNAS | January 28, 2020 | vol. 117 | no. 4 | 1929 Downloaded by guest on September 26, 2021 transitioned along gradients between categories of subjective and energizing. Thus, the feelings associated with music are experience. (Here, we collapsed data across cultures, given the neither entirely discrete nor entirely miscible, but rather are similarities revealed in Fig. 3.) To do so, we first determined if distributed along specific gradients of subjective experience. continuous variations in the proportions of participants who assigned each category to each music sample more accurately Experiment 2. Examining the Primacy of Categories in Feelings predicted each sample’s affective features than the one category Associated with Western and Traditional Chinese Music Targeting that was assigned most frequently. This was determined to be Valence and Arousal. In a second experiment, we sought to ad- true: the 13 dimensions explained 81.0% of the variance in dress 2 alternative explanations for the finding that specific judgments of the affective features; a model comprising 13 bi- feelings associated with music were better preserved across cul- nary features designating the dimension to which each music tures than valence and arousal. One explanation is that music sample was most strongly assigned explained 57.9% of the vari- most reliably conveys whatever it is that it intended to convey, ance in judgments of the affective features; and a similar model which would entail that the music samples in Experiment 1 re- that included magnitude (the score on the most strongly assigned liably conveyed specific feelings rather than broad affective dimension) explained 64.0% of the variance in judgments of the features across the 2 cultures because people provided music affective features. The 13-dimensional model performed signif- samples targeted toward specific feelings. A second alternative icantly better than either of the mutually exclusive models (P < explanation is that the primacy of specific feelings is a property 0.002, bootstrap test; SI Appendix, Methods S9), consistent with not of music in general but of Western music. To test these the notion that valence, arousal, and other affective features hypotheses, we examined the primacy of categories in responses steadily transition along gradients between categories of sub- to both Western and traditional Chinese music samples gathered jective experience. Moreover, to determine if these findings on the basis of evoked valence and arousal rather than specific could be attributed to correlated ambiguity in judgments of the feelings. categories and affective features—for example, the same music To this end, we assembled 2 additional sets of music samples. being judged as scary and negative by some participants and One set was gathered by 22 US participants (6 women, mean triumphant and positive by others—we correlated the SD of the age = 30.7) who were each asked to contribute 5-s music samples category judgments of each music sample with that of the af- conveying 12 different levels of valence and arousal: very fective feature judgments of each music sample. These had a pleasant, somewhat pleasant, very unpleasant, somewhat un- mildly negative, rather than positive, correlation (Pearson’s r = pleasant, very stimulating, somewhat stimulating, very subdued, −0.112; Spearman’s ρ = −0.109). Hence the smooth gradients somewhat subdued, pleasant and stimulating, pleasant and sub- between categories most likely cannot be explained by inter- dued, unpleasant and stimulating, and unpleasant and subdued. subject ambiguity; rather, they point to intermediate blends of A second set of music samples was gathered by a group of categories of subjective experience traditionally thought of 6 native Chinese participants familiar with Chinese traditional as discrete. music (5 women, ages 30 to 50), who were asked to provide Fig. 4 presents the number of music samples that loaded sig- Chinese traditional music tracks that conveyed as many as pos- nificantly on each of the 13 dimensions of subjective experience sible of the 12 valence and arousal levels. From each Chinese that we uncovered and on each combination of dimensions. This traditional music track, 3 evenly spaced 5-s samples were then analysis reveals the many blended feelings that can be evoked by extracted. Using these procedures, we compiled a total of brief music segments. For example, inspection of Fig. 4 reveals 138 modern Western music samples and 189 traditional Chinese that music traverses gradients from “anxious” to “energizing,” music samples. from “dreamy” to “scary,” from “indignant” to “triumphant,” We recruited new participants from the United States (n = and from “calm” to “sad”. However, not all categories of sub- 580, 288 women, mean age = 35.9) and China (n = 363, jective experience can be blended together in a single music 249 women, mean age = 23.2) to judge the new music samples in sample; for example, no music samples were labeled both sad the same response formats used in Experiment 1. For each music sample we collected an average of 23.8 responses in each re- sponse format in each culture, amounting to a total of 151,367 individual judgments of the 327 newly gathered music samples A. Amusing 512 B. Annoying (19,147 forced-choice categorical judgments and 132,220 9-point- C. Anxious, tense 256 scale judgments). Interrater agreement rates were similar to D. Beautiful 128 Experiment 1, with 37.8% of raters on average choosing the most E. Calm, relaxing, serene 64 agreed-upon category of subjective experience for each music F. Dreamy sample (versus 42.3% in Experiment 1). Interestingly, Chinese G. Energizing, pump-up 32 raters converged slightly more in their ratings of traditional H. Erotic, desirous 16 Chinese music samples than US raters (SI Appendix, Fig. S6), I. Indignant, defiant 8 suggesting that ratings are likely also informed by culture-specific J. Joyful, cheerful 4 understandings of music (5). K. Sad, depressing Even so, as summarized in Fig. 5, our results regarding the L. Scary, fearful 2 M. Triumphant, heroic primacy of categories of subjective experience were closely rep- <=1 licated when the models trained in Experiment 1 were validated A B C D E F G H I J K L M on judgments of music samples selected based on their valence Fig. 4. The 13 distinct dimensions of subjective experience associated with and arousal levels. Correlations across the 2 cultures in the music can be blended together in a number of ways. Represented here are projections of judgments of the new music samples onto the the number of music samples that loaded significantly on each dimension, or 13 varieties, or kinds of subjective experience that we uncov- kind, of subjective experience (diagonal) and on pairwise combinations of ered, were uniformly positive [P ≤ 0.003, q(FDR) ≤ 0.003, < dimensions [q 0.05, Monte Carlo simulation using rates of each category bootstrap test; because the dimensions were derived in Experi- judgment, Benjamini–Hochberg FDR corrected (39); see SI Appendix, Meth- ment 1, the data from Experiment 2 enable us to examine these ods S10, for details]. Categories are often blended together, combining, “ ” for example, “amusement” with “joy,”“indignance” with “triumph,” correlations without double-dipping ]. Correlations for 5 of the “dreaminess” (or “eeriness”) with “sadness,” and “anxiety/tension” with dimensions (“annoying,”“beautiful,”“energizing/pumped-up,” “triumph.” “joyful/cheerful,” and “scary/fearful”) exceeded the correlation

1930 | www.pnas.org/cgi/doi/10.1073/pnas.1910704117 Cowen et al. Downloaded by guest on September 26, 2021 for valence [P ≤ 0.002, q(FDR) ≤ 0.05, bootstrap test, 2-tailed A 1 comparison]. .8 These results further support the conclusion that judgments of .6 g music in terms of affective features derive from the cross-

.4 c

c culturally preserved experience of specific feelings. In particu-

.2 musing nnoying nxious rousal alence Er- oti tense pump-up Scary fearful Joyful cheerful Beautiful Calm relaxing Defiant Heroi Energizin A A A A Dreamy Sad V

Correlation 0 lar, we found, using the model trained in Experiment 1, that GL JADBCE E IMF FKH category judgments again predicted valence and arousal judg- B ments from the other culture as robustly as, or more robustly US Chinese than, valence and arousal judgments, even though the music category category multiplied judgments judgments samples were selected on the basis of their valence and arousal – equals levels (Fig. 5 B E). This finding rules out the possibility that r correlates music reliably conveys whatever it is selected to target and in- US Chinese stead supports the notion of the primacy of specific feelings as- category category sociated with music. Furthermore, we found that this holds true to valence to valence weights from weights from both for Western music samples (Fig. 5D) and for traditional Exp. 1 Exp. 1 Chinese music samples contributed by native Chinese partici- US Chinese valence valence pants (Fig. 5E), indicating that the primacy of the induction of judgments YXUS Chinese Zjudgments specific feelings is not merely a property of Western music. predicted .81 valence .67 valence .80 predicted Given the surprising range of feelings associated with music from Chinese judgments judgments from US samples selected based only on their valence and arousal levels, categories categories we measured the number of distinct dimensions of subjective 1 1 * 1 experience associated with these music samples across cultures. C.8 * * .8 * .8 * .6 .6 .6 We did so by computing partial correlations in judgments pro-

All .4 .4 .4 .2 .2 .2 jected onto each of the 13 dimensions. Even after controlling for XYZ XYZ XYZ XYZ Chinese XYZ XYZ Traditional

0 US Modern 0 0 Valence Arousal Valence Arousal Valence Arousal every other dimension, the partial correlation across cultures for each of the 13 dimensions was significantly greater than zero C: Chinese Traditional Sample D E USA [raw r ≥ 0.13, P ≤ 0.007, q(FDR) ≤ 0.05, bootstrap test], in- U: US Sample UU UUU U UU U Amusing U C UUUUUU UU UU UUU CUUUUUUUUUUU U Energizing, pump-up UC C UUU UU dicating that all 13 varieties of subjective experience uncovered U U UUU U UCCCC UU UU UU PSYCHOLOGICAL AND COGNITIVE SCIENCES U C UU U C U UU U U U UU UUCU U UU U UU U U U U U UUU U UCU U U UU in Experiment 1 were also conveyed by distinct music samples C UUU U U C U U Indignant, defiant C CU U C UCU C C UU UU U CC CCUC U U UCC C U U U U CC C UC CCUCCU across cultures in Experiment 2. (Here we measured di- U C C U U U C C UU U UCCC CCU U U U C U U mensionality using a confirmatory approach rather than PPCA, U Annoying UU C U C U U U U C U CUC U U C UU U U UUU UU U UU U UU C C CC U C UU CCCCCC C given that Experiment 2 was designed to further investigate the U UU C C C CCC U C Scary, fearful UUCCCCCCC CCCCC CC C C U C U CCCC C C C U C UU CCCC C CCCCCC C CCC C U C C C CU C CCC CC C cultural universality of the dimensions of subjective experience Joyful, U C CC C U U UC C CC UC CCC C CC C C C U U U CCCCCCC CU CCCCC C CCCCCC C C CCUCU CCCCC UC CC CCCC CCC C C UCC C C CCC CCUCCC cheerful UUC U C C CC C C U U UUUU C U that we had already uncovered in a larger sample, not to discover CUU Anxious, UCCC C new dimensions.) When considering the Western and Chinese tense Beautiful UU U Triumphant, heroic music samples separately, 12 distinct dimensions were preserved UU China Erotic, desirous CUC Dreamy F C U U UUU across cultures in the 138 Western samples alone (all except UU U C U U U U U U U CUUUUUU U U U CU UUUUUUUUU C C C U U UUU UU “ ” C C C UCCCC UUUUU UU anxious/tense ) and 9 were preserved in the 189 Chinese sam- U CC C C U U C U C C UUCC U U UU U C U U UUU UCU U UU U U C C C CC UU UU U “ ”“ ”“ ” UU C C C C C U CU U CUC ples alone (all except amusing, annoying, erotic/desirous, U C C CC CC C C CC C C U CCCCCUCU C CC C C C CC C UC CCUCCU C C C C U U “ ” CU C C C C C CUU and indignant/defiant )(SI Appendix, Fig. S7). It speaks to the C C C CC C C C C UCCC C C C C C C U C C CU C C C U U U U C C C C UU C C C CC C C C C U CUC richness of the subjective experiences that are routinely associ- C U CC CC C C C C U U UUU U C CCCC C C CCC CC C C CC CC C C C C CC C UU CCCCCC C CC C U C C CC C C UU C C C CCC C C C C C C CC UCC UUCCCCCCC CCCCC CC C ated with music that, even across just a few hundred music U C C C C C U CCCC C C C C CCCC C CCCCCC C CCC U U C C C C CU C CCC CC C U U C C C C U UC C CC UC CCC C CC C UU C C U CCCCCCC CU CCCCC C CCCCCC samples not targeting specific feelings, so many distinct feelings U U U U CCCCC UC CC CCCC CCC U UCC C C CCC CCUCCC UUC U C C CC Calm, relaxing, serene Sad, depressing UUUU C U were reliably distinguished across cultures. Finally, inspection of the distribution of the feelings associated Fig. 5. The primacy of categories in the feelings associated with Western with music samples gathered on the basis of evoked valence and and traditional Chinese music targeting valence and arousal. (A) Correlations arousal (Fig. 5 D–F) reveals that they were structured similarly to in the feelings associated with music across cultures. The cross-cultural signal those gathered on the basis of specific feelings (Fig. 3). In par- correlation for each dimension derived in Experiment 1 (orange bars) and ticular, we still find gradients of music both “calm” and “sad,” valence/arousal (green bars) captures the degree to which each attribute is “sad and “dreamy,” and so on through “triumphant,”“scary,” preserved across Chinese and US listeners in the 327 music samples gathered “ ”“ ”“ ”“ ”“ ” on the basis of valence and arousal. Many feelings were better preserved anxious, annoying, indignant, energizing, amusing, than valence or arousal. For 5 categories of subjective experience, the dif- “joyful,”“beautiful,” and “erotic.” Again, this was found despite ference from valence was significant [P ≤ 0.002, q(FDR) ≤ 0.05, bootstrap the music samples having been gathered without targeting any of test, 2-tailed]. Error bars represent SE. See SI Appendix, Fig. S7, for separate these feelings. This structure was preserved among both US (Fig. correlations across US and Chinese traditional samples. (B) Specific feelings 5E) and Chinese (Fig. 5F) perceivers. These findings validate the account for the preserved experience of valence and arousal across cultures, high-dimensional, continuous structure of the feelings conveyed even in response to valence and arousal targeted music samples. Category by music across cultures. judgments were used to predict affective feature judgments using the model trained in Experiment 1. (C) Category judgments from each culture were significantly better than valence/arousal judgments from each culture at predicting valence (Left;*P < 0.002, 0.01; bootstrap test) judgments from t-SNE (42) to the concatenated US and Chinese scores of the 327 new music the other culture and nominally better at predicting arousal judgments. This samples on the 13 categorical judgment dimensions derived in Experiment 1. held true for the modern US music samples (Middle;*P = 0.002, 0.02) and the The resulting map reveals a highly similar structure to that revealed in Ex- traditional Chinese music samples (Right;*P < 0.002) separately. These re- periment 1, with similar gradients in subjective experience, such as a gra- sults are consistent with the hypothesis that specific feelings are elicited by dient from anxious to triumphant music. Again, reported experiences of music and then subsequently used to construct valence/arousal judgments these 13 feelings and smooth gradients between them were largely pre- in a more culture-specific process of inference. (D) Visualizing the 13- served across the 2 cultures (E and F). To navigate an interactive version of dimensional structure of subjective experience incidentally evoked by mu- this map and explore the valence and arousal targeted music samples, visit sic samples gathered on the basis of only valence and arousal. We applied https://www.ocf.berkeley.edu/∼acowen/musicval.html.

Cowen et al. PNAS | January 28, 2020 | vol. 117 | no. 4 | 1931 Downloaded by guest on September 26, 2021 Discussion emotional vocalization (46). Further study is required to de- Music is a universal and vital part of social life across cultures (4, termine how the rich array of subjective experiences uncovered 5). Central to its meaning, and likely to its cultural significance, is here map onto a wide array of acoustic and perceptual features its well-documented power to evoke rich subjective experiences of music, perhaps in complex, nonlinear fashion. (1–3, 23–27, 33). What is less well understood is the taxonomy of Our second interest was to examine the distribution of feelings — feelings evoked by music—that is, how these feelings are arranged within a semantic space of subjective experience in particular, within a semantic space—and the extent to which such a taxonomy boundaries between categories of subjective experience. In of subjective experience is preserved across cultures. contrast to discrete theories of emotion (47), we found that the With quantitative and conceptual approaches (9, 11, 12), we feelings associated with music were not characterized by discrete examined the shared semantic space of subjective experience clusters of states, but by smooth gradients between categories of “ associated with Western and Chinese music by participants from subjective experience. Music occupies gradients from amuse- ” “ ”“ ” “ ” “ ” “ ” the United States and China. Our focus was to test hypotheses ment to joy, sadness to calmness, and fear to tension “ ” related to 3 properties of this semantic space: its conceptuali- to triumph, among many others (Figs. 4 and 5). These findings zation, focusing on how specific feelings and broad affective support a shift from the predominant scientific focus on how features contribute to the feelings associated with music; its di- discrete patterns of expression, physiology, and neural activation mensionality, or number of distinct kinds of feelings associated distinguish discrete emotional states (15, 22) toward the in- with music; and its distribution of states, focusing on the nature vestigation of how the variability and blending together of sub- of the boundaries between categories of subjective experience. jective experience may be represented in continuously varying Guided by this conceptual framework, 2,777 US and Chinese patterns of expression, physiology, and neural activation. participants judged over 2,000 samples from modern and clas- Our final focus was to examine a critical issue related to the sical Western music as well as traditional Chinese music. Judg- conceptualization of subjective experience: whether the feelings ments were made either in terms of which category of subjective associated with music are explained at a more basic, cross- cultural level by categorical labels or scales of affect. It has been experience, from a list of 28, was evoked, or along scales that posited that, in subjective experience, valence and arousal can be captured 11 affective features proposed by appraisal and com- thought of as core, low-level interpretive processes from which ponential theories to underlie emotional experience. Applying more specific feelings are constructed (13, 14, 38). In contrast to large-scale statistical inference techniques, we compared the this claim, we found that associations of music with specific preservation of the recognition of 28 categories and 11 scales of feelings such as amusement and triumph were better preserved affect across 2 different cultures, modeled the latent space that across cultures than associations with valence and arousal (Fig. captured the shared variance in judgments between cultures, and 1). Furthermore, judgments of these affective features were interrogated the boundaries between the 13 categories that were better predicted by category judgments from the other culture found to underlie this latent space. than by the same affective feature judgments from the other With respect to the dimensionality of the semantic space of culture (Fig. 1B), even in music samples gathered on the basis of subjective experience, many studies have targeted 6 or fewer valence and arousal associations (Fig. 5). These results suggest categories of emotion and relied on either interrater agreement that categories of subjective experience such as “amusement” rates (1, 29, 43) or factor analysis (3, 14, 37) to characterize and “desire” may be more directly associated with music, subjective experience (11). With data-driven statistical tech- whereas judgments of broader affective features may be inferred niques applied to a vast array of stimuli, we uncovered 13 distinct in a more culture-specific manner from these categories. These dimensions of subjective experience that were associated with findings contrast with the claim that valence and arousal are music in China and the United States: “amusing,”“annoying,” “ ”“ ”“ ”“ ” basic building blocks of subjective experience (13, 14, 17, 38), anxious/tense, beautiful, calm/relaxing/serene, dreamy, instead suggesting that they are higher-order inferences (11). “ ”“ ”“ ” energizing/pump-up, erotic/desirous, indignant/defiant, One might argue, however, that people have less conscious “ ”“ ”“ ” “ joyful/cheerful, sad/depressing, scary/fearful, and tri- access to valence and arousal levels than to more specific cate- ” umphant/heroic (Fig. 3B). It is noteworthy that the subjective gories of feeling states, causing reports of valence and arousal to experiences reliably associated with music not only include be more subject to bias and therefore less predictive. However, “ ”“ ” commonly studied emotional experiences ( fear, sadness, given how well valence and arousal judgments within each cul- “ ” joy ), but also less commonly studied subjective experiences, ture were predicted by category judgments (Figs. 1 and 5), it “ including traditionally understudied positive emotions ( amuse- seems that the valence and arousal judgments were reliable and ment,”“awe,”“calmness,”“triumph”) (11, 16). It will be im- quite precise—even if the meanings that they were precise in portant to examine whether these feelings associated with music capturing were culture-specific. Perhaps, then, participants’ va- emerge in cultures beyond the United States and China. Ger- lence and arousal ratings capture something distinct from “real” mane to the more general study of aesthetic experience will be valence and arousal levels to which people do not have conscious the study of whether the feelings elicited by other kinds of aes- access. But falling back on this hypothesis seems to undermine thetic stimuli—painting, poetry, drama, dance, sculpture—are the original motivation behind the constructs of valence and distributed in semantic spaces similar to the one revealed for arousal, which emerged in self-report data (14). music. Understanding the rich variety of feelings associated with In considering the evidence for universals in music-related music may be particularly useful for studies of the physiological feeling, it is likely that the cultural similarities that we observed and neural representations of distinct subjective experiences and were influenced by participants’ past exposure to music. For the emerging literature of neuroaesthetics (8, 15). example, the association of music with an “erotic/desirous” di- Inspection of the music that evokes each dimension (https:// mension may derive from previous exposure in both cultures to www.ocf.berkeley.edu/∼acowen/music.html) seems to point to music played during romantic scenes in films. Such semantic distinctive acoustic and perceptual underpinnings. For example, associations may have influenced our findings. However, this is “indignant/defiant” music tends to involve a “growl-like” timbre, less likely to be the case for Westerners’ responses to Chinese consistent with observed similarities between aggressive music traditional music, a genre with which most Westerners have and threatening vocal expressions in various species (44). “Scary/ limited familiarity, yet for which valence and arousal judgments fearful” music tends to involve harmonic dissonance, an example were still best predicted by category judgments from the other of how some feelings may be associated with specific harmonic culture. These findings render it unlikely that the primacy of (45) and melodic intervals, potentially in a manner that mirrors specific feelings over broad affective features could be fully

1932 | www.pnas.org/cgi/doi/10.1073/pnas.1910704117 Cowen et al. Downloaded by guest on September 26, 2021 explained by learned semantic associations. Nevertheless, given and China. To understand how and why music evokes these the influence of globalization, we avoid inferences about bi- feelings, future studies will need to move beyond the pervasive ologically ingrained universals. focus on simple models of subjective experience composed of In introducing quantitative approaches to study US and Chi- 4 to 6 emotion categories or levels of valence and arousal (1, 29), nese experiences of music, we hope to set the stage for a broader given that these models confound many distinct variables (11). project documenting shared and culture-specific understandings Studies of the human response to music—its universality (32– of music around the world. Different experiences may emerge 34), acoustic underpinnings (1), and neural representation (8, when studying musical traditions from other regions, such as 15)—should therefore incorporate a rich taxonomic structure of Africa and South America. It will be particularly informative to subjective experience. Similarly, technological endeavors to build study the semantic space of subjective experience associated with machines that curate, compose, or alter music should in- music in small-scale cultures with limited Western contact. corporate the wide range of feelings uncovered here if they aim We also recognize that, although participants in the United to reproduce the full complexity of human musical composition. States and China differ culturally on average, neither group is Finally, it is worth noting how the present findings dovetail homogeneous. In deriving a semantic space that captures the with the implications of recent research on subjective responses similarities across countries in how participants represent their to strongly evocative videos. Cowen and Keltner (12) found that, subjective experiences, we set aside variation in responses within when people report their feelings in response to a wide range of each country, an important topic for further study. We also did evocative videos, they reliably distinguish among at least 27 va- not delve deeply into what past work suggests may be rich rieties of subjective experience. In that study, specific feelings culture-specific understandings of music. For example, it has such as “awe” also emerged as more primary than broad affective been found that when listening to Hindustani ragas known to features such as valence and arousal in determining the structure have specific emotional connotations, Westerners can reliably of subjective experience and were also found to be organized identify the intended emotions, but do so with less granularity along continuous gradients. However, those results emerged than Indian listeners (5). While some evidence was observed for within a single culture and may have been influenced by the culture-specific understandings of traditional Chinese music (SI semantic content of the videos. The present study examined — Appendix, Fig. S6), the precise nature of these understandings subjective responses to a largely nonrepresentational stimulus or, for that matter, understandings of music from more specific modality across 2 different cultures. Together, these results — subcultures is an interesting topic for further study. converge on a taxonomy of subjective experience consisting of a Importantly, the present study did not focus on the distinctions rich array of distinct categories bridged by smooth gradients. between experienced and perceived feelings in music. Recent Questions regarding the structure of subjective experience PSYCHOLOGICAL AND COGNITIVE SCIENCES studies have found these constructs to be highly correlated, but bear upon central theoretical claims in affective science. They – not identical (48 50). It would be interesting for future studies to guide studies in fields ranging from neuroscience to machine interrogate the possible differences in structure between per- learning. Our method of interrogating how subjective experi- ceived and evoked feelings in music, perhaps incorporating ences associated with music are situated within a semantic space physiological measures. Moreover, it could be argued that the reveals a more complex taxonomy of feelings than is typical in brief, 5-s duration of our stimuli makes them unlikely to evoke — existing accounts of how subjective experience is organized. By certain feelings to cause people to become fixated and en- revealing that the feelings evoked by music span at least 13 dis- tranced, evoke memories that give rise to nostalgia, engage tinct dimensions and are distributed along continuous gradients, physiological changes that evoke empowerment (51), or enable a we provide a road map for the study of how subjective experi- deeper appreciation of their historical and artistic context, for ences are associated with physiological responses, are repre- example. Future studies could examine the structure of such sented in the brain, and can be predicted using machine learning. responses using longer music samples. Although the present study cannot determine the full range of Methods subjective experiences that can be evoked by musical composi- Judgments were obtained using Amazon Mechanical Turk and Qualtrics. A total tions or whether these responses are innate, it places a lower of 2,777 participants (1,591 from the United States, 1,258 from China) took part bound on the richness of the feelings reliably associated with in the study. The experimental procedures were approved by the Institutional music in 2 rather different cultures. Even brief, 5-s music samples Review Board at the University of California, Berkeley. All participants gave are reliably associated with at least 13 distinct dimensions of their informed consent. Data and analysis code can be requested at https:// subjective experience by participants within the United States forms.gle/73Diih84zksjkASe9.SeeSI Appendix for details.

1. P. N. Juslin, P. Laukka, Expression, perception, and induction of musical emotions: A review 11. A. Cowen, D. Sauter, J. L. Tracy, D. Keltner, Mapping the passions: Toward a high- and a questionnaire study of everyday listening. J. New Music Res. 33,217–238 (2004). dimensional taxonomy of emotional experience and expression. Psychol. Sci. Public 2. P. N. Juslin, S. Liljeström, D. Västfjäll, G. Barradas, A. Silva, An experience sampling study Interest 20,69–90 (2019). of emotional reactions to music: Listener, music, and situation. Emotion 8,668–683 (2008). 12. A. S. Cowen, D. Keltner, Self-report captures 27 distinct categories of emotion 3. M. Zentner, D. Grandjean, K. R. Scherer, Emotions evoked by the sound of music: bridged by continuous gradients. Proc. Natl. Acad. Sci. U.S.A. 114, E7900–E7909 – Characterization, classification, and measurement. Emotion 8, 494 521 (2008). (2017). — 4. G. Rouget, M. Buckner, Musical efficacy: Musicking to survive The case of the pygmies. 13. L. F. Barrett, Z. Khan, J. Dy, D. Brooks, Nature of emotion categories: Comment on Yearbook Tradit. Music 43,89–121 (2011). Cowen and Keltner. Trends Cogn. Sci. 22,97–99 (2018). 5. W. F. Thompson, L.-L. Balkwill, “Cross-cultural similarities and differences” in Hand- 14. J. A. Russell, Core affect and the psychological construction of emotion. Psychol. Rev. book of Music and Emotion: Theory, Research, Applications, P. N. Juslin, J. A. Sloboda, 110, 145–172 (2003). Eds. (Oxford University Press, 2010), pp. 755–788. 15. P. A. Kragel, K. S. LaBar, Decoding the nature of emotion in the brain. Trends Cogn. 6. P. N. Juslin, P. Laukka, Communication of emotions in vocal expression and music Sci. 20, 444–455 (2016). performance: Different channels, same code? Psychol. Bull. 129, 770–814 (2003). 16. Y. Wu, P. Muentener, L. E. Schulz, One- to four-year-olds connect diverse positive 7. D. A. Schwartz, C. Q. Howe, D. Purves, The statistical structure of human speech emotional vocalizations to their probable causes. Proc. Natl. Acad. Sci. U.S.A. 114, sounds predicts musical universals. J. Neurosci. 23, 7160–7168 (2003). – 8. S. Koelsch, Brain correlates of music-evoked emotions. Nat. Rev. Neurosci. 15, 170–180 11896 11901 (2017). (2014). 17. J. Posner, J. A. Russell, B. S. Peterson, The circumplex model of affect: An integrative 9. A. S. Cowen, D. Keltner, Clarifying the conceptualization, dimensionality, and struc- approach to affective neuroscience, cognitive development, and psychopathology. ture of emotion: Response to Barrett and colleagues. Trends Cogn. Sci. 22, 274–276 Dev. Psychopathol. 17, 715–734 (2005). (2018). 18. K. Oatley, J. M. Jenkins, Human emotions: Function and dysfunction. Annu. Rev. 10. A. S. Cowen, P. Laukka, H. A. Elfenbein, R. Liu, D. Keltner, The primacy of categories in Psychol. 43,55–85 (1992). the recognition of 12 emotions in speech prosody across two cultures. Nat. Hum. 19. D. Watson, K. Stanton, Emotion blends and mixed emotions in the hierarchical Behav. 3, 369–382 (2019). structure of affect. Emot. Rev. 9,99–104 (2017).

Cowen et al. PNAS | January 28, 2020 | vol. 117 | no. 4 | 1933 Downloaded by guest on September 26, 2021 20. A. S. Cowen, H. A. Elfenbein, P. Laukka, D. Keltner, Mapping 24 emotions conveyed 36. T. Hamamura, S. J. Heine, D. L. Paulhus, Cultural differences in response styles: The by brief human vocalization. Am. Psychol. 74, 698–712 (2019). role of dialectical thinking. Pers. Individ. Dif. 44, 932–942 (2008). 21. A. S. Cowen, D. Keltner, What the face displays: Mapping 28 emotions conveyed by 37. C. A. Smith, P. C. Ellsworth, Patterns of cognitive appraisal in emotion. J. Pers. Soc. naturalistic expression. Am Psychol., 10.1037/amp0000488 (2019). Psychol. 48, 813–838 (1985). 22. F. E. Harrell, Regression Modeling Strategies (Springer, 2001). 38. L. F. Barrett, Valence is a basic building block of emotional life. J. Res. Pers. 40,35–55 23. A. Goldstein, Thrills in response to music and other stimuli. Physiol. Psychol. 8, 126– (2006). 129 (1980). 39. Y. Benjamini, Y. Hochberg, Controlling the false discovery rate: A practical and 24. J. A. Sloboda, Musical structure and emotional response: Some empirical findings. powerful approach to multiple testing. J. R. Stat. Soc. B 57, 289–300 (1995). Psychol. Music 19, 110–120 (1991). 40. M. G. G’Sell, S. Wager, A. Chouldechova, R. Tibshirani, Sequential selection proce- – 25. W. Tsitsos, Rules of rebellion: Slamdancing, moshing, and the American alternative dures and false discovery rate control. J. R. Stat. Soc. Series B Stat. Methodol. 78, 423 scene. Pop. Music 18, 397–414 (1999). 444 (2016). 26. D. A. Hodges, Bodily Responses to Music (The Oxford Handbook of Music Psychology, 41. F. Wilcoxon, Individual comparisons by ranking methods. Biom. Bull. 1, 80 (1945). – 2008). 42. L. V. D. Maaten, G. Hinton, Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579 27. P. J. Silvia, K. Fayn, E. C. Nusbaum, R. E. Beaty, Openness to experience and awe in 2605 (2008). response to nature and music: Personality and profound aesthetic experiences. Psychol. 43. H. C. Lench, S. A. Flores, S. W. Bench, Discrete emotions predict changes in cognition, judgment, experience, behavior, and physiology: A meta-analysis of experimental Aesth. Creat. Arts 9,376–384 (2015). emotion elicitations. Psychol. Bull. 137, 834–855 (2011). 28. A. Gabrielsson, Emotion perceived and emotion felt: Same or different? Music. Sci. 5, 44. C. G. Tsai et al., Aggressiveness of the growl-like timbre: Acoustic characteristics, 123–147 (2001). musical implications, and biomechanical mechanisms. Music Percept. 27, 209–221 29. J. K. Vuoskoski, T. Eerola, Measuring music-induced emotion: A comparison of (2010). emotion models, personality biases, and intensity of experiences. Music. Sci. 15, 159– 45. M. Costa, P. E. Ricci Bitti, L. Bonfiglioli, Psychological connotations of harmonic mu- 173 (2011). sical intervals. Psychol. Music 28,4–22 (2000). 30. P. A. Kragel, K. S. Labar, Multivariate pattern classification reveals autonomic and 46. D. L. Bowling, J. Sundararajan, S. Han, D. Purves, Expression of emotion in Eastern and – experiential representations of discrete emotions. Emotion 13, 681 690 (2013). Western music mirrors vocalization. PLoS One 7, e31942 (2012). 31. A. Norenzayan, S. J. Heine, Psychological universals: What are they and how can we 47. P. Ekman, “All emotions are basic” in The Nature of Emotion, D. Ekman, Ed. (Oxford – know? Psychol. Bull. 131, 763 784 (2005). University Press, New York, 1992), pp. 15–19. 32. T. Fritz et al., Universal recognition of three basic emotions in music. Curr. Biol. 19, 48. L. O. Lundqvist, F. Carlsson, P. Hilmersson, P. N. Juslin, Emotional responses to music: – 573 576 (2009). Experience, expression, and physiology. Psychol. Music 37,61–90 (2009). 33. H. Egermann, N. Fernando, L. Chuen, S. McAdams, Music induces universal emotion- 49. P. G. Hunter, E. G. Schellenberg, U. Schimmack, Feelings and perceptions of happiness related psychophysiological responses: Comparing Canadian listeners to Congolese and sadness induced by music: Similarities, differences, and mixed emotions. Psychol. pygmies. Front. Psychol. 5, 1341 (2015). Aesthet. Creat. Arts 4,47–56 (2010). 34. J. H. McDermott, A. F. Schultz, E. A. Undurraga, R. A. Godoy, Indifference to disso- 50. M. Komosinski, A. Mensfelt, Emotions perceived and emotions experienced in re- nance in native Amazonians reveals cultural variation in music perception. Nature sponse to computer-generated music. Music Percept. 33, 432–445 (2016). 535, 547–550 (2016). 51. W. F. Thompson, K. N. Olsen, On the enjoyment of violence and aggression in music. 35. G. Hofstede, Dimensionalizing cultures: The hofstede model in context. Online Read. Comment on “An integrative review of the enjoyment of sadness associated with Psychol. Cult. 2,1–26 (2011). music” by Tuomas Eerola et al. Phys. Life Rev. 25, 128–130 (2018).

1934 | www.pnas.org/cgi/doi/10.1073/pnas.1910704117 Cowen et al. Downloaded by guest on September 26, 2021