War Versus Inspirational in Forrest Gump: Cultural Effects in Tagging

Proceedings of the Sixth International AAAI Conference on Weblogs and Social Media

War∗ vs 励志† in Forrest Gump: Cultural Effects in Tagging Communities Zhenhua Dong1, Chuan Shi2, Shilad Sen3, Loren Terveen2, John Riedl2

1Dept. of Information Technical Science 2Grouplens Research 3MSCS Department Nankai University Tianjin Department of Computer Science Macalester College Tianjin, China University of Minnesota St. Paul, Minnesota [email protected] Minneapolis, Minnesota [email protected] {chshi,terveen,riedl}@cs.umn.edu

Abstract Our research studies the role of culture in tagging. We specifically conducted an online survey in which subjects People from different cultures vary in cognition, emo- were given the opportunity to tag 20 movies. We ad- tion, and behavior. We explore cultural differences ministered the survey to English-speaking American and in a tagging system. We developed a model of cul- Mandarin-speaking Chinese participants (each in their own tural differences and performed a controlled empirical study with American and Chinese subjects to investi- native language), and we selected suggested tags for each gate questions that arise from the model. American and movie from either an American or Chinese source. This Chinese subjects differed in many ways: the number study design lets us pose three research questions: and types of tags they applied; the extent to which they 1. Do users from these two cultures have distinct personal applied suggested tags or entered new tags of their own; tagging tendencies? Left to their own devices, do they tag and how often they applied tags that originated from a differently? different culture. Our results are consistent with theo- ries of cultural differences between Asian and Western 2. Do users from these two cultures respond differently to cultures. Our findings suggest new opportunities and tag suggestions? mechanisms for shaping user behavior to produce use- 3. Do users from these two cultures respond differently ful tag repositories. when the source of a suggested tag is from their own culture or the other culture? To preview our results, the answer to all these questions is 1 Introduction “yes”. These results are important for both researchers and designers of tagging systems. They help researchers prop- Tagging is a ubiquitous feature in social web sites.Users can erly scope and interpret observed patterns of user behavior, associate short text labels of their choice with items such for example, in identifying factors that cause the behavior. as movies, electronics, photos, web pages, and blog posts. They offer guidance to designers who want to meet user Features like tag clouds help people understand and explore needs and shape user behavior, for example, by suggesting the space of items. Users’ individual tendencies drive their new techniques for tag suggestion algorithms. tagging choices: one user might tag the movie “The King’s The remainder of the paper is organized as follows. We Speech” as a “period drama” while another tags it “abdi- first survey related work, detailing how we draw on and ex- cation.” However, many sites also use auto-complete tag tend it. We then present our research framework, including suggestions to encourage coherent tag repositories. For ex- a model of how culture relates to the tagging process. We ample, a user who begins typing “period drama” may in- next present our research results and then discuss their im- stead choose “period piece” if the system tells them other plications. Finally, we close with a brief summary. taggers prefer it (both tags have been applied to “The King’s Speech” on Amazon.com). 2 Related Work Culture is a set of attitudes, beliefs, values, and practices shared by a people. We can speak of the culture of a nation, Since we study tagging behavior of Chinese and Americans, an ethnic group, an organization, etc. A culture’s core values we first survey relevant work on differences between West- shape how its people make decisions, interpret the world, ern and Asian culture. We then review studies of culture in and relate to others (Hofstede, Hofstede, and Minkov 2010). online communities and collaborative interaction, then nar- Not surprisingly, work in CSCW has found that culture plays row our focus to studies of cultural influences on tagging. a significant role in how people use social systems. Such Cultural Differences A large body of social science re- studies are crucial, because they help in properly interpreting search has identified systematic differences between cul- and generalizing findings and techniques. tures. We focus on research comparing Asian and West- ern cultures. Individualism vs. Collectivism. Asian cul- ∗ 战争, in Chinese tures emphasize the fundamental relatedness of individu- †inspirational, in English als to each other, with a focus on living harmoniously with

82 others (Hofstede, Hofstede, and Minkov 2010; Markus and Kitayama 1991; Nisbett et al. 2001). On the other hand, American culture emphasizes connectedness among individuals less. Instead, individuals seek to maintain their in- dependence by seeking out and expressing their unique in- ner attributes (Lee, Joshi, and McIvor 2007). Analytic vs. Holistic. (Nisbett et al. 2001) finds that Westerners tend to favor context-independent, analytic thought, focusing on salient objects independent of context. Asians, tend to favor context-dependent, holistic thought, attending to the relationship between the object and its context. These studies inspired our analyses and guided the interpretation of our results. Specifically, we applied Nisbett et al.’s holistic vs. analytic distinction to create a novel tag categorization scheme. Figure 1: A model of cultural differences in tagging behav- Cultural differences in online communities As comput- ior. Users are grounded in a culture’s values, which lead ing and communication technologies have spread around to initial personal tendencies in tagging behavior. Personal the world, researchers have studied cultural differences in tendencies evolve via two forces: social influence through technology use. (Chau et al. 2002) found that consumers exposure to other users’ tags, and through an investment in from different ethnic origins use the Web for different pur- one’s own tags. Users may choose tags based on their per- poses and formed different impressions of the same Web sonal tendencies, or they may choose context-specific tag sites. (Lee, Joshi, and McIvor 2007) studied how culture suggestions offered by the system. Users’ adoption of tag influences online consumer satisfaction. (Kayan, Fussell, suggestions is mediated by their desire to conform and the and Setlock 2006) found significant differences in the use consistency of suggested tags with their own beliefs and at- of instant messaging between Asians and North Americans: titudes (cultural resonance). Asians used multi-party chat, emoticons and audio-video chatting more than Americans. These results were consistent with general distinctions between Western (individual- (Chen and Tsoi 2011) studied tagging on two different istic, low-context) culture and Eastern (collectivistic, high- music sites, one popular with Chinese users (SongTaste) context) culture. (Pfeil, Zaphiris, and Ang 2006) found dif- and another popular with European and American users ferences in edits to Wikipedia among people from differ- (Last.fm). The sets of tags on the two sites exhibited sev- ent cultures. (Wang, Fussell, and Setlock 2009) found that eral distinct patterns, including differing in the types of tags working in a mixed-culture group led Chinese but not Amer- applied and the usefulness of the tags. Our work differed ican participants to adapt their communication style, and significantly in the methods used. We did a controlled study: (Wang, Fussell, and Cosley 2011) found that cultural diver- we knew the cultural and linguistic background of our sub- sity can be used to enhance task outcomes through the use jects, all subjects used the same interface, and we carefully of appropriate technological mediation. Our work provides controlled contextual information available to them. Chen specific new insights about cultural differences in tagging and Tsoi compared differences between the global sets of systems, a ubiquitous feature of social online communities. tags on two different web sites, with completely different user interfaces and unknown user demographics. Culture and tagging (Sen et al. 2006) investigated factors that influence how users choose to apply tags, including their 3 Research Framework personal preferencesand tags applied by other members of To organize our research, we modified the model of (Sen et the community that they observe. As detailed below, we al. 2006) to account for cultural differences in the tagging build on this work to account for cultural differences. process, specifically we study how a person’s culture relates A few researchers have studied tagging in cross-cultural to their personal tendency, conformity, and cultural reso- settings. (Dong and Fu 2010) found order differences in how nance. Figure 1 shows the model. The model focuses on Chinese and Americans tagged images. Americans tended cultural elements of tagging and does not attempt to include to tag the main objects in an image first, and background all social and technical aspects of tagging systems. For ex- objects and overall properties later, while Chinese tended ample, the tags a user chooses are obviously based heavily to apply tags about the overall properties of the image at on the characteristics of a particular item (e.g. a movie may first, and the objects in the image later. They attribute these be a drama). However, such aspects less related to culture differences to the analytic/holistic distinction. (Peesapati, are not included. We next detail the model’s components Wang, and Cosley 2010) studied photo tagging, presenting and motivate the research questions we ask about it. subjects with photos from their own culture and from other two other cultures. They found that subjects “felt closer” to Users’ culture implicitly guides how they interpret items photos from their own culture and that there were cultural and apply tags. Clearly, it is simplistic to assume that a per- differences in how subjects tagged. Compared to these stud- son has a single culture: what about someone who is born ies, we conduct a more extensive user study and reveal more and grows up in India, studies in the United States, then specific cultural differences in tagging behavior. works in Germany? However, as Hofstede et al. note, “...it is

83 immensely easier to obtain data for nations than for organic homogeneous societies (Hofstede, Hofstede, and Minkov 2010).” Therefore, to recruit clear experimental groups, we selected members of two distinct cultures and language groups: native speakers of Mandarin Chinese living in China and native speakers of English living in the United States.

Personal tendency is a tagger’s set of preferences and beliefs about tags and their use (Sen et al. 2006). Users bring a pre-existing personal tendency, which affects the number and type of tags they choose to apply, and which is partially determined by their cultural background (Chau et al. 2002). Over time, their personal tendency evolves as they interact with the items and tags in a system. Our first research question studies how culture affects this process: RQ1: Personal-Tendency: Do users from different cul- Figure 2: Tagging Interface. Users may apply tags by typing tures exhibit different tagging tendencies? them or (in appropriate conditions) clicking a suggested tag. Hofstede et al. differentiates between culture’s affect on a person’s values, the underlying beliefs and attitudes ac- 4.1 Experimental Conditions quired early in life, and practices, which can be externally We refined our basic survey idea into a 2 x 4 between- observed (Hofstede, Hofstede, and Minkov 2010). Since subjects design. values are difficult or impossible to measure accurately, our The first factor, subject culture, had two values: Amer- study focused on differences in tagging practices. ican and Chinese. As mentioned earlier, we define cultural groups based on a subject’s nationality. We selected Tag Suggestions Prior work showed users create tags sim- these nationalities for several reasons. First, they are his- ilar to those they have viewed (Sen et al. 2006; Suchanek, torically quite distinct. Second, the dominant languages in Vojnovic, and Gunawardena 2008). We study cultural dif- each culture, English and Mandarin Chinese, are very differences in two mechanisms affecting a user’s adoption of ferent. Third, our research team included native speakers of suggested tags: conformity and cultural resonance. English and Chinese (the Chinese speakers also speak En- glish), which greatly facilitated our research. Conformity One reason users apply suggested tags is The second factor, tag suggestion type, had four values: the psychological tendency towards conformity (Allen and Porter 1983). Research shows that people from differ- • None. No suggested tags were presented to subjects. ent cultures have different degrees of conformity (Kim and • Own. When subjects were presented a movie, they were Markus 1999; Markus and Kitayama 1991; Nisbett et al. shown the 8 most popular tags for that movie from a 2001). We thus pose a second research question studying movie web site from their own culture and language. the relationship between conformity and culture: We used MovieLens.org as the American source and Douban.com as the Chinese source. Subjects were not RQ2: Conformity: Do users from different cultures vary told the source of suggested tags. in their degree of conformity to suggested tags? • Other. For each movie, subjects were shown the 8 most Cultural Resonance. The tags suggested by a system are popular tags from the web site of the other culture: Chi- applied by people from a specific culture and may therefore nese subjects saw tags from MovieLens, and American reflect the culture of the original tagger. A user may be more subjects saw tags from Douban. Tags were translated into likely to select tags that reflect their own cultural beliefs and the native language of the subjects (see below for details). attitudes, even if they are not aware the tag came from their • Combined. Subjects were shown 4 popular tags from own culture. We call this effect cultural resonance, and our MovieLens and 4 popular tags from Douban. We detail third research question studies it: below how we select tags for this condition. RQ3: Resonance: Are people from one culture more This gives 8 experimental conditions, which we refer to likely to adopt tags from their own or another culture? by the subject culture and tag suggestion type, i.e. Chinese- None, American-None, Chinese-Own, American-Own, etc. 4 Experimental design 4.2 Survey Design To study our research questions, we designed a survey in We administered the survey via the Web. Subjects were which subjects were shown a sequence of movies and asked presented with an initial screen describing the experiment. to apply tags to each of the movies. In some conditions, They then were shown a sequence of 20 screens, one for subjects were shown a set of suggested tags for each movie each of 20 movies. Subjects were asked whether they had (the common practice in online tagging communities), while seen each movie. They were free to apply as many tags as in a control condition, no suggested tags were presented. they wanted to each movie they had seen. After viewing

84 all the movies, on the final screen they were asked for three Condition Subjects Distinct tags Tag apps optional items, their country, native language, and gender. American-None 26 615 1108 Chinese-None 54 692 1110 4.3 Screen Design / Tagging Interface Design American-Own 23 310 1164 Chinese-Own 29 217 1011 Since we wanted to study how suggested tags influenced the American-Other 15 286 755 tags subjects chose to apply, we wanted to minimize other Chinese-Other 26 140 929 textual content that might influence their choice. Thus, we American-Combined 14 198 500 did not display the type of metadata commonly available for Chinese-Combined 23 159 880 movies, such as the director, actors, plot summary, or re- views. On the other hand, we wanted to present information Table 1: Subjects and basic descriptive statistics. that would “jog the memory” of our subjects, so they could recall whether they had seen a movie and be able to select This method raised three issues: (a) how to compute pop- appropriate tags. ularity consistently across two different sites, (b) how to We thus decided to present the title and five photos for build a list of tags from both MovieLens and Douban for each movie. The five photos showed two posters for the the combined condition, and (c) how to translate tags. movie, two images associated with important plot elements of the movie, and one image showing the main actor(s). We Computing popularity The obvious starting point for the located photos using IMDB and Google Image Search. Fig- popularity of a tag is the number of times it was applied. ure 2 shows the photos for the movie “Forrest Gump”. However, Douban has many more users and therefore many When subjects indicated they had seen a movie, they were more tag applications than MovieLens. Therefore, we nor- shown a screen where they could apply tags (Figure 2). malized tag popularity as follows. For each chosen movie, Users in the None condition were only able to type in tags. we summed the number of applications of the 8 most popu- Users in all other conditions could either type in a tag or lar tags. Then for each of the 8 tags, we calculated the per- click on a suggested tag to apply it. All entered tags were cent of the sum the tag accounted for and used that number. displayed at the bottom of the screen. The average normalized popularity across each tag position was nearly identical for both cultures (adjusted-R2 = 0.99). 4.4 Selecting movies Combining tags from MovieLens and Douban In the We selected the 20 movies to include in the survey based on Combined conditions, we presented users with 4 popular four principles. The first three principles ensured that users tags from the MovieLens and 4 popular tags from Douban. would be familiar with many movies. The fourth ensured We began by selecting the 4 most popular tags from each that we balanced foreign and U.S. movies. source; if there were no common tags in the two sets (after Popularity. The movie should be popular in both China and translation), we had our 8 tags. If there were any common the United States. We measured popularity by the number tags, we proceeded as follows. If there was a single common of rating in MovieLens and Douban. tag, this left us with only 7 unique popular tags. We there- Genre. We selected movies across a range of genres. fore selected the 5th had been selected, we continued down Year. We selected movies ranging from 1972 to 2010. the list of popular tags from the site until we found one that had not been selected. If there were two common tags, we Language/Culture. American movies produced in English did this procedure for each of the sites. And if there were are popular in many parts of the world, including China. more than two common tags, we continued this procedure On the other hand, relatively Americans see relatively few in the obvious way. Finally, we computed a tag’s combined movies produced outside the United States in other lan- normalized popularity score as the average of the normalized guages. We wanted to balance American/English and non- popularity scores of the tag on the two sites. About 18% of American/non-English movies as much as possible. There- the suggested tags in the combined group were common tags fore, while respecting the Popularity principle, we chose chosen in this manner. 10 American/English movies and 10 non-American/non- English movies (including two Chinese). Translating tags The Other and Combined conditions re- quired us to show tags produced in Chinese to American 4.5 Selecting and translating suggested tags subjects and tags produced in English to Chinese subjects. As explained above, subjects in the tag suggestion condi- The bilingual members of our research team did the trans- tions were shown 8 suggested tags for each movie, either lation in collaboration with a bilingual professor of Chinese from their own culture, the other culture, or combined cul- Literature at our university and a bilingual graduate student tures. This is a primary source of cultural influence on sub- who studies Chinese film. jects’ tagging choices. We also realized that even if a tag is used in both our source sites, Douban and MovieLens, it 4.6 Subjects and recruitment might be much more popular in one than the other; this too, We created two versions of our survey, one in English and is a cultural difference. Therefore, we also represented the one in Chinese. Subjects in each language version were ran- popularity of each tag as a number in parentheses; for exam- domly assigned to one of the tag suggestion type conditions. ple, Figure 2 shows that “Vietnam” has a popularity of 10 We solicited participants through posts to relevant email lists for the movie “Forrest Gump”. and social network sites such as Twitter, Facebook, Douban,

85 Tag suggestion Avg num tags per user and Renren. The survey was live from March 8 through p-value April 18 of 2011. type per movie 499 people filled out at least part of the survey. How- Americans Chinese < 0.01 ever, to test our conditions clearly, we took a conservative None 3.27 1.96 Own 4.85 3.19 < 0.01 approach and analyzed data only for subjects who (a) made Other 4.37 3.05 < 0.01 it to the end of the survey, and (b) took the Chinese sur- Combined 4.58 3.20 < 0.01 vey and indicated their country was China and native language was Chinese, or took the English survey and indi- Table 2: Comparing the number of tags Applied by Ameri- cated their country was the United States and native lan- can and Chinese subjects in each tag suggestion type condi- guage was English. This resulted in a total of 78 American tion. subjects and 132 Chinese subjects. Both Chinese and Amer- Americans Chinese p-value ican subjects added tags to an average of about 10 movies Factual tags 77.78% 51.20% < 0.0001 (µ=9.8, µ=10.6 respectively). Subjects also tagged similar Subjective tags 21.47% 47.67% < 0.0001 numbers of American movies (µ=6.3, µ=6.7) and foreign Personal tags 0.76% 0.80% 0.95 movies (µ=3.4, µ=4.9). Table 1 shows the number of sub- Table 3: How Chinese and American subjects differed in jects in each condition and basic descriptive statistics. their use of Factual, Subjective, and Personal tags. A limitation of our survey is the possible selection-bias differences between Chinese and American subjects. Char- acteristics such as age and tagging experience level may dif- FSP Classification Schema The FSP scheme was defined fer between subjects in each culture. These differences may by (Sen et al. 2006). Factual tags identify objective prop- influence tagging behavior or overall reaction to online ex- erties of an item, Subjective tags express a user’s opinions periments in each subject group. However, the use of similar related to items, and Personal tags organize a user’s own recruitment channels between cultures should help mitigate items. We manually classified each distinct tag into one of these effects. the three classes. Tags that do not fit the FSP scheme are classified as other (unless mentioned otherwise, we ignore 5 RQ1: Personal-Tendency "other" tags in our research). Do users from different cultures exhibit different personal Four researchers did the classification, which was divided tagging tendencies? into two stages. In stage one, two native English speak- We measured two outcomes of taggers’ personal tagging ers classified the English tags from Group 1, and two native tendencies that contribute to the usefulness of a tag repos- Chinese speakers classified the Chinese tags from Group itory and that might differ across cultures: the number (tag 2. Inter-rater agreement metrics in stage one indicated volume) and type (tag class) of tags they apply. We com- that coders categorized tags reliably: Chinese and Ameri- puted the tag class metric only for users in the None condi- can coders agreed on 84% and 90% of their respective tag tions since the presence of suggested tags in the other con- classifications. Cohen’s kappa values also showed reli- ditions would have confounded the results. able coding (Chinese = 0.80, English = 0.63) (Landis and Koch 1977). In stage two, the coders for each language 5.1 Tag Volume discussed and tried to resolve any disagreements. If the Prior research found that Western users applied more tags to coders could not agree on a tag’s classification, it was not describe images and music than Chinese users (Dong and Fu included in our analysis. After stage two, English and Chi- 2010; Kayan, Fussell, and Setlock 2006; Peesapati, Wang, nese coders reached consensus for 97% and 96% of their and Cosley 2010). We found similar results: in each tag sug- respective tags. gestion type condition (None, Own, Other, and Combined), Table 3 shows the average proportion of tags in each class American subjects applied significantly more tags per movie for the American-None and Chinese-None groups. Chinese 1 than their Chinese counterparts (Table 2). subjects preferred subjective tags, and American subjects preferred factual tags. Figure 3 details these differences by 5.2 Tag Class showing the proportion of subjective tags for different users People from different cultures perceive and describe items in in each culture. Most Chinese users apply similar numbers different ways (Dong and Fu 2010; Kayan, Fussell, and Set- of factual and subjective tags, and the overall distribution lock 2006). To understand these differences, we classified looks relatively normal. However, the distribution of Amer- tags with two classification schemes. The Factual, Subjec- ican users looks very different (p < .00001, χ2 test). The 26 tive, Personal scheme (FSP) (Sen et al. 2006) is an estab- American users divide into two clear groups: those who use lished tag description scheme. We developed the Holistic, 80% or more factual tags (22 users), and those who use 80% Analytic, Genre (HAG) scheme based on social science re- or more subjective tags (4 users). The graph for factual tags search describing cultural differences between Eastern and shows the same pattern, but reversed. Western thought. As far as we know, (Kayan, Fussell, and Setlock 2006) are 1We excluded as outliers subjects whose average number of the only other researchers to use the FSP classification to tags/movie was more than 5 times the overall average. Two users analyze cultural differences in tagging. They found that a (both American-None) were outliers under this criterion. larger proportion of factual tags in a site popular with Chi-

86 Holistic Analytic Epic (7) Holocaust (11) classic (7) prison (8) historical (5) ﬁsh (7) surreal (5) Pixar (7) visual-treat (3) Leonardo DiCaprio (6) romantic (2) French (5) multiple storylines (1) Australia (3) Table 4: Examples of holistic and analytic tags.

Americans Chinese p-value Figure 3: Proportion of subjective tags for different users of Holistic 27.94% 56.01% < 0.0001 Chinese-None (one peak) and American-None (two-peaks). Analytic 52.94% 35.94% < 0.0001 Genre 19.02% 8.05% 0.01 Table 5: How Chinese and American subjects differed in nese users (SongTaste) and a larger proportion of subjec- their use of Holistic vs. Analytic vs. Genre tags. tive tags in a site popular with Europeans and Americans (Last.fm). These results appear to be contrary to ours. How- ever, as we described earlier, major differences in study and Chinese coders reached consensus on categorizations methodologies mean that the results are not directly com- for 98% and 99% of their respective (tag, movie) pairs. parable. For example, user interface differences between SongTaste and Last.fm and unclear user demographics of Table 4 gives a few examples of tags that we classified as the two sites may account for the differences. Analytic and tags we classified as Holistic. Recall that we actually classify (tag, movie) pairs; however, for simplicity HAG Classification Scheme We reviewed Nisbett’s work of exposition, Table 4 shows tags that were classified con- (see above) on holistic vs. analytic thought to create the sistently as Holistic or consistently as Analytic. HAG categorization scheme: Table 5 shows the average proportion of tags in each class • Holistic tags describe a movie as whole, relationships be- for the American-None and Chinese-None groups. Chinese tween objects in a movie, or context-dependent aspects of subjects applied more Holistic tags, while American users the objects in a movie. were more likely to use Analytic tags and Genre tags. The • Analytic tags describe specific parts or aspects of a movie. proportion of analytic tags for each Chinese and American We first had to define criteria for classifying tags as Holis- tagger (Figure 4) resembles the distributions for subjective tic or Analytic. One English coder and one Chinese coder tags (Figure 3). Most Chinese users apply a balance of worked together to create the criteria. Their work revealed Holistic and Analytic tags. The American users split into a problem: it was difficult to classify genre tags like drama, two groups. The same four users account for the right-hand horror, or comedy since they have both Holistic and Ana- peaks in Figures 3 and 4. This reflects the overall tendency lytic aspects. Like Holistic tags, genre tags describe an item of holistic tags to be subjective. as a whole; like analytic tags, genre tags describe attributes of an item. Rather than force genre tags into the Holistic or The differences in Holistic and Analytic usage are con- Analytic category, we defined a new category: sistent with Nisbett (Nisbett et al. 2001). The higher usage of Genre tags by Americans reflects Nisbett’s finding • Genre tags describe a movie’s genre. We used genres that Western cultures prefer to use rules and categorization from IMDB (www.imdb.org), which is popular in both methods to organize objects. China and the U.S. We classified tags using the HAG schema similarly to the FSP schema. One key difference is that we found it nec- essary to classify (tag, movie) pairs in the HAG schema, because the same tag may be used in different senses for different movies. For example, the tag “war” was applied to both “Braveheart” and “Forrest Gump” in our survey. We judged the former a Genre tag, because “Braveheart” is a war movie. However, we judged the latter an Analytic tag, because while one portion of “Forrest Gump” relates to the Vietnam War, it is not a war movie. As with FSP, coders were reliable in stage one codings according to both percentage agreement (Chinese = 89%, Figure 4: Proportion of Analytic tags for different users of English = 84%) and Cohen’s kappa (Landis and Koch 1977) Chinese-None (one peak) and American-None (two peaks). (Chinese = 0.82, English = 0.69). After stage two, English

87 Tag suggestion Percent conformity Adoption of: Subject p-value type Americans Chinese Culture Own tags Other tags Own 81.5% 96.7% American 40.4% 23.7% < 0.01 Other 69.2% 91.0% Chinese 47.5% 24.4% < 0.01 Combined 83.4% 95.6% Table 8: Cultural Resonance: Subjects who got tag sugges- Table 6: Comparing the conformity of American and Chi- tions originating from both cultures adopted more suggested nese subjects in each tag suggestion type condition. tags from their own culture.

Tag suggestion Percent “entirely con- p-value type formant” users 7 RQ3: Resonance Americans Chinese Are people from one culture more likely to adopt tags from Own 22% 73% 0.001 their own or another culture? Other 0% 55% 0.001 Combined 29% 74% 0.02 We investigated this question in two ways: (a) a between- subjects comparison for the Own and Other groups in each Table 7: Percent of subjects who only applied suggested tags culture, and (b) a within-subjects comparison for the Com- in each experimental condition. P-values are calculated us- bined group in each culture. ing a χ2 test. Own vs. Other We again use percent-conformity as our 6 RQ2: Conformity metric. Table 6 shows that for both cultures, subjects who were exposed to tags that originated from their own culture adopted more suggested tags than those who were exposed Do users from different cultures vary in their degree of con- to tags that originated from the other culture. formity to suggested tags? Note that 6 also illustrates the greater influence of con- 3 Conformity in tagging systems can be measured by the ex- formity on Chinese subjects’ tagging choices : they adopted tent to which users apply suggested tags. We measure con- a higher proportion of suggested tags from an American formity using percent-conformity, defined as follows: source than Americans did! For a user u, consider a movie m that u tagged. Calculate Combined As described above, the Combined condi- the percent of tags u applied to m that were suggested by tions use suggested tags from both American and Chinese the system. Average this value for all the movies tagged by sources, with all tags translated into subjects’ native lan- u to get the percent-conformity for u. Finally, to compare guage. Therefore, for the combined conditions, we com- experimental conditions, average this value for all users in pared the proportion of applied tags that were tag sugges- each condition. tions from subjects’ own culture and the proportion of applied tags that were suggestions from the other culture (we We compare the conformity of American and Chinese ignore suggestions that were common to both cultures). subjects for the three tag suggestion type conditions: (a) Table 7 shows that both American- and Chinese- Com- Own: users are suggested tags from their own culture, (b) bined subjects adopted more suggested tags that originated Other: users are suggested tags from the other culture, and from their own culture even though they did not know the (c) Combined: users are suggested tags from both cultures. cultural origin of suggested tags. As Table 6 shows, Chinese subjects exhibited stronger conformity than American users: in each of the three tag sug- 8 Discussion gestion type conditions, Chinese subjects applied more suggested tags than Americans.2 Some users were “entirely Our results suggest implications for designers of tagging conformant”; they only applied tags suggested by the sys- systems and for future research. To situate the implications, tem. Table 7 shows a dramatic cultural difference in entirely we recall a primary goal for a tagging system: to collect a conformant users: while a majority of Chinese users only useful repository of tags. To make the goal more focused, apply suggested tags, relatively few Americans did so (p < we also must specify useful for what. As (Sen et al. 2006) 0.02, χ2 test). These results are consistent with Western and found, different classes of tags (in the FSP scheme) are use- Asian emphasis on individualism and collectivism. ful for different purposes; for example, factual tags are most useful for finding items, while subjective tags are most help- We also wondered whether users from one culture were ful for evaluating items. We should also specify useful for more likely to choose popular tags. To analyze this, we mea- whom. Our results suggest that users from different cultures sured the Pearson correlation between the displayed popu- may have different preferences regarding tags. Thus, we can larity of a tag and the likelihood of a user to adopt the tag. restate the goal as: to collect a repository of tags that support Although we found stronger correlations with tags from a the users of a system in performing relevant tasks. user’s own culture (own = 0.66, other = 0.42), we did not System designers can intervene to achieve this goal at two find significant differences between cultures. points: tag application time and task execution time. Sys- tems can shape tag applications by carefully selecting dis-

2We do not report signiﬁcance because the users who only ap- 3And likewise the greater inﬂuence of individualism on Ameri- plied suggested tags led to a right-skewed distribution. can subjects’ choices.

88 played and suggested tags (this work and (Sen et al. 2006)). Extended Abstracts of ACM Conference on Computer Sup- Likewise, systems may optimize tasks by designing intelli- ported Cooperative Work (CSCW’11). ACM. gent algorithms and interfaces to be culture-aware. Dong, W., and Fu, W. 2010. Cultural difference in image With this context we can state several design guidelines tagging. In Proceedings of the 28th international conference and several areas for future work. First, designers should on Human factors in computing systems, 981–984. ACM. consider what tagging vocabulary “deficits” might result Hofstede, G.; Hofstede, G.; and Minkov, M. 2010. Cul- from users’ cultural tendencies. For example, a system with tures and organizations: software of the mind: intercultural mainly Chinese users may produce a low proportion of fac- cooperation and its importance for survival. McGraw-Hill tual tags, which may reduce the effectiveness of tag searches Professional. (Sen et al. 2006). In response, a designer might encourage factual tags by favoring them as suggested tags or by Kayan, S.; Fussell, S.; and Setlock, L. 2006. Cultural dif- offering “reputation points” for applying them. Second, de- ferences in the use of instant messaging in asia and north signers should consider when to suggest tags from the target america. In Proceedings of the 2006 conference on Com- user’s own culture. For example, a system may increase tag puter supported cooperative work, 525–528. ACM. adoption if it suggests Chinese tags to Chinese users. These Kim, H., and Markus, H. 1999. Deviance or uniqueness, two guidelines highlight an interesting tension for system harmony or conformity? a cultural analysis. Journal of Per- designers. The first focuses on improving the overall tagging sonality and Social Psychology 77(4):785. vocabulary, and the second focuses on matching the natu- Landis, J., and Koch, G. 1977. The measurement of observer ral tendencies of individual users. Supporting users’ natu- agreement for categorical data. Biometrics 159–174. ral tendencies may increase their short-term satisfaction, but Lee, K.; Joshi, K.; and McIvor, R. 2007. Understanding produce a problematic overall tagging vocabulary. multicultural differences in online satisfaction. In Proceed- We propose several areas of future work to establish a firm ings of the 2007 ACM SIGMIS CPR conference on Computer basis for these guidelines, we. First, while we showed that personnel research, 209–212. ACM. people from two cultures can apply different tags, we also would like to see whether the tags are useful for different Markus, H., and Kitayama, S. 1991. Culture and the self: specific tasks. Our intuition is that prior results concerning Implications for cognition, emotion, and motivation. Psy- tag usefulness (Sen et al. 2006) generally will hold for non- chological review 98(2):224. Westerners because the results are based on inherent proper- Nisbett, R.; Peng, K.; Choi, I.; and Norenzayan, A. 2001. ties of the tasks. For example, factual tags may be useful in Culture and systems of thought: Holistic versus analytic finding items because people agree on them and apply them cognition. Psychological review 108(2):291. consistently. However, this should be verified. Second, there Peesapati, S.; Wang, H.; and Cosley, D. 2010. Intercul- are no data about the relative usefulness of Holistic, Ana- tural human-photo encounters: how cultural similarity af- lytic, Genre tags for the key tagging tasks. This too should fects perceiving and tagging photographs. In Proceedings of be studied for users from various cultures. Third, existing the 3rd international conference on Intercultural collabora- research exploring incentive mechanisms that influence the tion, 203–206. ACM. amount and type of work people do in online communities Pfeil, U.; Zaphiris, P.; and Ang, C. 2006. Cultural differ- has focused on American contexts. Since these mechanisms ences in collaborative authoring of wikipedia. Journal of offer potential for eliciting useful tags, their effectiveness for Computer-Mediated Communication 12(1):88–113. other cultures should be explored. Sen, S.; Lam, S.; Rashid, A.; Cosley, D.; Frankowski, D.; 9 Acknowledgements Osterhouse, J.; Harper, F.; and Riedl, J. 2006. Tagging, communities, vocabulary, evolution. In Proceedings of the We would like to thank Joseph Allen, Jessica Ka Yee Chan, 2006 conference on Computer supported cooperative work, and Brandon Maus for the help in coding tags. We are also 181–190. ACM. grateful for the support of the members of GroupLens Re- search, particularly Jesse Vig and Daniel Kluver who pro- Suchanek, F.; Vojnovic, M.; and Gunawardena, D. 2008. vided feedback on the study design. This work is funded by Social tags: meaning and suggestions. In Proceeding of the the NSF (grants IIS 09-64697, IIS 09-64695 IIS 08-08692) 17th ACM conference on Information and knowledge man- and by the China Scholarship Council. agement, 223–232. ACM. Wang, H.; Fussell, S.; and Cosley, D. 2011. From diver- References sity to creativity: Stimulating group brainstorming with cultural differences and conversationally-retrieved pictures. In Allen, R., and Porter, L. 1983. Organizational influence Proceedings of the ACM 2011 conference on Computer sup- processes. Scott, Foresman. ported cooperative work, 265–274. ACM. Chau, P.; Cole, M.; Massey, A.; Montoya-Weiss, M.; and Wang, H.; Fussell, S.; and Setlock, L. 2009. Cultural differ- O’Keefe, R. 2002. Cultural differences in the online behav- ence and adaptation of communication styles in computer- ior of consumers. Communications of the ACM 45(10):138– mediated group brainstorming. In Proceedings of the 27th 143. international conference on Human factors in computing Chen, L., and Tsoi, H. K. 2011. Analysis of user tags in systems, 669–678. ACM. social music sites: Implications for cultural differences. In