Running head: TWEETING FOR EQUALITY 1

"Tweeting for Equality" Disparities in the Portrayal and the Amount of Coverage addressed to Male and Female Tennis Players on Twitter.

By Sven Kurt Köppel 12546577

Master’s Thesis

Graduate School of Communication

Master’s programme Communication Science

Dr. A. C. (Anne) Kroon

26th of June 2020

TWEETING FOR EQUALITY 2

Abstract

Despite the fact that women in sport are increasingly getting more opportunities to participate, it is generally assumed that sportswomen are portrayed negatively with myriad gender-specific descriptors and receive less attention than their male counterparts. This Master's Thesis examined this gender bias within user-generated comments on Twitter, addressing the current best female and male tennis players in the ATP and WTA rankings. About 80 thousand Tweets were analysed for gender-stereotyping with the help of an automated content analysis, theoretically based on Fiske et al.' (2002) stereotype content model dimensions. For the dictionary-based approach, the most similar words to the Eastman and Billings' (2001) stereotype-categorisation were inductively elaborated by a word embeddings model with the

Tweets as foundation. The findings revealed that in the social network Twitter, proportionally more Tweets portrayed sportswomen on the basis of their physical appearance and their life outside of sport than Tweets to their male colleagues. Apart from that, Tweets addressed to sportsmen tended to have a higher percentage of stereotyping based on their athletic and mental skills. It was concluded that negative reactions in the form of sexism and objectification were commonly activated due to the ascribed competencies of sportswomen. And this resulted in a higher proportion of descriptors that placed the women's appearance (e.g. beauty and grace) in the foreground. It therefore appears that Twitter users continues to depict competent women according to traditional norms that are thereby established and reproduced in and by society.

KEYWORDS bias, gender stereotyping, sports, tennis players, Twitter, word embeddings

Introduction

Although opportunities of women in sport have been gradually increasing in the last few decades (Grappendorf, 2011), sportswomen are continued to be portrayed differently and covered on a much smaller scale in the traditional media (Yip, 2018). As an example, Anna

TWEETING FOR EQUALITY 3

Kournikova, one of the most photographed sports celebrities in the world and winner of several tournaments, was covered in several articles in the English Tabloid Press, although the majority of the articles were not task related and took up topics besides sports (Harris & Clayton, 2002).

Additionally, Eugenie Bouchard and Serena Williams were asked by a commentator in a off- court interview to perform a so-called twirl, which was heavily criticized from various sides, with no sportsmen being called upon to perform the same action. This unfair demand essentially brought the girlish side of female tennis professionals to the fore and objectified them for the male gaze (Fredrickson & Roberts, 1997). These examples demonstrates in exemplary fashion that sports journalists continue to maintain socially shared gender stereotypes with their specific offences. The media's portrayal of male ideals, masculinity and male supremacy while underrepresenting and infantilizing sportswomen as the inferior "other" maintains the attributed female connotations of subordination, fragility, sensuality, and heterosexuality within traditional newspapers (Harris & Clayton, 2002). This implies, furthermore, that in sports media coverage not only male supremacy is preserved, but also hegemonic femininity, which confusingly serves to establish male hegemony, rather than implying hegemony of women

(Harris & Clayton, 2002).

There are several indications that the Internet could establish alternative ways to disrupt these traditional structures through inclusive and empowering reporting and may help to disrupt the power of dominant groups in other media (Jones, 2006). In fact, infinite space for alternative content could be made available through social media platforms, which could embrace more balanced representations of sportswomen (Litchfield & Kavanagh, 2019). Moreover, Twitter's microblogging format has opened up new communication options for teams, athletes, coaches, fans and sports media to produce their own content (Sanderson, 2012).

The increasing presence of sports celebrities and organisations on Twitter would seem to call for an increasing amount of relevant empirical research (Clavio & Kian, 2010). However,

TWEETING FOR EQUALITY 4 the emergence of Twitter as a legitimate communication and social networking medium has not resulted in sufficient research within and outside sport to address Twitter and its impact on the communication landscape. In fact, new media in sports communication itself is a relatively scarcely researched area. This is not surprising, as many scholars are not familiar enough with different forms of new media to be comfortable analysing the phenomena that occur in the communication methods of sport in new media (Dart, 2009). Nevertheless, important findings on the impact of stereotyping in sports on qualitative (Yip, 2018; Kian et al., (2011) and quantitative (Crossman, Vincent, & Speed, 2007) mass media reporting have been obtained from various side. This Master's Thesis has the incentive to contribute a substantial amount of scientific insights to this scholarly field in order to add relevant information to that body of knowledge. In contrast to the majority of research projects, the present paper focused not on the evaluation of journalistic content, but on user-generated Twitter commentaries, something that has not yet been extensively examined in this form. Thus, the research question shall be read as follows:

RQ: To what extent are male and female tennis players portrayed differently within user- generated Tweets and are women more likely to be addressed on the basis of their external characteristics than their male counterparts?

Existing stereotyping contributions, highlighted in the stereotypes content model of

Fiske et al. (2002) and their implications for social reactions and evaluations were applied to illustrate the gender-specific stereotyping of athletes based on the dimensions "warmth" and

"competence". Furthermore, the role of Twitter as a digital form of communication in developing and reproducing stereotype content was examined in more detail in order to include the communication science aspect. In doing so, references were drawn to common causal relationships in traditional mass media in the implementation and maintaining of socially shared stereotype content. Eventually, an automated content analysis was applied to distinguish

TWEETING FOR EQUALITY 5

Twitter communication into the two dimensions based on the occurrence of certain competence and appearance related keywords, elaborated by Eastman and Billings (2001).

Literature Review

In the following paragraph, an attempt has been made to elaborate a theoretical interrelationship between the stereotype content model (SCM), conceptualized by Fiske et al.

(2002) and existing gender bias literature regarding sports coverage and sports media portrayals.

Stereotype Content Model

Prior to discussing the existing research literature regarding gender bias in news content, the mechanism by which distinctions are established and reproduced in and by society needs to be clarified. Because gender stereotypes are prevalent in society (Heilman & Okimoto, 2007) and often influence attitudes and behaviours without conscious awareness (Wegener, Clark, &

Petty, 2006). Stereotypes offer generalizations "about people on the basis of their group membership" (Donelson, 1999, p. 40), often maintaining and reinforcing the power of the in- group while subordinating members of out-groups (Fiske, Xu, Cuddy, & Glick, 1999).

Fiske (1992) suggested that stereotypical dimensions emerge from interpersonal and intergroup interactions. When people become acquainted with individuals or groups, they expect to identify and assess their intentions and goals towards their peers as well as their effectiveness in pursuing these purposes (Fiske et al., 2002). The functional idea that people need to be capable of anticipating the intentions of out-groups (warmth), thereby estimating their capacities for a thriving realisation of these aims (competence), supports the representative classification of groups. Additionally, out-groups are distinguished according to their potential influence on the group or themselves (Fiske et al., 2002).

TWEETING FOR EQUALITY 6

The SCM constructed by Fiske (1998) implies that a simple evaluative antipathy towards out-groups must not necessarily occur. Stereotype content could reflect separate dimensions of (dis)like and (dis)respect, whereas some out-group being disrespected for perceived lack of competence (e.g. disabled & elderly people, housewives); others being disregarded due to their perceived lack of warmth (e.g. career women, Jews, Asians). A little further down the line, Fiske et al. (2002) reconfigured the established model and assumed a mixed stereotype content model. Herein, stereotypes mostly internalize a combination of more or less socially desirable traits, whereas a general antipathy towards out-groups is believed to be rather uncommon. The differing distribution of warmth and competence was expressed by

Fiske et al. (2002) in four categories: (1) paternalistic prejudice, (2) envious prejudice, (3) contemptuous prejudice and (4) admiration.

Table 1

The Mixed Stereotype Content Model expressed in four Categories.

Competence

Warmth Low High

High Paternalistic Prejudice Admiration Low status, not competitive, High status, not competitive, pride sympathy (e.g., elderly people, (e.g., in-group, close allies) disabled people, housewives)

Low Contemptuous prejudice Envious prejudice Low status, competitive, disgust, High status, competitive, envy, anger (e.g., welfare recipients, poor jealousy (e.g., Asians, Jews, rich people) people, feminists, female athletes)

Paternalistic prejudices describe out-groups, perceived as low on competence but high on warmth, whereas "stereotypes portray a group disrespected but pitied, which carries overtones of compassion, sympathy, and even tenderness, under the right conditions" (Fiske et al., 2002, p. 880). In contrast, envious prejudices describe out-groups categorized as very

TWEETING FOR EQUALITY 7 competent, but low on warmth (Glick & Fiske, 2001c; 2001a). They are deemed too competent, highly ambitious, hardworking, and thus not sociable (Hurh & Kim, 1989). In cases out-groups are categorized as low on both dimensions (e.g., refugees, poor people), they are perceived as

"social parasites" (Fiske et al., 2002, p. 881). Conversely, out-groups, considered both competent and warm (e.g., middle class) are presumed as close allies (Fiske et al., 2002).

Culturally shared stereotypes result from structural relation between groups in society, whereby out-groups are perceived as more competent to the extent that they are classified as powerful and high status or as less competent when they are ranked as powerless and low status

(Fiske et al., 2002). Furthermore, out-groups are classified as relatively warm, when individuals do not compete with others. In contrast, competitive out-groups are perceived as having negative intent and thus frustrate, annoy and tantalize (Fiske et al., 2002). Although the SCM has two separate dimensions 2 × 2 (competence and warmth), they are not psychologically inconsistent. The classification of groups along either dimension is free of uncomfortable feelings, because envious and paternalistic prejudices maintain the status quo and sustain the positions of social reference groups (Fiske et al., 2002).

For the purpose of this Master's Thesis, the importance of integrating athletes into the

SCM-dimensions was crucial. In consulting the study of Eckes (2002), a gender-classification was made accessible. He revealed that career women, to which sportswomen can be assigned, is a female subgroup with the most ascribed competencies, although they remain at a low warmth level. In contrast to housewives, motherly women and other traditional subgroups, which are commonly characterized in a way congruent with paternalistic stereotypes, feminists, career women and non-traditional women are perceived in accordance with envious stereotypes

(Eckes, 2002). While paternalistic stereotypes reflect the wishes of the dominant group about what women should be like, envious stereotypes reflect fears about the characteristics that certain subgroups of women are thought to have (Glick & Fiske, 2001b). Equipping these sub-

TWEETING FOR EQUALITY 8 groups of women with respected, positive (i.e. competence-related) characteristics can nevertheless serve to justify their discrimination because they are considered potentially harmful or unfair competitors who must be relegated to their place (Eckes, 2002). Furthermore, judgments about warmth and competence can trigger active and passive behaviour. The subjectively assessed high level of warmth elicits active facilitation. Conversely, groups that are not considered warm are actively harmed (Cuddy, Fiske, & Glick, 2008). Negative attributions that ascribe a lack of warmth and bad intentions to these women further serve to rationalise acts of discrimination (Eckes, 2002).

These active and passive implications of the gender-classification into the SCM were formerly described in the ambivalent sexism theory of Glick and Fiske (1996), where competent women were commonly incorporated within nontraditional subgroups (e.g., career women, sportswomen). This theory is essential in order to provide an elaborate illustration of the categorisation consequences for competent athletes. In the sexisms ideology, women were given special privileges as long as they fulfil the stereotypes they are ascribed with (Fiske et al., 1999). Herein, hostile and benevolent sexism consolidate the current male hegemony and functions as a ideological belief system (Glick & Fiske, 1996) and as "complementary tools of control, the stick and the carrot, that motivate women to accept a sexist system" (Glick & Fiske,

2001b, p. 139). While hostile sexism predicts negative attitudes towards non-traditional women, benevolent sexism predicts positive reactions towards traditional women (Glick,

Diebold, Bailey-Werner, & Zhu, 1997). Hostile sexism attempts to justify male power, traditional gender roles and the exploitation of women as sexual objects by men through derogatory descriptions of women. Benevolent sexism, conversely, is based on gentler and kinder justifications of male supremacy and stipulated gender roles; it acknowledges men's dependence on women and it encompasses a romanticized view of sexual relations with women

(Glick & Fiske, 1997).

TWEETING FOR EQUALITY 9

In other words, although competent athletes are commonly characterised in a way congruent with envious stereotypes, they trigger differing active and passive behaviour in social interactions. On the one hand, sportsmen as a competent male subgroup do not violate traditional conventions and negative reactions (e.g., hostile, benevolent sexism) are therefore not common. On the other hand, women violating the traditional conventions perpetuated by dominant social groups are discriminated because they are considered unfair and harmful competitors. Moreover, the sexualization and objectification of competent women using derogatory descriptors serves to justify male supremacy and status (Glick & Fiske, 1997).

SCM-Media-Interdependence

Since user-generated Tweets mentioning the world's top tennis players were investigated, the implementation and reproduction of prejudices by the mass media has to be reviewed more precisely. The traditional mass media, including sports media, offer a "contested terrain" for male supremacy, as they allow the presentation of conceptions and ideas that undermine the dominant paradigm and allow subtle shifts in the way groups are framed and eventually culturally perceived (Hargreaves, 1994; Messner, 2002, p. 93). The SCM provides suitable insights, which contribute to the study of media representations of marginalized groups in a unique way and provide insights into how media representations can ultimately influence emotions and behaviour between groups (Sink, Mastro, & Dragojevic, 2018). Firstly, since society's cultural systems are defined to a large extent by institutions such as the media, their messages serve to define perceptions of the status and competence of groups (Atwell Seate &

Mastro, 2017). Dominant groups (church and state, political groups) are thereby endowed with the ability to distribute symbolic content to a large, heterogeneous and geographically dispersed population (Carter-Francique & Richardson, 2016). These dominant groups have the resources to determine the desired texts and images in sports articles and influence mass media in their production and distribution (Rowe, 1999). According to Van Sterkenbrug and Knoppers (2004),

TWEETING FOR EQUALITY 10 these dominant groups are primarily white men from the middle to upper classes, who manage to defend the socio-cultural position of their peer groups through traditions and ideological concepts by producing suitable content. Secondly, the existence of these representations legitimizes and validates such perceptions of status and competence. Thirdly, the normalization of these views systematizes the known prejudiced and discriminatory reactions associated with these perceptions and makes them easier to identify and predict (Sink et al., 2018).

In other words, mass media exert a significant influence on the classification of groups into the dimensions of Fiske et al. (2002), whereby dominant groups within society claim a substantial influence on this framing process by determining appropriate content. Furthermore, these media representations are legitimized by their mere existence, thus also reproducing discrimination against sportswomen. As a consequence, hostile sexism towards competent female subgroups and male supremacy is regarded as a given order in society (Sink et al., 2018).

It often affects women and girls in various life spheres, such as negative working conditions or qualitative and quantitative differences in sports coverage (Fink, 2016). The following paragraph provides specific details on these quantitative discrepancies of news coverage within traditional media content.

Content of News Coverage

Past studies highlighted the fact that sportswomen received fewer attention in written media (Crossman et al., 2007; Fink & Kensicki, 2002; Harris & Clayton, 2002; Kian & Hardin,

2009; Vincent, Imwold, Masemann, & Johnson, 2002), broadcast media (Angelini, MacArthur,

& Billings, 2012; Billings, Halone, & Denham, 2002; Greer, Hardin, & Homan, 2009) and even new media (Burch, Eagleman, & Pedersen, 2012; Clavio & Eagleman, 2011; Kian, Mondello,

& Vincent, 2009; Litchfield & Kavanagh, 2019; Pedersen & Macafee, 2007). Billings et al.

(2002) and Kian and Hardin (2009) examined the basketball sports coverage and revealed that sportsmen were covered almost twice as often as sportswomen. According to Kian and Hardin

TWEETING FOR EQUALITY 11

(2009), these findings could be attributed to the fact that the gender of journalists exerts a significant influence on the gender-specific selection of media content. Journalists mostly covered sports of their gender, but the sheer majority of male journalists changed the balance in favour of male reporting (Kian & Hardin, 2009).

While analysing the coverage of transnational events such as the Olympic Games, gender news coverage inequalities slightly decreased. Angelini et al. (2012), Greer et al. (2009) and Vincent et al. (2002) made significant discoveries indicating that these transnational events bring the country-specific discrepancies of athletes into focus and gender characteristics became obscured. Despite this fact, sportsmen still received more news media coverage at the

2010 Vancouver Winter Olympic, both in terms of the athletes overall mentions during NBC’s prime-time telecast and the overall clock-time (Angelini et al., 2012).

Digital forms of communication such as online media might reduce the inequality of coverage, as it alleviates the spatial limitations found in traditional media outlets (Kian

& Hardin, 2009). But even the proliferation of digital media cannot overcome the male supremacy in sports reporting. This can be illustrated by Burch et al. (2012), where digital content was analysed during the Vancouver 2010 Olympic Games. Even there, sportsmen were more frequently represented than sportswomen. Through analysing the sports-blog "Deadspin",

Lisec and McDonald (2012) supports these findings, revealing that within three years only 37 of daily blog entries covered professional women's basketball; and that with dozens of blog entries daily. Litchfield and Kavanaghs (2019) Twitter study involving two teams participating in the 2016 Rio di Janeiro Olympics found similar findings, but the gender discrepancies were marginal. This could be explained due to the fact that sportswomen won more than half of the medals, which made it impossible for the editors to ignore them (Litchfield & Kavanagh, 2019).

Although it is widely assumed that news coverage is more balanced in a perceived gender-appropriate sport like tennis (Yip, 2018), female tennis players still received fewer

TWEETING FOR EQUALITY 12 attention than their male counterparts (Crossman et al., 2007; Kian et al., 2011). Vincent (2004) contradicted these findings, as sports media content covering sportswomen accounted for a relatively high proportion of the news coverage of the 2000 Wimbledon Championship. While analysing three newspapers during the 2004 Wimbledon Championship, Crossman et al. (2007) revealed an almost equal sports reporting. Although male content received 1.5 times as much space in newspapers when the entire space was considered, sportswomen participating in perceived gender-appropriate sports receive approximately the same amount of sports coverage

(Crossman et al., 2007).

Although several findings in the cited literature anticipated a balanced coverage in a perceived gender-appropriate sport such as Tennis, no clear signs of a definitive convergence have been found. Due to the overwhelming amount of evidence specifying gender-specific discrepancies in sports coverage, the first hypothesis also requires the assumption that in comparison to sportsmen, sportswomen receive less attention in the form of Tweets addressed to them. Moreover, it is expected that the gender discrepancies will even increase, as due to

Covid-19 no influences from transnational events (e.g., Wimbledon Championship), which could bring about a convergence of coverage (Vincent et al., 2002), are to be assumed. Lastly, there is evidence that the emergence of digital media has not been able to compensate the gender discrepancies in sports coverage. Thus, the first hypothesis was conceived as follows:

H1: The number of Tweets addressing sportswomen is lower compared to the number of Tweets addressing sportsmen.

Attention to Media Portrayals

In cases sportswomen are being covered, they are nevertheless portrayed differently by the mass media. A wide range of studies have been conducted on this topic in recent years (e.g.

Angelini, 2008; Chalabaev, Sarrazin, Fontayne, Boiché, & Clément-Guillotin, 2013; Fink,

2015; (Messner, Duncan, & Jensen, 1993); Crossman et al., 2007; Yip, 2018). Their research

TWEETING FOR EQUALITY 13 on sports coverage has precisely demonstrated that media portrayals differ in production, tone, and focus, leading to a more negative depiction of the women's sport and performance. In summary, it follows that sports journalists and the media organisations often "infantilise" highly qualified sportswomen, describing them as "girls" or "young ladies", while their male counterparts are rarely (if ever) called "boys" (Messner et al., 1993). Journalists further infantilise female athletes by calling them by their first name only, whereas this rarely occurs with male athletes. Messner et al. (1993) found that sport commentators called women's tennis players by their first names almost eight times more than male players. These linguistic discrepancies reflect the lower reputation of sportswomen, while reproducing negative or ambivalent attitudes (Messner et al., 1993).

Moreover, sports journalists frame the performance of female athletes differently and women's athletic abilities are systematically devalued (Billings, Angelini, & Duke, 2010).

While male achievements are often linked to talent and dedicated performance, the success of female athletes is generally based on luck, emotions and strong male support (Eastman &

Billings, 1999). Another way in which the sports media maintains male supremacy is by focusing on the (hyper-) femininity of sportswomen, resulting in athletic achievement and ability fading into the background (Daniels, 2012). While sportsmen prefer traditional masculine qualities such as aggression, power and speed and thus receive more coverage

(Duncan, 2006), sportswomen receive more attention from heterosexual men when they are considered sexually attractive and more feminine (Vincent et al., 2002). Additionally, it has been observed within all media and sports that more sports coverage is distributed to sportswomen when participating in those sports that include and emphasize female ideals such as beauty, glamor and grace (Daddario, 1997). Sports reporting tends to focus primarily on the heterosexuality of sportswomen, focusing on aspects of life outside of sport, or rather their role as housewife, mother and friend (Billings et al., 2002; Fink & Kensicki, 2002).

TWEETING FOR EQUALITY 14

According to Angelini (2008), this less-favoured media portrayals of sportswomen serves to maintain implicit gender-specific stereotypical schemata in order to portray the athletic abilities of sportswomen as inferior. There is an important mechanism underlying this interdependence. Exceptional female athletes, strong and skilled, threaten male supremacy and are therefore considered unnatural, deviant or lesbian (Sartore & Cunningham, 2009).

However, making female participation more likely to be perceived appropriate by society, the gender-neutrality of Tennis might result in equal media portrayals (Yip, 2018) and might encourage a balanced display of the competencies of sportswomen. Nevertheless, the strengths and successes of sportswomen are still framed ambivalently and in a sexualised way in tennis reporting (Messner et al., 1993). While the losses and failures of men are attributed to their strong opponents, the women are blamed for their deficiencies such as nervousness and lack of confidence when they lose. Vincent (2004) holds a similar view, noting that women are often described as psychologically fragile and emotionally vulnerable. On the contrary, men are independent, strong, stable, and sporty. Furthermore, Harris and Clayton (2002) pointed out that the media, when reporting on the individual winner of the 2000 Wimbledon championship

- Venus Williams - the media did not focus on her athletic and mental skills that led her to victory. Instead, they focus on her personal life, such as her favourite food and hobbies. With these remarks it is demonstrated how sportswomen are sexualized and objectified, even though they are categorized according to envious stereotypes. Thus, in contrast to men, competent women trigger more negative descriptors and experience more hostile sexism.

Twitter Comments

This Master's Thesis intends to contribute additional insights to the existing scientific knowledge about the various forms of stereotyping of athletes on the digital medium Twitter.

Twitter is a microblogging platform and social network where users can post short status updates called Tweets (Rudat, Buder, & Hesse, 2014). It connects users with user-generated

TWEETING FOR EQUALITY 15 messages, ideas and opinions from around the world, the moment it happens, allowing users to actively respond to these messages in the form of comments (Twitter Inc., 2020). Social media platforms like Twitter offer fan groups many opportunities to interact with athletes at both professional and amateur levels (Sanderson, 2011). Moreover, teams and athletes have been very responsive to the new possibilities of Twitter. For example, many athletes use Twitter to more actively present themselves to the public (Sanderson, 2008) and to create their own identity without the traditional media channels (Sanderson, 2013). Furthermore, fans now have the possibility to send messages directly to athletes (Sanderson & Truax, 2014), especially when mentioning their official Twitter accounts. Mentions on Twitter implies the integration of a another accounts Twitter username, whereby according to Twitter (2020) an "@" symbol must be written in addition to the username (e.g., @rogerfederer, @ashbarty).

As new opportunities arise with the emergence of social network sites like Twitter, portrayals of sportswomen might evolve in a way that social media users no longer need to adhere to prevailing organisational guidelines and male supremacy. However, sportswomen are still trivialised in web posts, blogs and online sports news and their achievements continue to be downgraded (Lisec & McDonald, 2012). Jones (2013) found that women's performances at the 2008 Beijing Olympics were less prominent and associated with traditional stereotypes, such as being emotionally weaker. Nevertheless, alternative findings were also detectable;

Eagleman et al. (2014) found only minimal gender biases in online coverage of the 2012

London Olympics. These mixed conclusions suggest the possibility that social media and the emergence of the Internet might lead to gender-neutral reporting (Kian et al., 2009).

The following hypotheses incorporate these findings into their theoretical construction.

In doing so, it is hypothesized that by classifying athletes in accordance with envious stereotypes, negative reactions in form of hostile sexism for competent women occur, which leads to a concentration of female body aspects (e.g., appearance, beauty, physical features). In

TWEETING FOR EQUALITY 16 contrast to sportsmen, this allegedly unnatural behaviour of women against any traditional norms is penalised with sexualisation and objectification (Glick & Fiske, 1997). Conversely, the ascribed competencies are predicted not to provoke negative reactions for sportsmen because "[m]en’s overall structural power gives them, as a group, high status, which presupposes men’s competence as a group" (Fiske et al., 1999, p. 484). That is why it is further assumed that sportsmen are more likely to be described in Twitter mentions in accordance to their athletic and mental abilities. Based on these arguments, the second and third hypotheses were formed as follows:

H2: Tweets, addressing sportswomen contain fewer adjective descriptors that discuss mental and physical skills than Tweets, addressing sportsmen.

H3: Tweets, addressing sportswomen contain more adjective descriptors that discuss physical appearance, attractivity and the social background than Tweets, addressing sportsmen.

Methodological Approach

In the Methods section the procedure by which the Tweets were made available for the inductive construction of a dictionary with a word embeddings model is displayed, enabling the extraction of the most similar words according to Eastman and Billings (2001) stereotype- categorisation. This established dictionary was then utilized for the execution of a dictionary- based automated content analysis, validated by the means of precision, recall and F-scores. The

Python function is accessible in its entirety in the following public github.com repository: https://github.com/SvenKoeppel1992/MasterThesis/blob/master/MasterThesis.ipynb and in the Appendix A.

Data Collection

The sample of Tweets required for the analysis was obtained using the REST API in

Python, with which the organization provides a limited insight into millions of accounts and

TWEETING FOR EQUALITY 17 billions of Tweets (Morstatter, Pfeffer, Liu, & Carley, 2013). REST APIs, enabling users to retrieve recent or popular Tweets (Gu, Qian, & Chen, 2016). "The REST query format includes

[…] a set of keywords with the support of operators, including AND, OR, and EXCLUDE […].

Tweets data consist of user information, text, time posted, times of re-tweets […]" (Gu et al.,

2016, p. 325). After implementing the REST API in Python, Tweets with an @-mention addressing male and female tennis players (e.g., @rogerfederer, @ashbarty) listed in the official rankings established by the Association of Tennis Professionals (ATP) and the Women's Tennis

Association (WTA) were retrieved (a detailed table of the reviewed players is provided in

Appendix B). For comprehension, the ATP is the governing body of the men's professional tennis circuits (ATP Tour, 2020), while the women equivalent, the WTA, is the global leader in women’s professional sport (WTA Tennis, 2020).

While a corresponding official Twitter account was identified for each sportswoman, two male players were not possessing official accounts (Matteo Berrettini & Christian Garin), which is why they were excluded from the analysis. For this reason, the sample was expanded to include those athletes who were listed next in the ATP ranking (Alex de Minaur & Hubert

Hurkacz). Totally 532.084 user-generated comments were downloaded, containing textual information and language details. However, a fairly high percentage of these Tweets were either repeatedly listed or not authored in English, which rendered them useless for the purpose of the study. After the duplicates and the non-English Tweets were removed, 86.868 Tweets remained for the upcoming dictionary construction. The high number of unusable Tweets was most likely caused by the huge number of players not publishing in English.

Dictionary construction

In order to develop dictionaries with the help of an inductive neural network language model, text must be quantified into structured data (Franzosi, 2004). To this aim, the Tweets were pre-processed in the following manner.

TWEETING FOR EQUALITY 18

Text Normalization

In the traditional sense, text normalization represents the first step of a speech synthesis system, whereby numbers, dates, acronyms, etc., found in real-world texts (e.g., Tweets), are converted into standard dictionary words so that the system can pronounce the words correctly

(Liu, Weng, & Jiang, 2012). The application of text normalization brings the informal language of Tweets closer to texts, allowing further analysis in existing models (Farzindar & Inkpen,

2017). The first step in text pre-processing is to separate words from punctuation and other symbols (Farzindar & Inkpen, 2017). Tokenization means the splitting of data records into a list of tokens. A token is a part of a whole, which means that a word is a token in a sentence, where a sentence is a token of a paragraph (Perking, 2014). By using the Natural Language

Toolkit (NLTK), which is a comprehensive Python library for natural language processing and text analytics (Perking, 2014), the Tweets were distinguished into separate word tokens.

Following the tokenization, in a second and third step any tokens that were no longer essential for further analysis were discarded. This included hashtags, punctuations, website URLs, usernames, stop words, etc. This set of tokens was then applied to train a word embeddings model, as explained hereafter.

Identifying synonyms

For identifying synonyms, word representations were trained in a word embeddings model, whereby each word written in the Tweets was assigned to a high-dimensional vector so that the geometry of the vector captures semantic relationships between the corresponding words (Collobert, Weston, Bottou, Kavukcuoglu, & Kuksa, 2011). By using the continuous skip-gram method, a current word was used to predict the surrounding window of context words. This architecture weights thereby the nearby words more heavily than words that are more distant from the context word (Mikolov, Chen, Corrado, & Dean, 2013a). The advantage of this neural network language and the representation of words as vectors, is that similar words

TWEETING FOR EQUALITY 19 are attributed to similar vectors (Mikolov, Yih, & Zweig, 2013b), making it feasible to extract the most similar words. Although these models are typically developed on the basis of large corpora of texts, such as collections of Google news articles or Wikipedia, and are known to find relationships that do not occur in a simple co-occurrence analysis (Garg, Schiebinger,

Jurafsky, & Zou, 2018), the current model was grounded on the generated tokens to ensure the inductive approach. This indicates that when performing a search for the most similar words, only those words can be retrieved that actually occurred in the reviewed Tweets. According to

Schwartz and Ungar (2015), the main advantage of deriving dictionaries from texts is that such dictionaries are based on real word distributions and not on presumed word usage.

Accordingly, the continuous skip-gram model was trained on the corpus of Tweets using the Gensim word2vec module within Python. According to Mikolov et al. (2013a) and Khattak et al. (2019), dimensionality and the context window are the key parameters for training word2vec embeddings. The quality of the current word embedding was warranted as the dimensionality was set to 300 dimensions.

Dependent Variables

By using the word2vec embeddings model, the most similar words were identified according to Eastman and Billings (2001) stereotype-categorisation. The authors subdivided college basketball television annotations into 15 categories, reflecting the athletic skills, leadership abilities and physical appearance of the players featured in the coverage. Out of these categories, the 10 which best reflected the dimensions "competences" and "warmth" of Fiske et al. (2002) were selected for the current analysis. Those emphasizing the physical appearance of the players, their personality and background (8. Personality; 9. Looks and Appearance; 10.

Background) were grouped in the first dependent variable "Appearance". This variable represents, inter alia, discriminatory, and sexist descriptors attributed to competent women due to their lack of warmth. The others, indicating athletic and mental abilities (1. Physicality and

TWEETING FOR EQUALITY 20

Athleticism; 2. Intelligence and Mental Skill; 3. Hard Work and Effort; 4. Determination and

Motivation; 5. Speed; 6. Physical Power; and 7. Mental Power) were combined in the category

"Skills" (see Table 2); representing the second dependent variable. Thereby, the competences of the athletes were emphasized predominantly.

Table 2

Keywords conducted by Gensim word2vec Model

Categories of Eastman and Variables Dictionaries Billings (2001) unique, dedication, impressive, talent, passion, 1. physicality / athleticism gesture, genius, highest-paid, athletes, player, greatest, status, strength, fitness, creative, 2. intelligence / mental determination, health, courage, energy, balance, skill focus, efforts, serve, slice, backhand, volleys, technique, smash, volley, grip, hits, footwork, onehanded, tweener, average, rally, opponents, 3. hard work / effort rate, motion, spin, rivals, superb, unbelievable,

unreal, performance, master, leading, underrated, 4. determination / rivalry, body, risk, successful, confidence, soul, motivation working, motivation , mentally, train, practising, Skills grow, training, job, work, gesture, drive, gym, 5. speed appreciated, art, ambassador, spirit, humble, peace, business, blessing, goal, frontline, happiness, success, force, willing, towards, wrist, position, 6. physical power ideal, feet, shoe, movement, cross, arm, storm, knees, impressed, moving, bounce, fit, fighting, creative, hander, model, greatness, beyond, 7. mental power athletic, challenging, competitive, boring, perspective, fair, jump, education, fancy, faster

8. personality attitude, generous, classy, gentleman, funny, hilarious, jealous, cool, smart, interesting, idiot, cute, precious, sweet, cutest, beautiful, dancing,

gorgeous, sexy, heart, shine, crying, adore, sweetie, dislike, horrible, shy, racist, cutie, sounds, sound, 9. looks / appearance looked, haircut, yummy, pink, looking, grace, eye, beast, colour, beau, nails, girlfriend, selfie, ring, lovely, wonderful, amazing, adorable, handsome, Appearance beard, goodness, girl, girls, lit, bear, Jesus, darling, families, kids, moms, raised, brothers, couple, 10. background mom, mothers, mum, madam, wife, husband, silly, students, anniversary, celebrating, son, daughter

TWEETING FOR EQUALITY 21

Automated Content Analysis

After the words for the dictionaries were identified for both variables, a deductive content analysis on the corpora of Tweets was performed to automatically classify text into the categories. This was done by a dictionary-based approach, which is the most powerful content analysis method that computers have the potential to automate for the benefit of scholars (Riffe,

Lacy, & Fico, 2014). Dictionaries are used to find out how often a certain keyword occurs in a document and can be selected to classify this document into a predefined category (Grimmer &

Stewart, 2013). Compared to traditional manual content analysis, the dictionary-based approach greatly enhances the effectiveness of text classification tasks (Guo, Vargo, Pan, Ding, & Ishwar,

2016). Therefore, a Python function was coded for the automatic examination of the text for the occurrence of the words compiled in the dictionaries. In doing so, the Tweets were then automatically assigned to one of the categories, depending on which word occurred in the text.

Validation

Whether the applied analysis tools categorized the Tweets properly was verified by means of precision, recall and F-score. Precision and recall have been used to characterize document retrieval performance for many years. Recall is the number of retrieved relevant items in relation to the total number of relevant items (Buckland & Gey, 1994). Unlike recall, precision defines the proportion of the items the models says was relevant actually were relevant

(Koehrsen, 2018). For both indicators, an approximation to 100 percent is considered scientifically desirable (Buckland & Gey, 1994). For the calculation of precision, recall and F- score, a randomly generated sample (N = 190) of the total amount of Tweets was manually coded for the occurrence of the words in order to validate the automatic classification of the

Tweets into the two variables. The sample was examined with the assistance of a Qualtrics survey. As shown in Appendix C, the Qualtrics survey posed two questions: The former aims to evaluate the identification number (ID) of the Tweets; the latter distinguishes the Tweets into

TWEETING FOR EQUALITY 22 the two categories "Skills" and "Appearance". The ID was required for later merging of the automatically generated and the manually recorded sample.

A confusion matrix was then applied to determine the overlap of the manually inspected sample with the automated classified one (see Table 4). The overall accuracy with a value of

.89 indicates that 89 percent of the examined Tweets were equally encoded manually.

Furthermore, the F-scores for both categories are extremely valid with a value of .89 (N = 96) and .90 (N = 94). The F-score can be interpreted as the weighted average of precision and recall, whereby values approximating to 1 are considered very valid (scikit-learn, 2019). The automated content analysis can therefore be considered a highly valid analysis tool.

Table 3

Precision and Recall Measures

Category Precision Recall F1-Score Support Accuracy

Skills .90 .88 .89 96 Appearance .90 .90 .90 94 .89 (190)

Results

The current study was conducted with the aim to investigate gender discrepancies in

Tweets, mentioning female and male athletes. It was expected in the first hypothesis, that sportsmen receive a significant higher amount of coverage in form of Tweets. Of the total number of Tweets, 34.105 (39.3%) addressed sportswomen and 52.753 (60.7%) sportsmen.

Furthermore, it was hypothesized that Tweets addressing sportsmen are more likely to be placed in the "Skills" category in terms of percentage, in comparison to Tweets addressing

Sportswomen (H2). From Tweets regarding sportswomen, 2.797 (.082) could be assigned to

TWEETING FOR EQUALITY 23

the category "Skills" (SD = .27); the corresponding amount for sportsmen is 5.275 (SD = .3), which means that .1 of male Tweets could be attributed to the this category (see Figure 2).

Eventually, the last hypothesis (H3) proposed that Tweets addressing sportswomen are more likely to be categorized according to appearance descriptors than those addressing sportsmen. Totally 3.513 (.103) Tweets referring to sportswomen were subdivided into the category "Appearance" (SD = .3), as well as 4.326 (.082) of the total number of Tweets, mentioning sportsmen (SD = .27).

Number of Tweets

60000

50000

40000

52753 30000 60.7% 34105 20000 39.3%

10000

0 Female Athletes Male Athletes

Figure 1: Total Amount of Tweets assigned to the Athletes

The descriptive discrepancies between sportsmen and sportswomen are in line with what was expected. It can be concluded that sportsmen received more attention in the form of Tweets than sportswomen. Moreover, Tweets addressing sportswomen have a higher probability of being categorized with descriptors of the variable "Appearance". Conversely, Tweets addressing sportsmen are more likely to be categorized in the category "Skills". However, statistical methods must be applied in order to adequately analyse these interdependencies. In order to evaluate H1, a one-sample-t-test was applied to check whether the discrepancies were

TWEETING FOR EQUALITY 24 significant. Sportsmen received more commentaries to their accounts (M = .61, SD = .39) than sportswomen (M = .39, SD = .61), t (86857) = 366,536, p < .001. Thus, H1 can be accepted, as the official accounts of sportsmen generated significantly more attention in form of Tweets than those of sportswomen.

Number of male Tweets for each category Number of female Tweets for each category

9000 8000 7000 4326 6000 5275 5000 4000 3000 3513 2000 2797 1000 0 Skills Appearance Figure 2: Number of Tweets assigned to the two Categories

A first logistic regression (see Table 4) was conducted to determine the gender- discrepancies in number of Tweets which were assigned to the category "Skills" and whether these differences were significant (H2). Thereby, a significant regression equation was found

(F = 77.94, p < .001), with an R² of .001. The likelihood of Tweets addressing sportsmen to be categorized into the category "Skills" is equal to .1 (p < .001). Conversely, the probability of

Tweets, addressing sportswomen, being classified into the same category is lower (.082, p <

.001). Due to the significant outcomes of the logistic regression, it can be concluded that

Tweets, mentioning sportswomen are less likely to be assigned to the category "Skills" than

Tweets, addressing sportsmen.

TWEETING FOR EQUALITY 25

Table 4

Logistic Regression Coefficients for the Variable "Skills"

Independent variable Beta β -SE P-value

Constant; Female .0821 .002 < .001

Gender; Male .0178 .002 < .001

Another logistic regression (see Table 5) was applied to predict whether the gender- different classification of Tweets into the category "Appearance" was statistically significant

(H3). Once again, a significant regression equation was found (F = 114.6, p < .001), with an R² of .001. The likelihood of Tweets addressing sportsmen to be categorized into the category

"Appearance" is equal to .082 (p < .001). Conversely, the probability of Tweets, addressing sportswomen, being classified into the same category is higher (.103, p < .001). Due to the significant outcomes of the logistic regression, it can be concluded that Tweets, mentioning sportswomen are more likely to be assigned to the category "Appearance" than Tweets, addressing sportsmen.

Table 5

Logistic Regression Coefficient for the Variable "Appearance"

Independent variable Beta β -SE P-value

Constant; Female .103 .002 < .001

Gender; Male -.021 .002 < .001

Conclusion and Discussion

The aim of this Master's Thesis was to examine the varying stereotyping of tennis players within the social network site Twitter. The assumptions were based on the premise that sportswomen are stereotyped to a greater extent due to their physical characteristics (H3) and

TWEETING FOR EQUALITY 26 receive less attention in terms of the number of commentaries (H1). Additionally, it was hypothesized that sportsmen tend to be described more by commentaries on their athletic and mental abilities (H2). The findings of the study uncovered significant discrepancies between

Tweets addressed to sportsmen and those addressed to sportswomen. It was revealed that sportswomen received significantly less attention from Twitter users than sportsmen.

Furthermore, having received attention in the form of Tweets, they were more likely to be stereotyped based on the category "Appearance" than Tweets addressing sportsmen.

Conversely, sportsmen were more likely to obtain attention in form of Tweets, with a significantly higher probability that these Tweets addressed the players' athletic and mental skills, than Tweets addressing sportswomen.

The findings of the gender-specific coverage discrepancies reflect the results of previous studies, both in traditional media (Crossman et al., 2007) and in digital media (Litchfield

& Kavanagh, 2019). According to Mislove et al. (2011), the usage behaviour of women and men on Twitter differs significantly. Accordingly, the overwhelming majority of Twitter users were male; around two-thirds of users. This point is even more delicate, as Kian and Hardin

(2009) found a significant influence of gender on content selection, with men more likely to address male content and women more likely to discuss female issues. The gender-specific majority regarding the attention in form of Tweets ascribed to the athletes mirrors this interdependence of Mislove et al. (2011) and Kian and Hardin (2009). Approximately 60% of the Tweets were addressed to sportsmen, which roughly reflects the gender-related usage behaviour of the Twitter users.

Furthermore, the findings regarding the differing gender stereotyping of Tweets addressing male and female athletes are comparable to prior outcomes (Crossman et al., 2007;

Messner et al., 1993; Yip, 2018). As in prior investigations, proportionally to the total number of Tweets, more comments addressing sportswomen were associated with descriptors that

TWEETING FOR EQUALITY 27 depict the physical appearance (e.g., sexy, beautiful, cutie) and the life outside the sport (e.g., family, birthday, mother). These findings offer significant support for the assumption that sportswomen trigger negative reactions and attitudes in terms of hostile sexism and objectification due to the ascribed competencies and low warmth (Glick & Fiske, 1997), and due to the fact that they are commonly considered nontraditional (Glick & Fiske, 1996). The negative intentions ascribed to them thus arouse fears that induce malicious reactions. Through traditional gender roles and the depiction of women as sex objects by means of derogatory descriptors, the male power and its supremacy is maintained (Atwell Seate & Mastro, 2017;

Sink et al., 2018). Moreover, the findings also provide indicators regarding the mass media's influence on the opinions and reactions of Twitter users, as "the day-to-day selection and display of news by journalists focuses the public`s attention and influences its perceptions"

(Carroll & McCombs, 2003), whereby also in Twitter journalists are the dominant group in shaping the public opinion (Buhl, Günther, & Quandt, 2018). The prevailing media portrayals of sportswomen were also observed in user-generated comments on Twitter, which suggests that such users were impacted by the stereotypes reproduced in the mass media. Thus, in line with past findings (Jones, 2013; Lisec & McDonald, 2012), the emergence of social networks as a digital medium has hardly overcome traditional gender structures. Sportswomen continue to be associated with traditional stereotypes, which also contradicts the findings of Eagleman et al. (2014) regarding a gender-neutral representation of women.

Conclusively, mass media, as a society's cultural system, still have a certain sovereignty of interpretation, which can influence people's emotions and behaviours. This is also reflected in the relative difference in how competent women are stereotyped compared to their male peers. Although sportsmen were also classified as competent and not warm (Eckes, 2002), they triggered negative content in form of Tweets to a far lower amount, as their competence is used as tool to maintain traditional gender roles.

TWEETING FOR EQUALITY 28

Limitations

The study did not consider other traits of tennis players which might interfere with the content of Tweets, such as ethnicity, race, or nationality. Thus, certain negative descriptors may be attributed to non-gender-relevant causes. Vincent (2004) argues that such variables could have a major impact on the predominant gender order. It is therefore highly possible that the sexualisation of sportswomen could be attributed to their race, or that the athletic and mental skills of particular sportsmen could be explained by their nationality; and so on.

Furthermore, due to the limited time frame of two months, not enough Tweets could have been collected for an accurate word embeddings model. Scholarly research thereby assumed that a higher amount of training data improves the accuracy of the model (Mikolov et al., 2013a). Normally, word embeddings models were, according to the authors, constructed with dictionaries containing around 1 million words and 6 billion tokens (e.g., Google News corpora). This accuracy requirement could not be fulfilled with the Tweets used in this analysis.

This fact had an impact on the inductive conception of the dictionary for the automated content analysis, which thus could not accurately reflect every dimension of the stereotype- categorization of Eastman and Billings (2001).

Eventually, although the validity of the dictionary was tested sufficiently for the automated content analysis with an overall accuracy of .89, words may have multiple meanings depending on how they were incorporated in the sentence, or a synonym may not be recognized, making it impossible to assign it to the corresponding variable (Conway, 2006). This leads to a significant loss of relevant information, which cannot be included in the subsequent analysis.

Implications

Despite these limitations, this Master's Thesis provided additional insights into gender- based discrepancies regarding the portrayal and the amount of attention of tennis players in a digital medium that has not yet been sufficiently researched. It might also contribute to the

TWEETING FOR EQUALITY 29 impetus for intensified research by official bodies in order to identify possible sexist and degrading digital content concerning women and thus implement data-based societal education and prevention.

Furthermore, the application of inductive automated content analysis by means of neural network languages has so far been neglected in communication studies. This study provides knowledge about the conception and execution of such word embedding models for the construction of dictionary-based approaches, which is of utmost importance for the further development of existing analysis tools in this research field.

Nevertheless, it would be highly interesting to consider additional traits of the players in order to ensure a holistic stereotyping approach. Extensive data collection over several months or years could lead to more comprehensive word embeddings model that would make the automated content analysis even more accurate.

Reference

Angelini, J. R. (2008). Television Sports and Athlete Sex: Looking at the Differences in Watching Male and Female Athletes. Journal of Broadcasting & Electronic Media, 52(1), 16–32, from doi:10.1080/10934520701820752. Angelini, J. R., MacArthur, P. J., & Billings, A. C. (2012). What's The Gendered Story? Vancouver's Prime Time Olympic Glory on NBC. Journal of Broadcasting & Electronic Media, 56(2), 261–279. ATP Tour (2020). About, from https://www.atptour.com/en/corporate/about. Atwell Seate, A., & Mastro, D. (2017). Exposure to Immigration in the News: The Impact of Group-Level Emotions on Intergroup Behavior. Communication Research, 44(6), 817– 840, from doi:10.1177/0093650215570654. Billings, A. C., Angelini, J. R., & Duke, A. H. (2010). Gendered profiles of Olympic history: Sportscaster dialogue in the 2008 Beijing Olympics. Journal of Broadcasting & Electronic Media, 54, 9–23, from doi:10.1080/08838150903550352. Billings, A. C., Halone, K. K., & Denham, B. E. (2002). "Man, That Was a Pretty Shot": An Analysis of Gendered Broadcast Commentary Surrounding the 2000 Men's and Women's NCAA Final Four Basketball Championships. Mass Communication & Society, 5(3), 295– 315, from doi:10.1123/ssj.19.3.315. Buckland, M., & Gey, F. (1994). The Relationship between Recall and Precision. Journal of the American Society for Information Science, 45(1), 12–19.

TWEETING FOR EQUALITY 30

Buhl, F., Günther, E., & Quandt, T. (2018). Observing the Dynamics of the Online News Ecosystem. Journalism Studies, 19(1), 79–104, from doi:10.1080/1461670x.2016.1168711. Burch, L. M., Eagleman, A. N., & Pedersen, P. M. (2012). NEW MEDIA COVERAGE OF GENDER IN THE 2010 WINTER OLYMPICS: AN EXAMINATION OF ONLINE MEDIA CONTENT. International Journal of Sport Management, 13, 1–17. Carroll, C. E., & McCombs, M. (2003). Agenda-setting Effects of Business News on the Public’s Images and Opinions about Major Corporations. Corporate Reputation Review, 6(1), 36–46. Carter-Francique, A. R., & Richardson, M. F. (2016). CONTROLLING MEDIA, CONTROLLING ACCESS: THE ROLE OF SPORT MEDIA ON BLACK WOMEN’S SPORT PARTICIPATION. Race, Gender & Class, 23(1-2), 7–33. Chalabaev, A., Sarrazin, P., Fontayne, P., Boiché, J., & Clément-Guillotin, C. (2013). The influence of sex stereotypes and gender roles on participation and performance in sport and exercise: Review and future directions. Psychology of Sport & Exercise, 14(2), 136–144, from doi:10.1016/j.psychsport.2012.10.005. Clavio, G., & Eagleman, A. N. (2011). Gender and Sexually Suggestive Images in Sports Blogs. Journal of Sport Management, 7, 295–304, from doi:10.1123/jsm.25.4.295. Clavio, G., & Kian, T. M. (2010). Uses and Gratifications of a Retired Female Athlete’s Twitter Followers. International Journal of Sport Communication, 3, 485–500, from doi:10.1123/ijsc.3.4.485. Collobert, R., Weston, J., Bottou, L., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (Almost) from scratch. Journal of Machine Learning Research, 12, 2493–2537. Conway, M. (2006). The Subjective Precision of Computers: A Methodological Comparison with Human Coding in Content Analysis. Journalism & Mass Communication Quarterly, 83(1), 186–200, from doi:10.1177/107769900608300112. Crossman, J., Vincent, J., & Speed, H. (2007). ‘THE TIMES THEY ARE A-CHANGIN’: Gender Comparisons in Three National Newspapers of the 2004 Wimbledon Championships. International Review for the Sociology of Sport, 42(1), 27–41. Cuddy, A. J.C., Fiske, S. T., & Glick, P. (2008). Warmth and Competence as Universal Dimensions of Social Perception: The Stereotype Content Model and the BIAS Map. Advances in Experimental Social Psychology, 40, 61–149, from doi:10.1016/s0065- 2601(07)00002-0. Daddario, G. (1997). Gendered sports programming: 1992 summer Olympic coverage and the feminine narrative form. Sociology of Sport Journal, 14, 103–120, from doi:10.1123/ssj.14.2.103. Daniels, E. A. (2012). Sexy versus strong: What girls and women think of female athletes. Journal of Applied Developmental Psychology, 33, 79–90, from doi:10.1016/j.appdev.2011.12.002. Dart, J. J. (2009). Blogging the 2006 FIFA World Cup Finals. Sociology of Sport Journal, 26, 107–126, from doi:10.1123/ssj.26.1.107. Donelson, F. E. (1999). Women’s experiences: A psychological perspective. Mountain View, CA: Mayfield Publishing Company. Duncan, M. C. (2006). Gender warriors in sport: Women and the media. In A. A. Raney & J. Bryant (Eds.), Handbook of Sports and Media (pp. 231–252). Mahwah, NJ: Lawrence Erlbaum Associates.

TWEETING FOR EQUALITY 31

Eagleman, A., Burch, L., & Vooris, R. (2014). A unified version of London 2012: New-media coverage of gender, nationality, and sport for Olympics consumers in six countries. Journal of Sport Management, 28(4), 457–470, from doi:10.1123/jsm.2013-0151. Eastman, S. T., & Billings, A. C. (1999). Gender parity in the Olympics: Hyping women athletes, favoring men athletes. Journal of Sports and Social Issues, 23, 140–170, from doi:10.1177/0193723599232003. Eastman, S. T., & Billings, A. C. (2001). Biased Voices of Sports: Racial and Gender Stereotyping in College Basketball Announcing. The Howard Journal of Communications, 12, 183–201, from doi:10.1080/106461701753287714. Eckes, T. (2002). Paternalistic and Envious Gender Stereotypes: Testing Predictions From the Stereotype Content Model. Sex Roles, 47(3/4), 99–114. Farzindar, A., & Inkpen, D. (2017). Natural Language Processing for Social Media (Second Edition). Toronto, CA: Morgan & Claypool. Fink, J. S. (2015). Female athletes, women’s sport, and the sport media commercial complex: Have we really ‘‘come a long way, baby’’? Sport Management Review, 18, 331–342. Fink, J. S. (2016). Hiding in Plain Sight: The Embedded Nature of Sexism in Sport. Journal of Sport Management, 30, 1–7, from doi:10.1123/jsm.2015-0278. Fink, J. S., & Kensicki, L. J. (2002). An Imperceptible Difference: Visual and Textual Constructions of Femininity in Sports Illustrated and Sports Illustrated for Women. Mass Communication & Society, 5(3), 317–339, from doi:10.1207/s15327825mcs0503_5. Fiske, S. T. (1992). Thinking is for doing: Portraits of social cognition form Daguerreotype to laserphoto. Journal of Personality and Social Psychology, 63, 877–889, from doi:10.1037/0022-3514.63.6.877. Fiske, S. T. (1998). Stereotyping, prejudice, and dicrimination. In D. T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.), Handbook of social psychology (4th ed., pp. 357–411). Boston: McGraw-Hill. Fiske, S. T., Xu, J., Cuddy, A. C., & Glick, P. (1999). (Dis)respecting versus (dis)liking: Status and interdependence predict ambivalent stereotypes of competence and warmth. Journal of Social Issues, 55(3), 473–489, from doi:10.1111/0022-4537.00128. Fiske, S. T., Cuddy, A. J. C., Glick, P., & Xu, J. (2002). A Model of (Often Mixed) Stereotype Content: Competence and Warmth Respectively Follow From Perceived Status and Competition. Journal of Personality and Social Psychology, 82(6), 878–902, from doi:10.1037/0022-3514.82.6.878. Franzosi, R. (2004). From Words to Numbers. Narrative, Data, and Social Sciences. Cambridge, UK: Cambridge University Press. Fredrickson, B. L., & Roberts, T.A. (1997). Objectification theory: Toward understanding women’s lived experiences and mental health risks. Psychology of Women Quarterly, 21(2), 173–206. Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018). Word embeddings quantify 100 years of gender and ethnic stereotypes, from https://www.pnas.org/content/suppl/2018/03/30/1720347115.DCSupplemental. Glick, P., Diebold, J., Bailey-Werner, B., & Zhu, L. (1997). The two faces of Adam: Ambivalent sexism and polarized attitudes towards women. Personality and Social Psychology Bulletin, 23, 1323–1334, from doi:10.1177/01461672972312009. Glick, P., & Fiske, S. T. (1996). The Ambivalent Sexism Inventory: Differentiating hostile and benevolent sexism. Journal of Personality and Social Psychology, 70, 491–512, from doi:10.1037/0022-3514.70.3.491.

TWEETING FOR EQUALITY 32

Glick, P., & Fiske, S. T. (1997). Hostile and Benevolent Sexism. Measuring Ambivalent Sexist Attitudes Toward Women. Psychology of Women Quarterly, 21, 119–135, from doi:10.1111/j.1471-6402.1997.tb00104.x. Glick, P., & Fiske, S. T. (2001a). An ambivalent alliance: Hostile and benevolent sexism as complementary justifications for gender inequality. American Psychologist, 56, 109–118, from doi:10.1037/0003-066x.56.2.109. Glick, P., & Fiske, S. T. (2001b). Ambivalent sexism. In M. P. Zanna (Ed.), Advances in experimental social psychology (pp. 115–188). Thousand Oaks, CA: Academic Press. Glick, P., & Fiske, S. T. (2001c). Ambivalent stereotypes as legitimizing ideologies: Differentiating paternalistic and envious prejudice. In J. Jost & B. Major (Eds.), The psychology of legitimacy (pp. 278–306). Thousand Oaks, CA: Academic Press. Grappendorf, H. (2011). Where We`ve Been, Where we Are, Where We`re Going: Girls and Women in Sport and Physical Activity. Women in Sport and Physical Activity Journal, 20(1), 93–94, from doi:10.1123/wspaj.20.1.93. Greer, J. D., Hardin, M., & Homan, C. (2009). “Naturally” Less Exciting? Visual Production of Men's and Women's Track and Field Coverage During the 2004 Olympics. Journal of Broadcasting & Electronic Media, 53(2), 173–189. Grimmer, J., & Stewart, B. M. (2013). Text as Data: The Promis and Pitfalls of Automatic Content Analysis Methods for Political Texts. Society for Political Methodology, 21(3), 267–297, from doi:10.1093/pan/mps028. Gu, Y., Qian, Z. S., & Chen, F. (2016). From Twitter to detector: Real-time traffic incident detection using social media data. Transportation Research Part C, 67, 321–342, from doi:10.1016/j.trc.2016.02.011. Guo, L., Vargo, C. J., Pan, Z., Ding, W., & Ishwar, P. (2016). Big Social Data Analytics in Journalism and Mass Communication: Comparing Dictionary-Based Text Analysis and Unsupervised Topic Modeling. Information Access and Control in an Age of Big Data, 93(2), 332–359, from doi:10.1177/1077699016639231. Hargreaves, J. (1994). Sporting females: Critical issues in the history and sociology and women’s sports. New York, NY: Routledge. Harris, J., & Clayton, B. (2002). FEMININITY, MASCULINITY, PHYSICALITY AND THE ENGLISH TABLOID PRESS: The Case of Anna Kournikova. International Review for the Sociology of Sport, 37(3-4), 397–413, from doi:10.1177/1012690202037004024. Heilman, M. E., & Okimoto, T. G. (2007). Why are women penalized for success at male tasks? The implied communality deficit. Journal of Applied Psychology, 92, 81–92, from 10.1037/0021-9010.92.1.81. Hurh, W. M., & Kim, K. C. (1989). The "success" image of Asian Americans: Its validity, and its practical and theoretical implications. Ethnic and Racial Studies, 12, 512–538, from doi:10.1080/01419870.1989.9993650. Jones, D. (2013). Online coverage of the 2008 Olympic Games on the ABC, BBC, CBC and TVNZ. Pacific Journalism Review, 19(1), 244–263, from doi:10.24135/pjr.v19i1.248. Jones, D. (2006). The Representation of Female Athletes in Online Images of Successive Olympic Games. Pacific Journalism Review, 12(1), 108–129, from doi:10.24135/pjr.v12i1.848. Khattak, F. K., Jeblee, S., Pou-Prom, C., Abdalla, M., Meaney, C., & Rudzicz, F. (2019). A survey of word embeddings for clinical text. Journal of Biomedical Informatics: X, 4, 1– 18, from doi:10.1016/j.yjbinx.2019.100057.

TWEETING FOR EQUALITY 33

Kian, E. M., Fink, J. S., & Hardin, M. (2011). Examining the Impact of Journalists’ Gender in Online and Newspaper Tennis Articles. Women in Sport and Physical Activity Journal, 20(2), 2–21, from doi:10.1123/wspaj.20.2.3. Kian, E. M., & Hardin, M. (2009). Framing of Sport Coverage Based on the Sex of Sports Writers: Female Journalists Counter the Traditional Gendering of Media Coverage. International Journal of Sport Communication, 2, 185–204, from doi:10.1123/ijsc.2.2.185. Kian, E. M., Mondello, M., & Vincent, J. (2009). ESPN--the women's sports network? A content analysis of Internet coverage of March madness. Journal of Broadcasting & Electronic Media, 53(3), 1–9. Koehrsen, W. (2018). Beyond Accuracy: Precision and Recall, from https://towardsdatascience.com/beyond-accuracy-precision-and-recall- 3da06bea9f6c#:~:text=Visualizing%20Precision%20and%20Recall&text=First%20up%20 is%20the%20confusion,true%20negative%2C%20and%20false%20negative. Lisec, J., & McDonald, M. G. (2012). Gender Inequality in the New Millennium: An Analysis of WNBA Representations in Sport Blogs. Journal of Sports Media, 7(2), 153–178, from doi:10.1353/jsm.2012.0019. Litchfield, C., & Kavanagh, E. (2019). Twitter, Team GB and the Australian Olympic Team: representations of gender in social media spaces. Sport in Society, 22(7), 1148–1164, from doi:10.1080/17430437.2018.1504775. Liu, F., Weng, F., & Jiang, X. (Eds.). 2012. A Broad-Coverage Normalization System for Social Media Language. Messner, M. A. (2002). Taking the field: Women, men, & sports. Minneapolis, MN: University of Minnesota Press. Messner, M. A., Duncan, M. C., & Jensen, K. (1993). Separating the Men from the Girls: The Gendered Language of Televised Sports. Gender and Society, 7(1), 121–137, from doi:10.1177/089124393007001007. Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013a). Efficient Estimation of Word Representations in Vector Space, from https://arxiv.org/pdf/1301.3781.pdf. Mikolov, T., Yih, W., & Zweig, G. (2013b). Linguistic Regularities in Continuous Space Word Representations, from Microsoft Research: https://www.aclweb.org/anthology/N13- 1090.pdf. Mislove, A., Lehmann, S., Ahn, Y., Onnela, J. P., & Rosenquist, N. J. (Eds.). 2011. Understanding the Demographics of Twitter Users. Morstatter, F., Pfeffer, J., Liu, H., & Carley, K. M. (Eds.). 2013. Is the Sample Good Enough? Comparing Data from Twitters Streaming API with Twitters Firehose. Pedersen, S., & Macafee, C. (2007). Gender Differences in British Blogging. Journal of Computer-Mediated Communication, 12, 1472–1492, from doi:10.1111/j.1083- 6101.2007.00382.x. Perking, J. (2014). Python 3 Text Processing with NLTK 3 Cookbook: Over 80 practical recipes on natural language processing techniques using Pythons NLTK 3.0. Birmingham, UK: Packt Publishing. Riffe, D., Lacy, S., & Fico, F. (2014). Analyzing media messages: Using quantitative content analysis in research. New York, NY: Routledge. Rowe, D. (1999). Sport, culture, and the media: The unruly trinity. Buckingham, UK: Open University Press.

TWEETING FOR EQUALITY 34

Rudat, A., Buder, J., & Hesse, F. W. (2014). Audience design in Twitter: Retweeting behavior between informational value and followers’ interests. Computers in Human Behavior, 35, 132–139, from doi:10.1016/j.chb.2014.03.006. Sanderson, J. (2008). The blog is serving its purpose: Self-presentation strategies on 38pitches.com. Journal of Computer-Mediated Communication, 13, 912–936, from doi:10.1111/j.1083-6101.2008.00424.x. Sanderson, J. (2011). It’s a whole new ballgame: How social media is changing sports. New York, NY: Hampton Press. Sanderson, J. (2012). Introduction to this special issue of IJSC on Twitter. International Journal of Sport Communication, 5(4), 433–434, from doi:10.1123/ijsc.5.4.433. Sanderson, J. (2013). Stepping into the (social media) game: Building athlete identity via Twitter. In R. Luppicini (Ed.), Handbook of research on technoself: Identity in a technological society (pp. 419–438). New York, NY: IGI Global. Sanderson, J., & Truax, C. (2014). "I hate you man!": Exploring Maladaptive Parasocial Interaction Expressions to College Athletes via Twitter. Journal of Issues in Intercollegiate Athletics, 7, 333–351. Sartore, M. L., & Cunningham, G. B. (2009). Gender, sexual prejudice, and sport participation: Implications for sexual minorities. Sex Roles, 60, 100–113, from doi:10.1007/s11199-008-9502-7. Schwartz, A. H., & Ungar, L. H. (2015). Data-Driven Content Analysis of Social Media: A Systematic Overview of Automated Methods. The Annals of the American Academy, 659, 78–94, from doi:10.1177/0002716215569197. scikit-learn (2019). sklearn.metrics.f1_score, from https://scikit- learn.org/stable/modules/generated/sklearn.metrics.f1_score.html. Sink, A., Mastro, D., & Dragojevic, M. (2018). Competent or Warm? A Stereotype Content Model Approach to Understanding Perceptions of Masculine and Effeminate Gay Television Characters. Journalism & Mass Communication Quarterly, 95(3), 588–606, from doi:10.1177/1077699017706483. Twitter Inc. (2020). About replies and mentions, from https://help.twitter.com/en/using- twitter/mentions-and- replies#:~:text=A%20mention%20is%20a%20Tweet,the%20body%20of%20the%20Twee t.&text=Visiting%20another%20account's%20profile%20page,for%20Tweets%20mention ing%20their%20username. Van Sterkenbrug, J., & Knoppers, A. (2004). Dominant discourses about race/ethnicity and gender in sport practice and performance. International Review for the Sociology of Sport, 39(3), 301–321, from doi:10.1177/1012690204045598. Vincent, J. (2004). Game, sex, and match: The construction of gender in British newspaper coverage of the 2000 Wimbledon Championships. Sociology of Sport Journal, 21, 435– 456, from doi:10.1123/ssj.21.4.435. Vincent, J., Imwold, C., Masemann, C., & Johnson, J. T. (2002). A COMPARISON OF SELECTED ‘SERIOUS’ AND ‘POPULAR’ BRITISH, CANADIAN, AND UNITED STATES NEWSPAPER COVERAGE OF FEMALE AND MALE ATHLETES COMPETING IN THE CENTENNIAL OLYMPIC GAMES: Did Female Athletes Receive Equitable Coverage in the ‘Games of the Women’? International Review for the Sociology of Sport, 37(3-4), 319–335. Wegener, D. T., Clark, J. K., & Petty, R. E. (2006). Not all stereotyping is created equal: Differential consequences of thoughtful versus non-thoughtful stereotyping. Journal of Personality and Social Psychology, 90, 42–59, from doi:10.1037/0022-3514.90.1.42.

TWEETING FOR EQUALITY 35

WTA Tennis (2020). About, from https://www.wtatennis.com/about. Yip, A. (2018). Deuce or advantage? Examining gender bias in online coverage of professional tennis. International Review for the Sociology of Sport, 53(5), 517–532, from doi:10.1177/1012690216671020.

TWEETING FOR EQUALITY 36

Appendix A

The functions displayed in the Appendix A were partly distributed in the course "Digital

Analytics" in connection with an Elective at the University of Amsterdam. The Python function is also accessible in its entirety in the following public GitHub.com repository: https://github.com/SvenKoeppel1992/MasterThesis/blob/master/MasterThesis.ipynb

Downloading the Sample of Tweets At the beginning, all the necessary packages must be imported into the Jupyter Notebook. from twython import Twython, TwythonRateLimitError import pandas as pd import os import time import sys import pickle import json import math import random import spacy import tweepy import nltk import sklearn from sklearn.metrics import confusion_matrix from sklearn.metrics import accuracy_score from sklearn.metrics import classification_report import seaborn as sns pd.set_option('display.float_format', lambda x: '%.3f' % x) %matplotlib inline import matplotlib.pyplot as plt from sklearn.linear_model import LogisticRegression, LinearRegression import statsmodels.api as sm

With the access keys and the tokens generated in the Twitter APP, downloads via REST APIs are permitted by Twitter Inc. For privacy issues, they are not displayed in the following step. consumer_key = consumer_secret = access_token = access_token_secret =

With the following function, the download of the @-mentions to the official accounts of the reviewed tennis players is made accessible: def search_tweets(query, lang, rounds): print('collecting tweets for the query', query) results = pd.DataFrame()

TWEETING FOR EQUALITY 37

counter = 0 max_id = None

if lang: tweets = twitter.search(q=query, lang=lang) else: tweets = twitter.search(q=query)

tweets = tweets['statuses']

results = results.append(pd.DataFrame(tweets)) max_id = tweets[-1]['id'] results.to_pickle('@May29Female.pkl') if rounds: while counter < rounds: if lang: tweets = twitter.search(q=query, lang=lang, max_id = max_id, count=100) else: tweets = twitter.search(q=query, max_id = max_id, count = 100)

tweets = tweets['statuses'] old_max_id = max_id max_id = tweets[-1]['id'] results = results.append(pd.DataFrame(tweets)) results.to_pickle('@May29Female.pkl') if max_id == old_max_id: print('no more tweets found for', query) break else: print('collected', len(tweets), 'tweets in round', counter + 1, 'last id', max_id, '|| waiti ng for 15 seconds') time.sleep(15) counter += 1 else: while True: if lang: tweets = twitter.search(q=query, lang=lang, max_id = max_id, count=100) else: tweets = twitter.search(q=query, max_id = max_id, count = 100)

tweets = tweets['statuses'] old_max_id = max_id max_id = tweets[-1]['id'] results = results.append(pd.DataFrame(tweets)) results.to_pickle('@May29Female.pkl') if max_id == old_max_id: print('no more tweets found for', query) break

TWEETING FOR EQUALITY 38

else: print('collected', len(tweets), 'tweets in round', counter + 1, 'last id', max_id, '|| waiti ng for 15 seconds') time.sleep(15) counter += 1

print('completed') results = results.drop_duplicates(subset='id_str') results.to_pickle('@May29Female.pkl')

return results twitter = Twython(consumer_key, consumer_secret, access_token, access_token_secret)

The official Twitter accounts of the male players are queried in the following manner. The accounts were distinguished by an OR, which lead to a search for all the listed accounts. query = "@DjokerNole OR @RafaelNadal OR @ThiemDomi OR @rogerfederer OR @DaniilMedwed OR @StefTsitsipas OR @AlexZverev OR @Gael_Monfils OR @fabiofogna OR @BautistaAgut OR @dieschwartzman OR @AndreyRub lev97 OR @karenkhachanov OR @denis_shapo OR @stanwawrinka OR @Gr igorDimitrov OR @felixtennis OR @JohnIsner OR @benoitpaire OR @Dutze e OR @Taylor_Fritz97 OR @pablocarreno91 OR @alexdeminaur OR @Hube rtHurkacz"

lang = None rounds = None results = search_tweets(query, lang, rounds)

The next line of code is to download the @-mentions referring to the TOP25 female players. query = "@ashbarty OR @KaPliskova OR @naomiosaka OR @Simona_Halep OR @ ElinaSvitolina OR @Petra_Kvitova OR @BelindaBencic OR @kikibertens O R @serenawilliams OR @SabalenkaA OR @JohannaKonta OR @Madison_K eys OR @SofiaKenin OR @elise_mertens OR @Riske4rewards OR @Donna Vekic OR @AngeliqueKerber OR @D_Yastremska OR @mariasakkari OR @ AnisimovaAmanda OR @SloaneStephens OR @juliagoerges OR @NastiaPav OR @BaraStrycova OR @geniebouchard OR @MariaSharapova"

lang = None rounds = None results = search_tweets(query, lang, rounds)

Every generated dataset must be read into Python. Due to a lack of knowledge, each player was searched for individually in the beginning. However, this misconstruction was corrected and did not lead to any disadvantages for the study.

DF1 = pd.read_pickle('@DjokerNole.pkl') DF52 = pd.read_pickle('@DjokerNole1.pkl')

TWEETING FOR EQUALITY 39

DF102 = pd.read_pickle('@DjokerNole2.pkl') DF2 = pd.read_pickle('@RafaelNadal.pkl') DF53 = pd.read_pickle('@RafaelNadal1.pkl') DF103 = pd.read_pickle('@RafaelNadal2.pkl') DF3 = pd.read_pickle('@ThiemDomi.pkl') DF54 = pd.read_pickle('@ThiemDomi1.pkl') DF104 = pd.read_pickle('@ThiemDomi2.pkl') and so on…

Loading the Twitter datasets of the sportswomen.

DF26 = pd.read_pickle('@ashbarty.pkl') DF76 = pd.read_pickle('@ashbarty1.pkl') DF126 = pd.read_pickle('@ashbarty2.pkl') DF27 = pd.read_pickle('@KaPliskova.pkl') DF77 = pd.read_pickle('@KaPliskova1.pkl') DF127 = pd.read_pickle('@KaPliskova2.pkl') DF28 = pd.read_pickle('@naomiosaka.pkl') DF78 = pd.read_pickle('@naomiosaka1.pkl') DF128 = pd.read_pickle('@naomiosaka2.pkl') and so on…

The following code was to append all the generated datasets of the male Tweets.

Final = DF1.append(DF2) Final = Final.append(DF3) Final = Final.append(DF4) and so on… Final.reset_index(drop=True)

The following step was to save the male dataset as a csv-file.

Final.to_csv("Full_Tweets_Male.csv")

The following codes was to append all the generated pickle files, addressing to female athletes.

Final_Female = DF26.append(DF27) Final_Female = Final_Female.append(DF28) Final_Female = Final_Female.append(DF29) Final_Female.reset_index(drop=True)

Eventually, the female dataset must be saved as a csv-file for further analysis.

Final_Female.to_csv("Full_Tweets_Female.csv") Data Cleaning Loading and appending the Tweets, which were generated after fixing the misconstruction; from now on, the comments referring to players were downloaded together.

DF154 = pd.read_pickle('@May20Male.pkl') DF157 = pd.read_pickle('@May23Male.pkl') DF158 = pd.read_pickle('@May29Male.pkl')

TWEETING FOR EQUALITY 40

Final_Tweets_Male = Final_Tweets_Male.append(DF154) Final_Tweets_Male = Final_Tweets_Male.append(DF157) Final_Tweets_Male = Final_Tweets_Male.append(DF158) Final_Tweets_Male.reset_index(drop=True) DF155 = pd.read_pickle('@May20Female.pkl') DF156 = pd.read_pickle("@May23Female.pkl") DF159 = pd.read_pickle("@May29Female.pkl") Final_Tweets_Female = Final_Tweets_Female.append(DF155) Final_Tweets_Female = Final_Tweets_Female.append(DF156) Final_Tweets_Female = Final_Tweets_Female.append(DF159) Final_Tweets_Female.reset_index(drop=True)

Tracking down the duplicates and deleting them; while doing so, a new datasets (FTF1 & FT1) was created.

FTF1 = Final_Tweets_Female.drop_duplicates(subset='text') FT1 = Final_Tweets_Male.drop_duplicates(subset='text')

With the help of the column "lang" the language of every Tweet could be examined and afterwards, all the non-English Tweets could have been sorted out.

FT2 = FT1[FT1['lang'] == "en"] FTF2 = FTF1[FTF1['lang'] == "en"]

Both datasets (FT2 = Male Tweets & FTF2 = Female Tweets) were saved.

FT2.to_csv("FT2.csv") FTF2.to_csv("FTF2.csv")

The column "Gender" is being built to prevent losing this information in the further course of the analysis. The Tweets, referring to sportswomen were coded as 0; The Tweets, referring to sportsmen were coded as 1. FemaleTweets["Gender"] = 0 MaleTweets["Gender"] = 1

Appending the female and male Tweets to one dataset. AllTweets = FemaleTweets.append(MaleTweets) AllTweets.reset_index(drop=True)

By appending the male and female datasets, 86.858 Tweets remained, which represents the whole dataset examined. For the further analysis, just the columns "id", "text" and "Gender" were necessary. The dataset was saved afterwards as "AllTweets.csv" and functioned as the foundation of the word vectors.

AllTweets = AllTweets[["id" , "text" , "Gender"]] AllTweets.head() AllTweets.to_csv("AllTweets.csv")

TWEETING FOR EQUALITY 41

Natural Language Processing and word2vectors print('CSV cleaning started! Please, wait...') # Compile regular expressions for cleaning re_mentions_hashtags=re.compile('([@#][\w_-]+)') re_nums=re.compile('[0-9]') re_URLs=re.compile('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a- fA-F]))+') re_special_chars=re.compile(r'[\,\|\-\:\_\*\+\.\?\!\…\"]') # Includes …(char code 8230 instead of real 3 dots) re_quotes=re.compile(r'\"\"') re_html_amp=re.compile('\&\;') re_html_amp=re.compile('\&\;') re_nn=re.compile(r'[\n\n]') re_emoji=re.compile(r"\ implement all the stopwords from the "stopwords-iso.json" file

# Function to remove stop words from string(using the structure of the stopwords-iso.json file) def remove_stops_from_str(str, mystop): L_str=str.split(' ') new_str=''

for word in L_str: in_stop=False for L_lang in list(mystop.values()): if word.rstrip() in L_lang: in_stop=True break

if in_stop==False: new_str+=word.rstrip()+' '

return new_str

# Function to remove stop words from list(using the structure of the stopwords-iso.json file) def remove_stops_from_list(L, mystop): L_filtered=[] for word in L: in_stop=False for L_lang in list(mystop.values()): if word.strip() in L_lang: in_stop=True break

if in_stop==False: L_filtered.append(word)

return L_filtered

# Creates a dataset from "text" column of the .csv file

TWEETING FOR EQUALITY 42 df = pd.read_csv('AllTweets.csv', usecols=['text']) # Change the location of the file!

# As there are different sizes of JSON, we should use below instead of pd.read_json with open('stopwords-iso.json' , encoding="utf-8") as json_stop_data: # Change the location of the file! stop_data=json.load(json_stop_data)

# Creates a dataset for stop words st=pd.DataFrame.from_dict(stop_data,orient='index')

# Remove mentions and hashtags df=df.replace(regex=re_mentions_hashtags, value='')

# Remove links df=df.replace(regex=re_URLs, value='')

# Remove numbers df=df.replace(regex=re_nums, value='')

# Remove special characters df=df.replace(regex=re_special_chars, value='')

# Remove HTML amp df=df.replace(regex=re_html_amp, value='')

# Remove emoji - works much longer now... but cleaning text df=df.replace(regex=re_emoji, value='')

#Replace new lines inside of the string/tweet by space df=df.replace(regex=re_nn, value=' ')

L_tweets=df['text'].str.strip().tolist() df_tweets_sent=pd.DataFrame(L_tweets) df_tweets_sent.to_csv('./cleaned_tweets.csv', header=['tweets'], index=None, mode='w') # Change the location of the file! print('CSV cleaning finished!') print('Model training started! Please, wait...') fmin_count = 4 fsize = 50 fwindow = 4

# df = pd.read_csv('/Users/buttersmai/Documents/sven/simpsons_dataset.csv') df = pd.read_csv('./cleaned_tweets.csv') # Change the location of the file!

L_sentences=df['tweets'].tolist() # for tweets # L_sentences=df['spoken_words'].tolist() #for Simpsons test dataset

TWEETING FOR EQUALITY 43 sentences=[list(gensim.utils.tokenize(str(sent), deacc=True, lowercase=True)) for sent in L_sentences] print('Dataset: Tweets') # for tweets # print('Dataset: Simpsons example') #for Simpsons test dataset print('The number of tweets:',len(sentences))

# Build the model model = Word2Vec(min_count=20, window=2, size=300, sample=6e-5, alpha=0.03, min_alpha=0.0007, negative=20)

# Build vocabulary model.build_vocab(sentences) t=time()

# Train the model model.train(sentences, total_examples=model.corpus_count, epochs=30, report_delay=1) print('Time to train the model: {} mins'.format(round((time() - t) / 60, 2))) vocab_list=list(model.wv.vocab.keys()) df_vocab_list=pd.DataFrame(vocab_list) df_vocab_list.to_csv('output_vocab.txt', header=None, index=None, sep=' ', mode='w') # Change the location of the file!

# Save the model to further use in model_use.py to save the RAM model.save('./w2v.model') print('Model training started! Please, wait...') print('Model saved:', model) model = Word2Vec.load('./w2v.model') print('Program started. Type \"stop\" or use Ctrl+C to stop it:') test_word='' while test_word!='stop': try: test_word=input('\nWord:') counter=0 L_similar=list(model.wv.most_similar(positive=test_word, topn=15)) print('\nMost similar for \"'+test_word+'\":') for word_tuple in L_similar: print(str(counter+1)+'. '+word_tuple[0])

TWEETING FOR EQUALITY 44

counter+=1 except KeyError: print('This word not in the vocabulary')

print('Program stopped!') Automated Content Analysis The automated content analysis is being feed with the words generated by the word embeddings model. def wordlist_any_present(text, query): import re text = str(text).lower() newquery = [] for word in query: newquery.append(str(word).lower()) tokens = re.findall(r"[\w']+|[.,!?;$@#]", text)

for word in newquery: if word in tokens: return 1 return 0

The first category "Skills" is being built as follows and with the following keywords:

AllTweets['Skills'] = AllTweets['text'].apply(wordlist_any_present, args=(["unique" , "dedication", "impressive" , "talent" , "passion" , "gest ure" , "genius", "highestpaid", "athletes" , "player", "greatest", "status" , "strength" , "fitness" , "Fitness" , "creative", "determination" , "health" , "courage", "energy", "balance" , "focus" , "e fforts" , "effort" , "serve" , "serving" , "slice" , "slicing" , "backhand" , "bh" , "volleys" , "volle y", "technique" , "smash", "grip", "hits", "hit" , "footwork" , "onehanded" , "tweener" , "tween ers", "average" , "rally" , "rallies" , "opponents" , "opponent" , "rate" , "motion" , "spin" , "riv als" , "rival", "superb" , "unbelievable" , "unreal" , "performance" , "master" , "Master" , "lead ing" , "underrated" , "rivalry" , "body" , "risk" , "successful" , "confidence" , "soul" , "working " , "motivation" , "mentally" , "train" , "practising", "training" , "train" , "job" , "work" , "gest ure" , "drive" , "gym" , "appreciated" , "art" , "ambassador" , "spirit" , "humble", "peace" , "bu siness" , "blessing" , "goal" , "goals" , "frontline" , "happiness" , "success" , "force" , "willing" , "towards" , "wrist" , "position" , "ideal" , "feet" , "foot" , "shoe" , "shoes" , "movement" , "cr oss" , "arm" , "storm" , "knees" , "impressed" , "moving" , "bounce" , "mentally strong" , "fit" , "fighting" , "creative" , "hander" , "lack of power" , "grown in power" , "pure power" , "mod el" , "greatness" , "beyond" , "athletic" , "mentally tough" , "challenging" , "competitive" , "b oring" , "perspective" , "fair" , "fairness" , "jump" , "jumping" , "education" , "educated" , "fa ncy" , "faster" ],))

The second category "Appearance" is being built as follows by detecting the following words:

AllTweets['Appearance'] = AllTweets['text'].apply(wordlist_any_present, args=(["attitude" , "generous" , "classy" , "gentleman" , "Gentleman" , "funny" , "hilarious" , "jealous" , "cool" , "smart" , "interesting" , "idiot" , "cute" , "precious" , "sweet" , "cutest" , "beautiful" , "dancing" , "gorgeous" , "sexy" , "heart" , "shine" , "crying" , "adore" ,

TWEETING FOR EQUALITY 45

"sweetie" , "dislike" , "horrible" , "shy" , "racist" , "cutie" , "sounds" , "sound" , "looked" , "ha ircut" , "hair" , "yummy" , "pink" , "lil", "grace" , "eye" , "beast" , "color" , "beau" , "nails" , " girlfriend" , "selfie" , "ring" , "lovely" , "wonderful" , "amazing" , "adorable" , "handsome" , " beard" , "goodness" , "girl" , "girls" , "lit" , "bear" , "jesus", "darling" , "families" , "kids" , " moms" , "raised" , "brothers" , "couple" , "mom" , "mothers" , "mum", "mother", "father" , "fa thers", "dad" , "daddy" , "wife" , "husband" , "silly" , "students" , "anniversary" , "bday" , "cel ebrating" , "son" , "daughter"],))

The dataset with the two new columns "Skills" and "Appearance are saved for further analysis

AllTweets.to_csv("Final_Tweets.csv") Precision and Recall Generating a new column "Stereotyping" for adding the previous made categories. def generate_category(row): if row['Skills'] == 1: row['Stereotyping'] = 1 if row['Appearance'] == 1: row['Stereotyping'] = 2 return row Tweets_Sample = Tweets_Sample.apply(generate_category, axis=1)

Dropping the missing values in the column "Stereotyping".

Tweets_Sample.isna().sum() Tweets_Sample = Tweets_Sample.dropna(subset=["Stereotyping"])

A sample was generated from the whole population of Tweets for the manual coding with the help of the following steps:

Test_Sample = Tweets_Sample.sample(frac=0.012, random_state = 42)

To manually code the Tweets, we just need the columns "id", "text" and "Stereotyping". The csv-file "pd.train" serves as the foundation for the manual coding.

Test_Sample = Test_Sample[['id', "text" , "Stereotyping"]] Test_Sample.to_csv("pd.train.csv") #pd.train is the dataset to manually code the tweets

After generating the subsample, just the columns id and gender are necessary to align the two datasets.

AllTweets_Test_after_manually_coding = Test_Sample[['id', "Stereotyping"]] AllTweets_Test_after_manually_coding.reset_index(drop=True) #after the manually coding, we just need the columns "id" and "Stereotyping"

The dataset with the automatically coded Tweets is saved to the dataset "Test".

AllTweets_Test_after_manually_coding.to_csv("Test.csv")

TWEETING FOR EQUALITY 46

The dataset, which was conducted within Qualtrics and which serves as the Train-Dataset is being implemented with the next steps.

AllTweets_Train_after_manually_coding = pd.read_csv("StereotypingTest.csv") #this is the d ataset which was downloaded from Qualtrics

Choosing just the columns that are useful.

AllTweets_Train_after_manually_coding = AllTweets_Train_after_manually_coding[["Q7" , "Q1"]] #the column Q7 is the answer-column for the id and Q1 is the answer-column for stere otyping

Renaming the columns for a better understanding.

AllTweets_Train_after_manually_coding = AllTweets_Train_after_manually_coding.rename (columns={'Q7': 'id', 'Q1': 'Bias'}) #more advantageous for the clarity

Subsequently, the data set downloaded from Qualtrics is also categorized in the same way as the original sample (Test.csv). def generate_category_manually_coding(row): #important that we have numerous categorie s for the further analysis if row['Bias'] == "Appearance & Looks": row['Stereotyping_Train'] = 3 if row['Bias'] == 'Physical & Mental Skills': row['Stereotyping_Train'] = 1 return row AllTweets_Train_after_manually_coding = AllTweets_Train_after_manually_coding.apply(g enerate_category_manually_coding, axis=1)

The first two rows must be deleted and the columns (id, Stereotyping_Train) were chosen to align with the automatically generated dataset "Test". Eventually, the Train dataset must be saved for further analysis.

AllTweets_Train_after_manually_coding = AllTweets_Train_after_manually_coding.drop(0, 1) #there are rows we do not need for the analysis AllTweets_Train_after_manually_coding.reset_index(drop=True) #reset the index AllTweets_Train_after_manually_coding = AllTweets_Train_after_manually_coding[["id" , " Stereotyping_Train"]] AllTweets_Train_after_manually_coding.to_csv("Train.csv")

Precision and Recall with the two Datasets "Train" and "Test": Merging the two datasets to "TrainTest" with the two columns "Stereotyping" and "Stereotyping_Train".

TrainTest = Test.merge(Train, on='id', how="left")

Are there any missing values in the merged dataset?

TrainTest.isna().sum()

TWEETING FOR EQUALITY 47

For further analysis is important to delete the missing variables within the dataset.

TrainTest = TrainTest.dropna(subset=["Stereotyping_Train"])

Generating a dataset with the confusion matrix included for Classification Report and Accuracy testing. from sklearn.metrics import confusion_matrix Matrix = (confusion_matrix(TrainTest['Stereotyping_Train'], TrainTest['Stereotyping']))

Precision and Recall / Classification Report and Accuracy Score. print(Matrix) print(accuracy_score(TrainTest["Stereotyping_Train"], TrainTest["Stereotyping"])) print(classification_report(TrainTest["Stereotyping_Train"], TrainTest["Stereotyping"])) Statistical Procedures The descriptive statistics were coded to adequately represent the independent variable "gender" and the dependent variables "skills" and "appearance". In the following step, the independent variable "Gender" will be displayed. The statistical procedure was executed with the help of the dataset "Final_Tweets", which was saved after the automated content analysis. Final_Tweets = pd.read_csv("Final_Tweets.csv") Final_Tweets["Gender"].value_counts() Final_Tweets["Gender"].describe()

Descriptive Statistics for the variable "Gender"; Retrieved from: https://stackoverflow.com/questions/33179122/seaborn-countplot-with-frequencies ncount = len(Final_Tweets) plt.figure(figsize=(12,8)) ax = sns.countplot(x="Gender", data=Final_Tweets, order=[0,1]) plt.title('Distribution of Gender') plt.xlabel('Gender')

# Make twin axis ax2=ax.twinx()

# Switch so count axis is on right, frequency on left ax2.yaxis.tick_left() ax.yaxis.tick_right()

# Also switch the labels over ax.yaxis.set_label_position('right') ax2.yaxis.set_label_position('left') ax2.set_ylabel('Frequency [%]')

TWEETING FOR EQUALITY 48 for p in ax.patches: x=p.get_bbox().get_points()[:,0] y=p.get_bbox().get_points()[1,1] ax.annotate('{:.1f}%'.format(100.*y/ncount), (x.mean(), y), ha='center', va='bottom') # set the alignment of the text

# Use a LinearLocator to ensure the correct number of ticks ax.yaxis.set_major_locator(ticker.LinearLocator(11))

# Fix the frequency range to 0-100 ax2.set_ylim(0,100) ax.set_ylim(0,ncount)

# And use a MultipleLocator to ensure a tick spacing of 10 ax2.yaxis.set_major_locator(ticker.MultipleLocator(10))

# Need to turn the grid on ax2 off, otherwise the gridlines end up on top of the bars ax2.grid(None) plt.savefig('Distribution_of_Gender.png')

Descriptive Statistics for the variable "Skills". Retrieved from: https://stackoverflow.com/questions/33179122/seaborn-countplot-with-frequencies ncount = len(Final_Tweets) plt.figure(figsize=(12,8)) ax = sns.countplot(x="Skills", data=Final_Tweets, order=[0,1]) plt.title('Mental & Athletic Skills') plt.xlabel('Mental & Athletic Skills')

# Make twin axis ax2=ax.twinx()

# Switch so count axis is on right, frequency on left ax2.yaxis.tick_left() ax.yaxis.tick_right()

# Also switch the labels over ax.yaxis.set_label_position('right') ax2.yaxis.set_label_position('left') ax2.set_ylabel('Frequency [%]') for p in ax.patches: x=p.get_bbox().get_points()[:,0] y=p.get_bbox().get_points()[1,1] ax.annotate('{:.1f}%'.format(100.*y/ncount), (x.mean(), y), ha='center', va='bottom') # set the alignment of the text

# Use a LinearLocator to ensure the correct number of ticks

TWEETING FOR EQUALITY 49 ax.yaxis.set_major_locator(ticker.LinearLocator(11))

# Fix the frequency range to 0-100 ax2.set_ylim(0,100) ax.set_ylim(0,ncount)

# And use a MultipleLocator to ensure a tick spacing of 10 ax2.yaxis.set_major_locator(ticker.MultipleLocator(10))

# Need to turn the grid on ax2 off, otherwise the gridlines end up on top of the bars ax2.grid(None) plt.savefig('Distribution_of_Mental & Athletic_Skills.png')

Which sex has received more Tweets within the category "Skills"?

Final_Tweets.groupby('Gender')[['Skills']].describe().transpose()

Which Gender has received more Tweets within the category "Appearance"? Descriptive stati stics for the variable "Appearance". Retrieved from: https://stackoverflow.com/questions/331 79122/seaborn-countplot-with-frequencies

Final_Tweets["Appearance"].value_counts(normalize=True)

ncount = len(Final_Tweets) plt.figure(figsize=(12,8)) ax = sns.countplot(x="Appearance", data=Final_Tweets, order=[0,1]) plt.title('Appearance & Background') plt.xlabel('Appearance & Background')

# Make twin axis ax2=ax.twinx()

# Switch so count axis is on right, frequency on left ax2.yaxis.tick_left() ax.yaxis.tick_right()

# Also switch the labels over ax.yaxis.set_label_position('right') ax2.yaxis.set_label_position('left') ax2.set_ylabel('Frequency [%]') for p in ax.patches: x=p.get_bbox().get_points()[:,0] y=p.get_bbox().get_points()[1,1] ax.annotate('{:.1f}%'.format(100.*y/ncount), (x.mean(), y), ha='center', va='bottom') # set the alignment of the text

TWEETING FOR EQUALITY 50

# Use a LinearLocator to ensure the correct number of ticks ax.yaxis.set_major_locator(ticker.LinearLocator(11))

# Fix the frequency range to 0-100 ax2.set_ylim(0,100) ax.set_ylim(0,ncount)

# And use a MultipleLocator to ensure a tick spacing of 10 ax2.yaxis.set_major_locator(ticker.MultipleLocator(10))

# Need to turn the grid on ax2 off, otherwise the gridlines end up on top of the bars ax2.grid(None) plt.savefig('Distribution_of_Appearance & Background.png') Hypotheses Testing The logistic regression for the variables "Skills" and "Gender" was conducted with the help of the following code. ols_stat_skills = sm.OLS(Final_Tweets["Skills"], sm.add_constant(Final_Tweets["Gender"])) result_ols_skills = ols_stat_skills.fit() print(result_ols_skills.summary())

The logistic regression for the variables "Appearance" and "Gender" was conducted with the help of the following code. ols_stat_appearance = sm.OLS(Final_Tweets["Appearance"], sm.add_constant(Final_Tweets[ "Gender"])) result_ols_appearance = ols_stat_appearance.fit() print(result_ols_appearance.summary())

The t-test for the analysis of gender differences in the number of tweets was performed within SPSS, generating a data set that included only the variable "Gender". a = Final_Tweets[["Gender"]] a = a.reset_index() a = a.drop(['index'],axis=1) a.to_csv("SPSS.csv")

TWEETING FOR EQUALITY 51

Appendix B

Reviewed Athletes and their official Twitter Accounts

The following table shows all female and male players included in the analysis of this study. You can also see the exact wording of their official Twitter account.

Table 6

Female and male tennis players reviewed in the analysis

Number WTA Twitter Account ATP Twitter Account 1 Ashleigh Barty @ashbarty Novak Djokovic @DjokerNole 2 Karolina Pliskova @KaPliskova Rafael Nadal @RafaelNadal 3 Naomi Osaka @naomiosaka Dominic Thiem @ThiemDomi 4 Simona Halep @Simona_Halep Roger Federer @rogerfederer 5 Bianca Andreescu @Bandreescu_ Daniil Medvedev @DaniilMedwed 6 Elina Svitolina @ElinaSvitolina Stefanos Tsitsipas @StefTsitsipas 7 Petra Kvitova @Petra_Kvitova Alexander Zverev @AlexZverev 8 Belinda Bencic @BelindaBencic Matteo Berrettini N. A 9 Kiki Bertens @kikibertens Gael Monfils @Gael_Monfils 10 Serena Williams @serenawilliams David Goffin @David__Goffin 11 Aryna Sabalenka @SabalenkaA Fabio Fognini @fabiofogna 12 Johanna Konta @JohannaKonta Roberto Bautista Agut @BautistaAgut 13 Madison Keys @Madison_Keys Diego Schwartzman @dieschwartzman 14 Sofia Kenin @SofiaKenin Andrey Rublev @AndreyRublev97 15 Petra Martic @PetraMartic1991 Karen Khachanov @karenkhachanov 16 Marketa Vondrousova @VondrousovaM Denis Shapovalov @denis_shapo 17 Elise Mertens @elise_mertens Stan Wawrinka @stanwawrinka 18 Alison Riske-Amritraj @Riske4rewards Christian Garin N. A 19 Donna Vekic @DonnaVekic Grigor Dimitrov @GrigorDimitrov 20 Angelique Kerber @AngeliqueKerber Felix Auger-Aliassime @felixtennis 21 Karolina Muchova @karomuchova7 John Isner @JohnIsner 22 Dayana Yastremska @D_Yastremska Benoit Paire @benoitpaire 23 Maria Sakkari @mariasakkari Dusan Lajovic @Dutzee 24 Amanda Anisimova @AnisimovaAmanda Taylor Fritz @Taylor_Fritz97 25 Sloane Stephens @SloaneStephens Pablo Carreno Busta @pablocarreno91 26 N. A N. A Alex De Minaur @alexdeminaur 27 N. A N. A Hubert Hurkacz @HubertHurkacz

TWEETING FOR EQUALITY 52

Appendix C

Instructions for the Manual Content Analysis

To ensure that the categorization is adequately checked, a certain number of Tweets are tested manually for the occurrence of the following keywords. 1. When one of the following keywords appear in the reviewed Tweet, the Tweet will be placed in the category "Athletic and Mental Skills":  "unique", "dedication", "impressive", "talent", "passion", "gesture", "genius", "highest-paid", "athletes", "player", "greatest", "status", "strength", "fitness", "creative", "determination", "health", "courage", "energy", "balance", "focus", "efforts", "serve", "slice", "backhand", "volleys", "technique", "smash", "volley", "grip", "hits", "footwork", "onehanded", "tweener", "average", "rally", "opponents", "rate", "motion", "spin", "rivals", "superb", "unbelievable", "unreal", "performance", "master", "leading", "underrated", "rivalry", "body", "risk", "successful", "confidence", "soul", "working", "motivation" , "mentally", "train", "practising", "grow", "training", "job", "work", "gesture", "drive", "gym", "appreciated", "art", "ambassador", "spirit", "humble", "peace", "business", "blessing", "goal", "frontline", "happiness", "success", "force", "willing", "towards", "wrist", "position", "ideal", "feet", "shoe", "movement", "cross", "arm", "storm", "knees", "impressed", "moving", "bounce", "mentally strong", "fit", "fighting", "creative", "safe", "hander", "lack of power", "grown in power", "pure power", "model", "greatness", "beyond", "athletic", "mentally tough", "challenging", "competitive", "boring", "perspective", "fair", "jump", "education", "fancy", "faster" 2. When one of the following keywords appear in the reviewed Tweet, the Tweet will be placed in the category "Physical Appearance":  "attitude", "generous", "classy", "gentleman", "funny", "hilarious", "jealous", "cool", "smart", "interesting", "idiot", "cute", "precious", "sweet", "cutest", "beautiful", "dancing", "gorgeous", "sexy", "heart", "shine", "crying", "adore", "sweetie", "dislike", "horrible", "shy", "racist", "cutie", "sounds", "sound", "looked", "haircut", "yummy", "pink", "looking", "grace", "eye", "beast", "colour", "beau", "nails", "girlfriend", "selfie", "ring", "lovely", "wonderful", "amazing", "adorable", "handsome", "beard", "goodness", "girl", "girls", "lit", "bear", "Jesus", "darling", "families", "kids", "moms", "raised", "brothers", "couple", "mom", "mothers", "mum", "madam", "wife", "husband", "silly", "students", "anniversary", "celebrating", "son", "daughter" Instructions: 1. A Tweet cannot be divided into both categories, as they must be mutually exclusive. Therefore, the category is preferred, where the importance is more given.

2. The identification number (ID) of the Tweet must be specified, otherwise the manually coded data set will not align with the automated data set

TWEETING FOR EQUALITY 53

Qualtrics survey

In the Figure 3 the survey is displayed in a screenshot. It can be noticed that two questions appear after the introduction; the first question is about the ID of the examined

Tweets and the second question is about the categorization of the Tweets into the two categories.

Figure 3: Screenshot of the Qualtrics Survey used for the manual coding procedure